otelsql

👁 ci
👁 codecov
👁 Go Report Card
👁 Documentation

It is an OpenTelemetry instrumentation for Golang database/sql, a port from https://github.com/open-telemetry/opentelemetry-go-contrib/pull/505.

It instruments traces and metrics.

Install

$ go get github.com/XSAM/otelsql

Usage

This project provides four different ways to instrument database/sql:

otelsql.Open, otelsql.OpenDB, otesql.Register and otelsql.WrapDriver.

And then use otelsql.RegisterDBStatsMetrics to instrument sql.DBStats with metrics.

db, err := otelsql.Open("mysql", mysqlDSN, otelsql.WithAttributes(
	semconv.DBSystemMySQL,
))
if err != nil {
	panic(err)
}

reg, err := otelsql.RegisterDBStatsMetrics(db, otelsql.WithAttributes(
	semconv.DBSystemMySQL,
))
if err != nil {
	panic(err)
}
defer func() {
	_ = db.Close()
	_ = reg.Unregister()
}()

Check Option for more features like adding context propagation to SQL queries when enabling WithSQLCommenter.

See godoc for details.

Blog

Getting started with otelsql, the OpenTelemetry instrumentation for Go SQL, is a blog post that explains how to use otelsql in miutes.

Examples

This project provides two docker-compose examples to show how to use it.

• The stdout example is a simple example to show how to use it with a MySQL database. It prints the trace data to stdout and serves metrics data via prometheus client.
• The otel-collector example is a more complex example to show how to use it with a MySQL database and an OpenTelemetry Collector. It sends the trace data and metrics data to an OpenTelemetry Collector. Then, it shows data visually on Jaeger and Prometheus servers.

Semantic Convention Stability Migration

The environment variable OTEL_SEMCONV_STABILITY_OPT_IN will be supported for at least six months. After this period, support for legacy metrics and Semantic Conventions v1.24.0 may be removed in the next release.

Check the CHANGELOG.md for more details.

Trace Instruments

It creates spans on corresponding methods.

Use SpanOptions to adjust creation of spans.

Trace Semantic Convention Stability

The instrumentation supports different OpenTelemetry semantic convention stability levels, configured through the OTEL_SEMCONV_STABILITY_OPT_IN environment variable:

Metric Instruments

Two types of metrics are provided depending on the semantic convention stability setting:

Legacy Metrics (default)

• db.sql.latency: The latency of calls in milliseconds
  • Unit: milliseconds
  • Attributes: method (method name), status (ok, error)

OpenTelemetry Semantic Convention Metrics

• db.client.operation.duration: Duration of database client operations
  • Unit: seconds
  • Attributes: db.operation.name (method name), error.type (if error occurs)

Connection Statistics Metrics (from Go's sql.DBStats)

• db.sql.connection.max_open: Maximum number of open connections to the database
• db.sql.connection.open: The number of established connections
  • Attributes: status (idle, inuse)
• db.sql.connection.wait: The total number of connections waited for
• db.sql.connection.wait_duration: The total time blocked waiting for a new connection (ms)
• db.sql.connection.closed_max_idle: The total number of connections closed due to SetMaxIdleConns
• db.sql.connection.closed_max_idle_time: The total number of connections closed due to SetConnMaxIdleTime
• db.sql.connection.closed_max_lifetime: The total number of connections closed due to SetConnMaxLifetime

Metric Semantic Convention Stability

The instrumentation supports different OpenTelemetry semantic convention stability levels, configured through the OTEL_SEMCONV_STABILITY_OPT_IN environment variable:

Connection statistics metrics (db.sql.connection.*) are always emitted regardless of the stability setting.

This allows users to gradually migrate to the new OpenTelemetry semantic conventions while maintaining backward compatibility with existing dashboards and alerts.

Error Type Attribution

When errors occur during database operations, the error.type attribute is automatically populated with the type of the error. This provides more detailed information for debugging and monitoring:

1. For standard driver errors: Special handling for common driver errors:

   • database/sql/driver.ErrBadConn
   • database/sql/driver.ErrSkip
   • database/sql/driver.ErrRemoveArgument

2. For custom errors: The fully qualified type name is used (e.g., github.com/your/package.CustomError).

3. For built-in errors: The type name is used (e.g., *errors.errorString for errors created with errors.New()).

Note: The error.type attribute is only available when using the new stable OpenTelemetry semantic convention metrics. This requires setting OTEL_SEMCONV_STABILITY_OPT_IN to either database/dup or database. With the default setting (empty), which only uses legacy metrics, the error.type attribute will not be populated.

Compatibility

This project is tested on the following systems.

While this project should work for other systems, no compatibility guarantees are made for those systems currently.

The project follows the Release Policy to support major Go releases.

Why port this?

Based on this comment, OpenTelemetry SIG team like to see broader usage and community consensus on an approach before they commit to the level of support that would be required of a package in contrib. But it is painful for users without a stable version, and they have to use replacement in go.mod to use this instrumentation.

Therefore, I host this module independently for convenience and make improvements based on users' feedback.

Communication

I use GitHub discussions/issues for most communications. Feel free to contact me on CNCF slack.

Constants ¶

Variables ¶

Functions ¶

func AttributesFromDSN(dsn string) []attribute.KeyValue

func Open(driverName, dataSourceName string, options ...Option) (*sql.DB, error)

func OpenDB(c driver.Connector, options ...Option) *sql.DB

func Register(driverName string, options ...Option) (string, error)

func RegisterDBStatsMetrics(db *sql.DB, opts ...Option) (metric.Registration, error)

func Version() string

func WrapDriver(dri driver.Driver, options ...Option) driver.Driver

Types ¶

type AttributesGetter func(ctx context.Context, method Method, query string, args []driver.NamedValue) []attribute.KeyValue

type Event string

const (
	// EventRowsNext is triggered during driver.Rows.Next iteration to track each row fetching operation.
	EventRowsNext Event = "sql.rows.next"
)

type InstrumentAttributesGetter func(ctx context.Context, method Method, query string, args []driver.NamedValue) []attribute.KeyValue

type InstrumentErrorAttributesGetter func(err error) []attribute.KeyValue

type Method string

const (
	// MethodConnectorConnect is used when driver.Connector.Connect is called to establish a new connection.
	MethodConnectorConnect Method = "sql.connector.connect"
	// MethodConnPing is used with driver.Pinger.Ping to verify database connection is alive.
	MethodConnPing Method = "sql.conn.ping"
	// MethodConnExec is used with driver.ExecerContext.ExecContext for direct query execution through a connection.
	MethodConnExec Method = "sql.conn.exec"
	// MethodConnQuery is used with driver.QueryerContext.QueryContext for executing queries directly through a connection.
	MethodConnQuery Method = "sql.conn.query"
	// MethodConnPrepare is used with driver.ConnPrepareContext.PrepareContext for creating prepared statements.
	MethodConnPrepare Method = "sql.conn.prepare"
	// MethodConnBeginTx is used with driver.ConnBeginTx.BeginTx for starting a new transaction.
	MethodConnBeginTx Method = "sql.conn.begin_tx"
	// MethodConnResetSession is used with driver.SessionResetter.ResetSession to reset connection session state.
	MethodConnResetSession Method = "sql.conn.reset_session"
	// MethodTxCommit is used with driver.Tx.Commit to commit a transaction.
	MethodTxCommit Method = "sql.tx.commit"
	// MethodTxRollback is used with driver.Tx.Rollback to rollback a transaction.
	MethodTxRollback Method = "sql.tx.rollback"
	// MethodStmtExec is used with driver.StmtExecContext.ExecContext to execute a prepared statement.
	MethodStmtExec Method = "sql.stmt.exec"
	// MethodStmtQuery is used with driver.StmtQueryContext.QueryContext to query using a prepared statement.
	MethodStmtQuery Method = "sql.stmt.query"
	// MethodRows is used to track the lifecycle of driver.Rows returned by query operations.
	MethodRows Method = "sql.rows"
)

type Option interface {
	// Apply sets the Option value of a config.
	Apply(cfg *config)
}

func WithAttributes(attributes ...attribute.KeyValue) Option

func WithAttributesGetter(attributesGetter AttributesGetter) Option

func WithDisableSkipErrMeasurement(disable bool) Option

func WithInstrumentAttributesGetter(instrumentAttributesGetter InstrumentAttributesGetter) Option

func WithInstrumentErrorAttributesGetter(instrumentErrorAttributesGetter InstrumentErrorAttributesGetter) Option

func WithMeterProvider(provider metric.MeterProvider) Option

func WithSQLCommenter(enabled bool) Option

SELECT * from FOO

SELECT * from FOO /*traceparent='00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01',tracestate='congo%3Dt61rcWkgMzE%2Crojo%3D00f067aa0ba902b7'*/

func WithSpanNameFormatter(spanNameFormatter SpanNameFormatter) Option

func WithSpanOptions(opts SpanOptions) Option

func WithTextMapPropagator(propagator propagation.TextMapPropagator) Option

func WithTracerProvider(provider trace.TracerProvider) Option

type OptionFunc func(*config)

func (f OptionFunc) Apply(c *config)

type SpanFilter func(ctx context.Context, method Method, query string, args []driver.NamedValue) bool

type SpanNameFormatter func(ctx context.Context, method Method, query string) string

type SpanOptions struct {
	// Ping, if set to true, will enable the creation of spans on Ping requests.
	Ping bool

	// RowsNext, if set to true, will enable the creation of events in spans on RowsNext
	// calls. This can result in many events.
	RowsNext bool

	// DisableErrSkip, if set to true, will suppress driver.ErrSkip errors in spans.
	DisableErrSkip bool

	// DisableQuery if set to true, will suppress db.statement in spans.
	DisableQuery bool

	// RecordError, if set, will be invoked with the current error, and if the func returns true
	// the record will be recorded on the current span.
	//
	// If this is not set it will default to record all errors (possible not ErrSkip, see option
	// DisableErrSkip).
	RecordError func(err error) bool

	// OmitConnResetSession if set to true will suppress sql.conn.reset_session spans
	OmitConnResetSession bool

	// OmitConnPrepare if set to true will suppress sql.conn.prepare spans
	OmitConnPrepare bool

	// OmitConnQuery if set to true will suppress sql.conn.query spans
	OmitConnQuery bool

	// OmitRows if set to true will suppress sql.rows spans
	OmitRows bool

	// OmitConnectorConnect if set to true will suppress sql.connector.connect spans
	OmitConnectorConnect bool

	// SpanFilter, if set, will be invoked before each call to create a span. If it returns
	// false, the span will not be created.
	SpanFilter SpanFilter
}