 |
VOOZH | about |
|
VOOZH | about |
Datadog Database Monitoring (DBM) for ClickHouse provides deep visibility into your ClickHouse Cloud services by collecting query metrics, live query samples, and completed query records to help you resolve issues and optimize query performance across your entire fleet.
Database Monitoring collects the following data from ClickHouse:
tags option are attached to the instance for filtering and grouping by environment, region, service, or any other custom dimensions.system.query_log.system.processes at a 1-second interval. Because ClickHouse queries often complete in under one second, short-lived queries may not always appear in samples.Create a dedicated datadog user:
CREATEUSERdatadogIDENTIFIEDBY'<PASSWORD>';Grant the required permissions on system tables:
GRANTSELECTONsystem.metricsTOdatadog;GRANTSELECTONsystem.eventsTOdatadog;GRANTSELECTONsystem.asynchronous_metricsTOdatadog;GRANTSELECTONsystem.partsTOdatadog;GRANTSELECTONsystem.replicasTOdatadog;GRANTSELECTONsystem.dictionariesTOdatadog;GRANTSELECTONsystem.processesTOdatadog;GRANTSELECTONsystem.query_logTOdatadog;Grant REMOTE permissions to allow cross-replica querying:
GRANTREMOTEON*.*TOdatadog;REMOTE privilege is required because the Agent uses ClickHouse's clusterAllReplicas() table function to aggregate data across all replicas in a ClickHouse Cloud service through the single endpoint. This privilege enables cross-node query execution—it does not grant access to any additional databases or tables beyond what was explicitly granted above. The ON *.* syntax is a ClickHouse requirement for this privilege type and does not expand the scope of data access.For ClickHouse Cloud, the Agent connects to the service endpoint directly. Data is collected at the service level (aggregate across all replicas), not per individual node.
# /etc/datadog-agent/conf.d/clickhouse.d/conf.yamlinit_config:instances:- dbm:trueserver:xyz.us-east-2.aws.clickhouse.cloudport:8443username:datadogpassword:<PASSWORD># Required for ClickHouse Cloudtls_verify:trueverify:truesingle_endpoint_mode:truetags:- env:production- deployment:cloudquery_metrics:enabled:truecollection_interval:10query_samples:enabled:truecollection_interval:1query_completions:enabled:truecollection_interval:10single_endpoint_mode: true is required for ClickHouse Cloud. It enables clusterAllReplicas() queries to collect data across all nodes behind a single endpoint.
The database_identifier option controls how the database instance appears in DBM. Datadog recommends using the service name as a custom tag for identification and grouping.
instances:- dbm:trueserver:xyz.us-east-2.aws.clickhouse.cloudport:8443# ... other settings ...database_identifier:template:"$env-$server:$port"tags:- env:production- service_name:user-serviceWith env:production, server: xyz.us-east-2.aws.clickhouse.cloud, and port: 8443, this produces:
| Template | Result |
|---|---|
$server:$port (default) | xyz.us-east-2.aws.clickhouse.cloud:8443 |
$env-$server:$port | production-xyz.us-east-2.aws.clickhouse.cloud:8443 |
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
server | string | Yes | - | ClickHouse Cloud service hostname (for example, xyz.us-east-2.aws.clickhouse.cloud). |
port | integer | No | 8123 | HTTP port. Use 8443 for ClickHouse Cloud (HTTPS). |
username | string | No | default | ClickHouse user account the Agent authenticates as. Datadog recommends a dedicated datadog user with limited permissions. |
password | string | No | - | Password for the specified user. |
db | string | No | default | Database to connect to. Most metrics come from system tables, so default is usually appropriate. |
| Field | Type | Default | Description |
|---|---|---|---|
tls_verify | boolean | false | Enable TLS. Required for ClickHouse Cloud. |
verify | boolean | true | Validate the server’s SSL certificate. Setting false in production is a security risk. |
tls_ca_cert | string | - | Path to a custom CA certificate file. Not needed for ClickHouse Cloud, which uses certificates from public CAs. |
| Field | Type | Default | Description |
|---|---|---|---|
dbm | boolean | false | Enable Database Monitoring. Required for query metrics, samples, and completions collection. |
single_endpoint_mode | boolean | false | Required for ClickHouse Cloud. Enables clusterAllReplicas() queries to collect data across all nodes behind a single endpoint. |
| Field | Type | Default | Description |
|---|---|---|---|
database_identifier.template | string | $server:$port | Template for the unique database identifier. Supports variables: $server, $port, and any custom tag keys (for example, $env, $service_name). |
Collects aggregated query statistics from system.query_log.
| Field | Type | Default | Description |
|---|---|---|---|
query_metrics.enabled | boolean | true | Enable query metrics collection. Requires dbm: true. |
query_metrics.collection_interval | number | 10 | Collection interval in seconds. |
Collects currently running queries from system.processes.
| Field | Type | Default | Description |
|---|---|---|---|
query_samples.enabled | boolean | true | Enable query samples collection. Requires dbm: true. |
query_samples.collection_interval | number | 1 | Collection interval in seconds. |
query_samples.payload_row_limit | integer | 1000 | Maximum number of active queries per snapshot. |
Collects records of individual completed queries from system.query_log.
| Field | Type | Default | Description |
|---|---|---|---|
query_completions.enabled | boolean | true | Enable query completions collection. Requires dbm: true. |
query_completions.collection_interval | number | 10 | Collection interval in seconds. |
query_completions.samples_per_hour_per_query | number | 15 | Maximum samples collected per hour per unique query signature. |
Additional helpful documentation, links, and articles:
| |
