VOOZH about

URL: https://docs.datadoghq.com/integrations/google-cloud-platform/

⇱ Google Cloud Platform

For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/integrations/google-cloud-platform.md. A documentation index is available at /llms.txt.

Google Cloud Platform

To find out if this integration is available in your organization, see your Datadog Integrations page or ask your organization administrator.

To initiate an exception request to enable this integration for your organization, email support@ddog-gov.com.

Overview

Use this guide to get started monitoring your Google Cloud environment. This approach simplifies the setup for Google Cloud environments with multiple projects, allowing you to maximize your monitoring coverage.

Setup

Set up Datadog’s Google Cloud integration to collect metrics and logs from your Google Cloud services.

Prerequisites

1. If your organization restricts identities by domain, you must add Datadog’s customer identity as an allowed value in your policy. Datadog’s customer identity: C0147pk0i

1. If your organization restricts identities by domain, you must add Datadog’s customer identity as an allowed value in your policy. Datadog’s customer identity: C03lf3ewa

2. Enable the following APIs for every project you want to monitor, including the project where the service account has been created.

Service account impersonation and automatic project discovery relies on you having certain roles and APIs enabled to monitor projects. Complete this step to avert integration issues.
Cloud Monitoring API
Allows Datadog to query your Google Cloud metric data.
Compute Engine API
Allows Datadog to discover compute instance data.
Cloud Asset API
Allows Datadog to request Google Cloud resources and link relevant labels to metrics as tags.
Cloud Resource Manager API
Allows Datadog to append metrics with the correct resources and tags.
IAM API
Allows Datadog to authenticate with Google Cloud.
Cloud Billing API
Allows developers to manage billing for their Google Cloud Platform projects programmatically. See the Cloud Cost Management (CCM) documentation for more information.

3. Ensure that any projects being monitored are not configured as scoping projects that pull in metrics from multiple other projects.

Metric collection

Installation

Organization-level (or folder-level) monitoring is recommended for comprehensive coverage of all projects, including any future projects that may be created in an org or folder.

Note: Your Google Cloud Identity user account must have the Admin role assigned to it at the desired scope to complete the setup in Google Cloud (for example, Organization Admin).

Metrics appear in Datadog approximately 15 minutes after setup.

Best practices for monitoring multiple projects

Enable per-project cost and API quota attribution

By default, Google Cloud attributes the cost of monitoring API calls, as well as API quota usage, to the project containing the service account for this integration. As a best practice for Google Cloud environments with multiple projects, enable per-project cost attribution of monitoring API calls and API quota usage. With this enabled, costs and quota usage are attributed to the project being queried, rather than the project containing the service account. This provides visibility into the monitoring costs incurred by each project, and also helps to prevent reaching API rate limits.

To enable this feature:

  1. Ensure that the Datadog service account has the Service Usage Consumer role at the desired scope (folder or organization).
  2. Click the Enable Per Project Quota toggle in the Projects tab of the Google Cloud integration page.

You can use service account impersonation and automatic project discovery to integrate Datadog with Google Cloud. Note: Projects with a prefix of sys- will not be picked up as part of project discovery.

This method enables you to monitor all projects visible to a service account by assigning IAM roles in the relevant projects. You can assign these roles to projects individually, or you can configure Datadog to monitor groups of projects by assigning these roles at the organization or folder level. Assigning roles in this way allows Datadog to automatically discover and monitor all projects in the given scope, including any new projects that may be added to the group in the future.

Validation

To view your metrics, use the left menu to navigate to Metrics > Summary and search for gcp:

Configuration

Leveraging the Datadog Agent

Use the Datadog Agent to collect the most granular, low-latency metrics from your infrastructure. Install the Agent on any host, including GKE, to get deeper insights from the traces and logs it can collect. For more information, see Why should I install the Datadog Agent on my cloud instances?

Log collection

See the Google Cloud Log Forwarding Setup page for log forwarding setup options and instructions.

Expanded BigQuery monitoring

Expanded BigQuery monitoring provides granular visibility into your BigQuery environments. See the BigQuery Data Observability documentation for more information.

Resource changes collection

Select Enable Resource Collection in the Resource Collection tab of the Google Cloud integration page. This allows you to receive resource events in Datadog when Google’s Cloud Asset Inventory detects changes in your cloud resources.

Then, follow the steps below to forward change events from a Pub/Sub topic to the Datadog Event Explorer.

Datadog recommends setting the asset-types parameter to the regular expression .* to collect changes for all resources.

Note: You must specify at least one value for either the asset-names or asset-types parameter.

See the gcloud asset feeds create reference for the full list of configurable parameters.

Enable resource changes collection

Click to Enable Resource Changes Collection in the Resource Collection tab of the Google Cloud integration page.

Validation

Find your asset change events in the Datadog Event Explorer.

Private Service Connect

Private Service Connect is only available for the US5 and EU Datadog sites.

Use the Google Cloud Private Service Connect integration to visualize connections, data transferred, and dropped packets through Private Service Connect. This gives you visibility into important metrics from your Private Service Connect connections, both for producers as well as consumers. Private Service Connect (PSC) is a Google Cloud networking product that enables you to access Google Cloud services, third-party partner services, and company-owned applications directly from your Virtual Private Cloud (VPC).

See Access Datadog privately and monitor your Google Cloud Private Service Connect usage in the Datadog blog for more information.

Data Collected

Metrics

gcp.gce.instance.cpu.utilization
(gauge)		Fraction of the allocated CPU that is currently in use on the instance. Note that some machine types allow bursting above 100% usage.
Shown as fraction

Cumulative metrics

Cumulative metrics are imported into Datadog with a .delta metric for each metric name. A cumulative metric is a metric where the value constantly increases over time. For example, a metric for sent bytes might be cumulative. Each value records the total number of bytes sent by a service at that time. The delta value represents the change since the previous measurement.

For example:

gcp.gke.container.restart_count is a CUMULATIVE metric. While importing this metric as a cumulative metric, Datadog adds the gcp.gke.container.restart_count.delta metric which includes the delta values (as opposed to the aggregate value emitted as part of the CUMULATIVE metric). See Google Cloud metric kinds for more information.

Events

All service events generated by your Google Cloud Platform are forwarded to your Datadog Events Explorer.

Service Checks

The Google Cloud Platform integration does not include any service checks.

Tags

Tags are automatically assigned based on a variety of Google Cloud Platform and Google Compute Engine configuration options. The project_id tag is added to all metrics. Additional tags are collected from the Google Cloud Platform when available, and varies based on metric type.

Additionally, Datadog collects the following as tags:

  • Any hosts with <key>:<value> labels.
  • Custom labels from Google Pub/Sub, GCE, Cloud SQL, and Cloud Storage.

Troubleshooting

Incorrect metadata for user defined gcp.logging metrics?

For non-standard gcp.logging metrics, such as metrics beyond Datadog’s out of the box logging metrics, the metadata applied may not be consistent with Google Cloud Logging.

In these cases, the metadata should be manually set by navigating to the metric summary page, searching and selecting the metric in question, and clicking the pencil icon next to the metadata.

Need help? Contact Datadog support.

Further Reading