VOOZH about

URL: https://docs.datadoghq.com/observability_pipelines/destinations/google_secops/

⇱ Google SecOps Destination

For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/observability_pipelines/destinations/google_secops.md. A documentation index is available at /llms.txt.

Google SecOps Destination

This product is not supported for your selected Datadog site. ().
Available for:

Logs

Overview

Use Observability Pipelines’ Google SecOps destination to send logs to Google SecOps.

The Observability Pipelines Worker uses standard Google authentication methods. See Authentication methods at Google for more information about choosing the authentication method for your use case.

Setup

Configure the Google SecOps destination when you set up a pipeline. You can set up a pipeline in the UI, using the API, or with Terraform. The steps in this section are configured in the UI.

For Secrets Management: Only enter the identifier for the Google SecOps endpoint URL. Do not enter the actual value.
If you enter secret identifiers and then choose to use environment variables, the environment variable is the identifier entered and prepended with DD_OP_. For example, if you entered PASSWORD_1 for a password identifier, the environment variable for that password is DD_OP_PASSWORD_1.

After you select the Google SecOps destination in the pipeline UI:

  1. Enter the identifier for your Google SecOps endpoint URL. If you leave it blank, the default is used.
  2. Enter the customer ID for your Google SecOps instance.
  3. If you have a credentials JSON file, enter the path to your credentials JSON file. The credentials file must be placed under DD_OP_DATA_DIR/config. Alternatively, you can use the GOOGLE_APPLICATION_CREDENTIALS environment variable to provide the credential path.
  4. Select JSON or Raw encoding in the dropdown menu.
  5. Enter the log type. See template syntax if you want to route logs to different log types based on specific fields in your logs.

Optional buffering

Toggle the switch to enable Buffering Options. Enable a configurable buffer on your destination to ensure intermittent latency or an outage at the destination doesn’t create immediate backpressure, and allow events to continue to be ingested from your source. Disk buffers can also increase pipeline durability by writing data to disk, ensuring buffered data persists through a Worker restart. See Destination buffers for more information.

  • If left unconfigured, your destination uses a memory buffer with a capacity of 500 events.
  • To configure a buffer on your destination:
    1. Select the buffer type you want to set (Memory or Disk).
    2. Enter the buffer size and select the unit.
      1. Maximum memory buffer size is 128 GB.
      2. Maximum disk buffer size is 500 GB.
    3. In the Behavior on full buffer dropdown menu, select whether you want to block events or drop new events when the buffer is full.

Note: Logs sent to the Google SecOps destination must have ingestion labels. For example, if the logs are from a A10 load balancer, it must have the ingestion label A10_LOAD_BALANCER. See Google Cloud’s Support log types with a default parser for a list of available log types and their respective ingestion labels.

Secret defaults

These are the defaults used for secret identifiers and environment variables.

  • Google Chronicle endpoint URL identifier:
    • The default identifier is DESTINATION_GOOGLE_CHRONICLE_UNSTRUCTURED_ENDPOINT_URL.
  • Google SecOps endpoint URL:
    • The default environment variable is DD_OP_DESTINATION_GOOGLE_CHRONICLE_UNSTRUCTURED_ENDPOINT_URL.

How the destination works

Event batching

A batch of events is flushed when one of these parameters is met. See event batching for more information.

Maximum EventsMaximum Size (MB)Timeout (seconds)
None115