VOOZH about

URL: https://docs.datadoghq.com/llm_observability/instrumentation/

⇱ Agent Observability Instrumentation

For AI agents: A markdown version of this page is available at https://docs.datadoghq.com/llm_observability/instrumentation.md. A documentation index is available at /llms.txt.

Agent Observability Instrumentation

This product is not supported for your selected Datadog site. ().

To get started with Agent Observability, instrument your LLM application or agent(s) by choosing from several approaches based on your programming language and setup. Datadog provides comprehensive instrumentation options designed to capture detailed traces, metrics, and evaluations from your LLM applications and agents with minimal code changes.

Instrumentation Options

You can instrument your application with the Python, Node.js, or Java SDKs, or by using the Agent Observability API.

SDK-based instrumentation (Recommended)

Datadog provides native SDKs that offer the most comprehensive Agent Observability features:

LanguageSDK AvailableAuto-InstrumentationCustom Instrumentation
PythonPython 3.7+
Node.jsNode.js 16+
JavaJava 8+

To instrument an LLM application with the SDK:

  1. Install the Agent Observability SDK
  2. Configure the SDK by providing the required environment variables in your application startup command, or programmatically in-code. Ensure you have configured your Datadog API key, Datadog site, and machine learning (ML) app name.

Auto-instrumentation

Auto-instrumentation captures LLM calls for Python, Node.js, and Java applications without requiring code changes. It allows you to get out-of-the-box traces and observability into popular frameworks and providers. For additional details and a full list of supported frameworks and providers, see the Auto-instrumentation Documentation.

Auto-instrumentation automatically captures:

  • Input prompts and output completions
  • Token usage and costs
  • Latency and error information
  • Model parameters (temperature, max_tokens, etc.)
  • Framework-specific metadata
When using supported frameworks, no manual span creation is required for LLM calls. The SDK automatically creates appropriate spans with rich metadata.

Custom instrumentation

All supported SDKs provide advanced capabilities for custom instrumentation of your LLM applications in addition to auto-instrumentation, including:

  • Manual span creation using function decorators or context managers
  • Complex workflow tracing for multi-step LLM applications
  • Agent monitoring for autonomous LLM agents
  • Custom evaluations and quality measurements
  • Session tracking for user interactions

To learn more, see the SDK Reference Documentation.

HTTP API instrumentation

If your language is not supported by the SDKs or you are using custom integrations, you can instrument your application using Datadog’s HTTP API.

The API allows you to:

  • Submit spans directly via HTTP endpoints
  • Send custom evaluations associated with spans
  • Include full trace hierarchies for complex applications
  • Annotate spans with inputs, outputs, metadata, and metrics

API endpoints:

  • Spans API: POST https://api./api/intake/llm-obs/v1/trace/spans
  • Evaluations API: POST https://api./api/intake/llm-obs/v2/eval-metric

To learn more, see the HTTP API Documentation.

Further Reading