VOOZH about

URL: https://deepwiki.com/friendsofhyperf/components/2.2-distributed-tracing-architecture

⇱ Distributed Tracing Architecture | friendsofhyperf/components | DeepWiki

Loading...
Loading...
Last indexed: 14 February 2026 (15d5ca)
Menu

Distributed Tracing Architecture

Purpose: This document explains Sentry's distributed tracing architecture, covering the transaction and span model, trace context propagation mechanisms, and how traces flow across service boundaries (HTTP, RPC, message queues). For configuration details, see Configuration and Initialization. For the AOP instrumentation implementation, see AOP-Based Instrumentation.

Overview

Sentry's distributed tracing system enables performance monitoring across multiple services and async boundaries in Hyperf applications. The architecture centers on three core concepts:

  1. Transactions - Top-level units representing complete operations (e.g., HTTP request, cron job, queue consumer)
  2. Spans - Child operations within a transaction (e.g., database query, Redis command, HTTP client request)
  3. Trace Context - Propagation data that links transactions and spans across service boundaries

The system is built on the official Sentry PHP SDK and extends it to support Hyperf's coroutine environment and message queue integrations.

Sources: src/sentry/src/Tracing/Listener/EventHandleListener.php1-755

Transaction and Span Hierarchy

Transaction Model

A transaction represents a complete unit of work and serves as the root of a trace. Every span must belong to a transaction. Transactions are created for:

  • HTTP requests (incoming)
  • RPC requests (incoming)
  • Console commands
  • Cron jobs
  • AMQP message consumption
  • Kafka message consumption
  • AsyncQueue job processing


Span Attributes


Each span carries structured data following OpenTelemetry semantic conventions:





































AttributeDescriptionExample
opOperation typedb.sql.query, http.client, cache.get
descriptionHuman-readable descriptionSELECT * FROM users WHERE id = ?
originInstrumentation sourceauto.db, auto.http.client, manual
dataStructured metadata{"db.system": "mysql", "server.address": "localhost"}
statusCompletion statusok, internal_error, unknown


Sources: src/sentry/src/Tracing/Listener/EventHandleListener.php170-216 src/sentry/src/Tracing/Aspect/GuzzleHttpClientAspect.php143-147




Starting Transactions


The startTransaction() Function


Transactions are initiated using the global startTransaction() helper, which wraps Sentry\State\Hub::startTransaction():




Transaction Context from Incoming Requests


When receiving an HTTP request, the system extracts trace context from headers to continue an existing trace:




Implementation in EventHandleListener:


src/sentry/src/Tracing/Listener/EventHandleListener.php267-341


The listener:


    

  1. Extracts trace context using Carrier::fromRequest($request) at line 305
    2. 

  2. Calls continueTrace($carrier->getSentryTrace(), $carrier->getBaggage()) at line 307
    3. 

  3. Configures transaction metadata (name, operation, data) at lines 308-313
    4. 

  4. Creates a child span for the request processing at lines 320-327
    5. 

  5. Uses defer() to ensure proper cleanup at line 331
    6. 



Sources: src/sentry/src/Tracing/Listener/EventHandleListener.php267-341




Creating Child Spans


The trace() Helper Function


Child spans are created using the trace() helper, which automatically manages span lifecycle:




The trace() function:


    

  1. Retrieves the current span from SentrySdk::getCurrentHub()->getSpan()
    2. 

  2. Creates a child span using $parentSpan->startChild($spanContext)
    3. 

  3. Executes the callback
    4. 

  4. Automatically calls $span->finish() on completion
    5. 



Span Creation in Aspects


AOP aspects use the trace() helper to instrument framework components. Example from database tracing:


src/sentry/src/Tracing/Aspect/DbAspect.php118-134


The aspect:


    

  • Wraps $proceedingJoinPoint->process() in a trace() call
    • 

  • Configures SpanContext with operation type, description, and metadata
    • 

  • Conditionally adds result data based on Feature::isTracingTagEnabled('db.result')
    • 



Sources: src/sentry/src/Tracing/Aspect/DbAspect.php118-134 src/sentry/src/Tracing/Listener/EventHandleListener.php206-216




Trace Context Propagation with Carrier


The Carrier Class


The Carrier class serializes and deserializes trace context for propagation across boundaries. It encapsulates:


    

  • sentry-trace header: Contains trace ID, span ID, and sampling decision
    • 

  • baggage header: Contains dynamic sampling context and custom data
    • 

  • Metadata: Additional context like message ID, publish time, producer info
    • 



Carrier Creation Methods



























MethodSourceUse Case
Carrier::fromRequest()HTTP request headersIncoming HTTP requests
Carrier::fromJson()Serialized JSON stringAMQP/Kafka message headers
Carrier::fromSpan()Existing spanOutgoing HTTP/RPC calls


Carrier Storage in Hyperf Context


The SentryContext class provides coroutine-safe storage:


src/sentry/src/SentryContext.php67-75


Methods:


    

  • SentryContext::setCarrier($carrier) - Store in current coroutine
    • 

  • SentryContext::getCarrier($coroutineId) - Retrieve from specific coroutine (defaults to current)
    • 



Sources: src/sentry/src/SentryContext.php67-75 src/sentry/src/Tracing/Listener/EventHandleListener.php305




Cross-Boundary Tracing


HTTP Client Instrumentation


When making HTTP requests via Guzzle, the GuzzleHttpClientAspect injects trace context into request headers:




Implementation:


src/sentry/src/Tracing/Aspect/GuzzleHttpClientAspect.php72-82


Key steps:


    

  1. Get current span from scope at line 74
    2. 

  2. Call $span->toTraceparent() to generate sentry-trace header at line 78
    3. 

  3. Call $span->toBaggage() to generate baggage header at line 77
    4. 

  4. Merge into existing headers and override options at line 82
    5. 



Opt-out mechanism: Set no_sentry_tracing option to true to disable tracing for specific requests (line 66).


Sources: src/sentry/src/Tracing/Aspect/GuzzleHttpClientAspect.php48-148 src/sentry/src/Constants.php63-65




RPC Instrumentation


RPC calls propagate trace context through Hyperf\Rpc\Context:




Two-phase instrumentation:


    

  1. 

    Path generation phase src/sentry/src/Tracing/Aspect/RpcAspect.php63-91:
    

      

    • Intercepts __generateRpcPath() at line 63
      • 

    • Extracts service and method information at lines 69-71
      • 

    • Creates SpanContext with RPC metadata at lines 80-88
      • 

    • Stores in SentryContext::setRpcSpanContext() at line 80
      • 

    

    2. 

  2. 

    Send phase src/sentry/src/Tracing/Aspect/RpcAspect.php93-130:
    

      

    • Retrieves stored SpanContext at line 95
      • 

    • Creates Carrier from current span at line 107
      • 

    • Injects into Rpc\Context as JSON at line 113
      • 

    • Wraps call in trace() at line 102
      • 

    

    3. 



Sources: src/sentry/src/Tracing/Aspect/RpcAspect.php1-132 src/sentry/src/Constants.php16




AMQP Message Queue Tracing


Producer Side (Outgoing)


The AmqpProducerAspect injects trace context into AMQP message headers:


src/sentry/src/Tracing/Aspect/AmqpProducerAspect.php (not in provided files, but referenced)


The aspect creates a Carrier from the current span and serializes it into the AMQP message's application_headers table.


Consumer Side (Incoming)


The EventHandleListener extracts trace context when consuming AMQP messages:


src/sentry/src/Tracing/Listener/EventHandleListener.php554-605


Processing flow:


    

  1. Extract AMQPMessage from event at line 566
    2. 

  2. Read application_headers table at line 568
    3. 

  3. Check for Constants::TRACE_CARRIER key at line 569
    4. 

  4. Deserialize with Carrier::fromJson() at line 570
    5. 

  5. Store in SentryContext::setCarrier($carrier) at line 571
    6. 

  6. Start transaction with continueTrace() at line 576
    7. 

  7. Add messaging metadata (system, operation, latency, etc.) at lines 581-595
    8. 



Sources: src/sentry/src/Tracing/Listener/EventHandleListener.php554-605 src/sentry/src/Constants.php16




Kafka Message Queue Tracing


Similar to AMQP, but uses Kafka message headers:


src/sentry/src/Tracing/Listener/EventHandleListener.php624-670


Key differences:


    

  1. Iterates through message headers at line 635
    2. 

  2. Checks header->getHeaderKey() for Constants::TRACE_CARRIER at line 636
    3. 

  3. Uses header->getValue() to deserialize at line 637
    4. 

  4. Transaction name includes topic: $consumer->getTopic() . ' process' at line 646
    5. 



Sources: src/sentry/src/Tracing/Listener/EventHandleListener.php624-670




AsyncQueue Job Tracing


AsyncQueue jobs use a two-phase approach due to job serialization:


Phase 1: Job Serialization (Producer)


The AsyncQueueJobMessageAspect intercepts job message creation:


[Referenced in ConfigProvider but source not provided]


The aspect wraps the job's serialization and includes the current trace context.


Phase 2: Job Execution (Consumer)


The EventHandleListener retrieves trace context from the parent coroutine:


src/sentry/src/Tracing/Listener/EventHandleListener.php685-717


Key line:




At line 691, it retrieves the carrier from the parent coroutine (the one that dispatched the job). This works because:


    

  1. AsyncQueue jobs run in child coroutines
    2. 

  2. The parent coroutine's context is copied during coroutine creation
    3. 

  3. SentryContext::getCarrier() accepts a $coroutineId parameter at line 72
    4. 



Sources: src/sentry/src/Tracing/Listener/EventHandleListener.php685-717 src/sentry/src/SentryContext.php72-75




gRPC Client Tracing


gRPC calls inject trace context directly into the request options:


src/sentry/src/Tracing/Aspect/GrpcAspect.php38-83


Flow:


    

  1. Get parent span at line 59
    2. 

  2. Check if sampled at line 62
    3. 

  3. Inject headers into options at lines 67-70:

      

    • sentry-trace header from $parent->toTraceparent()
      • 

    • baggage header from $parent->toBaggage()
      • 

    

    4. 

  4. Update request options at line 73
    5. 

  5. Execute in trace() wrapper at line 75
    6. 



Sources: src/sentry/src/Tracing/Aspect/GrpcAspect.php38-83 src/sentry/src/Constants.php63-65




Coroutine Context Propagation


The Challenge


Hyperf's coroutine model requires special handling to maintain trace context across coroutine boundaries. Without intervention, child coroutines would lose their parent's trace context.


Solution: CoroutineAspect


The Tracing\Aspect\CoroutineAspect intercepts coroutine creation:


src/sentry/src/Tracing/Aspect/CoroutineAspect.php45-106


Mechanism:




Implementation steps:


    

  1. 

    Check if should trace (lines 47-51):
    

      

    • Feature flag coroutine must be enabled
      • 

    • SentryContext::isTracingDisabled() must be false
      • 

    

    2. 

  2. 

    Detect calling function (line 54):
    

      

    • Uses CoroutineBacktraceHelper::foundCallingOnFunction() to identify the function creating the coroutine
      • 

    • Skips if not a recognized pattern
      • 

    

    3. 

  3. 

    Wrap callable (lines 67-95):
    

      

    • Restore Hyperf context keys (line 70)
      • 

    • Create new transaction with continueTrace() (lines 74-80)
      • 

    • Set transaction name to 'coroutine' and description to calling function
      • 

    • Execute original callable in nested trace() (lines 88-94)
      • 

    • Defer transaction finish and event flush (lines 83-86)
      • 

    

    4. 



Sources: src/sentry/src/Tracing/Aspect/CoroutineAspect.php45-106 src/sentry/src/SentryContext.php42-55




Trace Configuration


Tracing Feature Flags


The configuration provides granular control over tracing:


src/sentry/publish/sentry.php135-159


Top-level tracing flags:


    

  • tracing.request - HTTP/RPC request transactions
    • 

  • tracing.command - Console command transactions
    • 

  • tracing.crontab - Cron job transactions
    • 

  • tracing.amqp - AMQP consumer transactions
    • 

  • tracing.kafka - Kafka consumer transactions
    • 

  • tracing.async_queue - AsyncQueue job transactions
    • 

  • tracing.coroutine - Coroutine trace propagation
    • 



Span-level flags:


    

  • tracing_spans.db - Database query spans
    • 

  • tracing_spans.redis - Redis command spans
    • 

  • tracing_spans.elasticsearch - Elasticsearch operation spans
    • 

  • tracing_spans.cache - Cache operation spans
    • 

  • tracing_spans.guzzle - HTTP client request spans
    • 

  • tracing_spans.rpc - RPC call spans
    • 

  • tracing_spans.grpc - gRPC call spans
    • 



Tag flags (control data verbosity):


    

  • tracing_tags.db.sql.bindings - Include SQL parameter values
    • 

  • tracing_tags.db.result - Include query result
    • 

  • tracing_tags.redis.result - Include Redis command result
    • 

  • tracing_tags.http.response.body.contents - Include HTTP response body
    • 



Sampling Configuration


src/sentry/publish/sentry.php26-38


    

  • sample_rate (line 27) - Error sampling (1.0 = 100%)
    • 

  • traces_sample_rate (line 30) - Transaction sampling (1.0 = 100%)
    • 

  • traces_sampler (line 33) - Custom sampling function (commented example shows skipping /health endpoints)
    • 



Sources: src/sentry/publish/sentry.php26-38 src/sentry/publish/sentry.php135-171




Complete Tracing Flow Example


Multi-Service HTTP Request with Database and Cache




Resulting trace structure:


Transaction (Service A): GET /api/users/123
├── Span: db.sql.query - SELECT * FROM users
├── Span: db.redis - GET user:123:permissions
├── Transaction (Service B): RPC GetPermissions [linked via carrier]
│ └── Span: db.sql.query - SELECT * FROM permissions
└── Span: db.redis - SET user:123:permissions



Sources: src/sentry/src/Tracing/Listener/EventHandleListener.php267-341 src/sentry/src/Tracing/Aspect/RpcAspect.php93-130




Server Address Extraction


Purpose


To provide accurate server information in spans, the system needs to extract connection details (host, port) from various client libraries.


Extraction Aspects




Database Server Extraction


src/sentry/src/Tracing/Aspect/DbConnectionAspect.php38-67


Uses a WeakMap to associate PDO instances with server info:


    

  1. On createPdoConnection (line 45): Parse DSN with regex to extract host/port
    2. 

  2. On getPdo/getPdoForSelect (line 56): Retrieve from WeakMap and store in SentryContext
    3. 



Redis Server Extraction


src/sentry/src/Tracing/Aspect/RedisConnectionAspect.php38-87


    

  • For Redis instances: Call getHost() and getPort() at lines 49-50
    • 

  • For RedisCluster: Calculate slot from key, query CLUSTER SLOTS, find master node at lines 68-86

      

    • Uses RedisClusterKeySlot::get($key) for accurate slot calculation
      • 

    

    • 



RPC Server Extraction


src/sentry/src/Tracing/Aspect/RpcEndpointAspect.php33-65


Extracts from three transporter types:


    

  • Hyperf\RpcMultiplex\Socket: getName(), getPort()
    • 

  • Hyperf\LoadBalancer\Node: $result->host, $result->port
    • 

  • Hyperf\JsonRpc\Pool\RpcConnection: $socket->getSocketOption()->getHost()
    • 



Elasticsearch Server Extraction


src/sentry/src/Tracing/Aspect/ElasticsearchRequestAspect.php33-86


    

  • ES 8.x: Extract from RequestInterface URI at lines 41-48
    • 

  • ES 7.x: Extract from $client->transport->getLastConnection() at lines 54-62
    • 



Sources: src/sentry/src/Tracing/Aspect/DbConnectionAspect.php38-67 src/sentry/src/Tracing/Aspect/RedisConnectionAspect.php38-87 src/sentry/src/Tracing/Aspect/RpcEndpointAspect.php33-65 src/sentry/src/Tracing/Aspect/ElasticsearchRequestAspect.php33-86 src/sentry/src/SentryContext.php77-135
Refresh this wiki
On this page