Queue Monitoring

Purpose and Scope

This document describes the QueueFeature class, which integrates Sentry with Hypervel's queue system to provide distributed tracing and monitoring for queue jobs. The feature captures breadcrumbs for job lifecycle events, creates performance spans for queue operations, and propagates trace context across asynchronous job boundaries to maintain distributed tracing continuity.

For information about the base feature system and lifecycle methods, see Feature System Overview. For other monitoring features, see Cache Monitoring and Redis Monitoring.

Sources: src/Features/QueueFeature.php1-288

---

Overview

The QueueFeature class monitors Hypervel queue operations by listening to queue lifecycle events and creating appropriate Sentry breadcrumbs and performance spans. It implements distributed tracing by injecting trace context into job payloads when jobs are queued and extracting it when jobs are processed, allowing child jobs to continue their parent's trace.

The feature supports three distinct monitoring capabilities:

1. Breadcrumbs - Records job processing events for error context
2. Queue Job Spans - Traces job execution as child spans of existing transactions
3. Queue Job Transactions - Creates standalone transactions for jobs without parent traces

Sources: src/Features/QueueFeature.php33-64

---

Configuration

The QueueFeature is controlled by three configuration flags:

Configuration Key	Purpose	Scope
queue_info	Enable breadcrumb creation for job events	Breadcrumbs
queue_jobs	Enable span creation for jobs within transactions	Tracing
queue_job_transactions	Enable transaction creation for standalone jobs	Tracing

The feature is only applicable if the queue container binding exists and at least one of these flags is enabled.

Sources: src/Features/QueueFeature.php49-64

---

Queue Lifecycle Integration

Event Listeners

The QueueFeature registers listeners for seven queue events during the boot phase:

Event	Handler Method	Purpose
JobQueueing	handleJobQueueingEvent	Update span description with job name
JobQueued	handleJobQueuedEvent	Finish queue.publish span
JobProcessing	handleJobProcessingQueueEvent	Create breadcrumb and start queue.process span
JobProcessed	handleJobProcessedQueueEvent	Finish queue.process span with success status
JobFailed	handleJobFailedEvent	Finish span with error status and pop scope
JobExceptionOccurred	handleJobExceptionOccurredQueueEvent	Mark span as failed and flush events
WorkerStopping	handleWorkerStoppingQueueEvent	Flush pending events to Sentry

Lifecycle Sequence Diagram:

Sources: src/Features/QueueFeature.php66-106 src/Features/QueueFeature.php108-243

---

Distributed Tracing

Trace Context Propagation

The QueueFeature implements distributed tracing by modifying job payloads to carry trace context across process boundaries. This ensures that asynchronous jobs continue their parent transaction's trace rather than creating disconnected traces.

Payload Injection Keys:

Payload Key	Content	Purpose
sentry_baggage_data	Baggage header string	Carries dynamic sampling context and metadata
sentry_trace_parent_data	Traceparent header string	Carries trace ID, span ID, and sampling decision
sentry_publish_time	Microtime timestamp	Enables latency calculation between publish and process

Trace Propagation Flow:

Sources: src/Features/QueueFeature.php78-105 src/Features/QueueFeature.php196-199

Trace Context Extraction

When a job is processed, the feature extracts the trace context from the payload and uses Sentry's continueTrace() function to reconstruct the parent trace state:

$baggage = $jobPayload[static::QUEUE_PAYLOAD_BAGGAGE_DATA] ?? null;
$traceParent = $jobPayload[static::QUEUE_PAYLOAD_TRACE_PARENT_DATA] ?? null;
$context = continueTrace($traceParent ?? '', $baggage ?? '');

The continueTrace() function returns a TransactionContext that preserves the parent trace ID and sampling decision. If the parent was not sampled ($context->getParentSampled() === false), the job span is also not created, maintaining sampling consistency across the distributed trace.

Sources: src/Features/QueueFeature.php196-204

---

Span Operations

The QueueFeature creates two types of spans, each with a distinct operation identifier:

Queue Publish Span (queue.publish)

Created when a job is dispatched to the queue, this span measures the time taken to serialize and queue the job. It is a child of the current active span in the publishing context.

Span Data:

• messaging.system: "hypervel"
• messaging.message.id: Job UUID
• messaging.destination.name: Normalized queue name
• messaging.destination.connection: Queue connection name
• Description: Job class name or "Closure"

The span is started in the createPayloadUsing callback and finished when the JobQueued event fires.

Sources: src/Features/QueueFeature.php84-95 src/Features/QueueFeature.php108-132

Queue Process Span (queue.process)

Created when a worker begins processing a job, this span measures the job's execution time. Depending on context, it can be:

• A child span if the job is part of an existing transaction (e.g., triggered by a web request)
• A standalone transaction if the job starts a new trace

Span Data:

• messaging.system: "hypervel"
• messaging.destination.name: Normalized queue name
• messaging.destination.connection: Queue connection name
• messaging.message.id: Job UUID
• messaging.message.envelope.size: Raw job payload size in bytes
• messaging.message.body.size: Serialized job data size in bytes
• messaging.message.retry.count: Number of previous attempts
• messaging.message.receive.latency: Milliseconds between publish and process
• Operation: "queue.process"
• Origin: "auto.queue"

Sources: src/Features/QueueFeature.php181-224

Span Creation Logic

Sources: src/Features/QueueFeature.php165-224

---

Breadcrumb Creation

When the queue_info breadcrumb feature is enabled, the QueueFeature records a breadcrumb each time a job begins processing. This breadcrumb provides context for errors that occur during job execution.

Breadcrumb Properties:

Property	Value
Level	INFO
Type	DEFAULT
Category	"queue.job"
Message	"Processing queue job"

Breadcrumb Data:

• job: Job name (from $job->getName())
• queue: Queue name (from $job->getQueue())
• attempts: Number of execution attempts
• connection: Queue connection name
• resolved: Resolved job class name (from $job->resolveName())

Sources: src/Features/QueueFeature.php147-163

---

Scope Management

The QueueFeature uses the TracksPushedScopesAndSpans trait to maintain a stack of scopes and spans. This is critical in queue workers that process multiple jobs sequentially, ensuring isolation between jobs.

Scope Lifecycle

Scope Reset

When a new job starts, the pushScope() method performs three critical operations:

1. Clears breadcrumbs - Removes breadcrumbs from previous jobs to prevent cross-contamination
2. Resets propagation context - Generates a new propagation context with fresh trace/span IDs
3. Pushes a new scope - Creates a fresh scope layer for the new job

This ensures that each job has isolated error context and doesn't inherit breadcrumbs or context from previous jobs processed by the same worker.

Sources: src/Features/QueueFeature.php35-37 src/Features/QueueFeature.php277-287

---

Queue Name Normalization

The normalizeQueueName() method handles platform-specific queue naming conventions to extract the canonical queue name:

Queue Format	Example	Normalized Name
AWS SQS URL	https://sqs.us-east-1.amazonaws.com/123/my-queue	my-queue
Redis Queue	queues:default	default
Simple Name	high-priority	high-priority

This normalization ensures consistent queue name reporting across different queue drivers and prevents verbose URLs from appearing in Sentry traces.

Sources: src/Features/QueueFeature.php245-258

---

Event Flushing

The QueueFeature explicitly flushes pending Sentry events in two scenarios:

1. Worker Stopping - When the WorkerStopping event fires, ensuring all events are sent before the worker terminates
2. Job Exception - When the JobExceptionOccurred event fires, ensuring critical exception data is transmitted immediately

This prevents data loss when workers are terminated gracefully or when jobs fail catastrophically.

Sources: src/Features/QueueFeature.php233-236 src/Features/QueueFeature.php238-243

---

Metrics and Data Collection

Messaging Metrics

The feature collects comprehensive metrics aligned with OpenTelemetry semantic conventions for messaging systems:

Publish Metrics:

• Message ID (job UUID)
• Destination (queue name and connection)
• System identifier ("hypervel")

Process Metrics:

• All publish metrics, plus:
• Envelope size (total payload bytes)
• Body size (job data bytes)
• Retry count (attempts - 1)
• Receive latency (time between queue and process)

These metrics enable performance analysis of queue throughput, job size impact, and retry patterns.

Sources: src/Features/QueueFeature.php86-92 src/Features/QueueFeature.php183-193

---

Summary

The QueueFeature provides comprehensive monitoring for Hypervel's queue system through:

• Distributed tracing via baggage and traceparent injection into job payloads
• Dual span types for both queueing (queue.publish) and processing (queue.process) operations
• Automatic transaction continuation for asynchronous jobs across process boundaries
• Scope isolation ensuring clean context separation between sequential jobs
• Rich breadcrumbs capturing job execution details for error investigation
• Metrics collection for performance analysis and bottleneck identification

The feature integrates seamlessly with Hypervel's event system and queue infrastructure, requiring no application code changes to enable comprehensive queue monitoring.

Sources: src/Features/QueueFeature.php1-288