Job Dispatching and Processing

This document covers the job dispatching and processing system within Hypervel's queue infrastructure. It explains how job classes are created, dispatched to queues, processed by workers, and handled when they fail. This includes job serialization, unique job constraints, job batching, and the complete job lifecycle from dispatch to completion or failure.

For information about queue architecture, driver configuration, and worker setup, see Queue Architecture and Configuration. For asynchronous event broadcasting that uses the queue system, see Event Broadcasting and Anonymous Events.

Job Classes

Job classes represent units of work to be processed asynchronously. They are plain PHP classes that implement queueable interfaces and contain the business logic to be executed by queue workers.

Job Structure

Jobs in Hypervel follow a standard structure:

Sources: src/queue/composer.json1-64 src/bus/composer.json1-60

Job Serialization

Jobs are serialized before being pushed to the queue and deserialized when processed. This is handled by laravel/serializable-closure which allows closures and object state to be safely stored in queue storage.

Jobs can contain serializable closures, allowing inline job definitions:

Sources: src/queue/composer.json31-32 src/encryption/composer.json1-48 composer.json127

Job Dispatching Methods

Jobs are dispatched using several methods provided by the Dispatchable trait and helper functions.

Dispatch Methods

The dispatch() helper function and static dispatch() method create a PendingDispatch instance that allows fluent configuration before pushing to the queue:

Sources: src/bus/composer.json1-60 src/queue/composer.json1-64

Synchronous vs Asynchronous Dispatch

The dispatchSync() method executes the job immediately in the current process, useful for testing or when queue workers are not available:

Method	Execution	Use Case
dispatch()	Asynchronous via queue worker	Production job processing
dispatchSync()	Synchronous in current process	Testing, development
dispatchAfterResponse()	After HTTP response sent	Quick response, delayed processing

Sources: src/bus/composer.json1-60

Job Middleware

Jobs can have middleware that wraps the job execution, allowing cross-cutting concerns like rate limiting or transaction management.

Job middleware is defined in the job's middleware() method:

Sources: src/bus/composer.json1-60 src/queue/composer.json1-64

Unique Jobs

Unique jobs prevent duplicate jobs from being queued or processed simultaneously. This is implemented using cache locks with job identifiers.

Unique Job Implementation

Jobs implement unique constraints by:

1. Defining a uniqueId() method that returns a unique identifier
2. Implementing ShouldBeUnique interface
3. Optionally setting $uniqueFor property to control lock duration

Sources: src/bus/composer.json33-34 src/cache/composer.json1-53 src/queue/composer.json1-64

Job Batching

Job batching allows dispatching multiple jobs as a single unit, tracking their collective progress and handling batch completion or failure.

Batch Architecture

Batch Lifecycle:

Stage	Description	Event
Creation	Batch created with jobs	Bus::batch()
Dispatching	All jobs dispatched to queue	Jobs queued
Processing	Workers process jobs	Jobs run independently
Completion	All jobs finished successfully	then() callback
Failure	Any job failed	catch() callback
Finalization	Batch processing complete	finally() callback

Batch State Tracking

Batches track state in cache storage, allowing progress monitoring:

• $batch->totalJobs - Total jobs in batch
• $batch->processedJobs() - Jobs completed
• $batch->pendingJobs - Jobs remaining
• $batch->failedJobs - Jobs that failed
• $batch->progress() - Completion percentage
• $batch->finished() - All jobs complete
• $batch->cancelled() - Batch cancelled

Sources: src/bus/composer.json1-60 src/cache/composer.json1-53 composer.json127

Failed Jobs

When jobs fail after exhausting retry attempts, they are stored for inspection and manual retry.

Failed Job Handling Flow

Failed Job Storage:

Failed jobs are stored with complete context for debugging:

Field	Description
id	Unique identifier
uuid	Job UUID
connection	Queue connection name
queue	Queue name
payload	Serialized job
exception	Exception message and trace
failed_at	Timestamp

Job Retry Configuration

Jobs configure retry behavior through properties:

Sources: src/queue/composer.json1-64 composer.json141

Job Lifecycle

The complete lifecycle of a job from creation to completion involves multiple stages and components.

Job Processing Stages

1. Creation: Job instance created with dependencies
2. Serialization: Job and properties serialized using SerializableClosure
3. Encryption: Payload optionally encrypted with Encrypter
4. Queuing: Payload pushed to queue driver (Redis/Database)
5. Reservation: Worker reserves job from queue
6. Deserialization: Payload decrypted and deserialized
7. Middleware: Job middleware pipeline executed
8. Execution: handle() method invoked
9. Completion: Job deleted from queue or moved to failed jobs

Sources: src/queue/composer.json1-64 src/bus/composer.json1-60 src/encryption/composer.json1-48

Job Events

The queue system dispatches events during job processing for monitoring and logging.

Applications can listen to these events to:

• Log job execution
• Monitor queue performance
• Send failure notifications
• Track job metrics
• Debug job issues

Sources: src/queue/composer.json1-64 src/event/composer.json1-55 src/telescope/composer.json1-53

Job Chaining

Jobs can be chained to execute sequentially, with each job waiting for the previous to complete.

Job chains are configured using the withChain() method:

If any job in the chain fails, subsequent jobs are not dispatched. Chain jobs share the same queue and connection configuration.

Sources: src/bus/composer.json1-60 src/queue/composer.json1-64

Queue Worker Integration

Jobs are processed by queue workers which continuously poll queue drivers for new jobs.

Worker Process Flow

Workers are started via console commands:

• queue:work - Start processing jobs
• queue:listen - Start worker (restarts on each job)
• queue:restart - Gracefully restart all workers

Sources: src/queue/composer.json1-64 src/console/composer.json1-47 composer.json114

Coroutine Safety

Since Hypervel runs on Swoole, all queue operations are coroutine-safe through Context isolation.

Key coroutine safety features:

• Context Isolation: Each job runs in isolated coroutine context
• Connection Pooling: Queue connections pooled for concurrent access
• Atomic Operations: Cache locks ensure unique job atomicity
• Coordinator Integration: Graceful shutdown coordination across coroutines

Sources: src/queue/composer.json24-25 composer.json119 src/cache/composer.json1-53