Agent Service Architecture

The Agent Service Architecture is an experimental microservice-based system designed to serve autonomous LLM agents. Unlike the standard model-level inference services, the Agent Service exposes complete agent sessions—incorporating multi-turn conversation history, tool use, memory, and pluggable agent frameworks—via a set of independent HTTP microservices areal/experimental/agent_service/README.md1-8

System Overview

The architecture consists of four independent HTTP services that communicate via REST to provide scalable, session-affine agent execution areal/experimental/agent_service/README.md9-11

Key Components

Component	Role	Implementation
Gateway	Public entry point. Accepts WebSocket connections and HTTP requests (OpenResponses bridge) areal/experimental/agent_service/README.md34-36	areal.experimental.agent_service.gateway
Router	Session-affine routing service. Maintains session-to-DataProxy affinity and performs round-robin assignment areal/experimental/agent_service/README.md38-39	areal.experimental.agent_service.router
DataProxy	Stateful session proxy. Manages conversation history and pairs 1:1 with a Worker areal/experimental/agent_service/README.md41-43	areal.experimental.agent_service.data_proxy
Worker	Stateless agent execution server. Loads an AgentRunnable and executes single turns areal/experimental/agent_service/README.md45-48	areal.experimental.agent_service.worker
Controller	Orchestrator that manages the lifecycle of the entire microservice stack via Guards areal/experimental/agent_service/controller/controller.py66-75	AgentController

Sources: areal/experimental/agent_service/README.md32-49 areal/experimental/agent_service/controller/controller.py66-75

Component Interaction and Data Flow

The system ensures that multi-turn conversations are routed to the same stateful DataProxy to maintain context, while the Worker remains a stateless execution environment for the agent logic.

Multi-turn Conversation Flow

Sources: areal/experimental/agent_service/README.md13-30 areal/experimental/agent_service/README.md132-150

The Agent Service Controller

The AgentController manages the deployment and scaling of the service stack. It uses a Scheduler to launch Guard processes, which then fork the individual microservices areal/experimental/agent_service/controller/controller.py106-112

Lifecycle and Scaling

1. Guard Initialization: Creates Guard workers via the Scheduler (e.g., LocalScheduler). These workers run areal.experimental.agent_service.guard areal/experimental/agent_service/controller/controller.py126-146
2. Core Services: Forks the Router and Gateway on the first available Guard areal/experimental/agent_service/controller/controller.py148-186
3. Worker Pairs: Forks Worker and DataProxy pairs using scale_up(). The controller can add more pairs or use scale_down() to gracefully drain and remove them areal/experimental/agent_service/controller/controller.py223-248
4. Registration: Each DataProxy registers its address with the Router upon startup areal/experimental/agent_service/README.md38-39

Sources: areal/experimental/agent_service/controller/controller.py106-186 areal/experimental/agent_service/controller/controller.py223-248

Agent Protocol

Any agent implementation must satisfy the AgentRunnable protocol to be hosted by the Worker. This allows the service to remain agnostic to the specific agent framework areal/experimental/agent_service/README.md50-52

Key Interfaces

• AgentRunnable: Defines the run method which receives an AgentRequest (containing the message, session key, and history) and returns an AgentResponse areal/experimental/agent_service/README.md55-62 areal/experimental/agent_service/types.py19-24
• EventEmitter: A protocol for streaming deltas, tool calls, and tool results back through the service chain areal/experimental/agent_service/README.md89-94

Sources: areal/experimental/agent_service/README.md54-94 areal/experimental/agent_service/types.py19-26

Mapping Components to Code Entities

The following diagram bridges the logical agent service components to the specific code entities and file paths in the agent_service module.

Sources: areal/experimental/agent_service/README.md152-190 areal/experimental/agent_service/controller/controller.py66-75 areal/experimental/agent_service/gateway/__main__.py37-45 areal/experimental/agent_service/router/__main__.py34-39 areal/experimental/agent_service/data_proxy/__main__.py33-38 areal/experimental/agent_service/worker/__main__.py30-31

Integration Example: Claude Agent SDK

The architecture supports wrapping external SDKs into the microservice stack. The ClaudeAgent example demonstrates a session-persistent wrapper for the Anthropic Claude Agent SDK examples/agent_service/README.md5-9

Session Persistence in Worker

While the DataProxy manages conversation history for generic agents, some SDKs (like Claude's) maintain their own internal session state. In these cases, the Worker can maintain a persistent client instance mapped to the session key examples/agent_service/README.md66-68

1. Turn 1: Worker creates a ClaudeSDKClient for session "abc". The SDK runs autonomously (handling its own tool calls) and streams results back examples/agent_service/README.md71-74
2. Turn 2: Worker reuses the existing ClaudeSDKClient for session "abc", preserving internal context examples/agent_service/README.md76-78

Configuration and Deployment

The service is launched using the AgentController with an AgentConfig pointing to the ClaudeAgent implementation examples/agent_service/run_agent_service.py98-103 Environment variables like ANTHROPIC_API_KEY are passed via the SchedulingSpec to the child processes examples/agent_service/README.md111-112

Sources: examples/agent_service/README.md64-79 examples/agent_service/run_agent_service.py98-103 areal/experimental/agent_service/controller/controller.py126-131