Glossary

This page provides definitions and technical details for terms, jargon, and abbreviations specific to the AReaL codebase. It serves as a reference for onboarding engineers to understand the implementation and data flow of the system.

Core System Components

The AReaL architecture is divided into high-level orchestrators, training backends, and inference services.

Term	Definition	Key Code Entities
Trainer	The top-level orchestrator that manages the training loop, dataset loading, and coordination between rollout and optimization.	PPOTrainer, GRPOTrainer, SFTTrainer (in areal/trainer/)
TrainEngine	Abstract interface for training backends responsible for forward/backward passes, gradient updates, and weight synchronization.	TrainEngine areal/api/engine_api.py32-230 FSDPEngine areal/engine/fsdp_engine.py218-1240 MegatronEngine areal/engine/megatron_engine.py168-1240 ArchonEngine areal/experimental/engine/archon_engine.py147-1110
InferenceEngine	Interface for generation services. In AReaL, these are typically remote servers (SGLang/vLLM) managed via an async API.	InferenceEngine areal/api/engine_api.py234-315 SGLangBackend areal/engine/sglang_remote.py40-191 VLLMBackend areal/engine/vllm_remote.py41-182
RolloutWorkflow	Defines the logic for a single episode (e.g., multi-turn chat, tool use, or simple prompt-response).	RolloutWorkflow areal/api/workflow_api.py RLVRWorkflow areal/workflow/rlvr.py MultiTurnWorkflow areal/workflow/multi_turn.py
Scheduler	Manages the lifecycle of distributed workers across different backends (Local, Ray, Slurm).	Scheduler areal/api/engine_api.py133 RayScheduler areal/api/cli_args.py132

Sources: areal/api/engine_api.py32-315 areal/engine/fsdp_engine.py218-1240 areal/engine/megatron_engine.py168-1240 README.md15-39 areal/experimental/engine/archon_engine.py147-1110

---

Technical Jargon & Abbreviations

1. Parallelism Dimensions

AReaL supports multi-dimensional parallelism to scale large models.

• DP (Data Parallelism): Splitting data across multiple GPUs. Managed via dp_group in engines areal/engine/fsdp_engine.py211
• TP (Tensor Parallelism): Splitting individual weights/tensors across GPUs. Supported in MegatronEngine areal/engine/megatron_engine.py22-23 and ArchonEngine via parallel_dims areal/experimental/engine/archon_engine.py178
• PP (Pipeline Parallelism): Splitting model layers across different GPUs in a pipeline. Supported natively in ArchonEngine areal/experimental/engine/archon_engine.py183-187 and MegatronEngine areal/engine/megatron_engine.py29-30
• CP (Context Parallelism): Splitting the sequence dimension (e.g., Ulysses) for long-context support. areal/engine/fsdp_engine.py89-94
• EP (Expert Parallelism): For MoE models, routing different experts to different GPUs. areal/engine/fsdp_engine.py72-78

2. Weight Versioning

In Fully Asynchronous RL, the training engine updates weights while the inference engine generates rollouts. To prevent training on stale data, AReaL uses a versioning system.

• WeightUpdateMeta: A structure containing the weight version, storage path, and synchronization method (XCCL vs Disk). areal/api/io_struct.py183-211
• Off-policyness: The lag between the model version used for rollout and the current training version. Tracked via _version in engines. areal/engine/fsdp_engine.py186

3. Data Structures

• MicroBatchSpec: Configuration for splitting a global batch into smaller chunks to fit in GPU memory. areal/api/cli_args.py99-138
• ModelRequest / ModelResponse: Standardized IO structures for communicating with inference backends. areal/api/io_struct.py28-131
• TrieNode: Used in Tree Attention or Tree Training to represent shared prefixes in a batch of sequences to save computation. areal/engine/fsdp_engine.py162

Sources: areal/api/cli_args.py99-138 areal/api/io_struct.py28-211 areal/engine/fsdp_engine.py89-94 areal/engine/megatron_engine.py22-30 areal/experimental/engine/archon_engine.py124-144

---

Code Entity Space Mapping

The following diagrams bridge Natural Language concepts to specific Code Entities.

Diagram 1: Training Execution Flow

This diagram shows how a Trainer uses a TrainEngine to process data via the MicroBatch system.

Sources: areal/api/engine_api.py32-230 areal/engine/fsdp_engine.py118-124 areal/utils/data.py105-145 areal/engine/fsdp_utils/grad.py84

Diagram 2: Remote Inference & Weight Sync

This diagram illustrates the relationship between the InferenceEngine and the weight update protocol.

Sources: areal/api/io_struct.py183-211 areal/engine/sglang_remote.py129-159 areal/engine/sglang_remote.py44-89

---

Detailed Glossary Table

Term	Implementation Details	File Pointer
allocation_mode	A pattern-based GPU parallel strategy allocation mode (legacy, moving to per-engine backend fields).	BaseExperiment.allocation_mode docs/en/cli_reference.md104
Ulysses	A specific implementation of Context Parallelism that uses all_to_all to shard sequences across the sp_group.	ulysses_prepare_inputs areal/engine/fsdp_engine.py93
AnyPrecisionAdamW	A custom optimizer supporting flexible precision for weights and gradients.	AnyPrecisionAdamW areal/engine/fsdp_engine.py85
Tree Training	A technique to optimize training on multiple completions of the same prompt by using a Trie to avoid redundant computation.	build_packed_tree_batch areal/engine/fsdp_engine.py106
LoRA Versioning	A system where LoRA adapters are saved with version suffixes (e.g., -v1) to support async rollout.	get_versioned_lora_name areal/api/io_struct.py161-163
MicroBatchList	A container for tensors that have been split into micro-batches, handling padding and sequence packing metadata.	MicroBatchList areal/utils/data.py118
Archon	A torch-native training backend supporting pipeline parallelism and custom parallelism dimensions.	ArchonEngine areal/experimental/engine/archon_engine.py147-196
NormConfig	Configuration for reward and advantage normalization, supporting batch or group-level statistics.	NormConfig areal/api/cli_args.py42-96
GenerationHyperparameters	Controls text generation behavior (temperature, top_p, max_new_tokens, etc.) for rollout.	GenerationHyperparameters areal/api/cli_args.py163-212
Packing Algorithm	Algorithms like 'ffd' (First Fit Decreasing) or 'kk' (Karmarkar-Karp) used to allocate sequences into micro-batches.	MicroBatchSpec.packing_algorithm areal/api/cli_args.py126-138
InteractionCache	System for caching and tracking agent interactions, including parent-child relationships and rewards.	InteractionCache areal/experimental/openai/client.py56
ArealOpenAI	An OpenAI-compatible client wrapper that integrates with AReaL's training and reward systems.	ArealOpenAI areal/experimental/openai/client.py12-61

Sources: areal/api/cli_args.py42-212 areal/api/io_struct.py161-163 areal/engine/fsdp_engine.py85-106 areal/experimental/engine/archon_engine.py147-196 areal/utils/data.py118 areal/experimental/openai/client.py12-61