 |
VOOZH | about |
|
VOOZH | about |
This page provides a deep dive into AReaL's dataclass-based configuration system. Configuration dataclasses define the structure and validation logic for all experiment parameters, from algorithm hyperparameters to distributed training settings. For information on how to use these configurations in practice (CLI arguments, YAML files), see Configuration Overview and CLI Arguments Reference For specific backend configurations, see Training Engine Configurations and Inference Engine Configurations
AReaL uses Python dataclasses (via the @dataclass decorator) to define structured, type-safe configuration objects. These dataclasses serve as the single source of truth for all configurable parameters, providing:
__post_init__ methods validate complex constraints. areal/api/cli_args.py80-96All core configuration dataclasses are defined in areal/api/cli_args.py41-210
Sources: areal/api/cli_args.py1-50
The following diagram bridges the Natural Language configuration concepts to the specific Code Entities that implement them.
Title: Configuration Entity Mapping
Sources: areal/api/cli_args.py41-210
NormConfig controls reward and advantage normalization in RL algorithms. It supports batch-level and group-level normalization with configurable mean/std computation. areal/api/cli_args.py43-78
| Field | Type | Default | Description |
|---|---|---|---|
mean_level | `str | None` | Mean normalization level: "batch", "group", or None |
mean_leave1out | bool | False | Use leave-one-out averaging (RLOO-style) |
std_level | `str | None` | Std deviation normalization level: "batch", "group", or None |
std_unbiased | bool | True | Use unbiased std computation (Bessel's correction) |
eps | float | 1e-5 | Epsilon for numerical stability when dividing by std |
group_size | int | 1 | Group size for group-level normalization |
Validation: The __post_init__ method at areal/api/cli_args.py80-96 validates that mean_level and std_level are in the valid set {"batch", "group", None}, and that group_size >= 1 when using group normalization. areal/api/cli_args.py91-96
Sources: areal/api/cli_args.py43-97
MicroBatchSpec defines how training batches are split into micro-batches for gradient accumulation and memory optimization. areal/api/cli_args.py100-140
| Field | Type | Default | Description |
|---|---|---|---|
n_mbs | `int | None` | Number of micro-batches (or minimum if max_tokens_per_mb is set) |
granularity | int | 1 | Group adjacent sequences by this size when dividing |
max_tokens_per_mb | `int | None` | Maximum tokens per micro-batch; enables dynamic splitting |
n_mbs_divisor | int | 1 | Final micro-batch count must be divisible by this |
packing_algorithm | str | "ffd" | Sequence packing algorithm: "ffd" or "kk" (Karmarkar-Karp) |
Validation: The __post_init__ method validates that the packing_algorithm is supported (one of ffd or kk). areal/api/cli_args.py141-147 The kk algorithm is recommended for large-scale RL with variable-length sequences to ensure load balance across DP ranks. areal/api/cli_args.py135-136
Sources: areal/api/cli_args.py100-161
GenerationHyperparameters controls text generation behavior during rollout phases. This dataclass is used by inference engines to configure sampling. areal/api/cli_args.py164-210
| Field | Type | Default | Notable Features |
|---|---|---|---|
n_samples | int | 1 | Number of sequences per prompt |
max_new_tokens | int | 16384 | Maximum generation length |
temperature | float | 1.0 | Sampling temperature |
top_p | float | 1.0 | Nucleus sampling threshold |
top_k | int | int(1e8) | Top-k sampling |
greedy | bool | False | Deterministic max probability |
stop_token_ids | list[int] | [] | Stop generation on specific IDs |
ignore_eos | bool | False | Continue past EOS tokens |
Sources: areal/api/cli_args.py164-210
The TrainEngineConfig serves as a container for core model parameters and specific backend configurations.
Title: TrainEngineConfig Component Mapping
Sources: docs/en/cli_reference.md35-42 docs/en/cli_reference.md75-81
FSDPEngineConfig configures FSDP2 (Fully Sharded Data Parallel version 2) training backend. It includes settings for wrap_policy, offload_params, and memory-efficient loading. docs/en/cli_reference.md35-36
MegatronEngineConfig configures Megatron-LM training backend with support for multi-dimensional parallelism. docs/en/cli_reference.md81
| Field | Type | Description |
|---|---|---|
virtual_pipeline_parallel_size | int | Virtual pipeline stages for interleaved schedule (VPP) |
fp8_config | `FP8EngineConfig | None` |
bridge_type | str | Type of bridge to use (e.g., "mbridge") |
Sources: docs/en/cli_reference.md78-81
ArchonEngineConfig configures the Archon engine, a torch-native training backend. It includes specific settings for attention types and pipeline schedules. docs/en/cli_reference.md75-76
| Field | Type | Description |
|---|---|---|
attn_type | str | Attention backend: varlen, sdpa, or tree |
pp_schedule | str | Pipeline schedule type (e.g., ScheduleZBVZeroBubble) |
Sources: docs/en/cli_reference.md75-76
Configuration dataclasses implement validation in __post_init__ methods to enforce complex constraints beyond type checking.
Sources: areal/api/cli_args.py80-96
The configuration system uses a three-level precedence: (1) dataclass defaults provide fallback values, (2) YAML files specify comprehensive experiment setups via --config, (3) CLI arguments override specific fields using dot-notation (e.g., actor.lr=1e-4). docs/en/cli_reference.md7-19
Title: Configuration Processing Pipeline
Sources: areal/api/cli_args.py13-17 docs/en/cli_reference.md7-19
To add a new configuration field:
areal/api/cli_args.py with type annotation and default value using field(). areal/api/cli_args.py43-47__post_init__ if the field has specific constraints. areal/api/cli_args.py80-83TrainEngine implementation) to read the new field from the provided config object.Sources: areal/api/cli_args.py1-40 docs/en/cli_reference.md3-5
Refresh this wiki