 |
VOOZH | about |
|
VOOZH | about |
This page documents AReaL's continuous integration and continuous deployment (CI/CD) infrastructure, including GitHub Actions workflows, automated testing, Docker image builds, and release management.
AReaL's CI/CD system comprises:
All workflows are triggered by pull requests, releases, or manual dispatch, ensuring code quality and reproducibility across development, testing, and production environments.
Sources: .github/workflows/test-areal.yml1-35 .github/workflows/build-docker-image.yml1-20
| Workflow File | Trigger Events | Runner Type | Purpose |
|---|---|---|---|
test-areal.yml | PR with safe-to-test label, workflow_dispatch, workflow_call | GCP a2-highgpu-2g (2x A100) | Unit and integration tests (SGLang/vLLM variants) |
build-docker-image.yml | workflow_dispatch | GCP areal-docker-builder → ephemeral test runners | Development Docker image build, test, and promotion to :dev |
tag-release-image.yml | release (created), workflow_dispatch | GCP areal-docker-builder | Build and tag official release images (e.g., v1.0.4-sglang) |
deploy-docs.yml | Push to main/add_chinese_doc | ubuntu-latest | Jupyter Book documentation build/deploy |
install-test.yml | PR/Push (main) affecting pyproject.toml or areal/ | ubuntu-latest, macos-latest | Validates package installation and core imports |
stale-issues.yml | Daily schedule (0:00 UTC), workflow_dispatch | ubuntu-latest | Automatic issue/PR staleness management |
runner-heartbeat.yml | Weekly schedule (every 7 days), workflow_dispatch | GCP areal-docker-builder | Self-hosted runner health check |
bake-gcp-image.yml | workflow_dispatch, workflow_call | GCP a2-highgpu-2g | Pre-pulls Docker images into a new GCP OS image |
Sources: .github/workflows/test-areal.yml3-35 .github/workflows/build-docker-image.yml3-8 .github/workflows/tag-release-image.yml3-12 .github/workflows/deploy-docs.yml3-6 .github/workflows/install-test.yml3-23 .github/workflows/bake-gcp-image.yml3-31
The following diagram maps the Natural Language lifecycle of a code change to the specific GitHub Actions entities and GCP resources.
Natural Language Space to Code Entity Space: Development Pipeline
Sources: .github/workflows/test-areal.yml45-238 .github/workflows/build-docker-image.yml21-218 .github/workflows/tag-release-image.yml25-220 .github/workflows/install-test.yml28-182
The test-areal.yml workflow provisions ephemeral GCP instances with GPU hardware. It supports a matrix of inference engine variants (sglang, vllm) and test types (unit, integration) to ensure compatibility across different backends.
Sources: .github/workflows/test-areal.yml46-75
The workflow uses a custom OS image defined in env.GCP_OS_IMAGE (e.g., areal-cicd-test-20260506-432).
Startup Script Behavior: The rendered startup script (.github/workflows/test-areal.yml124-189):
areal-cicd using the specified CONTAINER_IMAGE with --runtime=nvidia --gpus all.--shm-size="58205394001.92b") required for distributed training.github-script (.github/workflows/test-areal.yml180-185).Resource Specifications:
a2-highgpu-2g (2x NVIDIA A100 GPUs).--max-run-duration).Sources: .github/workflows/test-areal.yml40-44 .github/workflows/test-areal.yml207-238
The install-test.yml workflow ensures the package remains installable across environments without requiring GPU hardware for the initial check.
uv sync and core imports (TrainController, RolloutController, WorkflowExecutor, StalenessManager) on Ubuntu and macOS (.github/workflows/install-test.yml53-73).sglang, vllm, and megatron extras on Linux. It specifically handles the conflict between sglang and vllm by swapping pyproject.toml and uv.lock files (.github/workflows/install-test.yml82-136).areal/tools/validate_docker_installation.py within the areal-runtime container to verify the pre-installed environment (.github/workflows/install-test.yml137-182).Sources: .github/workflows/install-test.yml1-182
The build-docker-image.yml workflow manages the lifecycle of the areal-runtime image for development:
areal-docker-builder instance on GCP (.github/workflows/build-docker-image.yml22-69).sglang and vllm variants are built using docker/build-push-action@v7 (.github/workflows/build-docker-image.yml135-154).:test tag.test-areal.yml as a downstream job using the newly built :test images (.github/workflows/build-docker-image.yml164-183).:test images are pulled, retagged as :dev, and pushed back to GHCR (.github/workflows/build-docker-image.yml184-210).bake-gcp-image.yml to create a new GCP OS image that has these :dev images pre-pulled (.github/workflows/build-docker-image.yml211-218).Sources: .github/workflows/build-docker-image.yml114-218 .github/workflows/bake-gcp-image.yml108-124
The tag-release-image.yml workflow is triggered by official releases. It extracts the version from pyproject.toml (.github/workflows/tag-release-image.yml133-145) and builds production-ready images for both sglang and vllm variants, tagging them with the release tag (e.g., v1.0.4-sglang) and updating the :latest tag (.github/workflows/tag-release-image.yml158-182).
Sources: .github/workflows/tag-release-image.yml1-220
The areal/version.py module provides the single source of truth for versioning by combining package metadata with Git state.
importlib.metadata.version("areal") (areal/version.py9-15).VersionInfo class executes shell commands to retrieve the current branch, short commit hash, and dirty state (areal/version.py61-88).0.1.0-a1b2c3d-dirty (areal/version.py43-58).Sources: areal/version.py1-98
The bake-gcp-image.yml workflow optimizes CI startup times by pre-pulling heavy Docker images into a GCP disk image.
Natural Language Space to Code Entity Space: Baking Workflow
Sources: .github/workflows/bake-gcp-image.yml73-132 .github/workflows/bake-gcp-image.yml211-238
The deploy-docs.yml workflow builds the Jupyter Book using docs/build_all.sh and deploys the resulting HTML to GitHub Pages whenever changes are pushed to main (.github/workflows/deploy-docs.yml47-72).
Refresh this wiki