Related
- Deploy MuleSoft App to CloudHub2 Using GitHub Actions CI/CD Pipeline
- Cutting Data Pipeline Costs and Data Freshness Issues With Netflix Maestro and Apache Iceberg: A Practical Tutorial
- Securing CI/CD Pipelines Against Supply Chain Attacks: Why Artifacts and Dependencies Matter More Than Ever
- Mastering GitHub Copilot in VS Code: Ask, Edit, Agent and the Build–Refine–Verify Workflow
Trending
- Engineering Closed-Loop Graph-RAG Systems, Part 2: From Prompts to Rules
- The Trust Problem in Modern SaaS: Why Your Authentication Succeeded, and You Still Got Breached
- Managing, Updating, and Organizing Agent Skills
- Observability for Agents and Workflows: Tracing Prompts, Tool Calls, and Business Outcomes End-to-End
- DZone
- Testing, Deployment, and Maintenance
- DevOps and CI/CD
- Advanced CI/CD Pipeline Optimization Techniques Using GitHub Actions
Advanced CI/CD Pipeline Optimization Techniques Using GitHub Actions
Explore advanced techniques to optimize CI/CD pipelines using GitHub Actions, which enhances efficiency and reliability for enterprise-level operations.
Join the DZone community and get the full member experience.
Join For FreeContinuous Integration and Continuous Deployment (CI/CD) pipelines are crucial for modern software development. This article explores advanced techniques to optimize these pipelines, enhancing efficiency and reliability for enterprise-level operations.
Parallelization Using Matrix Builds
GitHub Actions CI tests using the matrix strategy to run jobs in parallel:
jobs:
CI-Test:
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
node-version: [14.x, 16.x, 18.x]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
- run: npm ci
- run: npm test
The above job is executed across different platforms and Node.js versions simultaneously, thereby significantly reducing overall execution time.
Caching Dependencies for Faster Builds
Caching the dependencies and reusing the cache for future runs reduces build time.
- uses: actions/cache@v3
with:
path: ~/.npm
key: ${{ runner.OS }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-node-
The workflow caches npm dependencies and reuses the downloaded dependencies for future workflow runs, reducing build time and alleviating network congestion.
Reusable Workflows
Reusable workflows in GitHub Actions allow you to create a workflow with scripts that can be used across multiple workflows, improving efficiency and reducing code duplication.
Here is a reusable workflow that can be used to build Docker images.
repo/.github/workflows/reusable-docker-build-publish.yml@main
name: Docker Build and Publish Workflow
on:
workflow_call:
inputs:
dockerfile-path:
description: 'Path to Dockerfile'
required: true
image-name:
description: 'Name of the Docker image'
required: true
image-tag:
description: 'Image Tag'
required: true
secrets:
REGISTRY_URL:
description: 'Registry URL'
required: true
REGISTRY_USERNAME:
description: 'Registry username'
required: true
REGISTRY_PASSWORD:
description: 'Registry password'
required: true
jobs:
build-and-publish:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Log in to Registry
run: docker login "${{ secrets.REGISTRY_URL }}" -u "${{ secrets.REGISTRY_USERNAME }}" --password "${{ secrets.REGISTRY_PASSWORD }}"
- name: Build Docker image
run: docker build -t ${{ inputs.image-name }}:${{ inputs.image-tag }} -f ${{ inputs.dockerfile-path }} .
- name: Push Docker image
run: docker push ${{ inputs.image-name }}:${{ inputs.image-tag }}
The reusable workflow expects the following inputs:
| Input | Description | Required | Example |
|---|---|---|---|
| dockerfile-path | Path to Dockerfile | Yes | ./Dockerfile |
| image-name | Name of the Docker Image | Yes | my-image |
| image-tag | Name of the Docker Tag | Yes | 1.0.0, latest |
Other workflows can call the reusable workflow to build the Docker images.
name: Docker Build and Publish Workflow
on:
push:
branches:
- main
jobs:
build_publish_image:
uses: repo/.github/workflows/reusable-docker-build-publish.yml@main
with:
dockerfile-path: './Dockerfile'
image-name: 'my-org/my-app'
image-tag: 'latest'
secrets:
REGISTRY_URL: ${{ secrets.REGISTRY_URL }} # Make sure this matches the secret in your reusable workflow
REGISTRY_USERNAME: ${{ secrets.REGISTRY_USERNAME }} # Same here
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }} # And here
- Ensure that the secrets (
REGISTRY_URL,REGISTRY_USERNAME,REGISTRY_PASSWORD) are set in the repository's Secrets section under Settings > Secrets and Variables > Actions. - Modify the inputs as necessary to fit your repository's Docker image name, tag, and Dockerfile location.
Conditional Execution
Use conditional statements like "if" statements to avoid unnecessary workflow executions. For example, skip deployment steps based on certain conditions such as changes limited to specific branches, commits involving documentation changes, skipping PRs, etc.
jobs:
Branches:
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/test'
runs-on: ubuntu-latest
steps:
- run: npm test
// Check commit messages with words "docs"
CommitMessage:
if: "!contains(github.event.head_commit.message, 'docs')"
runs-on: ubuntu-latest
steps:
- run: echo "This is only executed if the commit message does not contain word 'docs'"
// Run the job only if there is a push to Main branch
Deploy:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
steps:
- run: echo "Deploying to production"
Job Concurrency
Cancel redundant jobs which could waste runner resources or overload the system.
concurrency:
group: ${{ github.ref }}
cancel-in-progress: true
Final Thought
By implementing the above strategies, you can greatly improve the speed and quality of software delivery pipelines.
Opinions expressed by DZone contributors are their own.
Related
-
Deploy MuleSoft App to CloudHub2 Using GitHub Actions CI/CD Pipeline
-
Cutting Data Pipeline Costs and Data Freshness Issues With Netflix Maestro and Apache Iceberg: A Practical Tutorial
-
Securing CI/CD Pipelines Against Supply Chain Attacks: Why Artifacts and Dependencies Matter More Than Ever
-
Mastering GitHub Copilot in VS Code: Ask, Edit, Agent and the Build–Refine–Verify Workflow
