While building my ETL pipeline project, I ran into a common beginner problem:

Running Apache Airflow locally on Windows with Docker was painful.

Problems included:

• Low disk space
• Docker setup issues
• Linux compatibility problems
• Environment debugging overhead

With only ~17GB free on my laptop, running multiple Airflow containers locally became difficult.

So I moved the entire setup to GitHub Codespaces.

What Codespaces Provided

Out of the box:

• Ubuntu Linux environment
• Docker pre-installed
• VS Code in browser
• Auto-cloned GitHub repo
• Port forwarding for Airflow UI

Workflow

docker-compose up

Then:

• Open Airflow UI
• Trigger ETL DAG
• Verify successful execution ✔️

Airflow was running in ~90 seconds.

Important Security Lesson

I accidentally committed my GCP service account key once.

GitHub Secret Scanning blocked the push automatically.

Immediately added:

*.json
.env

Never commit cloud credentials.

Why This Setup Helps Beginners

Codespaces removes a huge amount of local environment friction and lets you focus more on:

• Airflow orchestration
• ETL pipelines
• Docker workflows
• Cloud integrations

If you'd like:

• Beginner-friendly walkthrough → check Medium
• Engineering-focused breakdown → check Hashnode

Medium: Link
Hashnode: Link

Project Repo:ETL Pipeline

DataEngineering #Docker #ApacheAirflow #GitHubCodespaces