This project requires you to engineer a production-ready data storage architecture by managing data across multiple specialized layers. You will start by working with a MySQL OLTP (Online Transaction Processing)database to handle live, transactional e-commerce data. From there, you will design and implement a structured data warehouse solution, migrating the raw transactional records into a dedicated PostgreSQL staging warehouse before transforming them into production-ready schemas.

You will gain hands-on experience building lightweight, reliable ETL pipelines using core systems tools rather than heavy enterprise software. You will write functional Linux shell scripts to extract daily incremental transaction records from MySQL and load them into your staging environment. To ensure the pipeline operates without manual intervention, you will learn how to configure and schedule these automated scripts using Cron Jobs, a vital skill for managing routine data engineering workflows.

Data warehousing relies on optimized analytical structures. In this capstone, you will perform data transformations on staging tables to populate standard dimension tables and fact tables(forming a star schema model)optimized for rapid querying. Once your data is structured, you will export the tables into the production warehouse and connect them to IBM Cognos Analytics to design an executive reporting dashboard that reflects critical e-commerce business metrics.