Data Engineering with Rust
Ends soon! Keep adding new skills with 10,000+ programs for $239 (usually $399). Save now.
Data Engineering with Rust
This course is part of Rust Programming Specialization
Instructors: Noah Gift
6,743 already enrolled
Included with
Learn more
70 reviews
Recommended experience
70 reviews
Recommended experience
Skills you'll gain
Details to know
14 assignments
See how employees at top companies are mastering in-demand skills
Build your subject-matter expertise
- Learn new concepts from industry experts
- Gain a foundational understanding of a subject or tool
- Develop job-relevant skills with hands-on projects
- Earn a shareable career certificate
There are 4 modules in this course
Are you a data engineer, software developer, or a tech enthusiast with a basic understanding of Rust, seeking to enhance your skills and dive deep into the realm of data engineering with Rust? Or are you a professional from another programming language background, aiming to explore the efficiency, safety, and concurrency features of Rust for data engineering tasks? If so, this course is designed for you.
While a fundamental knowledge of Rust is expected, you should ideally be comfortable with the basics of data structures and algorithms, and have a working understanding of databases and data processing. Familiarity with SQL, the command line, and version control with git is advantageous. This four-week course focuses on leveraging Rust to create efficient, safe, and concurrent data processing systems. The journey begins with a deep dive into Rust's data structures and collections, followed by exploring Rust's safety and security features in the context of data engineering. In the subsequent week, you'll explore libraries and tools specific to data engineering like Diesel, async, Polars, and Apache Arrow, and learn to interface with data processing systems, REST, gRPC protocols, and AWS SDK for cloud-based data operations. The final week focuses on designing and implementing full-fledged data processing systems using Rust. By the end of this course, you will be well-equipped to use Rust for handling large-scale data engineering tasks, solving real-world problems with efficiency and speed. The hands-on labs and projects throughout this course will ensure you gain practical experience, putting your knowledge into action. This course is your gateway to mastering data engineering with Rust, preparing you for the next level in your data engineering journey.
This week, you will get started with the Rust development ecosystem, including AI coding tools, continuous integration, and cloud-based environments.
What's included
24 videos30 readings3 assignments5 ungraded labs
24 videosβ’Total 112 minutes
- Meet Instructor and Course Overviewβ’8 minutes
- Introduction to the AI Coding Paradigm Shiftβ’3 minutes
- Introduction to cloud-based development environmentsβ’11 minutes
- Introduction to GitHub Copilot Ecosystem for Rustβ’9 minutes
- Prompt Engineering with GCP BigQuery SQLβ’9 minutes
- Introduction to AWS CodeWhisperer for Rustβ’8 minutes
- Using Google Bard to Enhance Productivityβ’6 minutes
- Continuous Integration with Rust and GitHub Actionsβ’8 minutes
- Introducing Rust Sequences and Mapsβ’2 minutes
- Print Rust data structures demoβ’2 minutes
- Vector Fruit Salad demoβ’3 minutes
- VecDeque Fruit Salad demoβ’3 minutes
- Linkedin List Fruit Salad demoβ’2 minutes
- Fruit Salad CLI demoβ’4 minutes
- HashMap frequency counter demoβ’3 minutes
- HashMap language comparisoβ’3 minutes
- Analyzing UFC Fighter Network Using Graph Centrality in Rustβ’4 minutes
- Storing Unique Fruits Using HashSet in Rustβ’3 minutes
- Maintaining Sorted and Unique Fruits Using BTreeSet in Rustβ’3 minutes
- Creating a Fig Priority Fruit Salad Using Binary Heap in Rustβ’3 minutes
- PageRank algorithm for sports dataβ’4 minutes
- Showing shortest path with dijkstraβ’3 minutes
- Detecting Strongly Connected Components: A Deep Dive into Kosaraju's Algorithmβ’4 minutes
- Simple Charting of Data Structures in Rustβ’2 minutes
30 readingsβ’Total 295 minutes
- Prerequisites and Getting Startedβ’10 minutes
- Key Termsβ’10 minutes
- Using VS Code, Copilot, and Codespaces to Level Up to Rust from Pythonβ’10 minutes
- Harness the power of generative AI for software developmentβ’10 minutes
- The case for using Rust in MLOpsβ’10 minutes
- AWS CodeWhisperer FAQβ’10 minutes
- Lesson Reflectionβ’10 minutes
- Report a problem with the courseβ’5 minutes
- Key Termsβ’10 minutes
- External Lab: Creating a Fruit Salad with Rust Vectors in GitHub Codespacesβ’10 minutes
- External Lab: Fruit Salad Creation with VecDeque in GitHub Codespacesβ’10 minutes
- External Lab: Fruit Salad Creation with LinkedList in GitHub Codespacesβ’10 minutes
- External Lab: Command Line Fruit Salad Creator in GitHub Codespacesβ’10 minutes
- Rust Collections Docsβ’10 minutes
- Lesson Reflectionβ’10 minutes
- Key Termsβ’10 minutes
- Russian Troll Tweet Datasetsβ’10 minutes
- When to use a Rust Setβ’10 minutes
- Rust iteratorsβ’10 minutes
- Neo4J Graph Data Science Library Manualβ’10 minutes
- Calculating Centrality in a UFC Fighter Graph with Rustβ’10 minutes
- External GitHub Lab: Generating Unique Fruits with Rust and HashSetβ’10 minutes
- External GitHub Lab: Generating Unique Fruits with Rust and BTreeSetβ’10 minutes
- External GitHub Lab: Generating Fruit Salad with Rust and BinaryHeapβ’10 minutes
- External GitHub Lab: PageRank Algorithm in Rustβ’10 minutes
- External GitHub Lab: Shortest Path Algorithm in Rustβ’10 minutes
- External GitHub Lab: Community Detection in Rustβ’10 minutes
- External GitHub Lab: Graph Visualization in Rustβ’10 minutes
- Lesson Reflectionβ’10 minutes
- Final Week-Reflectionβ’10 minutes
3 assignmentsβ’Total 390 minutes
- Rust Collectionsβ’30 minutes
- Quiz-Getting Started With The Modern Rust Development Ecosystemβ’180 minutes
- Quiz-Rust Sequences and Mapsβ’180 minutes
5 ungraded labsβ’Total 300 minutes
- Data Engineering with Rust Sandboxβ’60 minutes
- Exploring Rust Data Structuresβ’60 minutes
- Counting Frequency with Rust HashMapsβ’60 minutes
- Weighting Programming Languages with Rust HashMapsβ’60 minutes
- Collections Week Challengeβ’60 minutes
This week, you will explore Rust's safety, security, and concurrency capabilities including encryption, network segmentation, thread safety, and web crawling.
What's included
22 videos21 readings4 assignments3 ungraded labs
22 videosβ’Total 94 minutes
- Multi-Factor Authenticationβ’2 minutes
- Network Segmentationβ’3 minutes
- Least Privilege Accessβ’2 minutes
- Encryptionβ’2 minutes
- Mutable fruit saladβ’4 minutes
- Customize fruit salad with a CLIβ’7 minutes
- Data Race exampleβ’3 minutes
- High Availabilityβ’3 minutes
- Understanding the Homophonic Cipher: A Cryptographic Techniqueβ’4 minutes
- Decoding the Secrets of the Caesar Cipherβ’3 minutes
- Creating a Decoder Ring: A Practical Guideβ’6 minutes
- Detecting Duplicates with SHA-3: A Data Integrity Toolβ’5 minutes
- Incident Responseβ’2 minutes
- Complianceβ’2 minutes
- Core Concepts in Concurrencyβ’5 minutes
- Dining Philosophersβ’6 minutes
- Web Crawl Wikipedia with Rayonβ’4 minutes
- Intelligent Chatbot with Tokioβ’5 minutes
- Multi-threaded deduplication with Rustβ’9 minutes
- Energy Efficiency Python vs Rustβ’6 minutes
- Concurrency Stress test with a GPUβ’8 minutes
- Host Efficiency Serverless Optimization problemβ’4 minutes
21 readingsβ’Total 210 minutes
- Key Termsβ’10 minutes
- Customize fruit salad with a CLIβ’10 minutes
- Meet Safe and Unsafeβ’10 minutes
- Ownership and Lifetimesβ’10 minutes
- The Perils Of Ownership-Based Resource Management (OBRM)β’10 minutes
- Lesson Reflectionβ’10 minutes
- Key Termsβ’10 minutes
- RustCrypto: Hashesβ’10 minutes
- Rust Software Security: A Current State Assessmentβ’10 minutes
- External GitHub Lab: Creating a Decoder Ring: A Practical Guideβ’10 minutes
- Lesson Reflectionβ’10 minutes
- Key Termsβ’10 minutes
- Concurrency and Parallelismβ’10 minutes
- Data Races and Race Conditionsβ’10 minutes
- Send and Syncβ’10 minutes
- Atomicsβ’10 minutes
- Distributed Computing and Concurrencyβ’10 minutes
- Challenges and Opportunities in Distributedβ’10 minutes
- Final Week-Reflectionβ’10 minutes
- Lesson Reflectionβ’10 minutes
- Rust Playground Rayon Challengeβ’10 minutes
4 assignmentsβ’Total 570 minutes
- Safety, Security and Concurrency with Rustβ’30 minutes
- Quiz-Rust Safety and Security Featuresβ’180 minutes
- Quiz-Security Programming with Rustβ’180 minutes
- Quiz-Rust Concurrencyβ’180 minutes
3 ungraded labsβ’Total 180 minutes
- Mutable fruit saladβ’60 minutes
- Data Raceβ’60 minutes
- Lab: Dining Philosophersβ’60 minutes
This week, you explore data processing in Rust. Manage data files and cloud storage. Build APIs and web scrapers. Power data engineering with Rust efficiency.
What's included
21 videos14 readings3 assignments3 ungraded labs
21 videosβ’Total 127 minutes
- Process CSV files in Rustβ’4 minutes
- Using Cargo Lambda with Rustβ’4 minutes
- List files on AWS EFS with Rustβ’8 minutes
- Use AWS S3 Storageβ’5 minutes
- Use AWS S3 Storage from Rustβ’5 minutes
- Write encrypted data to tables or Parquet filesβ’2 minutes
- What is Colab?β’6 minutes
- Using Bard to enhance notebook developmentβ’6 minutes
- Exploring Life Expectency in a Notebookβ’7 minutes
- Load a DataFrame with sensitive dataβ’2 minutes
- Using MLFlow with Databricks Notebooksβ’5 minutes
- End to End ML with MLFlow and Databricksβ’4 minutes
- Exploring global life expectancy with Polarsβ’5 minutes
- Cloud Developer Workspace Advantageβ’4 minutes
- Onboarding to GCP with Python and Rustβ’8 minutes
- Using GCP Cloud Shell with Rustβ’4 minutes
- Learn AWS CloudShellβ’12 minutes
- Prototyping AI APIs with AWS CloudShellβ’13 minutes
- Cloud9 with CodeWhispererβ’9 minutes
- Demo GCP App Engine Rust Deployβ’5 minutes
- Containerized Rust Actix Microservice on AWSβ’9 minutes
14 readingsβ’Total 140 minutes
- Key Termsβ’10 minutes
- Rust CSV Cookbookβ’10 minutes
- Apache Parquet Official Native Rust Implementationβ’10 minutes
- Lesson Reflectionβ’10 minutes
- Key Termsβ’10 minutes
- Running Rust in Jupyter Notebookβ’10 minutes
- External GitHub Lab: Using Polars DataFrame CLIβ’10 minutes
- Polars is a highly performant DataFrame library for manipulating structured dataβ’10 minutes
- Lesson Reflectionβ’10 minutes
- Key Termsβ’10 minutes
- Chapter 2-Week 2 (Up and Running with Cloud Computing)β’10 minutes
- Chapter 3 - Week 3: Virtualization and Containersβ’10 minutes
- What is the AWS SDK for Rust?β’10 minutes
- Final Week-Reflectionβ’10 minutes
3 assignmentsβ’Total 390 minutes
- Quiz3: Rust Data Engineering Libraries and Toolsβ’30 minutes
- Quiz-Using Rust to Manage Data, Files and Network Storageβ’180 minutes
- Quiz-DataFrames with Rust, Python and Notebooksβ’180 minutes
3 ungraded labsβ’Total 180 minutes
- Write CSV Filesβ’60 minutes
- Exploring Notebooks with Python Pandas and Jupyterβ’60 minutes
- Axum Greedy Coin Microserviceβ’60 minutes
This week, build data pipelines. Ingest, process, store data. Create workflows, Lambdas, microservices. Deploy to cloud. Monitor and scale.
What's included
22 videos18 readings4 assignments2 ungraded labs
22 videosβ’Total 124 minutes
- Jack and the Beanstalk Data Pipelinesβ’3 minutes
- Open Source Data Engineering - Pros and Consβ’5 minutes
- Core Components of Data Engineering Pipelinesβ’3 minutes
- Rust AWS Step Functions Pipelineβ’7 minutes
- Rust AWS Lambda Async S3 Size Calculatorβ’5 minutes
- What is Distroless?β’3 minutes
- Demo Deploying Rust Microservices on GCPβ’7 minutes
- Introduction to Hugging Face Hubβ’5 minutes
- Rust GPU Hugging Face Translatorβ’8 minutes
- Rust PyTorch High-Performance Optionsβ’8 minutes
- EFS ONNX Rust Inference with AWS Lambdaβ’10 minutes
- Theory behind model fine-tuningβ’3 minutes
- Doing Fine Tuningβ’8 minutes
- Selecting the correct database on GCPβ’4 minutes
- Rust SQLite Hugging Face Zero Shot Classifierβ’10 minutes
- Prompt Engineering for BigQuery β’9 minutes
- Big Query to Colab Pipelineβ’6 minutes
- Exploring Data with Big Queryβ’13 minutes
- Using Public Datasets for Data Scienceβ’2 minutes
- Querying Log files with BigQueryβ’4 minutes
- There is no one size databaseβ’2 minutes
- Course Conclusionβ’1 minute
18 readingsβ’Total 180 minutes
- Key Termsβ’10 minutes
- Architectural Patterns to Build End-to-End Data Driven Applications on AWSβ’10 minutes
- Data Preparation and Feature Engineering in MLβ’10 minutes
- "Distroless" Container Imagesβ’10 minutes
- Lesson Reflectionβ’10 minutes
- Key Termsβ’10 minutes
- Introduction to ONNXβ’10 minutes
- Ready-to-use NLP pipelines and Transformer-based modelsβ’10 minutes
- Hugging Face NLP Course documentationβ’10 minutes
- Hugging Face Fine-Tuning a pretrained modelβ’10 minutes
- Lesson Reflectionβ’10 minutes
- Key Termsβ’10 minutes
- About SQLiteβ’10 minutes
- Appropriate Uses For SQLiteβ’10 minutes
- What is BigQuery?β’10 minutes
- Final Week- Reflectionβ’10 minutes
- Next Stepsβ’10 minutes
- Share your learning experienceβ’10 minutes
4 assignmentsβ’Total 420 minutes
- Designing Data Processing Systems in Rustβ’30 minutes
- Final Course Quizβ’30 minutes
- Quiz-Getting Started with Rust Data Pipelines (Including ETL)β’180 minutes
- Quiz-Using Rust and Python for LLMs, ONNX, Hugging Face, and PyTorch Pipelinesβ’180 minutes
2 ungraded labsβ’Total 120 minutes
- ETLβ’60 minutes
- Rust Sandboxβ’60 minutes
Earn a career certificate
Add this credential to your LinkedIn profile, resume, or CV. Share it on social media and in your performance review.
Instructors
Offered by
Explore more from Machine Learning
- Status: Free TrialP
Pragmatic AI Labs
Specialization
- Status: Free TrialP
Pragmatic AI Labs
Course
- Status: Free TrialP
Pragmatic AI Labs
Course
- Status: Free TrialP
Pragmatic AI Labs
Course
Why people choose Coursera for their career
Learner reviews
- 5 stars
41.42%
- 4 stars
15.71%
- 3 stars
5.71%
- 2 stars
20%
- 1 star
17.14%
Showing 3 of 70
Reviewed on Aug 30, 2024
Love how the content is not watered down. Videos are very efficient at conveying information.
Reviewed on Feb 3, 2024
This course helps me to up skill during my career break
Frequently asked questions
To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.
Yes. In select learning programs, you can apply for financial aid or a scholarship if you canβt afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, youβll find a link to apply on the description page.
More questions
Financial aid available,
