Custom and Distributed Training with TensorFlow
Keep adding new skills with 10,000+ programs for $239 (usually $399). Save now.
Custom and Distributed Training with TensorFlow
This course is part of TensorFlow: Advanced Techniques Specialization
Instructors: Laurence Moroney
25,319 already enrolled
Ask Coursera
438 reviews
Recommended experience
438 reviews
Recommended experience
Tools you'll learn
Details to know
See how employees at top companies are mastering in-demand skills
Build your subject-matter expertise
- Learn new concepts from industry experts
- Gain a foundational understanding of a subject or tool
- Develop job-relevant skills with hands-on projects
- Earn a shareable career certificate
There are 4 modules in this course
In this course, you will:
• Learn about Tensor objects, the fundamental building blocks of TensorFlow, understand the difference between the eager and graph modes in TensorFlow, and learn how to use a TensorFlow tool to calculate gradients. • Build your own custom training loops using GradientTape and TensorFlow Datasets to gain more flexibility and visibility with your model training. • Learn about the benefits of generating code that runs in graph mode, take a peek at what graph code looks like, and practice generating this more efficient code automatically with TensorFlow’s tools. • Harness the power of distributed training to process more data and train larger models, faster, get an overview of various distributed training strategies, and practice working with a strategy that trains on multiple GPU cores, and another that trains on multiple TPU cores. The DeepLearning.AI TensorFlow: Advanced Techniques Specialization introduces the features of TensorFlow that provide learners with more control over their model architecture and tools that help them create and train advanced ML models. This Specialization is for early and mid-career software and machine learning engineers with a foundational understanding of TensorFlow who are looking to expand their knowledge and skill set by learning advanced TensorFlow features to build powerful models.
This week, you will get a detailed look at the fundamental building blocks of TensorFlow - tensor objects. For example, you will be able to describe the difference between eager mode and graph mode in TensorFlow, and explain why eager mode is very user friendly for you as a developer. You will also use TensorFlow tools to calculate gradients so that you don’t have to look for your old calculus textbooks next time you need to get a gradient!
What's included
12 videos3 readings1 assignment1 programming assignment2 ungraded labs
12 videos•Total 51 minutes
- A conversation with Andrew Ng: Overview of course 2•5 minutes
- What is a tensor?•4 minutes
- Creating tensors in code•6 minutes
- Math operations with tensors•2 minutes
- Basic Tensors code walkthrough•4 minutes
- Broadcasting, operator overloading and Numpy compatibility•6 minutes
- Evaluating variables and changing data types•4 minutes
- Gradient Tape•4 minutes
- Gradient Descent using Gradient Tape•4 minutes
- Calculate gradients on higher order functions•5 minutes
- Persistent=true and higher order gradients•3 minutes
- Gradient Tape basics code walkthrough•3 minutes
3 readings•Total 13 minutes
- Join the DeepLearning.AI Forum to ask questions, get support, or share amazing ideas!•2 minutes
- Reference: CNN for visual recognition•10 minutes
- Lecture Notes Week 1•1 minute
1 assignment•Total 30 minutes
- Tensors and Gradient Tape•30 minutes
1 programming assignment•Total 60 minutes
- Basic Tensor Operations•60 minutes
2 ungraded labs•Total 120 minutes
- Basic Tensors•60 minutes
- Gradient Tape Basics•60 minutes
This week, you will build custom training loops using GradientTape and TensorFlow Datasets. Being able to write your own training loops will give you more flexibility and visibility with your model training. You will also use a function to calculate the derivatives of functions so that you don’t have to look to your old calculus textbooks to calculate gradients.
What's included
8 videos2 readings1 assignment1 programming assignment2 ungraded labs
8 videos•Total 46 minutes
- Custom Training Loop steps•4 minutes
- Loss and gradient descent•4 minutes
- Define Training Loop and Validate Model•2 minutes
- Training Basics code walkthrough•6 minutes
- Training steps and data pipeline•5 minutes
- Define the training loop•5 minutes
- Gradients, metrics, and validation•5 minutes
- Fashion MNIST Custom Training Loop code walkthrough•16 minutes
2 readings•Total 11 minutes
- Reference: tf.keras.metrics•10 minutes
- Lecture Notes Week 2•1 minute
1 assignment•Total 30 minutes
- Custom Training•30 minutes
1 programming assignment•Total 60 minutes
- Breast Cancer Prediction•60 minutes
2 ungraded labs•Total 120 minutes
- Training Basics•60 minutes
- Fashion MNIST using Custom Training Loop•60 minutes
This week, you’ll learn about the benefits of generating code that runs in “graph mode”. You’ll take a peek at what graph code looks like, and you’ll practice generating this more efficient code automatically with TensorFlow’s tools, so that you don’t have to write the graph code yourself!
What's included
6 videos2 readings1 assignment1 programming assignment2 ungraded labs
6 videos•Total 35 minutes
- Benefits of graph mode•4 minutes
- Generating graph code•4 minutes
- AutoGraph Basics code walkthrough•6 minutes
- Control dependencies and flows•4 minutes
- Loops and tracing variables•5 minutes
- AutoGraph code walkthrough•12 minutes
2 readings•Total 11 minutes
- Reference: Fizz Buzz•10 minutes
- Lecture Notes Week 3•1 minute
1 assignment•Total 30 minutes
- AutoGraph•30 minutes
1 programming assignment•Total 120 minutes
- Horse or Human?•120 minutes
2 ungraded labs•Total 120 minutes
- AutoGraph Basics•60 minutes
- AutoGraph•60 minutes
This week, you will harness the power of distributed training to process more data and train larger models, faster. You’ll get an overview of various distributed training strategies and then practice working with two strategies, one that trains on multiple GPU cores, and the other that trains on multiple TPU cores. Get your cape ready, because you’re going to get some superpowers this week!
What's included
9 videos5 readings1 assignment2 programming assignments4 ungraded labs
9 videos•Total 56 minutes
- Intro to distribution strategies•4 minutes
- Types of distribution strategies•4 minutes
- Converting code to the Mirrored Strategy•4 minutes
- Mirrored Strategy code walkthrough•5 minutes
- Custom Training for Multiple GPU Mirrored Strategy•5 minutes
- Multi GPU Mirrored Strategy code walkthrough•13 minutes
- TPU Strategy•6 minutes
- TPU Strategy code walkthrough•10 minutes
- Other Distributed Strategies•4 minutes
5 readings•Total 33 minutes
- References used in Other Distributed Strategies•10 minutes
- Lecture Notes Week 4•1 minute
- [IMPORTANT] Reminder about end of access to Lab Notebooks•2 minutes
- References •10 minutes
- Acknowledgments•10 minutes
1 assignment•Total 30 minutes
- Distributed Strategy•30 minutes
2 programming assignments•Total 240 minutes
- Distributed Strategy•180 minutes
- Upload your model (optional)•60 minutes
4 ungraded labs•Total 240 minutes
- Mirrored Strategy•60 minutes
- Multi GPU Mirrored Strategy•60 minutes
- TPU Strategy•60 minutes
- One Device Strategy•60 minutes
Earn a career certificate
Add this credential to your LinkedIn profile, resume, or CV. Share it on social media and in your performance review.
Instructors
Offered by
Explore more from Machine Learning
- Status: Free TrialD
DeepLearning.AI
Course
- Status: Free TrialD
DeepLearning.AI
Specialization
- Status: Free Trial
- Status: Free TrialD
DeepLearning.AI
Course
Why people choose Coursera for their career
Learner reviews
- 5 stars
83.33%
- 4 stars
12.55%
- 3 stars
2.51%
- 2 stars
1.14%
- 1 star
0.45%
Showing 3 of 438
Reviewed on Jan 20, 2021
He is a very good instructor and the content is well prepared, also the course covers rare topics.
Reviewed on Jan 7, 2021
Difficult concepts are explained with simple words and simple examples. Great course
Reviewed on Jan 8, 2022
Another great course by Moroney sir. Loved how TF can be used to train models using different strategies. A great intro to the deep applications of TensorFlow
Frequently asked questions
To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.
Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.
More questions
Financial aid available,