Sequence Modeling, Transformers, and Transfer Learning
Keep adding new skills with 10,000+ programs for $239 (usually $399). Save now.
Sequence Modeling, Transformers, and Transfer Learning
This course is part of AI Engineer Professional Specialization
Included with
Learn more
Ask Coursera
Recommended experience
Recommended experience
What you'll learn
Understand the fundamentals of sequence modeling using RNNs, LSTMs, and GRUs.
Master the transformer architecture and attention mechanisms for NLP tasks.
Apply transfer learning to fine-tune pre-trained models for custom tasks.
Work on hands-on projects using RNNs, transformers, and transfer learning for text generation, translation, and summarization.
Skills you'll gain
Tools you'll learn
Details to know
February 2026
5 assignments
See how employees at top companies are mastering in-demand skills
Build your subject-matter expertise
- Learn new concepts from industry experts
- Gain a foundational understanding of a subject or tool
- Develop job-relevant skills with hands-on projects
- Earn a shareable career certificate
There are 3 modules in this course
This course features Coursera Coach!
A smarter way to learn with interactive, real-time conversations that help you test your knowledge, challenge assumptions, and deepen your understanding as you progress through the course. This course provides a comprehensive journey into sequence modeling, transformers, and transfer learning, equipping you with the skills to build powerful models for natural language processing (NLP) and other sequential data tasks. You'll begin by mastering Recurrent Neural Networks (RNNs), including their architecture, training techniques like backpropagation through time (BPTT), and specialized models such as Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRUs). The course then moves into sequence-to-sequence models, which are critical for tasks like translation, summarization, and text generation. The next phase of the course explores the groundbreaking transformer architecture, the backbone of modern NLP models like BERT and GPT. You will dive into attention mechanisms, self-attention, and multi-head attention, understanding how these components capture contextual relationships in text. You'll also gain hands-on experience with pre-trained transformer models and learn how to apply them to real-world NLP tasks such as text summarization and translation. In the final section, you'll focus on transfer learning, a technique that enables the reuse of pre-trained models to solve new tasks with fewer resources. This course teaches you how to fine-tune models for both computer vision and NLP applications, including domain adaptation strategies and challenges. With a hands-on project at the end of the course, youβll apply transfer learning to fine-tune a model for a custom task, demonstrating your ability to adapt state-of-the-art models to real-world problems. This course is ideal for learners with a foundational understanding of machine learning who want to advance their knowledge in deep learning, sequence modeling, and transfer learning. Prior knowledge of Python and basic machine learning concepts is recommended. The course is suitable for intermediate learners looking to deepen their understanding and practical skills in AI and deep learning. By the end of the course, you will be able to implement sequence models like RNNs, build transformers using attention mechanisms, apply transfer learning to fine-tune pre-trained models, and solve complex NLP tasks such as translation, summarization, and text generation.
In this module, we will explore the world of sequence modeling with Recurrent Neural Networks (RNNs). You'll learn about the architecture of RNNs, including how backpropagation through time works. We also cover advanced models like LSTMs and GRUs, and teach you how to preprocess text data and apply RNNs to sequence-to-sequence tasks. The module concludes with a hands-on project to implement RNNs for text generation or sentiment analysis.
What's included
7 videos2 readings1 assignment
7 videosβ’Total 165 minutes
- Day 1: Introduction to Sequence Modeling and RNNsβ’34 minutes
- Day 2: Understanding RNN Architecture and Backpropagation Through Time (BPTT)β’25 minutes
- Day 3: Long Short-Term Memory (LSTM) Networksβ’15 minutes
- Day 4: Gated Recurrent Units (GRUs)β’7 minutes
- Day 5: Text Preprocessing and Word Embeddings for RNNsβ’24 minutes
- Day 6: Sequence-to-Sequence Models and Applicationsβ’43 minutes
- Day 7: RNN Project β Text Generation or Sentiment Analysisβ’18 minutes
2 readingsβ’Total 20 minutes
- Introduction to the Course 'Sequence Modeling, Transformers, and Transfer Learning'β’10 minutes
- Full Specialization Resourcesβ’10 minutes
1 assignmentβ’Total 15 minutes
- Recurrent Neural Networks (RNNs) and Sequence Modeling - Assessmentβ’15 minutes
In this module, we introduce you to the transformative power of attention mechanisms in deep learning models. Youβll explore the architecture of transformers, learning about self-attention, multi-head attention, and positional encoding. With hands-on demonstrations of pre-trained transformer models like BERT and GPT, this section equips you to apply advanced NLP techniques to real-world projects like text summarization and translation.
What's included
7 videos1 assignment
7 videosβ’Total 134 minutes
- Day 1: Introduction to Attention Mechanismsβ’15 minutes
- Day 2: Introduction to Transformers Architectureβ’18 minutes
- Day 3: Self-Attention and Multi-Head Attention in Transformersβ’21 minutes
- Day 4: Positional Encoding and Feed-Forward Networksβ’20 minutes
- Day 5: Hands-On with Pre-Trained Transformers β BERT and GPTβ’20 minutes
- Day 6: Advanced Transformers β BERT Variants and GPT-3β’21 minutes
- Day 7: Transformer Project β Text Summarization or Translationβ’19 minutes
1 assignmentβ’Total 15 minutes
- Transformers and Attention Mechanisms - Assessmentβ’15 minutes
In this module, we dive into the concept of transfer learning, a powerful technique that leverages pre-trained models for a wide range of applications. You will learn how to use transfer learning for both computer vision and natural language processing (NLP), including fine-tuning strategies and domain adaptation. The section concludes with a project where you will fine-tune a model for a custom task, helping you apply these techniques to solve real-world problems.
What's included
7 videos1 reading3 assignments
7 videosβ’Total 139 minutes
- Day 1: Introduction to Transfer Learningβ’15 minutes
- Day 2: Transfer Learning in Computer Visionβ’26 minutes
- Day 3: Fine-Tuning Techniques in Computer Visionβ’22 minutes
- Day 4: Transfer Learning in NLPβ’17 minutes
- Day 5: Fine-Tuning Techniques in NLPβ’26 minutes
- Day 6: Domain Adaptation and Transfer Learning Challengesβ’15 minutes
- Day 7: Transfer Learning Project β Fine-Tuning for a Custom Taskβ’18 minutes
1 readingβ’Total 10 minutes
- Conclusion to the Course 'Sequence Modeling, Transformers, and Transfer Learning'β’10 minutes
3 assignmentsβ’Total 90 minutes
- Full Course Practice Assessmentβ’15 minutes
- Transfer Learning and Fine-Tuning - Assessmentβ’15 minutes
- Full Course Assessmentβ’60 minutes
Earn a career certificate
Add this credential to your LinkedIn profile, resume, or CV. Share it on social media and in your performance review.
Instructor
Offered by
Explore more from Machine Learning
Why people choose Coursera for their career
Frequently asked questions
Sequence modeling involves training machine learning models to work with sequential data, like time series or text, where the order of data matters. Transformers, a modern deep learning architecture, have revolutionized NLP tasks by using attention mechanisms to better capture long-range dependencies in sequences. Transfer learning allows models to leverage pre-trained knowledge from one task and apply it to another, significantly improving performance, especially when data is limited. These techniques are highly relevant as they are foundational to state-of-the-art AI models, particularly in natural language processing and computer vision.
The "Sequence Modeling, Transformers, and Transfer Learning" course explores advanced machine learning techniques for working with sequential data. It covers Recurrent Neural Networks (RNNs), including Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), as well as transformer models and attention mechanisms. The course also delves into transfer learning, including its applications in NLP and computer vision, with hands-on projects to reinforce learning. You will work with popular pre-trained models like BERT and GPT, and apply transfer learning to custom tasks.
After completing the course, you will have a deep understanding of sequence modeling, transformers, and transfer learning techniques. You will be capable of building and training RNNs and transformer models for tasks like text generation, sentiment analysis, text summarization, and translation. You will also be able to use transfer learning to fine-tune pre-trained models for specific applications in both NLP and computer vision, greatly enhancing your ability to solve real-world problems using AI.
More questions
Financial aid available,
