Natural Language Processing with Attention Models
Keep adding new skills with 10,000+ programs for $239 (usually $399). Save now.
Natural Language Processing with Attention Models
This course is part of Natural Language Processing Specialization
88,599 already enrolled
1,096 reviews
Recommended experience
1,096 reviews
Recommended experience
What you'll learn
Use encoder-decoder, causal, & self-attention to machine translate complete sentences, summarize text, and answer questions.
Skills you'll gain
Tools you'll learn
Details to know
3 assignments
See how employees at top companies are mastering in-demand skills
Build your subject-matter expertise
- Learn new concepts from industry experts
- Gain a foundational understanding of a subject or tool
- Develop job-relevant skills with hands-on projects
- Earn a shareable career certificate
There are 3 modules in this course
In Course 4 of the Natural Language Processing Specialization, you will:
a) Translate complete English sentences into Portuguese using an encoder-decoder attention model, b) Build a Transformer model to summarize text, c) Use T5 and BERT models to perform question-answering. By the end of this Specialization, you will have designed NLP applications that perform question-answering and sentiment analysis, and created tools to translate languages and summarize text! Learners should have a working knowledge of machine learning, intermediate Python including experience with a deep learning framework (e.g., TensorFlow, Keras), as well as proficiency in calculus, linear algebra, and statistics. Please make sure that youβve completed course 3 - Natural Language Processing with Sequence Models - before starting this course. This Specialization is designed and taught by two experts in NLP, machine learning, and deep learning. Younes Bensouda Mourri is an Instructor of AI at Stanford University who also helped build the Deep Learning Specialization. Εukasz Kaiser is a Staff Research Scientist at Google Brain and the co-author of Tensorflow, the Tensor2Tensor and Trax libraries, and the Transformer paper.
Discover some of the shortcomings of a traditional seq2seq model and how to solve for them by adding an attention mechanism, then build a Neural Machine Translation model with Attention that translates English sentences into German.
What's included
15 videos5 readings1 assignment1 programming assignment3 ungraded labs
15 videosβ’Total 88 minutes
- Course 4 Introductionβ’3 minutes
- Week Introductionβ’1 minute
- Seq2seqβ’6 minutes
- Seq2seq Model with Attentionβ’6 minutes
- Queries, Keys, Values, and Attentionβ’6 minutes
- Setup for Machine Translationβ’2 minutes
- Teacher Forcingβ’2 minutes
- NMT Model with Attentionβ’4 minutes
- BLEU Scoreβ’5 minutes
- ROUGE-N Scoreβ’5 minutes
- Sampling and Decodingβ’4 minutes
- Beam Searchβ’7 minutes
- Minimum Bayes Riskβ’3 minutes
- Week Conclusionβ’1 minute
- Andrew Ng with Oren Etzioniβ’35 minutes
5 readingsβ’Total 28 minutes
- Background on seq2seqβ’10 minutes
- Content Resourceβ’10 minutes
- Join the DeepLearning.AI Forum to ask questions, get support, or share amazing ideas!β’2 minutes
- Lecture Notes W1β’1 minute
- (Optional) Downloading your Notebook, Downloading your Workspace and Refreshing your Workspaceβ’5 minutes
1 assignmentβ’Total 30 minutes
- Neural Machine Translationβ’30 minutes
1 programming assignmentβ’Total 180 minutes
- NMT with Attention (Tensorflow)β’180 minutes
3 ungraded labsβ’Total 90 minutes
- Ungraded Lab: Basic Attentionβ’30 minutes
- Ungraded Lab: Scaled Dot-Product Attentionβ’30 minutes
- Ungraded Lab: BLEU Scoreβ’30 minutes
Compare RNNs and other sequential models to the more modern Transformer architecture, then create a tool that generates text summaries.
What's included
10 videos6 readings1 assignment1 programming assignment3 ungraded labs
10 videosβ’Total 39 minutes
- Week Introductionβ’1 minute
- Transformers vs RNNsβ’3 minutes
- Transformers overviewβ’5 minutes
- Transformer Applicationsβ’7 minutes
- Scaled and Dot-Product Attentionβ’4 minutes
- Masked Self Attentionβ’3 minutes
- Multi-head Attentionβ’6 minutes
- Transformer Decoderβ’5 minutes
- Transformer Summarizerβ’4 minutes
- Week Conclusionβ’1 minute
6 readingsβ’Total 51 minutes
- Transformers vs RNNsβ’10 minutes
- Transformer Applicationsβ’10 minutes
- Multi-head Attentionβ’10 minutes
- Transformer Decoderβ’10 minutes
- Content Resourceβ’10 minutes
- Lecture Notes W2β’1 minute
1 assignmentβ’Total 30 minutes
- Text Summarizationβ’30 minutes
1 programming assignmentβ’Total 180 minutes
- Transformer Summarizerβ’180 minutes
3 ungraded labsβ’Total 180 minutes
- Attentionβ’60 minutes
- Maskingβ’60 minutes
- Positional encodingβ’60 minutes
Explore transfer learning with state-of-the-art models like T5 and BERT, then build a model that can answer questions.
What's included
16 videos15 readings1 assignment1 programming assignment3 ungraded labs
16 videosβ’Total 98 minutes
- Week Introductionβ’1 minute
- Week 3 Overviewβ’7 minutes
- Transfer Learning in NLPβ’6 minutes
- ELMo, GPT, BERT, T5β’8 minutes
- Bidirectional Encoder Representations from Transformers (BERT)β’5 minutes
- BERT Objectiveβ’3 minutes
- Fine tuning BERTβ’2 minutes
- Transformer: T5β’4 minutes
- Multi-Task Training Strategyβ’6 minutes
- GLUE Benchmarkβ’2 minutes
- Hugging Face Introductionβ’3 minutes
- Hugging Face Iβ’4 minutes
- Hugging Face IIβ’3 minutes
- Hugging Face IIIβ’5 minutes
- Week Conclusionβ’0 minutes
- Andrew Ng with Quoc Leβ’41 minutes
15 readingsβ’Total 136 minutes
- Week 3 Overviewβ’10 minutes
- Transfer Learning in NLPβ’10 minutes
- ELMo, GPT, BERT, T5β’10 minutes
- Bidirectional Encoder Representations from Transformers (BERT)β’10 minutes
- BERT Objectiveβ’10 minutes
- Fine tuning BERTβ’10 minutes
- Transformer T5β’10 minutes
- Multi-Task Training Strategyβ’10 minutes
- GLUE Benchmarkβ’10 minutes
- Welcome to Hugging Face π€β’10 minutes
- Content Resourceβ’10 minutes
- Lecture Notes W3β’1 minute
- Acknowledgmentsβ’10 minutes
- Referencesβ’10 minutes
- (Optional) Opportunity to Mentor Other Learnersβ’5 minutes
1 assignmentβ’Total 30 minutes
- Question Answeringβ’30 minutes
1 programming assignmentβ’Total 180 minutes
- Question Answeringβ’180 minutes
3 ungraded labsβ’Total 240 minutes
- SentencePiece and BPEβ’120 minutes
- Question Answering with HuggingFace - Using a base modelβ’60 minutes
- Question Answering with HuggingFace 2 - Fine-tuning a modelβ’60 minutes
Earn a career certificate
Add this credential to your LinkedIn profile, resume, or CV. Share it on social media and in your performance review.
Instructors
Offered by
Explore more from Machine Learning
- Status: Free TrialD
DeepLearning.AI
Course
- Status: Free TrialD
DeepLearning.AI
Course
- Status: Free Trial
Course
- Status: Free Trial
Course
Why people choose Coursera for their career
Learner reviews
- 5 stars
68.60%
- 4 stars
14.37%
- 3 stars
8.28%
- 2 stars
4.64%
- 1 star
4.09%
Showing 3 of 1096
Reviewed on Oct 12, 2020
Great course! I really enjoyed extensive non-graded notebooks on LSH attention. Some content was pretty challenging, but always very rewarding!
Thank you!
Reviewed on Sep 23, 2021
It's a great way to get started with state-of-the-art NLP techniques, following the recommended papers is extremely useful.
Reviewed on Nov 3, 2022
The content is great, but it will be even better if we have a more in-depth understanding of the knowledge rather than a very quick crash course.
Frequently asked questions
To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.
Yes. In select learning programs, you can apply for financial aid or a scholarship if you canβt afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, youβll find a link to apply on the description page.
More questions
Financial aid available,