Generative AI: Fine-Tuning LLMs and Diffusion Models
Keep adding new skills with 10,000+ programs for $239 (usually $399). Save now.
Generative AI: Fine-Tuning LLMs and Diffusion Models
This course is part of Advanced Deep Learning Architectures Specialization
Instructor: Board Infinity
Included with
Learn more
Ask Coursera
Recommended experience
Recommended experience
What you'll learn
Build decoder-only transformer pipelines with KV caching optimizations
Fine-tune 7B+ LLMs using LoRA and QLoRA on consumer GPUs
Configure diffusers pipelines with ControlNet for controllable images
Train, export, and evaluate a domain-specialized LLM adapter
Skills you'll gain
Tools you'll learn
Details to know
May 2026
16 assignments
See how employees at top companies are mastering in-demand skills
Build your subject-matter expertise
- Learn new concepts from industry experts
- Gain a foundational understanding of a subject or tool
- Develop job-relevant skills with hands-on projects
- Earn a shareable career certificate
There are 4 modules in this course
"Master Generative AI with hands-on training in Large Language Models (LLMs), PEFT techniques (LoRA, QLoRA), and Diffusion Models using Hugging Face, diffusers, peft, trl, and bitsandbytes. This course takes you from the internals of decoder-only transformers to building a specialist fine-tuned LLM and generating high-quality, controllable images with ControlNet.
In Module 1, explore decoder-only transformer architectures, self-attention, causal masking, KV caching, and token flow mechanics. Module 2 focuses on Parameter-Efficient Fine-Tuning (PEFT), where you'll implement LoRA, QLoRA, and 4-bit quantization to fine-tune large models on consumer GPUs using SFT pipelines. Module 3 dives into diffusion models, covering forward/reverse processes, UNet, schedulers (DDIM, Euler, DPM++), and ControlNet conditioning. Module 4 is a capstone where you'll build a Specialist LLM β from dataset creation to adapter export and evaluation. By the end of this course, you will: - Build and optimize decoder-only transformer pipelines with KV caching - Fine-tune 7B+ LLMs using LoRA, QLoRA, and SFT pipelines on limited hardware - Configure diffusers pipelines with ControlNet for controllable image generation - Train, export, and evaluate a domain-specialized LLM adapter end-to-end" Disclaimer: This is an independent educational resource created by Board Infinity for informational and educational purposes only. This course is not affiliated with, endorsed by, sponsored by, or officially associated with any company, organization, or certification body unless explicitly stated. The content provided is based on industry knowledge and best practices but does not constitute official training material for any specific employer or certification program. All company names, trademarks, service marks, and logos referenced are the property of their respective owners and are used solely for educational identification and comparison purposes.
Explore the inner workings of decoder-only transformer architectures, including token flow, self-attention, causal masking, and KV cache optimization.
What's included
11 videos3 readings4 assignments
11 videosβ’Total 107 minutes
- Where LLM Engineers Work Todayβ’10 minutes
- Why Decoder-Only Models Dominate Industry Part 1β’11 minutes
- Why Decoder-Only Models Dominate Industry part -2β’9 minutes
- Skills Employers Wantβ’11 minutes
- Skills Employers Want Part 2β’9 minutes
- What Happens When You Type a Promptβ’9 minutes
- Decoder Block Components (Mental Model)β’9 minutes
- How Tokens Flow Through Layersβ’9 minutes
- Causal Masking & "No Peeking"β’10 minutes
- Why Attention Is Expensiveβ’10 minutes
- KV Cache: Why Inference Gets Fasterβ’10 minutes
3 readingsβ’Total 90 minutes
- The LLM Engineering Landscape (2026)β’30 minutes
- Inside the Decoder Block: Architecture & Data Flowβ’30 minutes
- KV Cache Optimization & Attention Behavior Analysisβ’30 minutes
4 assignmentsβ’Total 105 minutes
- Transformer Internals & Decoder-Only Architecturesβ’60 minutes
- Career Scope in LLM Engineeringβ’15 minutes
- Anatomy of Decoder-Only Transformersβ’15 minutes
- Self-Attention, Causal Masking & KV-Cache Behaviorβ’15 minutes
Master parameter-efficient fine-tuning techniques including LoRA, QLoRA with 4-bit quantization, and building supervised fine-tuning pipelines using peft and trl.
What's included
12 videos3 readings4 assignments
12 videosβ’Total 106 minutes
- The Core Idea Behind PEFTβ’9 minutes
- The Core Idea Behind PEFT part 2β’11 minutes
- The Core Idea Behind PEFT Part3β’6 minutes
- Implementing LoRA Layers in Attention Blocksβ’10 minutes
- Where PEFT Works (and Where It Doesn't)β’11 minutes
- LoRA Explainedβ’9 minutes
- QLoRA: Training Big Models on Small GPUsβ’9 minutes
- Setting QLoRA Hyperparameters for Stabilityβ’9 minutes
- What Good SFT Data Looks Like Part 1β’8 minutes
- What Good SFT Data Looks Like Part2β’6 minutes
- Building Full SFT Pipelines Using TRLβ’8 minutes
- Early Evaluation & Failure Signalsβ’10 minutes
3 readingsβ’Total 90 minutes
- LoRA Fundamentals & Design Decisionsβ’30 minutes
- QLoRA Implementation Guide for Large Modelsβ’30 minutes
- Constructing Reliable SFT Datasets for Behavior Modelingβ’30 minutes
4 assignmentsβ’Total 105 minutes
- PEFT - LoRA, QLoRA, & SFT Pipelinesβ’60 minutes
- Introduction to PEFT & Low-Rank Adaptationβ’15 minutes
- QLoRA & 4-Bit Quantization Pathwayβ’15 minutes
- Building SFT Pipelines with peft + trlβ’15 minutes
Understand the forward and reverse diffusion processes, configure diffusers pipelines with various schedulers, and apply ControlNets for conditioned image generation.
What's included
10 videos3 readings4 assignments
10 videosβ’Total 77 minutes
- Adding Noise: The Forward Processβ’7 minutes
- Removing Noise: The Reverse Processβ’8 minutes
- Why Timesteps Matterβ’8 minutes
- Diffusers Pipeline Componentsβ’7 minutes
- Diffusers Pipeline Components Part2β’10 minutes
- Choosing the Right Schedulerβ’5 minutes
- Style, Guidance & Sampling Tricksβ’7 minutes
- Why Prompting Alone Is Not Enoughβ’8 minutes
- ControlNet Concepts (Visual)β’8 minutes
- Practical: Conditioning an Image with ControlNetβ’8 minutes
3 readingsβ’Total 90 minutes
- Mathematical Intuition Behind Diffusion Timestepsβ’30 minutes
- Scheduler Comparison & Practical Recommendationsβ’30 minutes
- ControlNet for Structured Image Generationβ’30 minutes
4 assignmentsβ’Total 105 minutes
- Diffusion Models & Image Generationβ’60 minutes
- The Forward & Reverse Diffusion Processβ’15 minutes
- Configuring diffusers Pipelinesβ’15 minutes
- ControlNets & Conditioning Techniquesβ’15 minutes
Apply all course concepts in a capstone project building a specialist LLM through dataset creation, QLoRA training, and adapter exporting with rigorous evaluation.
What's included
12 videos3 readings4 assignments
12 videosβ’Total 81 minutes
- Converting Logs Into SFT-Ready Training Dataβ’9 minutes
- Designing Prompt Templates & Chat Formatsβ’6 minutes
- Cleaning, Normalizing & Validating Training Dataβ’8 minutes
- Cleaning, Normalizing & Validating Training Data Part 2β’5 minutes
- Cleaning, Normalizing & Validating Training Data Part 3β’7 minutes
- Training Workflow Using LoRA/QLoRAβ’5 minutes
- Monitoring Training Loss & Detecting Overfittingβ’6 minutes
- Memory-Saving Techniquesβ’7 minutes
- Exporting LoRA/QLoRA Adapters for Deploymentβ’8 minutes
- Exporting LoRAQLoRA Adapters for Deployment part 2β’5 minutes
- Loading the Adapter Into Base Models for Inferenceβ’6 minutes
- Evaluating Alignment: Perplexity, Behavior, Scenariosβ’7 minutes
3 readingsβ’Total 90 minutes
- Dataset Quality Standards for Persona-Level LLMsβ’30 minutes
- Troubleshooting PEFT Training Failuresβ’30 minutes
- Evaluation Framework for Domain-Specialized LLMsβ’30 minutes
4 assignmentsβ’Total 105 minutes
- The Hands-On Project - The Specialist LLMβ’60 minutes
- Dataset Creation & Prompt-Style Engineeringβ’15 minutes
- Training the Specialist LoRA/QLoRA Adapterβ’15 minutes
- Exporting, Loading & Evaluating the Adapterβ’15 minutes
Earn a career certificate
Add this credential to your LinkedIn profile, resume, or CV. Share it on social media and in your performance review.
Instructor
Offered by
Explore more from Machine Learning
- B
Board Infinity
Course
- B
Board Infinity
Course
Why people choose Coursera for their career
Frequently asked questions
A basic understanding of Python, PyTorch, and neural networks is helpful. Familiarity with transformers is a plus but not mandatory β we cover decoder internals from scratch.
You'll work with Hugging Face transformers, peft, trl, bitsandbytes, and diffusers for LLM fine-tuning and image generation.
To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
More questions
Financial aid available,
