VOOZH

URL: https://meta-pytorch.org/torchtune/main/

⇱ Welcome to the torchtune Documentation — torchtune main documentation

Get Started
Run PyTorch locally or get started quickly with one of the supported cloud platforms
Tutorials
Whats new in PyTorch tutorials
Learn the Basics
Familiarize yourself with PyTorch concepts and modules
PyTorch Recipes
Bite-size, ready-to-deploy PyTorch code examples
Intro to PyTorch - YouTube Series
Master PyTorch basics with our engaging YouTube tutorial series
Tools
Learn about the tools and frameworks in the PyTorch Ecosystem
Community
Join the PyTorch developer community to contribute, learn, and get your questions answered
Forums
A place to discuss PyTorch code, issues, install, research
Developer Resources
Find resources and get questions answered
Contributor Awards - 2024
Award winners announced at this year's PyTorch Conference
About PyTorch Edge
Build innovative and privacy-aware AI experiences for edge devices
ExecuTorch
End-to-end solution for enabling on-device inference capabilities across mobile and edge devices
ExecuTorch Docs
PyTorch
Explore the documentation for comprehensive guidance on how to use PyTorch
PyTorch Domains
Read the PyTorch Domains documentation to learn more about domain-specific libraries
PyTorch Blog
Catch up on the latest technical news and happenings
Community Blog
Stories from the PyTorch ecosystem
Videos
Learn about the latest PyTorch tutorials, new, and more
Community Stories
Learn how our community solves real, everyday machine learning problems with PyTorch
Events
Find events, webinars, and podcasts
Newsletter
Stay up-to-date with the latest updates
PyTorch Foundation
Learn more about the PyTorch Foundation
Governing Board Cloud Credit Program Technical Advisory Council Staff Contact Us
Become a Member

Table of Contents

Docs >
Welcome to the torchtune Documentation
👁 Image

Shortcuts

Welcome to the torchtune Documentation¶

torchtune is a Native-PyTorch library for LLM fine-tuning.

Getting Started¶

Topics in this section will help you get started with torchtune.

What is torchtune?

A gentle introduction to torchtune and how you can use the library in your projects.

Installation instructions

A step-by-step tutorial on how to install torchtune.

Finetune your first model

Follow a simple tutorial to finetune Llama2 with torchtune.

tutorials/first_finetune_tutorial.html

Tutorials¶

Ready to experiment? Check out some of the interactive torchtune tutorials.

Llama3 in torchtune

Topics: finetuning,llama3

Finetuning with LoRA in torchtune

Topics: finetuning,llama2,lora

Parameter-efficient finetuning of Llama2 using LoRA

Understanding QLoRA in torchtune

Topics: finetuning,llama2,qlora

Using QLoRA to quantize base model weights and maximize memory savings

Finetuning with QAT in torchtune

Topics: finetuning,llama3,qat,quantization,evals

Finetuning of Llama3 using QAT

End-to-End Workflow with torchtune

Topics: finetuning,quantization,inference,evals,llama2

Train, Evaluate, Quantize and then Generate with your LLM.

Distilling Llama3 8B into 1B

Topics: finetuning,llama3,kd

Distilling Llama3 8B into 1B using Knowledge Distillation

Multi-node training w/ Llama3.3 70B

Topics: multinode,llama3,slurm

Fine-tuning a large model on 2+ nodes

Docs

Access comprehensive developer documentation for PyTorch

Tutorials

Get in-depth tutorials for beginners and advanced developers

Resources

Find development resources and get your questions answered

To analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. As the current maintainers of this site, Facebook’s Cookies Policy applies. Learn more, including about available controls: Cookies Policy.