Recipes#

Recipes are bite-sized, actionable examples of how to use specific PyTorch features, different from our full-length tutorials.

Defining a Neural Network

Learn how to use PyTorch's torch.nn package to create and define a neural network for the MNIST dataset.

Basics

👁 Image

What is a state_dict in PyTorch

Learn how state_dict objects and Python dictionaries are used in saving or loading models from PyTorch.

Basics

👁 Image

Warmstarting model using parameters from a different model in PyTorch

Learn how warmstarting the training process by partially loading a model or loading a partial model can help your model converge much faster than training from scratch.

Basics

👁 Image

Zeroing out gradients in PyTorch

Learn when you should zero out gradients and how doing so can help increase the accuracy of your model.

Basics

👁 Image

PyTorch Benchmark

Learn how to use PyTorch's benchmark module to measure and compare the performance of your code

Basics

👁 Image

PyTorch Benchmark (quick start)

Learn how to measure snippet run times and collect instructions.

Basics

👁 Image

PyTorch Profiler

Learn how to use PyTorch's profiler to measure operators time and memory consumption

Basics

👁 Image

PyTorch Profiler with Instrumentation and Tracing Technology API (ITT API) support

Learn how to use PyTorch's profiler with Instrumentation and Tracing Technology API (ITT API) to visualize operators labeling in Intel® VTune™ Profiler GUI

Basics

👁 Image

Dynamic Compilation Control with ``torch.compiler.set_stance``

Learn how to use torch.compiler.set_stance

Compiler

👁 Image

Reasoning about Shapes in PyTorch

Learn how to use the meta device to reason about shapes in your model.

Basics

👁 Image

Tips for Loading an nn.Module from a Checkpoint

Learn tips for loading an nn.Module from a checkpoint.

Basics

👁 Image

(beta) Using TORCH_LOGS to observe torch.compile

Learn how to use the torch logging APIs to observe the compilation process.

Basics

👁 Image

Extension points in nn.Module for loading state_dict and tensor subclasses

New extension points in nn.Module.

Basics

👁 Image

torch.export AOTInductor Tutorial for Python runtime

Learn an end-to-end example of how to use AOTInductor for python runtime.

Basics

👁 Image

Demonstration of torch.export flow, common challenges and the solutions to address them

Learn how to export models for popular usecases

Compiler,TorchCompile

👁 Image

Model Interpretability using Captum

Learn how to use Captum attribute the predictions of an image classifier to their corresponding image features and visualize the attribution results.

Interpretability,Captum

👁 Image

How to use TensorBoard with PyTorch

Learn basic usage of TensorBoard with PyTorch, and how to visualize data in TensorBoard UI

Visualization,TensorBoard

👁 Image

DebugMode: Recording Dispatched Operations and Numerical Debugging

Inspect dispatched ops, tensor hashes, and module boundaries to debug eager and ``torch.compile`` runs.

Interpretability,Compiler

👁 Image

Automatic Mixed Precision

Use torch.cuda.amp to reduce runtime and save memory on NVIDIA GPUs.

Model-Optimization

👁 Image

Performance Tuning Guide

Tips for achieving optimal performance.

Model-Optimization

👁 Image

Optimizing CPU Performance on Intel® Xeon® with run_cpu Script

How to use run_cpu script for optimal runtime configurations on Intel® Xeon CPUs.

Model-Optimization

👁 Image

(beta) Utilizing Torch Function modes with torch.compile

Override torch operators with Torch Function modes and torch.compile

Model-Optimization

👁 Image

(beta) Compiling the Optimizer with torch.compile

Speed up the optimizer using torch.compile

Model-Optimization

👁 Image

(beta) Running the compiled optimizer with an LR Scheduler

Speed up training with LRScheduler and torch.compiled optimizer

Model-Optimization

👁 Image

(beta) Explicit horizontal fusion with foreach_map and torch.compile

Horizontally fuse pointwise ops with torch.compile

Model-Optimization

👁 Image

Using User-Defined Triton Kernels with ``torch.compile``

Learn how to use user-defined kernels with ``torch.compile``

Model-Optimization

👁 Image

Compile Time Caching in ``torch.compile``

Learn how to use compile time caching in ``torch.compile``

Model-Optimization

👁 Image

Compile Time Caching Configurations

Learn how to configure compile time caching in ``torch.compile``

Model-Optimization

👁 Image

Reducing torch.compile cold start compilation time with regional compilation

Learn how to use regional compilation to control cold start compile time

Model-Optimization

👁 Image

Intel® Neural Compressor for PyTorch

Ease-of-use quantization for PyTorch with Intel® Neural Compressor.

Quantization,Model-Optimization

👁 Image

Getting Started with DeviceMesh

Learn how to use DeviceMesh

Distributed-Training

👁 Image

Shard Optimizer States with ZeroRedundancyOptimizer

How to use ZeroRedundancyOptimizer to reduce memory consumption.

Distributed-Training

👁 Image

Direct Device-to-Device Communication with TensorPipe RPC

How to use RPC with direct GPU-to-GPU communication.

Distributed-Training

👁 Image

Getting Started with Distributed Checkpoint (DCP)

Learn how to checkpoint distributed models with Distributed Checkpoint package.

Distributed-Training

👁 Image

Asynchronous Checkpointing (DCP)

Learn how to checkpoint distributed models with Distributed Checkpoint package.

Distributed-Training

👁 Image

Getting Started with CommDebugMode

Learn how to use CommDebugMode for DTensors

Distributed-Training

👁 Image

Reducing AoT cold start compilation time with regional compilation

Learn how to use regional compilation to control AoT cold start compile time

Model-Optimization

👁 Image

URL: https://docs.pytorch.org/tutorials/recipes_index.html

⇱ Recipes — PyTorch Tutorials 2.12.0+cu130 documentation

Recipes#

Defining a Neural Network

What is a state_dict in PyTorch

Warmstarting model using parameters from a different model in PyTorch

Zeroing out gradients in PyTorch

PyTorch Benchmark

PyTorch Benchmark (quick start)

PyTorch Profiler

PyTorch Profiler with Instrumentation and Tracing Technology API (ITT API) support

Dynamic Compilation Control with ``torch.compiler.set_stance``

Reasoning about Shapes in PyTorch

Tips for Loading an nn.Module from a Checkpoint

(beta) Using TORCH_LOGS to observe torch.compile

Extension points in nn.Module for loading state_dict and tensor subclasses

torch.export AOTInductor Tutorial for Python runtime

Demonstration of torch.export flow, common challenges and the solutions to address them

Model Interpretability using Captum

How to use TensorBoard with PyTorch

DebugMode: Recording Dispatched Operations and Numerical Debugging

Automatic Mixed Precision

Performance Tuning Guide

Optimizing CPU Performance on Intel® Xeon® with run_cpu Script

(beta) Utilizing Torch Function modes with torch.compile

(beta) Compiling the Optimizer with torch.compile

(beta) Running the compiled optimizer with an LR Scheduler

(beta) Explicit horizontal fusion with foreach_map and torch.compile

Using User-Defined Triton Kernels with ``torch.compile``

Compile Time Caching in ``torch.compile``

Compile Time Caching Configurations

Reducing torch.compile cold start compilation time with regional compilation

Intel® Neural Compressor for PyTorch

Getting Started with DeviceMesh

Shard Optimizer States with ZeroRedundancyOptimizer

Direct Device-to-Device Communication with TensorPipe RPC

Getting Started with Distributed Checkpoint (DCP)

Asynchronous Checkpointing (DCP)

Getting Started with CommDebugMode

Reducing AoT cold start compilation time with regional compilation

Docs

Tutorials

Resources