AMD
AMD Instinct MI325X 256GB
InstinctDatacenterCDNA 4OAMROCm
Operating mode
Choose the operating mode for this hardware
Use this to bias workload recommendations toward responsiveness, background autonomy, lighter serving, or multi-GPU scale-out.
Current mode
Balanced
Balanced for general local use. Keeps the ranking neutral across personal and serving workflows.
About this GPU for AI
The AMD Instinct MI325X 256GB is AMD's enhanced CDNA 4-based datacenter GPU, offering 256 GB of HBM3e memory and 6 TB/s of bandwidth. It builds on the MI300X architecture with increased memory capacity and bandwidth, targeting the largest production LLM inference workloads. The 1307 TFLOPS FP16 compute is matched with a larger memory envelope, enabling inference of the largest open-source models (405B, Llama-3-405B) in full precision without quantization.
Beyond LLMs
AI Capability Matrix
What AI tasks this GPU can handle — from text generation to image and video creation.
| Capability | Status | Representative Model | Detail |
|---|
| LLM Chat (7B) | Runs natively | Llama 3.1 8B Q4 | — |
| LLM Coding (30B) | Runs natively | Qwen 3 30B Q4 | — |
| LLM Large (70B) |
rocm-supporteddatacenter-gradehigh-bandwidthhigh-vramflagship
Specifications
Compute
FP161307 TFLOPS
INT82614 TOPS
ArchitectureCDNA 4
Memory
VRAM256 GB
Bandwidth6000 GB/s
General
FamilyInstinct
SegmentDatacenter
InterconnectOAM
Compute PlatformROCM
MSRP$20,000
Key Features
CDNA 4 architecture (enhanced CDNA 3 platform)256 GB HBM3e across 8 stacks6 TB/s memory bandwidthImproved Matrix Core throughput with FP8/BF16/FP16AMD Infinity Fabric xGMI 3.0 multi-card interconnectFull ROCm support — production inference platform
For AI Workloads
Strengths
- 256 GB HBM3e — largest single-GPU memory available for inference
- 6 TB/s bandwidth exceeds MI300X for large-model decode throughput
- Enables 405B models in FP16 without multi-card splitting
- Full ROCm ecosystem support — vLLM, SGLang, PyTorch all validated
Considerations
- Extremely expensive ($20,000+) — datacenter-only product
- OAM form factor requires OCP/OAM server infrastructure
- 1307 TFLOPS FP16 is similar to MI300X — gain is memory, not compute
- NVIDIA H200 141GB offers competitive inference performance with larger CUDA ecosystem
CDNA 4 powers the next-generation Instinct MI325X and MI350X accelerators. Built on TSMC 3nm with up to 288 GB HBM3e memory and native FP4 support for maximum inference density.
AI Relevance
With up to 288 GB HBM3e and FP4 support, CDNA 4 targets the highest-density AI inference deployments. Directly competes with NVIDIA Blackwell B200 for large-scale model serving.
Process: TSMC 3nmPlatform: ROCMPrecisions: FP64, FP32, TF32, FP16, BF16, FP8, FP4, INT8
Recommendations by Workload
Qwen 3.5 122B A10B matches Chat and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 110.4 tok/s · 131K ctx · llama.cppEST.
DeepSeek V4 Flash is a specialized fit for Coding. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface.
Decode 82.5 tok/s · 686K ctx · llama.cppEST.
Just out of reach
Models you could run with an upgrade
High-quality models that need a bit more memory
1000BTier 100Needs ~640.2 GB
Also runs on 4× your GPU via Infinity Fabric — 83 tok/s
1000BTier 100Needs ~640.2 GB
Also runs on 4× your GPU via Infinity Fabric — 83 tok/s
1600BTier 100Needs ~889.4 GB
Also runs on 4× your GPU via Infinity Fabric — 61 tok/s
754BTier 92Needs ~496.0 GB
Also runs on 2× your GPU via Infinity Fabric — 35 tok/s
744BTier 91Needs ~489.9 GB
Also runs on 2× your GPU via Infinity Fabric — 37 tok/s
Image & Video Generation
Diffusion Model Compatibility
52 of 52 models can generate images or video on your AMD Instinct MI325X 256GB
Multi-GPU scaling
AMD Instinct MI325X 256GB — Up to 8× via Infinity Fabric
Scale out with multiple GPUs for larger models. Infinity Fabric provides 896 GB/s inter-GPU bandwidth with 12% overhead.
| Config | Effective memory | Models that fit | Est. bandwidth |
|---|
| 1× AMD | 256 GB | 363/374 | 6,000 GB/s |
| 2× AMD | 512 GB | 371/374 | 10,560 GB/s |
| 4× AMD | 1024 GB | 374/374 | 21,120 GB/s |
| 8× AMD | 2048 GB | 374/374 | 42,240 GB/s |
Model counts use default quantization at coding workload settings. Multi-GPU scaling factor: 0.88× per additional GPU.
Upgrade paths
Upgrade from AMD Instinct MI325X 256GB
See what you unlock with more powerful hardware
Upgrade options
Upgrade options
Frequently Asked Questions
AMD Instinct MI325X 256GBCategory AvgAMD Instinct MI350X 288GB
| Image Gen (SDXL) | Runs natively | SDXL 1.0 FP16 | ~300ms per image |
| Image Gen (Flux) | Runs natively | Flux.1 Dev FP16 | ~~1.2s per image |
| Image Gen (SD 3.5) | Runs natively | SD 3.5 Large FP16 | ~~1.4s per image |
| Video Short (25f) | Runs natively | LTX Video 2B | ~200ms/frame |
| Video Long (100f) | Runs natively | Wan Video 14B | ~700ms/frame |
S
DeepSeek V4 Flash is a specialized fit for Agentic Coding. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface.
Decode 82.5 tok/s · 686K ctx · llama.cppEST.
Devstral 2 123B Instruct matches Reasoning and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 39.8 tok/s · 256K ctx · llama.cppEST.
Qwen 3.5 122B A10B matches RAG and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 110.4 tok/s · 131K ctx · llama.cppEST.
93
122B103.4 GB176 tok/s131K ctx
Image
| MAGI-1Video | 1280×720 | 600ms/frame | S |
Image models estimated at 1024×1024 (28 steps, FP16). Video models estimated at 768×512 (25 frames, 30 steps, FP16). Actual performance varies with runtime and system load.
Buying advice
Should you buy AMD Instinct MI325X 256GB for local AI?
Excellent choice for local AI
Runs 41 of 50 top models well — a strong all-rounder for local inference.
What will limit you first
This setup is broadly balanced for this model.
No major red flags
This recommendation has enough memory headroom and acceptable estimated speed for the selected workload.
Best upgrade itinerary
Unlocks 1 additional models that do not fit on the current setup.
Want more headroom? AMD Instinct MI350X 288GB (288.0 GB VRAM) is the next step up.
