Blackwell DatacenterDatacenterBlackwellNVLINKCUDA
Operating mode
Choose the operating mode for this hardware
Use this to bias workload recommendations toward responsiveness, background autonomy, lighter serving, or multi-GPU scale-out.
Current mode
Balanced
Balanced for general local use. Keeps the ranking neutral across personal and serving workflows.
About this GPU for AI
The NVIDIA B200 is the flagship Blackwell datacenter GPU, delivering 180 GB of HBM3e and 2,250 TFLOPS of FP16 compute — roughly 2.3x the compute of an H100 at over twice the VRAM. Its new fourth-generation Tensor Cores add FP4 support, enabling up to 4,500 TOPS for FP4 inference, and the Blackwell architecture introduces a second-generation Transformer Engine. A single B200 can serve 70B models at FP16 with headroom for large batch sizes and long context windows, making it suitable for high-throughput production inference. At approximately 1,000W TDP, it targets next-generation liquid-cooled infrastructure.
Beyond LLMs
AI Capability Matrix
What AI tasks this GPU can handle — from text generation to image and video creation.
| Capability | Status | Representative Model | Detail |
|---|
| LLM Chat (7B) | Runs natively | Llama 3.1 8B Q4 | — |
| LLM Coding (30B) | Runs natively | Qwen 3 30B Q4 | — |
| LLM Large (70B) |
hbm-memorymassive-vramhigh-bandwidthblackwell-architecturebest-in-classpower-hungry
Specifications
Compute
FP162250 TFLOPS
INT84500 TOPS
ArchitectureBlackwell
Memory
VRAM180 GB
Bandwidth8000 GB/s
General
FamilyBlackwell Datacenter
SegmentDatacenter
InterconnectNVLINK
Compute PlatformCUDA
MSRP$30,000
Key Features
180 GB HBM3e — largest memory capacity in the B200 lineup8,000 GB/s memory bandwidth2,250 TFLOPS FP16 / 4,500 INT8 TOPS / FP4 Tensor Core support2nd-gen Transformer Engine for FP8 and FP4 inferenceNVLink 5.0 with 1.8 TB/s per-GPU bandwidth for multi-GPU scaling~1,000W TDP — requires liquid or next-gen air cooling
For AI Workloads
Strengths
- 180 GB HBM3e handles 70B models at FP16 and 405B+ models with Q4 on a single card
- 8 TB/s bandwidth is among the highest available, enabling fast token generation at large batch sizes
- FP4 Tensor Cores deliver up to 2.3x higher inference throughput vs. H100 FP8
- NVLink 5.0 enables efficient 8-GPU HGX B200 clusters with 1.44 TB pooled memory
Considerations
- ~1,000W TDP demands liquid cooling infrastructure — not compatible with legacy H100 SXM racks
- Extremely high cost — list pricing well above H100, with significant waitlists
- Software ecosystem still maturing — TensorRT-LLM and vLLM FP4 support launched recently
- Overkill for serving models below 30B parameters; ROI requires high-utilization production workloads
Blackwell is NVIDIA's fifth-generation RTX architecture, built on TSMC's 4NP process. It introduces 5th-generation Tensor Cores with native FP4 precision support, enabling double the inference throughput per watt compared to Ada Lovelace's FP8 operations. Key innovations include the Neural Rendering Pipeline for AI-driven shading and the debut of GDDR7 memory in consumer GPUs.
AI Relevance
FP4 Tensor Cores deliver the highest tokens-per-watt efficiency in any consumer architecture. Native FP4 quantization means models can run at lower precision with minimal quality loss, effectively doubling the effective VRAM for model weights.
Process: TSMC 4NPPlatform: CUDATensor Cores: Gen 5Precisions: FP32, FP16, BF16, FP8, FP4, INT8, INT4
Recommendations by Workload
Qwen 3.5 122B A10B matches Chat and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 211.0 tok/s · 131K ctx · llama.cppEST.
Devstral 2 123B Instruct is a specialized fit for Coding. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 76.1 tok/s · 179K ctx · llama.cppEST.
Just out of reach
Models you could run with an upgrade
High-quality models that need a bit more memory
397BTier 100Needs ~262.5 GB
Also runs on 2× your GPU via NVLink — 169 tok/s
1000BTier 100Needs ~632.6 GB
Also runs on 4× your GPU via NVLink — 135 tok/s
1000BTier 100Needs ~632.6 GB
Also runs on 4× your GPU via NVLink — 135 tok/s
1600BTier 100Needs ~881.8 GB
Also runs on 8× your GPU via NVLink — 199 tok/s
754BTier 92Needs ~488.4 GB
Also runs on 4× your GPU via NVLink — 159 tok/s
Image & Video Generation
Diffusion Model Compatibility
52 of 52 models can generate images or video on your NVIDIA B200 180GB
Multi-GPU scaling
NVIDIA B200 180GB — Up to 8× via NVLink
Scale out with multiple GPUs for larger models. NVLink provides 1800 GB/s inter-GPU bandwidth with 7% overhead.
| Config | Effective memory | Models that fit | Est. bandwidth |
|---|
| 1× NVIDIA | 180 GB | 359/374 | 8,000 GB/s |
| 2× NVIDIA | 360 GB | 364/374 | 14,880 GB/s |
| 4× NVIDIA | 720 GB | 373/374 | 29,760 GB/s |
| 8× NVIDIA | 1440 GB | 374/374 | 59,520 GB/s |
Model counts use default quantization at coding workload settings. Multi-GPU scaling factor: 0.93× per additional GPU.
Upgrade paths
Upgrade from NVIDIA B200 180GB
See what you unlock with more powerful hardware
Upgrade options
Upgrade options
Frequently Asked Questions
180
GB
NVIDIA B200 180GBCategory AvgAMD Instinct MI325X 256GB
| Image Gen (SDXL) | Runs natively | SDXL 1.0 FP16 | ~100ms per image |
| Image Gen (Flux) | Runs natively | Flux.1 Dev FP16 | ~600ms per image |
| Image Gen (SD 3.5) | Runs natively | SD 3.5 Large FP16 | ~700ms per image |
| Video Short (25f) | Runs natively | LTX Video 2B | ~100ms/frame |
| Video Long (100f) | Runs natively | Wan Video 14B | ~300ms/frame |
S
Devstral 2 123B Instruct is a specialized fit for Agentic Coding. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 76.1 tok/s · 179K ctx · llama.cppEST.
Devstral 2 123B Instruct matches Reasoning and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 76.1 tok/s · 179K ctx · llama.cppEST.
Qwen 3.5 122B A10B matches RAG and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 211.0 tok/s · 131K ctx · llama.cppEST.
96
122B95.8 GB270 tok/s131K ctx
Image
| MAGI-1Video | 1280×720 | 300ms/frame | S |
Image models estimated at 1024×1024 (28 steps, FP16). Video models estimated at 768×512 (25 frames, 30 steps, FP16). Actual performance varies with runtime and system load.
Buying advice
Should you buy NVIDIA B200 180GB for local AI?
Excellent choice for local AI
Runs 39 of 50 top models well — a strong all-rounder for local inference.
What will limit you first
This setup is broadly balanced for this model.
No major red flags
This recommendation has enough memory headroom and acceptable estimated speed for the selected workload.
Best upgrade itinerary
Unlocks 4 additional models that do not fit on the current setup.
Want more headroom? AMD Instinct MI325X 256GB (256.0 GB VRAM) is the next step up.
