Grace BlackwellDatacenterBlackwellNVLINKCUDA
Operating mode
Choose the operating mode for this hardware
Use this to bias workload recommendations toward responsiveness, background autonomy, lighter serving, or multi-GPU scale-out.
Current mode
Balanced
Balanced for general local use. Keeps the ranking neutral across personal and serving workflows.
About this GPU for AI
The NVIDIA GB200 Grace Blackwell Superchip is the per-module component of the GB200 NVL72 rack-scale system, pairing a Grace ARM CPU with two Blackwell B200 GPUs over 900 GB/s NVLink-C2C. Each module delivers 192 GB of HBM3e and up to 1,800 TFLOPS FP16, with FP4 Tensor Core support enabling even higher effective throughput for quantized inference. When combined in the GB200 NVL72 configuration — 72 GPUs acting as a single entity — the system delivers up to 30x more LLM inference throughput than an equivalent H100 cluster. This is NVIDIA's current cutting-edge platform for trillion-parameter model serving.
Beyond LLMs
AI Capability Matrix
What AI tasks this GPU can handle — from text generation to image and video creation.
| Capability | Status | Representative Model | Detail |
|---|
| LLM Chat (7B) | Runs natively | Llama 3.1 8B Q4 | — |
| LLM Coding (30B) | Runs natively | Qwen 3 30B Q4 | — |
| LLM Large (70B) |
massive-vramhbm-memoryblackwell-architecturebest-in-classpower-hungryenterprise-onlycpu-gpu-integrated
Specifications
Compute
FP161800 TFLOPS
INT83600 TOPS
ArchitectureBlackwell
Memory
VRAM192 GB
Bandwidth8000 GB/s
General
FamilyGrace Blackwell
SegmentDatacenter
InterconnectNVLINK
Compute PlatformCUDA
MSRP$60,000
Key Features
192 GB HBM3e across two B200 GPUs (per module)8,000 GB/s HBM3e bandwidth1,800 TFLOPS FP16 / 3,600 INT8 TOPS / FP4 Tensor Core supportGrace ARM CPU with 900 GB/s NVLink-C2C interconnectNVLink 5.0 with 1.8 TB/s per-GPU bandwidth~1,200W+ TDP — liquid cooling required
For AI Workloads
Strengths
- 192 GB HBM3e per module enables 70B models at FP16 with massive KV cache headroom
- FP4 Tensor Cores deliver the highest available inference throughput for quantized LLMs
- GB200 NVL72 pools 72 GPUs into a 13.5 TB single-domain system — the only current platform for real-time trillion-parameter inference
- 30x inference throughput improvement vs. equivalent H100 count for LLM workloads
Considerations
- Rack-scale liquid-cooled infrastructure required — incompatible with legacy air-cooled data centers
- Extremely high cost; GB200 NVL72 rack systems run into the tens of millions of dollars
- Software ecosystem for FP4 and Blackwell-specific optimizations still maturing
- Hardware backlog through mid-2026 — not readily acquirable even at enterprise budget levels
Blackwell is NVIDIA's fifth-generation RTX architecture, built on TSMC's 4NP process. It introduces 5th-generation Tensor Cores with native FP4 precision support, enabling double the inference throughput per watt compared to Ada Lovelace's FP8 operations. Key innovations include the Neural Rendering Pipeline for AI-driven shading and the debut of GDDR7 memory in consumer GPUs.
AI Relevance
FP4 Tensor Cores deliver the highest tokens-per-watt efficiency in any consumer architecture. Native FP4 quantization means models can run at lower precision with minimal quality loss, effectively doubling the effective VRAM for model weights.
Process: TSMC 4NPPlatform: CUDATensor Cores: Gen 5Precisions: FP32, FP16, BF16, FP8, FP4, INT8, INT4
Recommendations by Workload
Qwen 3.5 122B A10B matches Chat and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 169.4 tok/s · 131K ctx · llama.cppEST.
Devstral 2 123B Instruct is a specialized fit for Coding. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 76.1 tok/s · 212K ctx · llama.cppEST.
Just out of reach
Models you could run with an upgrade
High-quality models that need a bit more memory
397BTier 100Needs ~263.7 GB
Also runs on 2× your GPU via NVLink — 169 tok/s
1000BTier 100Needs ~633.8 GB
1000BTier 100Needs ~633.8 GB
1600BTier 100Needs ~883.0 GB
754BTier 92Needs ~489.6 GB
Image & Video Generation
Diffusion Model Compatibility
52 of 52 models can generate images or video on your NVIDIA GB200 192GB
Multi-GPU scaling
NVIDIA GB200 192GB — Up to 2× via NVLink
Scale out with multiple GPUs for larger models. NVLink provides 1800 GB/s inter-GPU bandwidth with 7% overhead.
| Config | Effective memory | Models that fit | Est. bandwidth |
|---|
| 1× NVIDIA | 192 GB | 359/374 | 8,000 GB/s |
| 2× NVIDIA | 384 GB | 366/374 | 14,880 GB/s |
Model counts use default quantization at coding workload settings. Multi-GPU scaling factor: 0.93× per additional GPU.
Upgrade paths
Upgrade from NVIDIA GB200 192GB
See what you unlock with more powerful hardware
Upgrade options
Upgrade options
Frequently Asked Questions
192
GB
NVIDIA GB200 192GBCategory AvgAMD Instinct MI325X 256GB
| Image Gen (SDXL) | Runs natively | SDXL 1.0 FP16 | ~200ms per image |
| Image Gen (Flux) | Runs natively | Flux.1 Dev FP16 | ~700ms per image |
| Image Gen (SD 3.5) | Runs natively | SD 3.5 Large FP16 | ~900ms per image |
| Video Short (25f) | Runs natively | LTX Video 2B | ~100ms/frame |
| Video Long (100f) | Runs natively | Wan Video 14B | ~400ms/frame |
S
Devstral 2 123B Instruct is a specialized fit for Agentic Coding. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 76.1 tok/s · 212K ctx · llama.cppEST.
Devstral 2 123B Instruct matches Reasoning and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 76.1 tok/s · 212K ctx · llama.cppEST.
Qwen 3.5 122B A10B matches RAG and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 169.4 tok/s · 131K ctx · llama.cppEST.
122B97.0 GB270 tok/s131K ctx
Image
| MAGI-1Video | 1280×720 | 400ms/frame | S |
Image models estimated at 1024×1024 (28 steps, FP16). Video models estimated at 768×512 (25 frames, 30 steps, FP16). Actual performance varies with runtime and system load.
Buying advice
Should you buy NVIDIA GB200 192GB for local AI?
Excellent choice for local AI
Runs 40 of 50 top models well — a strong all-rounder for local inference.
What will limit you first
This setup is broadly balanced for this model.
No major red flags
This recommendation has enough memory headroom and acceptable estimated speed for the selected workload.
Best upgrade itinerary
Unlocks 4 additional models that do not fit on the current setup.
Want more headroom? AMD Instinct MI325X 256GB (256.0 GB VRAM) is the next step up.
