Data CenterHopperNVLINKCUDA
Operating mode
Choose the operating mode for this hardware
Use this to bias workload recommendations toward responsiveness, background autonomy, lighter serving, or multi-GPU scale-out.
Current mode
Balanced
Balanced for general local use. Keeps the ranking neutral across personal and serving workflows.
About this GPU for AI
The NVIDIA H100 NVL is a unique dual-H100 card that fuses two H100 GPUs on a single PCIe Gen5 board, delivering 188 GB of HBM3 and 7.8 TB/s of combined bandwidth. The two GPUs are connected by three NVLink 4 bridges at 600 GB/s bidirectional, enabling them to act as a unified pool for large model inference. It is the highest-VRAM Hopper option available in a PCIe form factor, capable of running 70B models at FP16 with substantial KV cache and approaching 405B models at Q4. Benchmarks show up to 12x improvement over A100 systems for GPT-175B inference.
Beyond LLMs
AI Capability Matrix
What AI tasks this GPU can handle — from text generation to image and video creation.
| Capability | Status | Representative Model | Detail |
|---|
| LLM Chat (7B) | Runs natively | Llama 3.1 8B Q4 | — |
| LLM Coding (30B) | Runs natively | Qwen 3 30B Q4 | — |
| LLM Large (70B) |
hbm-memorymassive-vramhigh-bandwidthmulti-gpu-capabledatacenter-grade
Specifications
Compute
FP161979 TFLOPS
INT83958 TOPS
ArchitectureHopper
Memory
VRAM188 GB
Bandwidth7800 GB/s
General
FamilyData Center
SegmentData Center
InterconnectNVLINK
Compute PlatformCUDA
MSRP$60,000
Key Features
188 GB HBM3 total (94 GB per GPU × 2) — 7.8 TB/s combined bandwidth3,958 TFLOPS FP8 combined (1,979 per GPU)Dual H100 GPU on a single PCIe Gen5 board3× NVLink 4 bridges at 600 GB/s bidirectional GPU-GPU bandwidthMIG support: up to 7 instances per GPU (14 total)700–800W total TDP (350–400W per GPU)
For AI Workloads
Strengths
- 188 GB unified HBM3 pool eliminates GPU memory wall for 70B FP16 inference and enables 405B at Q4
- 7.8 TB/s combined bandwidth — near the top of the HBM3 class for decode throughput
- PCIe form factor with NVLink bridges fits in standard servers without SXM baseboard
- Up to 12x faster than A100 for GPT-175B inference; 5x faster for Llama 70B
Considerations
- Very high cost — dual-GPU card priced at premium above two individual H100 PCIe cards
- 600 GB/s NVLink bridge bandwidth between the two GPUs is lower than SXM's 900 GB/s intra-node fabric
- Niche form factor — few server designs accommodate the full thermal and power envelope
- Blackwell B100/B200 now available with comparable or higher VRAM at better compute-per-watt
Hopper is NVIDIA's datacenter-focused architecture succeeding Ampere. Built on TSMC 4N, it introduces the Transformer Engine with automatic FP8/FP16 mixed-precision training, HBM3/HBM3e memory, and NVLink 4.0 for multi-GPU scaling. The H100 flagship delivers up to 3x the AI training performance of A100.
AI Relevance
The Transformer Engine automatically manages FP8 precision for optimal training speed without accuracy loss. With up to 141 GB HBM3e (H200), Hopper GPUs can hold the largest open-weight models entirely in GPU memory, making them the workhorse of AI datacenters.
Process: TSMC 4NPlatform: CUDATensor Cores: Gen 4Precisions: FP64, FP32, TF32, FP16, BF16, FP8, INT8
Recommendations by Workload
Qwen 3.5 122B A10B matches Chat and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 159.3 tok/s · 131K ctx · llama.cppEST.
Devstral 2 123B Instruct is a specialized fit for Coding. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 71.5 tok/s · 201K ctx · llama.cppEST.
Just out of reach
Models you could run with an upgrade
High-quality models that need a bit more memory
397BTier 100Needs ~263.3 GB
1000BTier 100Needs ~633.4 GB
1000BTier 100Needs ~633.4 GB
1600BTier 100Needs ~882.6 GB
754BTier 92Needs ~489.2 GB
Image & Video Generation
Diffusion Model Compatibility
52 of 52 models can generate images or video on your H100 NVL 188GB
Upgrade paths
Upgrade from H100 NVL 188GB
See what you unlock with more powerful hardware
Upgrade options
Upgrade options
Frequently Asked Questions
H100 NVL 188GBCategory AvgAMD Instinct MI325X 256GB
| Image Gen (SDXL) | Runs natively | SDXL 1.0 FP16 | ~200ms per image |
| Image Gen (Flux) | Runs natively | Flux.1 Dev FP16 | ~700ms per image |
| Image Gen (SD 3.5) | Runs natively | SD 3.5 Large FP16 | ~900ms per image |
| Video Short (25f) | Runs natively | LTX Video 2B | ~100ms/frame |
| Video Long (100f) | Runs natively | Wan Video 14B | ~400ms/frame |
S
Devstral 2 123B Instruct is a specialized fit for Agentic Coding. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 71.5 tok/s · 201K ctx · llama.cppEST.
Devstral 2 123B Instruct matches Reasoning and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 71.5 tok/s · 201K ctx · llama.cppEST.
Qwen 3.5 122B A10B matches RAG and keeps a practical fit profile. It is a recent-generation family, which helps on current local SOTA workloads. It fits natively with comfortable headroom. Context coverage stays within the requested workload envelope. Known distribution channels: huggingface, lm-studio.
Decode 198.4 tok/s · 131K ctx · llama.cppEST.
95
122B96.6 GB254 tok/s131K ctx
Image
| MAGI-1Video | 1280×720 | 400ms/frame | S |
Image models estimated at 1024×1024 (28 steps, FP16). Video models estimated at 768×512 (25 frames, 30 steps, FP16). Actual performance varies with runtime and system load.
Buying advice
Should you buy H100 NVL 188GB for local AI?
Excellent choice for local AI
Runs 40 of 50 top models well — a strong all-rounder for local inference.
What will limit you first
This setup is broadly balanced for this model.
No major red flags
This recommendation has enough memory headroom and acceptable estimated speed for the selected workload.
Best upgrade itinerary
Unlocks 4 additional models that do not fit on the current setup.
Want more headroom? AMD Instinct MI325X 256GB (256.0 GB VRAM) is the next step up.
