Intel

Gaudi 3 128GB

Data CenterGaudiPCIe 5oneAPI

128GB

VRAM

3.7kGB/s

Bandwidth

900TFLOPS

FP16 Compute

1.8kTOPS

INT8 Inference

$15,000 MSRP

Gaudi 3 128GBCategory AvgNVIDIA H200 141GB

Operating mode

Choose the operating mode for this hardware

Use this to bias workload recommendations toward responsiveness, background autonomy, lighter serving, or multi-GPU scale-out.

Current mode

Balanced

Balanced for general local use. Keeps the ranking neutral across personal and serving workflows.

See Full AI Tier List for Gaudi 3 128GB →

About this GPU for AI

Intel Gaudi 3 is a dedicated AI accelerator — not a traditional GPU — designed from the ground up for large-scale LLM training and inference. It delivers 1.8 PFlops of BF16/FP8 compute and 3.7 TB/s of HBM2e bandwidth across 128 GB of on-package memory. Intel claims Gaudi 3 outperforms the NVIDIA H100 by 50% on average inference throughput for models like Llama 7B, 70B, and Falcon 180B, while delivering 40% better inference power efficiency. It integrates natively with PyTorch, Hugging Face, DeepSpeed, and vLLM via Intel's Synapse AI software stack.

Beyond LLMs

AI Capability Matrix

What AI tasks this GPU can handle — from text generation to image and video creation.

Capability	Status	Representative Model	Detail
LLM Chat (7B)	Runs natively	Llama 3.1 8B Q4	—
LLM Coding (30B)	Runs natively	Qwen 3 30B Q4	—
LLM Large (70B)	Runs natively	Llama 3.1 70B Q4	—
Image Gen (SDXL)	Runs natively	SDXL 1.0 FP16	~400ms per image
Image Gen (Flux)	Runs natively	Flux.1 Dev FP16	~~1.8s per image
Image Gen (SD 3.5)	Runs natively	SD 3.5 Large FP16	~~2.2s per image
Video Short (25f)	Runs natively	LTX Video 2B	~300ms/frame
Video Long (100f)	Runs natively	Wan Video 14B	~~1s/frame

datacenter-gradeai-acceleratorhbm-memoryhigh-vram

Specifications

Compute

FP16900 TFLOPS

INT81835 TOPS

ArchitectureGaudi

Memory

VRAM128 GB

Bandwidth3700 GB/s

General

FamilyData Center

SegmentData Center

InterconnectPCIe 5

Compute PlatformONEAPI

MSRP$15,000

Key Features

64 Tensor Processor Cores (TPCs) + dedicated GEMM engines for matrix operations128 GB HBM2e at 3.7 TB/s memory bandwidth1.8 PFlops BF16/FP8 compute — competitive with H10024x 200GbE RoCE networking for multi-node scale-outNative PyTorch, Hugging Face Transformers, DeepSpeed, and vLLM integrationOAM (Open Compute Accelerator Module) and PCIe Gen 5 form factors available

For AI Workloads

Strengths

Outperforms H100 by ~50% on Llama 7B/70B/Falcon 180B inference throughput per Intel benchmarks
3.7 TB/s HBM bandwidth enables very high token throughput — up to ~15,000 tokens/sec on Llama 3.1 8B
128 GB HBM2e fits 70B models at FP16 and 405B models across 8 cards
Open, standards-based Ethernet fabric for cluster scale-out — no proprietary interconnect required

Considerations

Synapse AI software stack is significantly less mature than CUDA — smaller community and fewer ready-made solutions
Not a drop-in replacement for NVIDIA in existing CUDA-based MLOps pipelines
Limited cloud availability compared to H100 — fewer managed service providers offer Gaudi 3 instances
Enterprise adoption and third-party tooling ecosystem substantially lags NVIDIA data center offerings

Architecture

Gaudi

Gaudi is Intel's purpose-built AI training and inference accelerator (acquired from Habana Labs). Gaudi 3 features 128 GB HBM2e and a dedicated Matrix Math Engine designed specifically for transformer workloads.

AI Relevance

Purpose-built for AI with integrated networking (24x 200GbE) for multi-node scaling. Gaudi 3 targets direct competition with NVIDIA H100 for transformer training and inference, with competitive TCO claims.

Process: TSMC 5nmPlatform: ONEAPIPrecisions: FP32, TF32, FP16, BF16, FP8, INT8

Buying advice

Should you buy Gaudi 3 128GB for local AI?

Excellent choice for local AI

Runs 36 of 50 top models well — a strong all-rounder for local inference.

128.0 GB

VRAM

$15,000

MSRP

$117/GB

Cost per GB VRAM

Best models for this GPU

Qwen 3.5 122B A10B — 99/100, 104 tok/s, 90.6 GB needed
Mistral Small 4 119B — 97/100, 113 tok/s, 91.7 GB needed
Devstral 2 123B Instruct — 97/100, 38 tok/s, 94.1 GB needed

What will limit you first

The raw memory story may look fine, but the software ecosystem is still a constraint here.

Runtime ecosystem is narrower than CUDA

Intel GPUs can look attractive on memory per dollar, but local AI tooling, kernels, and model coverage are still broader and easier on CUDA today.

Best upgrade itinerary

Prefer CUDA if you want the path of least resistance

If your goal is maximum runtime coverage, easier troubleshooting, and better support for new local AI releases, CUDA is usually still the safer upgrade path.

Unlocks 2 additional models that do not fit on the current setup.

Want more headroom? NVIDIA H200 141GB (141.0 GB VRAM) is the next step up.

Recommendations by Workload

Chat

Mistral Small 4 119B

This model is a direct match for chat. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, lm-studio.

Decode 112.9 tok/s · 124K ctx · llama.cppEST.

89.0 GB / 128.0 GB VRAM

Coding

Qwen3-Coder-Next

This model is a direct match for coding. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, ollama, lm-studio.

Decode 174.9 tok/s · 256K ctx · llama.cppEST.

64.0 GB / 128.0 GB VRAM

Agentic Coding

Devstral 2 123B Instruct

This model is still usable for agentic-coding, but it is not the most specialized pick. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, lm-studio.

Decode 37.5 tok/s · 117K ctx · llama.cppEST.

99.5 GB / 128.0 GB VRAM

Reasoning

Devstral 2 123B Instruct

This model is a direct match for reasoning. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, lm-studio.

Decode 37.5 tok/s · 117K ctx · llama.cppEST.

94.1 GB / 128.0 GB VRAM

RAG

Qwen 3.5 122B A10B

This model is a direct match for rag. It belongs to a current frontier family for local AI. It fits natively with comfortable headroom. Known channels: huggingface, lm-studio.

Decode 104.1 tok/s · 131K ctx · llama.cppEST.

93.0 GB / 128.0 GB VRAM

Full Model Compatibility

👁 Alibaba
Qwen 3.5 122B A10B

S99

122B90.6 GB104 tok/s131K ctx

moe

👁 Mistral
Mistral Small 4 119B

S97

119B91.7 GB113 tok/s124K ctx

moe

👁 Mistral
Devstral 2 123B Instruct

S97

123B94.1 GB38 tok/s117K ctx

dense

👁 OpenAI
GPT-OSS 120B

S95

117B90.0 GB40 tok/s131K ctx

dense

👁 Cohere
Command A 111B

S94

111B85.3 GB42 tok/s191K ctx

dense

👁 Mistral AI
Pixtral Large 124B

S93

124B94.7 GB37 tok/s115K ctx

dense

👁 Mistral
Leanstral 119B A6B

S93

119B95.1 GB104 tok/s76K ctx

moe

👁 Alibaba
Qwen3-Coder-Next

S92

80B64.0 GB175 tok/s256K ctx

moe

👁 Alibaba
Qwen 2.5 VL 72B

S92

72B62.5 GB64 tok/s33K ctx

dense

👁 Alibaba
Qwen 3.6 35B A3B

S91

35B39.2 GB329 tok/s262K ctx

+1moe

👁 Alibaba
Qwen3-Coder 30B A3B Instruct

S91

30.5B33.8 GB392 tok/s256K ctx

moe

👁 Alibaba
Qwen 3.5 27B

S90

27B33.3 GB170 tok/s131K ctx

dense

👁 Alibaba
Qwen3-VL 30B A3B Instruct

S90

30B33.5 GB405 tok/s256K ctx

moe

👁 Alibaba
Qwen 3.5 35B A3B

S90

35B36.5 GB358 tok/s131K ctx

moe

👁 Alibaba
Qwen 3.6 27B

S90

27B31.1 GB106 tok/s262K ctx

+1dense

👁 Alibaba
Qwen 3 32B

S89

32B37.1 GB144 tok/s131K ctx

dense

👁 Mistral
Magistral Small 2507

S88

24B30.8 GB190 tok/s131K ctx

dense

👁 Mistral
Devstral Small 2 24B Instruct

S88

24B30.8 GB190 tok/s256K ctx

dense

👁 Alibaba
Qwen 3 30B A3B

S88

30.5B33.8 GB392 tok/s131K ctx

moe

👁 NVIDIA
Nemotron 3 Nano 30B

S88

30B34.4 GB152 tok/s131K ctx

dense

👁 Alibaba
Qwen 3.5 9B

S87

9B21.4 GB126 tok/s131K ctx

dense

👁 Alibaba
Qwen 3 14B

S87

14B24.7 GB196 tok/s131K ctx

dense

👁 Google
Gemma 4 31B

S87

30.7B47.1 GB90 tok/s104K ctx

dense

👁 Mistral
Devstral Small 1.1

S87

24B30.8 GB190 tok/s131K ctx

dense

👁 Microsoft
Phi-4-reasoning-plus 14B

S86

14.7B25.7 GB206 tok/s33K ctx

dense

👁 NVIDIA
Nemotron Cascade 2 30B A3B

S85

30B34.9 GB400 tok/s262K ctx

moe

👁 Alibaba
Qwen 3 8B

S85

8B20.8 GB112 tok/s131K ctx

dense

👁 OpenAI
GPT-OSS 20B

S85

21B29.0 GB497 tok/s128K ctx

moe

👁 Alibaba
Qwen 3.5 4B

A83

4B18.3 GB56 tok/s131K ctx

dense

👁 LG AI
EXAONE 4.0 32B

A83

32B37.1 GB143 tok/s131K ctx

dense

👁 Google
Gemma 4 26B A4B

A82

25.2B32.7 GB421 tok/s256K ctx

moe

👁 Mistral
Ministral 3 14B

A81

14B24.7 GB196 tok/s262K ctx

multimodal

👁 NVIDIA
Nemotron Nano 8B

A80

8B20.5 GB112 tok/s131K ctx

dense

👁 Microsoft
Phi-4 Mini Reasoning 4B

A80

3.8B17.5 GB53 tok/s131K ctx

dense

👁 Jina AI
Jina Embeddings v3

A74

0.57B16.8 GB8 tok/s8K ctx

dense

👁 BAAI
BGE M3

A73

0.57B16.0 GB8 tok/s8K ctx

dense

👁 Alibaba
Qwen 3.5 397B A17B

397B258.7 GB6 tok/s4K ctx

moe

👁 Moonshot AI
Kimi K2.5

1000B631.1 GB2 tok/s4K ctx

moe

👁 Moonshot AI
Kimi K2.6

1000B631.1 GB2 tok/s4K ctx

+1moe

👁 DeepSeek
DeepSeek V4 Pro

1600B877.6 GB2 tok/s4K ctx

moe

👁 DeepSeek
DeepSeek V4 Flash

284B173.0 GB23 tok/s4K ctx

moe

👁 Z.ai
GLM-5.1

754B492.7 GB3 tok/s4K ctx

moe

👁 Z.ai
GLM-5

744B486.6 GB3 tok/s4K ctx

moe

👁 DeepSeek
DeepSeek V3.2

671B423.5 GB3 tok/s4K ctx

moe

👁 Alibaba
Qwen 3 235B A22B

235B159.9 GB26 tok/s4K ctx

moe

👁 Alibaba
Qwen3-Coder 480B A35B Instruct

480B309.4 GB4 tok/s4K ctx

moe

MiniMax M2.7

230B157.8 GB30 tok/s4K ctx

moe

👁 DeepSeek
DeepSeek Coder V2 236B

236B216.3 GB13 tok/s4K ctx

moe

👁 DeepSeek
DeepSeek R1 671B

671B482.6 GB3 tok/s4K ctx

moe

👁 DeepSeek
DeepSeek V3.1 671B

671B482.6 GB3 tok/s4K ctx

moe

Just out of reach

Models you could run with an upgrade

High-quality models that need a bit more memory

👁 Alibaba
Qwen 3.5 397B A17B

397BTier 100Needs ~257.3 GB

Runs on AMD Instinct MI350X 288GB (~$8,000)

Also runs on 2× your GPU via PCIe — 41 tok/s

👁 Moonshot AI
Kimi K2.5

1000BTier 100Needs ~627.4 GB

Also runs on 8× your GPU via PCIe — 95 tok/s

👁 Moonshot AI
Kimi K2.6

1000BTier 100Needs ~627.4 GB

Also runs on 8× your GPU via PCIe — 95 tok/s

👁 DeepSeek
DeepSeek V4 Pro

1600BTier 100Needs ~876.6 GB

Also runs on 8× your GPU via PCIe — 70 tok/s

👁 DeepSeek
DeepSeek V4 Flash

284BTier 98Needs ~172.4 GB

Runs on AMD Instinct MI350X 288GB (~$8,000)

Also runs on 2× your GPU via PCIe — 95 tok/s

Image & Video Generation

Diffusion Model Compatibility

52 of 52 models can generate images or video on your Gaudi 3 128GB

Model	Max Resolution	Gen Time	Grade
SD TurboImage	512×512	0ms	S
Stable Diffusion 1.5Image	512×768	100ms	S
Realistic Vision v5.1Image	512×768	100ms	S
DreamShaper 8Image	512×768	100ms	S
LCM DreamShaper v7Image	512×768	0ms	S
PixArt-SigmaImage	1024×1024	400ms	S
FramePack I2VVideo	1280×720	700ms/frame	S
SDXL TurboImage	512×512	0ms	S
SDXL LightningImage	1024×1024	100ms	S
Stable Diffusion XL 1.0Image	1024×1024	400ms	S
Playground v2.5Image	1024×1024	600ms	S
RealVisXL v5.0Image	1024×1024	400ms	S
DreamShaper XLImage	1024×1024	400ms	S
Juggernaut XL v9Image	1024×1024	400ms	S
Animagine XL 3.1Image	1024×1024	400ms	S
Pony Diffusion V6 XLImage	1024×1024	400ms	S
Animagine XL 4.0Image	1024×1024	400ms	S
Illustrious XLImage	1024×1024	400ms	S
Wan Video 2.1 1.3BVideo	480×832	300ms/frame	S
Stable Diffusion 3.5 MediumImage	1024×1024	700ms	S
Flux.2 Klein 4BImage	1024×1024	100ms	S
LTX Video 2BVideo	1280×720	300ms/frame	S
KolorsImage	1024×1024	800ms	S
Stable CascadeImage	1024×1024	~1s	S
AuraFlow v0.3Image	1536×1536	~1.8s	S
Stable Diffusion 3.5 LargeImage	1024×1024	~2.2s	S
Stable Diffusion 3.5 Large TurboImage	1024×1024	400ms	S
CogVideoX 2BVideo	720×480	300ms/frame	S
HunyuanVideoVideo	720×1280	700ms/frame	S
ChromaImage	1024×1024	400ms	S
Z-Image TurboImage	1536×1536	400ms	S
Flux.1 DevImage	1024×1024	~1.8s	S
Flux.1 SchnellImage	1024×1024	300ms	S
LTX Video 13BVideo	1280×720	700ms/frame	S
Flux.1 Kontext DevImage	1024×1024	~2s	S
AnimateDiff v1.5.3Video	512×768	200ms/frame	S
Cosmos Diffusion 7BVideo	1024×576	600ms/frame	S
CogVideoX 5BVideo	720×480	500ms/frame	S
Wan2.2 TI2V 5BVideo	832×480	500ms/frame	S
Flux.2 Klein 9BImage	1024×1024	200ms	S
Flux.1 Fill DevImage	1024×1024	~1.7s	S
Mochi 1 PreviewVideo	848×480	700ms/frame	S
HunyuanVideo 1.5Video	720×1280	600ms/frame	S
Helios 14BVideo	1280×720	700ms/frame	S
SkyReels V2 14BVideo	1280×720	700ms/frame	S
Wan Video 2.1 14BVideo	720×1280	700ms/frame	S
Wan Video 2.2 14BVideo	720×1280	700ms/frame	S
Qwen ImageImage	1024×1024	700ms	S
Qwen Image EditImage	1024×1024	700ms	S
Flux.2 DevImage	1024×1024	~18.7s	S
MAGI-1Video	1280×720	900ms/frame	S
HunyuanImage 3.0Image	256×256	~1.2s	D

Image models estimated at 1024×1024 (28 steps, FP16). Video models estimated at 768×512 (25 frames, 30 steps, FP16). Actual performance varies with runtime and system load.

Multi-GPU scaling

Gaudi 3 128GB — Up to 8× via PCIe

Scale out with multiple GPUs for larger models. PCIe interconnect with 15% scaling overhead.

Config	Effective memory	Models that fit	Est. bandwidth
1× Gaudi	128 GB	351/374	3,700 GB/s
2× Gaudi	256 GB	363/374	6,290 GB/s
4× Gaudi	512 GB	371/374	12,580 GB/s
8× Gaudi	1024 GB	374/374	25,160 GB/s

Model counts use default quantization at coding workload settings. Multi-GPU scaling factor: 0.85× per additional GPU.

Upgrade paths

Upgrade from Gaudi 3 128GB

See what you unlock with more powerful hardware

Upgrade options

👁 Intel
8× Gaudi 3 128GBMulti-GPU

8 × 128 GB = 1024 GB effectivevia PCIe

Unlocks 23 additional models that do not fit on the current setup.Unlocks Qwen 3.5 397B A17B, Kimi K2.5, Kimi K2.6+20 more · +116% faster avg

Unlocks 23 additional models that do not fit on the current setup.

Lifts average decode speed across fitting models by about 116%.

Scale-out only pays off if the host platform has enough PCIe lanes, slot spacing, power, and cooling.

The bigger the setup gets, the more the runtime matters. Multi-GPU and multi-user serving are where vLLM, SGLang, TGI, TensorRT-LLM, or tuned llama.cpp start to earn their complexity.

~$15,000 MSRP

👁 NVIDIA
NVIDIA H200 141GBNext step up

141 GB VRAM (+13)4800 GB/s (+1100)

Unlocks 2 additional models that do not fit on the current setup.Unlocks Qwen 3 235B A22B, MiniMax M2.7+20% faster avg

Unlocks 2 additional models that do not fit on the current setup.

Lifts average decode speed across fitting models by about 20%.

~$30,000 MSRP

AMD Instinct MI325X 256GBBiggest leap

256 GB VRAM (+128)6000 GB/s (+2300)

Unlocks 12 additional models that do not fit on the current setup.Unlocks Qwen 3.5 397B A17B, DeepSeek V4 Flash, Qwen 3 235B A22B+9 more · +23% faster avg

Unlocks 12 additional models that do not fit on the current setup.

Lifts average decode speed across fitting models by about 23%.