VOOZH about

URL: https://willitrunai.com/my-results/instinct-mi350x-288gb

⇱ AMD Instinct MI350X 288GB AI Tier List: 342 models ranked | WillItRunAI


AI Compatibility Tier List

AMD Instinct MI350X 288GB

374 AI models ranked by how well they run on your 288 GB VRAM hardware. Best match: DeepSeek V4 Flash (score: 98).

374

Models tested

342

Can run

288 GB

VRAM

126

Best tok/s

Tier Distribution

S 29
A 55
B 66
C 192
F 32
S: 29A: 55B: 66C: 192F: 32
S

S Tier -- Perfect Fit (29 models)

These models run flawlessly with headroom to spare.

👁 DeepSeek
DeepSeek V4 Flash
S98
284B189.0 GB126 tok/s1049K ctx
moe
👁 Alibaba
Qwen 3.5 397B A17B
S97
397B274.7 GB79 tok/s90K ctx
moe
👁 Alibaba
Qwen 3 235B A22B
S93
235B175.9 GB119 tok/s131K ctx
moe
👁 Mistral
Devstral 2 123B Instruct
S93
123B110.1 GB85 tok/s256K ctx
dense
👁 DeepSeek
DeepSeek Coder V2 236B
S93
236B232.3 GB109 tok/s31K ctx
moe
👁 Alibaba
Qwen 3.5 122B A10B
S92
122B106.6 GB235 tok/s131K ctx
moe
MiniMax M2.7
S91
230B173.8 GB135 tok/s205K ctx
moe
👁 DeepSeek
DeepSeek V2.5 236B
S91
236B232.3 GB109 tok/s31K ctx
moe
👁 Mistral
Mistral Small 4 119B
S90
119B107.7 GB255 tok/s256K ctx
moe
👁 OpenAI
GPT-OSS 120B
S90
117B106.0 GB89 tok/s131K ctx
dense
👁 Cohere
Command A 111B
S90
111B101.3 GB94 tok/s262K ctx
dense
👁 Mistral AI
Pixtral Large 124B
S89
124B110.7 GB84 tok/s131K ctx
dense
👁 Alibaba
Qwen3-Coder 30B A3B Instruct
S89
30.5B49.8 GB883 tok/s256K ctx
moe
👁 Alibaba
Qwen 3.6 35B A3B
S88
35B55.2 GB742 tok/s262K ctx
+1moe
👁 Alibaba
Qwen 2.5 VL 72B
S88
72B78.5 GB145 tok/s33K ctx
dense
👁 Alibaba
Qwen 3.5 27B
S88
27B49.3 GB378 tok/s131K ctx
dense
👁 Alibaba
Qwen 3.6 27B
S88
27B47.1 GB239 tok/s262K ctx
+1dense
👁 Alibaba
Qwen3-VL 30B A3B Instruct
S88
30B49.5 GB913 tok/s256K ctx
moe
👁 Meta
Llama 4 Maverick 17B 128E
S88
400B276.6 GB77 tok/s78K ctx
moe
👁 Alibaba
Qwen3-Coder-Next
S88
80B80.0 GB394 tok/s256K ctx
moe
👁 Alibaba
Qwen 3.5 35B A3B
S87
35B52.5 GB807 tok/s131K ctx
moe
👁 Mistral
Magistral Small 2507
S87
24B46.8 GB336 tok/s131K ctx
dense
👁 Alibaba
Qwen 3 32B
S87
32B53.1 GB325 tok/s131K ctx
dense
👁 Mistral
Devstral Small 2 24B Instruct
S87
24B46.8 GB336 tok/s256K ctx
dense
👁 Alibaba
Qwen 3.5 9B
S86
9B37.4 GB126 tok/s131K ctx
dense
👁 Alibaba
Qwen 3 30B A3B
S86
30.5B49.8 GB883 tok/s131K ctx
moe
👁 Mistral
Leanstral 119B A6B
S86
119B111.1 GB234 tok/s256K ctx
moe
👁 NVIDIA
Nemotron 3 Nano 30B
S86
30B50.4 GB343 tok/s131K ctx
dense
👁 Alibaba
Qwen 3 14B
S86
14B40.7 GB196 tok/s131K ctx
dense
A

A Tier -- Great Fit (55 models)

Excellent performance with comfortable memory margins.

👁 Mistral
Devstral Small 1.1
A85
24B46.8 GB336 tok/s131K ctx
dense
👁 Microsoft
Phi-4-reasoning-plus 14B
A85
14.7B41.7 GB206 tok/s33K ctx
dense
👁 Alibaba
Qwen 3 8B
A85
8B36.8 GB112 tok/s131K ctx
dense
👁 OpenAI
GPT-OSS 20B
A84
21B45.0 GB1121 tok/s128K ctx
moe
👁 Meta
Llama 3.1 405B
A84
405B284.4 GB26 tok/s23K ctx
dense
👁 Google
Gemma 4 31B
A83
30.7B63.1 GB204 tok/s256K ctx
dense
👁 NVIDIA
Nemotron Cascade 2 30B A3B
A83
30B50.9 GB903 tok/s262K ctx
moe
👁 Alibaba
Qwen 3.5 4B
A83
4B34.3 GB56 tok/s131K ctx
dense
👁 Meta
Llama 3.3 70B
A82
70B77.3 GB149 tok/s128K ctx
dense
👁 LG AI
EXAONE 4.0 32B
A81
32B53.1 GB323 tok/s131K ctx
dense
👁 Google
Gemma 4 26B A4B
A81
25.2B48.7 GB948 tok/s256K ctx
moe
👁 Mistral
Ministral 3 14B
A80
14B40.7 GB196 tok/s262K ctx
multimodal
👁 Microsoft
Phi-4 Mini Reasoning 4B
A80
3.8B33.5 GB53 tok/s131K ctx
dense
👁 NVIDIA
Nemotron Nano 8B
A80
8B36.5 GB112 tok/s131K ctx
dense
👁 Mistral
Codestral 2 25.08
A80
22B45.6 GB308 tok/s256K ctx
dense
👁 Alibaba
Qwen 2.5 32B
A79
32B53.1 GB323 tok/s131K ctx
dense
👁 Mistral
Mistral Small 3.2 24B
A79
24B46.8 GB336 tok/s131K ctx
vision
👁 Meta
Llama 3.1 70B
A79
70B77.3 GB149 tok/s128K ctx
dense
👁 Alibaba
Qwen 2.5 72B
A79
72B78.5 GB145 tok/s131K ctx
dense
👁 Moonshot AI
Kimi Linear 48B A3B
A79
48B59.9 GB200 tok/s1049K ctx
linear
👁 Google
Gemma 3 27B
A79
27B57.4 GB231 tok/s131K ctx
dense
A79
24B46.8 GB336 tok/s131K ctx
dense
👁 Alibaba
Qwen3-Coder 480B A35B Instruct
A79
480B325.4 GB35 tok/s4K ctx
moe
A78
19B43.7 GB266 tok/s8K ctx
dense
👁 Allen AI
OLMo 2 32B
A78
32B53.1 GB323 tok/s4K ctx
dense
👁 IBM
Granite 4.1 30B
A78
30B51.9 GB343 tok/s131K ctx
dense
👁 Mistral
Mistral Small 24B
A77
24B46.8 GB336 tok/s33K ctx
dense
👁 Meta
Llama 4 Scout 17B 16E
A77
109B99.1 GB223 tok/s1048K ctx
moe
👁 InternLM
InternVL2 8B
A77
8B36.5 GB112 tok/s8K ctx
dense
👁 Mistral AI
Mistral Small 3.1 24B
A77
24B46.8 GB336 tok/s131K ctx
dense
A77
14B41.3 GB196 tok/s16K ctx
dense
👁 Alibaba
Qwen 2.5 14B
A76
14B41.2 GB196 tok/s131K ctx
dense
👁 Mistral
Ministral 3 8B
A75
8B36.8 GB112 tok/s262K ctx
multimodal
👁 OpenBMB
MiniCPM-V 2.6 8B
A75
8B36.5 GB112 tok/s2K ctx
dense
👁 IBM
Granite Code 20B
A75
20B45.1 GB280 tok/s8K ctx
dense
👁 Google
Gemma 3 12B
A75
12B41.9 GB168 tok/s131K ctx
dense
👁 Alibaba
Qwen 3 4B
A74
4B34.3 GB56 tok/s33K ctx
dense
👁 Defog
SQLCoder 7B
A74
7B35.9 GB98 tok/s8K ctx
dense
👁 NVIDIA
Nemotron Nano 9B v2
A74
9B37.6 GB126 tok/s131K ctx
dense
👁 Alibaba
Qwen 2.5 VL 7B
A74
7B34.8 GB98 tok/s33K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 70B
A74
70B77.3 GB149 tok/s131K ctx
dense
👁 DeepSeek
DeepSeek Coder V2 16B
A74
16B42.8 GB1425 tok/s131K ctx
moe
👁 Alibaba
Qwen 2.5 Coder 32B
A74
32B53.1 GB323 tok/s131K ctx
dense
👁 Jina AI
Jina Embeddings v3
A73
0.57B32.8 GB8 tok/s8K ctx
dense
👁 IBM
Granite Code 34B
A73
34B54.1 GB305 tok/s8K ctx
dense
👁 BAAI
BGE M3
A72
0.57B32.0 GB8 tok/s8K ctx
dense
👁 Magistral AI
Magistral 7B
A72
7B35.9 GB98 tok/s8K ctx
dense
👁 Tsinghua/Zhipu
CodeGeeX 4 9B
A72
9B35.8 GB126 tok/s131K ctx
dense
👁 Cohere
Command R 35B
A72
35B53.5 GB298 tok/s131K ctx
dense
👁 Google
Gemma 4 E4B
A72
8B35.9 GB112 tok/s128K ctx
dense
👁 AllenAI
OLMo 2 13B
A72
13B40.1 GB182 tok/s33K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 32B
A71
32B53.1 GB323 tok/s33K ctx
dense
👁 BigCode
StarCoder 15B
A71
15B55.1 GB210 tok/s8K ctx
dense
👁 Meta
CodeLlama 13B Instruct
A71
13B49.8 GB182 tok/s16K ctx
dense
👁 Alibaba
Qwen 2.5 7B
A70
7B34.8 GB98 tok/s131K ctx
dense
B

B Tier -- Good Fit (66 models)

Solid performance, may need quantization or reduced context.

👁 IBM
Granite Code 8B
B70
8B36.5 GB112 tok/s8K ctx
dense
👁 DevStral AI
DevStral 7B
B70
7B35.9 GB98 tok/s8K ctx
dense
👁 Mistral AI
Codestral Mamba 7B
B70
7B34.5 GB98 tok/s262K ctx
state-space
👁 DeepSeek
DeepSeek R1 Distill 14B
B69
14B41.2 GB196 tok/s33K ctx
dense
👁 NVIDIA
Nemotron 70B
B69
70B77.3 GB149 tok/s131K ctx
dense
👁 Mixedbread AI
mxbai Embed Large
B69
0.34B31.9 GB5 tok/s1K ctx
dense
👁 LLaVA
LLaVA 1.6 13B
B69
13B49.8 GB182 tok/s4K ctx
dense
👁 IBM
Granite 4.1 8B
B69
8B37.0 GB112 tok/s131K ctx
dense
👁 Mistral AI
Pixtral 12B
B68
12B39.5 GB168 tok/s131K ctx
dense
👁 Meta
CodeLlama 7B Instruct
B68
7B41.8 GB98 tok/s16K ctx
dense
👁 BigCode
StarCoder 7B
B68
7B41.3 GB98 tok/s8K ctx
dense
👁 Snowflake
Snowflake Arctic Embed L
B68
0.34B31.9 GB5 tok/s1K ctx
dense
👁 MosaicML
MPT-30B-Instruct
B68
30B74.7 GB276 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 3B
B68
3B33.7 GB42 tok/s131K ctx
dense
👁 TII
Falcon 40B Instruct
B67
40B60.3 GB225 tok/s8K ctx
dense
👁 BAAI
BGE Large EN v1.5
B66
0.34B31.9 GB5 tok/s1K ctx
dense
👁 Google
Gemma 4 E2B
B66
5.1B33.3 GB71 tok/s128K ctx
dense
👁 Nous Research
Nous Dolphin 13B
B66
13B51.3 GB182 tok/s16K ctx
dense
👁 Cohere
Command R+ 104B
B66
104B96.6 GB100 tok/s131K ctx
dense
👁 WizardLM
WizardLM 13B
B66
13B49.8 GB182 tok/s8K ctx
dense
👁 LMSYS
Vicuna 13B
B66
13B49.8 GB182 tok/s4K ctx
dense
👁 Nous Research
Nous Hermes 1.0
B66
9B47.4 GB126 tok/s16K ctx
dense
👁 Zhipu
GLM-4 9B
B66
9B35.8 GB126 tok/s128K ctx
dense
👁 Google
Gemma 2 27B
B66
27B57.4 GB231 tok/s8K ctx
dense
👁 Meta
Llama 3.1 8B
B66
8B36.5 GB112 tok/s128K ctx
dense
👁 InternLM
InternLM 7B
B66
7B41.8 GB98 tok/s8K ctx
dense
👁 InternLM
InternLM Chat 7B
B66
7B41.8 GB98 tok/s8K ctx
dense
B65
7B36.9 GB98 tok/s8K ctx
dense
👁 WizardLM
WizardMath 7B
B65
7B35.9 GB98 tok/s4K ctx
dense
👁 Allen AI
OLMo 2 7B
B65
7B35.9 GB98 tok/s4K ctx
dense
👁 Nomic AI
Nomic Embed Text v1.5
B65
0.14B30.5 GB2 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 7B
B64
7B34.8 GB98 tok/s131K ctx
dense
👁 Mistral
Mixtral 8x22B
B64
141B119.1 GB141 tok/s66K ctx
moe
👁 Mistral
Ministral 3 3B
B64
3B32.3 GB42 tok/s262K ctx
multimodal
👁 Google
Gemma 3 4B
B64
4B34.2 GB56 tok/s128K ctx
dense
👁 Microsoft
Phi 4 Mini 4B
B63
4B33.6 GB56 tok/s128K ctx
dense
👁 LLaVA
LLaVA 1.5 7B
B63
7B41.8 GB98 tok/s4K ctx
dense
👁 Alibaba
Qwen 3.5 2B
B63
2B32.6 GB28 tok/s131K ctx
dense
👁 Baichuan
Baichuan 13B
B62
13B51.3 GB182 tok/s8K ctx
dense
👁 Mistral
Mixtral 8x7B
B62
47B60.3 GB420 tok/s33K ctx
moe
👁 Cerebras
Cerebras-GPT 13B
B62
13B48.8 GB182 tok/s131K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 7B
B61
7B34.8 GB98 tok/s33K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 8B
B61
8B36.5 GB112 tok/s33K ctx
dense
👁 MosaicML
MPT-7B-Instruct
B61
7B41.8 GB98 tok/s8K ctx
dense
👁 Baichuan
Baichuan 7B
B61
7B41.8 GB98 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 Math 72B
B61
72B78.5 GB145 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 3B
B61
3B33.7 GB42 tok/s131K ctx
dense
7B35.9 GB98 tok/s4K ctx
dense
👁 TII
Falcon 7B Instruct
B61
7B34.1 GB98 tok/s8K ctx
dense
👁 Microsoft
Phi 3 Mini 3.8B
B60
3.8B37.9 GB53 tok/s128K ctx
dense
👁 Alibaba
Qwen 3 1.7B
B60
1.7B32.4 GB24 tok/s33K ctx
dense
B60
0.02B30.0 GB2 tok/s0K ctx
dense
👁 Google
Gemma 2 9B
B59
9B40.3 GB126 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 14B
B59
14B41.2 GB196 tok/s131K ctx
dense
👁 Meta
Llama 3.2 11B Vision
B59
11B38.4 GB154 tok/s16K ctx
vision
👁 IBM
Granite 4.1 3B
B59
3B32.8 GB42 tok/s131K ctx
dense
👁 Microsoft
Phi 3.5 Mini 4B
B59
4B38.0 GB56 tok/s128K ctx
dense
👁 01.AI
Yi 1.5 34B
B59
34B54.1 GB306 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 1.5B
B58
1.5B31.0 GB21 tok/s33K ctx
dense
👁 IBM
Granite Code 3B
B58
3B34.0 GB42 tok/s8K ctx
dense
👁 DeepSeek
DeepSeek LLM 67B
B57
67B76.4 GB155 tok/s4K ctx
dense
👁 Mistral
Mistral Nemo 12B
B57
12B39.5 GB168 tok/s128K ctx
dense
👁 01.AI
Yi Coder 9B
B57
9B36.7 GB126 tok/s131K ctx
dense
👁 Mistral
Mistral 7B Instruct v0.3
B57
7B35.9 GB98 tok/s8K ctx
dense
👁 Microsoft
Phi 3 Medium 14B
B56
14B41.3 GB196 tok/s128K ctx
dense
👁 Mistral AI
Codestral 22B
B55
22B45.6 GB308 tok/s33K ctx
dense
C

C Tier -- Tight Fit (192 models)

Runs with offloading or significant compromises.

👁 InternLM
InternLM 20B
C54
20B62.4 GB280 tok/s8K ctx
dense
👁 Meta
Llama 3.2 3B
C54
3B33.2 GB42 tok/s128K ctx
dense
👁 Mistral
Ministral 8B
C54
8B36.8 GB112 tok/s131K ctx
dense
K EXAONE 236B A23B
C54
236B201.3 GB41 tok/s66K ctx
dense
Baichuan M3 235B
C54
235B200.6 GB41 tok/s67K ctx
dense
Baichuan M3 235B i1
C54
235B200.6 GB41 tok/s67K ctx
dense
👁 Cohere
Aya Expanse 32B
C51
32B51.7 GB325 tok/s8K ctx
dense
👁 DeepSeek
DeepSeek R1 1.5B
C50
1.5B31.0 GB21 tok/s33K ctx
dense
👁 TinyLlama
TinyLlama 1.1B
C50
1.1B30.7 GB15 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 122B A10B
C50
122B103.8 GB91 tok/s222K ctx
dense
C50
3B33.5 GB42 tok/s128K ctx
dense
Solar Open 100B
C49
100B102.4 GB96 tok/s269K ctx
dense
Solar Open 100B i1
C49
100B102.4 GB96 tok/s269K ctx
dense
👁 IBM
Granite 3.1 8B
C49
8B36.5 GB112 tok/s128K ctx
state-space
👁 01.AI
Yi 1.5 9B
C49
9B36.7 GB126 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 Math 7B
C49
7B34.8 GB98 tok/s4K ctx
dense
Llama 3.3 70B Instruct
C48
70B80.6 GB137 tok/s421K ctx
dense
👁 Alibaba
Qwen 2.5 1.5B
C48
1.5B31.0 GB21 tok/s131K ctx
dense
Solar Open 69B REAP i1
C48
69B79.9 GB139 tok/s428K ctx
dense
stabilityai japanese stablelm instruct beta 70b
C47
70B80.6 GB137 tok/s421K ctx
dense
BaichuanMed OCR 72B i1
C47
72B82.1 GB133 tok/s407K ctx
dense
👁 01.AI
Yi 34B Chat
C47
34B54.1 GB306 tok/s200K ctx
dense
👁 OpenChat
OpenChat 7B
C47
7B35.9 GB98 tok/s8K ctx
dense
Qwen3 48B A4B Savant Commander Distill 12X Closed Open Heretic Uncensored
C47
48B64.6 GB200 tok/s651K ctx
dense
👁 Unsloth
Qwen3.5 35B A3B
C46
35B55.2 GB274 tok/s924K ctx
dense
👁 BigCode
StarCoder2 15B
C46
15B41.7 GB210 tok/s16K ctx
dense
Qwen3.5 35B A3B
C46
35B55.2 GB274 tok/s924K ctx
dense
👁 Cohere
Aya Expanse 8B
C46
8B36.5 GB112 tok/s8K ctx
dense
👁 Google
Gemma 3 1B
C46
1B30.7 GB14 tok/s33K ctx
dense
👁 Unsloth
Qwen3.5 27B
C46
27B49.3 GB355 tok/s1223K ctx
dense
Hermes 4.3 36B
C46
36B55.9 GB266 tok/s896K ctx
dense
👁 Unsloth
gemma 3 27b it
C46
27B49.3 GB355 tok/s1223K ctx
dense
gemma 3 27b it
C46
27B49.3 GB355 tok/s1223K ctx
dense
EXAONE 4.0 32B
C46
32B53.0 GB299 tok/s1019K ctx
dense
aya expanse 32b heretic MPOA i1
C46
32B53.0 GB299 tok/s1019K ctx
dense
👁 LMSYS
Vicuna 7B
C46
7B41.8 GB98 tok/s4K ctx
dense
👁 Unsloth
Mistral Small 3.2 24B Instruct 2506
C46
24B47.2 GB336 tok/s1386K ctx
dense
Baichuan M2 32B Q4 K M
C45
32B53.0 GB299 tok/s1019K ctx
dense
cognitivecomputations Dolphin Mistral 24B Venice Edition
C45
24B47.2 GB336 tok/s1386K ctx
dense
Mistral Small 24B Instruct 2501
C45
24B47.2 GB336 tok/s1386K ctx
dense
baichuan inc Baichuan M2 32B
C45
32B53.0 GB299 tok/s1019K ctx
dense
👁 Stability AI
StableLM 2 12B
C45
12B50.5 GB168 tok/s4K ctx
dense
mistral small 3.1 24b instruct 2503 hf
C45
24B47.2 GB336 tok/s1386K ctx
dense
cognitivecomputations Dolphin3.0 R1 Mistral 24B
C45
24B47.2 GB336 tok/s1386K ctx
dense
👁 Teknium
OpenHermes 2.5 7B
C45
7B35.9 GB98 tok/s8K ctx
dense
Dolphin Mistral GLM 4.7 Flash 24B Venice Edition Thinking Uncensored i1
C45
24B47.2 GB336 tok/s1386K ctx
dense
8B36.5 GB112 tok/s33K ctx
dense
Codestral 22B v0.1
C45
22B45.7 GB308 tok/s1520K ctx
dense
Codestral 22B v0.1
C45
22B45.7 GB308 tok/s1520K ctx
dense
👁 Google
Gemma 2 2B
C45
2B32.5 GB28 tok/s8K ctx
dense
👁 Hugging Face H4
Zephyr 7B Beta
C45
7B35.9 GB98 tok/s33K ctx
dense
Codestral 22B v0.1
C45
22B45.7 GB308 tok/s1520K ctx
dense
cognitivecomputations Dolphin Mistral 24B Venice Edition
C45
24B47.2 GB336 tok/s1386K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Qwen 14B
C45
14B39.9 GB196 tok/s2436K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 0.5B
C45
0.5B30.2 GB7 tok/s131K ctx
dense
👁 Unsloth
Qwen3.5 9B
C45
9B36.2 GB126 tok/s3835K ctx
dense
Qwen3.5 9B Uncensored HauhauCS Aggressive
C45
9B36.2 GB126 tok/s3835K ctx
dense
Codestral 22B v0.1 IMat
C45
22B45.7 GB308 tok/s1520K ctx
dense
Meta Llama 3.1 8B Instruct
C45
8B35.5 GB112 tok/s4325K ctx
dense
gemma 3 12b it
C45
12B38.4 GB168 tok/s2856K ctx
dense
👁 UC Berkeley
Starling LM 7B
C45
7B35.9 GB98 tok/s8K ctx
dense
llava llama 3 8b v1 1
C45
8B35.5 GB112 tok/s4325K ctx
dense
Phi 4 reasoning vision 15B
C45
15B40.6 GB210 tok/s2268K ctx
dense
Qwen3.5 9B
C45
9B36.2 GB126 tok/s3835K ctx
dense
StarCoder2 15B
C45
15B40.6 GB210 tok/s2268K ctx
dense
Llama 2 7B Chat
C45
7B34.8 GB98 tok/s4955K ctx
dense
👁 Unsloth
DeepSeek R1 0528 Qwen3 8B
C45
8B35.5 GB112 tok/s4325K ctx
dense
internlm2 5 20b chat
C45
20B44.2 GB280 tok/s1680K ctx
dense
Codestral 22B v0.1 i1
C45
22B45.7 GB308 tok/s1520K ctx
dense
Codestral 21B Pruned i1
C45
21B45.0 GB294 tok/s1596K ctx
dense
internlm2 limarp chat 20b
C45
20B44.2 GB280 tok/s1680K ctx
dense
Mistral 7B Instruct v0.2
C45
7B34.8 GB98 tok/s4955K ctx
dense
NousResearch Hermes 4 14B
C45
14B39.9 GB196 tok/s2436K ctx
dense
Meta Llama 3 8B Instruct
C45
8B35.5 GB112 tok/s4325K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Llama 8B
C45
8B35.5 GB112 tok/s4325K ctx
dense
Dolphin3.0 Llama3.1 8B
C45
8B35.5 GB112 tok/s4325K ctx
dense
Llama 3 8B Instruct 32k v0.1
C45
8B35.5 GB112 tok/s4325K ctx
dense
Mistral 7B Instruct v0.3
C45
7B34.8 GB98 tok/s4955K ctx
dense
Meta Llama 3.1 8B Instruct
C45
8B35.5 GB112 tok/s4325K ctx
dense
vntl llama3 8b v2
C45
8B35.5 GB112 tok/s4325K ctx
dense
internlm2 math plus 20b i1
C45
20B44.2 GB280 tok/s1680K ctx
dense
DeepSeek R1 0528 Qwen3 8B
C45
8B35.5 GB112 tok/s4325K ctx
dense
Yi Coder 9B Chat
C45
9B36.2 GB126 tok/s3835K ctx
dense
glm 4 9b chat 1m
C45
9B36.2 GB126 tok/s3835K ctx
dense
SOLAR 10.7B Instruct v1.0 uncensored
C45
10.7B37.5 GB150 tok/s3213K ctx
dense
DeepSeek R1 0528 Qwen3 8B
C45
8B35.5 GB112 tok/s4325K ctx
dense
Nous Hermes 2 SOLAR 10.7B
C45
10.7B37.5 GB150 tok/s3213K ctx
dense
starcoder2 15b instruct v0.1
C45
15B40.6 GB210 tok/s2268K ctx
dense
Codestral RAG 19B Pruned i1
C45
19B43.5 GB266 tok/s1773K ctx
dense
starcoder2 15b i1
C45
15B40.6 GB210 tok/s2268K ctx
dense
Qwen3 8B DeepSeek v3.2 Speciale Distill
C45
8B35.5 GB112 tok/s4325K ctx
dense
Hermes 3 Llama 3.1 8B
C45
8B35.5 GB112 tok/s4325K ctx
dense
starcoder2 15b instruct v0.1
C45
15B40.6 GB210 tok/s2268K ctx
dense
GGUF SOLARized GraniStral 14B 2102 YeAM HCT 32QKV
C44
14B39.9 GB196 tok/s2436K ctx
dense
Mistral 7B Instruct v0.3
C44
7B34.8 GB98 tok/s4955K ctx
dense
Yi 1.5 9B Chat
C44
9B36.2 GB126 tok/s3835K ctx
dense
internlm JanusCoder 14B
C44
14B39.9 GB196 tok/s2436K ctx
dense
Hermes 2 Pro Llama 3 8B
C44
8B35.5 GB112 tok/s4325K ctx
dense
Hermes 2 Pro Mistral 7B
C44
7B34.8 GB98 tok/s4955K ctx
dense
Dolphin3.0 Llama3.1 8B
C44
8B35.5 GB112 tok/s4325K ctx
dense
GGUF SOLARized GraniStral 14B 1902 YeAM HCT
C44
14B39.9 GB196 tok/s2436K ctx
dense
HelpingAI 15B i1
C44
15B40.6 GB210 tok/s2268K ctx
dense
granite 8b code instruct 4k
C44
8B35.5 GB112 tok/s4325K ctx
dense
zephyr 7B beta
C44
7B34.8 GB98 tok/s4955K ctx
dense
dolphin 2.9.4 llama3.1 8b
C44
8B35.5 GB112 tok/s4325K ctx
dense
Nous Hermes 2 Mistral 7B DPO
C44
7B34.8 GB98 tok/s4955K ctx
dense
Falcon H1R 7B
C44
7B34.8 GB98 tok/s4955K ctx
dense
openchat 3.6 8b 20240522 IMat
C44
8B35.5 GB112 tok/s4325K ctx
dense
falcon mamba 7b instruct Q4 K M
C44
7B34.8 GB98 tok/s4955K ctx
dense
solar finalised finetuned Model 10.7B i1
C44
10.7B37.5 GB150 tok/s3213K ctx
dense
EXAONE 3.5 7.8B Instruct
C44
7.8B35.4 GB109 tok/s4438K ctx
dense
aya expanse 8b
C44
8B35.5 GB112 tok/s4325K ctx
dense
zephyr 7B alpha
C44
7B34.8 GB98 tok/s4955K ctx
dense
speechless zephyr code functionary 7b
C44
7B34.8 GB98 tok/s4955K ctx
dense
dolphin v2 8b abliterated i1
C44
8B35.5 GB112 tok/s4325K ctx
dense
EXAONE 3.5 7.8B Instruct
C44
7.8B35.4 GB109 tok/s4438K ctx
dense
StarCoder2 7B
C44
7B34.8 GB98 tok/s4955K ctx
dense
SOLAR 10.7B v1.0
C44
10.7B37.5 GB150 tok/s3213K ctx
dense
EXAONE 3.5 7.8B Instruct i1
C44
7.8B35.4 GB109 tok/s4438K ctx
dense
japanese stablelm instruct gamma 7B
C44
7B34.8 GB98 tok/s4955K ctx
dense
Yi 1.5 6B Chat
C44
6B34.1 GB84 tok/s5794K ctx
dense
Yi 9B Coder i1
C44
9B36.2 GB126 tok/s3835K ctx
dense
aya expanse 8b orthogonal heretic i1
C44
8B35.5 GB112 tok/s4325K ctx
dense
exaone 3.0 7.8b it
C44
7.8B35.4 GB109 tok/s4438K ctx
dense
Falcon H1 7B Instruct
C44
7B34.8 GB98 tok/s4955K ctx
dense
👁 Unsloth
Falcon H1R 7B
C44
7B34.8 GB98 tok/s4955K ctx
dense
HelpingAI2 9B
C44
9B36.2 GB126 tok/s3835K ctx
dense
stablelm 2 zephyr 1 6b
C44
6B34.1 GB84 tok/s5794K ctx
dense
aya expanse 8b orthogonal heretic
C44
8B35.5 GB112 tok/s4325K ctx
dense
HelpingAI2.5 10B i1
C44
10B37.0 GB140 tok/s3443K ctx
dense
Mamba Codestral 7B v0.1
C44
7B34.8 GB98 tok/s4955K ctx
dense
stabilityai japanese stablelm base gamma 7b
C44
7B34.8 GB98 tok/s4955K ctx
dense
Helply 10.2b chat i1
C44
10.2B37.1 GB143 tok/s3374K ctx
dense
HelpingAI2 9B i1
C44
9B36.2 GB126 tok/s3835K ctx
dense
HelpingAI 9B 200k i1
C44
9B36.2 GB126 tok/s3835K ctx
dense
baichuan2 7b chat
C44
7B34.8 GB98 tok/s4955K ctx
dense
DiscoPOP zephyr 7b gemma
C44
7B34.8 GB98 tok/s4955K ctx
dense
HelpingAI 9B i1
C44
9B36.2 GB126 tok/s3835K ctx
dense
internlm2 5 1 8b chat i1
C44
8B35.5 GB112 tok/s4325K ctx
dense
starcoder2 7b
C44
7B34.8 GB98 tok/s4955K ctx
dense
internlm3 8b instruct abliterated i1
C44
8B35.5 GB112 tok/s4325K ctx
dense
zephyr 7b beta Mistral 7B Instruct v0.2
C44
7B34.8 GB98 tok/s4955K ctx
dense
OpenChat 3.5 7B Qwen v2.0 i1
C44
7B34.8 GB98 tok/s4955K ctx
dense
OpenChat 3.5 7B Starling v2.0 i1
C44
7B34.8 GB98 tok/s4955K ctx
dense
internlm2 math plus 7b IMat
C44
7B34.8 GB98 tok/s4955K ctx
dense
CodeNinja 1.0 OpenChat 7B i1
C44
7B34.8 GB98 tok/s4955K ctx
dense
Yi 1.5 6B Chat
C44
6B34.1 GB84 tok/s5794K ctx
dense
internlm2 5 7b chat i1
C44
7B34.8 GB98 tok/s4955K ctx
dense
👁 01.AI
Yi 1.5 6B
C44
6B34.3 GB84 tok/s4K ctx
dense
stablelm 2 1 6b chat imatrix
C44
6B34.1 GB84 tok/s5794K ctx
dense
OpenSafetyLab MD Judge v0 2 internlm2 7b
C44
7B34.8 GB98 tok/s4955K ctx
dense
MD Judge v0 2 internlm2 7b i1
C44
7B34.8 GB98 tok/s4955K ctx
dense
zephyr 7b gemma sft african ultrachat 100k
C44
7B34.8 GB98 tok/s4955K ctx
dense
jointpreferences mistral 7b sft helpful
C44
7B34.8 GB98 tok/s4955K ctx
dense
zephyr 7b dpo full i1
C44
7B34.8 GB98 tok/s4955K ctx
dense
blossom v3 baichuan2 7b i1
C44
7B34.8 GB98 tok/s4955K ctx
dense
blossom v1 baichuan 7b i1
C44
7B34.8 GB98 tok/s4955K ctx
dense
👁 Intel
Neural Chat 7B
C44
7B35.9 GB98 tok/s8K ctx
dense
HelpingAI2 6B
C44
6B34.1 GB84 tok/s5794K ctx
dense
👁 BigCode
StarCoder2 7B
C44
7B34.5 GB98 tok/s16K ctx
dense
HelpingAI2 6B i1
C44
6B34.1 GB84 tok/s5794K ctx
dense
👁 NVIDIA
Nemotron Mini 4B
C44
4B34.1 GB56 tok/s4K ctx
dense
👁 DeepSeek
DeepSeek LLM 7B
C43
7B41.3 GB98 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 4B
C43
4B32.6 GB56 tok/s8733K ctx
dense
HelpingAI2.5 5B i1
C43
5B33.3 GB70 tok/s6970K ctx
dense
gemma 3 4b it
C43
4B32.6 GB56 tok/s8733K ctx
dense
👁 Alibaba
Qwen 3 0.6B
C43
0.6B30.9 GB8 tok/s33K ctx
dense
gemma 3 4b it
C43
4B32.6 GB56 tok/s8733K ctx
dense
👁 Meta
Llama 3.2 1B
C43
1B30.8 GB14 tok/s128K ctx
dense
Llama 3.2 3B Instruct
C42
3B32.2 GB42 tok/s11657K ctx
dense
👁 Qwen
Qwen2.5 3B Instruct
C42
3B31.9 GB42 tok/s11672K ctx
dense
Llama 3.2 3B Instruct
C42
3B31.9 GB42 tok/s11672K ctx
dense
Ministral 3 3B Instruct 2512
C42
3B31.9 GB42 tok/s11672K ctx
dense
HELVETE 3B
C42
3B31.9 GB42 tok/s11672K ctx
dense
Hermes 3 Llama 3.2 3B
C42
3B31.9 GB42 tok/s11672K ctx
dense
stablelm zephyr 3b
C42
3B31.9 GB42 tok/s11672K ctx
dense
StarCoder2 3B
C42
3B31.9 GB42 tok/s11672K ctx
dense
AI21 Jamba Reasoning 3B
C41
3B31.9 GB42 tok/s11672K ctx
dense
stablelm 3b 4e1t
C41
3B31.9 GB42 tok/s11672K ctx
dense
ai21labs AI21 Jamba Reasoning 3B
C41
3B31.9 GB42 tok/s11672K ctx
dense
ai21labs AI21 Jamba2 3B
C41
3B31.9 GB42 tok/s11672K ctx
dense
HelpingAI 3B hindi i1
C41
3B31.9 GB42 tok/s11672K ctx
dense
AI21 Jamba2 3B
C41
3B31.9 GB42 tok/s11672K ctx
dense
HelpingAI 3B hindi
C41
3B31.9 GB42 tok/s11672K ctx
dense
AI21 Jamba2 3B i1
C41
3B31.9 GB42 tok/s11672K ctx
dense
EXAONE 3.5 2.4B Instruct
C41
2.4B31.4 GB34 tok/s14611K ctx
dense
gemma 2 2b it
C41
2B31.6 GB28 tok/s17521K ctx
dense
👁 Google
gemma 2b
C41
2B31.2 GB28 tok/s17550K ctx
dense
Gemmasutra Mini 2B v1
C41
2B31.2 GB28 tok/s17550K ctx
dense
👁 Alibaba
Qwen 3.5 0.6B
C40
0.6B30.9 GB8 tok/s131K ctx
dense
gemma 2 2b it
C40
2B31.2 GB28 tok/s17550K ctx
dense
👁 BigCode
StarCoder2 3B
C40
3B32.0 GB42 tok/s16K ctx
dense
F

F Tier -- Can't Run (32 models)

Too heavy for this hardware.

👁 Qwen
Qwen2.5 1.5B Instruct
D40
1.5B30.8 GB21 tok/s23428K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Qwen 1.5B
D40
1.5B30.8 GB21 tok/s23428K ctx
dense
Yi Coder 1.5B Chat
D40
1.5B30.8 GB21 tok/s23428K ctx
dense
stablelm 2 zephyr 1.6b
D39
1.6B30.9 GB22 tok/s21958K ctx
dense
logos16v2 stablelm2 1.6b i1
D39
1.6B30.9 GB22 tok/s21958K ctx
dense
Yi Coder 1.5B
D39
1.5B30.8 GB21 tok/s23428K ctx
dense
👁 Unsloth
Falcon H1 1.5B Instruct
D39
1.5B30.8 GB21 tok/s23428K ctx
dense
TinyLlama 1.1B Chat v1.0
D39
1.1B30.5 GB15 tok/s31977K ctx
dense
👁 Alibaba
Qwen 2.5 0.5B
D39
0.5B30.2 GB7 tok/s131K ctx
dense
EXAONE 4.0 1.2B
D39
1.2B30.6 GB17 tok/s29306K ctx
dense
Llama 3.2 1B Instruct Q8 0
D38
1B30.6 GB14 tok/s35155K ctx
dense
TinyLlama 1.1B Chat v0.3
D38
1.1B30.5 GB15 tok/s31977K ctx
dense
👁 TinyLlama
TinyLlama 1.1B Chat v0.6
D38
1.1B30.5 GB15 tok/s31977K ctx
dense
👁 Unsloth
Qwen3.5 397B A17B
D38
397B318.4 GB15 tok/s6K ctx
dense
Llama 3.2 1B Instruct
D38
1B30.4 GB14 tok/s35183K ctx
dense
gemma 3 1b it
D38
1B30.4 GB14 tok/s35183K ctx
dense
TinyLlama 1.1B Chat v1.0 imatrix
D38
1.1B30.5 GB15 tok/s31977K ctx
dense
Falcon3 1B Instruct abliterated
D38
1B30.4 GB14 tok/s35183K ctx
dense
SmolVLM 500M Instruct
D36
0.5B30.2 GB7 tok/s41262K ctx
dense
embeddinggemma 300M
D35
0.3B30.0 GB4 tok/s41289K ctx
dense
granite embedding 107m multilingual
D33
0.11B29.9 GB2 tok/s41318K ctx
dense
Falcon H1 Tiny 90M Instruct
D32
0.09B29.9 GB2 tok/s41319K ctx
dense
👁 Mistral
Mistral Large 3
F0
675B446.8 GB13 tok/s4K ctx
+1moe
F0
744B502.6 GB9 tok/s4K ctx
moe
👁 Z.ai
GLM-5.1
F0
754B508.7 GB8 tok/s4K ctx
moe
F0
1000B647.1 GB5 tok/s4K ctx
moe
F0
1000B647.1 GB5 tok/s4K ctx
+1moe
👁 DeepSeek
DeepSeek V3 671B
F0
671B498.6 GB10 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V3.1 671B
F0
671B498.6 GB10 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V4 Pro
F0
1600B893.6 GB4 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek R1 671B
F0
671B498.6 GB10 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V3.2
F0
671B439.5 GB14 tok/s4K ctx
moe

Compare with...

See How Other Hardware Stacks Up

Check tier lists for similar GPUs and Apple Silicon configs

AMD Instinct MI325X 256GB

256 GB VRAM

View tier list

AMD Instinct MI300X 192GB

192 GB VRAM

View tier list

NVIDIA GB200 192GB

192 GB VRAM

View tier list

B100 192GB

192 GB VRAM

View tier list
Full AMD Instinct MI350X 288GB pageCompare this hardware