VOOZH about

URL: https://willitrunai.com/my-results/instinct-mi100-32gb

⇱ AMD Instinct MI100 32GB AI Tier List: 309 models ranked | WillItRunAI


AI Compatibility Tier List

AMD Instinct MI100 32GB

374 AI models ranked by how well they run on your 32 GB VRAM hardware. Best match: Qwen3-Coder 30B A3B Instruct (score: 100).

374

Models tested

309

Can run

32 GB

VRAM

121

Best tok/s

Tier Distribution

S 29
A 57
B 41
C 182
F 65
S: 29A: 57B: 41C: 182F: 65
S

S Tier -- Perfect Fit (29 models)

These models run flawlessly with headroom to spare.

👁 Alibaba
Qwen3-Coder 30B A3B Instruct
S100
30.5B24.2 GB121 tok/s102K ctx
moe
👁 Alibaba
Qwen3-VL 30B A3B Instruct
S99
30B23.9 GB125 tok/s105K ctx
moe
👁 Alibaba
Qwen 3.5 27B
S98
27B23.7 GB52 tok/s58K ctx
dense
👁 Alibaba
Qwen 3 30B A3B
S97
30.5B24.2 GB121 tok/s102K ctx
moe
👁 Alibaba
Qwen 3.6 35B A3B
S96
35B29.6 GB101 tok/s26K ctx
+1moe
👁 Mistral
Magistral Small 2507
S96
24B21.2 GB59 tok/s87K ctx
dense
👁 Mistral
Devstral Small 2 24B Instruct
S96
24B21.2 GB59 tok/s87K ctx
dense
👁 Alibaba
Qwen 3.5 35B A3B
S95
35B26.9 GB110 tok/s72K ctx
moe
👁 Alibaba
Qwen 3.6 27B
S95
27B21.5 GB33 tok/s187K ctx
+1dense
👁 NVIDIA
Nemotron 3 Nano 30B
S95
30B24.8 GB47 tok/s63K ctx
dense
👁 NVIDIA
Nemotron Cascade 2 30B A3B
S95
30B25.3 GB123 tok/s52K ctx
moe
👁 Mistral
Devstral Small 1.1
S94
24B21.2 GB59 tok/s87K ctx
dense
👁 OpenAI
GPT-OSS 20B
S93
21B19.4 GB153 tok/s99K ctx
moe
👁 Alibaba
Qwen 3 14B
S93
14B15.1 GB101 tok/s127K ctx
dense
👁 Alibaba
Qwen 3 32B
S93
32B27.5 GB45 tok/s34K ctx
dense
👁 Microsoft
Phi-4-reasoning-plus 14B
S92
14.7B16.1 GB96 tok/s33K ctx
dense
👁 Google
Gemma 4 26B A4B
S92
25.2B23.1 GB130 tok/s55K ctx
moe
👁 Alibaba
Qwen 3.5 9B
S91
9B11.8 GB126 tok/s131K ctx
dense
👁 Alibaba
Qwen 3 8B
S89
8B11.2 GB112 tok/s131K ctx
dense
👁 Mistral
Mistral Small 3.2 24B
S89
24B21.2 GB59 tok/s87K ctx
vision
👁 Mistral
Codestral 2 25.08
S88
22B20.0 GB60 tok/s95K ctx
dense
S88
24B21.2 GB59 tok/s87K ctx
dense
👁 Mistral
Ministral 3 14B
S87
14B15.1 GB101 tok/s127K ctx
multimodal
👁 Mistral
Mistral Small 24B
S87
24B21.2 GB59 tok/s33K ctx
dense
S87
19B18.1 GB74 tok/s8K ctx
dense
👁 LG AI
EXAONE 4.0 32B
S87
32B27.5 GB44 tok/s34K ctx
dense
👁 Alibaba
Qwen 3.5 4B
S86
4B8.7 GB56 tok/s131K ctx
dense
👁 Mistral AI
Mistral Small 3.1 24B
S86
24B21.2 GB59 tok/s87K ctx
dense
👁 Alibaba
Qwen 2.5 32B
S85
32B27.5 GB44 tok/s34K ctx
dense
A

A Tier -- Great Fit (57 models)

Excellent performance with comfortable memory margins.

👁 NVIDIA
Nemotron Nano 8B
A84
8B10.9 GB112 tok/s131K ctx
dense
👁 IBM
Granite 4.1 30B
A84
30B26.3 GB47 tok/s39K ctx
dense
A84
14B15.7 GB101 tok/s16K ctx
dense
👁 Allen AI
OLMo 2 32B
A84
32B27.5 GB44 tok/s4K ctx
dense
👁 IBM
Granite Code 20B
A84
20B19.5 GB71 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 14B
A83
14B15.6 GB101 tok/s106K ctx
dense
👁 Google
Gemma 3 27B
A83
27B31.8 GB32 tok/s16K ctx
dense
👁 Microsoft
Phi-4 Mini Reasoning 4B
A83
3.8B7.9 GB53 tok/s131K ctx
dense
👁 Meta
CodeLlama 13B Instruct
A82
13B24.2 GB101 tok/s16K ctx
dense
👁 DeepSeek
DeepSeek Coder V2 16B
A82
16B17.2 GB195 tok/s88K ctx
moe
👁 Google
Gemma 3 12B
A82
12B16.3 GB71 tok/s67K ctx
dense
👁 InternLM
InternVL2 8B
A81
8B10.9 GB112 tok/s8K ctx
dense
👁 LLaVA
LLaVA 1.6 13B
A80
13B24.2 GB101 tok/s4K ctx
dense
👁 Mistral
Ministral 3 8B
A80
8B11.2 GB112 tok/s168K ctx
multimodal
👁 OpenBMB
MiniCPM-V 2.6 8B
A80
8B10.9 GB112 tok/s2K ctx
dense
👁 NVIDIA
Nemotron Nano 9B v2
A79
9B12.0 GB126 tok/s131K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 32B
A79
32B27.5 GB44 tok/s34K ctx
dense
👁 AllenAI
OLMo 2 13B
A79
13B14.5 GB109 tok/s33K ctx
dense
👁 Defog
SQLCoder 7B
A78
7B10.3 GB98 tok/s8K ctx
dense
👁 BigCode
StarCoder 15B
A78
15B29.5 GB75 tok/s8K ctx
dense
👁 IBM
Granite Code 34B
A78
34B28.5 GB42 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 VL 7B
A78
7B9.2 GB98 tok/s33K ctx
dense
👁 Cohere
Command R 35B
A78
35B27.9 GB41 tok/s43K ctx
dense
👁 Alibaba
Qwen 3 4B
A78
4B8.7 GB56 tok/s33K ctx
dense
👁 WizardLM
WizardLM 13B
A78
13B24.2 GB101 tok/s8K ctx
dense
👁 LMSYS
Vicuna 13B
A78
13B24.2 GB101 tok/s4K ctx
dense
👁 Nous Research
Nous Dolphin 13B
A77
13B25.7 GB87 tok/s16K ctx
dense
👁 Nous Research
Nous Hermes 1.0
A77
9B21.8 GB126 tok/s16K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 32B
A77
32B27.5 GB44 tok/s33K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 14B
A77
14B15.6 GB101 tok/s33K ctx
dense
👁 Magistral AI
Magistral 7B
A77
7B10.3 GB98 tok/s8K ctx
dense
👁 Tsinghua/Zhipu
CodeGeeX 4 9B
A77
9B10.2 GB126 tok/s131K ctx
dense
👁 Google
Gemma 4 E4B
A76
8B10.3 GB109 tok/s128K ctx
dense
👁 Meta
CodeLlama 7B Instruct
A76
7B16.2 GB98 tok/s16K ctx
dense
👁 Jina AI
Jina Embeddings v3
A76
0.57B7.2 GB8 tok/s8K ctx
dense
👁 BigCode
StarCoder 7B
A76
7B15.7 GB98 tok/s8K ctx
dense
👁 Google
Gemma 4 31B
A75
30.7B37.5 GB15 tok/s10K ctx
dense
👁 Mistral AI
Pixtral 12B
A75
12B13.9 GB117 tok/s131K ctx
dense
👁 BAAI
BGE M3
A74
0.57B6.4 GB8 tok/s8K ctx
dense
👁 IBM
Granite Code 8B
A74
8B10.9 GB112 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 7B
A74
7B9.2 GB98 tok/s131K ctx
dense
👁 DevStral AI
DevStral 7B
A74
7B10.3 GB98 tok/s8K ctx
dense
👁 IBM
Granite 4.1 8B
A74
8B11.4 GB112 tok/s131K ctx
dense
👁 InternLM
InternLM 7B
A73
7B16.2 GB98 tok/s8K ctx
dense
👁 InternLM
InternLM Chat 7B
A73
7B16.2 GB98 tok/s8K ctx
dense
👁 Mistral AI
Codestral Mamba 7B
A73
7B8.9 GB98 tok/s262K ctx
state-space
👁 Baichuan
Baichuan 13B
A73
13B25.7 GB87 tok/s8K ctx
dense
👁 Cerebras
Cerebras-GPT 13B
A73
13B23.2 GB87 tok/s30K ctx
dense
👁 Mixedbread AI
mxbai Embed Large
A71
0.34B6.3 GB5 tok/s1K ctx
dense
👁 Moonshot AI
Kimi Linear 48B A3B
A71
48B34.3 GB18 tok/s4K ctx
linear
👁 LLaVA
LLaVA 1.5 7B
A71
7B16.2 GB98 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 3B
A70
3B8.1 GB42 tok/s131K ctx
dense
👁 Meta
Llama 3.1 8B
A70
8B10.9 GB112 tok/s128K ctx
dense
👁 Zhipu
GLM-4 9B
A70
9B10.2 GB126 tok/s128K ctx
dense
👁 Snowflake
Snowflake Arctic Embed L
A70
0.34B6.3 GB5 tok/s1K ctx
dense
A70
7B11.3 GB98 tok/s8K ctx
dense
👁 Google
Gemma 2 27B
A70
27B31.8 GB32 tok/s8K ctx
dense
B

B Tier -- Good Fit (41 models)

Solid performance, may need quantization or reduced context.

👁 WizardLM
WizardMath 7B
B69
7B10.3 GB98 tok/s4K ctx
dense
👁 Allen AI
OLMo 2 7B
B69
7B10.3 GB98 tok/s4K ctx
dense
👁 Google
Gemma 4 E2B
B69
5.1B7.7 GB71 tok/s128K ctx
dense
👁 MosaicML
MPT-7B-Instruct
B69
7B16.2 GB98 tok/s8K ctx
dense
👁 Baichuan
Baichuan 7B
B69
7B16.2 GB98 tok/s8K ctx
dense
👁 BAAI
BGE Large EN v1.5
B68
0.34B6.3 GB5 tok/s1K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 7B
B68
7B9.2 GB98 tok/s131K ctx
dense
👁 Google
Gemma 3 4B
B67
4B8.6 GB56 tok/s128K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 14B
B67
14B15.6 GB101 tok/s106K ctx
dense
👁 Mistral
Ministral 3 3B
B66
3B6.7 GB42 tok/s262K ctx
multimodal
👁 Microsoft
Phi 4 Mini 4B
B66
4B8.0 GB56 tok/s128K ctx
dense
👁 Google
Gemma 2 9B
B66
9B14.7 GB95 tok/s8K ctx
dense
👁 Nomic AI
Nomic Embed Text v1.5
B66
0.14B4.9 GB2 tok/s8K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 8B
B66
8B10.9 GB112 tok/s33K ctx
dense
👁 Microsoft
Phi 3 Mini 3.8B
B65
3.8B12.3 GB53 tok/s70K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 7B
B65
7B9.2 GB98 tok/s33K ctx
dense
👁 Alibaba
Qwen 3.5 2B
B65
2B7.0 GB28 tok/s131K ctx
dense
7B10.3 GB98 tok/s4K ctx
dense
👁 Meta
Llama 3.2 11B Vision
B65
11B12.8 GB128 tok/s16K ctx
vision
👁 Microsoft
Phi 3.5 Mini 4B
B64
4B12.4 GB56 tok/s70K ctx
dense
👁 Mistral AI
Codestral 22B
B64
22B20.0 GB64 tok/s33K ctx
dense
👁 01.AI
Yi 1.5 34B
B64
34B28.5 GB42 tok/s4K ctx
dense
👁 TII
Falcon 7B Instruct
B64
7B8.5 GB98 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 3B
B64
3B8.1 GB42 tok/s131K ctx
dense
👁 Mistral
Mistral Nemo 12B
B64
12B13.9 GB117 tok/s128K ctx
dense
👁 Microsoft
Phi 3 Medium 14B
B63
14B15.7 GB101 tok/s102K ctx
dense
👁 Alibaba
Qwen 3 1.7B
B62
1.7B6.8 GB24 tok/s33K ctx
dense
👁 01.AI
Yi Coder 9B
B61
9B11.1 GB126 tok/s131K ctx
dense
👁 IBM
Granite 4.1 3B
B61
3B7.2 GB42 tok/s131K ctx
dense
👁 IBM
Granite Code 3B
B61
3B8.4 GB42 tok/s8K ctx
dense
👁 Mistral
Mistral 7B Instruct v0.3
B61
7B10.3 GB98 tok/s8K ctx
dense
B60
0.02B4.4 GB2 tok/s0K ctx
dense
👁 Cohere
Aya Expanse 32B
B60
32B26.1 GB45 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 1.5B
B60
1.5B5.4 GB21 tok/s33K ctx
dense
👁 TII
Falcon 40B Instruct
B60
40B34.7 GB19 tok/s4K ctx
dense
👁 Mistral
Ministral 8B
B59
8B11.2 GB112 tok/s131K ctx
dense
👁 Meta
Llama 3.2 3B
B57
3B7.6 GB42 tok/s128K ctx
dense
👁 Mistral
Mixtral 8x7B
B56
47B34.7 GB36 tok/s4K ctx
moe
👁 Stability AI
StableLM 2 12B
B56
12B24.9 GB71 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 27B
B55
27B23.7 GB49 tok/s58K ctx
dense
👁 Unsloth
gemma 3 27b it
B55
27B23.7 GB49 tok/s58K ctx
dense
C

C Tier -- Tight Fit (182 models)

Runs with offloading or significant compromises.

gemma 3 27b it
C55
27B23.7 GB49 tok/s58K ctx
dense
👁 Unsloth
Mistral Small 3.2 24B Instruct 2506
C55
24B21.6 GB55 tok/s75K ctx
dense
cognitivecomputations Dolphin Mistral 24B Venice Edition
C55
24B21.6 GB55 tok/s75K ctx
dense
Mistral Small 24B Instruct 2501
C55
24B21.6 GB55 tok/s75K ctx
dense
mistral small 3.1 24b instruct 2503 hf
C55
24B21.6 GB55 tok/s75K ctx
dense
cognitivecomputations Dolphin3.0 R1 Mistral 24B
C55
24B21.6 GB55 tok/s75K ctx
dense
Dolphin Mistral GLM 4.7 Flash 24B Venice Edition Thinking Uncensored i1
C55
24B21.6 GB55 tok/s75K ctx
dense
cognitivecomputations Dolphin Mistral 24B Venice Edition
C54
24B21.6 GB55 tok/s75K ctx
dense
Codestral 22B v0.1
C54
22B20.1 GB60 tok/s90K ctx
dense
Codestral 22B v0.1
C54
22B20.1 GB60 tok/s90K ctx
dense
👁 IBM
Granite 3.1 8B
C54
8B10.9 GB112 tok/s128K ctx
state-space
👁 01.AI
Yi 1.5 9B
C54
9B11.1 GB126 tok/s4K ctx
dense
Codestral 22B v0.1
C54
22B20.1 GB60 tok/s90K ctx
dense
👁 BigCode
StarCoder2 15B
C54
15B16.1 GB82 tok/s16K ctx
dense
Codestral 22B v0.1 IMat
C54
22B20.1 GB60 tok/s90K ctx
dense
Codestral 22B v0.1 i1
C53
22B20.1 GB60 tok/s90K ctx
dense
👁 LMSYS
Vicuna 7B
C53
7B16.2 GB98 tok/s4K ctx
dense
Codestral 21B Pruned i1
C53
21B19.4 GB62 tok/s98K ctx
dense
internlm2 5 20b chat
C53
20B18.6 GB65 tok/s107K ctx
dense
👁 01.AI
Yi 34B Chat
C53
34B28.5 GB42 tok/s31K ctx
dense
internlm2 limarp chat 20b
C53
20B18.6 GB65 tok/s107K ctx
dense
internlm2 math plus 20b i1
C53
20B18.6 GB65 tok/s107K ctx
dense
Codestral RAG 19B Pruned i1
C52
19B17.9 GB69 tok/s117K ctx
dense
C52
3B7.9 GB42 tok/s128K ctx
dense
👁 Alibaba
Qwen 2.5 Math 7B
C52
7B9.2 GB98 tok/s4K ctx
dense
Phi 4 reasoning vision 15B
C52
15B15.0 GB87 tok/s171K ctx
dense
StarCoder2 15B
C52
15B15.0 GB87 tok/s171K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Qwen 14B
C52
14B14.3 GB94 tok/s189K ctx
dense
👁 DeepSeek
DeepSeek R1 1.5B
C51
1.5B5.4 GB21 tok/s33K ctx
dense
starcoder2 15b instruct v0.1
C51
15B15.0 GB87 tok/s171K ctx
dense
starcoder2 15b i1
C51
15B15.0 GB87 tok/s171K ctx
dense
starcoder2 15b instruct v0.1
C51
15B15.0 GB87 tok/s171K ctx
dense
👁 Unsloth
Qwen3.5 35B A3B
C51
35B29.6 GB37 tok/s26K ctx
dense
NousResearch Hermes 4 14B
C51
14B14.3 GB94 tok/s189K ctx
dense
HelpingAI 15B i1
C51
15B15.0 GB87 tok/s171K ctx
dense
👁 TinyLlama
TinyLlama 1.1B
C51
1.1B5.1 GB15 tok/s4K ctx
dense
Qwen3.5 35B A3B
C51
35B29.6 GB37 tok/s26K ctx
dense
GGUF SOLARized GraniStral 14B 2102 YeAM HCT 32QKV
C51
14B14.3 GB94 tok/s189K ctx
dense
internlm JanusCoder 14B
C51
14B14.3 GB94 tok/s189K ctx
dense
EXAONE 4.0 32B
C51
32B27.4 GB41 tok/s36K ctx
dense
GGUF SOLARized GraniStral 14B 1902 YeAM HCT
C51
14B14.3 GB94 tok/s189K ctx
dense
aya expanse 32b heretic MPOA i1
C51
32B27.4 GB41 tok/s36K ctx
dense
Baichuan M2 32B Q4 K M
C51
32B27.4 GB41 tok/s36K ctx
dense
baichuan inc Baichuan M2 32B
C51
32B27.4 GB41 tok/s36K ctx
dense
👁 OpenChat
OpenChat 7B
C51
7B10.3 GB98 tok/s8K ctx
dense
👁 DeepSeek
DeepSeek LLM 7B
C51
7B15.7 GB98 tok/s4K ctx
dense
👁 Cohere
Aya Expanse 8B
C51
8B10.9 GB112 tok/s8K ctx
dense
Hermes 4.3 36B
C51
36B30.3 GB36 tok/s23K ctx
dense
gemma 3 12b it
C51
12B12.8 GB109 tok/s234K ctx
dense
8B10.9 GB112 tok/s33K ctx
dense
SOLAR 10.7B Instruct v1.0 uncensored
C50
10.7B11.9 GB122 tok/s273K ctx
dense
Nous Hermes 2 SOLAR 10.7B
C50
10.7B11.9 GB122 tok/s273K ctx
dense
👁 Teknium
OpenHermes 2.5 7B
C50
7B10.3 GB98 tok/s8K ctx
dense
solar finalised finetuned Model 10.7B i1
C49
10.7B11.9 GB122 tok/s273K ctx
dense
SOLAR 10.7B v1.0
C49
10.7B11.9 GB122 tok/s273K ctx
dense
👁 Unsloth
Qwen3.5 9B
C49
9B10.6 GB126 tok/s340K ctx
dense
👁 Hugging Face H4
Zephyr 7B Beta
C49
7B10.3 GB98 tok/s33K ctx
dense
Qwen3.5 9B Uncensored HauhauCS Aggressive
C49
9B10.6 GB126 tok/s340K ctx
dense
Qwen3.5 9B
C49
9B10.6 GB126 tok/s340K ctx
dense
👁 UC Berkeley
Starling LM 7B
C49
7B10.3 GB98 tok/s8K ctx
dense
Yi Coder 9B Chat
C49
9B10.6 GB126 tok/s340K ctx
dense
Helply 10.2b chat i1
C49
10.2B11.5 GB128 tok/s290K ctx
dense
glm 4 9b chat 1m
C49
9B10.6 GB126 tok/s340K ctx
dense
HelpingAI2.5 10B i1
C49
10B11.4 GB131 tok/s298K ctx
dense
👁 Alibaba
Qwen 2.5 1.5B
C49
1.5B5.4 GB21 tok/s131K ctx
dense
Yi 1.5 9B Chat
C49
9B10.6 GB126 tok/s340K ctx
dense
Meta Llama 3.1 8B Instruct
C49
8B9.9 GB112 tok/s393K ctx
dense
llava llama 3 8b v1 1
C49
8B9.9 GB112 tok/s393K ctx
dense
👁 Unsloth
DeepSeek R1 0528 Qwen3 8B
C49
8B9.9 GB112 tok/s393K ctx
dense
Meta Llama 3 8B Instruct
C49
8B9.9 GB112 tok/s393K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Llama 8B
C49
8B9.9 GB112 tok/s393K ctx
dense
Dolphin3.0 Llama3.1 8B
C49
8B9.9 GB112 tok/s393K ctx
dense
Llama 3 8B Instruct 32k v0.1
C49
8B9.9 GB112 tok/s393K ctx
dense
Meta Llama 3.1 8B Instruct
C49
8B9.9 GB112 tok/s393K ctx
dense
vntl llama3 8b v2
C49
8B9.9 GB112 tok/s393K ctx
dense
DeepSeek R1 0528 Qwen3 8B
C49
8B9.9 GB112 tok/s393K ctx
dense
Yi 9B Coder i1
C49
9B10.6 GB126 tok/s340K ctx
dense
DeepSeek R1 0528 Qwen3 8B
C49
8B9.9 GB112 tok/s393K ctx
dense
HelpingAI2 9B
C49
9B10.6 GB126 tok/s340K ctx
dense
Qwen3 8B DeepSeek v3.2 Speciale Distill
C49
8B9.9 GB112 tok/s393K ctx
dense
Hermes 3 Llama 3.1 8B
C49
8B9.9 GB112 tok/s393K ctx
dense
HelpingAI2 9B i1
C48
9B10.6 GB126 tok/s340K ctx
dense
HelpingAI 9B 200k i1
C48
9B10.6 GB126 tok/s340K ctx
dense
Hermes 2 Pro Llama 3 8B
C48
8B9.9 GB112 tok/s393K ctx
dense
HelpingAI 9B i1
C48
9B10.6 GB126 tok/s340K ctx
dense
Dolphin3.0 Llama3.1 8B
C48
8B9.9 GB112 tok/s393K ctx
dense
granite 8b code instruct 4k
C48
8B9.9 GB112 tok/s393K ctx
dense
dolphin 2.9.4 llama3.1 8b
C48
8B9.9 GB112 tok/s393K ctx
dense
Llama 2 7B Chat
C48
7B9.2 GB98 tok/s461K ctx
dense
👁 InternLM
InternLM 20B
C48
20B36.8 GB32 tok/s8K ctx
dense
openchat 3.6 8b 20240522 IMat
C48
8B9.9 GB112 tok/s393K ctx
dense
Mistral 7B Instruct v0.2
C48
7B9.2 GB98 tok/s461K ctx
dense
aya expanse 8b
C48
8B9.9 GB112 tok/s393K ctx
dense
dolphin v2 8b abliterated i1
C48
8B9.9 GB112 tok/s393K ctx
dense
Mistral 7B Instruct v0.3
C48
7B9.2 GB98 tok/s461K ctx
dense
EXAONE 3.5 7.8B Instruct
C48
7.8B9.8 GB109 tok/s405K ctx
dense
aya expanse 8b orthogonal heretic i1
C48
8B9.9 GB112 tok/s393K ctx
dense
EXAONE 3.5 7.8B Instruct
C48
7.8B9.8 GB109 tok/s405K ctx
dense
EXAONE 3.5 7.8B Instruct i1
C48
7.8B9.8 GB109 tok/s405K ctx
dense
exaone 3.0 7.8b it
C48
7.8B9.8 GB109 tok/s405K ctx
dense
aya expanse 8b orthogonal heretic
C48
8B9.9 GB112 tok/s393K ctx
dense
Mistral 7B Instruct v0.3
C48
7B9.2 GB98 tok/s461K ctx
dense
👁 Intel
Neural Chat 7B
C48
7B10.3 GB98 tok/s8K ctx
dense
Hermes 2 Pro Mistral 7B
C48
7B9.2 GB98 tok/s461K ctx
dense
internlm2 5 1 8b chat i1
C48
8B9.9 GB112 tok/s393K ctx
dense
internlm3 8b instruct abliterated i1
C48
8B9.9 GB112 tok/s393K ctx
dense
zephyr 7B beta
C48
7B9.2 GB98 tok/s461K ctx
dense
Nous Hermes 2 Mistral 7B DPO
C48
7B9.2 GB98 tok/s461K ctx
dense
Falcon H1R 7B
C48
7B9.2 GB98 tok/s461K ctx
dense
falcon mamba 7b instruct Q4 K M
C48
7B9.2 GB98 tok/s461K ctx
dense
zephyr 7B alpha
C48
7B9.2 GB98 tok/s461K ctx
dense
speechless zephyr code functionary 7b
C48
7B9.2 GB98 tok/s461K ctx
dense
StarCoder2 7B
C48
7B9.2 GB98 tok/s461K ctx
dense
japanese stablelm instruct gamma 7B
C48
7B9.2 GB98 tok/s461K ctx
dense
Falcon H1 7B Instruct
C48
7B9.2 GB98 tok/s461K ctx
dense
👁 Unsloth
Falcon H1R 7B
C48
7B9.2 GB98 tok/s461K ctx
dense
Mamba Codestral 7B v0.1
C48
7B9.2 GB98 tok/s461K ctx
dense
stabilityai japanese stablelm base gamma 7b
C48
7B9.2 GB98 tok/s461K ctx
dense
baichuan2 7b chat
C48
7B9.2 GB98 tok/s461K ctx
dense
DiscoPOP zephyr 7b gemma
C48
7B9.2 GB98 tok/s461K ctx
dense
starcoder2 7b
C48
7B9.2 GB98 tok/s461K ctx
dense
zephyr 7b beta Mistral 7B Instruct v0.2
C48
7B9.2 GB98 tok/s461K ctx
dense
OpenChat 3.5 7B Qwen v2.0 i1
C48
7B9.2 GB98 tok/s461K ctx
dense
OpenChat 3.5 7B Starling v2.0 i1
C48
7B9.2 GB98 tok/s461K ctx
dense
internlm2 math plus 7b IMat
C48
7B9.2 GB98 tok/s461K ctx
dense
CodeNinja 1.0 OpenChat 7B i1
C48
7B9.2 GB98 tok/s461K ctx
dense
internlm2 5 7b chat i1
C48
7B9.2 GB98 tok/s461K ctx
dense
OpenSafetyLab MD Judge v0 2 internlm2 7b
C48
7B9.2 GB98 tok/s461K ctx
dense
MD Judge v0 2 internlm2 7b i1
C48
7B9.2 GB98 tok/s461K ctx
dense
zephyr 7b gemma sft african ultrachat 100k
C47
7B9.2 GB98 tok/s461K ctx
dense
jointpreferences mistral 7b sft helpful
C47
7B9.2 GB98 tok/s461K ctx
dense
zephyr 7b dpo full i1
C47
7B9.2 GB98 tok/s461K ctx
dense
blossom v3 baichuan2 7b i1
C47
7B9.2 GB98 tok/s461K ctx
dense
blossom v1 baichuan 7b i1
C47
7B9.2 GB98 tok/s461K ctx
dense
Yi 1.5 6B Chat
C47
6B8.5 GB84 tok/s552K ctx
dense
👁 Google
Gemma 2 2B
C47
2B6.9 GB28 tok/s8K ctx
dense
stablelm 2 zephyr 1 6b
C47
6B8.5 GB84 tok/s552K ctx
dense
👁 Google
Gemma 3 1B
C47
1B5.1 GB14 tok/s33K ctx
dense
👁 01.AI
Yi 1.5 6B
C47
6B8.7 GB84 tok/s4K ctx
dense
Yi 1.5 6B Chat
C47
6B8.5 GB84 tok/s552K ctx
dense
stablelm 2 1 6b chat imatrix
C47
6B8.5 GB84 tok/s552K ctx
dense
👁 BigCode
StarCoder2 7B
C47
7B8.9 GB98 tok/s16K ctx
dense
HelpingAI2 6B
C47
6B8.5 GB84 tok/s552K ctx
dense
HelpingAI2 6B i1
C47
6B8.5 GB84 tok/s552K ctx
dense
👁 NVIDIA
Nemotron Mini 4B
C47
4B8.5 GB56 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 0.5B
C46
0.5B4.6 GB7 tok/s131K ctx
dense
HelpingAI2.5 5B i1
C46
5B7.7 GB70 tok/s679K ctx
dense
👁 Unsloth
Qwen3.5 4B
C45
4B7.0 GB56 tok/s869K ctx
dense
gemma 3 4b it
C45
4B7.0 GB56 tok/s869K ctx
dense
gemma 3 4b it
C45
4B7.0 GB56 tok/s869K ctx
dense
Llama 3.2 3B Instruct
C44
3B6.6 GB42 tok/s1171K ctx
dense
👁 Alibaba
Qwen 3 0.6B
C44
0.6B5.3 GB8 tok/s33K ctx
dense
👁 Meta
Llama 3.2 1B
C44
1B5.2 GB14 tok/s128K ctx
dense
👁 Qwen
Qwen2.5 3B Instruct
C44
3B6.3 GB42 tok/s1186K ctx
dense
Llama 3.2 3B Instruct
C44
3B6.3 GB42 tok/s1186K ctx
dense
Ministral 3 3B Instruct 2512
C44
3B6.3 GB42 tok/s1186K ctx
dense
HELVETE 3B
C43
3B6.3 GB42 tok/s1186K ctx
dense
Hermes 3 Llama 3.2 3B
C43
3B6.3 GB42 tok/s1186K ctx
dense
stablelm zephyr 3b
C43
3B6.3 GB42 tok/s1186K ctx
dense
StarCoder2 3B
C43
3B6.3 GB42 tok/s1186K ctx
dense
AI21 Jamba Reasoning 3B
C43
3B6.3 GB42 tok/s1186K ctx
dense
stablelm 3b 4e1t
C43
3B6.3 GB42 tok/s1186K ctx
dense
ai21labs AI21 Jamba Reasoning 3B
C43
3B6.3 GB42 tok/s1186K ctx
dense
ai21labs AI21 Jamba2 3B
C43
3B6.3 GB42 tok/s1186K ctx
dense
HelpingAI 3B hindi i1
C43
3B6.3 GB42 tok/s1186K ctx
dense
AI21 Jamba2 3B
C43
3B6.3 GB42 tok/s1186K ctx
dense
HelpingAI 3B hindi
C43
3B6.3 GB42 tok/s1186K ctx
dense
AI21 Jamba2 3B i1
C43
3B6.3 GB42 tok/s1186K ctx
dense
gemma 2 2b it
C42
2B6.0 GB28 tok/s1793K ctx
dense
EXAONE 3.5 2.4B Instruct
C42
2.4B5.8 GB34 tok/s1504K ctx
dense
👁 BigCode
StarCoder2 3B
C42
3B6.4 GB42 tok/s16K ctx
dense
👁 Google
gemma 2b
C42
2B5.6 GB28 tok/s1821K ctx
dense
Gemmasutra Mini 2B v1
C42
2B5.6 GB28 tok/s1821K ctx
dense
gemma 2 2b it
C42
2B5.6 GB28 tok/s1821K ctx
dense
👁 Alibaba
Qwen 3.5 0.6B
C42
0.6B5.3 GB8 tok/s131K ctx
dense
👁 Qwen
Qwen2.5 1.5B Instruct
C41
1.5B5.2 GB21 tok/s2456K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Qwen 1.5B
C41
1.5B5.2 GB21 tok/s2456K ctx
dense
Yi Coder 1.5B Chat
C41
1.5B5.2 GB21 tok/s2456K ctx
dense
stablelm 2 zephyr 1.6b
C40
1.6B5.3 GB22 tok/s2298K ctx
dense
logos16v2 stablelm2 1.6b i1
C40
1.6B5.3 GB22 tok/s2298K ctx
dense
Yi Coder 1.5B
C40
1.5B5.2 GB21 tok/s2456K ctx
dense
👁 Unsloth
Falcon H1 1.5B Instruct
C40
1.5B5.2 GB21 tok/s2456K ctx
dense
F

F Tier -- Can't Run (65 models)

Too heavy for this hardware.

TinyLlama 1.1B Chat v1.0
D40
1.1B4.9 GB15 tok/s3380K ctx
dense
EXAONE 4.0 1.2B
D40
1.2B5.0 GB17 tok/s3091K ctx
dense
Llama 3.2 1B Instruct Q8 0
D40
1B5.0 GB14 tok/s3697K ctx
dense
👁 Alibaba
Qwen 2.5 0.5B
D39
0.5B4.6 GB7 tok/s131K ctx
dense
TinyLlama 1.1B Chat v0.3
D39
1.1B4.9 GB15 tok/s3380K ctx
dense
👁 TinyLlama
TinyLlama 1.1B Chat v0.6
D39
1.1B4.9 GB15 tok/s3380K ctx
dense
Llama 3.2 1B Instruct
D39
1B4.8 GB14 tok/s3726K ctx
dense
gemma 3 1b it
D39
1B4.8 GB14 tok/s3726K ctx
dense
TinyLlama 1.1B Chat v1.0 imatrix
D39
1.1B4.9 GB15 tok/s3380K ctx
dense
Falcon3 1B Instruct abliterated
D39
1B4.8 GB14 tok/s3726K ctx
dense
SmolVLM 500M Instruct
D37
0.5B4.6 GB7 tok/s4398K ctx
dense
embeddinggemma 300M
D35
0.3B4.4 GB4 tok/s4425K ctx
dense
granite embedding 107m multilingual
D33
0.11B4.3 GB2 tok/s4454K ctx
dense
Falcon H1 Tiny 90M Instruct
D33
0.09B4.3 GB2 tok/s4455K ctx
dense
👁 Meta
Llama 3.1 70B
F0
70B51.7 GB6 tok/s4K ctx
dense
👁 Meta
Llama 3.3 70B
F0
70B51.7 GB6 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 72B
F0
72B52.9 GB5 tok/s4K ctx
dense
👁 Alibaba
Qwen 3 235B A22B
F0
235B150.3 GB2 tok/s4K ctx
moe
👁 Alibaba
Qwen3-Coder-Next
F0
80B54.4 GB13 tok/s4K ctx
moe
👁 Alibaba
Qwen3-Coder 480B A35B Instruct
F0
480B299.8 GB2 tok/s4K ctx
moe
👁 Alibaba
Qwen 3.5 122B A10B
F0
122B81.0 GB5 tok/s4K ctx
moe
👁 Alibaba
Qwen 3.5 397B A17B
F0
397B249.1 GB2 tok/s4K ctx
moe
👁 Mistral
Mixtral 8x22B
F0
141B93.5 GB3 tok/s4K ctx
moe
👁 MosaicML
MPT-30B-Instruct
F0
30B49.1 GB12 tok/s4K ctx
dense
👁 Mistral
Devstral 2 123B Instruct
F0
123B84.5 GB2 tok/s4K ctx
dense
👁 Mistral
Mistral Small 4 119B
F0
119B82.1 GB5 tok/s4K ctx
moe
👁 Mistral
Leanstral 119B A6B
F0
119B85.5 GB5 tok/s4K ctx
moe
👁 Mistral
Mistral Large 3
F0
675B421.2 GB2 tok/s4K ctx
+1moe
F0
744B477.0 GB2 tok/s4K ctx
moe
👁 Z.ai
GLM-5.1
F0
754B483.1 GB2 tok/s4K ctx
moe
F0
1000B621.5 GB2 tok/s4K ctx
moe
F0
1000B621.5 GB2 tok/s4K ctx
+1moe
👁 DeepSeek
DeepSeek V3 671B
F0
671B473.0 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V3.1 671B
F0
671B473.0 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V4 Pro
F0
1600B868.0 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V4 Flash
F0
284B163.4 GB3 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek R1 671B
F0
671B473.0 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek Coder V2 236B
F0
236B206.7 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V2.5 236B
F0
236B206.7 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek LLM 67B
F0
67B50.8 GB6 tok/s4K ctx
dense
👁 Meta
Llama 4 Scout 17B 16E
F0
109B73.5 GB5 tok/s4K ctx
moe
👁 Meta
Llama 4 Maverick 17B 128E
F0
400B251.0 GB2 tok/s4K ctx
moe
👁 Cohere
Command R+ 104B
F0
104B71.0 GB2 tok/s4K ctx
dense
👁 Cohere
Command A 111B
F0
111B75.7 GB2 tok/s4K ctx
dense
👁 Mistral AI
Pixtral Large 124B
F0
124B85.1 GB2 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 VL 72B
F0
72B52.9 GB5 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 Math 72B
F0
72B52.9 GB5 tok/s4K ctx
dense
👁 NVIDIA
Nemotron 70B
F0
70B51.7 GB6 tok/s4K ctx
dense
👁 Meta
Llama 3.1 405B
F0
405B258.8 GB2 tok/s4K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 70B
F0
70B51.7 GB6 tok/s4K ctx
dense
👁 OpenAI
GPT-OSS 120B
F0
117B80.4 GB2 tok/s4K ctx
dense
👁 DeepSeek
DeepSeek V3.2
F0
671B413.9 GB2 tok/s4K ctx
moe
MiniMax M2.7
F0
230B148.2 GB3 tok/s4K ctx
moe
Solar Open 100B
F0
100B76.8 GB2 tok/s4K ctx
dense
Qwen3 48B A4B Savant Commander Distill 12X Closed Open Heretic Uncensored
F0
48B39.0 GB14 tok/s4K ctx
dense
K EXAONE 236B A23B
F0
236B175.7 GB2 tok/s4K ctx
dense
Llama 3.3 70B Instruct
F0
70B55.0 GB5 tok/s4K ctx
dense
Baichuan M3 235B
F0
235B175.0 GB2 tok/s4K ctx
dense
Baichuan M3 235B i1
F0
235B175.0 GB2 tok/s4K ctx
dense
BaichuanMed OCR 72B i1
F0
72B56.5 GB4 tok/s4K ctx
dense
Solar Open 100B i1
F0
100B76.8 GB2 tok/s4K ctx
dense
Solar Open 69B REAP i1
F0
69B54.3 GB5 tok/s4K ctx
dense
stabilityai japanese stablelm instruct beta 70b
F0
70B55.0 GB5 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 122B A10B
F0
122B78.2 GB2 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 397B A17B
F0
397B292.8 GB2 tok/s4K ctx
dense

Compare with...

See How Other Hardware Stacks Up

Check tier lists for similar GPUs and Apple Silicon configs

RTX 5090 32GB

32 GB VRAM

View tier list

RTX 5000 Ada 32GB

32 GB VRAM

View tier list

RTX PRO 4500 Blackwell 32GB

32 GB VRAM

View tier list

Radeon AI PRO R9700 32GB

32 GB VRAM

View tier list
Full AMD Instinct MI100 32GB pageCompare this hardware