VOOZH about

URL: https://willitrunai.com/my-results/a16-64gb

⇱ NVIDIA A16 64GB AI Tier List: 322 models ranked | WillItRunAI


AI Compatibility Tier List

NVIDIA A16 64GB

374 AI models ranked by how well they run on your 64 GB VRAM hardware. Best match: Qwen 3.6 35B A3B (score: 94).

374

Models tested

322

Can run

64 GB

VRAM

60

Best tok/s

Tier Distribution

S 22
A 59
B 54
C 187
F 52
S: 22A: 59B: 54C: 187F: 52
S

S Tier -- Perfect Fit (22 models)

These models run flawlessly with headroom to spare.

👁 Alibaba
Qwen 3.6 35B A3B
S94
35B32.8 GB60 tok/s138K ctx
+1moe
👁 Alibaba
Qwen3-Coder 30B A3B Instruct
S93
30.5B27.4 GB71 tok/s256K ctx
moe
👁 Alibaba
Qwen3-VL 30B A3B Instruct
S92
30B27.1 GB73 tok/s256K ctx
moe
👁 Alibaba
Qwen 3.5 35B A3B
S92
35B30.1 GB65 tok/s131K ctx
moe
👁 Alibaba
Qwen 3 30B A3B
S91
30.5B27.4 GB71 tok/s131K ctx
moe
👁 Alibaba
Qwen3-Coder-Next
S90
80B57.6 GB32 tok/s86K ctx
moe
👁 Alibaba
Qwen 3.5 27B
S90
27B26.9 GB31 tok/s131K ctx
dense
👁 Alibaba
Qwen 3 32B
S89
32B30.7 GB26 tok/s131K ctx
dense
👁 Alibaba
Qwen 3.5 9B
S88
9B15.0 GB92 tok/s131K ctx
dense
👁 NVIDIA
Nemotron Cascade 2 30B A3B
S88
30B28.5 GB72 tok/s210K ctx
moe
👁 Mistral
Magistral Small 2507
S88
24B24.4 GB34 tok/s131K ctx
dense
👁 Mistral
Devstral Small 2 24B Instruct
S88
24B24.4 GB34 tok/s256K ctx
dense
👁 Alibaba
Qwen 3.6 27B
S88
27B24.7 GB23 tok/s262K ctx
+1dense
👁 OpenAI
GPT-OSS 20B
S88
21B22.6 GB90 tok/s128K ctx
moe
👁 Alibaba
Qwen 2.5 VL 72B
S88
72B56.1 GB12 tok/s33K ctx
dense
👁 NVIDIA
Nemotron 3 Nano 30B
S87
30B28.0 GB28 tok/s131K ctx
dense
👁 Google
Gemma 4 31B
S87
30.7B40.7 GB20 tok/s41K ctx
dense
👁 Alibaba
Qwen 3 14B
S87
14B18.3 GB59 tok/s131K ctx
dense
👁 Alibaba
Qwen 3 8B
S87
8B14.4 GB103 tok/s131K ctx
dense
👁 Microsoft
Phi-4-reasoning-plus 14B
S86
14.7B19.3 GB56 tok/s33K ctx
dense
👁 Mistral
Devstral Small 1.1
S86
24B24.4 GB34 tok/s131K ctx
dense
👁 Google
Gemma 4 26B A4B
S85
25.2B26.3 GB76 tok/s181K ctx
moe
A

A Tier -- Great Fit (59 models)

Excellent performance with comfortable memory margins.

👁 Alibaba
Qwen 3.5 4B
A84
4B11.9 GB56 tok/s131K ctx
dense
👁 LG AI
EXAONE 4.0 32B
A83
32B30.7 GB26 tok/s131K ctx
dense
👁 NVIDIA
Nemotron Nano 8B
A82
8B14.1 GB103 tok/s131K ctx
dense
👁 Meta
Llama 3.3 70B
A82
70B54.9 GB12 tok/s46K ctx
dense
👁 Alibaba
Qwen 2.5 32B
A82
32B30.7 GB26 tok/s131K ctx
dense
👁 Mistral
Ministral 3 14B
A82
14B18.3 GB59 tok/s262K ctx
multimodal
👁 Google
Gemma 3 27B
A82
27B35.0 GB23 tok/s57K ctx
dense
👁 Moonshot AI
Kimi Linear 48B A3B
A81
48B37.5 GB16 tok/s473K ctx
linear
👁 Microsoft
Phi-4 Mini Reasoning 4B
A81
3.8B11.1 GB53 tok/s131K ctx
dense
👁 Mistral
Mistral Small 3.2 24B
A81
24B24.4 GB34 tok/s131K ctx
vision
👁 Mistral
Codestral 2 25.08
A81
22B23.2 GB34 tok/s256K ctx
dense
👁 Allen AI
OLMo 2 32B
A80
32B30.7 GB26 tok/s4K ctx
dense
👁 IBM
Granite 4.1 30B
A80
30B29.5 GB28 tok/s131K ctx
dense
A80
24B24.4 GB34 tok/s131K ctx
dense
A80
19B21.3 GB43 tok/s8K ctx
dense
👁 InternLM
InternVL2 8B
A79
8B14.1 GB103 tok/s8K ctx
dense
👁 Mistral
Mistral Small 24B
A79
24B24.4 GB34 tok/s33K ctx
dense
👁 Meta
Llama 3.1 70B
A78
70B54.9 GB12 tok/s46K ctx
dense
A78
14B18.9 GB59 tok/s16K ctx
dense
👁 Alibaba
Qwen 2.5 72B
A78
72B56.1 GB12 tok/s42K ctx
dense
👁 Mistral AI
Mistral Small 3.1 24B
A78
24B24.4 GB34 tok/s131K ctx
dense
👁 Alibaba
Qwen 2.5 14B
A78
14B18.8 GB59 tok/s131K ctx
dense
👁 DeepSeek
DeepSeek Coder V2 16B
A77
16B20.4 GB114 tok/s131K ctx
moe
👁 Mistral
Ministral 3 8B
A77
8B14.4 GB103 tok/s262K ctx
multimodal
👁 OpenBMB
MiniCPM-V 2.6 8B
A77
8B14.1 GB103 tok/s2K ctx
dense
👁 IBM
Granite Code 20B
A77
20B22.7 GB41 tok/s8K ctx
dense
👁 NVIDIA
Nemotron Nano 9B v2
A76
9B15.2 GB92 tok/s131K ctx
dense
👁 Google
Gemma 3 12B
A76
12B19.5 GB51 tok/s131K ctx
dense
👁 Defog
SQLCoder 7B
A76
7B13.5 GB98 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 32B
A76
32B30.7 GB26 tok/s131K ctx
dense
👁 Alibaba
Qwen 3 4B
A76
4B11.9 GB56 tok/s33K ctx
dense
👁 Alibaba
Qwen 2.5 VL 7B
A76
7B12.4 GB98 tok/s33K ctx
dense
👁 BigCode
StarCoder 15B
A76
15B32.7 GB44 tok/s8K ctx
dense
👁 IBM
Granite Code 34B
A75
34B31.7 GB24 tok/s8K ctx
dense
👁 Meta
CodeLlama 13B Instruct
A75
13B27.4 GB59 tok/s16K ctx
dense
👁 Jina AI
Jina Embeddings v3
A74
0.57B10.4 GB8 tok/s8K ctx
dense
👁 Cohere
Command R 35B
A74
35B31.1 GB24 tok/s131K ctx
dense
👁 Magistral AI
Magistral 7B
A74
7B13.5 GB98 tok/s8K ctx
dense
👁 Tsinghua/Zhipu
CodeGeeX 4 9B
A74
9B13.4 GB93 tok/s131K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 70B
A74
70B54.9 GB12 tok/s46K ctx
dense
👁 AllenAI
OLMo 2 13B
A74
13B17.7 GB64 tok/s33K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 32B
A73
32B30.7 GB26 tok/s33K ctx
dense
👁 BAAI
BGE M3
A73
0.57B9.6 GB8 tok/s8K ctx
dense
👁 Google
Gemma 4 E4B
A73
8B13.5 GB78 tok/s128K ctx
dense
👁 LLaVA
LLaVA 1.6 13B
A73
13B27.4 GB59 tok/s4K ctx
dense
👁 MosaicML
MPT-30B-Instruct
A73
30B52.3 GB22 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 7B
A72
7B12.4 GB98 tok/s131K ctx
dense
👁 IBM
Granite Code 8B
A72
8B14.1 GB103 tok/s8K ctx
dense
👁 Meta
CodeLlama 7B Instruct
A72
7B19.4 GB98 tok/s16K ctx
dense
👁 BigCode
StarCoder 7B
A72
7B18.9 GB98 tok/s8K ctx
dense
👁 DevStral AI
DevStral 7B
A72
7B13.5 GB98 tok/s8K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 14B
A71
14B18.8 GB59 tok/s33K ctx
dense
👁 Mistral AI
Codestral Mamba 7B
A71
7B12.1 GB98 tok/s262K ctx
state-space
👁 IBM
Granite 4.1 8B
A71
8B14.6 GB103 tok/s131K ctx
dense
👁 Nous Research
Nous Hermes 1.0
A71
9B25.0 GB85 tok/s16K ctx
dense
👁 Nous Research
Nous Dolphin 13B
A70
13B28.9 GB51 tok/s16K ctx
dense
👁 Mistral AI
Pixtral 12B
A70
12B17.1 GB69 tok/s131K ctx
dense
👁 WizardLM
WizardLM 13B
A70
13B27.4 GB59 tok/s8K ctx
dense
👁 LMSYS
Vicuna 13B
A70
13B27.4 GB59 tok/s4K ctx
dense
B

B Tier -- Good Fit (54 models)

Solid performance, may need quantization or reduced context.

👁 TII
Falcon 40B Instruct
B70
40B37.9 GB18 tok/s8K ctx
dense
👁 Mixedbread AI
mxbai Embed Large
B70
0.34B9.5 GB5 tok/s1K ctx
dense
👁 Snowflake
Snowflake Arctic Embed L
B69
0.34B9.5 GB5 tok/s1K ctx
dense
👁 NVIDIA
Nemotron 70B
B69
70B54.9 GB12 tok/s46K ctx
dense
👁 InternLM
InternLM 7B
B69
7B19.4 GB98 tok/s8K ctx
dense
👁 InternLM
InternLM Chat 7B
B69
7B19.4 GB98 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 3B
B69
3B11.3 GB42 tok/s131K ctx
dense
👁 Google
Gemma 2 27B
B69
27B35.0 GB23 tok/s8K ctx
dense
👁 Zhipu
GLM-4 9B
B68
9B13.4 GB93 tok/s128K ctx
dense
👁 Meta
Llama 3.1 8B
B68
8B14.1 GB103 tok/s128K ctx
dense
👁 Google
Gemma 4 E2B
B68
5.1B10.9 GB71 tok/s128K ctx
dense
B67
7B14.5 GB98 tok/s8K ctx
dense
👁 BAAI
BGE Large EN v1.5
B67
0.34B9.5 GB5 tok/s1K ctx
dense
👁 WizardLM
WizardMath 7B
B67
7B13.5 GB98 tok/s4K ctx
dense
👁 Allen AI
OLMo 2 7B
B67
7B13.5 GB98 tok/s4K ctx
dense
👁 Mistral
Mixtral 8x7B
B67
47B37.9 GB34 tok/s33K ctx
moe
👁 LLaVA
LLaVA 1.5 7B
B66
7B19.4 GB98 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 7B
B66
7B12.4 GB98 tok/s131K ctx
dense
👁 Baichuan
Baichuan 13B
B66
13B28.9 GB51 tok/s8K ctx
dense
👁 Nomic AI
Nomic Embed Text v1.5
B66
0.14B8.1 GB2 tok/s8K ctx
dense
👁 Google
Gemma 3 4B
B65
4B11.8 GB56 tok/s128K ctx
dense
👁 Mistral
Ministral 3 3B
B65
3B9.9 GB42 tok/s262K ctx
multimodal
👁 Cerebras
Cerebras-GPT 13B
B65
13B26.4 GB51 tok/s78K ctx
dense
👁 Microsoft
Phi 4 Mini 4B
B65
4B11.2 GB56 tok/s128K ctx
dense
👁 MosaicML
MPT-7B-Instruct
B64
7B19.4 GB98 tok/s8K ctx
dense
👁 Baichuan
Baichuan 7B
B64
7B19.4 GB98 tok/s8K ctx
dense
👁 Alibaba
Qwen 3.5 2B
B64
2B10.2 GB28 tok/s131K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 8B
B63
8B14.1 GB103 tok/s33K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 7B
B63
7B12.4 GB98 tok/s33K ctx
dense
7B13.5 GB98 tok/s4K ctx
dense
👁 Microsoft
Phi 3 Mini 3.8B
B62
3.8B15.5 GB53 tok/s128K ctx
dense
👁 Alibaba
Qwen 2.5 3B
B62
3B11.3 GB42 tok/s131K ctx
dense
👁 TII
Falcon 7B Instruct
B62
7B11.7 GB98 tok/s8K ctx
dense
👁 Google
Gemma 2 9B
B61
9B17.9 GB68 tok/s8K ctx
dense
👁 Microsoft
Phi 3.5 Mini 4B
B61
4B15.6 GB56 tok/s128K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 14B
B61
14B18.8 GB59 tok/s131K ctx
dense
👁 Meta
Llama 3.2 11B Vision
B61
11B16.0 GB75 tok/s16K ctx
vision
👁 01.AI
Yi 1.5 34B
B61
34B31.7 GB25 tok/s4K ctx
dense
👁 Alibaba
Qwen 3 1.7B
B61
1.7B10.0 GB24 tok/s33K ctx
dense
👁 Alibaba
Qwen 2.5 Math 72B
B61
72B56.1 GB12 tok/s4K ctx
dense
B60
0.02B7.6 GB2 tok/s0K ctx
dense
👁 InternLM
InternLM 20B
B60
20B40.0 GB33 tok/s8K ctx
dense
👁 IBM
Granite 4.1 3B
B60
3B10.4 GB42 tok/s131K ctx
dense
👁 IBM
Granite Code 3B
B59
3B11.6 GB42 tok/s8K ctx
dense
👁 Mistral
Mistral Nemo 12B
B59
12B17.1 GB69 tok/s128K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 1.5B
B59
1.5B8.6 GB21 tok/s33K ctx
dense
👁 01.AI
Yi Coder 9B
B59
9B14.3 GB93 tok/s131K ctx
dense
👁 Mistral
Mistral 7B Instruct v0.3
B59
7B13.5 GB98 tok/s8K ctx
dense
👁 Meta
Llama 4 Scout 17B 16E
B58
109B76.7 GB9 tok/s4K ctx
moe
👁 Microsoft
Phi 3 Medium 14B
B58
14B18.9 GB59 tok/s128K ctx
dense
👁 DeepSeek
DeepSeek LLM 67B
B57
67B54.0 GB13 tok/s4K ctx
dense
👁 Mistral AI
Codestral 22B
B57
22B23.2 GB36 tok/s33K ctx
dense
👁 Mistral
Ministral 8B
B56
8B14.4 GB103 tok/s131K ctx
dense
👁 Meta
Llama 3.2 3B
B55
3B10.8 GB42 tok/s128K ctx
dense
C

C Tier -- Tight Fit (187 models)

Runs with offloading or significant compromises.

👁 Cohere
Aya Expanse 32B
C53
32B29.3 GB26 tok/s8K ctx
dense
👁 Cohere
Command R+ 104B
C52
104B74.2 GB4 tok/s4K ctx
dense
👁 IBM
Granite 3.1 8B
C51
8B14.1 GB112 tok/s128K ctx
state-space
👁 01.AI
Yi 1.5 9B
C51
9B14.3 GB93 tok/s4K ctx
dense
C51
3B11.1 GB42 tok/s128K ctx
dense
👁 DeepSeek
DeepSeek R1 1.5B
C51
1.5B8.6 GB21 tok/s33K ctx
dense
👁 TinyLlama
TinyLlama 1.1B
C51
1.1B8.3 GB15 tok/s4K ctx
dense
Qwen3 48B A4B Savant Commander Distill 12X Closed Open Heretic Uncensored
C50
48B42.2 GB16 tok/s78K ctx
dense
👁 Alibaba
Qwen 2.5 Math 7B
C50
7B12.4 GB98 tok/s4K ctx
dense
👁 01.AI
Yi 34B Chat
C50
34B31.7 GB25 tok/s157K ctx
dense
👁 Stability AI
StableLM 2 12B
C49
12B28.1 GB51 tok/s4K ctx
dense
👁 LMSYS
Vicuna 7B
C49
7B19.4 GB98 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 35B A3B
C49
35B32.8 GB22 tok/s138K ctx
dense
Qwen3.5 35B A3B
C48
35B32.8 GB22 tok/s138K ctx
dense
👁 OpenChat
OpenChat 7B
C48
7B13.5 GB98 tok/s8K ctx
dense
Hermes 4.3 36B
C48
36B33.5 GB21 tok/s132K ctx
dense
👁 Alibaba
Qwen 2.5 1.5B
C48
1.5B8.6 GB21 tok/s131K ctx
dense
👁 Cohere
Aya Expanse 8B
C48
8B14.1 GB103 tok/s8K ctx
dense
👁 BigCode
StarCoder2 15B
C48
15B19.3 GB48 tok/s16K ctx
dense
EXAONE 4.0 32B
C48
32B30.6 GB24 tok/s159K ctx
dense
aya expanse 32b heretic MPOA i1
C48
32B30.6 GB24 tok/s159K ctx
dense
👁 Unsloth
Qwen3.5 27B
C47
27B26.9 GB28 tok/s203K ctx
dense
Baichuan M2 32B Q4 K M
C47
32B30.6 GB24 tok/s159K ctx
dense
baichuan inc Baichuan M2 32B
C47
32B30.6 GB24 tok/s159K ctx
dense
👁 Unsloth
gemma 3 27b it
C47
27B26.9 GB28 tok/s203K ctx
dense
8B14.1 GB103 tok/s33K ctx
dense
Llama 3.3 70B Instruct
C47
70B58.2 GB11 tok/s27K ctx
dense
👁 Teknium
OpenHermes 2.5 7B
C47
7B13.5 GB98 tok/s8K ctx
dense
gemma 3 27b it
C47
27B26.9 GB28 tok/s203K ctx
dense
👁 Hugging Face H4
Zephyr 7B Beta
C47
7B13.5 GB98 tok/s33K ctx
dense
👁 Unsloth
Mistral Small 3.2 24B Instruct 2506
C47
24B24.8 GB32 tok/s239K ctx
dense
Solar Open 69B REAP i1
C47
69B57.5 GB11 tok/s29K ctx
dense
cognitivecomputations Dolphin Mistral 24B Venice Edition
C47
24B24.8 GB32 tok/s239K ctx
dense
Mistral Small 24B Instruct 2501
C47
24B24.8 GB32 tok/s239K ctx
dense
mistral small 3.1 24b instruct 2503 hf
C47
24B24.8 GB32 tok/s239K ctx
dense
cognitivecomputations Dolphin3.0 R1 Mistral 24B
C47
24B24.8 GB32 tok/s239K ctx
dense
stabilityai japanese stablelm instruct beta 70b
C47
70B58.2 GB11 tok/s27K ctx
dense
👁 UC Berkeley
Starling LM 7B
C47
7B13.5 GB98 tok/s8K ctx
dense
Dolphin Mistral GLM 4.7 Flash 24B Venice Edition Thinking Uncensored i1
C47
24B24.8 GB32 tok/s239K ctx
dense
👁 Unsloth
Qwen3.5 9B
C47
9B13.8 GB85 tok/s777K ctx
dense
Qwen3.5 9B Uncensored HauhauCS Aggressive
C47
9B13.8 GB85 tok/s777K ctx
dense
👁 Google
Gemma 3 1B
C47
1B8.3 GB14 tok/s33K ctx
dense
Meta Llama 3.1 8B Instruct
C47
8B13.1 GB96 tok/s884K ctx
dense
llava llama 3 8b v1 1
C47
8B13.1 GB96 tok/s884K ctx
dense
👁 DeepSeek
DeepSeek LLM 7B
C47
7B18.9 GB98 tok/s4K ctx
dense
👁 Unsloth
DeepSeek R1 0528 Qwen3 8B
C47
8B13.1 GB96 tok/s884K ctx
dense
Qwen3.5 9B
C47
9B13.8 GB85 tok/s777K ctx
dense
Codestral 22B v0.1
C47
22B23.3 GB35 tok/s269K ctx
dense
Codestral 22B v0.1
C46
22B23.3 GB35 tok/s269K ctx
dense
BaichuanMed OCR 72B i1
C46
72B59.7 GB11 tok/s24K ctx
dense
Meta Llama 3 8B Instruct
C46
8B13.1 GB96 tok/s884K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Llama 8B
C46
8B13.1 GB96 tok/s884K ctx
dense
Dolphin3.0 Llama3.1 8B
C46
8B13.1 GB96 tok/s884K ctx
dense
Llama 3 8B Instruct 32k v0.1
C46
8B13.1 GB96 tok/s884K ctx
dense
Meta Llama 3.1 8B Instruct
C46
8B13.1 GB96 tok/s884K ctx
dense
cognitivecomputations Dolphin Mistral 24B Venice Edition
C46
24B24.8 GB32 tok/s239K ctx
dense
vntl llama3 8b v2
C46
8B13.1 GB96 tok/s884K ctx
dense
DeepSeek R1 0528 Qwen3 8B
C46
8B13.1 GB96 tok/s884K ctx
dense
Yi Coder 9B Chat
C46
9B13.8 GB85 tok/s777K ctx
dense
Llama 2 7B Chat
C46
7B12.4 GB98 tok/s1023K ctx
dense
DeepSeek R1 0528 Qwen3 8B
C46
8B13.1 GB96 tok/s884K ctx
dense
glm 4 9b chat 1m
C46
9B13.8 GB85 tok/s777K ctx
dense
Codestral 22B v0.1
C46
22B23.3 GB35 tok/s269K ctx
dense
Qwen3 8B DeepSeek v3.2 Speciale Distill
C46
8B13.1 GB96 tok/s884K ctx
dense
Mistral 7B Instruct v0.2
C46
7B12.4 GB98 tok/s1023K ctx
dense
Hermes 3 Llama 3.1 8B
C46
8B13.1 GB96 tok/s884K ctx
dense
Mistral 7B Instruct v0.3
C46
7B12.4 GB98 tok/s1023K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Qwen 14B
C46
14B17.5 GB55 tok/s470K ctx
dense
Hermes 2 Pro Llama 3 8B
C46
8B13.1 GB96 tok/s884K ctx
dense
Dolphin3.0 Llama3.1 8B
C46
8B13.1 GB96 tok/s884K ctx
dense
gemma 3 12b it
C46
12B16.0 GB64 tok/s562K ctx
dense
Yi 1.5 9B Chat
C46
9B13.8 GB85 tok/s777K ctx
dense
granite 8b code instruct 4k
C46
8B13.1 GB96 tok/s884K ctx
dense
Codestral 22B v0.1 IMat
C46
22B23.3 GB35 tok/s269K ctx
dense
dolphin 2.9.4 llama3.1 8b
C46
8B13.1 GB96 tok/s884K ctx
dense
👁 Google
Gemma 2 2B
C46
2B10.1 GB28 tok/s8K ctx
dense
SOLAR 10.7B Instruct v1.0 uncensored
C46
10.7B15.1 GB72 tok/s640K ctx
dense
openchat 3.6 8b 20240522 IMat
C46
8B13.1 GB96 tok/s884K ctx
dense
Nous Hermes 2 SOLAR 10.7B
C46
10.7B15.1 GB72 tok/s640K ctx
dense
aya expanse 8b
C46
8B13.1 GB96 tok/s884K ctx
dense
EXAONE 3.5 7.8B Instruct
C46
7.8B13.0 GB98 tok/s909K ctx
dense
Codestral 22B v0.1 i1
C46
22B23.3 GB35 tok/s269K ctx
dense
Mistral 7B Instruct v0.3
C46
7B12.4 GB98 tok/s1023K ctx
dense
dolphin v2 8b abliterated i1
C46
8B13.1 GB96 tok/s884K ctx
dense
Hermes 2 Pro Mistral 7B
C46
7B12.4 GB98 tok/s1023K ctx
dense
Phi 4 reasoning vision 15B
C46
15B18.2 GB51 tok/s433K ctx
dense
EXAONE 3.5 7.8B Instruct
C46
7.8B13.0 GB98 tok/s909K ctx
dense
Codestral 21B Pruned i1
C46
21B22.6 GB37 tok/s285K ctx
dense
zephyr 7B beta
C46
7B12.4 GB98 tok/s1023K ctx
dense
StarCoder2 15B
C46
15B18.2 GB51 tok/s433K ctx
dense
internlm2 5 20b chat
C46
20B21.8 GB38 tok/s304K ctx
dense
NousResearch Hermes 4 14B
C46
14B17.5 GB55 tok/s470K ctx
dense
EXAONE 3.5 7.8B Instruct i1
C46
7.8B13.0 GB98 tok/s909K ctx
dense
aya expanse 8b orthogonal heretic i1
C46
8B13.1 GB96 tok/s884K ctx
dense
Nous Hermes 2 Mistral 7B DPO
C46
7B12.4 GB98 tok/s1023K ctx
dense
Yi 9B Coder i1
C46
9B13.8 GB85 tok/s777K ctx
dense
internlm2 limarp chat 20b
C46
20B21.8 GB38 tok/s304K ctx
dense
exaone 3.0 7.8b it
C46
7.8B13.0 GB98 tok/s909K ctx
dense
Falcon H1R 7B
C46
7B12.4 GB98 tok/s1023K ctx
dense
falcon mamba 7b instruct Q4 K M
C46
7B12.4 GB98 tok/s1023K ctx
dense
internlm2 math plus 20b i1
C46
20B21.8 GB38 tok/s304K ctx
dense
aya expanse 8b orthogonal heretic
C46
8B13.1 GB96 tok/s884K ctx
dense
HelpingAI2 9B
C46
9B13.8 GB85 tok/s777K ctx
dense
zephyr 7B alpha
C46
7B12.4 GB98 tok/s1023K ctx
dense
speechless zephyr code functionary 7b
C46
7B12.4 GB98 tok/s1023K ctx
dense
StarCoder2 7B
C46
7B12.4 GB98 tok/s1023K ctx
dense
solar finalised finetuned Model 10.7B i1
C46
10.7B15.1 GB72 tok/s640K ctx
dense
HelpingAI2 9B i1
C46
9B13.8 GB85 tok/s777K ctx
dense
japanese stablelm instruct gamma 7B
C46
7B12.4 GB98 tok/s1023K ctx
dense
HelpingAI 9B 200k i1
C46
9B13.8 GB85 tok/s777K ctx
dense
internlm2 5 1 8b chat i1
C46
8B13.1 GB96 tok/s884K ctx
dense
Falcon H1 7B Instruct
C46
7B12.4 GB98 tok/s1023K ctx
dense
internlm3 8b instruct abliterated i1
C46
8B13.1 GB96 tok/s884K ctx
dense
👁 Unsloth
Falcon H1R 7B
C46
7B12.4 GB98 tok/s1023K ctx
dense
starcoder2 15b instruct v0.1
C46
15B18.2 GB51 tok/s433K ctx
dense
Codestral RAG 19B Pruned i1
C46
19B21.1 GB40 tok/s324K ctx
dense
HelpingAI 9B i1
C46
9B13.8 GB85 tok/s777K ctx
dense
SOLAR 10.7B v1.0
C46
10.7B15.1 GB72 tok/s640K ctx
dense
starcoder2 15b i1
C46
15B18.2 GB51 tok/s433K ctx
dense
GGUF SOLARized GraniStral 14B 2102 YeAM HCT 32QKV
C46
14B17.5 GB55 tok/s470K ctx
dense
starcoder2 15b instruct v0.1
C46
15B18.2 GB51 tok/s433K ctx
dense
internlm JanusCoder 14B
C46
14B17.5 GB55 tok/s470K ctx
dense
HelpingAI2.5 10B i1
C46
10B14.6 GB77 tok/s691K ctx
dense
GGUF SOLARized GraniStral 14B 1902 YeAM HCT
C46
14B17.5 GB55 tok/s470K ctx
dense
Mamba Codestral 7B v0.1
C46
7B12.4 GB98 tok/s1023K ctx
dense
👁 Intel
Neural Chat 7B
C46
7B13.5 GB98 tok/s8K ctx
dense
stabilityai japanese stablelm base gamma 7b
C46
7B12.4 GB98 tok/s1023K ctx
dense
baichuan2 7b chat
C46
7B12.4 GB98 tok/s1023K ctx
dense
Helply 10.2b chat i1
C46
10.2B14.7 GB75 tok/s676K ctx
dense
HelpingAI 15B i1
C46
15B18.2 GB51 tok/s433K ctx
dense
DiscoPOP zephyr 7b gemma
C46
7B12.4 GB98 tok/s1023K ctx
dense
Yi 1.5 6B Chat
C46
6B11.7 GB84 tok/s1207K ctx
dense
starcoder2 7b
C46
7B12.4 GB98 tok/s1023K ctx
dense
zephyr 7b beta Mistral 7B Instruct v0.2
C46
7B12.4 GB98 tok/s1023K ctx
dense
OpenChat 3.5 7B Qwen v2.0 i1
C46
7B12.4 GB98 tok/s1023K ctx
dense
OpenChat 3.5 7B Starling v2.0 i1
C46
7B12.4 GB98 tok/s1023K ctx
dense
internlm2 math plus 7b IMat
C46
7B12.4 GB98 tok/s1023K ctx
dense
CodeNinja 1.0 OpenChat 7B i1
C46
7B12.4 GB98 tok/s1023K ctx
dense
internlm2 5 7b chat i1
C46
7B12.4 GB98 tok/s1023K ctx
dense
stablelm 2 zephyr 1 6b
C46
6B11.7 GB84 tok/s1207K ctx
dense
OpenSafetyLab MD Judge v0 2 internlm2 7b
C46
7B12.4 GB98 tok/s1023K ctx
dense
MD Judge v0 2 internlm2 7b i1
C46
7B12.4 GB98 tok/s1023K ctx
dense
zephyr 7b gemma sft african ultrachat 100k
C45
7B12.4 GB98 tok/s1023K ctx
dense
jointpreferences mistral 7b sft helpful
C45
7B12.4 GB98 tok/s1023K ctx
dense
zephyr 7b dpo full i1
C45
7B12.4 GB98 tok/s1023K ctx
dense
blossom v3 baichuan2 7b i1
C45
7B12.4 GB98 tok/s1023K ctx
dense
blossom v1 baichuan 7b i1
C45
7B12.4 GB98 tok/s1023K ctx
dense
👁 01.AI
Yi 1.5 6B
C45
6B11.9 GB84 tok/s4K ctx
dense
Yi 1.5 6B Chat
C45
6B11.7 GB84 tok/s1207K ctx
dense
stablelm 2 1 6b chat imatrix
C45
6B11.7 GB84 tok/s1207K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 0.5B
C45
0.5B7.8 GB7 tok/s131K ctx
dense
👁 BigCode
StarCoder2 7B
C45
7B12.1 GB98 tok/s16K ctx
dense
HelpingAI2 6B
C45
6B11.7 GB84 tok/s1207K ctx
dense
HelpingAI2 6B i1
C45
6B11.7 GB84 tok/s1207K ctx
dense
👁 NVIDIA
Nemotron Mini 4B
C45
4B11.7 GB56 tok/s4K ctx
dense
HelpingAI2.5 5B i1
C44
5B10.9 GB70 tok/s1465K ctx
dense
👁 Unsloth
Qwen3.5 4B
C44
4B10.2 GB56 tok/s1852K ctx
dense
gemma 3 4b it
C44
4B10.2 GB56 tok/s1852K ctx
dense
gemma 3 4b it
C44
4B10.2 GB56 tok/s1852K ctx
dense
👁 Alibaba
Qwen 3 0.6B
C43
0.6B8.5 GB8 tok/s33K ctx
dense
👁 Meta
Llama 3.2 1B
C43
1B8.4 GB14 tok/s128K ctx
dense
Llama 3.2 3B Instruct
C43
3B9.8 GB42 tok/s2482K ctx
dense
👁 Qwen
Qwen2.5 3B Instruct
C43
3B9.5 GB42 tok/s2497K ctx
dense
Llama 3.2 3B Instruct
C43
3B9.5 GB42 tok/s2497K ctx
dense
Ministral 3 3B Instruct 2512
C43
3B9.5 GB42 tok/s2497K ctx
dense
HELVETE 3B
C42
3B9.5 GB42 tok/s2497K ctx
dense
Hermes 3 Llama 3.2 3B
C42
3B9.5 GB42 tok/s2497K ctx
dense
stablelm zephyr 3b
C42
3B9.5 GB42 tok/s2497K ctx
dense
StarCoder2 3B
C42
3B9.5 GB42 tok/s2497K ctx
dense
AI21 Jamba Reasoning 3B
C42
3B9.5 GB42 tok/s2497K ctx
dense
stablelm 3b 4e1t
C42
3B9.5 GB42 tok/s2497K ctx
dense
ai21labs AI21 Jamba Reasoning 3B
C42
3B9.5 GB42 tok/s2497K ctx
dense
ai21labs AI21 Jamba2 3B
C42
3B9.5 GB42 tok/s2497K ctx
dense
HelpingAI 3B hindi i1
C42
3B9.5 GB42 tok/s2497K ctx
dense
AI21 Jamba2 3B
C42
3B9.5 GB42 tok/s2497K ctx
dense
HelpingAI 3B hindi
C42
3B9.5 GB42 tok/s2497K ctx
dense
AI21 Jamba2 3B i1
C42
3B9.5 GB42 tok/s2497K ctx
dense
gemma 2 2b it
C41
2B9.2 GB28 tok/s3759K ctx
dense
EXAONE 3.5 2.4B Instruct
C41
2.4B9.0 GB34 tok/s3142K ctx
dense
👁 Google
gemma 2b
C41
2B8.8 GB28 tok/s3787K ctx
dense
Gemmasutra Mini 2B v1
C41
2B8.8 GB28 tok/s3787K ctx
dense
👁 BigCode
StarCoder2 3B
C41
3B9.6 GB42 tok/s16K ctx
dense
gemma 2 2b it
C41
2B8.8 GB28 tok/s3787K ctx
dense
👁 Alibaba
Qwen 3.5 0.6B
C41
0.6B8.5 GB8 tok/s131K ctx
dense
👁 Qwen
Qwen2.5 1.5B Instruct
C40
1.5B8.4 GB21 tok/s5078K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Qwen 1.5B
C40
1.5B8.4 GB21 tok/s5078K ctx
dense
Yi Coder 1.5B Chat
C40
1.5B8.4 GB21 tok/s5078K ctx
dense
F

F Tier -- Can't Run (52 models)

Too heavy for this hardware.

stablelm 2 zephyr 1.6b
D40
1.6B8.5 GB22 tok/s4755K ctx
dense
logos16v2 stablelm2 1.6b i1
D40
1.6B8.5 GB22 tok/s4755K ctx
dense
Yi Coder 1.5B
D40
1.5B8.4 GB21 tok/s5078K ctx
dense
👁 Unsloth
Falcon H1 1.5B Instruct
D40
1.5B8.4 GB21 tok/s5078K ctx
dense
TinyLlama 1.1B Chat v1.0
D39
1.1B8.1 GB15 tok/s6954K ctx
dense
👁 Alibaba
Qwen 2.5 0.5B
D39
0.5B7.8 GB7 tok/s131K ctx
dense
EXAONE 4.0 1.2B
D39
1.2B8.2 GB17 tok/s6368K ctx
dense
Llama 3.2 1B Instruct Q8 0
D39
1B8.2 GB14 tok/s7629K ctx
dense
TinyLlama 1.1B Chat v0.3
D39
1.1B8.1 GB15 tok/s6954K ctx
dense
👁 TinyLlama
TinyLlama 1.1B Chat v0.6
D39
1.1B8.1 GB15 tok/s6954K ctx
dense
Llama 3.2 1B Instruct
D39
1B8.0 GB14 tok/s7658K ctx
dense
gemma 3 1b it
D39
1B8.0 GB14 tok/s7658K ctx
dense
TinyLlama 1.1B Chat v1.0 imatrix
D38
1.1B8.1 GB15 tok/s6954K ctx
dense
Falcon3 1B Instruct abliterated
D38
1B8.0 GB14 tok/s7658K ctx
dense
SmolVLM 500M Instruct
D37
0.5B7.8 GB7 tok/s9006K ctx
dense
embeddinggemma 300M
D35
0.3B7.6 GB4 tok/s9033K ctx
dense
granite embedding 107m multilingual
D33
0.11B7.5 GB2 tok/s9062K ctx
dense
Falcon H1 Tiny 90M Instruct
D33
0.09B7.5 GB2 tok/s9063K ctx
dense
👁 Alibaba
Qwen 3 235B A22B
F0
235B153.5 GB2 tok/s4K ctx
moe
👁 Alibaba
Qwen3-Coder 480B A35B Instruct
F0
480B303.0 GB2 tok/s4K ctx
moe
👁 Alibaba
Qwen 3.5 122B A10B
F0
122B84.2 GB8 tok/s4K ctx
moe
👁 Alibaba
Qwen 3.5 397B A17B
F0
397B252.3 GB2 tok/s4K ctx
moe
👁 Mistral
Mixtral 8x22B
F0
141B96.7 GB4 tok/s4K ctx
moe
👁 Mistral
Devstral 2 123B Instruct
F0
123B87.7 GB3 tok/s4K ctx
dense
👁 Mistral
Mistral Small 4 119B
F0
119B85.3 GB8 tok/s4K ctx
moe
👁 Mistral
Leanstral 119B A6B
F0
119B88.7 GB7 tok/s4K ctx
moe
👁 Mistral
Mistral Large 3
F0
675B424.4 GB2 tok/s4K ctx
+1moe
F0
744B480.2 GB2 tok/s4K ctx
moe
👁 Z.ai
GLM-5.1
F0
754B486.3 GB2 tok/s4K ctx
moe
F0
1000B624.7 GB2 tok/s4K ctx
moe
F0
1000B624.7 GB2 tok/s4K ctx
+1moe
👁 DeepSeek
DeepSeek V3 671B
F0
671B476.2 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V3.1 671B
F0
671B476.2 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V4 Pro
F0
1600B871.2 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V4 Flash
F0
284B166.6 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek R1 671B
F0
671B476.2 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek Coder V2 236B
F0
236B209.9 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V2.5 236B
F0
236B209.9 GB2 tok/s4K ctx
moe
👁 Meta
Llama 4 Maverick 17B 128E
F0
400B254.2 GB2 tok/s4K ctx
moe
👁 Cohere
Command A 111B
F0
111B78.9 GB4 tok/s4K ctx
dense
👁 Mistral AI
Pixtral Large 124B
F0
124B88.3 GB3 tok/s4K ctx
dense
👁 Meta
Llama 3.1 405B
F0
405B262.0 GB2 tok/s4K ctx
dense
👁 OpenAI
GPT-OSS 120B
F0
117B83.6 GB3 tok/s4K ctx
dense
👁 DeepSeek
DeepSeek V3.2
F0
671B417.1 GB2 tok/s4K ctx
moe
MiniMax M2.7
F0
230B151.4 GB2 tok/s4K ctx
moe
Solar Open 100B
F0
100B80.0 GB4 tok/s4K ctx
dense
K EXAONE 236B A23B
F0
236B178.9 GB2 tok/s4K ctx
dense
Baichuan M3 235B
F0
235B178.2 GB2 tok/s4K ctx
dense
Baichuan M3 235B i1
F0
235B178.2 GB2 tok/s4K ctx
dense
Solar Open 100B i1
F0
100B80.0 GB4 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 122B A10B
F0
122B81.4 GB3 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 397B A17B
F0
397B296.0 GB2 tok/s4K ctx
dense

Compare with...

See How Other Hardware Stacks Up

Check tier lists for similar GPUs and Apple Silicon configs

AMD Instinct MI210 64GB

64 GB VRAM

View tier list

RTX 6000 Ada 48GB

48 GB VRAM

View tier list

RTX PRO 5000 Blackwell 48GB

48 GB VRAM

View tier list

NVIDIA A40 48GB

48 GB VRAM

View tier list
Full NVIDIA A16 64GB pageCompare this hardware