VOOZH about

URL: https://willitrunai.com/my-results/l40s-48gb

⇱ NVIDIA L40S 48GB AI Tier List: 320 models ranked | WillItRunAI


AI Compatibility Tier List

NVIDIA L40S 48GB

374 AI models ranked by how well they run on your 48 GB VRAM hardware. Best match: Qwen 3.6 35B A3B (score: 98).

374

Models tested

320

Can run

48 GB

VRAM

92

Best tok/s

Tier Distribution

S 22
A 59
B 51
C 188
F 54
S: 22A: 59B: 51C: 188F: 54
S

S Tier -- Perfect Fit (22 models)

These models run flawlessly with headroom to spare.

👁 Alibaba
Qwen 3.6 35B A3B
S98
35B31.2 GB92 tok/s82K ctx
+1moe
👁 Alibaba
Qwen 3.5 35B A3B
S96
35B28.5 GB100 tok/s131K ctx
moe
👁 Alibaba
Qwen3-Coder 30B A3B Instruct
S96
30.5B25.8 GB73 tok/s256K ctx
moe
👁 Alibaba
Qwen3-VL 30B A3B Instruct
S95
30B25.5 GB105 tok/s256K ctx
moe
👁 Alibaba
Qwen 3 30B A3B
S93
30.5B25.8 GB73 tok/s131K ctx
moe
👁 Alibaba
Qwen 3.5 27B
S92
27B25.3 GB31 tok/s130K ctx
dense
👁 NVIDIA
Nemotron Cascade 2 30B A3B
S92
30B26.9 GB104 tok/s131K ctx
moe
👁 Alibaba
Qwen 3 32B
S91
32B29.1 GB19 tok/s93K ctx
dense
👁 Mistral
Magistral Small 2507
S90
24B22.8 GB35 tok/s131K ctx
dense
👁 Mistral
Devstral Small 2 24B Instruct
S90
24B22.8 GB35 tok/s181K ctx
dense
👁 Alibaba
Qwen 3 14B
S90
14B16.7 GB98 tok/s131K ctx
dense
👁 OpenAI
GPT-OSS 20B
S90
21B21.0 GB116 tok/s128K ctx
moe
👁 Alibaba
Qwen 3.6 27B
S90
27B23.1 GB20 tok/s262K ctx
+1dense
👁 Alibaba
Qwen 3.5 9B
S89
9B13.4 GB111 tok/s131K ctx
dense
👁 NVIDIA
Nemotron 3 Nano 30B
S89
30B26.4 GB23 tok/s131K ctx
dense
👁 Microsoft
Phi-4-reasoning-plus 14B
S89
14.7B17.7 GB83 tok/s33K ctx
dense
👁 Mistral
Devstral Small 1.1
S88
24B22.8 GB35 tok/s131K ctx
dense
👁 Google
Gemma 4 26B A4B
S88
25.2B24.7 GB99 tok/s118K ctx
moe
👁 Google
Gemma 4 31B
S88
30.7B39.1 GB15 tok/s26K ctx
dense
👁 Alibaba
Qwen 3 8B
S88
8B12.8 GB120 tok/s131K ctx
dense
👁 LG AI
EXAONE 4.0 32B
S85
32B29.1 GB22 tok/s93K ctx
dense
👁 Alibaba
Qwen 3.5 4B
S85
4B10.3 GB60 tok/s131K ctx
dense
A

A Tier -- Great Fit (59 models)

Excellent performance with comfortable memory margins.

👁 Mistral
Ministral 3 14B
A84
14B16.7 GB89 tok/s221K ctx
multimodal
👁 Google
Gemma 3 27B
A84
27B33.4 GB19 tok/s37K ctx
dense
👁 Alibaba
Qwen 2.5 32B
A84
32B29.1 GB22 tok/s93K ctx
dense
👁 Moonshot AI
Kimi Linear 48B A3B
A83
48B35.9 GB14 tok/s225K ctx
linear
👁 Mistral
Mistral Small 3.2 24B
A83
24B22.8 GB35 tok/s131K ctx
vision
👁 Mistral
Codestral 2 25.08
A83
22B21.6 GB37 tok/s189K ctx
dense
👁 NVIDIA
Nemotron Nano 8B
A83
8B12.5 GB128 tok/s131K ctx
dense
👁 Allen AI
OLMo 2 32B
A83
32B29.1 GB22 tok/s4K ctx
dense
A82
24B22.8 GB35 tok/s131K ctx
dense
👁 IBM
Granite 4.1 30B
A82
30B27.9 GB23 tok/s98K ctx
dense
A82
19B19.7 GB52 tok/s8K ctx
dense
👁 Microsoft
Phi-4 Mini Reasoning 4B
A82
3.8B9.5 GB61 tok/s131K ctx
dense
A81
14B17.3 GB89 tok/s16K ctx
dense
👁 Mistral
Mistral Small 24B
A81
24B22.8 GB35 tok/s33K ctx
dense
👁 Alibaba
Qwen 2.5 14B
A80
14B17.2 GB90 tok/s131K ctx
dense
👁 Mistral AI
Mistral Small 3.1 24B
A80
24B22.8 GB35 tok/s131K ctx
dense
👁 BigCode
StarCoder 15B
A80
15B31.1 GB65 tok/s8K ctx
dense
👁 InternLM
InternVL2 8B
A80
8B12.5 GB128 tok/s8K ctx
dense
👁 Alibaba
Qwen3-Coder-Next
A79
80B56.0 GB25 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek Coder V2 16B
A79
16B18.8 GB148 tok/s131K ctx
moe
👁 IBM
Granite Code 20B
A79
20B21.1 GB48 tok/s8K ctx
dense
👁 Google
Gemma 3 12B
A79
12B17.9 GB69 tok/s115K ctx
dense
👁 Meta
CodeLlama 13B Instruct
A78
13B25.8 GB89 tok/s16K ctx
dense
👁 Mistral
Ministral 3 8B
A78
8B12.8 GB128 tok/s262K ctx
multimodal
👁 OpenBMB
MiniCPM-V 2.6 8B
A78
8B12.5 GB128 tok/s2K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 32B
A78
32B29.1 GB22 tok/s93K ctx
dense
👁 NVIDIA
Nemotron Nano 9B v2
A77
9B13.6 GB121 tok/s131K ctx
dense
👁 IBM
Granite Code 34B
A77
34B30.1 GB20 tok/s8K ctx
dense
👁 Defog
SQLCoder 7B
A77
7B11.9 GB112 tok/s8K ctx
dense
👁 LLaVA
LLaVA 1.6 13B
A77
13B25.8 GB89 tok/s4K ctx
dense
👁 Alibaba
Qwen 3 4B
A77
4B10.3 GB60 tok/s33K ctx
dense
👁 Cohere
Command R 35B
A76
35B29.5 GB20 tok/s131K ctx
dense
👁 Alibaba
Qwen 2.5 VL 7B
A76
7B10.8 GB112 tok/s33K ctx
dense
👁 AllenAI
OLMo 2 13B
A76
13B16.1 GB96 tok/s33K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 32B
A76
32B29.1 GB22 tok/s33K ctx
dense
👁 Alibaba
Qwen 2.5 VL 72B
A76
72B54.5 GB6 tok/s4K ctx
dense
👁 Jina AI
Jina Embeddings v3
A75
0.57B8.8 GB9 tok/s8K ctx
dense
👁 Magistral AI
Magistral 7B
A75
7B11.9 GB112 tok/s8K ctx
dense
👁 Tsinghua/Zhipu
CodeGeeX 4 9B
A75
9B11.8 GB124 tok/s131K ctx
dense
👁 Google
Gemma 4 E4B
A75
8B11.9 GB93 tok/s128K ctx
dense
👁 Nous Research
Nous Dolphin 13B
A74
13B27.3 GB77 tok/s16K ctx
dense
👁 BAAI
BGE M3
A74
0.57B8.0 GB9 tok/s8K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 14B
A74
14B17.2 GB90 tok/s33K ctx
dense
👁 WizardLM
WizardLM 13B
A74
13B25.8 GB89 tok/s8K ctx
dense
👁 LMSYS
Vicuna 13B
A74
13B25.8 GB89 tok/s4K ctx
dense
👁 Meta
CodeLlama 7B Instruct
A73
7B17.8 GB112 tok/s16K ctx
dense
👁 Nous Research
Nous Hermes 1.0
A73
9B23.4 GB113 tok/s16K ctx
dense
👁 BigCode
StarCoder 7B
A73
7B17.3 GB112 tok/s8K ctx
dense
👁 Alibaba
Qwen 2.5 7B
A73
7B10.8 GB112 tok/s131K ctx
dense
👁 IBM
Granite Code 8B
A73
8B12.5 GB128 tok/s8K ctx
dense
👁 Mistral AI
Pixtral 12B
A72
12B15.5 GB104 tok/s131K ctx
dense
👁 DevStral AI
DevStral 7B
A72
7B11.9 GB112 tok/s8K ctx
dense
👁 IBM
Granite 4.1 8B
A72
8B13.0 GB128 tok/s131K ctx
dense
👁 TII
Falcon 40B Instruct
A72
40B36.3 GB15 tok/s8K ctx
dense
👁 Mistral AI
Codestral Mamba 7B
A72
7B10.5 GB112 tok/s262K ctx
state-space
👁 Google
Gemma 2 27B
A71
27B33.4 GB19 tok/s8K ctx
dense
👁 Mixedbread AI
mxbai Embed Large
A71
0.34B7.9 GB5 tok/s1K ctx
dense
👁 InternLM
InternLM 7B
A70
7B17.8 GB112 tok/s8K ctx
dense
👁 InternLM
InternLM Chat 7B
A70
7B17.8 GB112 tok/s8K ctx
dense
B

B Tier -- Good Fit (51 models)

Solid performance, may need quantization or reduced context.

👁 Mistral
Mixtral 8x7B
B70
47B36.3 GB44 tok/s33K ctx
moe
👁 Snowflake
Snowflake Arctic Embed L
B70
0.34B7.9 GB5 tok/s1K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 3B
B70
3B9.7 GB48 tok/s131K ctx
dense
👁 Baichuan
Baichuan 13B
B70
13B27.3 GB77 tok/s8K ctx
dense
👁 Meta
Llama 3.3 70B
B70
70B53.3 GB6 tok/s4K ctx
dense
👁 Google
Gemma 4 E2B
B69
5.1B9.3 GB82 tok/s128K ctx
dense
👁 Zhipu
GLM-4 9B
B69
9B11.8 GB124 tok/s128K ctx
dense
👁 Cerebras
Cerebras-GPT 13B
B69
13B24.8 GB77 tok/s54K ctx
dense
👁 Meta
Llama 3.1 8B
B68
8B12.5 GB128 tok/s128K ctx
dense
B68
7B12.9 GB112 tok/s8K ctx
dense
👁 BAAI
BGE Large EN v1.5
B68
0.34B7.9 GB5 tok/s1K ctx
dense
👁 WizardLM
WizardMath 7B
B68
7B11.9 GB112 tok/s4K ctx
dense
👁 Allen AI
OLMo 2 7B
B68
7B11.9 GB112 tok/s4K ctx
dense
👁 LLaVA
LLaVA 1.5 7B
B68
7B17.8 GB112 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 7B
B67
7B10.8 GB112 tok/s131K ctx
dense
👁 Google
Gemma 3 4B
B67
4B10.2 GB64 tok/s128K ctx
dense
👁 Meta
Llama 3.1 70B
B66
70B53.3 GB6 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 72B
B66
72B54.5 GB6 tok/s4K ctx
dense
👁 Nomic AI
Nomic Embed Text v1.5
B66
0.14B6.5 GB2 tok/s8K ctx
dense
👁 MosaicML
MPT-7B-Instruct
B66
7B17.8 GB112 tok/s8K ctx
dense
👁 Baichuan
Baichuan 7B
B66
7B17.8 GB112 tok/s8K ctx
dense
👁 Mistral
Ministral 3 3B
B66
3B8.3 GB48 tok/s262K ctx
multimodal
👁 Microsoft
Phi 4 Mini 4B
B66
4B9.6 GB64 tok/s128K ctx
dense
👁 Alibaba
Qwen 3.5 2B
B64
2B8.6 GB30 tok/s131K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 8B
B64
8B12.5 GB128 tok/s33K ctx
dense
👁 Microsoft
Phi 3 Mini 3.8B
B64
3.8B13.9 GB61 tok/s109K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 14B
B64
14B17.2 GB90 tok/s131K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 7B
B64
7B10.8 GB112 tok/s33K ctx
dense
7B11.9 GB112 tok/s4K ctx
dense
👁 Google
Gemma 2 9B
B63
9B16.3 GB81 tok/s8K ctx
dense
👁 01.AI
Yi 1.5 34B
B63
34B30.1 GB20 tok/s4K ctx
dense
👁 Alibaba
Qwen 2.5 3B
B63
3B9.7 GB48 tok/s131K ctx
dense
👁 Meta
Llama 3.2 11B Vision
B63
11B14.4 GB113 tok/s16K ctx
vision
👁 Microsoft
Phi 3.5 Mini 4B
B63
4B14.0 GB64 tok/s109K ctx
dense
👁 TII
Falcon 7B Instruct
B63
7B10.1 GB112 tok/s8K ctx
dense
👁 InternLM
InternLM 20B
B62
20B38.4 GB39 tok/s8K ctx
dense
👁 DeepSeek
DeepSeek R1 Distill 70B
B62
70B53.3 GB6 tok/s4K ctx
dense
👁 Alibaba
Qwen 3 1.7B
B61
1.7B8.4 GB26 tok/s33K ctx
dense
👁 Mistral
Mistral Nemo 12B
B61
12B15.5 GB104 tok/s128K ctx
dense
👁 IBM
Granite 4.1 3B
B61
3B8.8 GB48 tok/s131K ctx
dense
👁 Microsoft
Phi 3 Medium 14B
B61
14B17.3 GB89 tok/s128K ctx
dense
👁 IBM
Granite Code 3B
B60
3B10.0 GB48 tok/s8K ctx
dense
B60
0.02B6.0 GB2 tok/s0K ctx
dense
👁 01.AI
Yi Coder 9B
B60
9B12.7 GB123 tok/s131K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 1.5B
B60
1.5B7.0 GB24 tok/s33K ctx
dense
👁 Mistral
Mistral 7B Instruct v0.3
B59
7B11.9 GB112 tok/s8K ctx
dense
👁 Mistral AI
Codestral 22B
B59
22B21.6 GB39 tok/s33K ctx
dense
👁 MosaicML
MPT-30B-Instruct
B58
30B50.7 GB12 tok/s8K ctx
dense
👁 Mistral
Ministral 8B
B57
8B12.8 GB128 tok/s131K ctx
dense
👁 NVIDIA
Nemotron 70B
B57
70B53.3 GB6 tok/s4K ctx
dense
👁 Meta
Llama 3.2 3B
B56
3B9.2 GB48 tok/s128K ctx
dense
C

C Tier -- Tight Fit (188 models)

Runs with offloading or significant compromises.

👁 Cohere
Aya Expanse 32B
C55
32B27.7 GB22 tok/s8K ctx
dense
👁 Stability AI
StableLM 2 12B
C53
12B26.5 GB72 tok/s4K ctx
dense
👁 IBM
Granite 3.1 8B
C52
8B12.5 GB128 tok/s128K ctx
state-space
👁 01.AI
Yi 1.5 9B
C52
9B12.7 GB123 tok/s4K ctx
dense
👁 01.AI
Yi 34B Chat
C52
34B30.1 GB20 tok/s94K ctx
dense
C52
3B9.5 GB48 tok/s128K ctx
dense
👁 DeepSeek
DeepSeek R1 1.5B
C51
1.5B7.0 GB24 tok/s33K ctx
dense
👁 TinyLlama
TinyLlama 1.1B
C51
1.1B6.7 GB18 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 35B A3B
C51
35B31.2 GB18 tok/s82K ctx
dense
👁 Alibaba
Qwen 2.5 Math 7B
C51
7B10.8 GB112 tok/s4K ctx
dense
Qwen3.5 35B A3B
C51
35B31.2 GB18 tok/s82K ctx
dense
Hermes 4.3 36B
C51
36B31.9 GB18 tok/s77K ctx
dense
👁 LMSYS
Vicuna 7B
C50
7B17.8 GB112 tok/s4K ctx
dense
👁 BigCode
StarCoder2 15B
C50
15B17.7 GB71 tok/s16K ctx
dense
EXAONE 4.0 32B
C50
32B29.0 GB20 tok/s97K ctx
dense
aya expanse 32b heretic MPOA i1
C50
32B29.0 GB20 tok/s97K ctx
dense
Baichuan M2 32B Q4 K M
C50
32B29.0 GB20 tok/s97K ctx
dense
baichuan inc Baichuan M2 32B
C49
32B29.0 GB20 tok/s97K ctx
dense
👁 Unsloth
Qwen3.5 27B
C49
27B25.3 GB26 tok/s131K ctx
dense
👁 Unsloth
gemma 3 27b it
C49
27B25.3 GB25 tok/s131K ctx
dense
👁 OpenChat
OpenChat 7B
C49
7B11.9 GB112 tok/s8K ctx
dense
gemma 3 27b it
C49
27B25.3 GB25 tok/s131K ctx
dense
👁 Unsloth
Mistral Small 3.2 24B Instruct 2506
C49
24B23.2 GB33 tok/s157K ctx
dense
👁 Cohere
Aya Expanse 8B
C49
8B12.5 GB128 tok/s8K ctx
dense
cognitivecomputations Dolphin Mistral 24B Venice Edition
C49
24B23.2 GB33 tok/s157K ctx
dense
Mistral Small 24B Instruct 2501
C49
24B23.2 GB33 tok/s157K ctx
dense
mistral small 3.1 24b instruct 2503 hf
C49
24B23.2 GB33 tok/s157K ctx
dense
cognitivecomputations Dolphin3.0 R1 Mistral 24B
C49
24B23.2 GB33 tok/s157K ctx
dense
👁 Alibaba
Qwen 2.5 1.5B
C49
1.5B7.0 GB24 tok/s131K ctx
dense
Dolphin Mistral GLM 4.7 Flash 24B Venice Edition Thinking Uncensored i1
C49
24B23.2 GB33 tok/s157K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Qwen 14B
C49
14B15.9 GB83 tok/s329K ctx
dense
Codestral 22B v0.1
C49
22B21.7 GB38 tok/s179K ctx
dense
Codestral 22B v0.1
C49
22B21.7 GB38 tok/s179K ctx
dense
Phi 4 reasoning vision 15B
C49
15B16.6 GB75 tok/s302K ctx
dense
StarCoder2 15B
C49
15B16.6 GB75 tok/s302K ctx
dense
NousResearch Hermes 4 14B
C49
14B15.9 GB83 tok/s329K ctx
dense
cognitivecomputations Dolphin Mistral 24B Venice Edition
C49
24B23.2 GB33 tok/s157K ctx
dense
Codestral 22B v0.1
C48
22B21.7 GB38 tok/s179K ctx
dense
👁 Alibaba
Qwen 2.5 Math 72B
C48
72B54.5 GB6 tok/s4K ctx
dense
gemma 3 12b it
C48
12B14.4 GB97 tok/s398K ctx
dense
starcoder2 15b instruct v0.1
C48
15B16.6 GB75 tok/s302K ctx
dense
Codestral 22B v0.1 IMat
C48
22B21.7 GB38 tok/s179K ctx
dense
GGUF SOLARized GraniStral 14B 2102 YeAM HCT 32QKV
C48
14B15.9 GB83 tok/s329K ctx
dense
starcoder2 15b i1
C48
15B16.6 GB75 tok/s302K ctx
dense
starcoder2 15b instruct v0.1
C48
15B16.6 GB75 tok/s302K ctx
dense
internlm JanusCoder 14B
C48
14B15.9 GB83 tok/s329K ctx
dense
GGUF SOLARized GraniStral 14B 1902 YeAM HCT
C48
14B15.9 GB83 tok/s329K ctx
dense
HelpingAI 15B i1
C48
15B16.6 GB75 tok/s302K ctx
dense
Codestral 22B v0.1 i1
C48
22B21.7 GB38 tok/s179K ctx
dense
Codestral 21B Pruned i1
C48
21B21.0 GB41 tok/s192K ctx
dense
internlm2 5 20b chat
C48
20B20.2 GB45 tok/s205K ctx
dense
internlm2 limarp chat 20b
C48
20B20.2 GB45 tok/s205K ctx
dense
8B12.5 GB128 tok/s33K ctx
dense
internlm2 math plus 20b i1
C48
20B20.2 GB45 tok/s205K ctx
dense
Codestral RAG 19B Pruned i1
C48
19B19.5 GB49 tok/s221K ctx
dense
👁 Teknium
OpenHermes 2.5 7B
C48
7B11.9 GB112 tok/s8K ctx
dense
👁 DeepSeek
DeepSeek LLM 7B
C48
7B17.3 GB112 tok/s4K ctx
dense
SOLAR 10.7B Instruct v1.0 uncensored
C48
10.7B13.5 GB108 tok/s456K ctx
dense
Nous Hermes 2 SOLAR 10.7B
C48
10.7B13.5 GB108 tok/s456K ctx
dense
👁 Hugging Face H4
Zephyr 7B Beta
C48
7B11.9 GB112 tok/s33K ctx
dense
👁 Unsloth
Qwen3.5 9B
C48
9B12.2 GB113 tok/s558K ctx
dense
Qwen3.5 9B Uncensored HauhauCS Aggressive
C48
9B12.2 GB113 tok/s558K ctx
dense
solar finalised finetuned Model 10.7B i1
C48
10.7B13.5 GB108 tok/s456K ctx
dense
Qwen3.5 9B
C48
9B12.2 GB113 tok/s558K ctx
dense
Qwen3 48B A4B Savant Commander Distill 12X Closed Open Heretic Uncensored
C48
48B40.6 GB13 tok/s37K ctx
dense
SOLAR 10.7B v1.0
C47
10.7B13.5 GB108 tok/s456K ctx
dense
👁 UC Berkeley
Starling LM 7B
C47
7B11.9 GB112 tok/s8K ctx
dense
Yi Coder 9B Chat
C47
9B12.2 GB113 tok/s558K ctx
dense
glm 4 9b chat 1m
C47
9B12.2 GB113 tok/s558K ctx
dense
Meta Llama 3.1 8B Instruct
C47
8B11.5 GB127 tok/s639K ctx
dense
llava llama 3 8b v1 1
C47
8B11.5 GB127 tok/s639K ctx
dense
👁 Unsloth
DeepSeek R1 0528 Qwen3 8B
C47
8B11.5 GB127 tok/s639K ctx
dense
👁 Google
Gemma 3 1B
C47
1B6.7 GB16 tok/s33K ctx
dense
Meta Llama 3 8B Instruct
C47
8B11.5 GB127 tok/s639K ctx
dense
Yi 1.5 9B Chat
C47
9B12.2 GB113 tok/s558K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Llama 8B
C47
8B11.5 GB127 tok/s639K ctx
dense
Dolphin3.0 Llama3.1 8B
C47
8B11.5 GB127 tok/s639K ctx
dense
Llama 3 8B Instruct 32k v0.1
C47
8B11.5 GB127 tok/s639K ctx
dense
Helply 10.2b chat i1
C47
10.2B13.1 GB114 tok/s483K ctx
dense
Meta Llama 3.1 8B Instruct
C47
8B11.5 GB127 tok/s639K ctx
dense
vntl llama3 8b v2
C47
8B11.5 GB127 tok/s639K ctx
dense
HelpingAI2.5 10B i1
C47
10B13.0 GB102 tok/s494K ctx
dense
DeepSeek R1 0528 Qwen3 8B
C47
8B11.5 GB127 tok/s639K ctx
dense
DeepSeek R1 0528 Qwen3 8B
C47
8B11.5 GB127 tok/s639K ctx
dense
Qwen3 8B DeepSeek v3.2 Speciale Distill
C47
8B11.5 GB127 tok/s639K ctx
dense
Hermes 3 Llama 3.1 8B
C47
8B11.5 GB127 tok/s639K ctx
dense
Llama 2 7B Chat
C47
7B10.8 GB112 tok/s742K ctx
dense
👁 Google
Gemma 2 2B
C47
2B8.5 GB32 tok/s8K ctx
dense
Yi 9B Coder i1
C47
9B12.2 GB113 tok/s558K ctx
dense
Mistral 7B Instruct v0.2
C47
7B10.8 GB112 tok/s742K ctx
dense
Hermes 2 Pro Llama 3 8B
C47
8B11.5 GB127 tok/s639K ctx
dense
Dolphin3.0 Llama3.1 8B
C47
8B11.5 GB127 tok/s639K ctx
dense
Mistral 7B Instruct v0.3
C47
7B10.8 GB112 tok/s742K ctx
dense
granite 8b code instruct 4k
C47
8B11.5 GB127 tok/s639K ctx
dense
HelpingAI2 9B
C47
9B12.2 GB113 tok/s558K ctx
dense
dolphin 2.9.4 llama3.1 8b
C47
8B11.5 GB127 tok/s639K ctx
dense
HelpingAI2 9B i1
C47
9B12.2 GB113 tok/s558K ctx
dense
openchat 3.6 8b 20240522 IMat
C47
8B11.5 GB127 tok/s639K ctx
dense
HelpingAI 9B 200k i1
C47
9B12.2 GB113 tok/s558K ctx
dense
aya expanse 8b
C47
8B11.5 GB127 tok/s639K ctx
dense
HelpingAI 9B i1
C47
9B12.2 GB113 tok/s558K ctx
dense
EXAONE 3.5 7.8B Instruct
C47
7.8B11.4 GB125 tok/s657K ctx
dense
dolphin v2 8b abliterated i1
C47
8B11.5 GB127 tok/s639K ctx
dense
EXAONE 3.5 7.8B Instruct
C47
7.8B11.4 GB125 tok/s657K ctx
dense
aya expanse 8b orthogonal heretic i1
C47
8B11.5 GB127 tok/s639K ctx
dense
EXAONE 3.5 7.8B Instruct i1
C47
7.8B11.4 GB125 tok/s657K ctx
dense
Mistral 7B Instruct v0.3
C47
7B10.8 GB112 tok/s742K ctx
dense
Hermes 2 Pro Mistral 7B
C47
7B10.8 GB112 tok/s742K ctx
dense
exaone 3.0 7.8b it
C47
7.8B11.4 GB125 tok/s657K ctx
dense
zephyr 7B beta
C47
7B10.8 GB112 tok/s742K ctx
dense
Nous Hermes 2 Mistral 7B DPO
C47
7B10.8 GB112 tok/s742K ctx
dense
aya expanse 8b orthogonal heretic
C47
8B11.5 GB127 tok/s639K ctx
dense
Falcon H1R 7B
C47
7B10.8 GB112 tok/s742K ctx
dense
falcon mamba 7b instruct Q4 K M
C47
7B10.8 GB112 tok/s742K ctx
dense
zephyr 7B alpha
C47
7B10.8 GB112 tok/s742K ctx
dense
speechless zephyr code functionary 7b
C47
7B10.8 GB112 tok/s742K ctx
dense
internlm2 5 1 8b chat i1
C47
8B11.5 GB127 tok/s639K ctx
dense
internlm3 8b instruct abliterated i1
C47
8B11.5 GB127 tok/s639K ctx
dense
StarCoder2 7B
C46
7B10.8 GB112 tok/s742K ctx
dense
japanese stablelm instruct gamma 7B
C46
7B10.8 GB112 tok/s742K ctx
dense
Yi 1.5 6B Chat
C46
6B10.1 GB96 tok/s879K ctx
dense
👁 Intel
Neural Chat 7B
C46
7B11.9 GB112 tok/s8K ctx
dense
Falcon H1 7B Instruct
C46
7B10.8 GB112 tok/s742K ctx
dense
👁 Unsloth
Falcon H1R 7B
C46
7B10.8 GB112 tok/s742K ctx
dense
stablelm 2 zephyr 1 6b
C46
6B10.1 GB96 tok/s879K ctx
dense
Mamba Codestral 7B v0.1
C46
7B10.8 GB112 tok/s742K ctx
dense
stabilityai japanese stablelm base gamma 7b
C46
7B10.8 GB112 tok/s742K ctx
dense
baichuan2 7b chat
C46
7B10.8 GB112 tok/s742K ctx
dense
DiscoPOP zephyr 7b gemma
C46
7B10.8 GB112 tok/s742K ctx
dense
👁 01.AI
Yi 1.5 6B
C46
6B10.3 GB96 tok/s4K ctx
dense
starcoder2 7b
C46
7B10.8 GB112 tok/s742K ctx
dense
zephyr 7b beta Mistral 7B Instruct v0.2
C46
7B10.8 GB112 tok/s742K ctx
dense
OpenChat 3.5 7B Qwen v2.0 i1
C46
7B10.8 GB112 tok/s742K ctx
dense
OpenChat 3.5 7B Starling v2.0 i1
C46
7B10.8 GB112 tok/s742K ctx
dense
internlm2 math plus 7b IMat
C46
7B10.8 GB112 tok/s742K ctx
dense
CodeNinja 1.0 OpenChat 7B i1
C46
7B10.8 GB112 tok/s742K ctx
dense
Yi 1.5 6B Chat
C46
6B10.1 GB96 tok/s879K ctx
dense
internlm2 5 7b chat i1
C46
7B10.8 GB112 tok/s742K ctx
dense
stablelm 2 1 6b chat imatrix
C46
6B10.1 GB96 tok/s879K ctx
dense
OpenSafetyLab MD Judge v0 2 internlm2 7b
C46
7B10.8 GB112 tok/s742K ctx
dense
MD Judge v0 2 internlm2 7b i1
C46
7B10.8 GB112 tok/s742K ctx
dense
zephyr 7b gemma sft african ultrachat 100k
C46
7B10.8 GB112 tok/s742K ctx
dense
jointpreferences mistral 7b sft helpful
C46
7B10.8 GB112 tok/s742K ctx
dense
zephyr 7b dpo full i1
C46
7B10.8 GB112 tok/s742K ctx
dense
blossom v3 baichuan2 7b i1
C46
7B10.8 GB112 tok/s742K ctx
dense
blossom v1 baichuan 7b i1
C46
7B10.8 GB112 tok/s742K ctx
dense
👁 NVIDIA
Nemotron Mini 4B
C46
4B10.1 GB64 tok/s4K ctx
dense
HelpingAI2 6B
C46
6B10.1 GB96 tok/s879K ctx
dense
HelpingAI2 6B i1
C46
6B10.1 GB96 tok/s879K ctx
dense
👁 Alibaba
Qwen 2.5 Coder 0.5B
C46
0.5B6.2 GB8 tok/s131K ctx
dense
👁 BigCode
StarCoder2 7B
C46
7B10.5 GB112 tok/s16K ctx
dense
👁 DeepSeek
DeepSeek LLM 67B
C45
67B52.4 GB6 tok/s4K ctx
dense
HelpingAI2.5 5B i1
C45
5B9.3 GB80 tok/s1072K ctx
dense
👁 Unsloth
Qwen3.5 4B
C45
4B8.6 GB64 tok/s1361K ctx
dense
gemma 3 4b it
C45
4B8.6 GB64 tok/s1361K ctx
dense
gemma 3 4b it
C45
4B8.6 GB64 tok/s1361K ctx
dense
👁 Alibaba
Qwen 3 0.6B
C44
0.6B6.9 GB10 tok/s33K ctx
dense
👁 Meta
Llama 3.2 1B
C44
1B6.8 GB16 tok/s128K ctx
dense
Llama 3.2 3B Instruct
C44
3B8.2 GB48 tok/s1827K ctx
dense
👁 Qwen
Qwen2.5 3B Instruct
C44
3B7.9 GB48 tok/s1842K ctx
dense
Llama 3.2 3B Instruct
C43
3B7.9 GB48 tok/s1842K ctx
dense
Ministral 3 3B Instruct 2512
C43
3B7.9 GB48 tok/s1842K ctx
dense
HELVETE 3B
C43
3B7.9 GB48 tok/s1842K ctx
dense
Hermes 3 Llama 3.2 3B
C43
3B7.9 GB48 tok/s1842K ctx
dense
stablelm zephyr 3b
C43
3B7.9 GB48 tok/s1842K ctx
dense
StarCoder2 3B
C43
3B7.9 GB48 tok/s1842K ctx
dense
AI21 Jamba Reasoning 3B
C43
3B7.9 GB48 tok/s1842K ctx
dense
stablelm 3b 4e1t
C43
3B7.9 GB48 tok/s1842K ctx
dense
ai21labs AI21 Jamba Reasoning 3B
C43
3B7.9 GB48 tok/s1842K ctx
dense
ai21labs AI21 Jamba2 3B
C43
3B7.9 GB48 tok/s1842K ctx
dense
HelpingAI 3B hindi i1
C43
3B7.9 GB48 tok/s1842K ctx
dense
AI21 Jamba2 3B
C43
3B7.9 GB48 tok/s1842K ctx
dense
HelpingAI 3B hindi
C43
3B7.9 GB48 tok/s1842K ctx
dense
AI21 Jamba2 3B i1
C43
3B7.9 GB48 tok/s1842K ctx
dense
gemma 2 2b it
C42
2B7.6 GB32 tok/s2776K ctx
dense
EXAONE 3.5 2.4B Instruct
C42
2.4B7.4 GB38 tok/s2323K ctx
dense
👁 Google
gemma 2b
C42
2B7.2 GB32 tok/s2804K ctx
dense
Gemmasutra Mini 2B v1
C42
2B7.2 GB32 tok/s2804K ctx
dense
👁 BigCode
StarCoder2 3B
C42
3B8.0 GB48 tok/s16K ctx
dense
gemma 2 2b it
C42
2B7.2 GB32 tok/s2804K ctx
dense
👁 Alibaba
Qwen 3.5 0.6B
C42
0.6B6.9 GB10 tok/s131K ctx
dense
👁 Qwen
Qwen2.5 1.5B Instruct
C41
1.5B6.8 GB24 tok/s3767K ctx
dense
👁 Unsloth
DeepSeek R1 Distill Qwen 1.5B
C41
1.5B6.8 GB24 tok/s3767K ctx
dense
Yi Coder 1.5B Chat
C41
1.5B6.8 GB24 tok/s3767K ctx
dense
stablelm 2 zephyr 1.6b
C40
1.6B6.9 GB26 tok/s3526K ctx
dense
logos16v2 stablelm2 1.6b i1
C40
1.6B6.9 GB26 tok/s3526K ctx
dense
Yi Coder 1.5B
C40
1.5B6.8 GB24 tok/s3767K ctx
dense
👁 Unsloth
Falcon H1 1.5B Instruct
C40
1.5B6.8 GB24 tok/s3767K ctx
dense
F

F Tier -- Can't Run (54 models)

Too heavy for this hardware.

TinyLlama 1.1B Chat v1.0
D40
1.1B6.5 GB18 tok/s5167K ctx
dense
EXAONE 4.0 1.2B
D40
1.2B6.6 GB19 tok/s4730K ctx
dense
👁 Alibaba
Qwen 2.5 0.5B
D40
0.5B6.2 GB8 tok/s131K ctx
dense
Llama 3.2 1B Instruct Q8 0
D40
1B6.6 GB16 tok/s5663K ctx
dense
TinyLlama 1.1B Chat v0.3
D39
1.1B6.5 GB18 tok/s5167K ctx
dense
👁 TinyLlama
TinyLlama 1.1B Chat v0.6
D39
1.1B6.5 GB18 tok/s5167K ctx
dense
Llama 3.2 1B Instruct
D39
1B6.4 GB16 tok/s5692K ctx
dense
gemma 3 1b it
D39
1B6.4 GB16 tok/s5692K ctx
dense
TinyLlama 1.1B Chat v1.0 imatrix
D39
1.1B6.5 GB18 tok/s5167K ctx
dense
Falcon3 1B Instruct abliterated
D39
1B6.4 GB16 tok/s5692K ctx
dense
SmolVLM 500M Instruct
D37
0.5B6.2 GB8 tok/s6702K ctx
dense
embeddinggemma 300M
D36
0.3B6.0 GB5 tok/s6729K ctx
dense
Solar Open 69B REAP i1
D35
69B55.9 GB5 tok/s4K ctx
dense
granite embedding 107m multilingual
D33
0.11B5.9 GB2 tok/s6758K ctx
dense
Falcon H1 Tiny 90M Instruct
D33
0.09B5.9 GB2 tok/s6759K ctx
dense
Llama 3.3 70B Instruct
D29
70B56.6 GB5 tok/s4K ctx
dense
stabilityai japanese stablelm instruct beta 70b
D28
70B56.6 GB5 tok/s4K ctx
dense
👁 Alibaba
Qwen 3 235B A22B
F0
235B151.9 GB2 tok/s4K ctx
moe
👁 Alibaba
Qwen3-Coder 480B A35B Instruct
F0
480B301.4 GB2 tok/s4K ctx
moe
👁 Alibaba
Qwen 3.5 122B A10B
F0
122B82.6 GB7 tok/s4K ctx
moe
👁 Alibaba
Qwen 3.5 397B A17B
F0
397B250.7 GB2 tok/s4K ctx
moe
👁 Mistral
Mixtral 8x22B
F0
141B95.1 GB3 tok/s4K ctx
moe
👁 Mistral
Devstral 2 123B Instruct
F0
123B86.1 GB2 tok/s4K ctx
dense
👁 Mistral
Mistral Small 4 119B
F0
119B83.7 GB7 tok/s4K ctx
moe
👁 Mistral
Leanstral 119B A6B
F0
119B87.1 GB6 tok/s4K ctx
moe
👁 Mistral
Mistral Large 3
F0
675B422.8 GB2 tok/s4K ctx
+1moe
F0
744B478.6 GB2 tok/s4K ctx
moe
👁 Z.ai
GLM-5.1
F0
754B484.7 GB2 tok/s4K ctx
moe
F0
1000B623.1 GB2 tok/s4K ctx
moe
F0
1000B623.1 GB2 tok/s4K ctx
+1moe
👁 DeepSeek
DeepSeek V3 671B
F0
671B474.6 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V3.1 671B
F0
671B474.6 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V4 Pro
F0
1600B869.6 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V4 Flash
F0
284B165.0 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek R1 671B
F0
671B474.6 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek Coder V2 236B
F0
236B208.3 GB2 tok/s4K ctx
moe
👁 DeepSeek
DeepSeek V2.5 236B
F0
236B208.3 GB2 tok/s4K ctx
moe
👁 Meta
Llama 4 Scout 17B 16E
F0
109B75.1 GB7 tok/s4K ctx
moe
👁 Meta
Llama 4 Maverick 17B 128E
F0
400B252.6 GB2 tok/s4K ctx
moe
👁 Cohere
Command R+ 104B
F0
104B72.6 GB2 tok/s4K ctx
dense
👁 Cohere
Command A 111B
F0
111B77.3 GB2 tok/s4K ctx
dense
👁 Mistral AI
Pixtral Large 124B
F0
124B86.7 GB2 tok/s4K ctx
dense
👁 Meta
Llama 3.1 405B
F0
405B260.4 GB2 tok/s4K ctx
dense
👁 OpenAI
GPT-OSS 120B
F0
117B82.0 GB2 tok/s4K ctx
dense
👁 DeepSeek
DeepSeek V3.2
F0
671B415.5 GB2 tok/s4K ctx
moe
MiniMax M2.7
F0
230B149.8 GB2 tok/s4K ctx
moe
Solar Open 100B
F0
100B78.4 GB2 tok/s4K ctx
dense
K EXAONE 236B A23B
F0
236B177.3 GB2 tok/s4K ctx
dense
Baichuan M3 235B
F0
235B176.6 GB2 tok/s4K ctx
dense
Baichuan M3 235B i1
F0
235B176.6 GB2 tok/s4K ctx
dense
BaichuanMed OCR 72B i1
F0
72B58.1 GB4 tok/s4K ctx
dense
Solar Open 100B i1
F0
100B78.4 GB2 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 122B A10B
F0
122B79.8 GB2 tok/s4K ctx
dense
👁 Unsloth
Qwen3.5 397B A17B
F0
397B294.4 GB2 tok/s4K ctx
dense

Compare with...

See How Other Hardware Stacks Up

Check tier lists for similar GPUs and Apple Silicon configs

RTX 6000 Ada 48GB

48 GB VRAM

View tier list

RTX PRO 5000 Blackwell 48GB

48 GB VRAM

View tier list

NVIDIA A40 48GB

48 GB VRAM

View tier list

Radeon Pro W7900 48GB

48 GB VRAM

View tier list
Full NVIDIA L40S 48GB pageCompare this hardware