VOOZH
about
URL: https://willitrunai.com/browse/best-for/8gb
⇱ Best AI Models for 8GB VRAM — Local LLMs | WillItRunAI
1
Qwen 3.5 4B
4B
Q4_K_M
6.3 GB VRAM
56.0 tok/s
Runs great
S
Excellent
2
Phi-4 Mini Reasoning 4B
3.799999952316284B
Q4_K_M
5.5 GB VRAM
53.2 tok/s
Runs great
S
Excellent
3
Jina Embeddings v3
0.5720000267028809B
F16
4.8 GB VRAM
8.0 tok/s
Runs great
A
Great
4
BGE M3
0.5680000185966492B
F16
4.0 GB VRAM
8.0 tok/s
Runs great
A
Great
5
Nemotron Nano 8B
8B
Q4_K_M
8.5 GB VRAM
51.9 tok/s
Needs offload
A
Great
6
mxbai Embed Large
0.33500000834465027B
F16
3.9 GB VRAM
4.7 tok/s
Runs great
A
Great
7
Snowflake Arctic Embed L
0.33500000834465027B
F16
3.9 GB VRAM
4.7 tok/s
Runs great
A
Great
8
Nomic Embed Text v1.5
0.13699999451637268B
F16
2.5 GB VRAM
2.0 tok/s
Runs great
B
Good
9
InternVL2 8B
8B
Q4_K_M
8.5 GB VRAM
51.9 tok/s
Needs offload
A
Great
10
Qwen 3 4B
4B
Q4_K_M
6.3 GB VRAM
56.0 tok/s
Runs great
S
Excellent
11
BGE Large EN v1.5
0.33500000834465027B
F16
3.9 GB VRAM
4.7 tok/s
Runs great
A
Great
12
MiniCPM-V 2.6 8B
8B
Q4_K_M
8.5 GB VRAM
51.9 tok/s
Needs offload
A
Great
13
Qwen 2.5 VL 7B
7B
Q4_K_M
6.8 GB VRAM
91.5 tok/s
Tight fit
A
Great
14
SQLCoder 7B
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
A
Great
15
Magistral 7B
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
A
Great
16
CodeGeeX 4 9B
9B
Q4_K_M
7.8 GB VRAM
71.7 tok/s
Needs offload
A
Great
17
Gemma 4 E4B
8B
Q4_K_M
7.9 GB VRAM
60.1 tok/s
Needs offload
A
Great
18
All MiniLM L6 v2
0.023000000044703484B
F16
2.0 GB VRAM
2.0 tok/s
Runs great
B
Good
19
Qwen 2.5 7B
7B
Q4_K_M
6.8 GB VRAM
91.5 tok/s
Tight fit
A
Great
20
Qwen 2.5 Coder 3B
3B
Q4_K_M
5.7 GB VRAM
42.0 tok/s
Runs great
A
Great
21
DevStral 7B
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
A
Great
22
Codestral Mamba 7B
7B
Q4_K_M
6.5 GB VRAM
97.0 tok/s
Runs great
A
Great
23
Granite Code 8B
8B
Q4_K_M
8.5 GB VRAM
51.9 tok/s
Needs offload
B
Good
24
Gemma 4 E2B
5.099999904632568B
Q4_K_M
5.3 GB VRAM
71.4 tok/s
Runs great
A
Great
25
Ministral 3 3B
3B
Q4_K_M
4.3 GB VRAM
42.0 tok/s
Runs great
A
Great
26
Qwen 3.5 2B
2B
Q4_K_M
4.6 GB VRAM
28.0 tok/s
Runs great
A
Great
27
GLM-4 9B
9B
Q4_K_M
7.8 GB VRAM
71.7 tok/s
Needs offload
A
Great
28
Gemma 3 4B
4B
Q4_K_M
6.2 GB VRAM
56.0 tok/s
Runs great
A
Great
29
Llama 3.1 8B
8B
Q4_K_M
8.5 GB VRAM
51.9 tok/s
Needs offload
B
Good
30
WizardMath 7B
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
A
Great
31
OLMo 2 7B
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
A
Great
32
Phi 4 Mini 4B
4B
Q4_K_M
5.6 GB VRAM
56.0 tok/s
Runs great
A
Great
33
Qwen 2.5 Coder 7B
7B
Q4_K_M
6.8 GB VRAM
91.5 tok/s
Tight fit
A
Great
34
Qwen 3 1.7B
1.7000000476837158B
Q4_K_M
4.4 GB VRAM
23.8 tok/s
Runs great
B
Good
35
Qwen 2.5 Coder 1.5B
1.5B
Q4_K_M
3.0 GB VRAM
21.0 tok/s
Runs great
B
Good
36
Qwen 2.5 3B
3B
Q4_K_M
5.7 GB VRAM
42.0 tok/s
Runs great
A
Great
37
Granite 4.1 3B
3B
Q4_K_M
4.8 GB VRAM
42.0 tok/s
Runs great
B
Good
38
DeepSeek R1 Distill 7B
7B
Q4_K_M
6.8 GB VRAM
91.5 tok/s
Tight fit
A
Great
39
DeepSeek R1 Distill 8B
8B
Q4_K_M
8.5 GB VRAM
51.9 tok/s
Needs offload
B
Good
40
Falcon 7B Instruct
7B
Q4_K_M
6.1 GB VRAM
92.6 tok/s
Runs great
A
Great
41
Samantha 7B
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
B
Good
42
Granite Code 3B
3B
Q4_K_M
6.0 GB VRAM
42.0 tok/s
Runs great
A
Great
43
Llama 3.2 3B
3B
Q4_K_M
5.2 GB VRAM
42.0 tok/s
Runs great
B
Good
44
Mistral 7B Instruct v0.3
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
B
Good
45
TinyLlama 1.1B
1.100000023841858B
Q4_K_M
2.7 GB VRAM
15.4 tok/s
Runs great
B
Good
46
DeepSeek R1 1.5B
1.5B
Q4_K_M
3.0 GB VRAM
21.0 tok/s
Runs great
B
Good
47
Qwen 2.5 Coder 0.5B
0.5B
Q4_K_M
2.2 GB VRAM
7.0 tok/s
Runs great
C
Usable
48
Qwen 2.5 1.5B
1.5B
Q4_K_M
3.0 GB VRAM
21.0 tok/s
Runs great
C
Usable
49
Gemma 3 1B
1B
Q4_K_M
2.7 GB VRAM
14.0 tok/s
Runs great
C
Usable
50
SmolLM3 3B
3B
Q4_K_M
5.5 GB VRAM
42.0 tok/s
Runs great
B
Good
51
Qwen 3 0.6B
0.6000000238418579B
Q4_K_M
2.9 GB VRAM
8.4 tok/s
Runs great
C
Usable
52
Gemma 2 2B
2B
Q4_K_M
4.5 GB VRAM
28.0 tok/s
Runs great
C
Usable
53
Llama 3.2 1B
1B
Q4_K_M
2.8 GB VRAM
14.0 tok/s
Runs great
C
Usable
54
Granite 3.1 8B
8B
Q4_K_M
8.5 GB VRAM
59.7 tok/s
Needs offload
C
Usable
55
Qwen 2.5 Math 7B
7B
Q4_K_M
6.8 GB VRAM
91.5 tok/s
Tight fit
B
Good
56
Qwen 3.5 0.6B
0.6000000238418579B
Q4_K_M
2.9 GB VRAM
8.4 tok/s
Runs great
C
Usable
57
Qwen 2.5 0.5B
0.5B
Q4_K_M
2.2 GB VRAM
7.0 tok/s
Runs great
C
Usable
58
OpenChat 7B
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
B
Good
59
Aya Expanse 8B
8B
Q4_K_M
8.5 GB VRAM
51.9 tok/s
Needs offload
C
Usable
60
Nemotron Mini 4B
4B
Q4_K_M
6.1 GB VRAM
56.0 tok/s
Runs great
B
Good
61
OpenHermes 2.5 7B
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
C
Usable
62
Dolphin 2.9 8B
8B
Q4_K_M
8.5 GB VRAM
51.9 tok/s
Needs offload
C
Usable
63
Zephyr 7B Beta
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
C
Usable
64
Qwen3.5 9B
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
65
gemma 2b
2B
Q4_K_M
3.2 GB VRAM
28.0 tok/s
Runs great
C
Usable
66
Qwen3.5 9B Uncensored HauhauCS Aggressive
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
67
gemma 2 2b it
2B
Q6_K
3.6 GB VRAM
28.0 tok/s
Runs great
C
Usable
68
Meta Llama 3.1 8B Instruct
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
69
Llama 3.2 3B Instruct
3B
Q5_K_M
4.2 GB VRAM
42.0 tok/s
Runs great
C
Usable
70
Qwen3.5 4B
4B
Q4_K_M
4.6 GB VRAM
56.0 tok/s
Runs great
C
Usable
71
Llama 2 7B Chat
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
72
llava llama 3 8b v1 1
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
73
Llama 3.2 1B Instruct Q8 0
1B
Q6_K
2.6 GB VRAM
14.0 tok/s
Runs great
C
Usable
74
Starling LM 7B
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
C
Usable
75
Qwen2.5 3B Instruct
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
76
Qwen2.5 1.5B Instruct
1.5B
Q4_K_M
2.8 GB VRAM
21.0 tok/s
Runs great
C
Usable
77
SmolVLM 500M Instruct
0.5B
Q6_K
2.2 GB VRAM
7.0 tok/s
Runs great
D
Poor
78
Mistral 7B Instruct v0.2
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
79
DeepSeek R1 0528 Qwen3 8B
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
80
TinyLlama 1.1B Chat v1.0
1.100000023841858B
Q4_K_M
2.5 GB VRAM
15.4 tok/s
Runs great
C
Usable
81
Gemmasutra Mini 2B v1
2B
Q4_K_M
3.2 GB VRAM
28.0 tok/s
Runs great
C
Usable
82
Qwen3.5 9B
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
83
Mistral 7B Instruct v0.3
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
84
gemma 3 4b it
4B
Q4_K_M
4.6 GB VRAM
56.0 tok/s
Runs great
C
Usable
85
embeddinggemma 300M
0.30000001192092896B
Q6_K
2.0 GB VRAM
4.2 tok/s
Runs great
D
Poor
86
Meta Llama 3 8B Instruct
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
87
DeepSeek R1 Distill Llama 8B
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
88
Dolphin3.0 Llama3.1 8B
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
89
Llama 3 8B Instruct 32k v0.1
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
90
DeepSeek R1 Distill Qwen 1.5B
1.5B
Q4_K_M
2.8 GB VRAM
21.0 tok/s
Runs great
C
Usable
91
Meta Llama 3.1 8B Instruct
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
92
vntl llama3 8b v2
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
93
DeepSeek R1 0528 Qwen3 8B
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
94
gemma 3 4b it
4B
Q4_K_M
4.6 GB VRAM
56.0 tok/s
Runs great
C
Usable
95
Yi Coder 1.5B Chat
1.5B
Q4_K_M
2.8 GB VRAM
21.0 tok/s
Runs great
C
Usable
96
Llama 3.2 1B Instruct
1B
Q4_K_M
2.4 GB VRAM
14.0 tok/s
Runs great
C
Usable
97
gemma 2 2b it
2B
Q4_K_M
3.2 GB VRAM
28.0 tok/s
Runs great
C
Usable
98
Llama 3.2 3B Instruct
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
99
gemma 3 1b it
1B
Q4_K_M
2.4 GB VRAM
14.0 tok/s
Runs great
C
Usable
100
DeepSeek R1 0528 Qwen3 8B
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
101
Yi 1.5 6B Chat
6B
Q4_K_M
6.1 GB VRAM
84.0 tok/s
Runs great
B
Good
102
Ministral 3 3B Instruct 2512
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
103
Yi Coder 9B Chat
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
104
glm 4 9b chat 1m
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
105
Qwen3 8B DeepSeek v3.2 Speciale Distill
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
106
Hermes 3 Llama 3.1 8B
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
107
Mistral 7B Instruct v0.3
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
108
stablelm 2 zephyr 1 6b
6B
Q4_K_M
6.1 GB VRAM
84.0 tok/s
Runs great
B
Good
109
Hermes 2 Pro Mistral 7B
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
110
TinyLlama 1.1B Chat v0.3
1.100000023841858B
Q4_K_M
2.5 GB VRAM
15.4 tok/s
Runs great
C
Usable
111
TinyLlama 1.1B Chat v0.6
1.100000023841858B
Q4_K_M
2.5 GB VRAM
15.4 tok/s
Runs great
C
Usable
112
zephyr 7B beta
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
113
Hermes 2 Pro Llama 3 8B
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
114
Dolphin3.0 Llama3.1 8B
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
115
Nous Hermes 2 Mistral 7B DPO
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
116
granite 8b code instruct 4k
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
117
HELVETE 3B
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
118
dolphin 2.9.4 llama3.1 8b
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
119
Yi 1.5 9B Chat
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
120
Falcon H1R 7B
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
121
granite embedding 107m multilingual
0.10700000077486038B
Q4_K_M
1.9 GB VRAM
2.0 tok/s
Runs great
D
Poor
122
falcon mamba 7b instruct Q4 K M
7B
Q4_K_M
6.8 GB VRAM
97.0 tok/s
Tight fit
C
Usable
123
Hermes 3 Llama 3.2 3B
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
124
zephyr 7B alpha
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
125
speechless zephyr code functionary 7b
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
126
stablelm zephyr 3b
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
127
Yi 1.5 6B Chat
6B
Q4_K_M
6.1 GB VRAM
84.0 tok/s
Runs great
B
Good
128
openchat 3.6 8b 20240522 IMat
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
129
EXAONE 4.0 1.2B
1.2000000476837158B
Q4_K_M
2.6 GB VRAM
16.8 tok/s
Runs great
C
Usable
130
StarCoder2 3B
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
131
stablelm 2 1 6b chat imatrix
6B
Q4_K_M
6.1 GB VRAM
84.0 tok/s
Runs great
B
Good
132
EXAONE 3.5 2.4B Instruct
2.4000000953674316B
Q4_K_M
3.4 GB VRAM
33.6 tok/s
Runs great
C
Usable
133
EXAONE 3.5 7.8B Instruct
7.800000190734863B
Q4_K_M
7.4 GB VRAM
75.7 tok/s
Tight fit
C
Usable
134
StarCoder2 7B
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
135
Yi 1.5 6B
6B
Q4_K_M
6.3 GB VRAM
84.0 tok/s
Runs great
B
Good
136
aya expanse 8b
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
137
japanese stablelm instruct gamma 7B
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
138
Yi Coder 1.5B
1.5B
Q4_K_M
2.8 GB VRAM
21.0 tok/s
Runs great
C
Usable
139
EXAONE 3.5 7.8B Instruct
7.800000190734863B
Q4_K_M
7.4 GB VRAM
75.7 tok/s
Tight fit
C
Usable
140
dolphin v2 8b abliterated i1
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
141
Falcon H1 7B Instruct
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
142
Falcon H1 Tiny 90M Instruct
0.09000000357627869B
Q4_K_M
1.9 GB VRAM
2.0 tok/s
Runs great
D
Poor
143
EXAONE 3.5 7.8B Instruct i1
7.800000190734863B
Q4_K_M
7.4 GB VRAM
75.7 tok/s
Tight fit
C
Usable
144
Falcon H1R 7B
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
145
AI21 Jamba Reasoning 3B
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
146
exaone 3.0 7.8b it
7.800000190734863B
Q4_K_M
7.4 GB VRAM
75.7 tok/s
Tight fit
C
Usable
147
aya expanse 8b orthogonal heretic i1
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
148
Falcon3 1B Instruct abliterated
1B
Q4_K_M
2.4 GB VRAM
14.0 tok/s
Runs great
C
Usable
149
Yi 9B Coder i1
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
150
stablelm 3b 4e1t
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
151
Mamba Codestral 7B v0.1
7B
Q4_K_M
6.8 GB VRAM
97.0 tok/s
Tight fit
C
Usable
152
ai21labs AI21 Jamba Reasoning 3B
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
153
stabilityai japanese stablelm base gamma 7b
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
154
stablelm 2 zephyr 1.6b
1.600000023841858B
Q4_K_M
2.9 GB VRAM
22.4 tok/s
Runs great
C
Usable
155
Falcon H1 1.5B Instruct
1.5B
Q4_K_M
2.8 GB VRAM
21.0 tok/s
Runs great
C
Usable
156
aya expanse 8b orthogonal heretic
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
157
baichuan2 7b chat
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
158
logos16v2 stablelm2 1.6b i1
1.600000023841858B
Q4_K_M
2.9 GB VRAM
22.4 tok/s
Runs great
C
Usable
159
DiscoPOP zephyr 7b gemma
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
160
HelpingAI2 9B
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
161
ai21labs AI21 Jamba2 3B
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
162
TinyLlama 1.1B Chat v1.0 imatrix
1.100000023841858B
Q4_K_M
2.5 GB VRAM
15.4 tok/s
Runs great
C
Usable
163
starcoder2 7b
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
164
zephyr 7b beta Mistral 7B Instruct v0.2
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
165
OpenChat 3.5 7B Qwen v2.0 i1
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
166
OpenChat 3.5 7B Starling v2.0 i1
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
167
HelpingAI2 6B
6B
Q4_K_M
6.1 GB VRAM
84.0 tok/s
Runs great
B
Good
168
internlm2 math plus 7b IMat
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
169
CodeNinja 1.0 OpenChat 7B i1
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
170
HelpingAI2 9B i1
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
171
HelpingAI2.5 5B i1
5B
Q4_K_M
5.3 GB VRAM
70.0 tok/s
Runs great
C
Usable
172
internlm2 5 1 8b chat i1
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
173
HelpingAI 9B 200k i1
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
174
internlm2 5 7b chat i1
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
175
HelpingAI 3B hindi i1
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
176
internlm3 8b instruct abliterated i1
8B
Q4_K_M
7.5 GB VRAM
73.8 tok/s
Tight fit
C
Usable
177
HelpingAI2 6B i1
6B
Q4_K_M
6.1 GB VRAM
84.0 tok/s
Runs great
B
Good
178
OpenSafetyLab MD Judge v0 2 internlm2 7b
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
179
AI21 Jamba2 3B
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
180
MD Judge v0 2 internlm2 7b i1
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
181
HelpingAI 3B hindi
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
182
zephyr 7b gemma sft african ultrachat 100k
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
183
HelpingAI 9B i1
9B
Q4_K_M
8.2 GB VRAM
46.2 tok/s
Needs offload
C
Usable
184
jointpreferences mistral 7b sft helpful
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
185
zephyr 7b dpo full i1
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
186
blossom v3 baichuan2 7b i1
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
187
AI21 Jamba2 3B i1
3B
Q4_K_M
3.9 GB VRAM
42.0 tok/s
Runs great
C
Usable
188
blossom v1 baichuan 7b i1
7B
Q4_K_M
6.8 GB VRAM
84.3 tok/s
Tight fit
C
Usable
189
Neural Chat 7B
7B
Q4_K_M
7.9 GB VRAM
90.6 tok/s
Needs offload
C
Usable
190
StarCoder2 7B
7B
Q4_K_M
6.5 GB VRAM
92.0 tok/s
Runs great
B
Good
191
StarCoder2 3B
3B
Q4_K_M
4.0 GB VRAM
42.0 tok/s
Runs great
C
Usable
