Voozh

Qwen 3.5 9B9B

Q4_K_M9.8 GB VRAM70.5 tok/sRuns great

SExcellent

Qwen 3.5 4B4B

Q4_K_M6.7 GB VRAM56.0 tok/sRuns great

SExcellent

Qwen 3 8B8B

Q4_K_M9.2 GB VRAM79.3 tok/sRuns great

SExcellent

Phi-4 Mini Reasoning 4B3.799999952316284B

Q4_K_M5.9 GB VRAM53.2 tok/sRuns great

SExcellent

Jina Embeddings v30.5720000267028809B

F165.2 GB VRAM8.0 tok/sRuns great

AGreat

BGE M30.5680000185966492B

F164.4 GB VRAM8.0 tok/sRuns great

AGreat

Nemotron Nano 8B8B

Q4_K_M8.9 GB VRAM79.3 tok/sRuns great

SExcellent

mxbai Embed Large0.33500000834465027B

F164.3 GB VRAM4.7 tok/sRuns great

AGreat

Snowflake Arctic Embed L0.33500000834465027B

F164.3 GB VRAM4.7 tok/sRuns great

AGreat

Nomic Embed Text v1.50.13699999451637268B

F162.9 GB VRAM2.0 tok/sRuns great

BGood

InternVL2 8B8B

Q4_K_M8.9 GB VRAM79.3 tok/sRuns great

SExcellent

Qwen 3 4B4B

Q4_K_M6.7 GB VRAM56.0 tok/sRuns great

AGreat

BGE Large EN v1.50.33500000834465027B

F164.3 GB VRAM4.7 tok/sRuns great

AGreat

Ministral 3 8B8B

Q4_K_M9.2 GB VRAM79.3 tok/sRuns great

SExcellent

MiniCPM-V 2.6 8B8B

Q4_K_M8.9 GB VRAM79.3 tok/sRuns great

SExcellent

Qwen 2.5 VL 7B7B

Q4_K_M7.2 GB VRAM91.5 tok/sRuns great

AGreat

SQLCoder 7B7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

SExcellent

Nemotron Nano 9B v29B

Q4_K_M10.0 GB VRAM70.5 tok/sTight fit

AGreat

Magistral 7B7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

AGreat

CodeGeeX 4 9B9B

Q4_K_M8.2 GB VRAM71.7 tok/sRuns great

AGreat

Gemma 4 E4B8B

Q4_K_M8.3 GB VRAM60.1 tok/sRuns great

AGreat

OLMo 2 13B13B

Q4_K_M12.5 GB VRAM33.9 tok/sNeeds offload

AGreat

All MiniLM L6 v20.023000000044703484B

F162.4 GB VRAM2.0 tok/sRuns great

BGood

Qwen 2.5 7B7B

Q4_K_M7.2 GB VRAM91.5 tok/sRuns great

AGreat

Qwen 2.5 Coder 3B3B

Q4_K_M6.1 GB VRAM42.0 tok/sRuns great

AGreat

DevStral 7B7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

AGreat

Codestral Mamba 7B7B

Q4_K_M6.9 GB VRAM97.0 tok/sRuns great

AGreat

Granite Code 8B8B

Q4_K_M8.9 GB VRAM79.3 tok/sRuns great

AGreat

Granite 4.1 8B8B

Q4_K_M9.4 GB VRAM79.3 tok/sRuns great

AGreat

Pixtral 12B12B

Q4_K_M11.9 GB VRAM52.9 tok/sNeeds offload

AGreat

Gemma 4 E2B5.099999904632568B

Q4_K_M5.7 GB VRAM71.4 tok/sRuns great

AGreat

Ministral 3 3B3B

Q4_K_M4.7 GB VRAM42.0 tok/sRuns great

AGreat

Qwen 3.5 2B2B

Q4_K_M5.0 GB VRAM28.0 tok/sRuns great

BGood

GLM-4 9B9B

Q4_K_M8.2 GB VRAM71.7 tok/sRuns great

AGreat

Gemma 3 4B4B

Q4_K_M6.6 GB VRAM56.0 tok/sRuns great

AGreat

Llama 3.1 8B8B

Q4_K_M8.9 GB VRAM79.3 tok/sRuns great

AGreat

Solar 7B7B

Q4_K_M9.3 GB VRAM90.6 tok/sRuns great

AGreat

WizardMath 7B7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

AGreat

OLMo 2 7B7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

AGreat

Phi 4 Mini 4B4B

Q4_K_M6.0 GB VRAM56.0 tok/sRuns great

AGreat

Qwen 2.5 Coder 7B7B

Q4_K_M7.2 GB VRAM91.5 tok/sRuns great

AGreat

Qwen 3 1.7B1.7000000476837158B

Q4_K_M4.8 GB VRAM23.8 tok/sRuns great

BGood

Qwen 2.5 Coder 1.5B1.5B

Q4_K_M3.4 GB VRAM21.0 tok/sRuns great

BGood

Qwen 2.5 3B3B

Q4_K_M6.1 GB VRAM42.0 tok/sRuns great

BGood

Granite 4.1 3B3B

Q4_K_M5.2 GB VRAM42.0 tok/sRuns great

BGood

Phi 3 Mini 3.8B3.799999952316284B

Q4_K_M10.3 GB VRAM53.2 tok/sTight fit

BGood

DeepSeek R1 Distill 7B7B

Q4_K_M7.2 GB VRAM91.5 tok/sRuns great

AGreat

DeepSeek R1 Distill 8B8B

Q4_K_M8.9 GB VRAM79.3 tok/sRuns great

AGreat

Falcon 7B Instruct7B

Q4_K_M6.5 GB VRAM92.6 tok/sRuns great

BGood

Samantha 7B7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

AGreat

Granite Code 3B3B

Q4_K_M6.4 GB VRAM42.0 tok/sRuns great

BGood

Phi 3.5 Mini 4B4B

Q4_K_M10.4 GB VRAM56.0 tok/sTight fit

BGood

Gemma 2 9B9B

Q4_K_M12.7 GB VRAM34.6 tok/sNeeds offload

BGood

Llama 3.2 11B Vision11B

Q4_K_M10.8 GB VRAM57.7 tok/sTight fit

BGood

Llama 3.2 3B3B

Q4_K_M5.6 GB VRAM42.0 tok/sRuns great

BGood

Mistral Nemo 12B12B

Q4_K_M11.9 GB VRAM52.9 tok/sNeeds offload

BGood

Mistral 7B Instruct v0.37B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

BGood

Yi Coder 9B9B

Q4_K_M9.1 GB VRAM71.3 tok/sRuns great

BGood

TinyLlama 1.1B1.100000023841858B

Q4_K_M3.1 GB VRAM15.4 tok/sRuns great

CUsable

DeepSeek R1 1.5B1.5B

Q4_K_M3.4 GB VRAM21.0 tok/sRuns great

CUsable

Ministral 8B8B

Q4_K_M9.2 GB VRAM79.3 tok/sRuns great

BGood

Qwen 2.5 Coder 0.5B0.5B

Q4_K_M2.6 GB VRAM7.0 tok/sRuns great

CUsable

Qwen 2.5 1.5B1.5B

Q4_K_M3.4 GB VRAM21.0 tok/sRuns great

CUsable

Gemma 3 1B1B

Q4_K_M3.1 GB VRAM14.0 tok/sRuns great

CUsable

SmolLM3 3B3B

Q4_K_M5.9 GB VRAM42.0 tok/sRuns great

BGood

Qwen 3 0.6B0.6000000238418579B

Q4_K_M3.3 GB VRAM8.4 tok/sRuns great

CUsable

Gemma 2 2B2B

Q4_K_M4.9 GB VRAM28.0 tok/sRuns great

CUsable

Llama 3.2 1B1B

Q4_K_M3.2 GB VRAM14.0 tok/sRuns great

CUsable

Granite 3.1 8B8B

Q4_K_M8.9 GB VRAM91.2 tok/sRuns great

BGood

Yi 1.5 9B9B

Q4_K_M9.1 GB VRAM71.3 tok/sRuns great

BGood

Qwen 2.5 Math 7B7B

Q4_K_M7.2 GB VRAM91.5 tok/sRuns great

BGood

Qwen 3.5 0.6B0.6000000238418579B

Q4_K_M3.3 GB VRAM8.4 tok/sRuns great

CUsable

Qwen 2.5 0.5B0.5B

Q4_K_M2.6 GB VRAM7.0 tok/sRuns great

CUsable

OpenChat 7B7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

BGood

Aya Expanse 8B8B

Q4_K_M8.9 GB VRAM79.3 tok/sRuns great

BGood

Nemotron Mini 4B4B

Q4_K_M6.5 GB VRAM56.0 tok/sRuns great

CUsable

OpenHermes 2.5 7B7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

BGood

Dolphin 2.9 8B8B

Q4_K_M8.9 GB VRAM79.3 tok/sRuns great

BGood

Zephyr 7B Beta7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

BGood

Qwen3.5 9B9B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

gemma 2b2B

Q4_K_M3.6 GB VRAM28.0 tok/sRuns great

CUsable

Qwen3.5 9B Uncensored HauhauCS Aggressive9B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

gemma 2 2b it2B

Q6_K4.0 GB VRAM28.0 tok/sRuns great

CUsable

Meta Llama 3.1 8B Instruct8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

Llama 3.2 3B Instruct3B

Q5_K_M4.6 GB VRAM42.0 tok/sRuns great

CUsable

Qwen3.5 4B4B

Q4_K_M5.0 GB VRAM56.0 tok/sRuns great

CUsable

Llama 2 7B Chat7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

llava llama 3 8b v1 18B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

Llama 3.2 1B Instruct Q8 01B

Q6_K3.0 GB VRAM14.0 tok/sRuns great

CUsable

Starling LM 7B7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

BGood

Qwen2.5 3B Instruct3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

Qwen2.5 1.5B Instruct1.5B

Q4_K_M3.2 GB VRAM21.0 tok/sRuns great

CUsable

SmolVLM 500M Instruct0.5B

Q6_K2.6 GB VRAM7.0 tok/sRuns great

DPoor

Mistral 7B Instruct v0.27B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

DeepSeek R1 0528 Qwen3 8B8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

TinyLlama 1.1B Chat v1.01.100000023841858B

Q4_K_M2.9 GB VRAM15.4 tok/sRuns great

CUsable

Gemmasutra Mini 2B v12B

Q4_K_M3.6 GB VRAM28.0 tok/sRuns great

CUsable

Qwen3.5 9B9B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

Mistral 7B Instruct v0.37B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

100

gemma 3 4b it4B

Q4_K_M5.0 GB VRAM56.0 tok/sRuns great

CUsable

101

embeddinggemma 300M0.30000001192092896B

Q6_K2.4 GB VRAM4.2 tok/sRuns great

DPoor

102

Meta Llama 3 8B Instruct8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

103

DeepSeek R1 Distill Llama 8B8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

104

Dolphin3.0 Llama3.1 8B8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

105

Llama 3 8B Instruct 32k v0.18B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

106

DeepSeek R1 Distill Qwen 1.5B1.5B

Q4_K_M3.2 GB VRAM21.0 tok/sRuns great

CUsable

107

Meta Llama 3.1 8B Instruct8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

108

vntl llama3 8b v28B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

109

DeepSeek R1 0528 Qwen3 8B8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

110

DeepSeek R1 Distill Qwen 14B14B

Q4_K_M12.3 GB VRAM30.1 tok/sNeeds offload

CUsable

111

gemma 3 4b it4B

Q4_K_M5.0 GB VRAM56.0 tok/sRuns great

CUsable

112

Yi Coder 1.5B Chat1.5B

Q4_K_M3.2 GB VRAM21.0 tok/sRuns great

CUsable

113

gemma 3 12b it12B

Q4_K_M10.8 GB VRAM49.2 tok/sTight fit

CUsable

114

Llama 3.2 1B Instruct1B

Q4_K_M2.8 GB VRAM14.0 tok/sRuns great

CUsable

115

gemma 2 2b it2B

Q4_K_M3.6 GB VRAM28.0 tok/sRuns great

CUsable

116

Llama 3.2 3B Instruct3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

117

gemma 3 1b it1B

Q4_K_M2.8 GB VRAM14.0 tok/sRuns great

CUsable

118

DeepSeek R1 0528 Qwen3 8B8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

119

Yi 1.5 6B Chat6B

Q4_K_M6.5 GB VRAM84.0 tok/sRuns great

CUsable

120

Ministral 3 3B Instruct 25123B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

121

Yi Coder 9B Chat9B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

122

glm 4 9b chat 1m9B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

123

Qwen3 8B DeepSeek v3.2 Speciale Distill8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

124

Hermes 3 Llama 3.1 8B8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

125

Mistral 7B Instruct v0.37B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

126

stablelm 2 zephyr 1 6b6B

Q4_K_M6.5 GB VRAM84.0 tok/sRuns great

CUsable

127

Hermes 2 Pro Mistral 7B7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

128

TinyLlama 1.1B Chat v0.31.100000023841858B

Q4_K_M2.9 GB VRAM15.4 tok/sRuns great

CUsable

129

TinyLlama 1.1B Chat v0.61.100000023841858B

Q4_K_M2.9 GB VRAM15.4 tok/sRuns great

CUsable

130

zephyr 7B beta7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

131

SOLAR 10.7B Instruct v1.0 uncensored10.699999809265137B

Q4_K_M9.9 GB VRAM55.2 tok/sTight fit

CUsable

132

Hermes 2 Pro Llama 3 8B8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

133

Dolphin3.0 Llama3.1 8B8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

134

Nous Hermes 2 Mistral 7B DPO7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

135

Nous Hermes 2 SOLAR 10.7B10.699999809265137B

Q4_K_M9.9 GB VRAM55.2 tok/sTight fit

CUsable

136

granite 8b code instruct 4k8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

137

HELVETE 3B3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

138

dolphin 2.9.4 llama3.1 8b8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

139

Yi 1.5 9B Chat9B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

140

Falcon H1R 7B7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

141

NousResearch Hermes 4 14B14B

Q4_K_M12.3 GB VRAM30.1 tok/sNeeds offload

CUsable

142

granite embedding 107m multilingual0.10700000077486038B

Q4_K_M2.3 GB VRAM2.0 tok/sRuns great

DPoor

143

falcon mamba 7b instruct Q4 K M7B

Q4_K_M7.2 GB VRAM97.0 tok/sRuns great

CUsable

144

Hermes 3 Llama 3.2 3B3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

145

zephyr 7B alpha7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

146

speechless zephyr code functionary 7b7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

147

stablelm zephyr 3b3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

148

Yi 1.5 6B Chat6B

Q4_K_M6.5 GB VRAM84.0 tok/sRuns great

CUsable

149

openchat 3.6 8b 20240522 IMat8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

150

EXAONE 4.0 1.2B1.2000000476837158B

Q4_K_M3.0 GB VRAM16.8 tok/sRuns great

CUsable

151

StarCoder2 3B3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

152

stablelm 2 1 6b chat imatrix6B

Q4_K_M6.5 GB VRAM84.0 tok/sRuns great

CUsable

153

EXAONE 3.5 2.4B Instruct2.4000000953674316B

Q4_K_M3.8 GB VRAM33.6 tok/sRuns great

CUsable

154

EXAONE 3.5 7.8B Instruct7.800000190734863B

Q4_K_M7.8 GB VRAM75.7 tok/sRuns great

BGood

155

StarCoder2 7B7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

156

Yi 1.5 6B6B

Q4_K_M6.7 GB VRAM84.0 tok/sRuns great

CUsable

157

aya expanse 8b8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

158

japanese stablelm instruct gamma 7B7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

159

Yi Coder 1.5B1.5B

Q4_K_M3.2 GB VRAM21.0 tok/sRuns great

CUsable

160

EXAONE 3.5 7.8B Instruct7.800000190734863B

Q4_K_M7.8 GB VRAM75.7 tok/sRuns great

CUsable

161

dolphin v2 8b abliterated i18B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

BGood

162

Falcon H1 7B Instruct7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

163

Falcon H1 Tiny 90M Instruct0.09000000357627869B

Q4_K_M2.3 GB VRAM2.0 tok/sRuns great

DPoor

164

EXAONE 3.5 7.8B Instruct i17.800000190734863B

Q4_K_M7.8 GB VRAM75.7 tok/sRuns great

CUsable

165

Falcon H1R 7B7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

166

AI21 Jamba Reasoning 3B3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

167

exaone 3.0 7.8b it7.800000190734863B

Q4_K_M7.8 GB VRAM75.7 tok/sRuns great

CUsable

168

aya expanse 8b orthogonal heretic i18B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

CUsable

169

Falcon3 1B Instruct abliterated1B

Q4_K_M2.8 GB VRAM14.0 tok/sRuns great

CUsable

170

GGUF SOLARized GraniStral 14B 2102 YeAM HCT 32QKV14B

Q4_K_M12.3 GB VRAM30.1 tok/sNeeds offload

CUsable

171

Yi 9B Coder i19B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

172

solar finalised finetuned Model 10.7B i110.699999809265137B

Q4_K_M9.9 GB VRAM55.2 tok/sTight fit

CUsable

173

stablelm 3b 4e1t3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

174

Mamba Codestral 7B v0.17B

Q4_K_M7.2 GB VRAM97.0 tok/sRuns great

CUsable

175

ai21labs AI21 Jamba Reasoning 3B3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

176

stabilityai japanese stablelm base gamma 7b7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

177

internlm JanusCoder 14B14B

Q4_K_M12.3 GB VRAM30.1 tok/sNeeds offload

CUsable

178

stablelm 2 zephyr 1.6b1.600000023841858B

Q4_K_M3.3 GB VRAM22.4 tok/sRuns great

CUsable

179

Falcon H1 1.5B Instruct1.5B

Q4_K_M3.2 GB VRAM21.0 tok/sRuns great

CUsable

180

aya expanse 8b orthogonal heretic8B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

CUsable

181

baichuan2 7b chat7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

182

logos16v2 stablelm2 1.6b i11.600000023841858B

Q4_K_M3.3 GB VRAM22.4 tok/sRuns great

CUsable

183

SOLAR 10.7B v1.010.699999809265137B

Q4_K_M9.9 GB VRAM55.2 tok/sTight fit

CUsable

184

GGUF SOLARized GraniStral 14B 1902 YeAM HCT14B

Q4_K_M12.3 GB VRAM30.1 tok/sNeeds offload

CUsable

185

DiscoPOP zephyr 7b gemma7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

186

HelpingAI2 9B9B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

187

ai21labs AI21 Jamba2 3B3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

188

TinyLlama 1.1B Chat v1.0 imatrix1.100000023841858B

Q4_K_M2.9 GB VRAM15.4 tok/sRuns great

CUsable

189

starcoder2 7b7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

190

zephyr 7b beta Mistral 7B Instruct v0.27B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

191

OpenChat 3.5 7B Qwen v2.0 i17B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

192

OpenChat 3.5 7B Starling v2.0 i17B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

193

HelpingAI2 6B6B

Q4_K_M6.5 GB VRAM84.0 tok/sRuns great

CUsable

194

internlm2 math plus 7b IMat7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

195

CodeNinja 1.0 OpenChat 7B i17B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

196

HelpingAI2 9B i19B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

197

HelpingAI2.5 5B i15B

Q4_K_M5.7 GB VRAM70.0 tok/sRuns great

CUsable

198

internlm2 5 1 8b chat i18B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

CUsable

199

HelpingAI 9B 200k i19B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

200

internlm2 5 7b chat i17B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

201

HelpingAI 3B hindi i13B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

202

internlm3 8b instruct abliterated i18B

Q4_K_M7.9 GB VRAM73.8 tok/sRuns great

CUsable

203

HelpingAI2 6B i16B

Q4_K_M6.5 GB VRAM84.0 tok/sRuns great

CUsable

204

OpenSafetyLab MD Judge v0 2 internlm2 7b7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

205

AI21 Jamba2 3B3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

206

MD Judge v0 2 internlm2 7b i17B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

207

HelpingAI2.5 10B i110B

Q4_K_M9.4 GB VRAM59.0 tok/sRuns great

CUsable

208

HelpingAI 3B hindi3B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

209

zephyr 7b gemma sft african ultrachat 100k7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

210

HelpingAI 9B i19B

Q4_K_M8.6 GB VRAM65.6 tok/sRuns great

BGood

211

jointpreferences mistral 7b sft helpful7B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

212

zephyr 7b dpo full i17B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

213

blossom v3 baichuan2 7b i17B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

214

Helply 10.2b chat i110.199999809265137B

Q4_K_M9.5 GB VRAM57.9 tok/sRuns great

CUsable

215

AI21 Jamba2 3B i13B

Q4_K_M4.3 GB VRAM42.0 tok/sRuns great

CUsable

216

blossom v1 baichuan 7b i17B

Q4_K_M7.2 GB VRAM84.3 tok/sRuns great

CUsable

217

Neural Chat 7B7B

Q4_K_M8.3 GB VRAM90.6 tok/sRuns great

BGood

218

StarCoder2 7B7B

Q4_K_M6.9 GB VRAM92.0 tok/sRuns great

CUsable

219

StarCoder2 3B3B

Q4_K_M4.4 GB VRAM42.0 tok/sRuns great

CUsable

URL: https://willitrunai.com/browse/best-for/12gb

⇱ Best AI Models for 12GB VRAM — Local LLMs | WillItRunAI