Voozh

Qwen3-Coder 30B A3B Instruct30.5B

Q4_K_M27.4 GB VRAM54.4 tok/sRuns great

SExcellent

Qwen 3.5 27B27B

Q4_K_M26.9 GB VRAM23.6 tok/sRuns great

SExcellent

Qwen 3.6 27B27B

Q4_K_M24.7 GB VRAM17.9 tok/sRuns great

SExcellent

Qwen3-VL 30B A3B Instruct30B

Q4_K_M27.1 GB VRAM56.3 tok/sRuns great

SExcellent

Qwen 3.6 35B A3B35B

Q4_K_M32.8 GB VRAM45.7 tok/sRuns great

SExcellent

Qwen 3.5 9B9B

Q4_K_M15.0 GB VRAM70.5 tok/sRuns great

SExcellent

Qwen 3.5 35B A3B35B

Q4_K_M30.1 GB VRAM49.8 tok/sRuns great

SExcellent

Magistral Small 250724B

Q4_K_M24.4 GB VRAM26.4 tok/sRuns great

SExcellent

Devstral Small 2 24B Instruct24B

Q4_K_M24.4 GB VRAM26.4 tok/sRuns great

SExcellent

Qwen 3 32B32B

Q4_K_M30.7 GB VRAM20.1 tok/sRuns great

SExcellent

Qwen 3 14B14B

Q4_K_M18.3 GB VRAM45.5 tok/sRuns great

SExcellent

Qwen 3 30B A3B30.5B

Q4_K_M27.4 GB VRAM54.4 tok/sRuns great

SExcellent

Qwen 2.5 VL 72B72B

Q4_K_M56.1 GB VRAM8.9 tok/sTight fit

SExcellent

Nemotron 3 Nano 30B30B

Q4_K_M28.0 GB VRAM21.1 tok/sRuns great

SExcellent

Qwen 3.5 4B4B

Q4_K_M11.9 GB VRAM56.0 tok/sRuns great

AGreat

Qwen 3 8B8B

Q4_K_M14.4 GB VRAM79.3 tok/sRuns great

SExcellent

Qwen3-Coder-Next80B

Q4_K_M57.6 GB VRAM24.3 tok/sTight fit

SExcellent

Phi-4-reasoning-plus 14B14.699999809265137B

Q4_K_M19.3 GB VRAM43.2 tok/sRuns great

SExcellent

Devstral Small 1.124B

Q4_K_M24.4 GB VRAM26.4 tok/sRuns great

SExcellent

GPT-OSS 20B21B

Q4_K_M22.6 GB VRAM69.1 tok/sRuns great

SExcellent

Nemotron Cascade 2 30B A3B30B

Q4_K_M28.5 GB VRAM55.7 tok/sRuns great

SExcellent

Phi-4 Mini Reasoning 4B3.799999952316284B

Q4_K_M11.1 GB VRAM53.2 tok/sRuns great

AGreat

Jina Embeddings v30.5720000267028809B

F1610.4 GB VRAM8.0 tok/sRuns great

AGreat

Gemma 4 31B30.700000762939453B

Q4_K_M40.7 GB VRAM15.3 tok/sRuns great

SExcellent

BGE M30.5680000185966492B

F169.6 GB VRAM8.0 tok/sRuns great

AGreat

Nemotron Nano 8B8B

Q4_K_M14.1 GB VRAM79.3 tok/sRuns great

AGreat

Ministral 3 14B14B

Q4_K_M18.3 GB VRAM45.3 tok/sRuns great

AGreat

EXAONE 4.0 32B32B

Q4_K_M30.7 GB VRAM19.9 tok/sRuns great

AGreat

Gemma 4 26B A4B25.200000762939453B

Q4_K_M26.3 GB VRAM58.5 tok/sRuns great

AGreat

Llama 3.3 70B70B

Q4_K_M54.9 GB VRAM9.2 tok/sTight fit

AGreat

Codestral 2 25.0822B

Q4_K_M23.2 GB VRAM27.1 tok/sRuns great

AGreat

mxbai Embed Large0.33500000834465027B

F169.5 GB VRAM4.7 tok/sRuns great

BGood

Mistral Small 3.2 24B24B

Q4_K_M24.4 GB VRAM26.4 tok/sRuns great

AGreat

Qwen 2.5 32B32B

Q4_K_M30.7 GB VRAM19.9 tok/sRuns great

AGreat

Snowflake Arctic Embed L0.33500000834465027B

F169.5 GB VRAM4.7 tok/sRuns great

BGood

CogVLM2 19B19B

Q4_K_M21.3 GB VRAM33.4 tok/sRuns great

AGreat

LFM2 24B24B

Q4_K_M24.4 GB VRAM26.4 tok/sRuns great

AGreat

Gemma 3 27B27B

Q4_K_M35.0 GB VRAM17.4 tok/sRuns great

AGreat

Granite 4.1 30B30B

Q4_K_M29.5 GB VRAM21.1 tok/sRuns great

AGreat

Kimi Linear 48B A3B48B

Q4_K_M37.5 GB VRAM12.3 tok/sRuns great

AGreat

Nomic Embed Text v1.50.13699999451637268B

F168.1 GB VRAM2.0 tok/sRuns great

BGood

InternVL2 8B8B

Q4_K_M14.1 GB VRAM79.3 tok/sRuns great

AGreat

OLMo 2 32B32B

Q4_K_M30.7 GB VRAM19.9 tok/sRuns great

AGreat

Mistral Small 24B24B

Q4_K_M24.4 GB VRAM26.4 tok/sRuns great

AGreat

Phi-4 14B14B

Q4_K_M18.9 GB VRAM45.3 tok/sRuns great

AGreat

Qwen 3 4B4B

Q4_K_M11.9 GB VRAM56.0 tok/sRuns great

AGreat

BGE Large EN v1.50.33500000834465027B

F169.5 GB VRAM4.7 tok/sRuns great

BGood

Mistral Small 3.1 24B24B

Q4_K_M24.4 GB VRAM26.4 tok/sRuns great

AGreat

Llama 3.1 70B70B

Q4_K_M54.9 GB VRAM9.2 tok/sTight fit

AGreat

Qwen 2.5 72B72B

Q4_K_M56.1 GB VRAM8.9 tok/sTight fit

AGreat

Qwen 2.5 14B14B

Q4_K_M18.8 GB VRAM45.5 tok/sRuns great

AGreat

Ministral 3 8B8B

Q4_K_M14.4 GB VRAM79.3 tok/sRuns great

AGreat

MiniCPM-V 2.6 8B8B

Q4_K_M14.1 GB VRAM79.3 tok/sRuns great

AGreat

Gemma 3 12B12B

Q4_K_M19.5 GB VRAM39.1 tok/sRuns great

AGreat

Qwen 2.5 VL 7B7B

Q4_K_M12.4 GB VRAM91.5 tok/sRuns great

AGreat

Granite Code 20B20B

Q4_K_M22.7 GB VRAM31.9 tok/sRuns great

AGreat

SQLCoder 7B7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

AGreat

Nemotron Nano 9B v29B

Q4_K_M15.2 GB VRAM70.5 tok/sRuns great

AGreat

DeepSeek Coder V2 16B16B

Q4_K_M20.4 GB VRAM87.8 tok/sRuns great

AGreat

Magistral 7B7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

AGreat

CodeGeeX 4 9B9B

Q4_K_M13.4 GB VRAM71.7 tok/sRuns great

AGreat

Qwen 2.5 Coder 32B32B

Q4_K_M30.7 GB VRAM19.9 tok/sRuns great

AGreat

Gemma 4 E4B8B

Q4_K_M13.5 GB VRAM60.1 tok/sRuns great

AGreat

OLMo 2 13B13B

Q4_K_M17.7 GB VRAM49.0 tok/sRuns great

AGreat

All MiniLM L6 v20.023000000044703484B

F167.6 GB VRAM2.0 tok/sRuns great

BGood

Granite Code 34B34B

Q4_K_M31.7 GB VRAM18.8 tok/sRuns great

AGreat

Command R 35B35B

Q4_K_M31.1 GB VRAM18.3 tok/sRuns great

AGreat

DeepSeek R1 Distill 70B70B

Q4_K_M54.9 GB VRAM9.2 tok/sTight fit

AGreat

Qwen 2.5 7B7B

Q4_K_M12.4 GB VRAM91.5 tok/sRuns great

AGreat

Qwen 2.5 Coder 3B3B

Q4_K_M11.3 GB VRAM42.0 tok/sRuns great

BGood

DeepSeek R1 Distill 32B32B

Q4_K_M30.7 GB VRAM19.9 tok/sRuns great

AGreat

StarCoder 15B15B

Q5_K_M32.7 GB VRAM34.0 tok/sRuns great

AGreat

CodeLlama 13B Instruct13B

Q4_K_M27.4 GB VRAM45.4 tok/sRuns great

AGreat

DevStral 7B7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

AGreat

Codestral Mamba 7B7B

Q4_K_M12.1 GB VRAM97.0 tok/sRuns great

AGreat

Granite Code 8B8B

Q4_K_M14.1 GB VRAM79.3 tok/sRuns great

AGreat

DeepSeek R1 Distill 14B14B

Q4_K_M18.8 GB VRAM45.5 tok/sRuns great

AGreat

Granite 4.1 8B8B

Q4_K_M14.6 GB VRAM79.3 tok/sRuns great

AGreat

Pixtral 12B12B

Q4_K_M17.1 GB VRAM52.9 tok/sRuns great

BGood

StarCoder 7B7B

Q4_K_M18.9 GB VRAM84.3 tok/sRuns great

AGreat

CodeLlama 7B Instruct7B

Q4_K_M19.4 GB VRAM84.3 tok/sRuns great

AGreat

LLaVA 1.6 13B13B

Q4_K_M27.4 GB VRAM45.4 tok/sRuns great

AGreat

Gemma 4 E2B5.099999904632568B

Q4_K_M10.9 GB VRAM71.4 tok/sRuns great

BGood

Ministral 3 3B3B

Q4_K_M9.9 GB VRAM42.0 tok/sRuns great

BGood

Qwen 3.5 2B2B

Q4_K_M10.2 GB VRAM28.0 tok/sRuns great

BGood

Nemotron 70B70B

Q4_K_M54.9 GB VRAM9.2 tok/sTight fit

BGood

GLM-4 9B9B

Q4_K_M13.4 GB VRAM71.7 tok/sRuns great

BGood

Gemma 3 4B4B

Q4_K_M11.8 GB VRAM56.0 tok/sRuns great

BGood

Llama 3.1 8B8B

Q4_K_M14.1 GB VRAM79.3 tok/sRuns great

BGood

Falcon 40B Instruct40B

Q5_K_M37.9 GB VRAM13.9 tok/sRuns great

BGood

WizardLM 13B13B

Q4_K_M27.4 GB VRAM45.4 tok/sRuns great

BGood

MPT-30B-Instruct30B

Q5_K_M52.3 GB VRAM17.0 tok/sRuns great

AGreat

InternLM 7B7B

Q4_K_M19.4 GB VRAM84.3 tok/sRuns great

BGood

InternLM Chat 7B7B

Q4_K_M19.4 GB VRAM84.3 tok/sRuns great

BGood

Solar 7B7B

Q4_K_M14.5 GB VRAM90.6 tok/sRuns great

BGood

Nous Hermes 1.09B

Q4_K_M25.0 GB VRAM65.6 tok/sRuns great

BGood

Nous Dolphin 13B13B

Q5_K_M28.9 GB VRAM39.2 tok/sRuns great

BGood

Vicuna 13B13B

Q4_K_M27.4 GB VRAM45.4 tok/sRuns great

BGood

WizardMath 7B7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

BGood

100

OLMo 2 7B7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

BGood

101

Phi 4 Mini 4B4B

Q4_K_M11.2 GB VRAM56.0 tok/sRuns great

BGood

102

Qwen 2.5 Coder 7B7B

Q4_K_M12.4 GB VRAM91.5 tok/sRuns great

BGood

103

Qwen 3 1.7B1.7000000476837158B

Q4_K_M10.0 GB VRAM23.8 tok/sRuns great

BGood

104

Gemma 2 27B27B

Q4_K_M35.0 GB VRAM17.4 tok/sRuns great

BGood

105

Qwen 2.5 Coder 1.5B1.5B

Q4_K_M8.6 GB VRAM21.0 tok/sRuns great

BGood

106

Qwen 2.5 3B3B

Q4_K_M11.3 GB VRAM42.0 tok/sRuns great

BGood

107

LLaVA 1.5 7B7B

Q4_K_M19.4 GB VRAM84.3 tok/sRuns great

BGood

108

Granite 4.1 3B3B

Q4_K_M10.4 GB VRAM42.0 tok/sRuns great

BGood

109

Phi 3 Mini 3.8B3.799999952316284B

Q4_K_M15.5 GB VRAM53.2 tok/sRuns great

BGood

110

DeepSeek R1 Distill 7B7B

Q4_K_M12.4 GB VRAM91.5 tok/sRuns great

BGood

111

DeepSeek R1 Distill 8B8B

Q4_K_M14.1 GB VRAM79.3 tok/sRuns great

BGood

112

Falcon 7B Instruct7B

Q4_K_M11.7 GB VRAM92.6 tok/sRuns great

BGood

113

MPT-7B-Instruct7B

Q4_K_M19.4 GB VRAM84.3 tok/sRuns great

BGood

114

Baichuan 7B7B

Q4_K_M19.4 GB VRAM84.3 tok/sRuns great

BGood

115

Baichuan 13B13B

Q5_K_M28.9 GB VRAM39.2 tok/sRuns great

BGood

116

Cerebras-GPT 13B13B

Q5_K_M26.4 GB VRAM39.2 tok/sRuns great

BGood

117

Samantha 7B7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

BGood

118

Granite Code 3B3B

Q4_K_M11.6 GB VRAM42.0 tok/sRuns great

BGood

119

Phi 3.5 Mini 4B4B

Q4_K_M15.6 GB VRAM56.0 tok/sRuns great

BGood

120

Mixtral 8x7B47B

Q4_K_M37.9 GB VRAM25.9 tok/sRuns great

BGood

121

Gemma 2 9B9B

Q4_K_M17.9 GB VRAM52.2 tok/sRuns great

BGood

122

Llama 3.2 11B Vision11B

Q4_K_M16.0 GB VRAM57.7 tok/sRuns great

BGood

123

Qwen 2.5 Coder 14B14B

Q4_K_M18.8 GB VRAM45.5 tok/sRuns great

BGood

124

Qwen 2.5 Math 72B72B

Q4_K_M56.1 GB VRAM8.9 tok/sTight fit

BGood

125

Yi 1.5 34B34B

Q4_K_M31.7 GB VRAM18.8 tok/sRuns great

BGood

126

Llama 3.2 3B3B

Q4_K_M10.8 GB VRAM42.0 tok/sRuns great

BGood

127

Mistral Nemo 12B12B

Q4_K_M17.1 GB VRAM52.9 tok/sRuns great

BGood

128

Mistral 7B Instruct v0.37B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

BGood

129

Yi Coder 9B9B

Q4_K_M14.3 GB VRAM71.3 tok/sRuns great

BGood

130

TinyLlama 1.1B1.100000023841858B

Q4_K_M8.3 GB VRAM15.4 tok/sRuns great

CUsable

131

Phi 3 Medium 14B14B

Q4_K_M18.9 GB VRAM45.3 tok/sRuns great

BGood

132

DeepSeek R1 1.5B1.5B

Q4_K_M8.6 GB VRAM21.0 tok/sRuns great

CUsable

133

Codestral 22B22B

Q4_K_M23.2 GB VRAM28.8 tok/sRuns great

BGood

134

DeepSeek LLM 67B67B

Q4_K_M54.0 GB VRAM9.6 tok/sTight fit

BGood

135

Ministral 8B8B

Q4_K_M14.4 GB VRAM79.3 tok/sRuns great

BGood

136

Qwen 2.5 Coder 0.5B0.5B

Q4_K_M7.8 GB VRAM7.0 tok/sRuns great

CUsable

137

Qwen 2.5 1.5B1.5B

Q4_K_M8.6 GB VRAM21.0 tok/sRuns great

CUsable

138

Gemma 3 1B1B

Q4_K_M8.3 GB VRAM14.0 tok/sRuns great

CUsable

139

InternLM 20B20B

Q5_K_M40.0 GB VRAM25.5 tok/sRuns great

BGood

140

SmolLM3 3B3B

Q4_K_M11.1 GB VRAM42.0 tok/sRuns great

CUsable

141

Qwen 3 0.6B0.6000000238418579B

Q4_K_M8.5 GB VRAM8.4 tok/sRuns great

CUsable

142

Aya Expanse 32B32B

Q4_K_M29.3 GB VRAM20.1 tok/sRuns great

CUsable

143

Gemma 2 2B2B

Q4_K_M10.1 GB VRAM28.0 tok/sRuns great

CUsable

144

Llama 3.2 1B1B

Q4_K_M8.4 GB VRAM14.0 tok/sRuns great

CUsable

145

Granite 3.1 8B8B

Q4_K_M14.1 GB VRAM91.2 tok/sRuns great

CUsable

146

Yi 1.5 9B9B

Q4_K_M14.3 GB VRAM71.3 tok/sRuns great

CUsable

147

Qwen 2.5 Math 7B7B

Q4_K_M12.4 GB VRAM91.5 tok/sRuns great

CUsable

148

Qwen 3.5 0.6B0.6000000238418579B

Q4_K_M8.5 GB VRAM8.4 tok/sRuns great

CUsable

149

Qwen 2.5 0.5B0.5B

Q4_K_M7.8 GB VRAM7.0 tok/sRuns great

DPoor

150

OpenChat 7B7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

CUsable

151

Aya Expanse 8B8B

Q4_K_M14.1 GB VRAM79.3 tok/sRuns great

CUsable

152

Yi 34B Chat34B

Q4_K_M31.7 GB VRAM18.8 tok/sRuns great

CUsable

153

StarCoder2 15B15B

Q5_K_M19.3 GB VRAM37.1 tok/sRuns great

CUsable

154

Nemotron Mini 4B4B

Q4_K_M11.7 GB VRAM56.0 tok/sRuns great

CUsable

155

OpenHermes 2.5 7B7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

CUsable

156

Dolphin 2.9 8B8B

Q4_K_M14.1 GB VRAM79.3 tok/sRuns great

CUsable

157

Zephyr 7B Beta7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

CUsable

158

Vicuna 7B7B

Q4_K_M19.4 GB VRAM84.3 tok/sRuns great

CUsable

159

Qwen3.5 27B27B

Q4_K_M26.9 GB VRAM21.9 tok/sRuns great

CUsable

160

Qwen3.5 35B A3B35B

Q4_K_M32.8 GB VRAM16.9 tok/sRuns great

CUsable

161

Qwen3.5 9B9B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

162

gemma 2b2B

Q4_K_M8.8 GB VRAM28.0 tok/sRuns great

CUsable

163

Qwen3.5 9B Uncensored HauhauCS Aggressive9B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

164

gemma 2 2b it2B

Q6_K9.2 GB VRAM28.0 tok/sRuns great

CUsable

165

Meta Llama 3.1 8B Instruct8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

166

Llama 3.2 3B Instruct3B

Q5_K_M9.8 GB VRAM42.0 tok/sRuns great

CUsable

167

Qwen3.5 4B4B

Q4_K_M10.2 GB VRAM56.0 tok/sRuns great

CUsable

168

Llama 2 7B Chat7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

169

llava llama 3 8b v1 18B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

170

Llama 3.2 1B Instruct Q8 01B

Q6_K8.2 GB VRAM14.0 tok/sRuns great

DPoor

171

Starling LM 7B7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

CUsable

172

Qwen2.5 3B Instruct3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

173

Qwen2.5 1.5B Instruct1.5B

Q4_K_M8.4 GB VRAM21.0 tok/sRuns great

CUsable

174

SmolVLM 500M Instruct0.5B

Q6_K7.8 GB VRAM7.0 tok/sRuns great

DPoor

175

Mistral 7B Instruct v0.27B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

176

DeepSeek R1 0528 Qwen3 8B8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

177

TinyLlama 1.1B Chat v1.01.100000023841858B

Q4_K_M8.1 GB VRAM15.4 tok/sRuns great

DPoor

178

gemma 3 27b it27B

Q4_K_M26.9 GB VRAM21.9 tok/sRuns great

CUsable

179

Gemmasutra Mini 2B v12B

Q4_K_M8.8 GB VRAM28.0 tok/sRuns great

CUsable

180

Qwen3.5 9B9B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

181

Mistral 7B Instruct v0.37B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

182

gemma 3 4b it4B

Q4_K_M10.2 GB VRAM56.0 tok/sRuns great

CUsable

183

embeddinggemma 300M0.30000001192092896B

Q6_K7.6 GB VRAM4.2 tok/sRuns great

DPoor

184

Meta Llama 3 8B Instruct8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

185

Qwen3.5 35B A3B35B

Q4_K_M32.8 GB VRAM16.9 tok/sRuns great

CUsable

186

DeepSeek R1 Distill Llama 8B8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

187

Dolphin3.0 Llama3.1 8B8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

188

Llama 3 8B Instruct 32k v0.18B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

189

DeepSeek R1 Distill Qwen 1.5B1.5B

Q4_K_M8.4 GB VRAM21.0 tok/sRuns great

CUsable

190

Meta Llama 3.1 8B Instruct8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

191

vntl llama3 8b v28B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

192

Mistral Small 3.2 24B Instruct 250624B

Q4_K_M24.8 GB VRAM24.6 tok/sRuns great

CUsable

193

DeepSeek R1 0528 Qwen3 8B8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

194

DeepSeek R1 Distill Qwen 14B14B

Q4_K_M17.5 GB VRAM42.2 tok/sRuns great

CUsable

195

gemma 3 4b it4B

Q4_K_M10.2 GB VRAM56.0 tok/sRuns great

CUsable

196

Llama 3.3 70B Instruct70B

Q4_K_M58.2 GB VRAM8.4 tok/sTight fit

CUsable

197

Yi Coder 1.5B Chat1.5B

Q4_K_M8.4 GB VRAM21.0 tok/sRuns great

CUsable

198

gemma 3 12b it12B

Q4_K_M16.0 GB VRAM49.2 tok/sRuns great

CUsable

199

Llama 3.2 1B Instruct1B

Q4_K_M8.0 GB VRAM14.0 tok/sRuns great

DPoor

200

gemma 2 2b it2B

Q4_K_M8.8 GB VRAM28.0 tok/sRuns great

CUsable

201

Llama 3.2 3B Instruct3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

202

gemma 3 1b it1B

Q4_K_M8.0 GB VRAM14.0 tok/sRuns great

DPoor

203

cognitivecomputations Dolphin Mistral 24B Venice Edition24B

Q4_K_M24.8 GB VRAM24.6 tok/sRuns great

CUsable

204

DeepSeek R1 0528 Qwen3 8B8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

205

Mistral Small 24B Instruct 250124B

Q4_K_M24.8 GB VRAM24.6 tok/sRuns great

CUsable

206

Yi 1.5 6B Chat6B

Q4_K_M11.7 GB VRAM84.0 tok/sRuns great

CUsable

207

Ministral 3 3B Instruct 25123B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

208

gemma 3 27b it27B

Q4_K_M26.9 GB VRAM21.9 tok/sRuns great

CUsable

209

Yi Coder 9B Chat9B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

210

glm 4 9b chat 1m9B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

211

Qwen3 8B DeepSeek v3.2 Speciale Distill8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

212

Hermes 3 Llama 3.1 8B8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

213

Mistral 7B Instruct v0.37B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

214

stablelm 2 zephyr 1 6b6B

Q4_K_M11.7 GB VRAM84.0 tok/sRuns great

CUsable

215

Hermes 2 Pro Mistral 7B7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

216

mistral small 3.1 24b instruct 2503 hf24B

Q4_K_M24.8 GB VRAM24.6 tok/sRuns great

CUsable

217

TinyLlama 1.1B Chat v0.31.100000023841858B

Q4_K_M8.1 GB VRAM15.4 tok/sRuns great

DPoor

218

cognitivecomputations Dolphin3.0 R1 Mistral 24B24B

Q4_K_M24.8 GB VRAM24.6 tok/sRuns great

CUsable

219

TinyLlama 1.1B Chat v0.61.100000023841858B

Q4_K_M8.1 GB VRAM15.4 tok/sRuns great

DPoor

220

zephyr 7B beta7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

221

Codestral 22B v0.122B

Q4_K_M23.3 GB VRAM26.8 tok/sRuns great

CUsable

222

SOLAR 10.7B Instruct v1.0 uncensored10.699999809265137B

Q4_K_M15.1 GB VRAM55.2 tok/sRuns great

CUsable

223

Hermes 2 Pro Llama 3 8B8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

224

Dolphin3.0 Llama3.1 8B8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

225

Nous Hermes 2 Mistral 7B DPO7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

226

Codestral 22B v0.122B

Q4_K_M23.3 GB VRAM26.8 tok/sRuns great

CUsable

227

Nous Hermes 2 SOLAR 10.7B10.699999809265137B

Q4_K_M15.1 GB VRAM55.2 tok/sRuns great

CUsable

228

Phi 4 reasoning vision 15B15B

Q4_K_M18.2 GB VRAM39.3 tok/sRuns great

CUsable

229

granite 8b code instruct 4k8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

230

Dolphin Mistral GLM 4.7 Flash 24B Venice Edition Thinking Uncensored i124B

Q4_K_M24.8 GB VRAM24.6 tok/sRuns great

CUsable

231

HELVETE 3B3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

232

StarCoder2 15B15B

Q4_K_M18.2 GB VRAM39.3 tok/sRuns great

CUsable

233

dolphin 2.9.4 llama3.1 8b8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

234

Yi 1.5 9B Chat9B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

235

Falcon H1R 7B7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

236

NousResearch Hermes 4 14B14B

Q4_K_M17.5 GB VRAM42.2 tok/sRuns great

CUsable

237

granite embedding 107m multilingual0.10700000077486038B

Q4_K_M7.5 GB VRAM2.0 tok/sRuns great

DPoor

238

falcon mamba 7b instruct Q4 K M7B

Q4_K_M12.4 GB VRAM97.0 tok/sRuns great

CUsable

239

Hermes 3 Llama 3.2 3B3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

240

zephyr 7B alpha7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

241

Qwen3 48B A4B Savant Commander Distill 12X Closed Open Heretic Uncensored48B

Q4_K_M42.2 GB VRAM12.3 tok/sRuns great

CUsable

242

speechless zephyr code functionary 7b7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

243

stablelm zephyr 3b3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

244

Yi 1.5 6B Chat6B

Q4_K_M11.7 GB VRAM84.0 tok/sRuns great

CUsable

245

Hermes 4.3 36B36B

Q4_K_M33.5 GB VRAM16.4 tok/sRuns great

CUsable

246

openchat 3.6 8b 20240522 IMat8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

247

EXAONE 4.0 1.2B1.2000000476837158B

Q4_K_M8.2 GB VRAM16.8 tok/sRuns great

DPoor

248

StarCoder2 3B3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

249

stablelm 2 1 6b chat imatrix6B

Q4_K_M11.7 GB VRAM84.0 tok/sRuns great

CUsable

250

EXAONE 3.5 2.4B Instruct2.4000000953674316B

Q4_K_M9.0 GB VRAM33.6 tok/sRuns great

CUsable

251

EXAONE 3.5 7.8B Instruct7.800000190734863B

Q4_K_M13.0 GB VRAM75.7 tok/sRuns great

CUsable

252

StarCoder2 7B7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

253

Yi 1.5 6B6B

Q4_K_M11.9 GB VRAM84.0 tok/sRuns great

CUsable

254

aya expanse 8b8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

255

EXAONE 4.0 32B32B

Q4_K_M30.6 GB VRAM18.4 tok/sRuns great

CUsable

256

japanese stablelm instruct gamma 7B7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

257

Yi Coder 1.5B1.5B

Q4_K_M8.4 GB VRAM21.0 tok/sRuns great

DPoor

258

EXAONE 3.5 7.8B Instruct7.800000190734863B

Q4_K_M13.0 GB VRAM75.7 tok/sRuns great

CUsable

259

dolphin v2 8b abliterated i18B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

260

Codestral 22B v0.122B

Q4_K_M23.3 GB VRAM26.8 tok/sRuns great

CUsable

261

StableLM 2 12B12B

Q5_K_M28.1 GB VRAM38.9 tok/sRuns great

CUsable

262

Solar Open 69B REAP i169B

Q4_K_M57.5 GB VRAM8.6 tok/sTight fit

CUsable

263

aya expanse 32b heretic MPOA i132B

Q4_K_M30.6 GB VRAM18.4 tok/sRuns great

CUsable

264

Falcon H1 7B Instruct7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

265

Falcon H1 Tiny 90M Instruct0.09000000357627869B

Q4_K_M7.5 GB VRAM2.0 tok/sRuns great

DPoor

266

EXAONE 3.5 7.8B Instruct i17.800000190734863B

Q4_K_M13.0 GB VRAM75.7 tok/sRuns great

CUsable

267

Falcon H1R 7B7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

268

AI21 Jamba Reasoning 3B3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

269

exaone 3.0 7.8b it7.800000190734863B

Q4_K_M13.0 GB VRAM75.7 tok/sRuns great

CUsable

270

aya expanse 8b orthogonal heretic i18B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

271

Falcon3 1B Instruct abliterated1B

Q4_K_M8.0 GB VRAM14.0 tok/sRuns great

DPoor

272

starcoder2 15b instruct v0.115B

Q4_K_M18.2 GB VRAM39.3 tok/sRuns great

CUsable

273

starcoder2 15b i115B

Q4_K_M18.2 GB VRAM39.3 tok/sRuns great

CUsable

274

GGUF SOLARized GraniStral 14B 2102 YeAM HCT 32QKV14B

Q4_K_M17.5 GB VRAM42.2 tok/sRuns great

CUsable

275

Baichuan M2 32B Q4 K M32B

Q4_K_M30.6 GB VRAM18.4 tok/sRuns great

CUsable

276

Yi 9B Coder i19B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

277

solar finalised finetuned Model 10.7B i110.699999809265137B

Q4_K_M15.1 GB VRAM55.2 tok/sRuns great

CUsable

278

stablelm 3b 4e1t3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

279

cognitivecomputations Dolphin Mistral 24B Venice Edition24B

Q4_K_M24.8 GB VRAM24.6 tok/sRuns great

CUsable

280

Mamba Codestral 7B v0.17B

Q4_K_M12.4 GB VRAM97.0 tok/sRuns great

CUsable

281

ai21labs AI21 Jamba Reasoning 3B3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

282

starcoder2 15b instruct v0.115B

Q4_K_M18.2 GB VRAM39.3 tok/sRuns great

CUsable

283

stabilityai japanese stablelm base gamma 7b7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

284

internlm JanusCoder 14B14B

Q4_K_M17.5 GB VRAM42.2 tok/sRuns great

CUsable

285

stablelm 2 zephyr 1.6b1.600000023841858B

Q4_K_M8.5 GB VRAM22.4 tok/sRuns great

DPoor

286

Falcon H1 1.5B Instruct1.5B

Q4_K_M8.4 GB VRAM21.0 tok/sRuns great

DPoor

287

aya expanse 8b orthogonal heretic8B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

288

internlm2 5 20b chat20B

Q4_K_M21.8 GB VRAM29.5 tok/sRuns great

CUsable

289

baichuan2 7b chat7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

290

Codestral 22B v0.1 IMat22B

Q4_K_M23.3 GB VRAM26.8 tok/sRuns great

CUsable

291

logos16v2 stablelm2 1.6b i11.600000023841858B

Q4_K_M8.5 GB VRAM22.4 tok/sRuns great

DPoor

292

stabilityai japanese stablelm instruct beta 70b70B

Q4_K_M58.2 GB VRAM8.4 tok/sTight fit

CUsable

293

SOLAR 10.7B v1.010.699999809265137B

Q4_K_M15.1 GB VRAM55.2 tok/sRuns great

CUsable

294

GGUF SOLARized GraniStral 14B 1902 YeAM HCT14B

Q4_K_M17.5 GB VRAM42.2 tok/sRuns great

CUsable

295

baichuan inc Baichuan M2 32B32B

Q4_K_M30.6 GB VRAM18.4 tok/sRuns great

CUsable

296

DiscoPOP zephyr 7b gemma7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

297

HelpingAI2 9B9B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

298

ai21labs AI21 Jamba2 3B3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

299

internlm2 limarp chat 20b20B

Q4_K_M21.8 GB VRAM29.5 tok/sRuns great

CUsable

300

TinyLlama 1.1B Chat v1.0 imatrix1.100000023841858B

Q4_K_M8.1 GB VRAM15.4 tok/sRuns great

DPoor

301

starcoder2 7b7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

302

zephyr 7b beta Mistral 7B Instruct v0.27B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

303

OpenChat 3.5 7B Qwen v2.0 i17B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

304

OpenChat 3.5 7B Starling v2.0 i17B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

305

HelpingAI2 6B6B

Q4_K_M11.7 GB VRAM84.0 tok/sRuns great

CUsable

306

HelpingAI 15B i115B

Q4_K_M18.2 GB VRAM39.3 tok/sRuns great

CUsable

307

internlm2 math plus 7b IMat7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

308

CodeNinja 1.0 OpenChat 7B i17B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

309

HelpingAI2 9B i19B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

310

HelpingAI2.5 5B i15B

Q4_K_M10.9 GB VRAM70.0 tok/sRuns great

CUsable

311

Codestral 21B Pruned i121B

Q4_K_M22.6 GB VRAM28.1 tok/sRuns great

CUsable

312

internlm2 5 1 8b chat i18B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

313

internlm2 math plus 20b i120B

Q4_K_M21.8 GB VRAM29.5 tok/sRuns great

CUsable

314

HelpingAI 9B 200k i19B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

315

internlm2 5 7b chat i17B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

316

HelpingAI 3B hindi i13B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

317

internlm3 8b instruct abliterated i18B

Q4_K_M13.1 GB VRAM73.8 tok/sRuns great

CUsable

318

HelpingAI2 6B i16B

Q4_K_M11.7 GB VRAM84.0 tok/sRuns great

CUsable

319

OpenSafetyLab MD Judge v0 2 internlm2 7b7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

320

AI21 Jamba2 3B3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

321

MD Judge v0 2 internlm2 7b i17B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

322

HelpingAI2.5 10B i110B

Q4_K_M14.6 GB VRAM59.0 tok/sRuns great

CUsable

323

HelpingAI 3B hindi3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

324

zephyr 7b gemma sft african ultrachat 100k7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

325

HelpingAI 9B i19B

Q4_K_M13.8 GB VRAM65.6 tok/sRuns great

CUsable

326

Codestral 22B v0.1 i122B

Q4_K_M23.3 GB VRAM26.8 tok/sRuns great

CUsable

327

jointpreferences mistral 7b sft helpful7B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

328

zephyr 7b dpo full i17B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

329

Codestral RAG 19B Pruned i119B

Q4_K_M21.1 GB VRAM31.1 tok/sRuns great

CUsable

330

blossom v3 baichuan2 7b i17B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

331

Helply 10.2b chat i110.199999809265137B

Q4_K_M14.7 GB VRAM57.9 tok/sRuns great

CUsable

332

AI21 Jamba2 3B i13B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

333

blossom v1 baichuan 7b i17B

Q4_K_M12.4 GB VRAM84.3 tok/sRuns great

CUsable

334

BaichuanMed OCR 72B i172B

Q4_K_M59.7 GB VRAM8.2 tok/sTight fit

CUsable

335

Neural Chat 7B7B

Q4_K_M13.5 GB VRAM90.6 tok/sRuns great

CUsable

336

StarCoder2 7B7B

Q4_K_M12.1 GB VRAM92.0 tok/sRuns great

CUsable

337

StarCoder2 3B3B

Q4_K_M9.6 GB VRAM42.0 tok/sRuns great

CUsable

338

DeepSeek LLM 7B7B

Q4_K_M18.9 GB VRAM84.3 tok/sRuns great

CUsable

URL: https://willitrunai.com/browse/best-for/64gb

⇱ Best AI Models for 64GB VRAM — Local LLMs | WillItRunAI