Voozh

Qwen3-Coder 30B A3B Instruct30.5B

Q4_K_M25.8 GB VRAM54.4 tok/sRuns great

SExcellent

Qwen 3.5 27B27B

Q4_K_M25.3 GB VRAM23.6 tok/sRuns great

SExcellent

Qwen 3.6 27B27B

Q4_K_M23.1 GB VRAM17.9 tok/sRuns great

SExcellent

Qwen3-VL 30B A3B Instruct30B

Q4_K_M25.5 GB VRAM56.3 tok/sRuns great

SExcellent

Qwen 3.6 35B A3B35B

Q4_K_M31.2 GB VRAM45.7 tok/sRuns great

SExcellent

Qwen 3.5 9B9B

Q4_K_M13.4 GB VRAM70.5 tok/sRuns great

SExcellent

Qwen 3.5 35B A3B35B

Q4_K_M28.5 GB VRAM49.8 tok/sRuns great

SExcellent

Magistral Small 250724B

Q4_K_M22.8 GB VRAM26.4 tok/sRuns great

SExcellent

Devstral Small 2 24B Instruct24B

Q4_K_M22.8 GB VRAM26.4 tok/sRuns great

SExcellent

Qwen 3 32B32B

Q4_K_M29.1 GB VRAM20.1 tok/sRuns great

SExcellent

Qwen 3 14B14B

Q4_K_M16.7 GB VRAM45.5 tok/sRuns great

SExcellent

Qwen 3 30B A3B30.5B

Q4_K_M25.8 GB VRAM54.4 tok/sRuns great

SExcellent

Nemotron 3 Nano 30B30B

Q4_K_M26.4 GB VRAM21.1 tok/sRuns great

SExcellent

Qwen 3.5 4B4B

Q4_K_M10.3 GB VRAM56.0 tok/sRuns great

AGreat

Qwen 3 8B8B

Q4_K_M12.8 GB VRAM79.3 tok/sRuns great

SExcellent

Phi-4-reasoning-plus 14B14.699999809265137B

Q4_K_M17.7 GB VRAM43.2 tok/sRuns great

SExcellent

Devstral Small 1.124B

Q4_K_M22.8 GB VRAM26.4 tok/sRuns great

SExcellent

GPT-OSS 20B21B

Q4_K_M21.0 GB VRAM69.1 tok/sRuns great

SExcellent

Nemotron Cascade 2 30B A3B30B

Q4_K_M26.9 GB VRAM55.7 tok/sRuns great

SExcellent

Phi-4 Mini Reasoning 4B3.799999952316284B

Q4_K_M9.5 GB VRAM53.2 tok/sRuns great

AGreat

Jina Embeddings v30.5720000267028809B

F168.8 GB VRAM8.0 tok/sRuns great

AGreat

Gemma 4 31B30.700000762939453B

Q4_K_M39.1 GB VRAM15.3 tok/sRuns great

SExcellent

BGE M30.5680000185966492B

F168.0 GB VRAM8.0 tok/sRuns great

AGreat

Nemotron Nano 8B8B

Q4_K_M12.5 GB VRAM79.3 tok/sRuns great

AGreat

Ministral 3 14B14B

Q4_K_M16.7 GB VRAM45.3 tok/sRuns great

AGreat

EXAONE 4.0 32B32B

Q4_K_M29.1 GB VRAM19.9 tok/sRuns great

AGreat

Gemma 4 26B A4B25.200000762939453B

Q4_K_M24.7 GB VRAM58.5 tok/sRuns great

SExcellent

Codestral 2 25.0822B

Q4_K_M21.6 GB VRAM27.1 tok/sRuns great

AGreat

mxbai Embed Large0.33500000834465027B

F167.9 GB VRAM4.7 tok/sRuns great

AGreat

Mistral Small 3.2 24B24B

Q4_K_M22.8 GB VRAM26.4 tok/sRuns great

AGreat

Qwen 2.5 32B32B

Q4_K_M29.1 GB VRAM19.9 tok/sRuns great

AGreat

Snowflake Arctic Embed L0.33500000834465027B

F167.9 GB VRAM4.7 tok/sRuns great

BGood

CogVLM2 19B19B

Q4_K_M19.7 GB VRAM33.4 tok/sRuns great

AGreat

LFM2 24B24B

Q4_K_M22.8 GB VRAM26.4 tok/sRuns great

AGreat

Gemma 3 27B27B

Q4_K_M33.4 GB VRAM17.4 tok/sRuns great

AGreat

Granite 4.1 30B30B

Q4_K_M27.9 GB VRAM21.1 tok/sRuns great

AGreat

Kimi Linear 48B A3B48B

Q4_K_M35.9 GB VRAM12.3 tok/sRuns great

AGreat

Nomic Embed Text v1.50.13699999451637268B

F166.5 GB VRAM2.0 tok/sRuns great

BGood

InternVL2 8B8B

Q4_K_M12.5 GB VRAM79.3 tok/sRuns great

AGreat

OLMo 2 32B32B

Q4_K_M29.1 GB VRAM19.9 tok/sRuns great

AGreat

Mistral Small 24B24B

Q4_K_M22.8 GB VRAM26.4 tok/sRuns great

AGreat

Phi-4 14B14B

Q4_K_M17.3 GB VRAM45.3 tok/sRuns great

AGreat

Qwen 3 4B4B

Q4_K_M10.3 GB VRAM56.0 tok/sRuns great

AGreat

BGE Large EN v1.50.33500000834465027B

F167.9 GB VRAM4.7 tok/sRuns great

BGood

Mistral Small 3.1 24B24B

Q4_K_M22.8 GB VRAM26.4 tok/sRuns great

AGreat

Qwen 2.5 14B14B

Q4_K_M17.2 GB VRAM45.5 tok/sRuns great

AGreat

Ministral 3 8B8B

Q4_K_M12.8 GB VRAM79.3 tok/sRuns great

AGreat

MiniCPM-V 2.6 8B8B

Q4_K_M12.5 GB VRAM79.3 tok/sRuns great

AGreat

Gemma 3 12B12B

Q4_K_M17.9 GB VRAM39.1 tok/sRuns great

AGreat

Qwen 2.5 VL 7B7B

Q4_K_M10.8 GB VRAM91.5 tok/sRuns great

AGreat

Granite Code 20B20B

Q4_K_M21.1 GB VRAM31.9 tok/sRuns great

AGreat

SQLCoder 7B7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

AGreat

Nemotron Nano 9B v29B

Q4_K_M13.6 GB VRAM70.5 tok/sRuns great

AGreat

DeepSeek Coder V2 16B16B

Q4_K_M18.8 GB VRAM87.8 tok/sRuns great

AGreat

Magistral 7B7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

AGreat

CodeGeeX 4 9B9B

Q4_K_M11.8 GB VRAM71.7 tok/sRuns great

AGreat

Qwen 2.5 Coder 32B32B

Q4_K_M29.1 GB VRAM19.9 tok/sRuns great

AGreat

Gemma 4 E4B8B

Q4_K_M11.9 GB VRAM60.1 tok/sRuns great

AGreat

OLMo 2 13B13B

Q4_K_M16.1 GB VRAM49.0 tok/sRuns great

AGreat

All MiniLM L6 v20.023000000044703484B

F166.0 GB VRAM2.0 tok/sRuns great

BGood

Granite Code 34B34B

Q4_K_M30.1 GB VRAM18.8 tok/sRuns great

AGreat

Command R 35B35B

Q4_K_M29.5 GB VRAM18.3 tok/sRuns great

AGreat

Qwen 2.5 7B7B

Q4_K_M10.8 GB VRAM91.5 tok/sRuns great

AGreat

Qwen 2.5 Coder 3B3B

Q4_K_M9.7 GB VRAM42.0 tok/sRuns great

BGood

DeepSeek R1 Distill 32B32B

Q4_K_M29.1 GB VRAM19.9 tok/sRuns great

AGreat

StarCoder 15B15B

Q5_K_M31.1 GB VRAM34.0 tok/sRuns great

AGreat

CodeLlama 13B Instruct13B

Q4_K_M25.8 GB VRAM45.4 tok/sRuns great

AGreat

DevStral 7B7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

AGreat

Codestral Mamba 7B7B

Q4_K_M10.5 GB VRAM97.0 tok/sRuns great

AGreat

Granite Code 8B8B

Q4_K_M12.5 GB VRAM79.3 tok/sRuns great

AGreat

DeepSeek R1 Distill 14B14B

Q4_K_M17.2 GB VRAM45.5 tok/sRuns great

AGreat

Granite 4.1 8B8B

Q4_K_M13.0 GB VRAM79.3 tok/sRuns great

AGreat

Pixtral 12B12B

Q4_K_M15.5 GB VRAM52.9 tok/sRuns great

AGreat

StarCoder 7B7B

Q4_K_M17.3 GB VRAM84.3 tok/sRuns great

AGreat

CodeLlama 7B Instruct7B

Q4_K_M17.8 GB VRAM84.3 tok/sRuns great

AGreat

LLaVA 1.6 13B13B

Q4_K_M25.8 GB VRAM45.4 tok/sRuns great

AGreat

Gemma 4 E2B5.099999904632568B

Q4_K_M9.3 GB VRAM71.4 tok/sRuns great

BGood

Ministral 3 3B3B

Q4_K_M8.3 GB VRAM42.0 tok/sRuns great

BGood

Qwen 3.5 2B2B

Q4_K_M8.6 GB VRAM28.0 tok/sRuns great

BGood

GLM-4 9B9B

Q4_K_M11.8 GB VRAM71.7 tok/sRuns great

BGood

Gemma 3 4B4B

Q4_K_M10.2 GB VRAM56.0 tok/sRuns great

BGood

Llama 3.1 8B8B

Q4_K_M12.5 GB VRAM79.3 tok/sRuns great

BGood

Falcon 40B Instruct40B

Q5_K_M36.3 GB VRAM13.9 tok/sRuns great

AGreat

WizardLM 13B13B

Q4_K_M25.8 GB VRAM45.4 tok/sRuns great

AGreat

MPT-30B-Instruct30B

Q5_K_M50.7 GB VRAM11.3 tok/sNeeds offload

BGood

InternLM 7B7B

Q4_K_M17.8 GB VRAM84.3 tok/sRuns great

AGreat

InternLM Chat 7B7B

Q4_K_M17.8 GB VRAM84.3 tok/sRuns great

AGreat

Solar 7B7B

Q4_K_M12.9 GB VRAM90.6 tok/sRuns great

BGood

Nous Hermes 1.09B

Q4_K_M23.4 GB VRAM65.6 tok/sRuns great

AGreat

Nous Dolphin 13B13B

Q5_K_M27.3 GB VRAM39.2 tok/sRuns great

AGreat

Vicuna 13B13B

Q4_K_M25.8 GB VRAM45.4 tok/sRuns great

AGreat

WizardMath 7B7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

BGood

OLMo 2 7B7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

BGood

Phi 4 Mini 4B4B

Q4_K_M9.6 GB VRAM56.0 tok/sRuns great

BGood

Qwen 2.5 Coder 7B7B

Q4_K_M10.8 GB VRAM91.5 tok/sRuns great

BGood

Qwen 3 1.7B1.7000000476837158B

Q4_K_M8.4 GB VRAM23.8 tok/sRuns great

BGood

Gemma 2 27B27B

Q4_K_M33.4 GB VRAM17.4 tok/sRuns great

AGreat

Qwen 2.5 Coder 1.5B1.5B

Q4_K_M7.0 GB VRAM21.0 tok/sRuns great

BGood

Qwen 2.5 3B3B

Q4_K_M9.7 GB VRAM42.0 tok/sRuns great

BGood

100

LLaVA 1.5 7B7B

Q4_K_M17.8 GB VRAM84.3 tok/sRuns great

BGood

101

Granite 4.1 3B3B

Q4_K_M8.8 GB VRAM42.0 tok/sRuns great

BGood

102

Phi 3 Mini 3.8B3.799999952316284B

Q4_K_M13.9 GB VRAM53.2 tok/sRuns great

BGood

103

DeepSeek R1 Distill 7B7B

Q4_K_M10.8 GB VRAM91.5 tok/sRuns great

BGood

104

DeepSeek R1 Distill 8B8B

Q4_K_M12.5 GB VRAM79.3 tok/sRuns great

BGood

105

Falcon 7B Instruct7B

Q4_K_M10.1 GB VRAM92.6 tok/sRuns great

BGood

106

MPT-7B-Instruct7B

Q4_K_M17.8 GB VRAM84.3 tok/sRuns great

BGood

107

Baichuan 7B7B

Q4_K_M17.8 GB VRAM84.3 tok/sRuns great

BGood

108

Baichuan 13B13B

Q5_K_M27.3 GB VRAM39.2 tok/sRuns great

BGood

109

Cerebras-GPT 13B13B

Q5_K_M24.8 GB VRAM39.2 tok/sRuns great

BGood

110

Samantha 7B7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

BGood

111

Granite Code 3B3B

Q4_K_M10.0 GB VRAM42.0 tok/sRuns great

BGood

112

Phi 3.5 Mini 4B4B

Q4_K_M14.0 GB VRAM56.0 tok/sRuns great

BGood

113

Mixtral 8x7B47B

Q4_K_M36.3 GB VRAM25.9 tok/sRuns great

BGood

114

Gemma 2 9B9B

Q4_K_M16.3 GB VRAM52.2 tok/sRuns great

BGood

115

Llama 3.2 11B Vision11B

Q4_K_M14.4 GB VRAM57.7 tok/sRuns great

BGood

116

Qwen 2.5 Coder 14B14B

Q4_K_M17.2 GB VRAM45.5 tok/sRuns great

BGood

117

Yi 1.5 34B34B

Q4_K_M30.1 GB VRAM18.8 tok/sRuns great

BGood

118

Llama 3.2 3B3B

Q4_K_M9.2 GB VRAM42.0 tok/sRuns great

BGood

119

Mistral Nemo 12B12B

Q4_K_M15.5 GB VRAM52.9 tok/sRuns great

BGood

120

Mistral 7B Instruct v0.37B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

BGood

121

Yi Coder 9B9B

Q4_K_M12.7 GB VRAM71.3 tok/sRuns great

BGood

122

TinyLlama 1.1B1.100000023841858B

Q4_K_M6.7 GB VRAM15.4 tok/sRuns great

CUsable

123

Phi 3 Medium 14B14B

Q4_K_M17.3 GB VRAM45.3 tok/sRuns great

BGood

124

DeepSeek R1 1.5B1.5B

Q4_K_M7.0 GB VRAM21.0 tok/sRuns great

CUsable

125

Codestral 22B22B

Q4_K_M21.6 GB VRAM28.8 tok/sRuns great

BGood

126

Ministral 8B8B

Q4_K_M12.8 GB VRAM79.3 tok/sRuns great

BGood

127

Qwen 2.5 Coder 0.5B0.5B

Q4_K_M6.2 GB VRAM7.0 tok/sRuns great

CUsable

128

Qwen 2.5 1.5B1.5B

Q4_K_M7.0 GB VRAM21.0 tok/sRuns great

CUsable

129

Gemma 3 1B1B

Q4_K_M6.7 GB VRAM14.0 tok/sRuns great

CUsable

130

InternLM 20B20B

Q5_K_M38.4 GB VRAM25.5 tok/sRuns great

BGood

131

SmolLM3 3B3B

Q4_K_M9.5 GB VRAM42.0 tok/sRuns great

CUsable

132

Qwen 3 0.6B0.6000000238418579B

Q4_K_M6.9 GB VRAM8.4 tok/sRuns great

CUsable

133

Aya Expanse 32B32B

Q4_K_M27.7 GB VRAM20.1 tok/sRuns great

CUsable

134

Gemma 2 2B2B

Q4_K_M8.5 GB VRAM28.0 tok/sRuns great

CUsable

135

Llama 3.2 1B1B

Q4_K_M6.8 GB VRAM14.0 tok/sRuns great

CUsable

136

Granite 3.1 8B8B

Q4_K_M12.5 GB VRAM91.2 tok/sRuns great

CUsable

137

Yi 1.5 9B9B

Q4_K_M12.7 GB VRAM71.3 tok/sRuns great

CUsable

138

Qwen 2.5 Math 7B7B

Q4_K_M10.8 GB VRAM91.5 tok/sRuns great

CUsable

139

Qwen 3.5 0.6B0.6000000238418579B

Q4_K_M6.9 GB VRAM8.4 tok/sRuns great

CUsable

140

Qwen 2.5 0.5B0.5B

Q4_K_M6.2 GB VRAM7.0 tok/sRuns great

DPoor

141

OpenChat 7B7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

CUsable

142

Aya Expanse 8B8B

Q4_K_M12.5 GB VRAM79.3 tok/sRuns great

CUsable

143

Yi 34B Chat34B

Q4_K_M30.1 GB VRAM18.8 tok/sRuns great

CUsable

144

StarCoder2 15B15B

Q5_K_M17.7 GB VRAM37.1 tok/sRuns great

CUsable

145

Nemotron Mini 4B4B

Q4_K_M10.1 GB VRAM56.0 tok/sRuns great

CUsable

146

OpenHermes 2.5 7B7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

CUsable

147

Dolphin 2.9 8B8B

Q4_K_M12.5 GB VRAM79.3 tok/sRuns great

CUsable

148

Zephyr 7B Beta7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

CUsable

149

Vicuna 7B7B

Q4_K_M17.8 GB VRAM84.3 tok/sRuns great

CUsable

150

Qwen3.5 27B27B

Q4_K_M25.3 GB VRAM21.9 tok/sRuns great

CUsable

151

Qwen3.5 35B A3B35B

Q4_K_M31.2 GB VRAM16.9 tok/sRuns great

CUsable

152

Qwen3.5 9B9B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

153

gemma 2b2B

Q4_K_M7.2 GB VRAM28.0 tok/sRuns great

CUsable

154

Qwen3.5 9B Uncensored HauhauCS Aggressive9B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

155

gemma 2 2b it2B

Q6_K7.6 GB VRAM28.0 tok/sRuns great

CUsable

156

Meta Llama 3.1 8B Instruct8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

157

Llama 3.2 3B Instruct3B

Q5_K_M8.2 GB VRAM42.0 tok/sRuns great

CUsable

158

Qwen3.5 4B4B

Q4_K_M8.6 GB VRAM56.0 tok/sRuns great

CUsable

159

Llama 2 7B Chat7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

160

llava llama 3 8b v1 18B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

161

Llama 3.2 1B Instruct Q8 01B

Q6_K6.6 GB VRAM14.0 tok/sRuns great

DPoor

162

Starling LM 7B7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

CUsable

163

Qwen2.5 3B Instruct3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

164

Qwen2.5 1.5B Instruct1.5B

Q4_K_M6.8 GB VRAM21.0 tok/sRuns great

CUsable

165

SmolVLM 500M Instruct0.5B

Q6_K6.2 GB VRAM7.0 tok/sRuns great

DPoor

166

Mistral 7B Instruct v0.27B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

167

DeepSeek R1 0528 Qwen3 8B8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

168

TinyLlama 1.1B Chat v1.01.100000023841858B

Q4_K_M6.5 GB VRAM15.4 tok/sRuns great

DPoor

169

gemma 3 27b it27B

Q4_K_M25.3 GB VRAM21.9 tok/sRuns great

CUsable

170

Gemmasutra Mini 2B v12B

Q4_K_M7.2 GB VRAM28.0 tok/sRuns great

CUsable

171

Qwen3.5 9B9B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

172

Mistral 7B Instruct v0.37B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

173

gemma 3 4b it4B

Q4_K_M8.6 GB VRAM56.0 tok/sRuns great

CUsable

174

embeddinggemma 300M0.30000001192092896B

Q6_K6.0 GB VRAM4.2 tok/sRuns great

DPoor

175

Meta Llama 3 8B Instruct8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

176

Qwen3.5 35B A3B35B

Q4_K_M31.2 GB VRAM16.9 tok/sRuns great

CUsable

177

DeepSeek R1 Distill Llama 8B8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

178

Dolphin3.0 Llama3.1 8B8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

179

Llama 3 8B Instruct 32k v0.18B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

180

DeepSeek R1 Distill Qwen 1.5B1.5B

Q4_K_M6.8 GB VRAM21.0 tok/sRuns great

CUsable

181

Meta Llama 3.1 8B Instruct8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

182

vntl llama3 8b v28B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

183

Mistral Small 3.2 24B Instruct 250624B

Q4_K_M23.2 GB VRAM24.6 tok/sRuns great

CUsable

184

DeepSeek R1 0528 Qwen3 8B8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

185

DeepSeek R1 Distill Qwen 14B14B

Q4_K_M15.9 GB VRAM42.2 tok/sRuns great

CUsable

186

gemma 3 4b it4B

Q4_K_M8.6 GB VRAM56.0 tok/sRuns great

CUsable

187

Yi Coder 1.5B Chat1.5B

Q4_K_M6.8 GB VRAM21.0 tok/sRuns great

CUsable

188

gemma 3 12b it12B

Q4_K_M14.4 GB VRAM49.2 tok/sRuns great

CUsable

189

Llama 3.2 1B Instruct1B

Q4_K_M6.4 GB VRAM14.0 tok/sRuns great

DPoor

190

gemma 2 2b it2B

Q4_K_M7.2 GB VRAM28.0 tok/sRuns great

CUsable

191

Llama 3.2 3B Instruct3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

192

gemma 3 1b it1B

Q4_K_M6.4 GB VRAM14.0 tok/sRuns great

DPoor

193

cognitivecomputations Dolphin Mistral 24B Venice Edition24B

Q4_K_M23.2 GB VRAM24.6 tok/sRuns great

CUsable

194

DeepSeek R1 0528 Qwen3 8B8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

195

Mistral Small 24B Instruct 250124B

Q4_K_M23.2 GB VRAM24.6 tok/sRuns great

CUsable

196

Yi 1.5 6B Chat6B

Q4_K_M10.1 GB VRAM84.0 tok/sRuns great

CUsable

197

Ministral 3 3B Instruct 25123B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

198

gemma 3 27b it27B

Q4_K_M25.3 GB VRAM21.9 tok/sRuns great

CUsable

199

Yi Coder 9B Chat9B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

200

glm 4 9b chat 1m9B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

201

Qwen3 8B DeepSeek v3.2 Speciale Distill8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

202

Hermes 3 Llama 3.1 8B8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

203

Mistral 7B Instruct v0.37B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

204

stablelm 2 zephyr 1 6b6B

Q4_K_M10.1 GB VRAM84.0 tok/sRuns great

CUsable

205

Hermes 2 Pro Mistral 7B7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

206

mistral small 3.1 24b instruct 2503 hf24B

Q4_K_M23.2 GB VRAM24.6 tok/sRuns great

CUsable

207

TinyLlama 1.1B Chat v0.31.100000023841858B

Q4_K_M6.5 GB VRAM15.4 tok/sRuns great

DPoor

208

cognitivecomputations Dolphin3.0 R1 Mistral 24B24B

Q4_K_M23.2 GB VRAM24.6 tok/sRuns great

CUsable

209

TinyLlama 1.1B Chat v0.61.100000023841858B

Q4_K_M6.5 GB VRAM15.4 tok/sRuns great

DPoor

210

zephyr 7B beta7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

211

Codestral 22B v0.122B

Q4_K_M21.7 GB VRAM26.8 tok/sRuns great

CUsable

212

SOLAR 10.7B Instruct v1.0 uncensored10.699999809265137B

Q4_K_M13.5 GB VRAM55.2 tok/sRuns great

CUsable

213

Hermes 2 Pro Llama 3 8B8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

214

Dolphin3.0 Llama3.1 8B8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

215

Nous Hermes 2 Mistral 7B DPO7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

216

Codestral 22B v0.122B

Q4_K_M21.7 GB VRAM26.8 tok/sRuns great

CUsable

217

Nous Hermes 2 SOLAR 10.7B10.699999809265137B

Q4_K_M13.5 GB VRAM55.2 tok/sRuns great

CUsable

218

Phi 4 reasoning vision 15B15B

Q4_K_M16.6 GB VRAM39.3 tok/sRuns great

CUsable

219

granite 8b code instruct 4k8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

220

Dolphin Mistral GLM 4.7 Flash 24B Venice Edition Thinking Uncensored i124B

Q4_K_M23.2 GB VRAM24.6 tok/sRuns great

CUsable

221

HELVETE 3B3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

222

StarCoder2 15B15B

Q4_K_M16.6 GB VRAM39.3 tok/sRuns great

CUsable

223

dolphin 2.9.4 llama3.1 8b8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

224

Yi 1.5 9B Chat9B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

225

Falcon H1R 7B7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

226

NousResearch Hermes 4 14B14B

Q4_K_M15.9 GB VRAM42.2 tok/sRuns great

CUsable

227

granite embedding 107m multilingual0.10700000077486038B

Q4_K_M5.9 GB VRAM2.0 tok/sRuns great

DPoor

228

falcon mamba 7b instruct Q4 K M7B

Q4_K_M10.8 GB VRAM97.0 tok/sRuns great

CUsable

229

Hermes 3 Llama 3.2 3B3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

230

zephyr 7B alpha7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

231

Qwen3 48B A4B Savant Commander Distill 12X Closed Open Heretic Uncensored48B

Q4_K_M40.6 GB VRAM12.3 tok/sTight fit

CUsable

232

speechless zephyr code functionary 7b7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

233

stablelm zephyr 3b3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

234

Yi 1.5 6B Chat6B

Q4_K_M10.1 GB VRAM84.0 tok/sRuns great

CUsable

235

Hermes 4.3 36B36B

Q4_K_M31.9 GB VRAM16.4 tok/sRuns great

CUsable

236

openchat 3.6 8b 20240522 IMat8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

237

EXAONE 4.0 1.2B1.2000000476837158B

Q4_K_M6.6 GB VRAM16.8 tok/sRuns great

DPoor

238

StarCoder2 3B3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

239

stablelm 2 1 6b chat imatrix6B

Q4_K_M10.1 GB VRAM84.0 tok/sRuns great

CUsable

240

EXAONE 3.5 2.4B Instruct2.4000000953674316B

Q4_K_M7.4 GB VRAM33.6 tok/sRuns great

CUsable

241

EXAONE 3.5 7.8B Instruct7.800000190734863B

Q4_K_M11.4 GB VRAM75.7 tok/sRuns great

CUsable

242

StarCoder2 7B7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

243

Yi 1.5 6B6B

Q4_K_M10.3 GB VRAM84.0 tok/sRuns great

CUsable

244

aya expanse 8b8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

245

EXAONE 4.0 32B32B

Q4_K_M29.0 GB VRAM18.4 tok/sRuns great

CUsable

246

japanese stablelm instruct gamma 7B7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

247

Yi Coder 1.5B1.5B

Q4_K_M6.8 GB VRAM21.0 tok/sRuns great

DPoor

248

EXAONE 3.5 7.8B Instruct7.800000190734863B

Q4_K_M11.4 GB VRAM75.7 tok/sRuns great

CUsable

249

dolphin v2 8b abliterated i18B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

250

Codestral 22B v0.122B

Q4_K_M21.7 GB VRAM26.8 tok/sRuns great

CUsable

251

StableLM 2 12B12B

Q5_K_M26.5 GB VRAM38.9 tok/sRuns great

CUsable

252

aya expanse 32b heretic MPOA i132B

Q4_K_M29.0 GB VRAM18.4 tok/sRuns great

CUsable

253

Falcon H1 7B Instruct7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

254

Falcon H1 Tiny 90M Instruct0.09000000357627869B

Q4_K_M5.9 GB VRAM2.0 tok/sRuns great

DPoor

255

EXAONE 3.5 7.8B Instruct i17.800000190734863B

Q4_K_M11.4 GB VRAM75.7 tok/sRuns great

CUsable

256

Falcon H1R 7B7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

257

AI21 Jamba Reasoning 3B3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

258

exaone 3.0 7.8b it7.800000190734863B

Q4_K_M11.4 GB VRAM75.7 tok/sRuns great

CUsable

259

aya expanse 8b orthogonal heretic i18B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

260

Falcon3 1B Instruct abliterated1B

Q4_K_M6.4 GB VRAM14.0 tok/sRuns great

DPoor

261

starcoder2 15b instruct v0.115B

Q4_K_M16.6 GB VRAM39.3 tok/sRuns great

CUsable

262

starcoder2 15b i115B

Q4_K_M16.6 GB VRAM39.3 tok/sRuns great

CUsable

263

GGUF SOLARized GraniStral 14B 2102 YeAM HCT 32QKV14B

Q4_K_M15.9 GB VRAM42.2 tok/sRuns great

CUsable

264

Baichuan M2 32B Q4 K M32B

Q4_K_M29.0 GB VRAM18.4 tok/sRuns great

CUsable

265

Yi 9B Coder i19B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

266

solar finalised finetuned Model 10.7B i110.699999809265137B

Q4_K_M13.5 GB VRAM55.2 tok/sRuns great

CUsable

267

stablelm 3b 4e1t3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

268

cognitivecomputations Dolphin Mistral 24B Venice Edition24B

Q4_K_M23.2 GB VRAM24.6 tok/sRuns great

CUsable

269

Mamba Codestral 7B v0.17B

Q4_K_M10.8 GB VRAM97.0 tok/sRuns great

CUsable

270

ai21labs AI21 Jamba Reasoning 3B3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

271

starcoder2 15b instruct v0.115B

Q4_K_M16.6 GB VRAM39.3 tok/sRuns great

CUsable

272

stabilityai japanese stablelm base gamma 7b7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

273

internlm JanusCoder 14B14B

Q4_K_M15.9 GB VRAM42.2 tok/sRuns great

CUsable

274

stablelm 2 zephyr 1.6b1.600000023841858B

Q4_K_M6.9 GB VRAM22.4 tok/sRuns great

CUsable

275

Falcon H1 1.5B Instruct1.5B

Q4_K_M6.8 GB VRAM21.0 tok/sRuns great

DPoor

276

aya expanse 8b orthogonal heretic8B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

277

internlm2 5 20b chat20B

Q4_K_M20.2 GB VRAM29.5 tok/sRuns great

CUsable

278

baichuan2 7b chat7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

279

Codestral 22B v0.1 IMat22B

Q4_K_M21.7 GB VRAM26.8 tok/sRuns great

CUsable

280

logos16v2 stablelm2 1.6b i11.600000023841858B

Q4_K_M6.9 GB VRAM22.4 tok/sRuns great

CUsable

281

SOLAR 10.7B v1.010.699999809265137B

Q4_K_M13.5 GB VRAM55.2 tok/sRuns great

CUsable

282

GGUF SOLARized GraniStral 14B 1902 YeAM HCT14B

Q4_K_M15.9 GB VRAM42.2 tok/sRuns great

CUsable

283

baichuan inc Baichuan M2 32B32B

Q4_K_M29.0 GB VRAM18.4 tok/sRuns great

CUsable

284

DiscoPOP zephyr 7b gemma7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

285

HelpingAI2 9B9B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

286

ai21labs AI21 Jamba2 3B3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

287

internlm2 limarp chat 20b20B

Q4_K_M20.2 GB VRAM29.5 tok/sRuns great

CUsable

288

TinyLlama 1.1B Chat v1.0 imatrix1.100000023841858B

Q4_K_M6.5 GB VRAM15.4 tok/sRuns great

DPoor

289

starcoder2 7b7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

290

zephyr 7b beta Mistral 7B Instruct v0.27B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

291

OpenChat 3.5 7B Qwen v2.0 i17B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

292

OpenChat 3.5 7B Starling v2.0 i17B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

293

HelpingAI2 6B6B

Q4_K_M10.1 GB VRAM84.0 tok/sRuns great

CUsable

294

HelpingAI 15B i115B

Q4_K_M16.6 GB VRAM39.3 tok/sRuns great

CUsable

295

internlm2 math plus 7b IMat7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

296

CodeNinja 1.0 OpenChat 7B i17B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

297

HelpingAI2 9B i19B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

298

HelpingAI2.5 5B i15B

Q4_K_M9.3 GB VRAM70.0 tok/sRuns great

CUsable

299

Codestral 21B Pruned i121B

Q4_K_M21.0 GB VRAM28.1 tok/sRuns great

CUsable

300

internlm2 5 1 8b chat i18B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

301

internlm2 math plus 20b i120B

Q4_K_M20.2 GB VRAM29.5 tok/sRuns great

CUsable

302

HelpingAI 9B 200k i19B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

303

internlm2 5 7b chat i17B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

304

HelpingAI 3B hindi i13B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

305

internlm3 8b instruct abliterated i18B

Q4_K_M11.5 GB VRAM73.8 tok/sRuns great

CUsable

306

HelpingAI2 6B i16B

Q4_K_M10.1 GB VRAM84.0 tok/sRuns great

CUsable

307

OpenSafetyLab MD Judge v0 2 internlm2 7b7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

308

AI21 Jamba2 3B3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

309

MD Judge v0 2 internlm2 7b i17B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

310

HelpingAI2.5 10B i110B

Q4_K_M13.0 GB VRAM59.0 tok/sRuns great

CUsable

311

HelpingAI 3B hindi3B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

312

zephyr 7b gemma sft african ultrachat 100k7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

313

HelpingAI 9B i19B

Q4_K_M12.2 GB VRAM65.6 tok/sRuns great

CUsable

314

Codestral 22B v0.1 i122B

Q4_K_M21.7 GB VRAM26.8 tok/sRuns great

CUsable

315

jointpreferences mistral 7b sft helpful7B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

316

zephyr 7b dpo full i17B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

317

Codestral RAG 19B Pruned i119B

Q4_K_M19.5 GB VRAM31.1 tok/sRuns great

CUsable

318

blossom v3 baichuan2 7b i17B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

319

Helply 10.2b chat i110.199999809265137B

Q4_K_M13.1 GB VRAM57.9 tok/sRuns great

CUsable

320

AI21 Jamba2 3B i13B

Q4_K_M7.9 GB VRAM42.0 tok/sRuns great

CUsable

321

blossom v1 baichuan 7b i17B

Q4_K_M10.8 GB VRAM84.3 tok/sRuns great

CUsable

322

Neural Chat 7B7B

Q4_K_M11.9 GB VRAM90.6 tok/sRuns great

CUsable

323

StarCoder2 7B7B

Q4_K_M10.5 GB VRAM92.0 tok/sRuns great

CUsable

324

StarCoder2 3B3B

Q4_K_M8.0 GB VRAM42.0 tok/sRuns great

CUsable

325

DeepSeek LLM 7B7B

Q4_K_M17.3 GB VRAM84.3 tok/sRuns great

CUsable

URL: https://willitrunai.com/browse/best-for/48gb

⇱ Best AI Models for 48GB VRAM — Local LLMs | WillItRunAI