VOOZH about

URL: https://willitrunai.com/models/phi-3.5-mini-4b

โ‡ฑ Phi 3.5 Mini 4B VRAM Requirements โ€” GPU Compatibility


๐Ÿ‘ Microsoft
Microsoft

Phi 3.5 Mini 4B

Legacy
๐Ÿ‘ huggingface
HuggingFace๐Ÿ‘ ollama
Ollama
1.2MDownloads993LikesAug 2024Released128K tokensContextMITLicense39 BasicQuality

Phi 3.5 Mini 4B (4B parameters) requires approximately 10.1 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 12 GB of VRAM.

Get started

โ€” copy & paste to run locally

Copy-paste commands to run Phi 3.5 Mini 4B on your machine.

Run

ollama run phi3.5

Quick specs

Parameters4B
Architecturedense
Context128K tokens
Modalitytext
Min RAM1.6 GB
Rec. RAM2.4 GB (Q4_K_M)
LicenseMIT
FamilyPhi
โœ“ Chat

About this model

Phi-3.5-mini is a lightweight, state-of-the-art open model built upon datasets used for Phi-3 - synthetic data and filtered publicly available websites - with a focus on very high-quality, reasoning dense data. The model belongs to the Phi-3 model family and supports 128K token context length. The model underwent a rigorous enhancement process, incorporating both supervised fine-tuning, proximal policy optimization, and direct preference optimization to ensure precise instruction adherence and robust safety measures.

  • โ€ขMemory/compute constrained environments
  • โ€ขLatency bound scenarios
  • โ€ขStrong reasoning (especially code, math and logic)

Related models

Your hardware

Detecting...

Quick picks

๐Ÿ‘ Intel
Best budgetB
Intel Arc B580 12GB~$249 โ€” 56 tok/s
๐Ÿ‘ NVIDIA
Best overallA
RTX 5070 Ti 16GB~$749 โ€” 76 tok/s

Best hardware

Top picks for Phi 3.5 Mini 4B

RTX 5070 Ti 16GBA
16 GB
RTX 5080 16GBA
16 GB
RTX 5060 Ti 16GBA
16 GB
RTX 4060 Ti 16GBA
16 GB
RTX 4070 Ti Super 16GBA
16 GB

Run this model

Phi 3.5 Mini 4B on RTX 5070 Ti 16GBPhi 3.5 Mini 4B on RTX 5080 16GBPhi 3.5 Mini 4B on RTX 5060 Ti 16GB

Quantization options

VRAM estimates by quant level

No hardware detected โ€” fit column shows raw VRAM estimates

QuantBitsVRAMQualityFit
Q2_K
2
1.6 GB
Lowโ€”
Q3_K_S
3
2.0 GB
Lowโ€”
NVFP4
4
2.2 GB
Mediumโ€”
Q4_K_M
4
2.4 GB
Mediumโ€”
Q5_K_M
5
2.9 GB
Highโ€”
Q6_K
6
3.3 GB
Highโ€”
Q8_0
8
4.3 GB
Very Highโ€”
F16
16
8.2 GB
Maximumโ€”

Quality benchmarks

Phi 3.5 Mini 4B benchmark scores

Benchmark verified

Coding

SWE-bench Verifiedโ€”
HumanEval+62.8%
Aider Polyglotโ€”
LiveCodeBenchโ€”

Reasoning

MMLU-Pro69.0%
GPQA Diamond12.0%
MATH-50019.6%
ARC Challenge84.6%

General

Chatbot Arenaโ€”
IFEval57.7%

Source: official ยท 2024-08-20

Hardware compatibility

Fit estimates across all hardware

Open calculator

Computing compatibility...

Memory breakdown

Reference: RTX 2060 6GB

Weights2.4 GB
KV Cache5.9 GB
Runtime1.2 GB
Headroom0.6 GB

Frequently asked questions

FAQ โ€” Phi 3.5 Mini 4B

See also

Quantization GuideScoring MethodologyVRAM Calculator