VOOZH about

URL: https://willitrunai.com/models/phi-3-medium-14b

โ‡ฑ Phi 3 Medium 14B VRAM Requirements โ€” GPU Compatibility


๐Ÿ‘ Microsoft
Microsoft

Phi 3 Medium 14B

Current
๐Ÿ‘ huggingface
HuggingFace๐Ÿ‘ ollama
Ollama
7.9KDownloads388LikesMay 2024Released128K tokensContextMITLicense29 EntryQuality

Phi 3 Medium 14B (14B parameters) requires approximately 13.4 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 16 GB of VRAM.

Get started

โ€” copy & paste to run locally

Copy-paste commands to run Phi 3 Medium 14B on your machine.

Run

ollama run phi3:medium

Quick specs

Parameters14B
Architecturedense
Context128K tokens
Modalitytext
Min RAM5.5 GB
Rec. RAM8.5 GB (Q4_K_M)
LicenseMIT
FamilyPhi
โœ“ Chatโœ“ Reasoning

About this model

The Phi-3-Medium-128K-Instruct is a 14B parameters, lightweight, state-of-the-art open model trained with the Phi-3 datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties. The model belongs to the Phi-3 family with the Medium version in two variants 4k and 128K which is the context length (in tokens) that it can support.

  • โ€ขMemory/compute constrained environments
  • โ€ขLatency bound scenarios
  • โ€ขStrong reasoning (especially code, math and logic)

Related models

Your hardware

Detecting...

Quick picks

Best budgetB
RX 7600 XT 16GB~$329 โ€” 21 tok/s
๐Ÿ‘ NVIDIA
Best overallB
RTX A4500 20GB~$2,000 โ€” 63 tok/s

Best hardware

Top picks for Phi 3 Medium 14B

RTX A4500 20GBB
20 GB
RX 7900 XT 20GBB
20 GB
RTX 4090 24GBB
24 GB
NVIDIA A30 24GBB
24 GB
RTX 5090 Laptop 24GBB
24 GB

Run this model

Phi 3 Medium 14B on RTX A4500 20GBPhi 3 Medium 14B on RX 7900 XT 20GBPhi 3 Medium 14B on RTX 4090 24GB

Quantization options

VRAM estimates by quant level

No hardware detected โ€” fit column shows raw VRAM estimates

QuantBitsVRAMQualityFit
Q2_K
2
5.5 GB
Lowโ€”
Q3_K_S
3
6.9 GB
Lowโ€”
NVFP4
4
7.8 GB
Mediumโ€”
Q4_K_M
4
8.5 GB
Mediumโ€”
Q5_K_M
5
10.1 GB
Highโ€”
Q6_K
6
11.5 GB
Highโ€”
Q8_0
8
15.0 GB
Very Highโ€”
F16
16
28.7 GB
Maximumโ€”

Quality benchmarks

Phi 3 Medium 14B benchmark scores

Benchmark verified

Coding

SWE-bench Verifiedโ€”
HumanEval+58.5%
Aider Polyglotโ€”
LiveCodeBenchโ€”

Reasoning

MMLU-Pro41.2%
GPQA Diamond11.5%
MATH-50019.2%
ARC Challenge91.0%

General

Chatbot Arenaโ€”
IFEval60.4%

Source: official ยท 2024-04-22

Hardware compatibility

Fit estimates across all hardware

Open calculator

Computing compatibility...

Memory breakdown

Reference: RTX 2060 6GB

Weights8.5 GB
KV Cache3.1 GB
Runtime1.2 GB
Headroom0.6 GB

Frequently asked questions

FAQ โ€” Phi 3 Medium 14B

See also

Quantization GuideScoring MethodologyVRAM Calculator