CodeLlama 13B Instruct

Legacy

2.3KDownloads160LikesAug 2023Released16K tokensContextCommunityLicense60 GoodQuality

CodeLlama 13B Instruct (13B parameters) requires approximately 21.9 GB of VRAM with Q4_K_M quantization. For the best balance of quality and speed, we recommend hardware with at least 26 GB of VRAM.

Get started

— copy & paste to run locally

Copy-paste commands to run CodeLlama 13B Instruct on your machine.

Run

lms load CodeLlama-13b-Instruct-hf && lms server start

Quick specs

Parameters13B

Architecturedense

Context16K tokens

Modalitycode

Min RAM5.1 GB

Rec. RAM7.9 GB (Q4_K_M)

LicenseCommunity

FamilyCodeLlama

✓ Code

About this model

Code Llama is a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 34 billion parameters. This is the repository for the 13 instruct-tuned version in the Hugging Face Transformers format. This model is designed for general code synthesis and understanding. Links to other models can be found in the index at the bottom.

•[x] Code completion
•[x] Instructions / chat
•[ ] Python specialist

Related models

Your hardware

Detecting...

Quick picks

Best budgetA

Mac mini M4 64GB~$1,099 — 10 tok/s

👁 NVIDIA

Best overallA

RTX 5090 32GB~$1,999 — 151 tok/s

Best hardware

Top picks for CodeLlama 13B Instruct

👁 NVIDIA

RTX 5090 32GBA

32 GB

👁 NVIDIA

RTX PRO 4500 Blackwell 32GBA

32 GB

AMD Instinct MI100 32GBA

32 GB

👁 NVIDIA

NVIDIA V100 32GBA

32 GB

AMD Instinct MI60 32GBA

32 GB

Run this model

CodeLlama 13B Instruct on RTX 5090 32GB CodeLlama 13B Instruct on RTX PRO 4500 Blackwell 32GB CodeLlama 13B Instruct on AMD Instinct MI100 32GB

Quantization options

VRAM estimates by quant level

No hardware detected — fit column shows raw VRAM estimates

Quant	Bits	VRAM	Quality	Fit
Q2_K	2	5.1 GB	Low	—
Q3_K_S	3	6.4 GB	Low	—
NVFP4	4	7.3 GB	Medium	—
Q4_K_M	4	7.9 GB	Medium	—
Q5_K_M	5	9.4 GB	High	—
Q6_K	6	10.7 GB	High	—
Q8_0	8	13.9 GB	Very High	—
F16	16	26.7 GB	Maximum	—

Quality benchmarks