VOOZH about

URL: https://www.hardware-corner.net/gpu-llm-benchmarks/rtx-5090/

⇱ RTX 5090 Local LLM Benchmarks, Context Scaling & Supported Models 2026 – Hardware Corner

Tier 1 Enthusiast

RTX 5090 LLM Performance

Local LLM Performance: 102.7 t/s average on 14B models at 16k context. Updated Benchmarks: March 2026.

Gen (14B 4-bit) 102.7 t/s
PP (14B 4-bit) 4,473 t/s
Max Model 34B
VRAM
32 GB GDDR7
Bandwidth 1,792 GB/s
Token Gen (14B @ 4k Ctx)

102.7T/s

Prompt Proc (14B @ 4k Ctx)

4,473T/s

Summary

We view the NVIDIA RTX 5090 as a strong contender, offering significantly higher performance and more usable context than its predecessors in our tests, though it is still bound by similar VRAM constraints regarding model size. We found it particularly impressive for running the latest Qwen 3.5 models in agentic workflows, where that extra speed and context handling make a tangible difference. However, we have to note that in the current market, the price of entry is extremely high.

Key Insights

Capable of running Qwen3 32B using GGUF (Q4_K) completely in VRAM.
Handles context splitting effectively up to 32k tokens on 32B model.
Handles gpt-oss 120B model in MXFP4 quantization at full 128K context.
Supports Flash Attention 2, significantly boosting prompt processing speeds.
Supports hardware acceleration for NVFP4 quantization

Current Price in US

$3,699

Avg. Market Value

Current Pricing

Hardware Specs
VRAM 32GB GDDR7
Capable of running 34B model
Bandwidth 1,792 GB/s
Architecture Blackwell
Memory speed 28 Gbps
Memory bus 512 bit
TDP 575 W
Suggested PSU 1,000 W
Price/GB VRAM $115.59
Price/(t/s) with 14B @ 16k $36.02

Biggest LLMs You Can Run on This GPU

The models below represent the largest language models that fit fully in VRAM on this GPU using 4-bit quantization (GGUF). Benchmarks include token generation and prompt processing speeds measured at their maximum supported context length.

Qwen3.5 35B (MXFP4) Max 256k
Token Generation 97.3 t/s @ 256k context
Prompt Processing 2,003.7 t/s @ 256k context
Qwen3 32B (Q4_K) Max 32k
Token Generation 43.8 t/s @ 32k context
Prompt Processing 1,451.1 t/s @ 32k context
Gemma4 31B (Q4_K) Max 128k
Token Generation 43.4 t/s @ 128k context
Prompt Processing 900.2 t/s @ 128k context

Note: Context values are grouped into standard tiers (4K, 16K, 32K, 64K, 128K). Models may support slightly higher context, but they remain in the lower tier unless they reach the next bracket.

RTX 5090 local LLM Inference Performance vs Similar GPUs

Compare prompt ingestion and token generation speeds against similar GPUs across widely used local models and extended context lengths up to 256K.

Local LLM Benchmarks

Prompt processing (t/s) and token generation speed (t/s) across different open weight models and context lengths.

Prompt Processing
Model 4k Ctx 16k Ctx 32k Ctx 64k Ctx 128k Ctx 256k Ctx
Qwen3 8B (Q4_K)
llama.cpp build 8189
 11,933.4 8,538.4 6,034.2 3,089.6 1,209.1 —
Qwen3 14B (Q4_K) 6,497.6 4,473.1 2,908.4 1,707.2 908.4 —
gpt-oss 20B (Q4_K) 9,443.8 7,168.1 5,183.1 3,019.7 1,636.4 —
Gemma4 26B (Q4_K) 8,799.2 7,733.2 6,292.6 4,360.6 2,839.1 1,707.2
Qwen3.5 27B (Q4_K) 3,004.1 2,721.5 2,341.8 1,606.4 1,019.8 —
Qwen3 30B A3B (Q4_K)
llama.cpp build 8189
 7,093.0 5,509.8 4,210.0 2,379.7 985.0 —
Gemma4 31B (Q4_K) 3,395.0 2,794.3 2,229.1 1,459.3 900.2 —
Qwen3 32B (Q4_K) 2,931.3 2,077.2 1,451.1 — — —
Qwen3.5 35B (MXFP4) 6,605.2 6,142.3 5,611.4 4,624.5 3,242.6 2,003.7
Token Generation
Model 4k Ctx 16k Ctx 32k Ctx 64k Ctx 128k Ctx 256k Ctx
Qwen3 8B (Q4_K)
llama.cpp build 8189
 200.4 162.3 129.8 91.8 58.8 —
Qwen3 14B (Q4_K) 123.8 102.7 82.4 57.6 37.2 —
gpt-oss 20B (Q4_K) 298.2 249.2 215.1 169.0 112.0 —
Gemma4 26B (Q4_K) 180.3 167.2 159.4 149.4 130.2 106.0
Qwen3.5 27B (Q4_K) 58.8 55.8 53.8 50.1 44.1 —
Qwen3 30B A3B (Q4_K)
llama.cpp build 8189
 226.1 170.4 143.1 110.8 76.8 —
Gemma4 31B (Q4_K) 61.1 59.2 55.4 51.6 43.4 —
Qwen3 32B (Q4_K) 61.4 50.9 43.8 — — —
Qwen3.5 35B (MXFP4) 165.2 148.3 143.2 133.5 118.2 97.3

Frequently Asked Questions

Common questions about running LLMs on the RTX 5090.