Latest Open-Source AMD Improvements Allowing For Better Llama.cpp AI Performance Against Windows 11

Written by Michael Larabel in Software on 17 September 2025 at 10:48 AM EDT. Page 5 of 5. 21 Comments.

When taking a geo mean of all the Llama.cpp benchmarks under Windows and Linux with both the CPU and GPU data points, this is the outcome:

👁 Geometric Mean Of All Test Results benchmark with settings of Result Composite, Llama.cpp Windows vs. Linux Performance. Linux 6.17 + Mesa 25.3-dev was the fastest.

Due in large part to slower Vulkan performance on Ubuntu 24.04.3 LTS, it took moving to Linux 6.17 and Mesa 25.3-devel for this AMD Ryzen 9 9950X3D + Radeon RX 9070 XT system to edge past Windows 11 for Llama.cpp. Moving to Linux 6.17 was also of benefit in some of the CPU-based benchmarks with the different LLMs. A very interesting first rodeo between Windows and Linux for Llama.cpp performance.

👁 ROCm vs. Vulkan Llama.cpp RDNA4 Radeon RX 9070 XT

Such strong Vulkan API performance with Linux 6.17 + Mesa 25.3-devel then made me curious... How does this compare to Llama.cpp with the AMD ROCm back-end? So from the same system while running Ubuntu 24.04.3 LTS + Linux 6.17, I ran some Vulkan vs. ROCm 6.4.3 benchmarks with Llama.cpp on the Radeon RX 9070 XT. (NOTE: This testing wrapped up last week, prior to yesterday's ROCm 7.0 release. ROCm 7.0 benchmarks will come later.) Curious about the practicality of Vulkan versus ROCm for AI inferencing:

👁 Llama.cpp benchmark with settings of Backend Comparison (Model: Llama-3.1-Tulu-3-8B-Q8_0, Test: Text Generation 128). Vulkan was the fastest.

👁 Llama.cpp benchmark with settings of Backend Comparison (Model: Qwen3-8B-Q8_0, Test: Text Generation 128). Vulkan was the fastest.

👁 Llama.cpp benchmark with settings of Backend Comparison (Model: Qwen3-8B-Q8_0, Test: Prompt Processing 512). Vulkan was the fastest.

👁 Llama.cpp benchmark with settings of Backend Comparison (Model: Qwen3-8B-Q8_0, Test: Prompt Processing 1024). Vulkan was the fastest.

👁 Llama.cpp benchmark with settings of Backend Comparison (Model: gpt-oss-20b-Q8_0, Test: Text Generation 128). Vulkan was the fastest.

👁 Llama.cpp benchmark with settings of Backend Comparison (Model: gpt-oss-20b-Q8_0, Test: Prompt Processing 512). Vulkan was the fastest.

👁 Llama.cpp benchmark with settings of Backend Comparison (Model: gpt-oss-20b-Q8_0, Test: Prompt Processing 1024). Vulkan was the fastest.

👁 Llama.cpp benchmark with settings of Backend Comparison (Model: gpt-oss-20b-Q8_0, Test: Prompt Processing 2048). Vulkan was the fastest.

👁 Llama.cpp benchmark with settings of Backend Comparison (Model: Mistral-7B-Instruct-v0.3-Q8_0, Test: Text Generation 128). Vulkan was the fastest.

To some surprise, the Vulkan performance using Mesa 25.3-dev RADV + Linux 6.17 was outperforming the current ROCm 6.4.3 release with Llama.cpp.

RESULTS

In some cases the Radeon RX 9070 XT with Vulkan was performing significantly faster than with ROCm.

END OF RESULTS

Llama.cpp with RADV Vulkan on Linux 6.17 + Mesa 25.3-dev outpaced using the Llama.cpp ROCm back-end in all of these benchmarks on the Radeon RX 9070 XT. This area will be explored more in future Phoronix articles with additional hardware.

That's the end of the data for today. Those appreciating my relentless Linux benchmarking can show their support via joining Phoronix Premium to help making such continued testing possible.

21 Comments

If you enjoyed this article consider joining Phoronix Premium to view this site ad-free, multi-page articles on a single page, and other benefits. PayPal or Stripe tips are also graciously accepted. Thanks for your support.

Page: 1 2 3 4 5

URL: https://www.phoronix.com/review/llama-cpp-windows-linux/5

⇱ Latest Open-Source AMD Improvements Allowing For Better Llama.cpp AI Performance Against Windows 11 - Phoronix

Latest Open-Source AMD Improvements Allowing For Better Llama.cpp AI Performance Against Windows 11