The Massive AI Performance Benefit With AMX On Intel Xeon 6 "Granite Rapids"

Written by Michael Larabel in Processors on 24 September 2025 at 10:20 AM EDT. Page 5 of 6. 15 Comments.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: Llama-3.1-Tulu-3-8B-Q8_0, Test: Text Generation 128. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: Llama-3.1-Tulu-3-8B-Q8_0, Test: Prompt Processing 512. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: Llama-3.1-Tulu-3-8B-Q8_0, Test: Prompt Processing 1024. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: Llama-3.1-Tulu-3-8B-Q8_0, Test: Prompt Processing 2048. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: Mistral-7B-Instruct-v0.3-Q8_0, Test: Text Generation 128. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: Mistral-7B-Instruct-v0.3-Q8_0, Test: Prompt Processing 512. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: Mistral-7B-Instruct-v0.3-Q8_0, Test: Prompt Processing 1024. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: Mistral-7B-Instruct-v0.3-Q8_0, Test: Prompt Processing 2048. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: granite-3.0-3b-a800m-instruct-Q8_0, Test: Text Generation 128. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: DeepSeek-R1-Distill-Llama-8B-Q8_0, Test: Prompt Processing 512. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: DeepSeek-R1-Distill-Llama-8B-Q8_0, Test: Prompt Processing 1024. AMX Enabled - Default was the fastest.

👁 Llama.cpp benchmark with settings of Backend: CPU BLAS, Model: DeepSeek-R1-Distill-Llama-8B-Q8_0, Test: Prompt Processing 2048. AMX Enabled - Default was the fastest.

From Llama to Granite to DeepSeek R1 LLMs, Intel AMX with Granite Rapids allowed for much better performance than what would otherwise be possible. For those curious how the fresh Intel Xeon 6980P benchmark numbers stack up against the AMD EPYC 9005 "Turin" series, I will be working on a fresh one-year comparison there in the coming weeks.

15 Comments - Next Page

Page: 1 2 3 4 5 6 Next Page

URL: https://www.phoronix.com/review/intel-xeon-6-granite-rapids-amx/5

⇱ The Massive AI Performance Benefit With AMX On Intel Xeon 6 "Granite Rapids" - Phoronix

The Massive AI Performance Benefit With AMX On Intel Xeon 6 "Granite Rapids"