The Massive AI Performance Benefit With AMX On Intel Xeon 6 "Granite Rapids"
Advanced Matrix Extensions (AMX) remains one of the exciting elements of the Intel Xeon 6 Granite Rapids processors and continues helping a lot for CPU-based AI workloads as a nice feature originally introduced back with Sapphire Rapids. AMX paired with MRDIMM memory can make for some very nice CPU-based AI inferencing performance when running the likes of OpenVINO and Llama.cpp, etc.
When running with AMX enabled for a number of the workloads there was a measurable increase to the server power consumption of ~60 Watts for this 2P air-cooled server while on average came out quite close. That typically small power increase was well worth it for the massive benefits provided to AI workloads on Granite Rapids.
For the combined dual Intel Xeon 6980P power consumption the averages were close but still periods of higher power usage when engaging AMX in OpenVINO and Llama.cpp. But still worthwhile for the performance benefit.
The power increase wasn't enough to lead to any abnormal thermal or clocking differences in this AMX comparison.
That's the fresh look at Intel AMX performance on Granite Rapids in 2025. A fresh comparison of Intel Granite Rapids against AMD Turin on the latest Linux server software is coming soon on Phoronix. Thanks again to Giga Computing for supplying the new Xeon 6900P 2U server platform for making this new Granite Rapids testing possible.
If you enjoyed this article consider joining Phoronix Premium to view this site ad-free, multi-page articles on a single page, and other benefits. PayPal or Stripe tips are also graciously accepted. Thanks for your support.