Linux's New "Sheaves" Per-CPU Caching Layer Showing Massive Wins For AMD Performance
Earlier this week I wrote about Sheaves as an opt-in, per-CPU array-based caching layer likely coming for Linux 6.18. The sheaves patches have been queued into the "slab/for-next" Git branch ahead of the Linux 6.18 kernel merge window. Patches posted now by Google are showing the Linux Sheaves code having a massive beneficial impact for large AMD systems.
Google engineer Sudarsan Mahendran posted some benchmarks of the SLUB per-CPU Sheaves patches on Friday. The patches were applied to a Linux 6.17 base and tested across AMD, Intel, and ARM servers. For the AMD EPYC Turin server the Sheaves work ended up being a massive win for performance on a number of benchmarks but also some regressions.
Sudarsan Mahendran commented on the mailing list:
Going over his benchmarks on that LKML thread were exciting when seeing "+28.58%" mean improvements to get started, but also some 13~20% regressions... But when getting to the higher process counts for these scalability benchmarks was when it was getting really wild with +70.59%, +126.89%, +112.89%, and other massive wins. See all of the Google engineer's data in this thread.
It will be exciting to see how the Sheaves patches play out in more real-world workloads. Once these patches hit the mainline kernel presumably for Linux 6.18, I'll be firing up a number of benchmarks on my own hardware and thankfully have a lot of AMD EPYC Turin hardware and more for some exciting benchmarks ahead.
Google engineer Sudarsan Mahendran posted some benchmarks of the SLUB per-CPU Sheaves patches on Friday. The patches were applied to a Linux 6.17 base and tested across AMD, Intel, and ARM servers. For the AMD EPYC Turin server the Sheaves work ended up being a massive win for performance on a number of benchmarks but also some regressions.
Sudarsan Mahendran commented on the mailing list:
"I ported this patch series on top of v6.17 and ran some benchmarks: will-it-scale, hackbench, redis, unixbench and kernbench. I ran the benchmarks on Intel Granite Rapids (480 cores), AMD Turin (512 cores) and ARM (80 cores)
Summary of the results:
- Significant change (meaning >10% difference between base and experiment) on will-it-scale tests in AMD.
- No significant change on other benchmarks ran."
Going over his benchmarks on that LKML thread were exciting when seeing "+28.58%" mean improvements to get started, but also some 13~20% regressions... But when getting to the higher process counts for these scalability benchmarks was when it was getting really wild with +70.59%, +126.89%, +112.89%, and other massive wins. See all of the Google engineer's data in this thread.
It will be exciting to see how the Sheaves patches play out in more real-world workloads. Once these patches hit the mainline kernel presumably for Linux 6.18, I'll be firing up a number of benchmarks on my own hardware and thankfully have a lot of AMD EPYC Turin hardware and more for some exciting benchmarks ahead.