Intel Linux Optimizations Help AMD EPYC "Genoa" Improve Scaling To 384 Threads
First up was the Memcached testing given that it was one of the focuses of Intel engineers when uncovering a bottleneck within the Linux kernel networking code. Like we saw with Xeon Scalable Sapphire Rapids, the AMD EPYC Genoa performance also benefited significantly as the core counts increased from the kernel optimizations carried on Intel's Clear Linux in relation to CentOS Stream and Ubuntu Linux. With booting the AMD server with just three cores/threads enabled, Ubuntu and Clear Linux exhibited very similar but by the time of running the EPYC 9654 2P server in the full 192 cores / 384 threads configuration, the three Linux distributions were exhibiting distinctly different results. CentOS Stream was closer to Clear Linux at 384 threads than Ubuntu Linux, which saw the worst scaling of the three operating systems with this benchmark. Clear Linux was scaling the best for Memcached with the 1:10 set:get test but even it dropped off after 96 cores/threads when embarking on the 2P layout. At 384 threads for this Memcached benchmark, Intel's optimized Linux platform was 55% faster than Ubuntu 23.04 out-of-the-box and 6% faster than CentOS Stream.
When pushing Memcached further with a 1:100 set to get ration, Intel's Linux distribution continued to show much greater scaling than the lousy showing out of Ubuntu and CentOS Stream.
With a more modest Memcached 1:5 benchmark on this 4th Gen EPYC server, Clear Linux continued to prove advantageous overall but by much slimmer margins than with the other configurations.
One of the items to keep in mind with Ubuntu's rather poor showing is that by default even Ubuntu Server runs on AMD EPYC (and Intel Xeon Scalable) hardware with the "schedutil" scheduler utilization governor while Clear Linux and CentOS Stream both default to using the performance governor with ACPI CPUFreq. Ubuntu's default schedutil (or prior ondemand/powersave) governor default is a bit silly particularly for large servers such as this EPYC 9654 2P configuration. Hopefully Ubuntu will better tune its kernel defaults for Ubuntu Server in the future or at least integrate some sort of option/profile into the installer.