Linux 6.18 To More Reliably Handle 255+ vCPUs On AMD EPYC Servers
For those wanting to run virtual machines with more than 255 vCPUs on modern AMD EPYC servers, an important code refactoring was merged for Linux 6.18 to ensure the proper topology information is exposed to KVM guest VMs.
A firmware bug was recently uncovered when running an AMD KVM-backed guest with QEMU and using more than 255 cores. AMD engineer K Prateek Nayak explained on the patch series:
That patch series for ensuring proper AMD CPU topology detection was merged this week as part of the x86/cpu changes.
It was also in that pull request that FreeBSD Bhyve hypervisor detection was added. That separate and unrelated non-AMD specific change is also to help out properly handling guests with 255+ vCPUs beginning with the upcoming FreeBSD 15.0 release.
A firmware bug was recently uncovered when running an AMD KVM-backed guest with QEMU and using more than 255 cores. AMD engineer K Prateek Nayak explained on the patch series:
"When running an AMD guest on QEMU with > 255 cores, the following FW_BUG was noticed with recent kernels when topoext feature wasn't explicitly enabled:
[Firmware Bug]: CPU 512: APIC ID mismatch. CPUID: 0x0000 APIC: 0x0200
QEMU provides the extended topology leaf 0xb for these guests but in an effort to keep all the topology parsing bits together during the enablement of the 0xb leaf for AMD, a pseudo dependency on X86_FETURE_TOPOEXT was created which prevents these guests from parsing the topology from the 0xb leaf.
...
The series was also tested on 255 and 512 vCPU (each vCPU is an individual core from QEMU topology being passed) EPYC-Genoa guest with and without x2apic and topoext enabled and this series solves the FW_BUG seen on guest with > 255 VCPUs. No changes observed in
"/sys/kernel/debug/x86/topo/" for all other cases without warning. 0xb leaf is provided unconditionally on these guests (with or without topoext, even with x2apic disabled on guests with <= 255 vCPU).
In all the cases initial_apicid matched the apicid in "/sys/kernel/debug/x86/topo/" after applying this series."
That patch series for ensuring proper AMD CPU topology detection was merged this week as part of the x86/cpu changes.
It was also in that pull request that FreeBSD Bhyve hypervisor detection was added. That separate and unrelated non-AMD specific change is also to help out properly handling guests with 255+ vCPUs beginning with the upcoming FreeBSD 15.0 release.