Intel FRED Suffers A Late "Incompatible Change" To The Architecture
Intel FRED has been seeing Linux software enablement going on for the past three years. FRED is the Flexible Return Event Delivery that overhauls CPU transitions between privilege levels with a goal of lower ring transition latency and more robust software support. Unfortunately it has a late breaking incompatible change to the architecture and the Linux kernel is now being patched for it.
Intel FRED has been a long time coming with the initial public specification in 2022 and since then there has been work across the Linux kernel and compiler toolchains for preparing for the Flexible Return Event Delivery architecture and the new CPU instructions. There's been the Linux kernel support upstreamed back in Linux 6.9 but now there is an important incompatible change needed prior to Intel CPUs shipping with FRED.
Catching my attention today is this patch queued up by an Intel engineer to remove the instruction from FRED entry points. Intel engineer Xin Li sums up the situation:
Interestingly this "incompatible change" to the FRED architecture was caught rather late in its development and seemingly raised by external researchers. Jennifer Miller and other Arizona State University researchers earlier this year around issues with Intel's FineIBT implementation and was covered in Linux's FineIBT Protections "Critically Flawed" Until Intel CPUs Appear With FRED.
This incompatible change to FRED is rather late with Intel said to be introducing this architecture with Panther Lake SoCs due out in the coming months followed by Diamond Rapids servers next year.
This patch is currently queued into a TIP branch (x86/entry) for submitting either during the ongoing Linux 6.17 kernel cycle or for the Linux 6.18 merge window since it wasn't immediately picked up by an "urgent" TIP branch but in any event the patch is marked for back-porting to the stable Linux kernel branches since Linux 6.9.
Update [18 August]: An Intel engineer reached out to provide further insight into the issue. This architectural change is incompatible in the context of the updated kernel will not work with Intel's early pre-production hardware unless either FRED or CET kernel support is disabled. But older kernels will work with what will be shipped as the production processors featuring FRED. They are also grateful to Arizona State University's timely disclosure for allowing this change to happen prior to FRED being found in production hardware.
Intel FRED has been a long time coming with the initial public specification in 2022 and since then there has been work across the Linux kernel and compiler toolchains for preparing for the Flexible Return Event Delivery architecture and the new CPU instructions. There's been the Linux kernel support upstreamed back in Linux 6.9 but now there is an important incompatible change needed prior to Intel CPUs shipping with FRED.
Catching my attention today is this patch queued up by an Intel engineer to remove the instruction from FRED entry points. Intel engineer Xin Li sums up the situation:
"The FRED specification has been changed in v9.0 to state that there is no need for FRED event handlers to begin with ENDBR64, because in the presence of supervisor indirect branch tracking, FRED event delivery does not enter the WAIT_FOR_ENDBRANCH state.
As a result, remove ENDBR64 from FRED entry points.
Then add ANNOTATE_NOENDBR to indicate that FRED entry points will never be used for indirect calls to suppress an objtool warning.
This change implies that any indirect CALL/JMP to FRED entry points causes #CP in the presence of supervisor indirect branch tracking.
Credit goes to Jennifer Miller and other contributors from Arizona State University whose research shows that placing ENDBR at entry points has negative value thus led to this change.
Note: This is obviously an incompatible change to the FRED architecture. But, it's OK because there no FRED systems out in the wild today. All production hardware and late pre-production hardware will follow the FRED v9 spec and be compatible with this approach.
[ dhansen: add note to changelog about incompatibility ]"
Interestingly this "incompatible change" to the FRED architecture was caught rather late in its development and seemingly raised by external researchers. Jennifer Miller and other Arizona State University researchers earlier this year around issues with Intel's FineIBT implementation and was covered in Linux's FineIBT Protections "Critically Flawed" Until Intel CPUs Appear With FRED.
This incompatible change to FRED is rather late with Intel said to be introducing this architecture with Panther Lake SoCs due out in the coming months followed by Diamond Rapids servers next year.
This patch is currently queued into a TIP branch (x86/entry) for submitting either during the ongoing Linux 6.17 kernel cycle or for the Linux 6.18 merge window since it wasn't immediately picked up by an "urgent" TIP branch but in any event the patch is marked for back-porting to the stable Linux kernel branches since Linux 6.9.
Update [18 August]: An Intel engineer reached out to provide further insight into the issue. This architectural change is incompatible in the context of the updated kernel will not work with Intel's early pre-production hardware unless either FRED or CET kernel support is disabled. But older kernels will work with what will be shipped as the production processors featuring FRED. They are also grateful to Arizona State University's timely disclosure for allowing this change to happen prior to FRED being found in production hardware.