Reimplementing A Linux Rust Scheduler In eBPF Shows Very Promising Results
NVIDIA software engineer Andrea Righi has implemented his "scx_rustland" Linux Rust scheduler within eBPF for very promising performance results.
The bottleneck to the scx_rustland Rust-written scheduler has been the overhead in communication between kernel and user-space. To address this, he's implemented scx_rustland fully within eBPF and called the new creation scx_bpfland.
The scx_bpfland scheduler employs the same logic as scx_rustland but without the kernel/user-space communication overhead. Andrea has run some benchmarks and the new bpfland code is showing very promising results. PostgreSQL is as much as 30~39% faster, FFmpeg is several percent faster, nginx is around 8% faster, and more.
Andrea concluded his blog post on scx_bpfland with:
Very interesting times with the great innovations happening within the Linux kernel and new doors being opened by both Rust and (e)BPF.
The bottleneck to the scx_rustland Rust-written scheduler has been the overhead in communication between kernel and user-space. To address this, he's implemented scx_rustland fully within eBPF and called the new creation scx_bpfland.
The scx_bpfland scheduler employs the same logic as scx_rustland but without the kernel/user-space communication overhead. Andrea has run some benchmarks and the new bpfland code is showing very promising results. PostgreSQL is as much as 30~39% faster, FFmpeg is several percent faster, nginx is around 8% faster, and more.
Andrea concluded his blog post on scx_bpfland with:
"In conclusion, prototyping new schedulers in user-space using Rust and then re-implementing them in BPF can be an effective workflow for designing new specialized schedulers.
The rapid edit/compile/test cycle provided by technologies like sched_ext is invaluable for quickly iterating on these prototypes, allowing developers to achieve meaningful results in a much shorter timeframe.
scx_bpfland is a practical example of this approach, demonstrating how initial development in a flexible Rust user-space environment can be effectively transitioned to BPF for enhanced performance.
This experiment not only highlights the powerful combination of sched_ext and eBPF in enabling efficient and adaptable scheduler development but also suggests that sched_ext could be a fundamental step toward pluggable modular scheduling in Linux."
Very interesting times with the great innovations happening within the Linux kernel and new doors being opened by both Rust and (e)BPF.