ZLUDA, a CUDA for Intel GPUs, Needs a New Maintainer

Work on an open source project that would help programs written for Nvidia’s proprietary CUDA platform to run on Intel GPUs has come to halt, as the project’s previous maintainer looks for someone else to lead the effort.

Back in 2020, Andrezej Janik, a software engineering manager for Intel working in Gdańsk, Poland, began work on ZLUDA project, which was to serve as a drop-in replacement for CUDA (Compute Unified Device Architecture) on Intel GPU technology.

ZLUDA (written in C++ and Rust) was going to be a big deal, as it provided an open source alternative to Nvidia’s proprietary CUDA. Even then ZLUDA provided a proof of concept to run unaltered CUDA-based applications on non-Nvidia hardware.

As the project’s GitHub page explains:

ZLUDA is a drop-in replacement for CUDA on Intel GPU. ZLUDA allows to run unmodified CUDA applications using Intel GPUs with near-native performance. It works with current integrated Intel UHD GPUs and will work with future Intel Xe GPUs

ZLUDA was built upon Intel‘s oneAPI Level Zero specs, and even in its early days, it looked incredibly promising. And given how it would make it possible to run CUDA software on Intel hardware, the promise was significant.

Soon after the ZLUDA buzz hit though, Janik updated the README .md file with this statement:

Due to private reasons, I am currently unable to continue developing this project. If you want to take it over, fork it and contact me at vosen@vosen.pl

At least according to one employee at Intel, who spoke anonymously, Intel itself may have requested development be halted, due to worries around licensure. At the very least, Janik’s work on CUDA probably represents a conflict of interest for Intel (even if the project is not an official Intel one).

Certainly, since the launch of CUDA back in 2006, Nvidia has kept the software proprietary. Obviously, if Nvidia wanted to allow other businesses to implement CUDA, it would have opened the source. But it has not.

What Is CUDA?

For those that don’t know, the CUDA library allows software developers to use a CUDA-enabled GPU for general purpose processing, also known as GPGPU (General-Purpose Computing on Graphics Processing Units). The layer that CUDA provides gives direct access to a Nvidia GPU’s virtual instruction set and parallel computational elements, which then makes it possible to execute compute kernels and is what most high-performance AI/ML frameworks use to access GPUs.

Simply put, CUDA makes GPU resources available for parallel programming, which means developers can speed up compute-intensive applications by making use of GPU resources for any parallelizable part of a computation. That’s huge, especially in cloud native development where scalability can require as much compute power as possible.

Where Is the Open Source Version?

Back in 2009, Apple and the Khronos Group launched OpenCL, which was an attempt to provide a standard for heterogeneous computing not limited to the combination of Intel/AMD CPUs and Nvidia GPUs. In other words, it is a version of CUDA that could be implemented on any hardware.

Unfortunately, OpenCL could never reach the performance levels of CUDA. Part of the problem is that with OpenLS only 1 GPU can be utilized, whereas with CUDA you get the power of 2 GPUs. That’s a significant difference (and perhaps why OpenCL has yet to gain much traction).

Why Is CUDA Important to the Cloud?

As stated earlier, with CUDA you gain added compute power with the addition of the GPU to your CPU. With cloud native applications, you need every bit of power you can get. This is especially true when you’re deploying containers that will be hit with massive demand. In fact, you can see up to 50x performance improvements when using CUDA and multiple P100 server GPUs (over standard CPUs).

Back in 2020, Folding@Home gained CUDA support. With that addition, the project saw a 15-30% performance boosts with most GPU-based projects and even larger gains for specific workloads. Case in point, using CUDA Core22 on COVID Moonshot projects, the project saw performance gains of 50-100% with CUDA. Using an Nvidia P106-100 GPU, Folding@Home discovered a 400% increase in performance in COVID Moonshot sprints.

Given the importance of this particular work, every performance gain that can be had is critical. And with CUDA offering that level of gain, it’s easy to see why this library has become so crucial.

Onward

Janik’s project received serious press from both HackerNews and Phoronix, which quickly spread through the AI/ML/HPC communities. ZLUDA was going to be a game-changer.

Without Janik, does that mean the project is dead in the water? No. As Janik said, if someone wants to take it over as a fork, he’s willing to hand the project over.

I reached out to Janik for comment but never heard back. Nor has Nvidia responded to a last-minute inquiry from The New Stack.

However, ZLUDA is an important project, one that could have significant implications for cloud-native development. So if you (or someone you know) are up to forking the project, reach out to Janik as soon as possible.