Chainguard Launches CPU/GPU Containers for AI Frameworks

The concept is simple. You take a container, usually Docker, with your application and code inside. It likely has some CVEs, if not many. Alternatively, you can build a container using a Chainguard container image, and usually, you can reduce those vulnerabilities to zero. After that, you can port your application, deploy it, share it, etc.

What Chainguard has recently done is apply this process to containers for AI models and large language models (LLMs).  How it achieved this is interesting, but the key point is that, according to Chainguard’s documentation, it works. And as I can confirm and will detail below, I was able to download the image with Docker pull to get started, and it worked seamlessly.

Additionally, how this process works, and information about Chainguard in general, is detailed and covered in its learning center, Chainguard Academy. There, you can get hands-on help, and the team will actually work with you once you’ve completed the course, especially when you pull your own container images from GitHub, for example.

The idea is to address the challenges of securely building ML and GenAI applications. These challenges involve entirely new types of stacks that must be configured to handle the additional complexities of both GPUs and CPUs. This includes neural networking, computing, applications and other computational challenges associated with ML and AI/ML stacks.

Many organizations are just getting started, and security may or may not be an afterthought, given the immense challenges involved in the production cycle, which now requires a new type of workflow. To address security concerns, Chainguard offers a growing suite of CPU and GPU-enabled container images tailored for AI. These images include PyTorch, Conda and Kafka, all of which are hardened and minimal. The goal is to use these containers seamlessly, without hindering the development of applications despite the significant challenges involved.

“While much of the AI security conversation revolves around model security and prompt injections, we can’t overlook foundational aspects of the infrastructure hosting these models and applications,” said Dan Fernandez, staff product manager at Chainguard, told The New Stack. “The entire AI Stack needs robust security. Chainguard Images help organizations tackle vulnerabilities in these often-overlooked components, strengthening the foundation of AI deployments.”

The vulnerabilities in these hardened containers are addressed in real time, with the status of these vulnerabilities being checked daily. For example, the runtime release of the official PyTorch image on Docker Hub contained one critical, 23 high, 1,189 medium and 72 low CVEs according to the Grype vulnerability scanner as run on July 24. However, these vulnerabilities are mitigated through the use of Chainguard’s hardened images.

Like all Chainguard images, its Chainguard AI Image for PyTorch has zero CVEs “as of today,” and any new CVEs will be rapidly patched, frequently within hours and not days, Chainguard says. ,

“The AI landscape is evolving incredibly fast with new application components emerging almost every day. In this fast-paced environment, using the latest, most secure versions of components is crucial,” Fernandez said. “Machine learning operations teams want to focus on developing better functionality, and they prefer that over the toil of vulnerability remediation, which slows down innovation.”

In addition to speaking with The New Stack,  Fernandez wrote in a blog post that by simplifying and securing the AI/ML stack, Chainguard allows businesses to harness the power of these technologies without the added burden of managing intricate infrastructure.

‍“As we champion the acceleration of AI technologies, we equally emphasize the critical need for cutting-edge security solutions to safeguard these advancements,” said Jacob Rideout, CTO of HiddenLayer, a computer and network security provider, in the blog post. “We have adopted Chainguard Images to reduce the burden of vulnerability triage on our developers so they can focus on building and establishing a secure avenue for the broad adoption of AI technologies.

‍In addition to patching the images on an ongoing basis, Chainguard communicated the following attributes for its hardened containers:

• Chainguard seeks to keep the “bloat” out of Chainguard Images, including, of course, its images for large AI frameworks.
• ‍Development and runtime versions of images are provided. Development images provide tools for training or development, while the use of production images are suggested for tasks such as inference.
• ‍Chainguard Images are smaller on disk than comparable official images, according to Chainguard. This is important for ultra-large frameworks and libraries that tend to break infrastructure such as automated container scanners.
• ‍Every Chainguard Image comes with an included software bill of materials (SBOM) itemizing all included software artifacts. Chainguard Images are also reproducible from their attestations (signed build configurations).

Starting Block

Getting started with Chainguard was simple. ‍‍

The versions of Chainguard’s PyTorch- and other GPU-enabled images are available to pull in the Chainguard Images Directory. It only required a “docker pull” command to get things going. The subsequent steps, as detailed in Chainguard’s documentation, are straightforward:

👁 Image

Chainguard’s documentation is solid for the rest of the steps required for pulling Chainguard’s PyTorch- and other GPU-enabled images. For running pytorch-cuda12, the documentation describes how PyTorch has prerequisites that need to be configured in the environment prior to running with GPUs (examples are provided in TESTING.md).

The container is launched this way:

👁 Image

Do and Learn

👁 Image

Another interesting aspect of Chainguard’s ML/AI hardened container images is its Chainguard Academy course, which gamifies the learning process. It teaches users not only how to use its newly introduced AI/ML hardened container images but also containers in general. Feel free to test your knowledge. It’s fun to get started, as I can confirm.

For the AI/ML course, Chainguard offers a minimal low-CVE image for what it calls “deep learning” with PyTorch. It includes support for the CUDA parallel computing platform for performing computation on supported GPUs.

This introductory guide to Chainguard’s Pytorch image will walk the learner through fine-tuning an image classification model, saving the model and running it securely for inference, Chainguard says. The security and footprint of the PyTorch Chainguard Image to the official runtime image distributed by PyTorch are compared and ways to adapt the resources in this tutorial are covered for PyTorch.

Stay tuned for a full tutorial and review of Chainguard Academy.