Canonical Begins Snap'ing Up Silicon-Optimized AI LLMs For Ubuntu Linux
Canonical's new push for their Snap app packaging/sandboxed format on Ubuntu Linux is for AI large language models (LLMs). Making it more interesting though is that they are working to deliver silicon-optimized AI LLMs for your hardware and to make it easily deployable for Ubuntu sers.
Canonical has begun rolling out optimized inference Snaps as a new means of deploying AI LLMs on Ubuntu-powered devices. The hope is that with the likes of sudo snap install deepseek-r1 --beta you can be easily up and running with the relevant quantized LLM modeland optimal acceleration support for your given system.
So far with the public beta announced today by Canonical, there are Intel and ARM64 Ampere optimized models just for DeepSeek R1 and Qwen 2.5 VL. The framework is open-source but so far no optimized versions for deployments with NVIDIA CUDA or AMD ROCm stacks/hardware.
It's an interesting and worthwhile concept to help users get the most optimized LLM for their system. Though there have been similar and more broad efforts like Llamafile trying to make easily redistributable LLMs that will work across different hardware and even operating systems from a single file, not confined to just Snap-enabled environments or where the Snap store is available.
It will be interesting to see what comes of this silicon-optimized LLM Snap work with how comprehensive the LLM coverage will be as well as ultimately how well it ends up working with more diverse compute environments not only for AMD ROCm and NVIDIA CUDA support but the growing presence of NPUs with Intel Core Ultra, Ryzen AI, and various other AI accelerators.
More details on today's public beta announcement via the Ubuntu blog.
Canonical has begun rolling out optimized inference Snaps as a new means of deploying AI LLMs on Ubuntu-powered devices. The hope is that with the likes of sudo snap install deepseek-r1 --beta you can be easily up and running with the relevant quantized LLM modeland optimal acceleration support for your given system.
So far with the public beta announced today by Canonical, there are Intel and ARM64 Ampere optimized models just for DeepSeek R1 and Qwen 2.5 VL. The framework is open-source but so far no optimized versions for deployments with NVIDIA CUDA or AMD ROCm stacks/hardware.
It's an interesting and worthwhile concept to help users get the most optimized LLM for their system. Though there have been similar and more broad efforts like Llamafile trying to make easily redistributable LLMs that will work across different hardware and even operating systems from a single file, not confined to just Snap-enabled environments or where the Snap store is available.
It will be interesting to see what comes of this silicon-optimized LLM Snap work with how comprehensive the LLM coverage will be as well as ultimately how well it ends up working with more diverse compute environments not only for AMD ROCm and NVIDIA CUDA support but the growing presence of NPUs with Intel Core Ultra, Ryzen AI, and various other AI accelerators.
More details on today's public beta announcement via the Ubuntu blog.