If you’ve been rocking a Pascal GPU, you may have heard about Nvidia finally pulling the plug on new drivers for the 10-series cards. As someone with a perfectly fine GTX 1080, I wasn’t too thrilled about this news. Sure, it may not be able to deliver ultra-high frame rates at 4K or process ray-traced graphics, but it’s more than capable of rendering several games in my library at playable frame rates – and it can even push upwards of 1440p in most indie titles.

I’ve got an entire home lab built from old nodes, so there was no way I’d toss my faithful companion away just because it won’t receive optimizations for new games. With a little bit of tinkering and lots of troubleshooting, I configured my GTX 1080 to host LLMs via an Ollama LXC running on a Proxmox node.

Installing the GPU drivers on my PVE host was a job and a half

After all, I couldn’t use the latest drivers with the Pascal GPU

Since I’ve got a spare first-gen Ryzen node with an empty PCIe slot in my Proxmox cluster, I figured it would be the perfect host for my aged Pascal card. As an avid supporter of the Proxmox VE Helper-Scripts repo, my initial plan was to spin up an Ollama LXC using the bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/ollama.sh)" command on a Proxmox node. That’s because the Ollama script automatically detects graphics cards, configures GPU passthrough for the container, and installs the essential drivers on the underlying distro. Unfortunately, this strategy failed miserably from the get-go, as the PVE host didn’t have the right Nvidia drivers, and the script couldn’t pass my graphics card to the LXC.

So, the first order of business was to install the right Nvidia drivers on the Proxmox rig. But since Team Green removed support for Pascal cards with 590.48.01, I went with 580.119.02 instead. But as with everything Nvidia-related, the installation process was a convoluted mess. After browsing a lot of documentation, I chanced upon this forum post, and it’s what helped me install the Nvidia drivers. Here’s a brief overview of the whole process:

First, I ran nano /etc/modprobe.d/blacklist-nouveau.conf in my main node file to blacklist the Nvidia GPU. This involved pasting the following code into the file and hitting Ctrl+Shift+X to save and Y to head back to the Proxmox Shell.

blacklist nouveau
options nouveau modeset=0

Then, I ran rmmod nouveau to delete any existing nouveau drivers on my PVE node. Before downloading the old Nvidia drivers, I ran the apt install build-essential pve-headers-$(uname -r) command to grab the packages needed to execute them. After that, I ran wget https://us.download.nvidia.com/XFree86/Linux-x86_64/580.119.02/NVIDIA-Linux-x86_64-580.119.02.run to pull the drivers onto my PVE host, used chmod +x NVIDIA-Linux-x86_64-580.119.02.run to make it executable, and entered ./NVIDIA-Linux-x86_64-580.119.02.run to initiate the installation wizard.

This process went by without a hitch, and soon, the GTX 1080 drivers were configured on Proxmox. I quickly ran nvidia-smi to confirm whether they were detected by the PVE host, and Nvidia’s monitoring utility sprang up in all its CLI glory.

The Ollama LXC gave me just as much trouble

With the drivers configured, it was time to re-execute the script for the Ollama LXC. I went with the Default options, as those tend to work well for most LXCs. For the first couple of minutes, there wasn’t anything amiss with the script. It was able to connect to the Internet, deploy the LXC, and even detect the GPU. Unfortunately, things fell apart when the script tried to install the Nvidia packages on the underlying Ubuntu template. The error code mentioned the packages didn’t have an installation candidate, which was pretty weird considering the script clearly detected the container.

Then, I ran the script with different parameters, and the only time it succeeded was when I explicitly disabled GPU passthrough in the initial configuration wizard. But the very point of this project was to use my GTX 1080 to run LLMs, so the CPU-powered Ollama LXC was useless to me. Clearly, it was time to change my game plan…

I had to set up Ollama inside a custom Debian LXC

Passing the GPU was a lot simpler than I'd anticipated

Let’s look at this problem from a different angle. The LXC script was essentially an Ubuntu template armed with some extra commands to configure GPU passthrough and install the Ollama packages. So, I could theoretically create a container for Ollama by performing all these steps manually.

Once I’d created a new (unprivileged) LXC, I ran the ls -l /dev/nvidia* command in the Shell tab of my Proxmox host. That’s because I needed the device number associated with the GPU components. In my particular setup, these were 195, 235, and 237. Then, I executed the nano /etc/pve/lxc/107.conf command to open the configuration file for my LXC and added the following lines of code to mount the GPU onto the container and tweak the permissions to let Ollama call upon it when running LLMs.

lxc.cgroup2.devices.allow: c 195:* rwm
lxc.cgroup2.devices.allow: c 235:* rwm
lxc.cgroup2.devices.allow: c 237:* rwm
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file

With the tweaks on the PVE host complete, I started the LXC and ran apt update && apt upgrade to install the latest packages. Then, I ran the same set of commands to install Nvidia drivers on the LXC. The only difference was the --no-kernel-modules argument when running the executable, as the installation would fail if I didn’t add it.

wget https://us.download.nvidia.com/XFree86/Linux-x86_64/580.119.02/NVIDIA-Linux-x86_64-580.119.02.run
chmod +x NVIDIA-Linux-x86_64-580.119.02.run
./NVIDIA-Linux-x86_64-580.119.02.run --no-kernel-modules

I ran nvidia-smi inside the LXC, and the same monitoring UI as earlier popped up. Before I could grab the Ollama package, I used apt install curl zstd lshw. The first two packages are necessary for Ollama installation script, while the last one lets the app detect the GPU. Finally, I executed the curl -fsSL https://ollama.com/install.sh | sh command, and within a few minutes, Ollama was up and running – this time with the graphics card all configured to run LLMs. Just to test things out, I downloaded Deepseek R1, ran it, and used the ollama ps command to check the device running the LLM. And sure enough, my GTX 1080 was engaged in running the 7B Deepseek model.

My old GPU may not be ideal for LLMs, but it gets the job done

If you’re an LLM-hosting veteran, you’ve probably realized that my GPU is far from perfect for this project. To be honest, you’re right about that. Not only is the retired Pascal architecture too slow for high-parameter models, the 8GB VRAM also acts as a bottleneck. But you know what? The GTX 1080 is more than enough for running anything between 0.7B to 8B models – and it’s quite the game-changer for certain self-hosted services.

Home Assistant, for example, supports Ollama’s API, meaning I can use my GTX 1080 to process queries on HASS. Paperless-GPT (which works with self-hosted LLMs) can use the models running on my Ollama instance to quickly generate tags for my documents. Likewise, LanguageTool can leverage LLMs to remind me of my grammatical mistakes, while Open Notebook can utilize my GTX 1080-powered to sort through my research notes.