Developments in the AI sector have taken the computing landscape by storm. From Dell, Lenovo, and other big-name laptop manufacturers to software companies like Microsoft and Adobe, every tech firm is making quick strides on the artificial intelligence front to earn a slice of the AI pie.

Despite all the uproar about next-gen hardware hitting insane TOPS and performance numbers in artificial intelligence-based workloads, older hardware can still hold its own when it comes to running LLMs and image generators. So, we’ve put together a comprehensive guide to help you turn your spare PC into a local AI-hosting machine.

What you’ll need

While it’s true that you can replicate this project on most machines released in the past decade, there are a few things to keep in mind. We’ll go with Ollama’s LLMs for generating text, and you might need to meet some minimum requirements to run models with a higher parameter count.

For instance, you shouldn’t expect to run the highly demanding Llama 2 (70B) LLM on a low-end system with 8GB of memory. Likewise, your old PC should be equipped with a somewhat decent (at least on the VRAM front) GPU if you want to generate images without suffering a lag-fest.

Finally, you’ll need an operating system pre-installed on your old system. You can technically use Windows 11, but since we’re trying to maximize performance, it’s a good idea to let Microsoft’s flagship OS sit this one out. Instead, we’ll go with a Linux distribution, preferably a lightweight one. I went with Pop_OS! Due to its amazing support for Nvidia GPUs, though, I'm sure you can find more optimized distros if you want even faster speeds when generating AI content.

👁 A screenshot of the initial Ubuntu setup with the Location option highlighted
How to install Ubuntu

Want to install Ubuntu in place of Windows or another operating system? We run you through the entire process

Setting up Ollama

Assuming you’ve already installed the OS, it’s time to install and configure Ollama on your PC.

  1. Open the terminal app.
  2. Run the following command to download and install the Ollama package: 
    curl -fsSL https://ollama.com/install.sh | sh
  3. Once Ollama has finished installing, you can start the app using this command: 
    sudo systemctl start ollama
  4. Next, you can install the LLM of your choice using the following command: 
    ollama pull llm_name: parameter_choice
    I’ve used Gemma’s 7B variant for this tutorial.
  5. (Optional) If you’ve downloaded multiple models, you can switch between them using the ollama run command: 
    ollama run llm_name: parameter_choice

With that, you’re free to experiment with Ollama’s multitude of language models. But our work is far from over…

Installing a web UI

Currently, the “AI PC” can only run the LLMs locally via the terminal. If you want a more organized user interface, then I recommend installing Open WebUI, To do so, you’ll need to set up Docker alongside a handful of dependencies and packages.

  1. Fire up the terminal app and install the Snap Store with this command. 
    sudo apt install snapd -y
  2. Using the snap command, install Docker on your system. 
    sudo snap install docker
  3. Execute the docker run command with the following parameters to create a new container for Open WebUI. 
    docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name ollama-webui --restart always ghcr.io/open-webui/open-webui:main
  4. Launch a web browser and type the following address into the URL bar: 
    localhost:8080
  5. Enter an Email, Username, and Password to create an admin account. You'll need it to access Ollama's LLMs over the Open WebUI.

Another neat functionality of Open WebUI is that the tool lets you run Ollama LLMs from other devices connected to the same network as your old PC. You can set up this facility with the help of this procedure:

  1. Switch to a different system and launch the web browser of your choice.
  2. Type the IP address of your spare PC followed by a colon (:) and port number 8080 into the Search Bar. You can use a network scanner application like Fing or Nmap to find out the IP address of the Ollama host machine.
  3. Enter the same credentials as earlier and click on the Sign in button.

By now, you might have noticed that Open WebUI bears a lot of resemblance to ChatGPT’s user interface. So, if you’re a long-time user of OpenAI’s primary chatbot, then you’ll feel right at home with this setup.

Setting up Stable Diffusion

If you have an ancient PC, it’s a good idea to stop at this point, as Stable Diffusion has significantly higher specifications to generate images. I’ve tested everything on a self-built PC with a Ryzen 5 1600, GTX 1080, and 16GB (single channel) of DDR4 memory. So, if your hardware can beat my dinosaur machine, you shouldn’t encounter too many issues when using Stable Diffusion.

  1. Run the following command inside the terminal to install the necessary dependencies for Automatic11’s Stable Diffusion WebUI package. 
    sudo apt install wget git python3 python3-venv libgl1 libglib2.0-0
  2. Use the wget command to grab the Stable Diffusion Web UI repository. 
    wget -q https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh
  3. Modify the permissions of the webui.sh file with this command: 
    chmod +x webui.sh
  4. Run the webui script with the api argument. 
    ./webui.sh --api

Next, you’ll need to link the local instance of Stable Diffusion to the web UI we’re using for Ollama

  1. Switch to the Open WebUI, click on your Username, and choose Settings.
  2. Navigate to the Images tab and set Image generation to On before entering http://127.0.0.1:7860 as the AUTOMATIC1111 Base URL. In case the Open WebUI shows an error when connecting to Stable Diffusion, re-execute the webui.sh file with the --listen argument. Similarly, if you keep getting VRAM-related errors, use the --lowvram argument to lower the VRAM consumed in exchange for a drop in performance.
  3. Click on the Save button.

Finally, our project requires a prompt generator for Stable Diffusion, which you install on Ollama and configure using the Open WebUI page.

  1. Switch to the terminal one last time and run this command to download the prompt generator: 
    ollama run brxce/stable-diffusion-prompt-generator
  2. Head to the Open WebUI page on your browser and pick the prompt generator you just downloaded using the Select a model button.
  3. Once you've entered a prompt, click on the Generate Image option to create an AI render with the help of Stable Diffusion.
  4. Alternatively, you can tap the Edit button to modify the prompt.

Harnessing the power of AI on old hardware

If you followed all the steps correctly, you should be able to generate images and text on your self-hosted AI server. Sure, the performance can be quite lackluster, especially when you try to generate images using an outdated PC. Speaking of performance, I recommend checking out this page about some performance-related arguments that you can use to optimize the webui.sh script.

That said, an old PC isn’t the only platform that can generate text using AI. If you have plenty of patience, you can even attempt to run LLMs and SLMs on an SBC, like the Raspberry Pi.