How to Easily Set Up a Neat User Interface for Your Local LLM
A step-by-step guide to run Llama3 locally with Open WebUI
1 Why Local LLMs
Whether it’s due to company restrictions or a desire to handle personal data securely, many have avoided using ChatGPT due to data privacy concerns.
Fortunately, there are solutions that allow unlimited use of LLMs without sending sensitive data to the cloud.
In my previous article, I explored one such solution by explaining how to run Llama 3 locally thanks to Ollama.
PREREQUISITE: by the end of that last article we had Llama 3 running locally thanks to Ollama and we could use it either through the terminal or within a Jupyter Notebook.
In this article I explain how to make the use of local LLMs more user-friendly through a neat UI in a matter of minutes!
So this article really is a follow-up where I explain step by step how to run a local LLM through a ChatGPT like user interface – Open WebUI. Spoiler alert: it’s surprisingly simple!
Let’s get started!
2 Brief Overview of Open WebUI
Open WebUI is an open-source interface for running AI models locally. It allows to chat with your LLM as you would with ChatGPT, through your web browser.
It is compatible with various different LLMs and has a very active community which ensures that the solution stays up to date with the latest models.
The interface is very clean and user-friendly which makes it super intuitive to use. There’s even the possibility to switch between local models and fine-tune their hyper-parameters.
Promising!
3 Installing Docker
Well, we did talk about Open WebUI, so what is Docker doing here?
Docker allows you to easily run applications in isolated "containers" on your local computer. This simplifies software installation and ensures consistent, reliable execution across different environments. In other words, it is what we need to run Open WebUI.
To download Docker, just go to docker.com and click download:
Once it’s done downloading, as with any other apps, just follow the very straightforward installation steps.
STEP 1/2:
1- Drag the docker app to your Applications folder to install it
2- Accept the Docker Service Agreement
STEP 3/4:
3- You can create a Docker account if you wish, or you can skip this part, it is not necessary in our case
4- Finally, go ahead with the recommended settings to complete the installation of Docker Desktop
At this stage, docker is now installed and should look like this (I immediately switch to dark mode for obvious reasons):
You can minimize Docker for now, as the next step is to install Open WebUI!
4 Installing Open WebUI
To install Open WebUI, go to https://docs.openwebui.com/ :
If you scroll down just a little bit, you’ll find the recommended installation section (through Docker):
What you’re after is the command that I highlighted in red here.
docker run -d -p 3000:8080 – add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data – name open-webui – restart always ghcr.io/open-webui/open-webui:main
Just running that command in the terminal will download the Open WebUI Docker image and run it:
If you now go back to Docker Desktop (which you had previously minimized), you will see this container running:
5 Running Open WebUI
In Docker Desktop, if you click on that 3000:8080 link in the "Port(s)" column, it will open the following url in your web browser:
and you should see the login page.
Open WebUI requires you to sign up before using it. Once it’s done you can just login:
And once logged in, you should end up on the main page of Open WebUI, running on localhost:3000, and it should look like this:
The next and last step is to select a model thanks to the dropdown menu on the top left. It shows the list of all the models previously downloaded through Ollama.
In my case I only downloaded llama3 8B, so that’s the one I’m going to use.
And the last thing left to do is to use it!
The response time of the model will typically depend on the GPU / RAM of your computer.
6 In Summary
To sum up, in order to run an LLM (Llama 3 for example) locally on your computer and through a neat user interface (Open WebUI) you need to:
- Install Ollama on your computer
- Download Llama 3 (or any other open source LLM)
- Install Docker on your computer
- Install and run Open WebUI locally thanks to Docker
- Run Llama 3 through the Open WebUI interface
Steps 1 and 2 are super easy and I described them in details in this article:
Steps 3, 4 and 5 were the focus of this article.
And hopefully you found it useful!
7 Final Thoughts
The intent of this article was to highlight the simplicity of running a local LLM through Open WebUI.
I just scratched the surface here, as Open WebUI can do much more than just running a model. Things like uploading files, generating images or feeding the model with your own knowledge base is very accessible through this tool.
Food for thought.
Thanks for reading all the way to the end of the article. Follow for more! Feel free to leave a message below, or reach out to me through LinkedIn / X if you have any questions / remarks!
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Write for TDS
