I was impressed with how quickly it was to get a local LLM up and running with Ollama. It was as simple as downloading the installer, pulling a model, and starting to send it queries to get its responses. But after that brief interaction to test the waters, I went back to using ChatGPT and Claude, and didn't open Ollama again for a while.

It made me realize that out of the box, Ollama doesn't have everything it needs to go toe-to-toe with the bigger AI names. What was missing was a way to integrate it properly into my workflow. Only then was I able to feel the power of having an LLM on my local system, with no limits, no fees, and no data sitting on third-party servers.

The first thing I install after Ollama

Open WebUI makes it feel like a polished product

Open WebUI provides a web frontend for Ollama. On Linux, this is a pretty much essential tool if you don't want to be squinting at your terminal to interact with Ollama. On Windows, Ollama now ships with its own graphical frontend, so it's no longer crucial to install Open WebUI there, but I still think it's a nice touch that rounds out the experience.

Using a browser to interact with Ollama feels a lot more like accessing cloud AI. Since Ollama runs as a service in the background, Open WebUI can connect to its API and hand off your requests. It provides a nice interface with chat history, search, and file uploads. The RAG (Retrieval Augmented Generation) pipeline is particularly nice, allowing the model to process and reason over file uploads. Not only does it make Ollama look like a cloud AI, it makes it behave like one as well.

Since you access Open WebUI in a browser, it means that any device on your local network can access your LLM. I'm not always on my main computer, so I like being able to use my phone to pull up the web interface and consult the local LLM. Open WebUI is easy to deploy and only requires a single command if you're using Docker.

Ditch the Copilot subscription

Continue brings local AI directly into VS Code for free

One of the handiest ways to use AI is within VS Code, where it automatically completes lines of code and answers questions about your codebase without ever leaving the editor. But I didn't want to pay for Copilot or send my code to Microsoft's servers, so I installed the Continue extension.

Continue supports all the essential features you need in VS Code, like autocomplete, inline edits, chat, and code explanation, except it lets you connect to any AI backend that you want. You can easily configure the extension to consult your Ollama background service, making it an effective replacement for what Copilot provides in the IDE.

I experimented with a few different models for this, and ultimately decided that a lighter one like Qwen2.5-Coder worked best on my machine. It's not too demanding on system resources and can still give decent results. If you find that autocomplete isn't as snappy as you'd like, you can downgrade to a leaner model. You can run a heavier model for chat and a lean one for autocomplete simultaneously in Continue. It all depends on the hardware you're running. Now that my local LLM is able to collaborate with me in VS Code, I find Ollama to be a lot more useful.

Get more done by automating tasks

n8n turns Ollama from a chatbot into something that does things

While having a chat interface and integration into VS Code is convenient, I think the power of AI really shines when we set it to automate tasks. You know, the mundane things that we don't actually want to do, but need to. This is the point where AI stops being something you talk to and starts being a thing that finishes work for you. n8n works seamlessly with Ollama, allowing you to configure local and private AI-powered workflows that don't have an API cost.

I pointed n8n at my main folder where data lands, and it parses the text to summarize its contents for me. It also renames files according to patterns that I've specified and organizes them by moving them to their proper folders. I've found that it works best for boring tasks that involve a lot of text, like isolating important parts of your meeting transcripts, for example.

You don't necessarily need to use n8n for automation, I just recommend it because it's probably one of the easiest ways to get started. I wrote a Python script that interacts with Ollama to automatically summarize saved articles overnight. Ollama exposes a straightforward API that anything can talk to, including n8n, plain Python scripts, or anything else that can make an HTTP request.

Ollama is nice, but extra tooling makes it great

With just these three tools, Ollama becomes much more powerful, and more likely to be something you'll actually use. Open WebUI gives the AI a face, Continue puts it in your editor, and n8n lets you utilize it for automation. The extra tooling is what makes Ollama feel useful, instead of a novelty.