If you've got a home lab and a smart home, you probably already have Home Assistant set up. You might even have it paired with a local Voice Assistant, so you can control your smart home devices without lifting a finger. Maybe you've got plenty of automation going on as well, but wouldn't it be cool if you could talk to the voice assistant conversationally?

You could add smart speakers from the big names in voice control, but those don't give you control over your data. Instead, you can run your own AI voice assistant at home that doesn't need to phone home to another server, tie that into Home Assistant, and create a voice assistant with personality customized for your smart home setup.

Now, I've got my Ollama AI running on my NAS, which isn't the best situation because I don't have a GPU to pass through for faster computation. But this was a proof of concept more than anything else, and now that I like it, it's going on a bare metal Proxmox server with an Nvidia GPU so that I can use CUDA for effective AI reasoning. At the very least, it's a good show of what the future of agentic AI could do, if companies stop chasing text-based chatbots and throw money at the type of computer-human interfaces any Star Trek fan has been waiting for all this time.

You'll want the full HAOS install for this one

It's faster and more convenient than using the Docker container

Okay, so the main reason you'd want to link Ollama, which runs local AI models against your queries, and Whisper, which is a speech-to-text model for transcribing the spoken word, is that you want an AI to talk to. Well, more than just talk to, because you can use it to control Home Assistant, and every part of your smart home that's linked to our favorite open-source project. You also need one more part, Piper, which is a text-to-speech model that completes the conversation between your mouth and the AI's silicon brain.

It's been surprisingly simple to set up, given that this is the first time I've used an AI chatbot outside of the web-based version of ChatGPT. Except for one thing. I originally had my Home Assistant installed as a Docker container on my NAS, and that's hard mode for adding services. What you want instead is either Home Assistant OS installed as the operating system on a Mini PC, or as a virtual machine.

That's because the add-on store for HAOS isn't available on the Docker version, and it makes it so much easier to get everything going. Otherwise, you need to set up Whisper, and Piper, and Ollama in their own Docker containers, and then link those into Home Assistant, when you could use the add on store and have them as parts of the OS.

Having a local AI model is pretty handy

But being able to talk to it is even better

The longest part of this process was waiting for the Llama 3 model to download. Integrating it into HAOS was simple, so was getting Whisper working. But it is kind of slow to respond, and it won't replace Alexa or Google Assistant, at least not yet. But it's not that far away, and considering that these models used to take a considerable amount of server hardware to run, it's pretty impressive they work at all when on consumer devices.

I set up Open UI to use the Ollama instance via text for testing, and while it takes a while and makes my NAS fans go brr in a way I've not heard any other task accomplish, it still gives me a little chill knowing that the reasoning is being done on a device in my home, that I own, and that I set up. Sure, someone else did the hard work of training the AI and getting the plugins to work with HAOS, but it's still an accomplishment, like changing a tire on your car.

And the neat thing is that you can change how the AI responds to you, by giving it text commands. The text-to-speech add-in isn't smart, it reads what it gets verbatim. But by telling Ollama that's the case, the AI will give better text to get converted. Things like the time, where 14:22 would get read out as one-four-colon-two-two, by telling the AI to adjust, it will change the output to "two twenty two pm" and give you a better response. That's a fascinating look into how conversational instructions can essentially program AI.

My smart home is almost smarter than I am

While not being beholden to Google or Amazon for voice assistance, having my own local Ollama-powered voice assistant is also great fun. You don't have to suffer from the stiff upper lip of corporate AI models. You can train your particular AI to be rude or sarcastic or have it limit the amount of useless information it returns when you ask it to turn the lights off. That makes it truly yours, and the frustration of figuring out how to run the model and connect everything has been worth it.