Smart homes appear perfect in video demos, but they can be laborious in real life. While I made the best of Home Assistant to set up my smart home, I struggled to make it work for my family. Not everyone was a fan of waiting for the lights or appliances to turn on. Since that made them physically turn smart things on and off, it broke several automations.

I wanted a better presence sensor that knows when a person is in the room, not one that detects movement from something that moved briefly, like the curtains. With that, I wanted my smart home to stop appearing dumb. After briefly trying a PIR motion sensor on an ESP32, I realized I was using the wrong sensor for the right task. To achieve a better solution, I created a multimodal presence detection system for my smart home.

Why does the usual presence detection fail

Based on a lot of assumptions​

Using a PIR sensor (HC-SR501) with an ESP32 only helped me detect motion when my family moves around. That assumption recorded "ghost" motions of something moving and caused several automations to trigger unexpectedly, especially when guests arrive and leave, leaving everyone confused. It dawned upon me that a human presence sensor is suitable rather than a simple motion sensor.

So, I set up a Home Assistant Companion app on their phone for presence detection. That didn’t work either, because they’d forgotten to charge their phones and left them in a drawer. Also, they don’t roam around carrying their phones from room to room either.

Turns out, phone-based presence detection was a hit-or-miss at my home. I realized that my home needed a human presence detection sensor, and I picked a cheap one to experiment with.

My colleague Adam Conway’s success with an ESP32-powered presence sensor using Bluetooth inspired me to try it out. In theory, it was perfect since even smartwatches carry Bluetooth. While researching the Bermuda project, I discovered ESPectre, which uses Wi-Fi signal information to detect motion and presence. ​

And that’s how I created a fusion with both projects by using two ESP32 boards and a mmWave sensor to create a presence sensor. The idea is to detect if someone is present, even if they're just breathing.

Picking the correct ESP32 board for the project

Capable of running Wi-Fi and Bluetooth together

To run Wi-Fi and Bluetooth radios together, and a capable ESP32-S3 or ESP32-C6 board can handle that. I chose two ESP32-C6 boards because they support Wi-Fi 6 and Bluetooth 5.3. First was an ESP32-C6 board with an IPEX connector that lets me use a pigtail connector and hook an RM-SMA-based antenna that I harvested from an old router. While this device also supports Zigbee and Thread, only Thread is usablewith Wi-Fi.

I had planned to add a temperature and humidity sensor for thermal signature recording. Owing to the complexity of correlating "thermal sensing" with presence, I chose to shelve that plan for later.

Using Wi-Fi and Bluetooth with a motion sensor on ESP32

One detects a presence, and the other estimates the location

When used individually, both Wi-Fi and Bluetooth exhibit their flaws. But when I combined them, both worked together as a better system.

Bermuda enables device proximity in a room without using any trackers or apps. Using an ESP32, it works as a Bluetooth proxy, picks up Bluetooth Low Energy device chatter, and passes that data to Home Assistant for processing and presenting where a person is likely to be.

When data from multiple Bluetooth proxies reaches Home Assistant, the signal data is used to estimate the device's distance and area. So it constantly listens to BLE data from your phones, watches, earbuds, and other devices.

The ESPectre project emits sci-fi vibes. It uses Wi-Fi signals to detect motion, including even micro-movements, to estimate your presence. And you don’t need to carry any device for that. All the math is used to process the radio waves. It works even in the dark and even with minor obstructions like furniture. Finally, Home Assistant only stores device state values and triggers automations, but doesn’t store any Wi-Fi signal data.

The challenging part was to create a reliable ESP32-based multimodal sensor. Since ESPectre recommends using specific YAML for better results, I flashed the ESP32-C6 YAML on a tiny board that supports an antenna with a tiny IPEX connector. On the second board, I used an mmWave sensor and BLE tracking after installing Bermuda integration through HACS.

A smarter upgrade, but not exactly plug-and-play

This multimodal presence-detection sensor isn’t a straightforward implementation. The mmWave presence sensor can detect motion and presence more effectively, and I recommend it. Putting together the sensor and software projects onto the ESP32 using ESPHome took me less than an hour. However, tweaking the code to avoid false positives took a couple of days to get it right.

Gradually, the automation, based on presence detection, becomes more responsive and triggers more quickly. No more “ghost” movements cause any automation to work. Finally, my smart home actually felt smarter.