I’ve been running my homelab on an 8-year-old laptop as a bare-metal Debian server. I rarely interacted with it directly, though. At my desk, I used my PC, and everywhere else, I relied on my MacBook. My MacBook had become my main control hub for the homelab server and my NAS. I didn’t mind managing my server manually, but I did mind having to sit down every time. Even for a routine check, I had to log into my MacBook, either SSH into the server or open Portainer in a browser, and then navigate from there.

That’s when I decided to try Claude’s new computer use feature. I had the Claude Pro subscription and an Apple device, so I already had everything to test it. Instead of setting up another remote-control layer,I used my phone and the Claude app’s Computer Use capability to open apps, navigate interfaces, and run commands with just plain English. For the first time, I didn’t have to sit down to manage my homelab; I was just giving instructions on what to do.

What "computer use" actually means in practice

More than just remote access

When people hear ‘computer use,' they might assume it’s just another remote access tool. But it works differently. Most remote tools connect you to your system by sharing the screen, and they let you control it directly. Claude’s computer use feature takes screenshots, observes them by reading, and then decides what to click, type, or navigate. It’s not a direct remote control; it’s Claude looking at the screen and acting on it, similar to how a human would.

Instead of executing the steps myself, which I usually did on my MacBook directly or remotely, I could now just describe what needed to be done. Claude handled everything on my MacBook and gave a response on my phone. I didn’t need to know which app to open or which menu to navigate. With this, I could manage my homelab from anywhere; the only requirement was that my MacBook stayed on.

My setup: Mac as control hub, Debian as the actual homelab

Two machines, one workflow

My setup is fairly straightforward. My server runs on a separate machine, and the control hub runs on a separate machine. Although the server isn’t headless, I rarely access it directly. The Debian server runs Docker services such as Jellyfin, Immich, and Nextcloud. The MacBook acts as my daily driver and the control hub for the server. I use it to navigate dashboards and access the server via SSH.

At the time of testing, the new computer use feature wasn’t available for non-Apple devices, which worked in my favor. I didn’t have to give Claude access to my server directly; it had access to my Mac only. The setup was simple: Debian as the server, a Mac as the control hub, and the Claude app as the interface. On my iPhone, I used the Claude app’s Dispatch feature for the phone-to-Mac link.

In practice, getting Computer Use to work wasn’t as seamless as it initially looked. Most of the Apple system apps were restricted, including Terminal. Safari was read-only, and the Brave browser couldn’t be detected at all. Chrome worked once I granted the necessary screen recording and accessibility permissions in macOS settings. The setup required a few adjustments, but once configured, I typed my first instruction from my phone and watched it execute without any human intervention, and I realized it was worth exploring further.

What I can now do from my phone (real homelab tasks)

This is where it becomes useful

The Claude app’s Dispatch feature is what proved most useful in my case. I tried accessing Nextcloud on the iPhone app, and it kept loading indefinitely. That’s when I decided to test the computer use feature to actually resolve the issue without directly accessing my Mac.

I instructed Claude to open Portainer on my Mac and to check Nextcloud logs. It first reported that Portainer wasn’t accessible on the default port, as it was trying to open on the Mac's local IP address. I then asked it to access Portainer via my server’s local IP. The Nextcloud logs came back clean. I followed up and asked it to check Cloudflared logs for Nextcloud, since I was using Cloudflare tunnels. That’s where it found the real issues. There were 491 errors within about an hour. Claude diagnosed it as a Docker network isolation issue since Cloudflare and Nextcloud were in separate networks and couldn’t reach each other. It suggested connecting them to the same Docker network. Once I applied the fix, I asked Claude to run a verification to see whether the problem was resolved or not.

Without it, I could have solved the issue myself eventually, but it would have taken longer than the few minutes it took with this setup; I would have had to open my Mac and go look for it.

Where it still falls short

Good, but not hands-off

It works, and for the right tasks, it works well. But it needs quite a bit of work to get started with. It didn’t work for me out of the box. Apple is very strict with third-party access to its system applications.

After running a few instructions, I figured that the Terminal app was click-only; Claude couldn’t type anything. I then asked it to access Safari, but it turned out to be read-only. Claude could only read what was open and couldn’t navigate. I then tried the Brave browser, but due to some limitations, Claude couldn’t detect it. Eventually, I got it working on Chrome, but only after installing the Claude for Chrome extension and enabling the ‘Allow all browser actions’ toggle in Dispatch.

It’s slower than direct execution, like in Claude Code. It has to read the screen via screenshots and then navigate step by step. It might need more follow-ups to complete a multi-step instruction, and sometimes it can misread the screen context. For instance, it initially used the wrong IP address to access Portainer. It may also be a concern for privacy-conscious users. While Claude can view your screen to interpret context, it processes images locally first and applies privacy safeguards before sending data to the cloud.

Since it’s currently limited to Macs, the system must stay awake, typically with the lid open — unless configured for clamshell mode with external peripherals. Limitations are there, but it won’t bother me, given the friction it can reduce.

Still early, but the direction is clear

Claude’s new computer use feature didn’t replace any traditional tools like SSH or Portainer. The server still did what it was supposed to, and the Mac was still used as a control hub; it only changed the way I interacted with them. Now, I didn’t need to sit down at my desk to do a few routine checks; I could do it from anywhere by typing the instructions on the Claude app on my phone. Right now, it works well for well-defined tasks and when the context is clear on the screen. Even in the early stage, it shows a shift toward a flexible way of managing homelabs.