Local LLMs are usually something that gets associated with desktop workflows, especially considering how resource-hungry they can be. So that's pretty much what I defaulted to for my local AI work, which primarily consists of interacting with private documents. But that whole privacy logic kind of stopped applying the second I walked away from my PC, because then I'd just default to ChatGPT or Claude on my phone like nothing happened. And the phone is actually where most of your personal stuff lives when you think about it.

Local LLMs on mobile genuinely never crossed my mind until pretty recently. The assumption was that phone hardware couldn't pull it off, but turns out, that's been less than true for a hot minute now and the recent edge-focused models are specifically built for this. So I finally bit the bullet and gave it a spin for the past couple of weeks to see if local AI on my phone could actually replace ChatGPT, Claude, and Gemini…

Finding a mobile LM Studio

I actually found something better

LM Studio doesn't have a mobile version, so of course I went looking for an equivalent on mobile that has a clean GUI and one-click model downloads. Reddit's local LLM crowd kept pointing to PocketPal: it's free, open-source, and available on both iOS and Android.

The model selection is what actually drew me to it over some of the other mobile runners. PocketPal pulls models directly from Hugging Face inside the app, so any GGUF model you'd run in LM Studio you can grab here too. Vision is supported with a separate projection model file you download alongside the main one, and the app has GPU acceleration where the hardware supports it. Recent updates added robust Android NNAPI hardware acceleration alongside Apple's Metal framework on iOS.

One thing though: PocketPal is built on React Native, so there's a bit of strain compared to native Apple-only apps like Private LLM, and at longer context lengths or heavier prompts, you might notice it (I definitely got some lag). I'll probably try a couple of others down the line; Locally AI for iOS and Maid on Android both come up a lot, but PocketPal is what I'm starting with here.

Gemma 4 E2B is built for this exact situation

The one Google designed specifically for phones

There are a couple of other mobile-optimized models worth flagging - Phi-4 Mini and Qwen3 1.7B both come up a lot, and I'm not ruling them out for future testing. But I went into this knowing I wanted Gemma 4 E2B before I even installed PocketPal. Partly because I've already been running E4B on my PC and I like the family, but mostly because E2B is the variant Google specifically designed for phones. It's the smallest of the Gemma 4 models alongside the E4B, and they support audio and image input too. Most edge models this size are text-only, so multimodal at 2B parameters is genuinely uncommon.

I went with Unsloth's GGUF because they're the most trusted re-packager in this space, their files use Dynamic 2.0 quantization which gets you better accuracy at the same file size, and their version sits well above a million downloads. Within that, I went with Q4_K_M, the standard speed-quality sweet spot for mobile, at around 3GB. The F16 multimodal projection file goes alongside it so vision actually works. And the context window is 128K tokens, which is comfortably more than I'm going to use on a phone. For context, I'm running this on iPhone 16 which has 8GB RAM.

I went into this planning to use it for the same stuff as my desktop setup, which is not coding or technical work, but just general chat. So quick explanations, short study guides, a bit of deeper research, plus image and doc analysis for files I'd rather not upload to cloud AI.

Ditching cloud AI apps for my new local model

Impressive, but with trade-offs

The actual chatting part was by far the most solid. Responses are conversational, easy to follow, and don't feel dumbed-down. For the kind of mobile use I described - private notes, quick questions, image stuff - it's covering what I'd otherwise reach a cloud app for. PocketPal also gives me way more knobs than ChatGPT, Claude, or Gemini will ever expose on mobile. All the usuals are there: per-session system prompts, savable presets, temperature, repetition penalties, the whole sampler panel. So I get to really fine-tune its behavior, which is pretty much necessary for AI that doesn't have memory or persistent context.

Image input is where E2B really earned its place. I can screenshot anything (in my case, usually something from an editing app), drop it in, and the responses actually show it reads and interprets everything that's in the image instead of guessing from context. PocketPal also lets you capture images in-app to then upload to the chat, which I found pretty useful for things like understanding product labels.

Where the cloud apps still win isn't the model, it's the wrapper. There are no projects or memory across chats, every session starts cold and disconnected from your workflow. There's no web search or MCP-style tool calling either, so anything time-sensitive is off the table. Also, the attachment + icon only gives you the image gallery for inputs, it doesn't have a PDF or file picker, so document interaction means copy-pasting from another app. This pretty much makes it unsuitable for research papers.

Chats can be exported as JSON to your files, which is technically more portable than what the cloud apps offer, but JSON isn't something I'm going to actually read - the use case here is copying the text back into another chat, or sending it to my local LLM on desktop for context. PocketPal also has TTS with three neural voice engines - Kitten, Kokoro, and Supertonic - that read responses aloud, but it's one-directional, so there's no speech-to-text on the input side yet.

My phone caught up to my desktop

Going in, I figured this would be a fun experiment that ended with me right back in ChatGPT after a few days. Instead, PocketPal and Gemma 4 E2B genuinely handle most of what I was reaching cloud apps for on mobile. There are real tradeoffs, like no folders or web access or proper document support. But for private, on-the-fly chatting with image input, my phone is now pulling its own weight. The cloud apps stay closed more often than not.