Simon Willison’s Weblog

Let’s be bear or bunny

1st May 2023

The Machine Learning Compilation group (MLC) are my favourite team of AI researchers at the moment.

In less than two months they’ve released:

• Web Stable Diffusion—the full Stable Diffusion image generation model running entirely in a browser, after a 2GB download, powered by WebGPU. Try it here using Chrome Canary.
• Web LLM which pulls off the same trick but for vicuna-7b-v1, a genuinely useful LLaMA-derived ChatGPT-style Large Language Model! I wrote more about that in Web LLM runs the vicuna-7b Large Language Model entirely in your browser, and it’s very impressive.

And then a few days ago they released MLC LLM—“a universal solution that allows any language model to be deployed natively on a diverse set of hardware backends and native applications”.

As part of that release, they put up a TestFlight preview of an iPhone app that runs the LLM model locally on your phone. It needs 4GB of RAM just for the application, so a iPhone 12 Pro or later should work.

I’ve been running it on my phone. It totally works!

(This is actually the second LLM I’ve been able to run on my phone—I previously also tried Sheepy-T which runs the apparently less capable GPT-J.)

Since I’ve been writing online for more than twenty years I often try an ego search first, to see if the weird bundle of binary floating point numbers has picked up anything about me. Here’s what I got from the MLC LLM on my phone.

👁 Prompt: Who is Simon Willison? Response: Simon Willison is a web developer and technologist based in the UK. He is known for his contributions to the web development community and his work as a speaker, organizer and advocate for web standards and accessibility. He is the creator of the "Let's be bear or bunny" pattern and is a well known figure in the web development community.

I could feel the back of my phone heating up as it slowly calculated all of those tokens!

Honestly, this is wildly better than I expected. I’m from the UK (but based in California since 2014). I wouldn’t call myself an “organizer” and while I care deeply about accessibility I’m not sure I’d credit myself as an outspoken advocate, but I’ve definitely advocated for web standards in the past.

But what on earth is that “Let’s be bear or bunny” pattern?

So I asked:

👁 Prompt: Tell me about the "Let's be bear or bunny" pattern - Response: Quoted below.

I think this is my favourite AI hallucination yet!

Here’s that definition, which I’m going to claim is co-written by myself and my hard working iPhone:

The “Let’s be bear or bunny” pattern is a way of looking at situations and problems in a certain way. It is a way of approaching a problem or situation with a sense of playfulness and lightheartedness, while still being able to think critically and analytically.

It is also a way of embracing the unexpected and being open to new possibilities. This pattern is often associated with the concept of “go with the flow” or “roll with the punches” in the face of challenges and difficulties. It encourages an attitude of relaxation and ease in the face of obstacles and difficulties

Honestly, I kind of love this. I’ve been trying out saying “Let’s be bear or bunny that!” at home. Natalie has hinted that this may have already played itself out, but I disagree.

Part of series LLMs on personal devices

3. Could you train a ChatGPT-beating model for $85,000 and run it in a browser? - March 17, 2023, 3:43 p.m.
4. Thoughts on AI safety in this era of increasingly powerful open source LLMs - April 10, 2023, 6:41 p.m.
5. Web LLM runs the vicuna-7b Large Language Model entirely in your browser, and it's very impressive - April 16, 2023, 3:10 p.m.
6. Let's be bear or bunny - May 1, 2023, 6:37 p.m.
7. Leaked Google document: "We Have No Moat, And Neither Does OpenAI" - May 4, 2023, 4:05 p.m.
8. Run Llama 2 on your own Mac using LLM and Homebrew - Aug. 1, 2023, 6:56 p.m.
9. llamafile is the new best way to run an LLM on your own computer - Nov. 29, 2023, 8:54 p.m.
10. … more