Running local LLMs is all the rage these days in the self-hosting circles. And if you've been intrigued, or have dabbled in it, you'd have heard of Koboldcpp and LM Studio both. While I'd previously dabbled in LM Studio, I recently discovered Koboldcpp and decided to give it a whirl to see if there are any notable differences. Turns out, despite running the same LLM, the two apps use fundamentally different approaches that give their own unique strengths. Specifically, if you are someone who cares about narrative workflow, like storytelling or even role playing, Koboldcpp stands out in a few ways over LM Studio. Here's what I discovered.

Built for storytelling and creative writing

Keep your writing and characters consistent

Launching Koboldcpp makes it immediately obvious that it's designed for story telling and writing first. Look, LM Studio is an excellent tool for managing and running local models. It lets you download, benchmark and interact with models all in one place, making it an extremely flexible tool for a variety of purposes. However, it treats each chat as a unique session, and once you close a chat, the context is gone.

Koboldcpp, on the other hand, carries over features from its roots in interactive fiction. So, when you open it, you're not just chatting with a model. The app lets you create stories, define characters, add world-building details and keep track of notes. These elements stay saved so you can return to them later. It's designed to help you build something that grows over time rather than starting from scratch every time you run it. Moreover, it is especially useful if you're working on long-form writing or role-playing scenarios. You can write a scene, stop and come back to it days or months later without having to reintroduce the context. It's not that LM Studio can't pull this off with manual setup, but Koboldcpp makes it a part of the workflow from the start. The difference is in intent. LM Studio is designed to be a more general-purpose tool, while Koboldcpp is focused on storytelling and creative use.

Better integration with role-playing frontends

Built to work seamlessly with existing storytelling tools

Another area where Koboldcpp stands out is broader integration. The app supports the Kobold API, which is broadly supported by a wide range of role-playing and story-based frontends. So, if you've used something like SillyTavern or other role-playing frontends, they should connect to a Kobold API backend without any extra leg work. You just run the app, copy the local endpoint, and it just works. Everything from character data to prompts, world building notes and more are transferred over as you'd expect without much configuration or troubleshooting.

LM Studio, in comparison, uses an Open AI style API format. That's great for developers building their own tools or scripts, or for conversational chats. However, if you are using it as part of your creative workflow with existing role-playing setups, it's not ideal. Can it work? Yes. But it's not meant for the task. This difference truly highlights the differing approaches taken by both projects. While LM Studio is designed to make local LLMs accessible for a broad range of uses like chat, experimentation and development, Koboldcpp aims to make them more accessible for storytelling. If you're already running an existing tool in the ecosystem, it just makes sense to opt in for Koboldcpp.

Lightweight and focused on interaction

A simpler setup that feels just a bit faster to use

Koboldcpp is built on top of llama.cpp and is distributed as a single executable file. You download one file, run it, and you're good to go. There's no extra set up needed and it starts quickly. Additionally, Koboldcpp supports the same GCUF and GGML formats that LM Studio supports, but remains lighter overall.

In day-to-day use, that light and breezy set-up goes a long way towards responsiveness. There's less overhead and responses come through quicker, especially during interactive sessions where quick to-fro is essential. That isn't an essential feature, but it's a big plus for smoother back-and-forth writing or brainstorming sessions. That's not to say that LM Studio is slow. It's just built for a different purpose. LM Studio runs more background processes and offers a broader set of tools ranging from model management to server hosting. That might be useful to some, but not all. The performance difference isn't night and day and largely comes down to the model being used as well as your hardware, but it is certainly noticeable.

Choosing the right tool for our workflow

Using both apps makes it clear that Koboldcpp and LM Studio serve very different audiences even though the core feature set is fairly similar. LM Studio is the obvious better choice if you want an all-in-one local LLM tool and manager. It's great for checking out new models, comparing quantization, running benchmarks, even using it as an LLM server. LM Studio is geared, ground up, for more technically proficient users.

Koboldcpp, on the other hand, focuses on one thing and does it well. It's meant for writers, storytellers, and role-playing enthusiasts who want AI assistant to create and revisit narratives. The built-in story management features and Kobold API compatibility with frontends make it a solid choice for a very specific kind of user. If you mostly just chat or experiment with models, LM Studio will suit you better. But if your goal is to build something creative and ongoing, Koboldcpp's focused approach can be a real advantage.