One of the tools I’ve been relying on a lot these days is my local LLM. I use mine through LM Studio, which also lets you upload documents, and it uses RAG technology to enable interaction with your files. Naturally, I started incorporating more of my notes into my local LLM workflow for learning and studying, and it turned out to be more powerful than I expected.

Why my local LLM is perfect for handling my notes

It’s private and organized

My local AI runner of choice is LM Studio, and it has built-in retrieval augmented generation (RAG) for versions 0.3.0 or later. This means that it lets you upload and interact with documents using RAG technology, regardless of the model you use and whether it has RAG capabilities. LM Studio automatically handles the retrieval and augmentation processes using its own internal mechanisms. I can load any of my models, upload my notes, and start prompting for retrieval.

LM Studio does have some limits. It supports up to five documents per chat session with a combined size limit of 30MB. So it works better with shorter documents and smaller note stacks. If you need to use more or larger documents, the native RAG might not be sufficient - it’s worth considering third-party tools in this case (such as the Big RAG plugin for LM Studio). It supports a range of document formats, including PDF, DOCX, CSV, and TXT. And non-supported file types will usually be read as plain text when you upload them.

Another thing I like about using LM Studio with my notes is its organizational features. Most AI tools have their own, but LM Studio’s is simpler and more practical. In the left chat panel, you can rename chats and organize them into folders; the folders feature is hierarchical and lets you nest them indefinitely. All of these features make it easier to keep track of where my chats with the documents are.

Getting my notes into LM Studio

It starts with an organized stack

Before even starting my research or studying sessions in my local AI, I ensure my notes are readable and organized first. I tend to take my notes very messily and leave them to “sort later”, but since incorporating tools like NotebookLM and LM Studio into my daily work, keeping structured and coherent notes is vital. Local LLMs and source-grounded tools don’t infer context the way cloud chatbots do, so you need to give them something solid to work with from the beginning, or they’ll interpret your mess literally.

This simply means using structure with characters like colons and hyphens, fleshing out text fully with proper grammar, and then organizing them into the relevant folders on my computer so I know exactly where to find them. It might sound obvious, but not doing these things is exactly why I ended up with such a messy note stack in the first place. You only need to upload your documents once in each chat session - LM Studio keeps them accessible for all subsequent prompts. And it automatically enables the Rag-V1 integration once you add documents; don’t turn this off.

Surfacing connections in my notes using my local LLM

How I prompt it to get the most out of my notes

Local models don’t really guess what you mean or jump between ideas the way cloud models sometimes do, so I have to be explicit about what counts as a “connection” among my notes and how I want the information given to me. Instead of asking it to just analyze them, I define what type of links it should look for and also give it examples of what that could look like.

I start by defining the role of the model first. For example, “You are analyzing research notes on [topic] to find relationships between ideas.” Then I define the connection, whether it be a theme, repeated terminology, similar problems, or ideas that could be part of the same project - as long as I lay out what I’m looking for clearly. I also include instructions on how to present the results, such as creating a table that lists the connections. And lastly, I include constraints to prevent my local LLM from guessing, for example, “if information is unclear, mark it as uncertain.”

These aren’t hard and fast rules. It will depend on the model you’re using, the runner you’re using, and the types of notes and content you’re working with. But with a local LLM, it’s always a good idea to:

• Start with clear and structured notes from the get-go
• Define what you mean by “connections”, “key points”, or “insights”
• Give it examples of what you’re looking for
• Give it examples of what you don’t want
• Give it a format for how you want the information presented

The hidden links were already there

Feeding my notes to my local AI didn’t magically create connections, it just surfaced the ones I didn’t know were already there. Once I started funneling them through my local AI, they actually became useful. And it also makes the effort of keeping them concise and organized worth it.