Advanced Retrieval-Augmented Generation (RAG) Techniques

Retrieval-Augmented Generation (RAG) has experienced a number of advancements in recent years alongside its increasing popularity. In my talk at All Things Open (ATO) 2024 on Oct. 28, I covered a number of the techniques needed to build better RAG. These include chunking, choosing an embedding model and metadata structuring. See below for the video of my presentation.

Considerations for Building a RAG System

One of the most important things when building a RAG system is making it work with the type of data you need. For example, there are many types of text — conversational, documentation, Q&A, lectures and formal documents. You also have to determine exactly what you need from the data: Is it just a dump of all text, or are you looking for specific insights, or only information from embedded charts and graphs?

Just like any other data project, you need to do an analysis to determine what data you are using, how you will ingest it, and what enrichments and transformations are required. Your decisions include cost, size, model license, time to embed data and whether it works with your data specifications.

An incredibly important part of using vector databases and RAG is determining what embedding model to use, from providers like HuggingFace, OpenAI, Google, Meta, PyTorch, Jina AI, Mistral AI or Nomic A. Some models are for dense embeddings such as BAAI/bge-base-en-v1.5, which produces vectors of 768 dimension floating point numbers. There are also sparse embedding models that produce mostly zeros.

You also need to decide which tools to use; many new tools make building RAG less manual, such as LangChain, LlamaIndex, LangChain4J or Spring AI. You can also use AI extract-transform-load (ETL) tools such as DataVolo, Cloudera DataFlow, Airbyte, StreamNative UniConn, Apache Spark, Apache Flink, Ray and Fivetran.

Looking to the Future of RAG

In addition to discussing new advances in the world of RAG, during my ATO talk, I shared some examples of new models, techniques, vector databases and AI advancements that will supercharge the entire concept. These include:

• Chunking
• Embedding model options
• Metadata structuring
• GraphRAG
• Multilingual vs. a specific language
• Multimodal data retrieval
• Query enhancement
• Query routing
• Hierarchical indexing
• Hybrid retrieval
• Agentic RAG
• Self-reflection
• Query routing
• Subqueries

I’ll also share a quick overview of a RAG system that uses Milvus, an open source vector database, to combine a retrieval system with a generative model. By adding smart context quickly retrieved from Milvus to your prompt, you can reduce LLM hallucinations, which is so important.

👁 Vector similarity search with RAG architecture

Watch My Presentation

Other Resources

• Retrieval-Augmented Generation (RAG) With Milvus and LlamaIndex
• Super Charge Your AI Applications With Easy High Speed Multi-Tenancy With Partition Keys
• Ranking for Relevance With BM25
• How Good Is Quantization in Milvus?