RAG (Retrieval-Augmented Generation)

RAG retrieves the most relevant chunks of your knowledge base, documents, or data before generation, so the model answers from real source material rather than its training data alone. It remains the dominant pattern for grounded business agents in 2026.

The pipeline: embed documents into a vector database, embed the user's query, retrieve the top-N most-similar chunks, feed those into the LLM as context, generate an answer. Variations stack reranking, hybrid keyword+vector search, query rewriting, and tool-use on top of the basic pattern.

When RAG works: well-bounded knowledge bases, fact retrieval, customer support over docs. When it struggles: queries that need synthesis across many documents, queries where the answer isn't a direct retrieval but a derivation, queries where chunk size hides the answer. Modern reasoning models with long context windows are starting to encroach on RAG's territory.