Was ist RAG (Retrieval-Augmented Generation)?

Retrieval-Augmented Generation (RAG) is a method in which a language model first retrieves relevant passages from a defined knowledge base and then generates its answer on the basis of those passages – instead of relying only on what it learned during training.

How RAG works

A RAG system runs in two phases. During indexing, documents are extracted, split into sections (chunks) and converted into embeddings – numerical vectors that represent meaning. During answering, the question is converted into a vector as well; the system searches for the most similar chunks, optionally re-ranks them and passes the best hits to the language model together with the question.

Extract and split documents
Compute embeddings and store them in a vector index
Find matching passages for each question
Generate the answer from those passages and cite the sources

Why companies use RAG

Language models do not know your price lists, contracts or project files. Training a model on them is expensive, slow and hard to control. RAG keeps knowledge outside the model: a new document is available as soon as it is indexed, and deleting a file removes it from future answers. Because the answer is based on concrete passages, users can check the sources – which reduces, but does not eliminate, hallucinations.

RAG and permissions

In a company, the decisive question is not only *which* passage fits, but *who* may see it. A RAG system must filter retrieval by the permissions of the person asking; otherwise the AI becomes a back door to confidential documents. Permissions therefore belong in the retrieval step itself, not just in the user interface.

RAG or fine-tuning?

Fine-tuning retrains a model on your own examples. That helps shape style, format or terminology, but is poorly suited to making current facts reliably retrievable: knowledge learned in training can neither be deleted selectively nor backed by sources, and every change requires new training. For company knowledge that changes constantly and is subject to permissions, RAG is therefore usually the better choice. The two approaches can be combined.

Limits of RAG

Poor source quality leads to poor answers: outdated or contradictory documents are retrieved as well.
Exact figures from spreadsheets are better calculated than retrieved as text snippets.
Questions about relationships across many documents benefit from an additional knowledge graph.

RAG in AIPE

AIPE extracts uploaded and connected documents, splits them into sections and indexes them semantically. Retrieval only searches documents the user is allowed to read, and answers reference the source documents used. Spreadsheet questions are calculated with a dedicated analysis tool instead of text snippets. Read more in the guide AI knowledge base.

What is RAG?