RAG (retrieval-augmented generation) is a technique where, before answering, the system searches your documents for the chunks that best match the question and passes them to the model along with it. The model answers with that text in view. It makes things up less, though it can still get things wrong.
A question from the app. Pick an answer.
A store's bot reads its up-to-date PDFs, yet quotes an old price. Why?
A store's prices do not come from training: the bot reads them from its files when you ask.
Before answering, it searches the documents for chunks. If it pulls last year's list, it answers from that, perfectly calm.
Having the text at hand cuts down a lot on made-up answers. When it fails, it is often because it searched badly, not because it did not search.
Before any question: the documents are split into chunks and each one gets its embedding, a list of numbers that represents its meaning. That gets stored.
When a question comes in: its embedding is computed, the closest chunks are found, they are placed in the context along with the question and the model answers. The model did not learn anything new: it reads those chunks on the spot, as if you had pasted them into the chat yourself.
The context window has a limit, and every token that goes in is paid for and processed. A hundred PDFs will not fit in one question, and even if they did, sending them whole every time would be slow and expensive. RAG sends only what looks useful for that question.
A small distributor uploads its catalog, shipping terms and opening hours, and connects a bot to WhatsApp. When someone asks "do you ship out of town?", the bot searches those files and answers with what they say, not with what sounds likely. For a small business it is the most direct way to get an assistant that knows its own data without training any model.
"With RAG it no longer hallucinates." It makes things up less, but it can pull the wrong chunk, mix two together or fill in with what is likely when the answer is not in the documents: that is still a hallucination. "If it cites the source, it is right." It can cite a document that does not say that: open the citation. "RAG trains the model on my data." It does not change the model: your documents are read on every question, and to update them you just change the file and recompute its embeddings.