A vector database stores data as lists of numbers, called embeddings, that represent its meaning, and quickly finds the ones closest to a query. So it searches by sense, not by exact words: "I was charged twice" finds "duplicate charge". RAG can use it to pick which passages to read.
A question from the app. Pick an answer.
Your bot answers from 20 FAQs. Do you need a vector database?
Embeddings can be compared without a separate database: with 20 texts, a simple program checks them all instantly.
Twenty short answers fit together in the context, and the model reads them all on every question. If the content grows, it pays to retrieve only what is relevant.
The model reads the text it gets in its context. A vector database helps retrieve content; it is not needed to read it.
A support team has thousands of solved cases. Someone writes "I paid twice". A word search looks for "paid" and "twice", and the case titled "duplicate charge on card" has neither word: it does not show up.
With a vector database, each case was stored with its embedding. The question gets its own, the closest points are found and "duplicate charge" shows up among them, because it means the same thing even without sharing a word.
A traditional database mostly looks up exact values in rows and columns: this customer, this date. A vector database is built to find what is most similar. To avoid comparing the query against millions of vectors one by one, it usually uses an approximate index that jumps straight to the right neighborhood: it saves time, but now and then it misses a close one.
In a RAG system with vector search, it is the middle step: a model turns your documents into embeddings, the database stores them and finds the passages closest to each question, and the model answers by reading them.
If you have thirty FAQs or a manual a few pages long, it probably fits whole in the context window: paste it in and you are done. With a few thousand passages, a simple program can compute their embeddings and compare them all in memory, with no separate server. A vector database pays off when there are too many to scan in time, they change often or you need to filter by date or customer.
And it does not replace word search: for a product code or an order number, exact matching wins. That is why many systems combine both.