Quick answer
Retrieval augmented generation (RAG) is a technique where an AI system first searches a trusted set of documents, then uses what it finds to write its answer. This keeps responses grounded in your own content, reduces made-up facts and lets the answer cite its sources.
Key takeaways
- RAG searches your content before the AI writes a reply.
- It reduces, but does not remove, wrong answers.
- Document quality decides answer quality.
- Always show sources so people can check.
How does RAG work?
- Your documents are split into small chunks.
- Each chunk is converted into a searchable numeric form called an embedding.
- A question arrives and the system finds the closest chunks.
- Those chunks are passed to the AI with the question.
- The AI writes an answer and links to the sources.
Why use RAG instead of training a model?
| Approach | Strength | Limit |
|---|---|---|
| RAG | Uses current documents and shows sources | Needs good search and clean content |
| Fine-tuning | Changes tone and behaviour | Hard to update with new facts |
| Plain chatbot | Quick to start | Guesses from general knowledge |
Where does it fit in a business?
- Staff help desks for policies and procedures.
- Customer support using manuals and FAQs.
- Sales teams searching product information.
- Legal or compliance teams searching internal guidance.
What makes it fail?
Out-of-date files, duplicate versions and messy formatting lead to poor retrieval. Weak chunking can split an answer across pieces.
Retrieval augmented generation also needs access controls, so people only see documents they are allowed to read.
How do you check quality?
- Build a test list of real questions with known answers.
- Check whether the right source was found.
- Look for answers that go beyond the source.
- Collect user feedback on every reply.
A simple example
Imagine a staff member asks, "How many days of carer's leave do I get?" A plain chatbot might guess.
With retrieval augmented generation, the system finds your leave policy, quotes the right paragraph and links to it. The employee can verify the answer in seconds.
That link back to the source is what builds trust.
Bottom line
Retrieval augmented generation gives AI a way to look things up instead of guessing. It works best with clean, current documents and answers that point back to their sources.
Sources and further reading
- Lewis et al.: Retrieval-Augmented Generation (original paper)
- Australian Government: Voluntary AI Safety Standard
- OWASP: Top 10 for LLM Applications
Checked against the sources above and our own project experience. Last reviewed 7 October 2026.
