What is RAG? (Retrieval-augmented generation in plain English)
Retrieval-augmented generation, or RAG, is the technology behind every "ask questions of your documents" tool you have heard about. Plain-English explanation, with the implications for SMEs.
You may have seen the phrase "RAG" or "retrieval-augmented generation" and wondered if it is something you should care about. The short answer: yes, but you do not need to understand the technology to use it. You need to understand what it makes possible.
This article gives the plain-English explanation, then the practical implications for a small business in North Wales.
The problem RAG solves
AI models like ChatGPT and Claude are trained on a huge amount of public data. They know a lot about the world. They know nothing about your specific business - your products, your client list, your contracts, your processes.
You can paste a document into the conversation and ask questions about it. That works, but only for one document at a time, only up to the context window limit, and only for as long as the conversation lasts.
What if you have 200 contracts and want to ask questions across all of them? What if you have five years of customer emails and want to find patterns? What if you want to build a customer-facing chatbot that knows your specific FAQ? Pasting individual documents does not scale.
How RAG works (the simple version)
RAG is a pattern that combines two things: a search system over your documents, and an AI model.
When you ask a question, the system first searches your documents and finds the relevant chunks. Then it gives those chunks to the AI as context, along with your question. The AI answers using the chunks - not from its general training, but from your actual material.
The result is an AI that answers questions using your business's specific knowledge - your contracts, your products, your processes - rather than from generic public training data.
The technical details (vector embeddings, chunking strategies, hybrid search) are interesting if you build these systems. For an SME owner deciding whether to use one, the mental model is "AI plus your filing cabinet".
A concrete example
Take a North Wales accountancy practice. They have:
- Five years of client emails (anonymised)
- Their internal advisory templates and notes
- Welsh and UK tax guidance documents
- Their previous client letters and explainers
A RAG system trained on this material lets a junior staff member ask: "What did we tell clients last year about the change to NIC thresholds?" - and get back the actual previous communications with sources cited.
Without RAG, the same question requires manually searching through email and document folders. With RAG, it is a 5-second answer with the source attached.
Why this matters for SMEs
RAG turns the documents you already have into a queryable knowledge base. For most small businesses, this is the most valuable AI use case after general drafting.
Three places where it lands well for SMEs:
1. Customer-facing FAQ chatbots. Trained on your actual help content, returning answers that reflect your actual business rather than the AI's generic guess. Hospitality businesses, online retailers, service businesses with deep FAQ libraries all benefit.
2. Internal knowledge search. Most businesses lose hours a week to "where did we save that thing?". A well-set-up RAG system over your shared drive answers in seconds.
3. Document analysis at scale. AI document processing for accountants, solicitors, surveyors. Search across years of past work to find precedents.
The cost
The good news: RAG is no longer expensive. What used to require a developer and £10,000 of setup is increasingly available as off-the-shelf products.
Three patterns for SMEs in 2026:
Built-in to your tools. Microsoft Copilot already does basic RAG over your Microsoft 365 content. Google Gemini does the same in Workspace. ChatGPT's "Custom GPTs" let you upload documents to a single GPT. These are the simplest starting point.
Specialist no-code tools. Tools like NotebookLM (Google), Custom GPTs (OpenAI), Claude Projects let you upload a folder of documents and ask questions. Free tier or low monthly cost.
Custom builds. For larger document sets or specific integrations (a customer-facing chatbot on your website, a tool that searches your CRM), you need a developer or an AI consultant. Costs are typically a few thousand pounds for a useful first version, far cheaper than two years ago.
The limits
RAG is not magic. Three honest limitations:
Quality depends on the source material. If your documents are scattered, inconsistent or out of date, the RAG system inherits those problems. The first task is usually a tidy-up of the source material.
Hallucination still happens. RAG reduces hallucination because the AI is grounded in your documents, but it does not eliminate it. The AI can still misinterpret a document or stitch together two pieces incorrectly. Always check critical answers. The hallucination guide covers the discipline.
Setup work is real. "Upload your documents and ask questions" sounds easy. The work is in deciding which documents, structuring them sensibly, and refining the system based on actual use. A weekend rather than an afternoon.
The honest summary
RAG is one of the most genuinely useful AI patterns for SMEs in 2026. It turns documents you already have into a knowledge base your team can query in plain English. The cost has come down sharply. The setup work is real but achievable.
Most North Wales businesses I work with end up using a built-in or no-code RAG tool first (Copilot, Gemini, NotebookLM, Custom GPTs) before considering a custom build. That is usually the right starting point.
If you would like to scope what a RAG setup would look like for your business, that is the kind of work a discovery call can structure.