Tech

RAG: AI Gets a Personal Touch

RAG (retrieval augmented generation) is how AI answers questions about your own data. Here's how it works, why chatbots on websites use it, and why it beats fine-tuning.

Have you ever asked an AI a question and thought, "this would be so much more useful if it actually knew my stuff"?

I hit this wall constantly. ChatGPT can explain quantum physics, but it has no idea what's in my project docs. Claude can write beautiful code, but it doesn't know how my blog's backup script is set up unless I paste it in. The model is brilliant and generic at the same time.

That gap, between what an AI knows in general and what it knows about you, is exactly what RAG was built to close. And once you understand it, you'll start seeing it everywhere. That helpful chatbot on your bank's website? RAG. The support bot that somehow knows the refund policy word for word? Also RAG.

Let me break down what it is, how it works, and why it's quietly become the most common pattern in applied AI.

The problem: smart, but frozen

Large language models like GPT and Claude are trained on huge snapshots of the internet. Two things follow from that:

They're frozen in time. Anything that happened after training, they simply don't know.
They only know public information. Your company wiki, your customer data, your personal notes were never part of the training set, and you wouldn't want them to be.

So when you ask a plain LLM "what's our refund policy?", it does the only thing it can do. It guesses. Confidently. This is where hallucinations come from, and it's why nobody sane wires a raw LLM directly to their customers.

You could paste your entire knowledge base into every prompt, but context windows have limits and tokens cost money. There had to be a better way.

RAG: the open book exam

RAG stands for Retrieval Augmented Generation. The name sounds academic, but the idea is beautifully simple.

Instead of expecting the model to memorise everything, you let it look things up.

Think of it like an exam. A plain LLM is a student answering from memory alone. Impressive, but it will confidently misremember things. A RAG system is the same student in an open book exam. Before answering, it flips to the relevant page, reads the actual content, and answers based on what's in front of it.

How RAG actually works

Every RAG system, from a weekend project to an enterprise chatbot, follows the same four steps.

Diagram of the RAG pipeline: chunk and embed documents, embed the question, retrieve matching chunks, then augment and generate the answer — The four steps of a RAG pipeline

Step 1: Chunk and embed your knowledge

You take your documents (blog posts, PDFs, help articles, whatever) and split them into small chunks. Each chunk gets converted into an embedding, which is a long list of numbers that captures its meaning. Chunks about similar topics end up with similar numbers.

These embeddings go into a vector database. This happens once, ahead of time.

Step 2: Embed the question

When a user asks something, their question gets converted into an embedding too, using the same model. Now the question and your documents live in the same mathematical space.

Step 3: Retrieve the relevant chunks

The vector database finds the chunks whose embeddings sit closest to the question's embedding. This is semantic search, so it matches meaning, not keywords. Ask "how do I get my money back?" and it finds the refund policy, even though the word "refund" never appeared in your question.

Step 4: Augment and generate

The top matching chunks get stuffed into the prompt alongside the original question, roughly like this:

Here is some context:
[chunk 1: refund policy...]
[chunk 2: return shipping...]

Using only the context above, answer:
"How do I get my money back?"

The LLM now answers from your actual content instead of its memory. Retrieval, augmentation, generation. That's the whole pipeline.

Why this feels like personalisation

Here's the part I find genuinely elegant. The model never changes. You're not retraining anything. You're just changing what it reads before it speaks.

Swap the vector database and the same model becomes a completely different assistant:

Feed it your codebase and it becomes your senior dev who knows the repo
Feed it your health logs and it becomes a coach who knows your history
Feed it this blog and it becomes... well, me, but faster at finding my own posts

That's why RAG feels personal. The intelligence is generic, but the knowledge is yours. It's the difference between talking to a smart stranger and talking to a smart friend who's read everything you've written.

Those chatbots on every website? Same recipe

Once you know the pattern, you can't unsee it. The vast majority of "AI assistants" you meet online are RAG systems wearing different outfits:

Customer support bots retrieve from help centre articles
Documentation assistants ("Ask AI" buttons on dev docs) retrieve from the docs themselves
Internal company bots retrieve from wikis and Slack history
Legal and finance tools retrieve from contracts and filings

Nobody is training a custom model per company. They're all running the same loop: embed, retrieve, augment, generate. The moat isn't the model. It's the quality of the knowledge base and the retrieval on top of it.

But why not just fine-tune a model?

Fair question, and it comes up every time I explain this. Fine-tuning means actually updating the model's weights on your data. It has its place, but for "make the AI know my stuff", RAG usually wins:

Comparison chart of RAG versus fine-tuning showing freshness, cost, traceability, and safety differences — RAG versus fine-tuning

Fresh by default. Update a document and the next answer reflects it. No retraining.
Cheaper. A vector database and an API key versus GPU hours.
Traceable. You can show which chunks the answer came from, which means citations instead of vibes.
Safer. Your data stays in your database, retrieved on demand, not baked into a model.

Rule of thumb I've settled on: fine-tuning changes how a model behaves, RAG changes what it knows. Most of us want the second one.

The best part: you can build this yourself

This isn't locked behind enterprise pricing. The entire stack is accessible to anyone comfortable with a bit of code. Postgres with the pgvector extension gives you a perfectly good vector database, and an embeddings API handles the numbers.

Which is exactly where this series is going next. In the follow-up post I'll build a working RAG pipeline step by step using pgvector and the OpenAI API: setting up the database, chunking and embedding real content, running similarity search, and wiring it into a chat completion. Same philosophy as everything else on this blog. No magic, just parts you can see and understand.