What Is RAG (Retrieval-Augmented Generation)?

Imagine you start a new job at a company that sells resume tools. On the first day, a customer calls and asks, “Can I use a Word file, or must I upload a PDF?” You do not know the answer from memory. But you are not stuck. You open the company help center, search for “file type”, read the right paragraph, and answer in your own words. You did not memorise the help center. You looked it up and then explained it.

That is exactly what Retrieval-Augmented Generation, or RAG, does for an AI model. The model is the person on the phone. Your documents are the help center. The “retrieval” part is the search. The “generation” part is the model writing a clear answer from what the search found.

In this lesson you will understand why RAG exists, learn the six steps of a RAG system, and build the first version in your AI Lab: a help-center bot that finds the right paragraphs in six small documents and answers only from them.

The problem: a model only knows what it was trained on

A language model like Claude learned from a huge amount of text, but that learning stopped at a certain date. After that it does not know anything new. It also never saw your private files: your price list, your company rules, your customers' contracts, or the PDF a user uploaded five minutes ago.

If you ask a model about something it does not know, it may do something dangerous: it can write a confident, smooth, wrong answer. This is called a hallucination. The model is not lying on purpose. It is built to continue text in a believable way, and a believable answer is not always a true one.

There are three ways to give a model knowledge it does not have:

• Put everything in the prompt. This works for a short document. It fails when you have hundreds of pages, because the prompt has a size limit and you pay for every token every time.
• Fine-tune the model. You train it further on your examples. This is good for teaching a style or a format. It is a poor way to teach facts, it is slow and costly, and you cannot easily remove or update one fact later.
• Retrieve and then ask (RAG). For each question, find the few pieces of your data that matter and put only those in the prompt. This is cheap, easy to update, and it lets you show your sources.

For most business apps with changing knowledge, RAG is the first thing to try.

The six steps of RAG

Every RAG system, from a weekend project to a large product, does these six jobs. Learn them by name, because the next lessons improve one step at a time.

1. Ingest. Load your documents: text, Markdown, PDF, Word, web pages. Clean them.
2. Chunk. Cut each document into small pieces, called chunks, so that you can retrieve just the useful part.
3. Embed. Turn each chunk into a list of numbers (a vector) that represents its meaning. We cover this in lesson 14.
4. Store. Save the chunks and their vectors in a place that can search them quickly. Lesson 15 covers vector databases.
5. Retrieve. When a question arrives, find the chunks that are most related to it.
6. Generate. Build a prompt with the question and those chunks, and ask the model to answer using only that text.

Steps 1 to 4 happen before any user asks anything. Steps 5 and 6 happen on every question. In this first lesson we keep it very simple on purpose. We chunk by paragraph and retrieve by plain keywords, with no vectors at all. A simple version that works end to end teaches you more than a clever version you cannot see inside.

A worked example

Suppose our help center has this paragraph in a file called resume-basics.md:

A user asks: “Which file type should I upload?” The system searches, finds that paragraph (the words “file” and “PDF” and “Word” are close to the question), and builds this prompt for the model:

<context>
<chunk id="1" source="resume-basics.md">
Use one simple font such as Arial or Calibri at size 10 or 11.
Save the file as a PDF unless the job post asks for Word.
</chunk>
</context>

Question: Which file type should I upload?

The model answers: “Upload a PDF, unless the job post asks for Word.” The answer is short, correct, and it came from your own text, not from the model's memory.

Notice the tags like <context> and <chunk>. They are not magic. They are a clear way to tell the model, “this part is data, that part is the question.” This habit becomes very important in the security lessons later in this course.

The most important sentence in a RAG prompt

Retrieval gives the model the facts. But the model still needs rules about how to use them. Our system prompt says three things:

• Answer using ONLY the context provided.
• If the context does not contain the answer, say exactly: “I don't have that in my notes.”
• Keep the answer short and in plain English.

The second rule is the one that protects you. Without it, when the search finds nothing useful, the model quietly falls back on its own memory and may invent an answer. With it, the bot admits it does not know. An honest “I don't know” is much better than a confident mistake.

There is a second layer of protection in our code. If the search finds no chunk at all, we do not call the model. We return the “I don't have that in my notes.” message directly. This saves money and removes any chance of a made-up answer for that case.

How keyword retrieval works

Our first retrieval method is simple. We break the question into words, throw away tiny common words like “the” and “is” (called stop words), and give each chunk a score. A chunk gets points for every question word that it also contains. A rare word gets more points than a common word. If the question says “Zendesk” and only one chunk contains it, that chunk should win. If the question says “resume” and every chunk contains it, the word tells us almost nothing.

This idea is called inverse document frequency: the fewer chunks contain a word, the more valuable the word is. The code uses Math.log(1 + total / count). You do not need to remember the formula. You only need to remember the idea: rare shared words matter most.

Keyword search is fast, free and easy to understand. Its weakness is vocabulary. If the user asks “How long should my resume be?” and your document says “one or two pages”, the words “long” and “pages” do not match, and keyword search misses it. Lessons 14 and 16 show ways to reduce this problem. For now, remember that retrieval is the step where most RAG systems fail. If the right paragraph is not found, the best model in the world cannot answer well.

Where RAG goes wrong

• The right chunk is not retrieved. The most common problem. Fix it by improving chunking, search and documents.
• The chunk is found but cut in half. An idea is split across two chunks and neither has the full answer.
• Too many chunks. If you send ten chunks, the model may get lost or mix them up. Three to five good chunks usually beat ten average ones.
• The model ignores the context. A clear system prompt and tags help. Testing (lesson 22) tells you if it still happens.
• The documents are old or wrong. RAG repeats what your documents say. If a document is out of date, the answer is out of date.

Your task: build the help-center bot

In this task you add your first RAG feature to the AI Lab: a page where a user asks a question, and the answer comes only from six small help-center documents.

What you will build

• A folder server/rag/docs with six short Markdown help articles (the solution zip has ready-made ones; you can write your own).
• A loader that reads the documents and cuts them into paragraph chunks.
• A keyword search that scores chunks and returns the best three.
• A route POST /api/rag that retrieves, builds a prompt with the chunks inside tags, and asks Claude to answer from them only.
• An Angular page called Ask docs that shows the answer and the sources it used.

Steps

1. Create server/rag/docs and add at least four Markdown files. Each file starts with a title line that begins with # and has three or four short paragraphs. Write them about any topic you know, for example resume tips.
2. Write server/rag/load.js with a function that reads every .md file and returns a list of chunks. One paragraph is one chunk. Give each chunk an id like resume-basics.md#2 and remember its source file.
3. Write server/rag/keyword.js. Split text into lower-case words, remove stop words, and score each chunk by the shared rare words. Return the best three chunks that scored above zero.
4. Write server/routes/rag.js. If there are no chunks, return the “I don't have that in my notes.” message without calling Claude. Otherwise put the chunks in a <context> block and call Claude with the rules from the lesson as the system prompt.
5. Build the Angular page. It needs one input, one button, the answer, and a list of sources with their scores. Add the page to the menu and routes.
6. Test four questions: two that your documents answer, one that they do not (for example “What is the capital of Australia?”), and one where your wording is different from the document. Write down what happens in each case.

Hints

• Keep the system prompt short and strict. Say what to do when the context has no answer.
• Cut the question to 500 characters on the server so nobody can send a huge prompt.
• Show the score next to each source. Seeing the scores teaches you how the search thinks.
• Do not forget to handle an empty question with a clear 400 message.

You know it works when a question in your documents gets a short, correct answer with the right source listed, an unrelated question gets “I don't have that in my notes.” without any model call, and the page shows a friendly message if the server key is wrong.

Try to build it yourself first. The struggle is where the learning happens. Then compare with the solution.

Solution

The solution zip contains the full AI Lab after this lesson. These are the new files. First the loader, which reads the documents and makes paragraph chunks:

import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

const here = path.dirname(fileURLToPath(import.meta.url));
export const DOCS_DIR = path.join(here, 'docs');

// Read every .md file in server/rag/docs. Returns [{ source, title, text }].
export function loadDocs() {
  return fs.readdirSync(DOCS_DIR)
    .filter((f) => f.endsWith('.md'))
    .sort()
    .map((file) => {
      const raw = fs.readFileSync(path.join(DOCS_DIR, file), 'utf8').trim();
      const title = (raw.match(/^#\s+(.+)$/m) || [])[1] || file;
      return { source: file, title, text: raw.replace(/^#\s+.+\n+/, '') };
    });
}

// Naive chunking for lesson 13: one paragraph = one chunk.
// Lesson 14 replaces this with better chunking.
export function paragraphChunks() {
  const chunks = [];
  for (const doc of loadDocs()) {
    doc.text.split(/\n{2,}/).map((p) => p.trim()).filter(Boolean).forEach((text, i) => {
      chunks.push({ id: `${doc.source}#${i + 1}`, source: doc.source, title: doc.title, text });
    });
  }
  return chunks;
}

The keyword search:

// Keyword retrieval: the simplest useful search.
// Score = how many rare words the question and the chunk share.

const STOP = new Set('a an the and or of to in on at for with from by is are was were be been it this that these those i you we they my your our their me do does did how what when where why which can should would will not no yes as if but so than then there here about into over'.split(' '));

// A very light "stemmer": pages -> page, formatting -> formatt, skills -> skill.
// It is crude on purpose. It only needs to make the same word look the same.
const stem = (w) => (w.length > 4 ? w.replace(/(ing|ed|es|s)$/, '') : w);

export const tokenize = (text) =>
  (String(text).toLowerCase().match(/[a-z0-9][a-z0-9+#.-]*/g) || [])
    .map((w) => w.replace(/[.-]+$/, ''))
    .filter((w) => w.length > 1 && !STOP.has(w))
    .map(stem);

export function keywordSearch(question, chunks, k = 3) {
  const q = [...new Set(tokenize(question))];
  const docFreq = new Map();
  const tokenized = chunks.map((c) => new Set(tokenize(c.text + ' ' + c.title)));
  for (const set of tokenized) for (const t of set) docFreq.set(t, (docFreq.get(t) || 0) + 1);

  return chunks
    .map((chunk, i) => {
      let score = 0;
      for (const t of q) {
        if (tokenized[i].has(t)) score += Math.log(1 + chunks.length / (docFreq.get(t) || 1));
      }
      return { ...chunk, score: Number(score.toFixed(3)) };
    })
    .filter((c) => c.score > 0)
    .sort((a, b) => b.score - a.score)
    .slice(0, k);
}

The route. Notice the three stages in the comments: retrieve, augment, generate.

import { Router } from 'express';
import { client, MODEL, textOf } from '../claude.js';
import { paragraphChunks } from '../rag/load.js';
import { keywordSearch } from '../rag/keyword.js';

const router = Router();
const chunks = paragraphChunks(); // read once when the server starts

const SYSTEM = `You answer questions for a resume website using ONLY the context the user provides.
If the context does not contain the answer, say exactly: "I don't have that in my notes."
Keep answers short and in plain English. Do not use outside knowledge.`;

// POST /api/rag  { question }  ->  { answer, sources, usedModel }
router.post('/rag', async (req, res) => {
  const question = String(req.body?.question || '').trim().slice(0, 500);
  if (!question) return res.status(400).json({ error: { message: 'Please type a question.' } });

  // 1. Retrieve: find the 3 best chunks.
  const sources = keywordSearch(question, chunks, 3);

  // No match at all: do not even call the model, it would only guess.
  if (!sources.length) {
    return res.json({ answer: "I don't have that in my notes.", sources: [], usedModel: false });
  }

  // 2. Augment: put the chunks into the prompt, inside tags so the model can tell data from instructions.
  const context = sources.map((s, i) => `<chunk id="${i + 1}" source="${s.source}">\n${s.text}\n</chunk>`).join('\n');

  // 3. Generate: ask the model to answer from that context only.
  const reply = await client.messages.create({
    model: MODEL,
    max_tokens: 400,
    system: SYSTEM,
    messages: [{ role: 'user', content: `<context>\n${context}\n</context>\n\nQuestion: ${question}` }],
  });

  res.json({ answer: textOf(reply), sources, usedModel: true });
});

export default router;

And the Angular page:

import { Component, inject, signal } from '@angular/core';
import { FormsModule } from '@angular/forms';
import { HttpClient } from '@angular/common/http';
import { firstValueFrom } from 'rxjs';
import { errorText } from '../util';

type Source = { id: string; source: string; text: string; score: number };

@Component({
  selector: 'app-rag',
  imports: [FormsModule],
  template: `
    <h1>Ask the help center (RAG)</h1>
    <p class="hint">The server finds the best paragraphs in our own documents, then asks Claude to answer from them only.</p>

    <div class="card">
      <label for="q">Your question</label>
      <input id="q" [(ngModel)]="question" (keyup.enter)="ask()" />
      <button (click)="ask()" [disabled]="busy() || !question.trim()">{{ busy() ? 'Searching...' : 'Ask' }}</button>
      @if (error()) { <div class="err">{{ error() }}</div> }
    </div>

    @if (answer()) {
      <div class="card">
        <h2>Answer</h2>
        <div class="answer">{{ answer() }}</div>
        <h2>Sources used ({{ sources().length }})</h2>
        @for (s of sources(); track s.id) {
          <p><strong>{{ s.source }}</strong> <span class="muted">score {{ s.score }}</span></p>
          <pre class="out">{{ s.text }}</pre>
        } @empty {
          <p class="muted">Nothing matched, so the model was not called.</p>
        }
      </div>
    }
  `,
})
export class RagPage {
  private http = inject(HttpClient);

  question = 'How many pages should my resume be?';
  busy = signal(false);
  error = signal('');
  answer = signal('');
  sources = signal<Source[]>([]);

  async ask() {
    this.busy.set(true);
    this.error.set('');
    this.answer.set('');
    try {
      const r = await firstValueFrom(this.http.post<{ answer: string; sources: Source[] }>('/api/rag', { question: this.question }));
      this.answer.set(r.answer);
      this.sources.set(r.sources);
    } catch (e) {
      this.error.set(errorText(e));
    } finally {
      this.busy.set(false);
    }
  }
}

What you learned

• A model only knows its training data. RAG gives it your data at question time.
• RAG has six steps: ingest, chunk, embed, store, retrieve, generate.
• A strict prompt (“answer only from the context, otherwise say you do not know”) matters as much as retrieval.
• Keyword search is simple and useful, but it misses answers when the words differ.

In the next lesson you will cut documents into better chunks and meet embeddings, which help the search understand more than exact words.