Chunking, Embeddings and Vector Search

In lesson 13 we cut documents into paragraphs and searched by exact words. It worked, but you saw the weak spot: if the user's words are different from the document's words, the search can miss the answer. In this lesson you improve two things. First, you learn to chunk well, because bad chunks hide good answers. Second, you meet embeddings, the idea that lets a search compare meaning instead of only spelling.

By the end you will build a Search page in your AI Lab where you can move two sliders, chunk size and overlap, and watch the results change. That page will teach you more about retrieval than any amount of reading.

Why chunking matters

A chunk is the unit of retrieval. The search finds chunks, and the model reads chunks. So the shape of your chunks controls everything.

• Chunks that are too big carry many ideas at once. The search score gets blurred, and you send extra text to the model that it does not need. You pay for it too.
• Chunks that are too small may not contain a complete answer. A chunk that only says “as shown above” is useless on its own.
• Chunks cut in the wrong place split a sentence or a list in half. The first half goes to one chunk and the second half to another, and neither one answers the question.

There is no perfect size. Common starting points are about 200 to 500 tokens per chunk (roughly 150 to 400 words), with a small overlap. Short, direct documents like FAQs do well with smaller chunks. Long, flowing articles do better with larger ones. The right way to choose is to try, measure and compare. Lesson 22 shows you how to measure.

Overlap: a safety net at the borders

Overlap means that the end of one chunk is repeated at the start of the next. Imagine a sentence that sits exactly on the border between chunk 3 and chunk 4. Without overlap, it is cut. With an overlap of 20 words, the last 20 words of chunk 3 also appear at the start of chunk 4, so at least one of the two chunks holds the sentence whole.

A typical overlap is 10 to 20 percent of the chunk size. More overlap means safer borders but more chunks, more storage and more duplicate results. Here is our chunking function. It moves a window of size words across the text, and each step moves forward by size - overlap words:

// Split text into overlapping chunks of about `size` words.
// Overlap repeats the last few words of one chunk at the start of the next,
// so a sentence cut at the border still appears whole in one of them.
export function chunkWords(text, { size = 80, overlap = 20 } = {}) {
  size = Math.max(10, Math.floor(size));
  overlap = Math.min(Math.max(0, Math.floor(overlap)), size - 1);
  const words = String(text).split(/\s+/).filter(Boolean);
  const out = [];
  for (let start = 0; start < words.length; start += size - overlap) {
    out.push(words.slice(start, start + size).join(' '));
    if (start + size >= words.length) break; // the last chunk reached the end
  }
  return out;
}

Smarter chunkers split on structure first: headings, then paragraphs, then sentences, and only cut inside a sentence as a last choice. They also keep the heading with each chunk so that the chunk knows what it is about. Our lab adds the document title in front of each chunk before embedding for the same reason.

Also keep metadata with every chunk: the source file, the title, the page number, the date. Metadata lets you show sources to the user, filter results (“only the interview guide”) and debug.

What is an embedding?

A computer cannot compare meaning directly. It can compare numbers. An embedding is a way to turn a piece of text into a list of numbers, called a vector, so that texts with similar meaning get similar vectors.

Here is a simple picture. Imagine a big map. On this map, “doctor” and “nurse” are close together, and “banana” is far away. Every piece of text is a point on the map. A real embedding map does not have 2 directions like a paper map. It has hundreds or thousands of directions, so a vector might be 1,024 numbers long. But the idea is the same: close points mean related meaning.

To search, you embed the question too. That puts the question on the same map. Then you look for the chunk points that are nearest to the question point. This is vector search, also called semantic search.

Measuring closeness: cosine similarity

The most common way to measure closeness is cosine similarity. It looks at the angle between two vectors. If they point the same way, the score is close to 1. If they have nothing in common, it is near 0. The code is short. When both vectors are scaled to length 1, cosine similarity is just a dot product: multiply the numbers in pairs and add them up.

// a and b are vectors of the same length, already scaled to length 1
const cosine = (a, b) => a.reduce((sum, x, i) => sum + x * b[i], 0);

Use cosine scores to rank, not as percentages. A score of 0.3 is not “30 percent sure”. What matters is that chunk A scored higher than chunk B, and how far above the noise level the best score is.

A real embedding model versus our teaching embedding

Real embeddings come from a trained model. The model has read huge amounts of text and learned that “car” and “automobile” belong close together. You call an embedding service with your text, and it returns the vector. Anthropic's Claude models generate text, and at the time of writing Anthropic does not sell its own embedding model. Anthropic's documentation points to other providers, for example Voyage AI, and many teams also use open models they run themselves. Check the current documentation and pricing before you choose.

To keep this lab free and key-less, we use a tiny embedding that we wrote ourselves. It hashes words and pairs of words into 1,024 slots. Look at how short it is:

// A tiny LOCAL embedding so the lab works without any key.
// It turns text into a vector of 512 numbers by hashing words and word pairs.
// Texts that share words get similar vectors. It does NOT understand synonyms:
// "car" and "automobile" look unrelated to it. A real embedding model does.
// In a real product, replace embed() with a call to an embedding model.
import { tokenize } from './keyword.js';

export const DIMS = 1024;

function hash(str) {
  let h = 2166136261; // FNV-1a
  for (let i = 0; i < str.length; i++) {
    h ^= str.charCodeAt(i);
    h = Math.imul(h, 16777619);
  }
  return h >>> 0;
}


export function embed(text) {
  const vec = new Float64Array(DIMS);
  const w = tokenize(text); // content words only: no "the" or "of"
  const features = [...w, ...w.slice(1).map((x, i) => `${w[i]}_${x}`)]; // words and word pairs
  for (const f of features) {
    const h = hash(f);
    vec[h % DIMS] += h & 0x100 ? 1 : -1; // a second bit of the hash picks the sign
  }
  for (let i = 0; i < DIMS; i++) vec[i] = Math.sign(vec[i]) * Math.log1p(Math.abs(vec[i]));
  const norm = Math.sqrt(vec.reduce((s, x) => s + x * x, 0)) || 1;
  return Array.from(vec, (x) => Math.round((x / norm) * 1e4) / 1e4); // length ~1, so cosine = dot product; 4 decimals keep files small
}

export const cosine = (a, b) => a.reduce((s, x, i) => s + x * b[i], 0);

Be honest about what this does. It matches shared words and word pairs, so it is smoother than exact keyword matching, but it does not understand synonyms. “Car” and “automobile” look unrelated to it. We use it because it lets you see every part of vector search with no account and no cost. In a real project you replace the function embed() with a call to an embedding model, and everything else in the lab stays the same. That is the reason we keep embed() in its own file.

The search page

The server builds an index: it chunks each document, adds the title, embeds the chunk and keeps the vector. To answer a query it embeds the query, scores every chunk with cosine similarity and returns the best ones. For a few hundred chunks, checking every chunk is fast enough. Vector databases (lesson 15) use clever indexes so that this stays fast with millions.

import { loadDocs } from './load.js';
import { chunkWords } from './chunk.js';
import { embed, cosine } from './embed.js';

const cache = new Map(); // "size:overlap" -> chunks with vectors

// Chunk every document and embed every chunk. Cached per setting.
export function buildIndex({ size = 80, overlap = 20 } = {}) {
  const key = `${size}:${overlap}`;
  if (cache.has(key)) return cache.get(key);
  const chunks = [];
  for (const doc of loadDocs()) {
    chunkWords(doc.text, { size, overlap }).forEach((text, i) => {
      chunks.push({ id: `${doc.source}#${i + 1}`, source: doc.source, title: doc.title, text, vector: embed(`${doc.title}. ${text}`) });
    });
  }
  cache.set(key, chunks);
  return chunks;
}

// Vector search: embed the question, rank chunks by cosine similarity.
export function vectorSearch(query, { size = 80, overlap = 20, k = 4 } = {}) {
  const q = embed(query);
  return buildIndex({ size, overlap })
    .map(({ vector, ...c }) => ({ ...c, score: Number(cosine(q, vector).toFixed(4)) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, k);
}

Notice that nothing in this lesson calls Claude. Retrieval is your own code, and testing it separately is cheap and fast. This is a good habit: test retrieval on its own before you test the whole RAG system. If retrieval is bad, no prompt can save you.

Your task: build the Search page

Add a page where you can change chunk size and overlap and see which chunks come back for a question.

What you will build

• server/rag/chunk.js with the overlapping word chunker.
• server/rag/embed.js with a local embedding and a cosine function.
• server/rag/index.js that builds the index (cached for each size and overlap) and searches it.
• A route POST /api/search that accepts query, size, overlap and k, and keeps all numbers inside safe limits.
• An Angular Search page with two sliders and a results list with similarity bars.

Steps

1. Write the chunker. Test it in the Node console with a text of 200 words, size 50 and overlap 10. You should get chunks that start at word 0, 40, 80 and so on.
2. Write the embedding function. Check that embed("resume") has length 1024 and that the sum of its squares is about 1.
3. Write the index and search. Print the top three chunks for the query “what should go in the skills section for the ATS”.
4. Write the route. Clamp size to between 20 and 300 words, overlap to between 0 and size minus 1, and k to between 1 and 10. Return the number of chunks too.
5. Build the Angular page with two range inputs, a query box and a button. Show each hit with its id, its score and a small bar.
6. Experiment. Try size 20, 80 and 200. Which setting finds the right chunk for each of your test questions? Write down your finding.

Hints

• Cache the index in a Map using a key like "80:20" so a slider move does not re-embed everything each time.
• Convert numbers with Number() and check them with Number.isFinite before you use them.
• Remember to include the title in the text you embed. It helps a chunk that does not repeat its topic.

You know it works when moving the chunk size slider changes the number of chunks and the ranking, a very long chunk size gives few chunks with blurry scores, and a nonsense query gives low scores.

Solution

The files you have just read are the heart of the solution. Here are the route and the Angular page. The zip contains everything together.

import { Router } from 'express';
import { buildIndex, vectorSearch } from '../rag/index.js';

const router = Router();
const num = (v, d, min, max) => Math.min(max, Math.max(min, Number.isFinite(Number(v)) ? Number(v) : d));

// POST /api/search  { query, size, overlap, k }  ->  { chunkCount, hits }
// No Claude call here: this lesson is about retrieval only, so it is free and fast.
router.post('/search', (req, res) => {
  const query = String(req.body?.query || '').trim().slice(0, 300);
  if (!query) return res.status(400).json({ error: { message: 'Please type something to search for.' } });
  const size = num(req.body?.size, 80, 20, 300);
  const overlap = num(req.body?.overlap, 20, 0, size - 1);
  const k = num(req.body?.k, 4, 1, 10);

  res.json({
    chunkCount: buildIndex({ size, overlap }).length,
    hits: vectorSearch(query, { size, overlap, k }),
  });
});

export default router;

import { Component, inject, signal } from '@angular/core';
import { FormsModule } from '@angular/forms';
import { HttpClient } from '@angular/common/http';
import { firstValueFrom } from 'rxjs';
import { errorText } from '../util';

type Hit = { id: string; source: string; text: string; score: number };

@Component({
  selector: 'app-search',
  imports: [FormsModule],
  template: `
    <h1>Chunks and vector search</h1>
    <p class="hint">Change the chunk size and overlap and watch which chunks come back. No Claude call, no cost.</p>

    <div class="card">
      <label for="q">Search for</label>
      <input id="q" [(ngModel)]="query" (keyup.enter)="run()" />
      <div class="grid2">
        <div><label for="s">Chunk size (words): {{ size }}</label><input id="s" type="range" min="20" max="200" step="10" [(ngModel)]="size" /></div>
        <div><label for="o">Overlap (words): {{ overlap }}</label><input id="o" type="range" min="0" max="60" step="5" [(ngModel)]="overlap" /></div>
      </div>
      <button (click)="run()" [disabled]="busy() || !query.trim()">{{ busy() ? 'Searching...' : 'Search' }}</button>
      @if (error()) { <div class="err">{{ error() }}</div> }
    </div>

    @if (hits().length) {
      <div class="card">
        <h2>Top {{ hits().length }} of {{ chunkCount() }} chunks</h2>
        @for (h of hits(); track h.id) {
          <p><strong>{{ h.id }}</strong> <span class="muted">similarity {{ h.score }}</span></p>
          <div class="bar"><span [style.width.%]="percent(h.score)"></span></div>
          <pre class="out">{{ h.text }}</pre>
        }
      </div>
    }
  `,
  styles: [`.bar { height:8px; background:#e2e8f0; border-radius:99px; margin:4px 0 6px; } .bar span { display:block; height:100%; background:#4f46e5; border-radius:99px; }`],
})
export class SearchPage {
  private http = inject(HttpClient);

  query = 'what should go in the skills section for the ATS';
  size = 80;
  overlap = 20;
  busy = signal(false);
  error = signal('');
  hits = signal<Hit[]>([]);
  chunkCount = signal(0);

  percent(score: number) {
    return Math.max(2, Math.min(100, Math.round(score * 100)));
  }

  async run() {
    this.busy.set(true);
    this.error.set('');
    try {
      const r = await firstValueFrom(
        this.http.post<{ chunkCount: number; hits: Hit[] }>('/api/search', { query: this.query, size: Number(this.size), overlap: Number(this.overlap), k: 4 }),
      );
      this.hits.set(r.hits);
      this.chunkCount.set(r.chunkCount);
    } catch (e) {
      this.error.set(errorText(e));
    } finally {
      this.busy.set(false);
    }
  }
}

What you learned

• Chunk size and overlap decide what your search can find. There is no perfect value, so test.
• An embedding turns text into a vector so that similar meanings are close together.
• Cosine similarity measures closeness. Use it to rank, not as a percentage.
• Our local embedding is a teaching tool. Real systems call an embedding model, and always embed questions and chunks with the same model.

In the next lesson you will put these vectors into a store with a clear interface, learn what vector databases do, and see how to move from this lab to PostgreSQL with pgvector, Qdrant or Pinecone.