Improving Retrieval

Table of Contents

1. Why the First Search Is Not Enough
2. Keyword Search and the Idea of BM25
3. Building Bm25
4. Reciprocal Rank Fusion: hybridSearch
5. Comparing the Three Searches
6. Using Hybrid Search in ask
7. Query Rewriting
8. Reranking
9. Removing Near-Duplicates (MMR-Lite)
10. Metadata Filters
11. Chunk-Size Experiments
12. Parent-Document Retrieval
13. What to Expect With Real Embeddings
14. Recipe: The bm25 and hybrid Modules
15. Try It Yourself
16. Quick Review: Common Questions

1. Why the First Search Is Not Enough

Your pipeline from Retrieve and Generate works, but you saw it stumble:

• "How long can I borrow a laptop?" ranked the right chunk (laptops.md > How long) only fourth. A general FAQ chunk about laptops scored higher.
• "Can I get a scholarship?" scored so low that a sensible minScore would have refused it.
• A paraphrase such as "What is the price to join?" has no word in common with "Membership costs 10 US dollars", so a search that counts words can never find it.

In RAG, retrieval quality sets the ceiling. If the right passage is not in the prompt, no model can use it. This chapter is a toolbox for raising that ceiling. Each tool fixes a specific kind of miss, and each has a cost, so you will learn to measure before you keep any of them. (Measuring properly is chapter 7. Here we use a small check of eight questions, enough to see the idea but not enough to prove anything.)

The plan:

1. Add keyword search (BM25), which is good at exact words.
2. Combine it with vector search, which is good at related meaning.
3. Try smaller tools: query rewriting, reranking, de-duplication, filters, chunk sizes, and parent documents.

Honest note about the mock. The mock's embeddings are bag-of-words vectors: they match shared words, not meaning. That makes them a weak search, and it changes what you will see. Hybrid search helps much less here than it usually does with real embeddings, because the vector side has little "meaning" to add. We will say so where it matters, and section 13 explains what to expect with real models.

Start the mock as before (node mock-llm.js). Everything below runs against it.

2. Keyword Search and the Idea of BM25

Before vectors, search engines worked by words. The classic scoring method that is still in wide use is called BM25 (the name is a historical label; you do not need it). You need the three intuitions behind it, and each one matches common sense.

1. Rare words matter more. Suppose a student asks "Who is the club treasurer?" The word club appears in almost every chunk, so finding it tells you nothing. The word treasurer appears in few chunks, so a chunk that has it is probably the one you want. BM25 gives every word a weight called IDF (inverse document frequency): high for rare words, low for common ones.

2. Repeating a word helps, but less each time. A chunk that mentions "laptop" twice is probably more about laptops than one that mentions it once. But a chunk that says "laptop" fifty times is not fifty times more relevant; it is probably spam. So the benefit saturates.

3. Long chunks are discounted. A long chunk contains many words, so it will match many queries by luck. BM25 compares each chunk's length to the average and lowers the score of the long ones.

Here is the formula, written once so you can see it is no more than those three ideas. For a question, add up the score of each question word w:

score(chunk) = sum over words w in the question of
    idf(w) * f * (k1 + 1) / ( f + k1 * (1 - b + b * len / avgLen) )

idf(w) = ln( 1 + (N - n + 0.5) / (n + 0.5) )

• f is how many times w appears in the chunk.
• len is the chunk length in words, and avgLen is the average.
• N is the number of chunks, and n is the number of chunks that contain w.
• k1 (we use 1.5) controls how fast repeats saturate.
• b (we use 0.75) controls how strongly length is discounted. With b = 0 length is ignored, with b = 1 it is fully applied.

These two settings are common defaults, not magic numbers. Let us look at saturation and the length discount on their own:

// demo-parts.js
// The "term frequency" part of BM25, with b = 0 so length plays no role yet.
const k1 = 1.5;
const tfPart = (f, lenRatio = 1, b = 0.75) => (f * (k1 + 1)) / (f + k1 * (1 - b + b * lenRatio));

console.log("repeats saturate (b = 0):");
for (const f of [1, 2, 3, 5, 10, 50])
  console.log(`  word appears ${String(f).padStart(2)}x -> ${tfPart(f, 1, 0).toFixed(2)}`);

console.log("long chunks are discounted (word appears once):");
for (const ratio of [0.5, 1, 2, 4])
  console.log(`  chunk is ${ratio}x the average length -> ${tfPart(1, ratio).toFixed(2)}`);
// output
// repeats saturate (b = 0):
//   word appears  1x -> 1.00
//   word appears  2x -> 1.43
//   word appears  3x -> 1.67
//   word appears  5x -> 1.92
//   word appears 10x -> 2.17
//   word appears 50x -> 2.43
// long chunks are discounted (word appears once):
//   chunk is 0.5x the average length -> 1.29
//   chunk is 1x the average length -> 1.00
//   chunk is 2x the average length -> 0.69
//   chunk is 4x the average length -> 0.43

Output is stable (pure arithmetic). A word appearing 50 times scores only 2.4 times as much as one appearing once, and a chunk four times the average length scores less than half. That is the whole idea.

3. Building Bm25

We need a tokenizer first: a function that turns text into a list of words. Keyword search is only as good as its tokenizer, so the choices matter:

• Lowercase everything, so "Laptop" and "laptop" match.
• Split on anything that is not a letter, a mark or a digit. (We keep Unicode marks, because Khmer vowel signs are marks, and cutting at them would shred the words.)
• Drop stopwords such as "the" and "is", which carry no meaning.
• Stem lightly: cut a few endings so "borrowed", "borrowing" and "borrow" become one word. Ours is crude on purpose; real search engines use better stemmers or lemmatizers.

Here is the tokenizer on a mixed English and Khmer sample:

// demo-tokens.js
import { tokenize } from "./src/bm25.js";

console.log(tokenize("Borrowed laptops, the club rules!"));
console.log(tokenize("សមាជិកភាព ១០ ដុល្លារ"));
// Khmer is written without spaces between words. One long phrase stays one token:
console.log(tokenize("សមាជិកភាពមានតម្លៃ១០ដុល្លារក្នុងមួយឆមាស").length);
// output
// [ 'borrow', 'laptop', 'club', 'rule' ]
// [ 'សមាជិកភាព', '១០', 'ដុល្លារ' ]
// 1

Output is stable. The English words were lowercased, stemmed and stripped of the stopword. The first Khmer line came out as three tokens, but only because the sample has spaces between phrases. Khmer is normally written without spaces between words, so the last line, a Khmer sentence with no spaces, became exactly one token (the printed 1 is the token count). A whole sentence as one huge "word" will almost never match anything. For real Khmer keyword search you need a word segmenter (a tool that finds word boundaries), or you can index character n-grams (overlapping pieces of a few characters). Both are real topics of their own; the important point is that BM25 as written here is an English-style search. Vector search with a multilingual embedding model has no such problem, which is a good reason to keep both.

Now the class. The complete file is in section 14. It stores each chunk's words, counts in how many chunks each word appears (df), and scores every chunk for each search. For a few thousand chunks this brute-force loop is fast enough; for millions you would use an inverted index (a table from each word to the chunks containing it), which is what real search engines do.

One choice is worth noticing: we index the heading plus the text. The heading, such as "How long", is a strong hint about what a chunk answers, and the vector search from chapter 4 embedded only the chunk text, so it never saw the heading. (Embedding heading + text is a good experiment, see the exercises.) Let us look at the weights and at some searches:

// demo-bm25.js
import { loadDocs } from "./src/load.js";
import { chunkDocs } from "./src/chunk.js";
import { Bm25 } from "./src/bm25.js";

const chunks = chunkDocs(await loadDocs("docs"));
const bm25 = new Bm25(chunks);
console.log("chunks:", chunks.length, " avg length:", bm25.avgLen.toFixed(1), "words");
for (const w of ["club", "fee", "scholarship", "rust"])
  console.log(`idf(${w}) = ${bm25.idf(w).toFixed(2)}  (in ${bm25.df.get(w) ?? 0} chunks)`);
for (const q of ["How long can I borrow a laptop?", "When is the Saturday workshop?", "Does the club teach Rust?"]) {
  console.log(q);
  for (const h of bm25.search(q, 3)) console.log(`  ${h.score.toFixed(2)}  ${h.chunk.source} > ${h.chunk.heading}`);
}
// output
// chunks: 33  avg length: 22.4 words
// idf(club) = 0.56  (in 19 chunks)
// idf(fee) = 2.02  (in 4 chunks)
// idf(scholarship) = 2.61  (in 2 chunks)
// idf(rust) = 4.22  (in 0 chunks)
// How long can I borrow a laptop?
//   7.85  laptops.md > How long
//   5.48  faq.md > Do I need my own laptop?
//   5.44  laptops.md > Who can borrow
// When is the Saturday workshop?
//   4.83  schedule.md > Saturday workshops
//   4.04  laptops.md > How long
//   3.17  mentors.md > Becoming a mentor
// Does the club teach Rust?
//   4.31  faq.md > What programming languages do you teach?
//   2.80  mentors.md > Who our mentors are
//   0.95  about.md > About Mekong Code Club

Output is stable. See how the weights match the intuition: club (in 19 of 33 chunks) has a low weight of 0.56, scholarship (in 2 chunks) has 2.61. For "How long can I borrow a laptop?", the first result is now laptops.md > How long, the chunk the vector search ranked fourth, because it is the only one that has both "long" and "borrow" (stemmed from "borrowed").

The third search shows a trap. "Does the club teach Rust?" has no chunk about Rust, but BM25 still returns faq.md > What programming languages do you teach? because of the word teach. A keyword search returns the best of what exists. It does not know whether that is good enough. (The weight printed for rust is just the formula applied to a word in zero chunks; it never matters, because no chunk contains it.) Same lesson as vector scores in chapter 5: a refusal needs more than "something matched".

4. Reciprocal Rank Fusion: hybridSearch

Now you have two searches. Vector search is good at related wording, BM25 at exact words. How do you combine them?

The obvious idea is to add the scores. It does not work well, because the scores live on different scales. Cosine scores here are between 0 and 1, BM25 scores can be 8 or 20, and the scale changes with the question. You can rescale ("normalise") both to 0..1, but one outlier then squashes everything else.

Reciprocal Rank Fusion (RRF) avoids the problem by ignoring scores altogether and using only rank: first, second, third. Each search gives every chunk it returns 1 / (c + rank) points, and you add up the points from all searches. The constant c (often 60) stops the first place from dominating. For example, a chunk that is 1st in the vector list and 5th in the keyword list gets:

1/(60+1) + 1/(60+5) = 0.01639 + 0.01538 = 0.03177

A chunk that is 2nd in both gets 2 * 1/62 = 0.03226. Being good in both lists beats being great in one, which is the point. Chunks that appear in only one list still get points from it.

We add one knob, alpha, which is the weight of the vector side. With alpha = 0.5 both searches have an equal say. Lower values trust keywords more:

// src/hybrid.js (the heart of it)
const add = (hits, weight, field) =>
  hits.forEach((h, i) => {
    const entry = fused.get(h.chunk.id) ?? { chunk: h.chunk, score: 0, cosine: null, bm25: null };
    entry.score += weight / (c + i + 1);   // i is 0-based, so rank = i + 1
    entry[field] = h.score;                // keep the original score for debugging
    fused.set(h.chunk.id, entry);
  });
add(byVector, alpha, "cosine");
add(byKeyword, 1 - alpha, "bm25");

Each fused hit keeps its original cosine and bm25 scores (or null when that search did not return it). They are useful when you debug, and you will need cosine if you want a refusal threshold. The fused score itself is only good for sorting: it is a rank-based number around 0.01 to 0.03 with no absolute meaning, so never compare it with a minScore made for cosine.

5. Comparing the Three Searches

Time to measure. We use eight questions, each with the one chunk that holds the answer. For every search we ask for 10 results, find the rank of the right chunk, and report:

• hit@1: how often the right chunk is first.
• hit@3: how often it is in the top 3.
• MRR (mean reciprocal rank): the average of 1/rank, which rewards putting the right chunk near the top. Perfect is 1.0.

// demo-compare.js
import { loadDocs } from "./src/load.js";
import { chunkDocs } from "./src/chunk.js";
import { buildIndex } from "./src/indexer.js";
import { embed } from "./src/embed.js";
import { Bm25 } from "./src/bm25.js";
import { hybridSearch } from "./src/hybrid.js";

const questions = [
  { q: "How much does membership cost per term?", source: "membership.md", heading: "Membership fee" },
  { q: "When is the Saturday workshop?", source: "schedule.md", heading: "Saturday workshops" },
  { q: "How long can I borrow a laptop?", source: "laptops.md", heading: "How long" },
  { q: "How many minutes does each team get on Showcase Day?", source: "projects.md", heading: "Showcase Day" },
  { q: "Who do I talk to about a scholarship?", source: "membership.md", heading: "Scholarships" },
  { q: "Where does the club meet?", source: "about.md", heading: "Where we meet" },
  { q: "How many seats are in the room?", source: "about.md", heading: "Where we meet" },
  { q: "What happens if I return a laptop late?", source: "laptops.md", heading: "Late returns" },
];

const store = await buildIndex("docs");
const bm25 = new Bm25(store.items.map((it) => it.chunk));

const searchers = {
  vector: async (q, k) => store.search((await embed([q]))[0], { k }),
  bm25: async (q, k) => bm25.search(q, k),
  "hybrid a=.5": async (q, k) => hybridSearch(q, { store, bm25, k, alpha: 0.5 }),
  "hybrid a=.3": async (q, k) => hybridSearch(q, { store, bm25, k, alpha: 0.3 }),
  "hybrid a=.7": async (q, k) => hybridSearch(q, { store, bm25, k, alpha: 0.7 }),
};

for (const [name, search] of Object.entries(searchers)) {
  let hit1 = 0, hit3 = 0, rr = 0;
  const ranks = [];
  for (const { q, source, heading } of questions) {
    const hits = await search(q, 10);
    const rank = hits.findIndex((h) => h.chunk.source === source && h.chunk.heading === heading) + 1;
    ranks.push(rank || "-");
    if (rank === 1) hit1++;
    if (rank >= 1 && rank <= 3) hit3++;
    if (rank) rr += 1 / rank;
  }
  const n = questions.length;
  console.log(`${name.padEnd(12)} hit@1 ${hit1}/${n}  hit@3 ${hit3}/${n}  MRR ${(rr / n).toFixed(2)}  ranks: ${ranks.join(" ")}`);
}
// output
// vector       hit@1 4/8  hit@3 6/8  MRR 0.65  ranks: 1 3 4 1 1 3 1 4
// bm25         hit@1 8/8  hit@3 8/8  MRR 1.00  ranks: 1 1 1 1 1 1 1 1
// hybrid a=.5  hit@1 4/8  hit@3 8/8  MRR 0.75  ranks: 1 2 2 1 1 2 1 2
// hybrid a=.3  hit@1 7/8  hit@3 8/8  MRR 0.94  ranks: 1 1 2 1 1 1 1 1
// hybrid a=.7  hit@1 4/8  hit@3 8/8  MRR 0.73  ranks: 1 2 3 1 1 2 1 2

The numbers are stable for this corpus and these settings. The last list shows the rank of the right chunk for each question. Read it honestly:

• BM25 won, clearly. It put the right chunk first for all eight questions. The vector search did so for four.
• Hybrid with equal weights (a=.5) was worse than BM25 alone. The weak vector side dragged it down. Giving the vectors less weight (a=.3) recovered most of the loss, and more weight (a=.7) did not help.

Do not conclude "BM25 beats vectors". Two things are going on. First, the mock's vectors are weak, as we said. Second, I wrote these eight questions after reading the documents, so they naturally reuse the documents' words, which is the situation where keyword search is at its best. Real users paraphrase. A fairer test uses questions from real users, and a bigger set, which is exactly what chapter 7 is about. This is a classic trap: a small, friendly test set makes whatever you built look good.

One thing the table does show correctly: the weights are data dependent. There is no best alpha; you find one by measuring your own documents and questions.

6. Using Hybrid Search in ask

Remember the search option we put in ask in chapter 5? Hybrid search plugs into it. We pass minScore: 0 because the fused score is not a cosine, and ask the model the same three questions as before:

// demo-hybrid-ask.js
import { buildIndex } from "./src/indexer.js";
import { Bm25 } from "./src/bm25.js";
import { hybridSearch } from "./src/hybrid.js";
import { ask } from "./src/rag.js";

const store = await buildIndex("docs");
const bm25 = new Bm25(store.items.map((it) => it.chunk));
const search = (q, { k }) => hybridSearch(q, { store, bm25, k, alpha: 0.3 });

for (const q of ["How long can I borrow a laptop?", "Who is the vice president?", "What is the Wi-Fi password?"]) {
  const plain = await ask(q, store);
  const hybrid = await ask(q, store, { search, minScore: 0 }); // RRF scores are tiny, so no cosine floor
  console.log(q);
  console.log("  vector:", plain.answer);
  console.log("  hybrid:", hybrid.answer);
}
const [h] = await search("How long can I borrow a laptop?", { k: 1 });
console.log(h.score.toFixed(4), h.cosine?.toFixed(3), h.bm25?.toFixed(2));
// output
// How long can I borrow a laptop?
//   vector: You can bring your own, or borrow one of the club laptops if you are eligible. [1]
//   hybrid: You can bring your own, or borrow one of the club laptops if you are eligible. [1]
// Who is the vice president?
//   vector: I don't know based on the provided documents.
//   hybrid: I don't know based on the provided documents.
// What is the Wi-Fi password?
//   vector: The fee pays for snacks, printing, and the Wi-Fi bill. [3]
//   hybrid: The room has 24 seats, a projector, and fast Wi-Fi. [1]
// 0.0162 0.480 5.48

Output is stable. Look at the first answer. With hybrid, the right chunk is now in the prompt in the second position (from the table in section 5, rank 2 at a=.3), yet the answer is the same wrong-ish sentence. That is the mock again: it picks the sentence with the most shared words, and "You can bring your own, or borrow one of the club laptops" beats "A laptop can be borrowed for up to 7 days" because the mock does not know "borrowed" means "borrow". A real model reads the passage and would answer "up to 7 days". This is a good example of why you must measure retrieval and answers separately, as chapter 7 does: retrieval improved, the (mock) generation did not.

The last question is more interesting. "What is the Wi-Fi password?" has no answer in the documents. The vector-only pipeline answered with a sentence about the Wi-Fi bill, from the membership page. The hybrid pipeline answered with "fast Wi-Fi" from the room description. Both are wrong, because the question is about a password, which is not in any document. The mock does not understand that; it matches the word "Wi-Fi". With a real model and rule 3 of the system prompt you would hope for a refusal. This is why chapter 7 tests unanswerable questions.

Finally, notice the line after the answers: 0.0162 0.480 5.48. It is the top hybrid hit: the fused score (about 0.016, a rank-based number), its original cosine (0.480) and its BM25 score (5.48). If you want a refusal threshold on hybrid results, test it on cosine or bm25, never on the fused score.

7. Query Rewriting

Sometimes the question is simply not the best search text. Some examples:

• It uses different words from the documents ("price" instead of "fee").
• It is vague, short, or part of a conversation (chapter 5, section 10).
• It asks several things at once.

Query rewriting fixes the question before searching. A real system asks a model: "Rewrite this question to be a good search query. Add likely synonyms. Reply with the query only." Variants include query expansion (add related words) and multi-query (generate three rewrites, search each, and merge the results with RRF, the same trick as before).

The mock cannot rewrite, so we use a stub: a tiny hand-made synonym table. The shape of the code is exactly what you would write with a real model, just with the expand function replaced by a model call:

// demo-expand.js
import { buildIndex } from "./src/indexer.js";
import { embed } from "./src/embed.js";
import { Bm25 } from "./src/bm25.js";
import { hybridSearch } from "./src/hybrid.js";

// A stub for "ask a model to rewrite the question". A real rewrite step
// would send the question to an LLM; here a tiny synonym table plays that role.
const SYNONYMS = { price: "fee cost", charge: "fee", money: "fee", computer: "laptop", keep: "borrow" };
export const expand = (q) =>
  q.replace(/[?.!]/g, "").split(/\s+/).flatMap((w) => [w, ...(SYNONYMS[w.toLowerCase()]?.split(" ") ?? [])]).join(" ");

const store = await buildIndex("docs");
const bm25 = new Bm25(store.items.map((it) => it.chunk));

for (const q of ["What is the price to join?", "How many days can I keep a computer?"]) {
  console.log(q, "->", expand(q));
  for (const query of [q, expand(q)]) {
    const hits = await hybridSearch(query, { store, bm25, k: 2 });
    console.log("  ", query === q ? "plain   " : "expanded", hits.map((h) => `${h.chunk.source} > ${h.chunk.heading}`).join(" | "));
  }
}
// output
// What is the price to join? -> What is the price fee cost to join
//    plain    about.md > Who can join | faq.md > How do I contact the club?
//    expanded membership.md > Membership fee | about.md > Who can join
// How many days can I keep a computer? -> How many days can I keep borrow a computer laptop
//    plain    mentors.md > Who our mentors are | faq.md > Can my parents visit?
//    expanded laptops.md > How long | faq.md > Do I need my own laptop?

Output is stable. Without the rewrite, "What is the price to join?" found only about.md > Who can join (it shares the word join), and the fee chunk was nowhere. With "fee" and "cost" added, the fee chunk was first. The second question went from two unrelated chunks to the laptop-loan chunks.

The honest costs:

• It adds a model call and delay before every search.
• The rewrite can go wrong (drop a key detail, or add words that point to the wrong topic), and then the bad query hides the right chunk. Log both the original and the rewrite.
• A real embedding model already handles synonyms like price and fee, so rewriting helps keyword search more than vector search. It helps most with vague follow-up questions and with jargon your users know but your documents spell differently.

8. Reranking

Search needs to look at thousands of chunks, so it uses cheap scoring. A reranker looks at only the few best candidates and scores each one more carefully against the question. The standard recipe has two steps:

1. Retrieve wide: get the top 20 or so candidates with cheap search.
2. Rerank narrow: score each candidate with a more expensive method and keep the top k (3 to 5) for the prompt.

Real rerankers are often a cross-encoder: a model that reads the question and one chunk together and outputs a relevance score. This is more accurate than comparing two separate embeddings, but slower, so you only run it on the candidates. Another option is to ask a general LLM to judge or order the candidates ("LLM rerank"), which costs tokens and time on every question. Both are valid. Neither is free, and neither is available on the mock.

For a demo we use a very cheap reranker: keyword coverage, the fraction of the question's words that appear in the chunk. It is crude (it ignores word importance, unlike BM25), but it shows the two-step pattern:

// demo-rerank.js
import { buildIndex } from "./src/indexer.js";
import { embed } from "./src/embed.js";
import { tokenize } from "./src/bm25.js";

// Cheap rerank: what fraction of the question's words does the chunk contain?
function rerank(question, hits) {
  const q = new Set(tokenize(question));
  return hits
    .map((h) => {
      const words = new Set(tokenize(`${h.chunk.heading} ${h.chunk.text}`));
      const covered = [...q].filter((w) => words.has(w)).length;
      return { ...h, coverage: covered / q.size };
    })
    .sort((a, b) => b.coverage - a.coverage || b.score - a.score);
}

const store = await buildIndex("docs");
const question = "How long can I borrow a laptop?";
const [v] = await embed([question]);
const candidates = store.search(v, { k: 6 });          // wide net first
console.log("before:");
candidates.forEach((h, i) => console.log(`  ${i + 1}. ${h.score.toFixed(3)}  ${h.chunk.source} > ${h.chunk.heading}`));
console.log("after rerank, keep 3:");
rerank(question, candidates).slice(0, 3)
  .forEach((h, i) => console.log(`  ${i + 1}. coverage ${h.coverage.toFixed(2)}  ${h.chunk.source} > ${h.chunk.heading}`));
// output
// before:
//   1. 0.480  faq.md > Do I need my own laptop?
//   2. 0.293  laptops.md > Who can borrow
//   3. 0.274  laptops.md > Rules
//   4. 0.241  laptops.md > How long
//   5. 0.144  laptops.md > Late returns
//   6. 0.120  faq.md > How do I contact the club?
// after rerank, keep 3:
//   1. coverage 1.00  laptops.md > How long
//   2. coverage 0.67  faq.md > Do I need my own laptop?
//   3. coverage 0.67  laptops.md > Who can borrow

Output is stable. Wide retrieval brought the right chunk, `laptops.md

How long`, into the candidates at position 4. Reranking by coverage moved it to first, because it is the only candidate containing all three question words (long, borrow, laptop). The pattern is the useful part: a wide net, then a careful sort, then a small prompt.

When should you add a real reranker? When your evaluation shows the right chunk is in the top 20 but not in the top 4. If it is not in the top 20, a reranker cannot help, and you should fix the first search.

9. Removing Near-Duplicates (MMR-Lite)

Real document folders contain repeats: an old version of a policy, the same paragraph in two files, a FAQ that copies the handbook. If your top 4 hits are four copies of one fact, you waste the prompt and hide other useful chunks.

MMR (maximal marginal relevance) is the standard fix. Pick the best hit first. Then, for each remaining spot, pick the candidate that has a high score and is least similar to what you already picked. Our "lite" version measures similarity as the overlap of words between two chunks. To try it, make a small folder with a duplicated policy:

mkdir docs-dup
cp docs/membership.md docs/laptops.md docs-dup/
sed 's/10 US dollars/ten US dollars/' docs/membership.md > docs-dup/membership-2023.md

membership-2023.md is a stale copy of the fee page with one word changed. Now compare a plain top 3 with the MMR-lite top 3:

// demo-mmr.js
import { buildIndex } from "./src/indexer.js";
import { embed } from "./src/embed.js";
import { tokenize } from "./src/bm25.js";

const words = (c) => new Set(tokenize(c.text));
const jaccard = (a, b) => [...a].filter((w) => b.has(w)).length / new Set([...a, ...b]).size;

// MMR-lite: pick the best hit, then keep picking the hit that scores well
// AND is least like what we already picked. lambda=1 means "ignore variety".
function mmr(hits, { k = 3, lambda = 0.6 } = {}) {
  const picked = [];
  const pool = [...hits];
  while (picked.length < k && pool.length) {
    let best = null, bestValue = -Infinity;
    for (const h of pool) {
      const sim = Math.max(0, ...picked.map((p) => jaccard(words(h.chunk), words(p.chunk))));
      const value = lambda * h.score - (1 - lambda) * sim;
      if (value > bestValue) { best = h; bestValue = value; }
    }
    picked.push(best);
    pool.splice(pool.indexOf(best), 1);
  }
  return picked;
}

const store = await buildIndex("docs-dup");
const [v] = await embed(["How much does membership cost per term?"]);
const hits = store.search(v, { k: 6 });
const show = (label, list) => {
  console.log(label);
  list.forEach((h) => console.log(`  ${h.score.toFixed(3)}  ${h.chunk.id}  ${h.chunk.heading}`));
};
show("plain top 3:", hits.slice(0, 3));
show("MMR-lite top 3:", mmr(hits));
// output
// plain top 3:
//   0.435  membership-2023.md#1  Membership fee
//   0.435  membership.md#1  Membership fee
//   0.217  membership-2023.md#4  Refunds
// MMR-lite top 3:
//   0.435  membership-2023.md#1  Membership fee
//   0.217  membership-2023.md#4  Refunds
//   0.143  membership-2023.md#2  Scholarships

Output is stable. In the plain list, the fee chunk appeared twice (the current and the old file, both scoring 0.435). With MMR-lite, the duplicate was dropped and a different chunk took its place.

Two warnings. First, de-duplicating is not the same as fixing the cause: you still have two conflicting versions of a policy in your index, and if they disagree (say, 10 dollars and 8 dollars) the model may quote the old one. Delete or mark stale documents at the source (chapter 8 covers keeping the index fresh). Second, lambda trades relevance against variety, and a low value can push out a chunk that was actually needed. Test it.

10. Metadata Filters

Each chunk carries source and heading (and later you may add date, language, team, permission). The store's search method accepts a filter function, so you can restrict a search before ranking:

// demo-filter.js
import { buildIndex } from "./src/indexer.js";
import { embed } from "./src/embed.js";

const store = await buildIndex("docs");
const question = "How long can I borrow a laptop?";
const [v] = await embed([question]);

const show = (label, hits) => {
  console.log(label);
  hits.forEach((h) => console.log(`  ${h.score.toFixed(3)}  ${h.chunk.source} > ${h.chunk.heading}`));
};
show("no filter:", store.search(v, { k: 3 }));
show("only laptops.md:", store.search(v, { k: 3, filter: (c) => c.source === "laptops.md" }));
// output
// no filter:
//   0.480  faq.md > Do I need my own laptop?
//   0.293  laptops.md > Who can borrow
//   0.274  laptops.md > Rules
// only laptops.md:
//   0.293  laptops.md > Who can borrow
//   0.274  laptops.md > Rules
//   0.241  laptops.md > How long

Output is stable. With a filter on laptops.md, the unrelated FAQ chunk disappears and laptops.md > How long enters the top 3.

Where does the filter value come from?

• From the user's context: the page they are on, the product they chose, their language, their team. This is the safest and most useful source.
• From parsing the question (rules, or a model that outputs a filter). This is powerful and easy to get wrong: a wrong filter silently hides the right answer.

Filters can also be about permissions, such as "only documents this user may read". That is a different and more serious use: the filter must run before any text reaches the model, and chapter 8 covers it.

11. Chunk-Size Experiments

In chapter 2 you chose a chunking strategy. Does the choice matter for retrieval? Try a small experiment: build the index with four settings and see whether the top 3 hits for each question contain the right file and the key fact. We also report how much text goes into the prompt:

// demo-sizes.js
import { buildIndex } from "./src/indexer.js";
import { embed } from "./src/embed.js";

// An answer is "found" when a top-3 chunk comes from the right file and
// contains the key fact. (Chapter 7 builds a proper evaluation set.)
const questions = [
  { q: "How much does membership cost per term?", source: "membership.md", fact: "10 US dollars" },
  { q: "When is the Saturday workshop?", source: "schedule.md", fact: "9:00 to 12:00" },
  { q: "How long can I borrow a laptop?", source: "laptops.md", fact: "up to 7 days" },
  { q: "How many minutes does each team get on Showcase Day?", source: "projects.md", fact: "7 minutes" },
  { q: "Who do I talk to about a scholarship?", source: "membership.md", fact: "Dara" },
  { q: "How many seats are in the room?", source: "about.md", fact: "24 seats" },
  { q: "What happens if I return a laptop late?", source: "laptops.md", fact: "loses its borrowing" },
  { q: "Who is the club president?", source: "conduct.md", fact: "Sophea" },
];

const strategies = {
  "heading            ": { strategy: "heading" },
  "size 200, overlap 20": { strategy: "size", size: 200, overlap: 20 },
  "size 400, overlap 40": { strategy: "size", size: 400, overlap: 40 },
  "size 800, overlap 80": { strategy: "size", size: 800, overlap: 80 },
};

for (const [name, opts] of Object.entries(strategies)) {
  const store = await buildIndex("docs", opts);
  const avg = Math.round(store.items.reduce((n, it) => n + it.chunk.text.length, 0) / store.size);
  let found = 0, sent = 0;
  const missed = [];
  for (const { q, source, fact } of questions) {
    const [v] = await embed([q]);
    const hits = store.search(v, { k: 3 });
    sent += hits.reduce((n, h) => n + h.chunk.text.length, 0);
    const ok = hits.some((h) => h.chunk.source === source && h.chunk.text.replace(/\s+/g, " ").includes(fact));
    if (ok) found++; else missed.push(fact);
  }
  console.log(`${name}  chunks ${String(store.size).padStart(3)}  avg ${avg} chars  found ${found}/${questions.length}  sent ~${Math.round(sent / questions.length)} chars/question${missed.length ? `  missed: ${missed.join(", ")}` : ""}`);
}
// output
// heading              chunks  33  avg 186 chars  found 6/8  sent ~551 chars/question  missed: up to 7 days, loses its borrowing
// size 200, overlap 20  chunks  41  avg 188 chars  found 8/8  sent ~562 chars/question
// size 400, overlap 40  chunks  24  avg 320 chars  found 8/8  sent ~989 chars/question
// size 800, overlap 80  chunks  16  avg 481 chars  found 8/8  sent ~1603 chars/question

Output is stable for this corpus. Heading chunks missed two questions. The size-based chunks found all eight, though the larger chunks sent roughly two to three times more text to the prompt. Do not conclude "size chunks are better" from this. The mock's vectors are crude, the corpus is tiny (the whole handbook is only 33 heading chunks), and there are only eight questions, so a single lucky ordering changes the score. What the experiment does teach:

• Chunking is a parameter to tune, not a fixed fact.
• Bigger chunks cost more per question (look at the last column).
• You must test with questions and an honest count, not by looking at one example.

Chapter 7 builds a proper evaluation set and compares these strategies on more questions and in more ways.

12. Parent-Document Retrieval

Small chunks are easier to match (a sentence has one idea). Big chunks are better to read (the model gets the surrounding context). Parent-document retrieval gives you both:

1. Split documents into big parents (our heading sections).
2. Split each parent into small children (here: sentences).
3. Embed and search the children.
4. When a child matches, send its parent to the model.

If three children match from the same parent, you send that parent once.

// demo-parent.js
import { loadDocs } from "./src/load.js";
import { chunkDocs } from "./src/chunk.js";
import { embed } from "./src/embed.js";
import { VectorStore } from "./src/store.js";

// Parents: the heading sections we send to the model.
const parents = chunkDocs(await loadDocs("docs"));
// Children: one sentence each. Small pieces are easier to match exactly.
const children = parents.flatMap((p) =>
  p.text.replace(/\s+/g, " ").split(/(?<=[.!?])\s+/).filter(Boolean)
    .map((text, i) => ({ id: `${p.id}/s${i + 1}`, source: p.source, heading: p.heading, text, parentId: p.id })));

const store = new VectorStore();
store.add(children, await embed(children.map((c) => c.text)));
const byId = new Map(parents.map((p) => [p.id, p]));

async function parentSearch(question, k = 2) {
  const [v] = await embed([question]);
  const out = [];
  for (const hit of store.search(v, { k: k * 4 })) {      // look at many children...
    if (out.some((h) => h.chunk.id === hit.chunk.parentId)) continue; // ...one hit per parent
    out.push({ chunk: byId.get(hit.chunk.parentId), score: hit.score, matched: hit.chunk.text });
    if (out.length === k) break;
  }
  return out;
}

for (const h of await parentSearch("What happens if I install games on a laptop?")) {
  console.log(`${h.score.toFixed(3)}  ${h.chunk.source} > ${h.chunk.heading}`);
  console.log(`   matched sentence: ${h.matched}`);
  console.log(`   sent to model   : ${h.chunk.text.replace(/\s+/g, " ").slice(0, 70)}... (${h.chunk.text.length} chars)`);
}
// output
// 0.612  laptops.md > Rules
//    matched sentence: Do not install games or pirated software on a club laptop.
//    sent to model   : Do not install games or pirated software on a club laptop. Do not chan... (295 chars)
// 0.302  laptops.md > Who can borrow
//    matched sentence: The club owns eight laptops that paid members and scholarship members can borrow.
//    sent to model   : Some students do not own a laptop. The club owns eight laptops that pa... (168 chars)

Output is stable. The sentence "Do not install games or pirated software on a club laptop." matched, and the model gets the whole Rules section, including the sentences about the administrator password and about what happens when a laptop is broken. The matching sentence alone would not have told the student what the rules say about repairs.

The trade-offs: you embed more vectors (one per sentence), the parent can be long, and a bad sentence match fetches a whole irrelevant parent. In this corpus, where sections are already short, the benefit is small. It grows when your documents have long sections, like manuals and contracts.

13. What to Expect With Real Embeddings

Be careful with the results above. They come from a toy search, and every conclusion about which method wins applies to the toy. Here is what to expect with a real embedding model, stated in general terms. We make no numeric claims, because the numbers depend on your documents, your questions and your model.

• Vectors will be much better at paraphrase. "What is the price to join?" would find the fee chunk without any rewrite, and so might the same question in another language, if the model is multilingual.
• Keyword search will still win on exact strings: names such as "Dara", error codes, product numbers, rare acronyms and quoted phrases. Embeddings tend to blur these, and a typo or an unusual word may map to something vaguely related.
• So hybrid usually helps, because each side covers the other's weak spot. It is a common default for production RAG. But "usually" is not "always". Measure on your own data, as in section 5, with real user questions.
• The gaps will be smaller than in this chapter. Here a weak vector search made hybrid and BM25 look dramatically different. With a good embedding model the vector search stands on its own, and hybrid adds a smaller gain.
• Rerankers and rewriting are optional extras. Add them one at a time and keep each only if the evaluation improves enough to justify the added cost and delay.

A useful habit: change one thing at a time, measure, and write down the numbers. When a change helps by a hair on eight questions, it is probably noise.

14. Recipe: The bm25 and hybrid Modules

Here is the full final src/bm25.js. It has no dependencies:

const STOPWORDS = new Set(
  "a an the is are was were be to of in on at for and or but with by from as it its this that these those i you we they he she what when where who which how do does did can could should would will my your our their me us about into than then there here not no if so also any all some more most".split(" "),
);

// Lowercase, split on anything that is not a letter, mark or digit, drop
// stopwords, and trim a few endings so "borrowed" and "borrow" match.
// (Marks \p{M} are kept so scripts such as Khmer are not cut in pieces.)
const ENDINGS = ["ing", "ed", "s"];
function stem(w) {
  for (const end of ENDINGS) {
    if (end === "s" && w.endsWith("ss")) continue;
    if (w.endsWith(end) && w.length - end.length >= 3) return w.slice(0, -end.length);
  }
  return w;
}

export function tokenize(text) {
  return text
    .toLowerCase()
    .split(/[^\p{L}\p{M}\p{N}]+/u)
    .filter((w) => w && !STOPWORDS.has(w))
    .map(stem);
}

export class Bm25 {
  constructor(chunks, { k1 = 1.5, b = 0.75 } = {}) {
    this.chunks = chunks;
    this.k1 = k1;
    this.b = b;
    // The heading is part of what a chunk is about, so index it too.
    this.docs = chunks.map((c) => tokenize(`${c.heading} ${c.text}`));
    this.avgLen = this.docs.reduce((n, d) => n + d.length, 0) / (this.docs.length || 1);
    this.df = new Map(); // word -> number of chunks containing it
    for (const d of this.docs) for (const w of new Set(d)) this.df.set(w, (this.df.get(w) ?? 0) + 1);
  }

  idf(word) {
    const n = this.df.get(word) ?? 0;
    return Math.log(1 + (this.docs.length - n + 0.5) / (n + 0.5));
  }

  search(query, k = 4) {
    const terms = [...new Set(tokenize(query))];
    const results = [];
    this.docs.forEach((doc, i) => {
      const tf = new Map();
      for (const w of doc) tf.set(w, (tf.get(w) ?? 0) + 1);
      let score = 0;
      for (const w of terms) {
        const f = tf.get(w);
        if (!f) continue;
        const norm = 1 - this.b + (this.b * doc.length) / this.avgLen;
        score += this.idf(w) * ((f * (this.k1 + 1)) / (f + this.k1 * norm));
      }
      if (score > 0) results.push({ chunk: this.chunks[i], score });
    });
    return results.sort((a, b) => b.score - a.score).slice(0, k);
  }
}

And the full final src/hybrid.js. It imports embed from the embedding module:

import { embed } from "./embed.js";

// Reciprocal Rank Fusion: each list gives a chunk 1 / (c + rank) points.
// alpha is the weight of the vector list (0.5 = equal say).
export async function hybridSearch(question, { store, bm25, k = 4, alpha = 0.5, c = 60, pool = 20 } = {}) {
  const [vector] = await embed([question]);
  const byVector = store.search(vector, { k: pool });
  const byKeyword = bm25.search(question, pool);

  const fused = new Map(); // chunk id -> { chunk, score, cosine, bm25 }
  const add = (hits, weight, field) =>
    hits.forEach((h, i) => {
      const entry = fused.get(h.chunk.id) ?? { chunk: h.chunk, score: 0, cosine: null, bm25: null };
      entry.score += weight / (c + i + 1);
      entry[field] = h.score;
      fused.set(h.chunk.id, entry);
    });
  add(byVector, alpha, "cosine");
  add(byKeyword, 1 - alpha, "bm25");
  return [...fused.values()].sort((a, b) => b.score - a.score).slice(0, k);
}

Options for hybridSearch: store and bm25 (required), k (default 4), alpha (vector weight, default 0.5), c (RRF constant, default 60) and pool (how many results to take from each search before fusing, default 20). The ideas in sections 7 to 12 (rewriting, reranking, MMR, filters, parents) are small demos, not modules. When you keep one, move it into its own file.

15. Try It Yourself

1. Tune alpha. Run demo-compare.js with alpha from 0.1 to 0.9 in steps of 0.1. Plot or list MRR. Is the best value near either end? What would change if the questions were paraphrases?
2. Write paraphrased questions. Write eight questions about the club in words that do not appear in the documents (for example "What do I pay to join?"). Run all three searches. Which one now wins, and why?
3. Tune the BM25 constants. Change k1 to 0.5 and then 3, and b to 0 and then 1. Which questions change rank? Which constants make a long chunk rank higher?
4. Rerank hybrid. Apply the coverage reranker from section 8 to the top 8 hybrid results and keep 3. Does hit@1 on the eight questions go up?
5. Embed the heading too. In indexer.js, embed ${c.heading}. ${c.text} instead of c.text. Re-run demo-compare.js. Does the vector search improve? What would you check before trusting the result?
6. Add a filter from the question. Write a function that picks a source filter when the question contains "laptop" or "laptops" and no filter otherwise. Find one question where this filter helps and one where it hurts.

16. Quick Review: Common Questions

Questions beginners often ask about this chapter. Try answering each one yourself first, then click the question to open the answer.

Where to go next. You now have three searches (vector, keyword and hybrid) and a toolbox of extras, but your only evidence is eight questions that you wrote yourself. In the next chapter, Evaluating a RAG System, you build a proper evaluation set, measure retrieval and answers separately, name the ways a RAG system fails, and put the checks into your tests.