Chapter 5 · concept 2 of 4 · show all
2Tokens, cost and your first RAG
4Chunking
- 4.1Why we chunk, and the main chunking strategies
- 4.2Structure aware chunking and tables
- 4.3Chunk size, overlap and token based sizing
- 4.4Contextual chunking and late chunking
- 4.5Parent child and multi representation indexing
- 4.6Chunk metadata, provenance and the chunking debugging playbook
- Chapter overview
7Finding the right chunks
8When retrieval fails
What embeddings cannot do
Embeddings blur negation, numbers and codes, and an English model barely links Dutch to English. Measured sentence pairs show where, and what to add.
A real question
“Reset the router” and “Do NOT reset the router”. Does search know the difference?
Hardly. Their vectors scored 0.939: almost the same point. Embeddings capture what a text is about, not every small word that flips its meaning.
The problem
Why this matters
A vector understands the text as well as a reader does.
Negation, prices and error codes barely move the vector. EUR 20.00 against EUR 21.00 scored 0.942; E4012 against E4021 scored 0.970.
Why not the simpler way
After RAG-19 it is tempting to use vector search for everything. Then a user asks about error E4021, gets the E4012 row, and the answer is confidently wrong. Or the system retrieves "Do not reset the router" for "should I reset the router?" and a careless prompt summarizes it as advice to reset. Knowing the blind spots tells you which other tools to add.
The idea
The idea in plain words
An embedding compresses a whole chunk into 384 numbers. Compression keeps the gist and drops detail. The detail it drops first is exactly what makes business text precise: a "not", a digit, a product code, a date, a language.
A photo taken from far away shows that a sign is a speed limit sign. It does not let you read whether it says 30 or 80.
How it works
Step by step
1Small words, big meaning
“not”, “never”, “except” change the meaning completely, but they are one token among many.
- 2
Numbers and codes blur
The model sees “a price” or “an error code”, not which one. Near-identical values get near-identical vectors.
3Add exact matching
Keyword search (BM25) matches the exact code or number. The answer model then reads the real words.
In detail
Negation and qualifiers. Training pairs teach the model that texts on the same topic belong together. "Do" and "do not" sentences are on the same topic, so they stay close.
Numbers and identifiers. Tokenizers split codes like E4012 into pieces, and the model learns little about which pieces matter. Values that differ by one digit end up next to each other.
Language. A model trained mostly on English places Dutch text in a different region. Multilingual models are trained on translated pairs so both languages share one space; e5-small did that here at 0.894. The catch: that model also scored our unrelated pair 0.780, so its scores sit higher across the board.
Long text. Our model reads at most 512 tokens; anything after that is cut off before embedding (RAG-13).
The code
Try it in Python
PAIRS = [
("Reset the router when the power light is solid red.",
"Do not reset the router when the power light is solid red."),
("Error E4012 means mesh pairing failed.", "Error E4021 means mesh pairing failed."),
("Claims must be reported within 30 days.", "Claims must be reported within 30 days, except for theft."),
]
for a, b in PAIRS:
va, vb = model.encode([a, b], normalize_embeddings=True)
print(f"{float(va @ vb):.3f} {a[:40]!r} vs {b[:40]!r}")
# If opposite meanings score close to your real matches, add keyword search and a reranker.
See it
What we measured
Measured · cosine similarity of sentence pairs
Sentence pairs we wrote for this test, embedded as passages. Xenova/bge-small-en-v1.5 and Xenova/multilingual-e5-small, q8, normalized.
Measured 2026-11-04 with corpus/scripts/measure-parts-4-7.mts. Pairs are hand-written, not from the corpus. Compare scores within one model only: e5 scores are higher for everything.
Try it on a real example
Pairs of sentences we wrote, scored with bge-small-en-v1.5
Pair
Do not reset the router when the power light is solid red.
0.939. Opposite advice, almost the same vector. The real manual says “Do not”; only reading the chunk tells you that.
Choosing
Use it when, avoid it when
- Vector search to find the right topic and paraphrases
- A multilingual model when questions and documents differ in language
- Vector search alone for codes, prices and dates
- Trusting a summary of a chunk without the exact words
What it costs · Testing blind spots takes a list of pairs and a few minutes. The guards (BM25, filters, a reranker) are covered in Part 5.
In production
Test your blind spots with a small set of pairs from your own domain: codes, prices, negations, other languages. If a pair scores above your top results for real questions, plan a guard. Typical guards are keyword search fused with vector search (RAG-28), metadata filters for versions and dates (RAG-29), a reranker that reads query and chunk together (RAG-31), and a prompt that tells the model to quote the exact value.
Watch out
Common mistakes
- Higher similarity means the same meaning. It means the same topic; opposite advice scored 0.939.
- Vector search cannot find codes. On our corpus it found all 4 code questions in the top 3. It just cannot tell close codes apart.
- A multilingual model is always better. It linked Dutch well, but its unrelated pair scored 0.780, so its thresholds need retuning.
Check yourself
Remember this
Remember these
- Negation barely changes a vector: 0.939 for opposite advice.
- Prices, dates and codes that differ by one character score above 0.9.
- An English-only model links a Dutch translation weakly: 0.602.
- Pair vector search with keyword search, filters and a model that reads the chunk.
Say it in an interview
- 01Embeddings capture topic, not exact meaning: a negated sentence scored 0.94 with the original.
- 02Codes and prices that differ by a digit score above 0.9, so I add BM25 for exact terms.
- 03An English model linked a Dutch translation at 0.60; a multilingual one at 0.89.
- 04I test blind spots with hand-made pairs from the domain before trusting vector search.
- 05Retrieval finds the neighbourhood; the reranker and the answer model read the exact words.
Quick quiz