Chapter 3 · concept 6 of 6 · show all
2Tokens, cost and your first RAG
4Chunking
- 4.1Why we chunk, and the main chunking strategies
- 4.2Structure aware chunking and tables
- 4.3Chunk size, overlap and token based sizing
- 4.4Contextual chunking and late chunking
- 4.5Parent child and multi representation indexing
- 4.6Chunk metadata, provenance and the chunking debugging playbook
- Chapter overview
7Finding the right chunks
8When retrieval fails
Updates, deletes, reindexing and ingestion pipelines
How to keep an index in step with changing documents, when to patch and when to rebuild, and how forgotten deletes leave stale answers behind.
A real question
We updated the tariff sheet. Why can the assistant still find the old plan?
Because the new chunks were added but the old ones were never deleted. Keeping an index in step with changing documents is a pipeline, not a one-off script.
The problem
Why this matters
When a document changes, index the new file and you are done.
The old chunks stay unless you delete them. They are called orphans, and search can still return them as if they were current.
Why not the simpler way
The simple pipeline is a script that indexes a folder once. Then documents change. If you rerun it, you index everything again and pay for it. If you add only the new files, the old versions stay. If a document is deleted at the source, nothing tells the index. Within weeks the assistant quotes prices and rules that are no longer true.
The idea
The idea in plain words
An ingestion pipeline keeps the index in step with the sources. For every source change it decides: add (new document), update (changed document), or delete (gone). An update can replace all of a document's chunks or patch only the chunks whose text changed. A full reindex rebuilds everything, which you need when the chunker or the embedding model changes.
It is a shop's stock list. A new price label replaces the old one on the shelf. Leaving the old label next to it is how customers get the wrong price.
How it works
Step by step
1Detect what changed
Compare a fingerprint (hash) of each document with the one you indexed last time.
2Compare old and new chunks
Some chunks are unchanged, some are new, some are gone.
3Add, and delete by document id
Embed the new chunks, delete the gone ones, and record the new version.
In detail
- Detect changes: compare content hashes or modified times with what was indexed last time.
- Chunk the changed document with the same settings as before.
- Diff old and new chunks by content hash: unchanged, added, removed.
- Write: embed and insert added chunks, delete removed ones (patch), or delete all and insert all (replace).
- Record the new version and hash, so the next run knows the state.
Patching saves embedding calls only when most chunks stay the same. Small documents, and any chunker whose boundaries shift when text near the top changes, make almost every chunk new.
The code
Try it in Python
import hashlib
def h(text: str) -> str:
return hashlib.sha256(text.encode()).hexdigest()
def plan_update(old_chunks: list[str], new_chunks: list[str]) -> dict:
old, new = {h(c) for c in old_chunks}, {h(c) for c in new_chunks}
return {
"embed": [c for c in new_chunks if h(c) not in old], # added or changed
"delete": [c for c in old_chunks if h(c) not in new], # gone: must be removed
"keep": len(old & new),
}
See it
What we measured
Measured · chunks per update, tariff 2025.1 to 2025.2
SYN-TEL-TARIFF, versions 2025.1 and 2025.2 (title and one mobile plan changed), chunked live in the Update Lab. Chunks matched by exact text.
Measured 2026-11-03. The orphans include the old 20 GB plan, which search can still return as if it were for sale. On these short documents patching saves little; it pays off on long documents with stable sections.
Try it on a real example
Tariff sheet 2025.1 is replaced by 2025.2. One plan changed: 20 GB became 25 GB.
Index after the update
Only the current sheet is searchable. Questions about the 20 GB plan get the right answer: no longer sold to new customers.
Choosing
Use it when, avoid it when
- Change detection by content hash and delete by document id
- Patch updates for long documents with stable sections
- A full reindex into a new index when the chunker or model changes
- Adding new versions without removing old chunks
- Rebuilding the whole index for every small change
What it costs · Patching needs per-chunk hashes and a diff step; replacing re-embeds the whole document. A full reindex re-embeds everything and should be rare.
In production
Delete by document id, never by guessing which chunks belonged to a file. Treat a source deletion as a real event: tombstone the document and remove its chunks. Run a full reindex into a new index when the chunker or embedding model changes, check it against the golden set, then switch over, so users never see a half-built index. Alert when an ingestion run fails; a stale index fails quietly.
Watch out
Common mistakes
- Patching is always cheaper. Here it saved nothing for recursive chunks on a short sheet.
- Deleting is optional. Forgotten deletes are the most common source of stale answers.
- Reindexing is only for disasters. It is the normal step after changing the chunker or the embedding model.
Check yourself
Remember this
Remember these
- Detect changes with a content hash; delete chunks by document id.
- Forgotten deletes leave orphans that make answers stale.
- Patching only changed chunks pays off on long documents with stable sections.
- A new chunker or embedding model means a full reindex into a new index.
Say it in an interview
- 01An ingestion pipeline turns source changes into adds, updates and deletes.
- 02I detect changes by content hash and always delete by document id.
- 03On my tariff update, recursive chunks kept 0 of 2 and heading chunks kept 2 of 4.
- 04Forgotten deletes left the old 20 GB plan searchable as if it were still for sale.
- 05Chunker or model changes mean a full reindex into a new index, checked before switching.
Quick quiz