RAGEngineering Hub
Menu
Chapter 9 · concept 4 of 6 · show all
  1. 1What RAG is, and why
    1. 1.1Why models need outside knowledge, and what hallucination means
    2. 1.2What RAG is: anatomy of a request
    3. 1.3RAG vs fine tuning vs long context and prompt caching
    4. 1.4Use cases and the quality, latency and cost triangle
    5. Chapter overview
  2. 2Tokens, cost and your first RAG
    1. 2.1Tokens and context windows for RAG
    2. 2.2The RAG lifecycle and your first RAG in miniature
    3. Chapter overview
  3. 3Getting documents in
    1. 3.1Content sources and file formats
    2. 3.2Parsing, layout and OCR
    3. 3.3Tables, images and mixed content
    4. 3.4Cleaning, normalization and quality gates
    5. 3.5Metadata, document identity and versioning
    6. 3.6Updates, deletes, reindexing and ingestion pipelines
    7. Chapter overview
  4. 4Chunking
    1. 4.1Why we chunk, and the main chunking strategies
    2. 4.2Structure aware chunking and tables
    3. 4.3Chunk size, overlap and token based sizing
    4. 4.4Contextual chunking and late chunking
    5. 4.5Parent child and multi representation indexing
    6. 4.6Chunk metadata, provenance and the chunking debugging playbook
    7. Chapter overview
  5. 5Embeddings
    1. 5.1Embeddings and similarity from first principles
    2. 5.2What embeddings cannot do
    3. 5.3Choosing an embedding model: size, cost, language
    4. 5.4Embedding model migration and index rebuilds
    5. Chapter overview
  6. 6Vector stores and scores
    1. 6.1Vector databases, ANN indexes and choosing a store
    2. 6.2Score calibration and thresholds
    3. 6.3Late interaction and multivector retrieval
    4. Chapter overview
  7. 7Finding the right chunks
    1. 7.1Query understanding, rewriting and expansion
    2. 7.2Dense retrieval and keyword search with BM25
    3. 7.3Hybrid retrieval and rank fusion
    4. 7.4topK, no good match, and metadata filtering
    5. 7.5Routing, diversity, deduplication and caching
    6. 7.6Reranking: two stage, cross encoder and LLM rerankers
    7. Chapter overview
  8. 8When retrieval fails
    1. 8.1Retrieval failure analysis playbook
    2. Chapter overview
  9. 9Building the answer
    1. 9.1Context selection, packing and citations
    2. 9.2Grounding prompts and abstention
    3. 9.3Lost in the middle and context management
    4. 9.4Conflicts, staleness and freshness rules
    5. 9.5Hallucination checks and answer verification
    6. 9.6Conversational RAG, answer UX and RAG APIs
    7. Chapter overview
  10. 10Measuring
    1. 10.1Why measure, and what to measure
    2. 10.2Building a golden dataset
    3. 10.3Retrieval metrics: precision, recall, MRR, nDCG
    4. 10.4Faithfulness and answer correctness
    5. 10.5LLM as judge, and judging the judge
    6. 10.6Offline and online evals, regression tests and slices
    7. Chapter overview
  11. 11Safety and operations
    1. 11.1Prompt injection and poisoned documents
    2. 11.2Access control, privacy and governance
    3. 11.3Index lifecycle operations
    4. 11.4Observability and tracing
    5. 11.5Latency, cost, caching and model routing
    6. 11.6Reliability, degraded modes and launch readiness
    7. Chapter overview
  12. 12Scale and advanced RAG
    1. 12.1Scaling, multi tenancy and enterprise architecture
    2. 12.2Choosing an architecture: naive, advanced, agentic, graph
    3. 12.3Agentic RAG and RAG as an MCP tool
    4. 12.4GraphRAG, structured data and multimodal RAG
    5. Chapter overview

Conflicts, staleness and freshness rules

When two sources disagree, or an old version sits next to the current one, the model needs rules. On our corpus 5 of 8 version questions had an outdated version in the top 3.

A real question

One document says 10 working days, another says 30 days. Which should the answer use?

Neither by luck. You decide the rules: valid version only, more authoritative source first, and when a conflict matters, say so in the answer. The model follows rules; it cannot invent them.

The problem

Why this matters

The model will work out which source is right.
The belief

The model will work out which source is right.

Old and new versions read alike, so both get retrieved. For 5 of our 8 version questions, another version of the same document was in the top 3. Without dates or rules, the model guesses.
What really happens

Old and new versions read alike, so both get retrieved. For 5 of our 8 version questions, another version of the same document was in the top 3. Without dates or rules, the model guesses.

Why not the simpler way

Real document collections contain old price lists, superseded policies, handbooks that simplify the official terms, and notices that override both for a while. Retrieval finds all of them, because they talk about the same thing. An answer built from the wrong one is wrong with a valid citation.

The idea

The idea in plain words

Three kinds of disagreement: versions of one document over time (staleness), different documents that say different things (conflict), and temporary overrides such as an endorsement that suspends cover for a period. Each needs metadata and a rule.

A train timetable from last year is still a well-written timetable. It is just not valid today.

How it works

Step by step

  1. Store effective_from and effective_to on every chunk. Keep only versions valid on the date in question: today, or the date the user asks about.
    1

    Filter by validity

    Store effective_from and effective_to on every chunk. Keep only versions valid on the date in question: today, or the date the user asks about.

  2. Policy conditions over a handbook, an endorsement over the base policy, an official notice over a blog post. Write the order down.
    2

    Rank sources by authority

    Policy conditions over a handbook, an endorsement over the base policy, an official notice over a blog post. Write the order down.

  3. Pass version and source type to the model. When sources disagree, answer by the rule and mention the other value.
    3

    Show conflicts that matter

    Pass version and source type to the model. When sources disagree, answer by the rule and mention the other value.

In detail

Validity. Every chunk carries effective_from and effective_to (RAG-11). A question about "today" uses today; a question about "September 2025" uses that date.

Authority. A small table maps source types to a rank. Pass the source type into each chunk's label so the model can apply the rule, and apply it in code where you can.

Overrides. Endorsements and notices are valid for a period and override the base text in that period. Our flood questions depend on postcode and date for exactly this reason (RAG-30).

Freshness rules. Some answers should expire: an outage status from yesterday is stale today. Store a time-to-live per source type.

The code

Try it in Python

AUTHORITY = {"policy-conditions": 3, "endorsement": 3, "tariff-sheet": 3, "product-sheet": 2, "handbook": 1}

def label(chunk) -> str:
    m = chunk.meta
    return f"({m['title']}, version {m['version']}, valid {m['effective_from']} to {m['effective_to'] or 'now'}, source type {m['type']})"

RULE = """If sources disagree, use the one with the higher source type in this order:
policy conditions and endorsements, then product sheets, then handbooks.
Say which value you used and mention the other."""

See it

What we measured

Measured · outdated versions in the top 3

Polder corpus, 34 recursive 512 token chunks, hybrid search. Version questions: 8 golden questions over the tariff sheets (3 versions) and the outsourcing policy (2 versions).

Version questions with another version in the top 35 of 8
Right version first, no filter (RAG-11)2 of 5
Right version first, validity filter (RAG-11)2 of 54 of 5
Conflict questions in the golden set2

Measured 2026-11-04 with corpus/scripts/measure-parts-6-7.mts; filter numbers measured 2026-11-03 on vector search. Model behaviour with the authority rule was not measured (not measured yet).

Try it on a real example

“How long does Polder Verzekeringen take to decide on a complete claim?”

Answer

Two retrieved sources

Claims handbook: “We decide on your claim within 10 working days…”
Policy conditions: “Polder Verzekeringen decides on a complete claim within 30 days.”

Both are retrieved. A model may pick either, or average them into something neither says.

Choosing

Use it when, avoid it when

Use it when
  • Validity dates and source types on every chunk
  • A written authority order, applied in code and in the prompt
Avoid it when
  • Leaving old versions searchable without end dates
  • Letting the model decide authority on its own

What it costs · Metadata at ingestion and a short rule in the prompt.

In production

Add conflict and version questions to the golden set; ours has 2 conflict and 8 version questions. Alert when two valid documents of the same authority disagree; that is a content issue to fix at the source. When documents are updated, end-date the old version's chunks in the same job that adds the new ones (RAG-12).

Watch out

Common mistakes

  • The newest document always wins. A handbook can be newer and still rank below the conditions.
  • Conflicts are rare. 5 of 8 version questions had an old version in the top 3.
  • Hiding conflicts is cleaner. When a user may have read the other value, say why it does not apply.

Check yourself

Remember this

Remember these

  • Old and new versions are retrieved together: 5 of 8 version questions here.
  • Filter by validity dates before ranking (RAG-11, RAG-29).
  • Write down which sources outrank which.
  • When sources conflict, answer by the rule and name the other value.

Say it in an interview

  1. 01Old and new versions are retrieved together; on my corpus 5 of 8 version questions had both.
  2. 02I filter by validity dates first; that lifted the right version first from 2 to 4 of 5.
  3. 03A written authority order decides conflicts, not the model.
  4. 04When sources disagree, the answer follows the rule and names the other value.
  5. 05Conflict and version questions are part of my golden set.

Quick quiz

Three questions

Question 1

Q1

The claims handbook says 10 working days, the policy conditions say 30 days. Which wins on our corpus?

Question 2

Q2

How many of our 8 version questions had another version of the same document in the top 3?

Question 3

Q3 · from RAG-12

When a document is updated, what must happen to the old version's chunks?

Saved on this device. No account needed.