RAGEngineering Hub
Menu
Chapter 9 · concept 1 of 6 · show all
  1. 1What RAG is, and why
    1. 1.1Why models need outside knowledge, and what hallucination means
    2. 1.2What RAG is: anatomy of a request
    3. 1.3RAG vs fine tuning vs long context and prompt caching
    4. 1.4Use cases and the quality, latency and cost triangle
    5. Chapter overview
  2. 2Tokens, cost and your first RAG
    1. 2.1Tokens and context windows for RAG
    2. 2.2The RAG lifecycle and your first RAG in miniature
    3. Chapter overview
  3. 3Getting documents in
    1. 3.1Content sources and file formats
    2. 3.2Parsing, layout and OCR
    3. 3.3Tables, images and mixed content
    4. 3.4Cleaning, normalization and quality gates
    5. 3.5Metadata, document identity and versioning
    6. 3.6Updates, deletes, reindexing and ingestion pipelines
    7. Chapter overview
  4. 4Chunking
    1. 4.1Why we chunk, and the main chunking strategies
    2. 4.2Structure aware chunking and tables
    3. 4.3Chunk size, overlap and token based sizing
    4. 4.4Contextual chunking and late chunking
    5. 4.5Parent child and multi representation indexing
    6. 4.6Chunk metadata, provenance and the chunking debugging playbook
    7. Chapter overview
  5. 5Embeddings
    1. 5.1Embeddings and similarity from first principles
    2. 5.2What embeddings cannot do
    3. 5.3Choosing an embedding model: size, cost, language
    4. 5.4Embedding model migration and index rebuilds
    5. Chapter overview
  6. 6Vector stores and scores
    1. 6.1Vector databases, ANN indexes and choosing a store
    2. 6.2Score calibration and thresholds
    3. 6.3Late interaction and multivector retrieval
    4. Chapter overview
  7. 7Finding the right chunks
    1. 7.1Query understanding, rewriting and expansion
    2. 7.2Dense retrieval and keyword search with BM25
    3. 7.3Hybrid retrieval and rank fusion
    4. 7.4topK, no good match, and metadata filtering
    5. 7.5Routing, diversity, deduplication and caching
    6. 7.6Reranking: two stage, cross encoder and LLM rerankers
    7. Chapter overview
  8. 8When retrieval fails
    1. 8.1Retrieval failure analysis playbook
    2. Chapter overview
  9. 9Building the answer
    1. 9.1Context selection, packing and citations
    2. 9.2Grounding prompts and abstention
    3. 9.3Lost in the middle and context management
    4. 9.4Conflicts, staleness and freshness rules
    5. 9.5Hallucination checks and answer verification
    6. 9.6Conversational RAG, answer UX and RAG APIs
    7. Chapter overview
  10. 10Measuring
    1. 10.1Why measure, and what to measure
    2. 10.2Building a golden dataset
    3. 10.3Retrieval metrics: precision, recall, MRR, nDCG
    4. 10.4Faithfulness and answer correctness
    5. 10.5LLM as judge, and judging the judge
    6. 10.6Offline and online evals, regression tests and slices
    7. Chapter overview
  11. 11Safety and operations
    1. 11.1Prompt injection and poisoned documents
    2. 11.2Access control, privacy and governance
    3. 11.3Index lifecycle operations
    4. 11.4Observability and tracing
    5. 11.5Latency, cost, caching and model routing
    6. 11.6Reliability, degraded modes and launch readiness
    7. Chapter overview
  12. 12Scale and advanced RAG
    1. 12.1Scaling, multi tenancy and enterprise architecture
    2. 12.2Choosing an architecture: naive, advanced, agentic, graph
    3. 12.3Agentic RAG and RAG as an MCP tool
    4. 12.4GraphRAG, structured data and multimodal RAG
    5. Chapter overview

Context selection, packing and citations

Turning ranked chunks into a prompt: how many, in what order, labelled how, and how to make every claim point at a numbered source. With measured context sizes and a citation check.

A real question

We have the right chunks. How should they go into the prompt?

Few, deduplicated, best first, each labelled with a number and its source. Then ask for a citation after every claim, so both readers and scripts can check where each fact came from.

The problem

Why this matters

Paste the retrieved text into the prompt and the model will sort it out.
The belief

Paste the retrieved text into the prompt and the model will sort it out.

What really happens

Unlabelled text gives the model no way to say where a fact came from, and you no way to check it. Numbered, labelled sources make every answer traceable to one chunk.

Why not the simpler way

Retrieval ends with a ranked list; the model needs a prompt. Many systems join the chunks with blank lines and move on. That loses the source of every fact, wastes tokens on duplicates, and hides the best chunk among the rest. Packing is a small step with a large effect on cost, trust and debuggability.

The idea

The idea in plain words

Selection picks which chunks go in. Packing decides order and format. Citations link each claim in the answer to a numbered source. Together they make the context small, readable and checkable.

A good report has footnotes. The reader can follow any claim to its source, and an editor can check every footnote.

How it works

Step by step

  1. Take the reranked top few. Drop near copies and old versions. Stop when the token budget is reached.
    1

    Select

    Take the reranked top few. Drop near copies and old versions. Stop when the token budget is reached.

  2. Give each chunk a number and a header: document title, version, section. Keep the original text.
    2

    Label

    Give each chunk a number and a header: document title, version, section. Keep the original text.

  3. Instruct the model to put [n] after every claim, and check each [n] exists and holds the claim.
    3

    Ask for citations

    Instruct the model to put [n] after every claim, and check each [n] exists and holds the claim.

In detail

Budget. Decide the context budget in tokens (RAG-03), then add chunks in rank order until it is full. Our Mini RAG prompt used about 1,026 tokens of context for 3 chunks.

Order. Best first. Models use text at the start of the context more reliably than in the middle (RAG-35).

Format. [n] (document, version, section) then the chunk text. Our Mini RAG Lab uses: "Answer the question using only the sources below. Cite sources like [1]. If the sources do not contain the answer, say so."

Citation check. Parse every [n]; reject numbers that do not exist; for each sentence with a number, code or date, check that value appears in the cited chunk (RAG-37).

Token budget of one RAG call: 39 instruction tokens, about 1,026 tokens of retrieved chunks, about 13 question tokens, and up to 1,024 tokens reserved for the answer, all inside the context window.ONE CALL · ABOUT 2,102 TOKENS OF A MUCH LARGER CONTEXT WINDOWInstruction ~393 retrieved chunks~1,026Question ~13Answer reserve~1,024
Median prompt in the Mini RAG Lab: 1,078 tokens across 47 golden questions (recursive 512 chunks, top 3). The answer reserve is the max_tokens the lab sends.

The code

Try it in Python

import re

def pack(chunks, budget_tokens=1200, count=len_tokens):
    out, used, seen = [], 0, set()
    for c in chunks:                                   # already reranked, best first
        key = c.text[:200]
        if key in seen:
            continue
        cost = count(c.text)
        if used + cost > budget_tokens:
            break
        seen.add(key); used += cost
        out.append(f"[{len(out) + 1}] ({c.doc}, version {c.version}, {c.section})\n{c.text.strip()}")
    return "\n\n".join(out)

def bad_citations(answer: str, n_sources: int) -> list[int]:
    return [int(n) for n in re.findall(r"\[(\d+)\]", answer) if not 1 <= int(n) <= n_sources]

See it

What we measured

Measured · context size and answers present, hybrid search

Polder corpus, 34 recursive 512 token chunks, 47 golden questions, BM25 + bge-small with RRF. Tokens are the chunk texts only, averaged per question.

Top 1309 tokens33 of 47
Top 3884 tokens47 of 47
Top 51,466 tokens47 of 47
Top 102,704 tokens47 of 47
Expected answers whose numbers all appear in the top 338 of 39

Measured 2026-11-04 with corpus/scripts/measure-parts-4-7.mts and measure-parts-6-7.mts. Citation behaviour of a real model was not measured (not measured yet: needs an API key).

Try it on a real example

The same question, two ways to pack the context

Pack

Prompt

…Price changes during a fixed term are announced at least 30 days in advance by e-mail. … We decide on your claim within 10 working days… Credit risk remained low…

Three documents run together. The answer “30 days” cannot be traced to the tariff sheet, or told apart from the insurance text next to it.

Choosing

Use it when, avoid it when

Use it when
  • Numbered, labelled sources, best first
  • A token budget and dedup before packing
  • A script that checks citations
Avoid it when
  • Joining chunks without labels
  • Filling the context window because it is there

What it costs · Labels add a few tokens per chunk. Every extra chunk adds about 300 tokens on our corpus.

In production

Return citations as structured data (source id, document, version, link), not just text, so the interface can show them and logs can store them (RAG-38). Measure citation coverage: the share of answer sentences with a valid citation. Track context tokens per question; it is usually the largest part of the cost.

Watch out

Common mistakes

  • Citations prove the answer is right. They make it checkable; the check is a separate step.
  • More context is safer. Top 3 already held all 47 answers here.
  • Order does not matter. It does, especially for long contexts (RAG-35).

Check yourself

Remember this

Remember these

  • Pack few chunks, best first: top 3 held every answer in about 900 tokens.
  • Label each chunk with a number, document, version and section.
  • Ask for a citation after every claim.
  • Check citations with a script: does source [n] contain the claimed value?

Say it in an interview

  1. 01I pack few chunks, deduplicated and best first, within a token budget.
  2. 02Each chunk gets a number and a source header; the answer cites [n] per claim.
  3. 03On my corpus the top 3 held every answer in about 900 tokens.
  4. 04A script checks that each citation exists and that the cited chunk holds the value.
  5. 05Citations go back to the client as structured data, not just text.
Try it yourself: Packing LabPick chunks, a token budget and an order, and see the prompt, where the answer lands and the cost.

Quick quiz

Three questions

Question 1

Q1

Why number each source in the prompt?

Question 2

Q2

With hybrid search, how big was a top-3 context on our corpus?

Question 3

Q3 · from RAG-31

What does a reranker change?

Saved on this device. No account needed.