Chapter 12 · concept 2 of 4 · show all

Choosing an architecture: naive, advanced, agentic, graph

Four families of RAG design, from one vector search to agents and knowledge graphs. Start simple, measure, and climb only when a measured failure needs it. On our corpus, two simple steps did most of the work.

A real question

Naive RAG, advanced RAG, agentic RAG, GraphRAG. Which one should we build?

The simplest one that passes your golden set. Start naive, measure, and add one technique for each failure you can name. Most teams need hybrid search and a reranker long before they need agents or graphs.

The problem

Why this matters

The belief

Start with the most advanced design, so you never have to rebuild.

What really happens

On our corpus, plain vector search put the answer first for 30 of 47 questions. Adding keyword search and a reranker made it 40, and all 47 in the top 3. Every extra layer adds cost, latency and ways to fail.

Why not the simpler way

New RAG patterns appear every month, each with an impressive demo. Teams adopt them before measuring the basics, and end up with slow, expensive systems that are hard to debug. This course measured each step on one corpus; this page puts them together into a decision.

The idea

The idea in plain words

Naive RAG: embed, retrieve top k, generate. Advanced RAG: improvements around the same single pass (better chunking, hybrid search, reranking, filters, query rewriting, verification). Agentic RAG: a model decides what to search, possibly many times, and can call other tools. GraphRAG: build a graph of entities and relations, and retrieve over it.

Choosing a vehicle: a bicycle for the commute, a van when you move house. Owning a lorry for the commute just in case is expensive.

How it works

Step by step

1
Measure the simple design
Chunk, embed, top k, prompt. Run the golden set and the slices. This is your baseline.
2
Fix named failures
Ranked low: hybrid and reranking. Old versions: filters. Follow-ups: rewriting. Multi-step questions: an agent. Relationships: a graph.
3
Pay only for gains
Keep a technique only if the golden set improves and the latency and cost budgets still hold.

In detail

Decision rules.

Answers present but ranked low → hybrid and reranking (RAG-28, RAG-31).
Wrong versions or roles → metadata filters (RAG-29, RAG-46).
Follow-ups fail → rewriting (RAG-38).
Questions need several look-ups that depend on each other, or calculations, or live systems → agentic (RAG-53).
Questions about relationships across many documents, or themes of the whole corpus → graph or hierarchical summaries (RAG-54).
Small, stable corpus → long context with caching may beat any RAG (RAG-04).

Costs to compare. Latency, cost per question, build effort, debuggability, and how many new failure modes it brings.

Quality is decided offline more often than online: a chunk that splits the answer cannot be rescued by a better prompt.

The code

Try it in Python

LADDER = [
    ("naive",    lambda: VectorRAG()),
    ("hybrid",   lambda: HybridRAG()),
    ("rerank",   lambda: HybridRAG(reranker="bge-reranker-base")),
]

def climb(golden, budget_ms: float, min_gain: int = 2):
    best_name, best = None, None
    for name, make in LADDER:
        system = make()
        score, p95 = run_golden(system, golden), p95_latency(system, golden)
        if p95 > budget_ms:
            break
        if best is None or score - best >= min_gain:
            best_name, best = name, score
    return best_name

See it

What we measured

Measured · naive against advanced RAG on 47 golden questions

Polder corpus, 34 recursive 512 token chunks. Naive: bge-small vector search. Advanced: BM25 + vector with RRF, top 10 reranked by bge-reranker-base.

Answer in top 3naive 44advanced 47

Answer ranked firstnaive 30advanced 40

Extra retrieval timeabout 1.4 s (reranking)

Agentic and graph designsnot measured

Measured 2026-11-04 with corpus/scripts/measure-parts-4-7.mts and measure-part-8.mts.

Try it on a real example

Four families, what they add, and what we measured

Design

One vector search, top k, prompt

Simple, fast, cheap.

Measured: 44 of 47 in the top 3, 31 first.

Choosing

Use it when, avoid it when

Use it when

Naive RAG as a measured baseline
Advanced techniques per named failure
Agents or graphs only for question types the golden set shows failing

Avoid it when

Choosing a design from a demo
Keeping components that no longer improve the numbers

What it costs · Each rung adds latency, cost and failure modes; measure all three.

In production

Write the decision down: which failures each component fixes, with the golden-set numbers. Revisit it when the question mix changes; new user needs, such as multi-step questions, are the real reason to climb. Remove components that no longer pay for themselves.

Watch out

Common mistakes

Naive RAG is a toy. It found 44 of 47 here; it is the right start.
Agentic RAG is the next version of RAG. It is a tool for specific questions.
GraphRAG replaces vector search. Most graph systems still use vector search inside.

Check yourself

Remember this

Remember these

Start naive and measure; it is your baseline.
Advanced RAG (hybrid, rerank, filters) fixed most failures here.
Agents and graphs solve specific problems at real cost.
Keep a technique only if the golden set and budgets agree.

Say it in an interview

01I start naive, measure, and add one technique per named failure.
02On my corpus hybrid plus reranking took answers first from 30 to 40 of 47.
03I use agents for multi-step questions and tools, not as a default.
04I consider graphs for relationship and corpus-wide questions.
05Every component has to justify itself on the golden set and the budgets.

Quick quiz

Three questions

On our corpus, which change moved answers at rank 1 from 30 to 40 of 47?

When does an agentic design earn its cost?

Q3 · from RAG-04

When is long context or caching better than RAG?

Saved on this device. No account needed.