Chapter 11 · concept 2 of 6 · show all
2Tokens, cost and your first RAG
4Chunking
- 4.1Why we chunk, and the main chunking strategies
- 4.2Structure aware chunking and tables
- 4.3Chunk size, overlap and token based sizing
- 4.4Contextual chunking and late chunking
- 4.5Parent child and multi representation indexing
- 4.6Chunk metadata, provenance and the chunking debugging playbook
- Chapter overview
7Finding the right chunks
8When retrieval fails
Access control, privacy and governance
Users must only get answers from documents they may read. Filter by role before ranking, never in the prompt. Without a filter, our restricted runbook reached a customer's top 3; with one, it cannot.
A real question
A customer asks a question whose answer is in an internal runbook. What should happen?
The runbook should never reach the model for that customer. Store who may read each document on every chunk, filter by the user's real roles before ranking, and log what was shown to whom.
The problem
Why this matters
Tell the model in the prompt not to reveal internal documents.
Once restricted text is in the prompt, it has already left the access boundary. A prompt instruction can be ignored or overridden by injection. Without a filter, our NOC-only runbook reached the top 3 for a customer's question.
Why not the simpler way
RAG often starts as a demo over public documents, then grows into internal ones: HR policies, incident runbooks, customer records. Search does not know who is asking. Without access control in retrieval, the assistant becomes the easiest way to read documents you were never meant to see.
The idea
The idea in plain words
Access control decides which chunks a user may retrieve, based on the permissions of their source documents. Privacy covers personal data in documents and in logs: minimise, protect, delete on request. Governance is knowing what is indexed, who owns it, how long it is kept, and being able to show that.
A library with a reading room for staff: the catalogue only shows staff books to people with a staff card, it does not ask visitors to promise not to read them.
How it works
Step by step
1Roles on every chunk
Copy the document's access list (allowed_roles, groups, tenant) to each of its chunks at ingestion, and update it when the document's permissions change.
2Filter by the session
Take the user's roles from the authenticated session, never from the question. Filter before ranking, in the index query.
3Log, retain, delete
Log which chunks each user received. Keep logs only as long as needed. Delete a person's data, and its chunks, on request.
In detail
Document-level ACLs copied to chunks: allowed_roles, groups, tenant_id. Multi-tenant systems also separate indexes or namespaces per tenant (RAG-51).
Query-time filter. The filter is part of the search call, built from the session. Pre-filtering is exact; check ANN recall with narrow filters (RAG-22).
Sync. Permission changes in the source must reach the index quickly. Store a permissions hash per document and re-sync when it changes.
Privacy. Detect and mask personal data at ingestion where the use case allows. Logs of questions and answers are personal data too: retention limits and deletion apply.
The code
Try it in Python
def allowed(chunk_meta: dict, session_roles: set[str]) -> bool:
roles = set(chunk_meta.get("allowed_roles") or [])
return not roles or bool(roles & session_roles) # empty list = public
def search_for(session, question, index):
roles = session.verified_roles # from the login, never from the question
pool = [c for c in index.chunks if allowed(c.meta, roles)]
results = hybrid_search(question, pool)
audit_log(session.user_id, question, [c.id for c in results])
return results
See it
What we measured
Measured · restricted chunks in the top 3, hybrid search
Polder corpus, 34 recursive 512 token chunks. One document, the NOC escalation runbook, has allowed_roles [noc_engineer]. 62 questions: 47 answerable plus 15 unanswerable.
Measured 2026-11-04 with corpus/scripts/measure-parts-4-7.mts. With the filter applied before ranking, restricted chunks cannot be returned to users without the role, by construction.
Try it on a real example
“When must a fibre outage be escalated to the duty manager?”
Allowed: the runbook lists noc_engineer. Answer: escalate within 15 minutes when more than 500 connections are affected.
Choosing
Use it when, avoid it when
- ACLs on every chunk, filtered before ranking
- Role pairs in the golden set
- Audit logs with retention limits
- Prompt instructions as access control
- Roles taken from user input
- Caches shared across roles
What it costs · Metadata at ingestion, a filter per query, and a sync job for permission changes.
In production
Test access with golden items per role: the same question asked by roles that may and may not see the answer, as in our g-38 and g-39 pair. Run these on every index rebuild. Audit regularly: which documents are indexed, who can see them, who owns them. Treat a cache keyed without the user's roles as a leak waiting to happen (RAG-30).
Watch out
Common mistakes
- The model will keep secrets if asked. Text in the prompt is already exposed.
- Filtering after ranking is fine. Restricted chunks may already be in the context or the cache.
- Internal tools do not need access control. Internal users have different rights too.
Check yourself
Remember this
Remember these
- Store access lists on every chunk and keep them in sync with the source.
- Filter by the session's roles before ranking.
- Without a filter our restricted runbook reached a customer's top 3.
- Log, limit retention, and support deletion of personal data.
Say it in an interview
- 01Access control lives in retrieval: ACLs on every chunk, filtered before ranking.
- 02Roles come from the authenticated session, never from the question.
- 03Without a filter my NOC-only runbook reached a customer's top 3; with it, it cannot.
- 04I test with role pairs: the same question for a role that may and one that may not see it.
- 05Question and answer logs are personal data, with retention and deletion rules.
Quick quiz