Finding the right chunks

Concept 7.1

Query understanding, rewriting and expansion

Users type short, vague or follow-up questions. Rewriting can fix that, or break it. We searched 12 questions in full and as short keywords; the same change helped one question and buried another.

Concept 7.2

Dense retrieval and keyword search with BM25

How BM25 scores shared rare words, how dense retrieval scores meaning, and a head to head on our 47 questions where the 30 year old method won, with the reason why.

Concept 7.3

Hybrid retrieval and rank fusion

Run keyword and vector search, then merge the two ranked lists with Reciprocal Rank Fusion. On our corpus fusion kept all 47 answers in the top 3 and fixed every vector miss.

Concept 7.4

topK, no good match, and metadata filtering

How many chunks to pass on, what to do when none fit, and how filters on version, date and access remove wrong candidates before ranking. Measured recall and token cost per k.

Concept 7.5

Routing, diversity, deduplication and caching

Four ways to shape the candidate list: send questions to the right index, spread results across sources, drop near copies, and reuse answers. Measured: diversity cost us 3 answers, and a semantic cache would confuse two flood questions.

Concept 7.6

Reranking: two stage, cross encoder and LLM rerankers

A fast search finds 10 candidates, then a cross-encoder reads each one with the question and reorders them. On our corpus, answers ranked first rose from 30 to 40 of 47, at about 1.5 seconds per question on a CPU.