How does query understanding affect retrieval quality?
- 1 min — the query/document asymmetry
- 3 min — rewrite, expand, decompose, filters, intent
- 1 min — HyDE and why it works
- 1 min — latency cost and caching
There's an asymmetry: queries are short, ambiguous, conversational and underspecified; documents are long, formal and self-contained. Embedding them into the same space directly means comparing unlike things. Query understanding closes that gap — rewriting ("it" → "the refund policy"), expanding with synonyms, decomposing multi-part questions, extracting filters ("last quarter" → a date range), and classifying intent so you can route.
In multi-turn chat, resolving pronouns against conversation history is usually the single highest-ROI retrieval improvement. "What about for contractors?" retrieves garbage standalone and retrieves perfectly once rewritten.
HyDE — generate a hypothetical answer, embed that instead of the question — works because it moves the query into document space. Mention the cost: every rewrite step is an extra LLM call in the critical path. Cache aggressively and consider a small fast model for rewriting. Org scope: query understanding sits in the shared retrieval service, not in each product's prompt, so a rewrite fix lands everywhere at once; buy a hosted small model for it and keep the rewrite prompts versioned with the retriever.
What the answer sounds like at each level
Rewriting vague queries helps retrieval find better documents.
Explains the query/document asymmetry and lists rewriting, decomposition, filter extraction and intent routing with examples.
Quantifies the latency each rewrite step adds, caches and uses a small model, and measures rewrite impact on Recall@k separately from answer quality.
Follow-ups the interviewer will push
1Now you are at 10× traffic and the rewrite model is the bottleneck.
Skip rewriting for single-turn queries with no pronouns (a cheap heuristic covers most of them), cache rewrites keyed on (history hash, query), and run a small model like Claude Haiku or a self-hosted 7–8B at low max_tokens.
2Now the rewrite model hallucinates a filter, e.g. invents a date range.
Validate extracted filters against a schema and against values that exist in the index; on failure, drop the filter and retrieve unfiltered rather than returning nothing.
Red flags interviewers listen for
- Ignores multi-turn context resolution.
- Adds LLM calls to the critical path without mentioning latency or caching.
- Cannot explain why HyDE works.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Mechanism | Explains why short queries and long documents embed poorly together. |
| Practicality | Prioritises conversational rewriting as the highest-ROI step. |
| Cost | Budgets the extra LLM calls and mitigates with caching and small models. |
Sources
- Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE) — Gao et al., 2022paper