How would you prevent one customer's documents from appearing in another customer's results?
- 1 min — stakes and the principle: defence in depth, fail closed.
- 3 min — four layers of enforcement.
- 3 min — leak paths beyond retrieval.
- 2 min — adversarial tests in CI and incident response.
Defence in depth, because this is the failure that ends the company. (1) Physical partitioning where possible. (2) Mandatory pre-filter on tenant ID enforced at the data-access layer, never by the caller. (3) Post-retrieval verification that every returned chunk's tenant matches the request before it reaches the LLM. (4) Fail closed — missing tenant context returns an error, not everything.
Keep tenant ID in the request context and derive it from the authenticated principal, never from a request body field a client controls. Then write the test suite as an adversarial one: tests that attempt cross-tenant retrieval and must fail, run in CI on every change.
Name the leak paths people miss: the cache (a semantic cache keyed only on query text serves tenant A's answer to tenant B — key on tenant plus query); logs and traces containing chunk text; eval and debug tooling running with elevated access; and embeddings themselves, which are partially invertible (Morris et al. 2023 recovered 92% of 32-token inputs exactly), so treat the vector store as containing the source text for classification purposes. Also: post-filtering is not an acceptable control here, because it's a correctness bug waiting for a race condition.
Org scope: security owns the threat model, the platform owns the enforcement layer and the shared adversarial test template, and every service team inherits that template in CI. Isolation is reviewed at design review for any new cache, log sink or tool that touches chunk text.
What the answer sounds like at each level
Filter every query by tenant id and test that tenants can't see each other's data.
Layers partitioning, data-layer pre-filter, post-retrieval verification and fail-closed defaults, with tenant derived from the authenticated principal.
Enumerates non-obvious leak paths (caches, traces, eval tooling, embedding inversion), and makes adversarial isolation tests a shared CI template every team inherits.
Follow-ups the interviewer will push
1A bug ships and one tenant saw another's chunk. First hour?
Kill switch the affected path (disable the cache or feature), preserve traces for scope, identify every affected request by joining trace tenant id against chunk tenant id, notify legal/security for breach-notification timelines (72 hours under GDPR), then fix and add the adversarial test that would have caught it.
2Why is the post-retrieval tenant check worth having if pre-filtering is correct?
Because pre-filtering is correct until a refactor, a new index or a misconfigured alias makes it not. The post-check is a cheap invariant (compare one field per chunk) that turns a silent leak into a loud error and an alert.
Red flags interviewers listen for
- Takes tenant id from a client-supplied request field.
- Considers post-filtering sufficient.
- Forgets caches, logs and debug tools as leak paths.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Layering | Multiple independent controls, each enforced below the caller. |
| Leak-path coverage | Names cache, logs, tooling and embedding inversion. |
| Verification | Adversarial cross-tenant tests in CI on every change. |
Closing a cross-tenant gap before it opened
Illustrative exampleWe were launching a semantic cache to cut LLM spend on our multi-tenant assistant. The design keyed cached answers on the normalised query embedding, and it was already in staging with good hit rates.
I was reviewing it as the engineer responsible for the retrieval platform. The cost savings were real and the team was close to shipping.
The cache key had no tenant component, which meant two customers asking the same question would share an answer built from one customer's documents. I wrote a failing test that demonstrated the leak concretely rather than describing it in review comments — that changed the conversation in about ten minutes. We added tenant to the key, which cut the hit rate roughly in half, so I proposed recovering it by caching the expensive reranking stage per tenant and caching only non-grounded general answers globally. Then I went looking for the same class of bug elsewhere and found two: trace logs capturing chunk text without tenant redaction, and an internal eval tool running with a service account that could read every namespace. I added cross-tenant adversarial tests to the shared CI template so every service inherits them.
We shipped the cache with about 70% of the projected savings and no isolation gap. The adversarial test template has since caught two similar mistakes in other teams' services before they reached staging, which is a better outcome than the original catch.
Sources
- Text Embeddings Reveal (Almost) As Much As Text — Morris et al., 2023paper
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — Greshake et al., 2023paper