Design enterprise search with AI answers across Google Drive, Slack, Confluence, Jira and Salesforce, respecting each tool's permissions.
- 5 min — scope: sources, users, freshness and permission requirements
- 10 min — connectors, identity mapping, ACL model
- 10 min — retrieval: hybrid + pre-filter + rerank + late binding
- 10 min — freshness, leak paths, red-team tests
- 5 min — estimate, ownership, build vs buy
Permission-aware retrieval. Connectors ingest via webhooks or change events plus periodic reconciliation crawls; documents become chunks carrying ACL metadata expressed as principal ids (users, groups, 'anyone in domain', 'anyone with link'). Retrieval is hybrid (BM25 + dense, fused with RRF) under a mandatory pre-filter on the requesting user's expanded principals, then a reranker, then a grounded answer with citations. An identity graph maps every SaaS identity to the corporate identity via the IdP (SCIM).
Query: 'what did we promise the customer about the SSO deadline?' → resolve the user to a principal set (user id + ~340 groups across Google groups, Slack channels and Salesforce roles), cached for 5 min → hybrid search filtered on acl_principals ∈ set → top 50 → cross-encoder rerank → top 8 → answer with links. Before rendering, a late-binding check confirms each cited document against a fresh ACL cache or the source API, so a share revoked a minute ago is caught even if the index is stale.
- Permission semantics are where bugs live. Slack private channels and DMs, Drive link sharing ('anyone with the link' is not 'anyone in the org'), Salesforce role hierarchy and sharing rules, Jira project roles. Model each conservatively and fail closed on anything unrecognised.
- Freshness SLOs: revocations reflected in ≤5 min via events, full ACL reconciliation every 24 h, content freshness ≤15 min. Monitor connector lag per source — rate-limit backoff causes silent staleness.
- Leak paths beyond retrieval: cached answers, derived summaries without ACLs, traces containing chunk text, and embeddings (partially invertible — classify the index like the source).
- Org scope: a search-platform team owns connectors and index; security owns the policy and audits. Buying connectors is reasonable; keep the ACL-enforcement layer in-house or contractually audited. Roll out three sources first, with cross-permission red-team tests in CI.
Assumptions: 20k employees; 50M documents × ~4 chunks = 200M chunks; 1,024-d embeddings; 10 queries/user/day; 2% of documents change daily; answers use 8k input / 400 output at illustrative $1/M input, $5/M output; embeddings at illustrative $0.02/M tokens.
- QPS: 200k/day → 2.3 average, ~25 peak (Monday morning 10×). Retrieval load is trivial; the LLM is the cost.
- Index memory: 200M × 1,024 B (int8) ≈ 205 GB; binary quantisation at 128 B/vector ≈ 26 GB in RAM with int8 on SSD to rescore the top 500. Three shards × two replicas on 64 GB nodes ≈ $5k/month.
- Ingest: 1M changes/day ≈ 12 docs/s. Initial backfill 50M × 2k tokens = 100B tokens × $0.02/M ≈ $2k of embeddings — connector API rate limits, not embedding, make backfill take days.
- Answers: 200k × 8k = 1.6B input ($1.6k) + 80M output ($0.4k) ≈ $2k/day ≈ $60k/month ≈ $3 per employee per month, plus ~$0.50 for infrastructure.
What the answer sounds like at each level
Indexes documents into a vector DB and filters results by what the user can see.
Stores principal ids per chunk, resolves group membership at query time, pre-filters, and handles ACL freshness with events plus reconciliation.
Models each tool's permission semantics explicitly, adds late-binding checks, sets freshness SLOs, enumerates non-retrieval leak paths, and splits ownership with security.
Follow-ups the interviewer will push
1A user leaves a private Slack channel. How fast does search reflect it?
The membership event updates the principal cache; no re-index is needed because membership is resolved at query time. SLO ≤5 min, with the late-binding check covering the gap.
2The corpus grows to 500M documents.
Binary quantisation with int8 rescoring; shard by source and tenant. Restrictive ACL filters hurt HNSW recall, so use an engine with filter-aware graph traversal (Qdrant, OpenSearch, Vespa) or partition by large groups.
3The EU subsidiary's data must stay in the EU.
A separate regional index and answer path; federated queries respect residency, and generation for EU documents runs in-region.
Red flags interviewers listen for
- Filters results after retrieval instead of pre-filtering.
- Copies ACLs at ingest only, with no event path for revocations.
- Treats 'anyone with link' as public.
- Ignores caches, traces and embeddings as leak paths.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Permissions | Principal-based pre-filter, query-time group resolution, per-tool semantics, fail closed. |
| Freshness | Event path plus reconciliation with explicit SLOs and lag monitoring. |
| Retrieval quality | Hybrid + RRF + reranker; handles filter selectivity at scale. |
| Org | Security co-owns policy; staged source rollout with adversarial tests. |
Sources
- Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods — Cormack et al., 2009paper
- Dense Passage Retrieval for Open-Domain Question Answering — Karpukhin et al., 2020paper
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — Greshake et al., 2023paper