Instruction prefix (query: / passage:)
A fixed string such as 'query: ' or a task instruction prepended before embedding, telling the model what role the text plays.
Some Embedding models expect a short label in front of the text, such as query: before a user's question and passage: before a document. The label tells the model what job the text is doing, so it can place questions near the passages that answer them.
If the model was trained with these labels and you leave them off, it still returns numbers without any error, just worse ones.
Handing a mail room a parcel marked 'outgoing' or 'incoming'. Same box, but the label decides which shelf it goes to.
Assuming a missing prefix would throw an error. It never does; it silently shifts vectors, and the damage hides inside blended averages until you slice recall by ingestion source.
Instruction-tuned embedders are trained with role or task text prepended, so the prefix is part of the model's input distribution. Conventions as of Oct 2026: E5 and multilingual-e5 use query: / passage: ; nomic-embed-text uses search_query: / search_document: (plus clustering: , classification: ); bge-en-v1.5 adds Represent this sentence for searching relevant passages: to queries only; Qwen3-Embedding and gte-Qwen take a free-text instruction on the query side ('Instruct: Given a legal question, retrieve clauses that answer it\nQuery: …'); Cohere, Voyage and Gemini expose an input_type or task_type parameter instead of a string.
The prefix is therefore part of your index contract: change it and every stored vector is stale, exactly as if you changed the model. Record the exact strings with the model version in the Document registry, apply them in one shared embedding client rather than per pipeline, and treat a prefix change as a Backfill. Task-specific instructions (domain wording on the query side) can add 1–3 points of recall on instruction-following models without touching documents.
One shared embedding client that owns model id, revision, query prefix and document prefix as a single versioned config; every chunk stores the config hash; ingestion refuses to write if hashes disagree. For instruction-following models, start with a one-line task instruction naming the domain and answer type.
Legal contracts search with multilingual-e5-large.
- Index:
passage: Either party may terminate this Agreement upon 90 days' written notice… - Query:
query: notice period to terminate for convenience
Six months later a second ingestion path for scanned amendments ships without the passage: prefix. Overall recall@10 falls only from 0.88 to 0.84, so nobody notices, but sliced by source it is 0.89 for the original contracts and 0.61 for amendments, which are exactly the documents lawyers search for most.
In a debugging story, name the prefix as an index contract: "I keep model id, revision and both prefix strings as one config object, version it in the registry, and fail ingestion if the config hash on a chunk does not match the index's." The follow-up tends to be "can you change the query instruction without re-embedding?" Yes, on query-side-only models like bge-en-v1.5 or Qwen3-Embedding; no, for E5-style models where documents carry a prefix too.
| Model family | Query side | Document side |
|---|---|---|
| E5 / multilingual-e5 | query: … | passage: … |
| nomic-embed-text v1.5 | search_query: … | search_document: … |
| bge-en-v1.5 | Represent this…: … | (none) |
| Qwen3-Embedding | Instruct: <task> Query: | (none) |
| Cohere / Voyage API | input_type=query | input_type=document |
Numbers worth knowing
- Prefix cost
- 2–20 tokens per inputfixed role strings vs free-text task instruction
- Recall loss from a missing document prefix
- 5–30 points on the affected sliceE5-style models; diluted in blended averages
- Gain from a domain task instruction
- 1–3 points recall@10instruction-following embedders, query side only (measure)
How to decide
| When | Do this |
|---|---|
| The model card lists prefixes or an input_type parameter | Apply them on both paths and test for them in CI. |
| The model takes a query-side instruction only | Tune the instruction per domain freely; documents need no re-embed. |
| You want to change a document-side prefix | Treat it as a new model: new index, Backfill, Index alias swap. |
Cost maths
A document-prefix mistake costs a re-embed of the affected slice. 1.2M amendment chunks × 500 tokens = 600M tokens × $0.02/M (text-embedding-3-small) = $12, or a few GPU-hours self-hosted. The real cost is the weeks of degraded answers before detection, which is why the per-source recall slice matters more than the bill.
Operating it
- More than one ingestion path writes to the same index: centralize the embedding client
- More than one model or prefix version live at once: add a config-hash check at query time
- Distinct embedding-config hashes per index: alert if more than 1 (outside a backfill window)
- Golden-set recall@10 per ingestion source: alert on any source 5+ points below the median
When it goes wrong
Lawyers reported that amendments never showed up in contract search, though the base contracts did.
Recall@10 sliced by source showed 0.61 for the OCR amendments pipeline vs 0.89 elsewhere.
The amendments pipeline called the model directly and skipped the passage: prefix.
Moved prefixes into a shared client, stored a config hash per chunk, re-embedded 1.2M chunks, added a CI test that embeds a fixture through every pipeline and compares vectors.
Sources
- Text Embeddings by Weakly-Supervised Contrastive Pre-training (E5) — Wang et al., 2022paper
- One Embedder, Any Task: Instruction-Finetuned Text Embeddings — Su et al., 2022paper
- Nomic Embed: Training a Reproducible Long Context Text Embedder — Nussbaum et al., 2024paper
- Embed API reference (input_type) — Cohere, 2025docs
Check yourself
Three questions. Every option is explained once you answer.