Dot product
Multiply two vectors position by position and sum. The cheapest similarity score; identical in ranking to cosine once vectors are unit length.
An Embedding is a list of numbers that stands for a piece of text. The dot product of two such lists multiplies the numbers in matching positions and adds the results into one score. A higher score means the two texts point the same way in meaning, so search systems use it to find which stored passage best matches a question.
If every vector is first scaled to length 1 (called normalizing), the dot product gives exactly the same ranking as Cosine similarity, with less arithmetic.
Scoring a recipe against your pantry: for each ingredient, multiply how much the recipe needs by how much you have, then add it up. A high total means you can cook it.
Treating dot product and cosine as interchangeable without checking normalization. With unnormalized vectors and an inner-product index, long or high-magnitude vectors win for reasons unrelated to meaning.
a·b = Σ aᵢbᵢ = |a||b|cos θ. It mixes direction (cos θ) with magnitude (|a||b|). For unit vectors the magnitude term is 1, so dot product equals cosine, and squared L2 distance is 2 − 2·cos θ: all three metrics give the same order. That is why the standard move is to normalize once at write time and index with inner product, which skips the two square roots and a division per comparison.
Cost is d multiply-adds and, more importantly, d × 4 bytes of memory read per float32 candidate: a 1024-dim vector is 4 KB. Brute-force search over N vectors is therefore memory-bandwidth-bound, which is why Approximate nearest neighbour (ANN) indexes like HNSW exist. Some models (older DPR-style retrievers, some recommender towers) are trained for raw inner product, where magnitude encodes popularity or confidence; normalizing them changes the ranking. Maximum inner product search on unnormalized vectors is also harder for ANN graphs than cosine search, because the top inner-product match need not be an angular neighbour.
L2-normalize every vector at write and query time, assert the norm in the ingestion path, and index with inner product: pgvector vector_ip_ops (the <#> operator returns the negative inner product, so order ascending), FAISS METRIC_INNER_PRODUCT, Qdrant Dot (or Cosine, which normalizes on insert). Keep raw dot product only when the model card says magnitude matters.
Code search over a 2M-function monorepo with 768-dim vectors. Query vector q and two function vectors, all unit length: q·f₁ = 0.82 (a retry_with_backoff helper), q·f₂ = 0.41 (an unrelated logger). Cosine gives the same 0.82 and 0.41.
Now a teammate adds a second ingestion path that forgets to normalize, so f₂ arrives with length 2.4. Its inner product becomes 0.98 and the logger outranks the right answer, while cosine still says 0.41. Nothing errors; ranking is just wrong for every vector from that path.
When asked which metric you configured, answer from the model card: "text-embedding-3 and Cohere embed-v4 return unit vectors, so I index with inner product. If the card says magnitude carries signal, I keep raw dot product and accept a harder ANN problem." Expect the follow-up "how would you detect a mixed index?" The answer is a nightly job that samples vector norms per ingestion source and alerts when any norm leaves 1.000 ± 0.001.
| Metric | a=[.6,.8] b=[.8,.6] | b scaled ×2 | Rank if unit |
|---|---|---|---|
| Dot product | 0.96 | 1.92 | Same |
| Cosine | 0.96 | 0.96 | Same |
| L2 distance | 0.28 | 1.08 | Same (reversed) |
Numbers worth knowing
- Bytes read per 1024-dim candidate
- 4 KB float32, 1 KB int8, 128 B binaryuncompressed vector, no index overhead
- Brute-force scan of 1M × 1024-dim vectors
- ≈4 GB read; ~40–200 ms per queryone CPU socket, 20–100 GB/s memory bandwidth (estimate)
- L2² for unit vectors
- 2 − 2·cos θexact identity; same ranking as cosine and dot product
How to decide
| When | Do this |
|---|---|
| The model returns unit vectors (text-embedding-3, Cohere embed-v4, most sentence-transformers with normalize=True) | Index with inner product; it is the cheapest of the three. |
| The model card says it was trained for unnormalized inner product | Keep magnitudes, use MIPS, and expect lower ANN recall (efSearch, nprobe) at the same HNSW settings; measure it. |
| You are under ~200k vectors and latency budget is >50 ms | Skip ANN and brute-force with FAISS IndexFlatIP or a pgvector sequential scan: recall is exactly 1.0. |
Cost maths
Brute force costs N × d × bytes per query in memory traffic. 5M chunks × 768 dims × 4 B = 15.4 GB per query; at 50 GB/s that is ~300 ms and caps one node at ~3 QPS. HNSW (M=16, ef_search=100) touches roughly 1–3k vectors per query ≈ 3–9 MB, so the same node serves hundreds of QPS. Quantizing to int8 cuts every term by 4×.
Operating it
- Brute-force p95 above 100 ms or corpus past ~1M vectors: move to an ANN index
- Norms drifting off 1.0 on any ingestion source: block that source until fixed
- Memory bandwidth saturated on the search node (perf counters >80%): quantize before adding nodes
- Vector norm distribution per ingestion source: alert if p1 or p99 leaves 1.000 ± 0.001
- Golden-set recall@10 per source: alert on a drop of >3 points week over week
- Search p95 latency: alert above 2× baseline (signals a scan fallback or bandwidth contention)
When it goes wrong
Developer-docs search started returning the same five long changelog pages for unrelated questions.
A weekly Golden set run showed recall@10 down from 0.86 to 0.71; slicing by source isolated the changelog importer.
A new importer called the embedding model without normalize_embeddings=True; its vectors had norms of 1.8–3.1 in an inner-product index.
Normalize in a shared embedding client instead of per importer, add a norm assertion at write time, and re-embed the 40k affected chunks.
Related terms
Sources
- The Faiss library — Douze et al., 2024paper
- pgvector: open-source vector similarity search for Postgres — pgvector, 2024docs
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks — Reimers & Gurevych, 2019paper
Check yourself
Three questions. Every option is explained once you answer.