OCR (optical character recognition)
Optical character recognition: turning images of text (scans, photos, faxes) into machine-readable text. Errors here propagate into every search.
Scanned documents and photos are just pictures; a computer cannot search the words in them until optical character recognition (OCR) turns the shapes of letters into real text.
If OCR misreads 'O' as '0' or '1' as 'l', search and AI answers inherit those mistakes, so it is worth checking OCR quality before building anything on top of it.
A typist copying a faded photocopy: fast and mostly right, but smudged digits and handwriting cause mistakes nobody notices until later.
Trusting OCR output because it looks fluent. A 2% character error rate means roughly one wrong character every ten words, enough to break identifiers and amounts.
Classic OCR (Tesseract 5, LSTM-based) detects text lines and recognises characters; on clean 300-DPI printed scans it achieves character error rates (CER) around 1–2%, rising sharply below 200 DPI, on skewed photos, tables and handwriting. Managed services (AWS Textract, Google Document AI, Azure AI Document Intelligence) add layout, tables, forms and handwriting at ~$1.50 per 1,000 pages for plain text detection (list prices, Oct 2026, illustrative). Open models such as PaddleOCR and docTR, and vision LLMs, handle harder layouts; vision LLMs can silently 'correct' or invent text, so they need confidence checks on numbers.
Production concerns: deskew and binarise before recognition, request word-level confidence and route low-confidence pages to a stronger engine or human review, keep bounding boxes so citations can highlight the source region, and store the OCR engine and version in Chunk metadata. OCR errors hurt BM25 most (an exact token 'P0L-4471' no longer matches 'POL-4471') and dense retrieval less. OCR usually runs inside Layout-aware parsing.
Preprocess (deskew, 300 DPI, binarise), Tesseract 5 or PaddleOCR with word confidences, escalate pages with mean confidence <80 to a managed OCR service, keep bounding boxes and engine version per page, and send identifier fields (policy numbers, MRNs, invoice ids) with confidence <90 to human review.
Insurance claims intake: 400k faxed and photographed forms per month. Tesseract on raw images: CER 6.8%, policy-number exact-match 71%. After deskew, 300-DPI resampling and routing pages with mean confidence <80 to Textract (22% of pages): CER 1.9%, policy-number match 96%.
Cost: 88k Textract pages × $1.50/1k ≈ $130/month, versus ~$600/month if every page went to Textract.
Show the routing: "Cheap OCR on everything, confidence per page, escalate the bottom 15–25% to a managed engine or vision model, and human review for fields that drive money, like policy numbers. I measure CER on a 200-page labelled sample per source." Expect: "why not just a vision LLM?" Cost at volume and silent fabrication of plausible digits.
| Engine | Strength | Price (Oct 2026) |
|---|---|---|
| Tesseract 5 | Clean print, free | CPU only |
| Textract / Doc AI | Tables, forms, handwriting | ~$1.50/1k pages text |
| Vision LLM | Odd layouts | ~1.5–2k tokens/page |
Numbers worth knowing
- Tesseract CER
- ~1–2% clean 300-DPI print; 5–10%+ on faxes/photosprinted Latin script, no handwriting
- Managed OCR price
- ~$1.50 per 1,000 pagesplain text detection, Textract / Document AI list price (illustrative, Oct 2026)
- Words per character error at 2% CER
- ~1 error per 8–10 wordsaverage English word ≈ 5–6 characters with space
How to decide
| When | Do this |
|---|---|
| Born-digital PDFs with a text layer | Skip OCR; extract the text layer. |
| Clean scans, high volume | Tesseract or PaddleOCR with confidence routing. |
| Forms, handwriting or tables in scans | Managed OCR with forms/tables features, human review for money fields. |
Cost maths
Monthly = pages × (escalation rate × managed price + review rate × human cost). 400k pages, 22% escalated at $1.50/1k = $132; 1% human review at an assumed $0.10/page = $400. All-managed would be $600 plus review. Self-hosted Tesseract at ~1 s/page needs ~110 CPU-hours/month ≈ $5.
Operating it
- CER on the labelled sample above 3% for any source: add preprocessing or escalate that source
- Escalation rate above 40%: the cheap engine is the wrong tool for this source
- Identifier exact-match below 95%: add human review for those fields
- Mean OCR confidence per source per day: alert on a drop >10 points
- Identifier validation failure rate (regex/checksum): alert above 2%
- Escalation rate to managed OCR: alert on a 2× jump (cost and quality signal)
When it goes wrong
Claims adjusters could not find claims by policy number for one broker's submissions.
Search-zero-results rate for policy-number queries was 30% for that broker vs 3% overall.
The broker switched to low-resolution phone photos; Tesseract read 'O' as '0' and '1' as 'I', and nothing checked confidence.
Confidence routing to managed OCR, a policy-number regex plus checksum validation, and fuzzy matching (edit distance 1) for identifier search.
Sources
- Tesseract OCR — tesseract-ocr, 2024docs
- Docling Technical Report — Auer et al. (IBM Research), 2024paper
Check yourself
Three questions. Every option is explained once you answer.