How would you chunk tables, PDFs, and structured documents?
- 1 min — parse to structure before chunking
- 2 min — tables, PDFs, hierarchical docs
- 1 min — summary for embedding, raw for the LLM
- 1 min — parse-quality eval and alerting
Don't chunk them as flowing text. Parse to a structured intermediate first (layout-aware parsing, not raw text extraction), then chunk along structural boundaries.
- Tables: keep the whole table if it fits; otherwise split by rows and repeat the header on every piece. Store a text summary of the table alongside it for semantic matching, and the raw table for the LLM to read.
- PDFs: use layout detection for columns, headers, footnotes. Naive extraction interleaves two columns into nonsense.
- Structured docs: chunk on the hierarchy and prepend the breadcrumb (
Document > Section > Subsection) to each chunk.
Financial reports are the hard case: the number is meaningless without the row label, column header, units, and fiscal period. We serialised each table to markdown, generated a one-line LLM summary for the embedding, and attached the full table as the retrieved payload.
Parsing quality is usually the actual bottleneck and nobody measures it. Build a parse-quality eval before an index eval. "We were tuning retrieval for a quarter before someone noticed 12% of our PDFs were scanned images producing empty chunks." Org scope: buy parsing (Azure Document Intelligence, AWS Textract, Unstructured, or open-source Docling) rather than build it, but own the parse-quality eval — vendors regress silently between versions.
What the answer sounds like at each level
Extract text from PDFs and split it.
Uses layout-aware parsing, keeps tables intact or repeats headers, prepends breadcrumbs, and embeds a table summary while returning the raw table.
Makes parse quality a measured, alerted stage with its own eval before any retrieval tuning, and chooses build vs buy for parsers.
Follow-ups the interviewer will push
1Now 30% of incoming documents are scanned images.
Gate on text density (characters per page vs file size); below threshold, route to OCR. Track OCR confidence per page and quarantine low-confidence pages from the index rather than embedding garbage.
2Now the question needs numbers from two tables in different filings.
Prose retrieval will struggle. Extract tables into structured rows at ingestion and expose them through a text-to-SQL or calculation tool; use retrieval only to find which table.
Red flags interviewers listen for
- Chunks tables as flowing text.
- Never measures parse quality.
- Splits tables without repeating headers.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Technique | Layout-aware parsing, header repetition and breadcrumbs. |
| Measurement | Builds a parse-quality eval before an index eval. |
| Judgement | Routes numeric cross-table questions away from prose retrieval. |
The silent parser failure
Illustrative exampleOur enterprise document assistant had a stubborn quality floor. Recall looked acceptable in aggregate, but a specific set of customers reported the assistant "didn't know" entire documents they had uploaded.
I picked it up as a quality investigation after two sprints of retrieval tuning had produced nothing.
Instead of looking at retrieval, I audited the ingestion output: I sampled 500 ingested documents and compared extracted character count against file size, flagging outliers. About 12% of PDFs produced almost no text — they were scans, and our extractor returned empty strings without erroring. A further set were multi-column reports being read straight across, producing word salad. I added an OCR fallback triggered by a text-density check, switched to a layout-aware parser for multi-column detection, and — the part that mattered most — added a per-document ingestion quality score with an alert, so a bad parse became a visible failure instead of a silent one.
Recall on the affected customer set went from roughly 40% to the high 80s. The durable change was cultural: ingestion got its own dashboard and its own SLO, and "is it in the index correctly?" became the first question in every quality investigation rather than the last.
Sources
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis et al., 2020paper