OCR models for invoices and receipts: structured OCR with provenance

Treat OCR on invoices as a text-extraction problem and you'll ship something that looks right in the demo and quietly rewrites your numbers in production. Extraction is the easy part. The hard part is proving, when finance asks why a total doesn't reconcile, exactly which pixels produced which digit.

The documents are supplier invoices, receipts, credit notes, and crumpled phone photos: faded thermal paper, shadows, skewed perspective, a misread logo, one total repeated across sections. You owe the business vendor, dates, totals, tax, line items, and a coordinate per field.

A vision-language model can help read layout the way a person does. It can also fail worse than garbling text, quietly swapping a smudged character for whatever reads naturally. The default stays boring: deterministic OCR, vendor templates, anchors, arithmetic checks, with a generative model earning its way in only for handwriting, irregular layouts, or real exceptions.

Why the pixels have to survive the pipeline

Lock the original file on arrival, assign a page and document id, and never let processing overwrite it. What survives is a chain, not a value:

file hash, capture device, page number
preprocessing steps and versions
recognizer, runtime revision
raw text, region, confidence, reviewer edit

Normalization is where evidence quietly disappears. If a character could be an O or zero, a decimal point, a minus sign, a diacritic, store what the recognizer saw next to what the rule decided. Collapse the two and nobody can tell which was a guess.

Judge the pipeline on field accuracy, arithmetic reconciliation, and exception rate, not character error rate, which treats a misplaced comma the same as a wrong account number. Score identifiers by exact match, and score table geometry apart from transcription: a model can read every character right and still misplace the row.

The invoice that reads perfectly and is wrong

The costly failure isn't the garbled scan. It's the total copied from the subtotal line, or a balance-forward figure sliding into amount-due, perfectly formed and wrong. Models continue plausible patterns, so a damaged total gets completed into whatever looks familiar. Store only the corrected value and you lose the one thing that would catch it.

Give the model a way to say no: a blurred region, a clipped edge, glare, illegible handwriting, a total that won't add up should raise an exception, not a guess. Confidence scores vary by vendor and are often miscalibrated; pair them with image-quality checks, format validators, and disagreement between passes.

Provenance justifies the storage cost. Attach a page number and bounding polygon to every important field, show the crop beside the value, and a reviewer resolves an ambiguous digit in seconds instead of hunting the page.

Where a vision-language model actually earns its keep

Used well, a VLM links a label to a distant value, reconstructs a table split across pages, or explains why two totals disagree. Keep observation and interpretation separate, and never let its output write to a database without validation sitting outside the model. Text on the document is data it's reading, not an instruction.

Multi-page batches need the same discipline: preserve page order and header identity, split deterministically, and decide whether a table may continue across a page boundary.

Local deployment keeps documents on hardware you control and gets cheap at steady volume, once storage, memory, and review time are counted. A hosted API buys elasticity and a higher exception ceiling, at the cost of retention, residency, and a running bill. Route between the two when privacy rules are firm and escalation stays visible.

None of this stops a reviewer from glancing at a crop and approving without reading the digit. Provenance makes verification possible. It doesn't make anyone do it.