OCR models for invoices and receipts: capture quality before recognition

Blame the photo before you blame the model. Not universally true, but most failures I trace sit upstream of the recognizer: an angled receipt, a faded thermal original, a crumple line through the total. No prompt fixes a pixel that was never captured.

The job isn't image-to-string conversion: what belongs on the page, in what order, which characters to trust, where every value traces back to. Invoices, receipts, credit notes, expense photos feed a schema wanting vendor, dates, totals, tax, currency, line items, source coordinates, fought by creases, shadows, skew, faded thermal paper, stray logos, repeated subtotals.

A vision-language model helps by tying a distant label to its value, and hurts by handing back a smooth sentence built from a smear of ink: fluent-and-wrong slips past review the way obviously-broken never does.

The real constraint is boring: the smallest character that matters needs clean pixels before any model sees it. The pipeline starts with OCR, vendor templates, anchors, arithmetic checks; generative tools enter only for messy handwriting, an odd layout, or a real exception, never as a replacement.

The pixel budget comes before the model

Check every page for blur, glare, clipping, skew, resolution, completeness before it's read; fail it and reshoot rather than hope the model compensates. Keep the file untouched, assign a page and document id, log the chain:

source hash, page id, device
crop / dewarp / denoise version
recognizer model, runtime revision
raw text, region, order, confidence

Don't let the tidy version quietly replace the messy one: if a character could be an O or a zero, a decimal or a stray mark, store what the recognizer saw beside what the business rule proposes. Collapse them and a total stays wrong.

When the wrong number looks completely normal

The costly failure isn't garbled output, it's a syntactically perfect amount copied from the subtotal or the row above the one you wanted. Feed an unreadable region to a big model and it hands back a fluent reconstruction: exactly the job it was trained for.

Give the system a way to say no: low-resolution patches, clipped pages, glare, unreadable handwriting, contradictory totals raise an exception, not a guess. Confidence scores alone won't do it, they're often miscalibrated; pair them with quality checks, format rules, disagreement between passes. Show the reviewer the crop beside an uncertain value: a bad digit in a small crop beats hunting a full page.

Measure the right thing: field accuracy, arithmetic reconciliation, duplicate detection, exception rate, not just character error rate, which weighs a missing comma the same as a wrong account number. Score identifiers and amounts by exact match; judge table geometry apart from transcription.

Where I'd spend the generative budget

A vision-language model earns its keep on the hard parts: a distant label-value pair, a split table, why two totals disagree. Keep observation and interpretation separate; never let it write to a database or call a tool unsupervised, that stays outside the model. Text on an invoice is data, not an instruction; a document telling it to skip a check deserves that suspicion.

Split multi-page documents deterministically, carry identifiers and header identity across pages, say whether a table continues past a break, instead of trusting the model to sort that out. Version models, preprocessing, and validation together, and replay a fixed test set, clean through unreadable, whenever a piece changes.

Local deployment keeps documents on hardware you control and turns cheap at high, steady volume, storage, idle accelerator time, and review staff included. A hosted API buys elasticity and handles exceptions better, for upload time, retention you don't control, cost scaling per page; hybrid routing works where privacy is strict and escalation stays visible.

Given the choice, I take the exception queue over the confident wrong answer: a lower straight-through rate than any demo promises, someone squinting at crops on a Friday afternoon. I'll take that trade.