OCR models for tables and statements: local hardware and hybrid OCR deployment

So before you spend a euro on an accelerator, ask what a plain CPU layout-and-table pipeline already gets right on this document. For statements and lab reports, that's usually most of it.

These pages are hard because meaning lives in relationships, not glyphs: which header a cell answers to, whether a row is a subtotal, whether a table continues under a repeated header. Faint rules, wrapped cells, and negative-number conventions trip up extraction before OCR misreads a character. A vision-language model can help, reading a distant label against a value or rebuilding a table a grid detector split apart. It can also swap an ambiguous character for something that just sounds right.

the digit lands in the wrong cell, not on the floor

That's the failure to design against. On tables the characteristic error isn't garbage output, it's a correctly recognized digit attached to the wrong row, column, or unit. Routing every page through the biggest VLM, or buying a GPU for work a CPU engine already handles, wastes money and hides the failure. Start with layout-aware OCR and deterministic validation, escalating to a generative model only where it earns its place: bad handwriting, irregular layout, or exceptions the cheap stage keeps rejecting.

A confident wrong answer is worse than a page you have to look at twice.

what a receipt for a page actually needs

Treat the source file as immutable, and log the chain: source hash, capture device, processing version, which model touched it, raw text and confidence, then the normalized field and reviewer change, kept separate. Never let normalization overwrite what was transcribed; if a character could be O or 0, keep both readings. Word error rate treats a dropped comma the same as a wrong account number. That's backwards. Score identifiers and amounts as exact matches, and store a bounding box per field so review means glancing at a crop.

let the model guess less, not smarter

Give the pipeline permission to say it doesn't know. Low resolution, glare, and totals that don't reconcile should raise an exception, not a guess. Confidence scores are poorly calibrated alone, so pair them with image-quality checks, format validators, and disagreement between recognition passes. Don't let a model merge documents in a bundle: split deterministically and carry identifiers through. Text on the page is data, never instructions.

where the GPU earns its keep

Local processing keeps documents under your control and gets cheaper as volume climbs, once you count storage, model memory, and review against it. A hosted API buys elasticity and a wider exception ceiling, at the cost of upload time, retention, and per-page pricing. Route locally by default, escalate only regions that earned it, and keep every escalation visible in the log. Version models and validation rules together; when one changes, replay a fixed batch of clean pages, common noise, and the cases you already know are nearly impossible.

Next thing I'd check: last month's exception queue, and what share of escalations to the expensive model were cases the validators should have caught themselves.