OCR models for tables and statements: preprocessing for OCR models

A table of numbers on a scanned statement carries none of the redundancy a paragraph of prose has. Miscopy a word and a reader's brain patches it from context. Miscopy a digit in a total and nothing catches it. That's the workload constraint here: you're extracting character-perfect values headed for a ledger, and the recognizer can't tell a cosmetic miss from a financial one.

Statements aren't paragraphs with numbers in them

Worry about financial statements, fee schedules, price lists, lab panels: the value lives in a cell, not a sentence. You're extracting cells, headers, row hierarchy, merged regions, units, footnotes, whether a table continues onto the next page. The damage is mundane: faint rules, wrapped cells, repeated headers, negatives in parentheses, type small enough to misread.

Start deterministic: layout-aware OCR, table detection, a validation pass that checks output like a spreadsheet, not prose. Bring in a generative or vision-language model only where it earns its place: bad handwriting, an odd layout, a label linked to a distant value, a genuine exception. It's a specialist you call in, not a replacement for stages that already work.

Keep a receipt for every pixel you touch

Version every preprocessing step, orientation, crop, dewarp, denoise, threshold, contrast, and test each against the untouched original. A filter that makes a page look cleaner to a human can still erase a decimal point the recognizer needed. Keep the source immutable, give it a page and document ID, and log every downstream decision instead of overwriting it:

source hash, page, capture device, orientation
crop/dewarp/threshold versions
OCR/layout/VLM model, runtime revision
raw text, regions, confidence, reviewer change

If a character is ambiguous, an O or a zero, a smudge that might be a decimal, store what the recognizer saw next to what the rule proposes. Never let the second overwrite the first. Character error rate hides the damage: it scores a dropped comma the same as a transposed account number. Score reconstruction and association separately: add exact-match checks on identifiers, dates, amounts, units.

The wrong number that reads perfectly

The failure worth losing sleep over isn't garbled text. It's correctly recognized digits sitting in the wrong row, column, or unit. A model tuned to make a page look tidy will drop a faint decimal point or a background box, and a damaged part number can get completed into whatever looks most plausible. Nothing flags it.

Build an abstention path instead of trusting confidence scores, model-specific and often miscalibrated. Route to a human on:

• a low-resolution region or clipped page edge
• glare or unreadable handwriting
• totals that don't reconcile against their own rows
• disagreement between two recognition passes

Store the bounding box for every field that matters and show the crop beside the value: review becomes verification, not retyping the page. A vision-language model doing associative reasoning stays on the observation side of that line: text printed inside a document is data, never an instruction, and it doesn't get to call a tool on its own say-so.

None of this is free. The deterministic-first version isn't the cheap option: a validation layer thorough enough to catch a misplaced decimal costs real engineering time, and on a messy enough scan, all that scaffolding can still wave a wrong number through with a straight face.