OCR models for scanned archives: an OCR evaluation that predicts production

Once an archive is big enough to run OCR locally instead of paying per page through an API, you've committed to owning the accelerator, the storage, and the backup discipline for as long as the archive exists. That's the hardware fact underneath the project, and it shapes the architecture more than the model choice does.

The material isn't friendly. Scanned archives are books, correspondence, reports, and historical records off paper or microfilm, arriving with skew, bleed-through, stains, obsolete typefaces, marginal handwriting, and scanning settings that drifted across however many years the collection took to digitize. The pipeline needs to produce text that keeps its page identity, its reading order, and an honest record of where the model was guessing.

Start with a conventional OCR engine, language packs, and layout segmentation: cheap, and it already handles most of the page. Bring in a generative or vision-language model only where it earns its place, difficult handwriting, irregular layouts, label-to-value association, exceptions the deterministic stages choke on. Don't let a flashier model replace stages that already work.

Weight errors by consequence, not by count

A practical test samples real capture conditions, annotates text and structure, and weights CER/WER plus critical-field and abstention metrics by downstream consequence and field frequency, not raw character count. CER and WER are a fine baseline, but they treat a missing comma and a wrong account number as roughly the same event. Add exact-match metrics for the fields that matter: identifiers, dates, amounts, warnings, units. For structured documents, score field association and table geometry separately from plain transcription accuracy. Gate the go/no-go decision on critical-field performance, not the aggregate score.

Everything the pipeline touched

Treat the original scan as immutable and assign page and document identifiers before anything else happens to the file. Log the full path, not just the final text:

source file hash, page, capture device
original dimensions, color, orientation
crop, dewarp, threshold, denoise versions
OCR/layout/VLM model and runtime revision
language hints and dictionaries
raw text, regions, reading order, confidence
normalized field, validation, reviewer change
latency, memory, energy or API cost

Never let normalization overwrite the raw recognition. When a character is ambiguous, O versus 0, I versus 1, a decimal point, a minus sign, a unit, a diacritic, store what the recognizer saw and what the business rule guessed as two separate values.

The confident wrong answer

The failure worth watching for shows up as clean, fluent prose: an unfamiliar surname, date, or part number swapped for something that reads more naturally than what was on the page. Language models are trained to continue plausible patterns, so a damaged word gets completed into whatever looks right and usually passes a spot check. If your evaluation only samples clean pages, or reports one aggregate score dominated by easy body text, this failure hides completely. Sample the ugly pages on purpose.

Give the model a way to say no

Build an explicit abstention path instead of trusting the model to always produce an answer. Route these straight to an exception queue:

• low-resolution or clipped regions
• glare or physical damage over the text
• handwriting nobody can confidently read
• totals that contradict each other on the same page

Confidence scores are model-specific and often badly calibrated, so don't rely on them alone; combine them with image-quality checks, format validators, cross-field arithmetic, known dictionaries, and disagreement between separate recognition passes. Make review fast: store the page number and bounding polygon for every important field and show the crop next to the proposed value. A reviewer resolves one uncertain digit quickly when they can see it; without coordinates, the same fix means searching the whole page, or pulling the physical document back out.

Keep the model out of the database

A vision-language model can associate a label with a distant value or explain why two totals disagree. Keep its output as observation, not authority: any write to a database or downstream tool needs schema validation and business rules that live outside the model. Text printed on a scanned page is untrusted input, not an instruction, never permission to change what the model does next.

For multi-page work, preserve page order and repeated-header identity, and don't hand the model an arbitrary bundle of pages and hope it keeps them straight. Split documents deterministically, attach identifiers up front, and state whether a table or paragraph may continue across a boundary: an ungoverned model will happily merge two unrelated invoices into one.

Version the model, preprocessing, dictionaries, schemas, and validation together, and replay a stratified set (clean pages, common noise, hard layouts, critical fields, known-impossible examples) whenever one of them changes. Local deployment keeps documents under direct control and earns its cost at high, steady volume, once you're already paying for storage, model memory, and accelerator time anyway. A hosted API or frontier vision model buys elasticity and a higher exception ceiling, at the price of upload time, retention terms, data residency, rate limits, and per-page or token cost. Route between the two when privacy rules are strict and escalation stays visible.

The one rule I'd keep: never let the system emit a value it can't point back to on the page.