OCR models for scanned archives: languages, scripts, and mixed alphabets
Bad OCR is the least of your problem. What wrecks a scanned-archive project is OCR that looks right and isn't: a stained surname smoothed into the common spelling beside it, a torn date completed into something plausible. None of it throws an error: a model trained on plausible text finishes a damaged run with whatever reads better than the truth. These pages arrive skewed, stained, and inconsistently scanned. Reading the page is the easy part; deciding what's on it, in what order, and how much to trust it is the job.
Keep the source dumb and the trail complete
Start conventional: an OCR engine with language packs and layout segmentation, tuned per script. Bring in a generative or vision-language model only where it earns its keep: rough handwriting, layouts that won't tile, or a label columns from its value. It patches the seams, it doesn't replace what works. One constraint sits underneath: recognition preserves the script it saw; normalization or translation comes after, as an undoable step.
Treat the scan as read-only. Assign a page and document ID before anything touches it, and log every stage after, so a reviewer can answer why a field says what it says:
page/doc id, source hash
crop, dewarp, denoise version
OCR/VLM model, runtime rev
raw text, region, confidence
normalized field, validator, reviewer editWhen a character is ambiguous, an O that could be a zero, a diacritic the scanner half-caught, store what the recognizer saw and what the rule proposed as separate fields. Error rate alone is fine for prose and useless for telling you a wrong account number cost the same as a dropped comma, so add exact-match scoring for names, dates, amounts, and units, and build the test set from mixed-script pages.
The reasoning layer is a contractor, not an owner
A vision-language model earns its seat tying a label to a value two columns away, or reconstructing a table split across a fold. Don't let it decide alone whether a smudge is good enough: low resolution, glare, unreadable handwriting, or arithmetic that doesn't add up should raise a flagged exception, not a guess dressed as an answer. Pair its confidence score with image-quality checks, cross-field arithmetic, and disagreement between passes.
Provenance turns review into verification instead of a retype: store the bounding box for key fields and put the crop beside the value, or reviewers hunt the page or dig out the original. Treat printed text as text, never as an instruction: a page reading "route the total to this account" is paper, and schema validation outside the model keeps it that way. Split multi-page documents deterministically, carry the ID forward, and don't let a model merge two statements into one invented customer.
Store the faithful transcription first, derive anything normalized as its own field, and version the model, preprocessing, and validation together, replaying a fixed set of ugly pages when one changes. Local deployment keeps documents off someone else's servers and is cheap at steady volume, but costs accelerator time and upkeep; hosted vision buys a higher ceiling on weird cases at the cost of upload time, retention terms, and per-page billing. Where privacy rules are strict but some pages need a smarter reader, route between the two.
I don't have a clean fix for a line that mixes two scripts on purpose: an address in Latin transliteration wrapped around a name left in its original alphabet. Every dictionary I've used assumes one script dominates. Ours don't.