OCR models for technical documents and labels: languages, scripts, and mixed alphabets

So the OCR model "fixes" the part number. Nobody asked it to. It sees a scratched zero, decides an O fits the pattern on the plate, and writes down a part number that never existed on any shelf.

That's the argument I keep having with people who point a vision-language model at manuals and plates and call it solved. Language models are pattern completion engines: fed a damaged surname, a clipped total, or a corroded serial number, they will confidently produce something that reads better than what was actually printed. A name transliterated two ways across two scripts, or a dropped diacritic, turns one identifier into another, cleanly. On a hazard label that's a correctness problem, not a style one: the string on screen has to match the string stamped on the object, across scripts your pipeline was never tuned for.

the two things a scanned zero can become

Treat script preservation as the constraint that outranks every other design choice, including the one where the demo looks smoother. Run high-resolution region OCR first, backed by dictionaries that validate a field's shape without correcting its identity. Bring in a generative model only where it earns its keep: rough handwriting, layouts nothing else parses, fields needing linking across a page. It sits on top of the deterministic stages you can audit.

When a character is genuinely ambiguous, an O that could be a 0, a decimal that could be a stray mark, don't let the business rule overwrite what the recognizer actually saw. Store both. Tag the page, region, model and dictionary versions, so someone can explain a changed value later. Score exact-string accuracy on identifiers, dates, and warnings, not character error rate alone: CER treats a dropped comma and a wrong account number alike.

The failure that costs you is the one that reads perfectly well and is wrong.

the part where a person still has to look

Build fixtures per language, mix scripts on the same page, and score diacritics and punctuation separately from raw accuracy, because averaging hides the errors that matter on a label. Give the pipeline somewhere to say it doesn't know: glare, a clipped corner, a contradictory total, handwriting nobody can read. Confidence scores from the model alone aren't reliable; pair them with format validators, cross-field arithmetic, and disagreement between passes. Store a bounding polygon with every field so a reviewer sees the crop beside the claim.

If a vision-language model reasons over the document, keep its interpretation separate from its observation, and never let text printed on the page count as an instruction. That's an injection surface someone will eventually try. For multi-page jobs, don't let the model merge invoices because they loaded together; split deterministically, keep page order.

Local processing keeps documents in your building and gets cheap once volume is real; a hosted model buys elasticity and a better ceiling on weird cases, at the cost of retention settings and per-page charges you have to track. What I still don't have is a clean rule for routing a document at the exact moment it's ambiguous which side it belongs on, before anyone has read a word of it.