Commercial vs free models for document extraction: the real cost per completed task

Every team running document extraction at real volume has hit this once: a model returns JSON that passes every schema check. It ships. Weeks later, one field turns out wrong, and nobody had been watching it.

Somebody catches it during an audit, or a customer does, and the team spends a week re-litigating which model to trust. The instinct is to treat "commercial versus free" as one axis with a price tag on it. It isn't. A paid API bundles inference with a service you don't have to operate. A free hosted tier is a commercial service with a strict allowance, not a gift. An open-weight model is an artifact, license attached, and the compute to run it is still an expense, it just lands on a different budget line: hardware, power, somebody's time.

For extraction work the unit that matters is one validated structured record pulled out of a real document: OCR text or page images, a schema, field definitions, source coordinates. Get that right, over and over, and the provider question stops being philosophical and starts being arithmetic.

The invoice that never shows up

The number on the pricing page is not the number that decides anything. Total cost has to include inference, idle hardware, the engineering time spent wiring validation, every retry a bad response triggers, and the minutes a reviewer spends fixing what the model got wrong. That's the same discipline behind where the other 99 percent of your cost lives: count everything, or don't count at all.

Put a local open-weight model plus ordinary validation code on one side. Put a commercial multimodal API or a managed document service on the other. Run it with real model IDs and today's prices, but don't bake those numbers into your architecture: providers reshuffle their catalogs, free tiers vanish with a week's notice, and someone quietly ships a better quantization of the model you passed on before.

Run the same stack of documents down both roads

Feed both routes the identical batch of documents and the identical acceptance bar. Log everything for every record you actually accept, not just the successes:

model + exact revision
input / cached input / output tokens, or energy draw
first-token time, completion time, timeouts
idle hardware allocation
pass, retry, abstain, or human repair
engineering and incident time
privacy, license, fallback constraints

Grade against field-level accuracy, schema validity, whether the model abstained, and how long a human spent on repair. Keep the failures, and label why each one happened: missing evidence, an ignored instruction, malformed structure, bad reasoning, a tool call that errored, or an unsafe action. Those labels make routing decisions useful later, instead of a hunch. A local model, in my experience, nails the boring cases and falls over right where context or ambiguity crosses some threshold you can usually find in the failure labels themselves.

Where the smaller model actually wins

The commercial route earns its price when traffic is spiky, a wrong answer is expensive, and the provider takes real operating work off your plate. That's not license to route everything to the flagship model. Put the provider's cheaper tier in the same experiment, cap the output length and the reasoning effort, and watch what breaks.

The free or open-weight route earns its place under a narrower set of conditions:

• volume is steady enough to keep hardware busy, not idle most of the week
• the task genuinely fits inside a smaller model's competence
• your team already has the hardware and the skill to run it cheaply
• no other workload loses that GPU to this one

Take any of those away and the math flips fast. A model that barely fits and reloads on every batch, or a runtime only one engineer understands, is not free even without an invoice.

The mistake that wrecks these comparisons is dividing a GPU's purchase price by some theoretical lifetime token count, or comparing nothing but the two providers' listed input prices. Price per million tokens and tokens per second are diagnostics, not business outcomes. Report cost per accepted task, p95 completion latency, how bad a failure is when one happens, and how much human attention is still baked into that number.

Keep the door open

Whichever way the numbers land, build so you can leave. Put provider-specific request and response handling behind one narrow adapter. Keep prompts and schemas in version control instead of a vendor console. Keep the job record itself outside any thread object that only one API understands. Contract-test streaming, tool calls, structured output, error handling, cancellation, plus token accounting, for every route you support.

Fallback needs to be a decision, not an accident. If the commercial API goes down and a local model steps in as a worse but working substitute, the kind of local-first cascade people build for exactly this, say so where a user can see it. If local capacity runs out, the answer is never to quietly route sensitive documents to a remote provider instead. Privacy and authorization checks run before any routing logic gets a vote. Full stop.

None of this stays true forever. Re-run the comparison when traffic, model versions, prices, licenses, hardware utilization, or your review practice moves. A route that made sense at ten thousand tasks a month can be wrong at a hundred thousand. A local model that choked on your documents last year might clear the bar now on a better quant. Free tiers disappear with no warning at all.

There's no universal winner here, and I'd stop looking for one. The durable version of this architecture is the smallest set of routes that clears your quality gate, keeps cost and data movement visible, and fails safely when a provider doesn't show up. "Free" and "commercial" are procurement labels, nothing more.

So before you commit to either: pull last month's extraction logs, tag every failure by the six mechanisms above, and total the human repair minutes nobody is billing to either column. That number is the one that decides it.