Commercial vs free models for document extraction: reliability and exit strategy

My rule for document extraction: don't pick commercial or local off a leaderboard, pick it on the cost of one validated record, and build the exit ramp before you ship. That's the whole verdict. Everything below is why I trust it.

Start by admitting "commercial versus free" is three different deals under one label. A paid API bundles inference with a service: uptime, support, a roadmap someone else owns. A free hosted tier is usually that same service on a leash. An open-weight model is just an artifact: license status unresolved, and running it is still somebody's line item, an invoice traded for hardware, electricity, maintenance, and capacity planning nobody scheduled.

the unit of work is a validated record, not a response

For document extraction you're buying one validated structured record out of a real document: OCR text or page images in, a schema plus field definitions plus source coordinates as constraints, JSON out. The costly failure isn't a blank response, it's JSON that looks fine and is wrong somewhere: a total off by a transposed digit, a date missing its qualifier. A model that fails validation, triggers a retry, or parks a reviewer for ten minutes on one record can cost more than the pricier call that finishes in one pass.

Cost per accepted record was always the real metric, not cost per token.

The gate I use is field-level accuracy, schema validity, whether the model abstains instead of guessing, and how long human repair takes when it doesn't. Keep the failures and label why each happened: missing evidence, an ignored instruction, malformed structure, wrong reasoning, a tool error, or an unsafe action. Those labels make routing useful later, because a local model often nails the boring cases and only fails once ambiguity or context length crosses a threshold you can actually go find.

test the whole path, not the best-case prompt

Reliability means dependency failure, model retirement, version drift, capacity limits, and recovery time, not whether the demo worked on a good day. Test a provider outage, losing the local host, rate limits, a model pulled from the catalog, your restore procedure, and an explicit fallback, using the same evidence and acceptance criteria while each keeps its own prompt template and API. If the commercial side hands you a native tool call you'd have to build by hand locally, count that integration gap as part of the comparison.

Run the test with real model IDs and dated prices, because a vague comparison is worthless, then keep those specifics out of the permanent rule: catalogs reshuffle, free allowances get cut, and open artifacts get better quantizations on nobody's announced schedule.

when paying wins, when owning wins

The commercial route earns its keep when the provider supplies availability and support your team can't reproduce for the price, not because a dashboard exists. That's no license to route everything to the priciest model either: put a cheaper commercial tier in the comparison, cap output length and reasoning effort, and see if it still clears the gate. Flat subscriptions for bursty capability rarely fit; usage pricing tracks an extraction job's shape better than a seat count, which is most of why a subscription is the wrong instrument here.

The local, open-weight route earns it when offline operation, version pinning, and independence from someone else's account are real requirements, not preferences. Its advantage compounds with steady utilization and a narrow, repeatable task, and shrinks fast when the model barely fits, reloads constantly, needs frequent manual repair, or runs on a box only one engineer understands. Spare hardware isn't free either, not when another workload loses its slot.

The recurring mistake is a silent fallback wired in mid-incident that changes privacy, cost, or behavior without anyone deciding it should. Attach every number to a workload and a time period instead: list price per million tokens and tokens per second are diagnostic, not outcomes for a budget meeting. What belongs on the page is cost per accepted task, p95 latency, failure severity, and how much human attention the route still eats.

wire your own way out before you need it

Put every provider-specific request and response shape behind one narrow adapter, keep prompts and schemas in version control, and keep a job's authoritative state outside any vendor's proprietary thread object. Contract-test streaming, tool calls, structured output, errors, cancellation, and token accounting for every route you support. A local-first cascade works for this, but only if the fallback runs both ways: a fine degraded mode when the commercial API is down, never a quiet channel for sensitive data out when local capacity runs out. Privacy and authorization rules run ahead of capability routing, not behind it.

Re-run the comparison whenever traffic, model versions, prices, licenses, hardware utilization, or review practice changes, because the answer expires even when your code doesn't. Right at ten thousand documents a month, wrong at a hundred thousand; a local model that choked last year might clear the bar now on a better quant, and a free tier that made the plan pencil out can vanish on a Tuesday nobody notices.

There's no universal winner here, and I stopped looking for one. What I'll commit to is the smallest set of routes that clears the quality gate, keeps cost and data movement visible, and fails safely instead of silently, because free and commercial are procurement labels, not an engineering answer. That means I accept slower, more boring iteration on model choice so I don't get paged the night a provider reshuffles its catalog or a local card dies overnight, and I give up the fantasy where someone hands me one model to use forever.