Commercial vs free models for document extraction: the operational burden
Say you've got a folder of scanned invoices, a twenty-field JSON schema, and two ways to fill it in: pay a multimodal API per page, or run a 14B-class open-weight vision model on a GPU you already own. Neither route is what its label promises: you're paying for a finished record you never touch again, and tokens are just the invoice's unit of measure.
the free label hides three different products
"Commercial versus free" collapses three separate arrangements into one word, and that collapse is where the bad decisions start. A paid API bundles inference with a service: uptime commitments, rate limits, a status page, somebody else's on-call rotation absorbing the 3am page. A free hosted tier is almost always a limited commercial service wearing a different sticker, capped in ways that show up exactly when you need it most. An open-weight model is a third thing: an artifact whose license may or may not qualify as open source, with compute that's still somebody's expense. Local inference doesn't make that expense disappear, it moves it, off the token meter and onto hardware and electricity, plus the capacity planning that now has a name attached.
one accepted record is the only unit that matters
For document extraction the unit that matters isn't a token or a call, it's one validated structured record pulled from a real document. The input is OCR text or page images plus a schema, field definitions, and source coordinates. The failure mode you actually fight rarely looks like a crash: it's JSON that looks fine and is quietly wrong, a total copied from the wrong line, a date missing its qualifier. A cheap generation that fails validation, triggers a retry, and eats ten minutes of a reviewer's afternoon has already lost to the pricier call that landed the field right the first time.
The free artifact never includes the part that costs money later: reliable serving, observability, backups, an owner who gets paged when the endpoint falls over. On the self-hosted side, the candidate is a local model plus whatever validation code you write; on the commercial side, a multimodal API or a managed extraction product. Run the comparison with real model IDs and dated prices, but keep those specifics out of the rule that survives it: catalogs get reshuffled, free allowances vanish, and last month's open artifact has three new quantizations by the time you check again.
what actually goes on the ledger
Compare whole paths, not headline numbers. Track setup time, monthly maintenance, how long failure recovery takes, the cost of testing an upgrade, and what an outage costs while it's happening. Give both routes the same evidence and acceptance bar, but let each keep its own prompt template and API shape: forcing a local model to imitate a commercial tool's calling convention tells you nothing. If one provider ships a native capability the other side has to hand-build, that gap is real engineering cost, not a rounding error.
| Track | Why |
|---|---|
| Route and exact model revision | the build you're grading, not a vendor name |
| Input, cached input, reasoning, output tokens | where the bill comes from |
| Queue, first token, completion, timeout | the latency users feel |
| Hardware energy and idle allocation | the real cost of an idle box |
| Pass, retry, abstain, human repair | the record you're paying for |
| Engineering and incident time | the line spreadsheets tend to skip |
| Privacy, license, fallback constraints | overrides every number above it |
The quality gate is field-level accuracy, schema validity, abstention rate, and how long human repair takes. Keep the failed outputs instead of deleting them, and label why each one failed: missing evidence, an ignored instruction, a malformed structure, bad reasoning, a tool error, an unsafe action. Those labels are what make routing decisions defensible later. A local model will often nail the straightforward cases and only fall apart once ambiguity crosses some threshold you can't see until you've logged enough failures to find it.
who wins, and what happens when it breaks
Commercial tends to win when a team wants a supported API and provider-managed scaling more than control over its own infrastructure. That's no license to route everything to the flagship model. Put a cheaper commercial tier in the same comparison, cap output length and reasoning effort, and see what still clears the quality gate. Managed capacity earns its keep when it removes work your team would otherwise do by hand, not because a dashboard exists.
Open-weight wins when the organization already runs accelerators and wants control over versions, scheduling, or deployment; the advantage compounds with steady, narrow, repeatable work. It shrinks fast when the model barely fits, reloads constantly, needs frequent hand-repair, or depends on a runtime only one person understands. Spare hardware isn't free either, not when another workload loses its slot for it.
The recurring mistake is pricing engineering and incident time at zero when comparing self-hosting against an API, so every number needs a workload and a time period attached, or the comparison is just marketing copy. List price per million tokens and tokens per second are diagnostic, not a business outcome; the scoreboard should read cost per accepted task, p95 latency, failure severity, and how much human attention the pipeline still needs.
Design the exit before you choose a side. Put provider-specific request and response handling behind one narrow adapter, keep prompts and schemas in version control, and keep the authoritative job state outside any vendor's thread object. Contract-test streaming, tool calls, structured output, errors and cancellation, plus token accounting on every route you support: the day you switch is not the day to discover the adapter was theoretical.
Fallback has to be explicit. A local model can be a fine degraded mode when the commercial API is down, as long as whoever's downstream knows the capability changed. What it can't be is a silent overflow valve: if local capacity runs out and the system quietly starts sending sensitive documents to a remote provider instead, call it what it is, a privacy incident with a delay on it. Privacy and authorization rules run before capability routing, full stop. If you're already leaning on a local-first cascade for other workloads, extraction fits into it without new plumbing.
Re-run the comparison whenever traffic, versions, prices, licenses, utilization, or review practice moves: a route right at ten thousand tasks a month can be wrong at one hundred thousand. A local model that failed last year might pass now on a better quant or a rewritten prompt. A free tier that looked permanent can vanish between one billing cycle and the next. None of this needs a universal winner: the durable version is whichever routes clear the quality gate, keep cost and data movement visible, and fail safe, the same cost-per-accepted-unit thinking behind a 99-percent cost architecture, pointed at documents instead of chat.
If I had to keep exactly one rule: never let a fallback cross a privacy boundary the primary route wasn't allowed to cross, no matter how good the deal looks.