Commercial vs free models for document extraction: licenses, terms, and redistribution
A 12 GB card running a 14B-class GGUF doesn't care what license the weights shipped under. It just runs. Confusing hardware with paperwork stalls a team on an untested route.
"Commercial versus free" hides three comparisons: a paid API bundles inference with a service contract, a free tier is a limited commercial service, and an open-weight model is an artifact whose license may not be open source. Local inference swaps the token invoice for hardware, electricity, and maintenance.
What one extracted record actually costs
The unit that matters is one validated structured record, not a token. Input is OCR text or page images, a schema, field definitions, and source coordinates. The failure mode: valid-looking JSON with a wrong value or a missing qualifier. A cheap generation that fails validation can cost more than the pricier call that finishes once.
Downloading a model doesn't grant every right to use, modify, or redistribute what you build on it. Local pairs an open-weight model with your own validation code; commercial buys a multimodal API or managed service instead. Log license version and acceptable-use terms, hold both routes to the same evidence, and keep real prices in the experiment, not in the rule, since catalogs and free tiers move:
- route and revision, plus input, cached, reasoning, and output cost
- queue time, first token, completion, timeout, and idle allocation
- pass, retry, abstain, human-repair rate, engineering time, and incident load
- privacy, license, and fallback constraints
Where each route actually pulls its weight
Judge outputs on field accuracy, schema validity, and abstention. Label failures by mechanism: missing evidence, bad instructions, malformed structure, wrong reasoning, tool or safety error. A local model handles easy cases fine, flagged only past some threshold of ambiguity.
Commercial wins when contractual support beats artifact obligations, though that's no argument for the flagship: try a cheaper capped tier before paying for capacity that only earns its keep replacing real work. Open or free wins when the license permits your use and deployment control matters, an edge that grows with steady, narrow work and shrinks once the model barely fits or needs one engineer's know-how.
The mistake I keep seeing: a model picked off a price sheet before anyone checks its license. Price and tokens per second are diagnostic, not outcomes; the outcome is cost per accepted record, p95 latency, and human attention still needed.
Keep the exit cheaper than the decision
Put provider-specific handling behind one narrow adapter, keep prompts and schemas in version control, and keep job state out of a vendor's thread object. Contract-test streaming, tool calls, and token accounting for every route, the discipline behind any real local-first cascade.
Fallback has to be explicit: dropping to local when the API is down is fine if users know capability changed, but privacy and authorization rules run first, so don't route sensitive documents to a remote provider once local capacity runs out. Re-run the comparison as traffic, prices, licenses, or hardware utilization shift: right at low volume can be wrong at high.
There's no permanent winner, only the smallest set of routes that clears the quality gate, keeps cost and data movement visible, and fails safely. And I'll admit the adapter-and-contract-tests version of this sounds tidier on paper than it feels the Friday the schema changes and you're holding the pager.