Commercial vs free models for document extraction: privacy and data control

A model that barely fits in the VRAM you have will reload between every batch of pages, and no prompt tuning fixes that. That decides more "local versus commercial" arguments than any privacy policy does, because a stalled pipeline never reaches one.

Start from the workload. Document extraction has one unit that matters: a validated structured record pulled from a real document. Feed it OCR text or page images, a schema, field definitions, source coordinates, and you get JSON back. The characteristic failure isn't a crash, it's output that parses cleanly and is wrong, a field with the wrong value, a qualifier nobody flags as missing. A cheap generation that fails validation and eats ten minutes of reviewer time can cost more than the pricier call that finishes it once.

three things wearing one name

"Commercial versus free model" sounds like one comparison. It's three. A paid API bundles inference with a service: uptime, support, a contract. A free hosted tier is almost always a rate-limited slice of that same commercial service. An open-weight model is neither, an artifact whose license may not be open source, whose compute is still somebody's line item. Running it locally swaps a token invoice for hardware, electricity, patching, and capacity planning.

List price per million tokens and tokens per second size one run, not a business case alone. Use concrete model IDs and dated prices, but keep them out of the permanent rule: providers change catalogs, allowances disappear, artifacts gain new quantizations.

draw the whole path before arguing about privacy

Privacy here isn't a slogan, it's a checklist: collection, transit, retention, logs, admin access, backups, and downstream tools. Locally that's an open-weight text or vision model plus your own validation code; commercially, a multimodal API or a managed document service. Test both against the real workflow, sensitive fields stripped, using each route's own prompt template.

Map every copy the data makes and every operator who can touch it. A native tool one provider ships that the other side has to hand-roll is real engineering cost. Compare both routes on the same short list:

• exact model or API revision
• input, cached input, output tokens, priced separately
• queue time, first-token latency, completion time
• hardware energy and idle allocation
• pass rate, retry rate, abstain, human-repair rate
• engineering and incident time
• privacy, licensing, and fallback terms

Score both against one quality gate: field-level accuracy, schema validity, abstention over guessing, repair time when wrong. Keep failed outputs and label why: missing evidence, a bad instruction, malformed structure, wrong reasoning, a tool error, something unsafe. A local model can nail routine documents and fail exactly where ambiguity crosses a threshold found only by testing.

what commercial actually buys

Commercial wins when the contract, certifications, residency, and managed security beat what your team secures in-house. That's no license to route everything to the priciest model; put a cheaper commercial tier in the comparison, and cap output length and reasoning effort. Managed capacity earns its cost only when it removes work.

what local actually buys

Local wins when data has to stay on controlled hardware and you can secure the whole lifecycle, not just inference. That advantage compounds with steady, repeatable volume and thins fast on anything ad hoc. It disappears once the model barely fits, needs constant reloading, or runs on a box only one engineer understands. Spare hardware someone else needed isn't free capacity, it's a resourcing decision made silently.

The recurring mistake is calling a workflow "private" because inference happens on a machine in your building. Fix that by attaching every number to a workload and a time window: cost per accepted task, p95 completion latency, failure severity, and how much human attention it still requires.

keep an exit, and know what it costs

Put provider-specific requests and responses behind one narrow adapter, and version-control the prompts and schemas. Keep authoritative job state out of a vendor's thread object you can't migrate off later. Contract-test streaming, tool calls, structured output, errors, cancellation, and token accounting for each supported route.

Fallback has to be a decision someone made, not a default nobody noticed: dropping to a local model when the commercial API is down is fine, but the system should say so. If local capacity is what's exhausted, shipping sensitive documents to a remote provider breaks the premise the system was built on. Privacy and authorization rules run before capability routing.

Re-run the comparison whenever traffic, model versions, prices, licenses, or hardware utilization shift enough to matter. A route right at ten thousand documents a month can be wrong at a hundred thousand, and a setup that failed last year might pass today on a better quant. "Free" and "commercial" are procurement labels; reliable completed work is the actual metric. None of this needs a permanent winner, just the smallest set of routes that clears the quality gate and fails safely. I run something closer to a local-first cascade than a single-vendor bet, and the tradeoff I accept is real: more adapter code to maintain, no vendor line to call when the local half misbehaves. That's the price of keeping the exit open, and I'd rather pay it than discover I don't have one.