Commercial vs free models for document extraction: long-context economics

Long context is oversold for document extraction. That's the claim. The qualifier: for repeatable extraction jobs, a huge window buys insurance against a scenario you'll rarely hit, and you pay for it every month regardless.

three bills hiding inside one word

Commercial versus free sounds like one comparison. It's three, and blending them is how budgets go sideways. A paid API bundles inference with a running service: uptime, rate limits, somebody else's pager. A free tier is that same commercial service with a limit that moves without notice. An open-weight model is an artifact instead: its license may or may not clear the open-source bar, and running it is never free, just moved to your power bill and maintenance hours. Local inference doesn't kill the cost, it moves it onto a different ledger: hardware, electricity, maintenance, capacity planning.

the record is the unit, not the token

For document extraction you're buying one validated record out of a real document: OCR text or page images in, a schema, field definitions, source coordinates, correct JSON out or nothing. The failure that matters isn't a refusal, it's JSON that looks fine and is quietly wrong, a missing qualifier nobody catches until something downstream breaks. A cheap call that fails validation, gets retried, and costs a reviewer real time can cost more than a pricier call that finishes once.

The number worth chasing isn't the max tokens either side accepts, it's evidence density, how much of what you hand the model per token helps it land the field right. Bucket prompts by length and run retrieval-and-compression against dumping the whole document in. Score both on accuracy and validity, then on abstention and repair time. Label every failure: missing evidence, a bad instruction, malformed output, wrong reasoning, or a bad tool call. A local model that nails the easy majority and only trips past a specific context threshold is telling you where to route it.

the adapter, not the model, is what you're actually buying

Commercial capacity earns its cost on rare jobs that need real context and don't justify owning hardware. That's not a license to route everything to the biggest model: put a cheaper commercial tier alongside it, cap its output and reasoning effort, and let it earn its keep by removing real work, not by existing.

The local or open-weight route earns its keep on routine, narrow, high-volume work that fits a controlled window after retrieval and a reused stable prompt prefix, close to a local-first cascade. It shrinks fast once the model barely fits, needs constant babysitting, or depends on one irreplaceable person, and spare hardware isn't free if something else needs it.

The recurring mistake is sending every document because the provider happens to accept it. Pin every number to a workload and a time period: price per token and tokens per second are diagnostic, not outcomes, more like the accounting behind a real cost architecture. Track cost per accepted task, p95 latency, and failure severity. Put one narrow adapter behind every provider's request shape, prompts and schemas in version control, job state outside any vendor's thread object.

Fallback has to be explicit: a local model covering for a down API is fine if someone notices capability changed, and shipping sensitive pages to a remote provider because local capacity ran out is not. Re-run the comparison whenever traffic, prices, or practice moves, a choice correct at ten thousand tasks a month can be wrong at a hundred thousand. Free and commercial are procurement labels. Reliable, completed work is the only metric that counts.

So don't settle this in the abstract. Pull last month's extraction jobs, sort the failures by why they failed, and check how many that choked on your local model would have passed on a bigger prompt budget alone. That number tells you whether you're short on context or short on retrieval.