Commercial vs free models for coding assistants: long-context economics

Bigger context windows do not save you money. They let you postpone the discipline that would have, and that debt collects interest.

A paid API bundles inference with a service. A free tier is a rationed slice of that service. An open-weight model is an artifact you run yourself, trading the invoice for hardware and power, and none of the three is inherently cheaper.

For a coding assistant the unit that matters is a change a human accepts and the test suite passes: a plausible patch that compiles locally but violates a hidden contract. A cheap generation that trips review and costs a reviewer ten minutes is the expensive one.

A patch that compiles is not a patch that passes

Bucket prompts by length, then compare full context against retrieval plus compression on identical evidence and acceptance criteria. Respect each provider's own template and API; if one side ships a native tool the other must reimplement, count that cost too.

Track route and model revision, the token split across input, cache, and output, and latency at every stage, judged against tests and diff review, not a benchmark score. Tag every failure by mechanism: missing evidence, malformed output, wrong reasoning, or an unsafe action. Those tags are what let you route work later instead of guessing.

Extreme context is occasional; routine is not

The commercial API earns its bill when a task is occasional and genuinely needs a window past what you'd sanely buy hardware for, when managed capacity absorbs the spike. That's a narrow case: put a cheaper tier in the comparison too, and see if the expensive option earns its premium.

The local or free route earns its keep when routine prompts fit a controlled window after retrieval and reused stable prefixes, and the advantage grows with steady use and narrow, repeatable tasks. It erodes once the model reloads constantly or needs manual repair past a threshold worth naming, and spare hardware isn't free if another workload loses its slot.

The mistake on both sides is the same: stuffing in every document the window accepts because it fits, instead of pricing the workload. Price per token and tokens per second suit tuning copilot-token-diet-style pruning, not deciding who wins; what decides is cost per accepted task, latency, failure severity, and human attention required.

Write the adapter before you need the exit

Build so changing your mind later doesn't require a rewrite: one narrow adapter for provider-specific handling, prompts and schemas in version control, and job state you own, never inside a vendor's thread object. Contract-test streaming, tool calls, structured output, and token accounting for every route you support.

Fallback has to be a decision made on purpose, not a side effect of a timeout. A local model quietly taking over as a local-first cascade when the commercial API drops is fine, as long as the user can tell capability changed. If local capacity runs out, don't ship sensitive data to a remote provider to keep answering: privacy rules run before routing gets a vote.

Re-run the comparison as traffic, prices, licenses, or review practice change: a route right at low volume can be wrong at scale, and a model that failed last year may pass now on a better quant. Free tiers vanish too. There's no universal winner; the durable design is the smallest route set that clears your quality gate, keeps cost and data movement visible, and fails safely when a provider doesn't.

So before the next context-window pitch lands in your inbox, skip the token count. Pull last month's routing log, find the ten most expensive reviewed changes, and check how many a bigger window would actually have prevented.