Commercial vs free models for coding assistants: the real cost per completed task

A team drops its commercial coding assistant for a free or local model, watches the invoice line hit near zero, and calls it a win. Months later two engineers spend an hour a day fixing patches that compiled clean but broke a contract the model never saw, and nobody adds that hour back into the spreadsheet that declared victory.

Free versus commercial is four things: a paid API bundling inference with a service you don't run; a free hosted tier, a limited commercial offer someone else subsidizes until they stop; an open-weight artifact whose license may not cover your use, its compute is still your line item; and local inference, trading a token invoice for hardware, power, and upkeep.

For a coding assistant the unit that matters is a reviewed change that clears the repo's test suite, not a token count. The worst failure is the plausible patch: it compiles, looks right, and breaks a contract nothing in the prompt showed, and a cheap generation that fails validation and eats ten minutes of review time can cost more than an expensive one that lands first try.

The ledger nobody keeps

Total cost is inference, idle hardware, engineering time, retries, validation, and the hour spent fixing what the model got almost right. Put a local open-weight coder or capped free tier on one side, a metered commercial API on the other, and run both against the same evidence and acceptance bar. Use today's model and price, since both go stale: free tiers get pulled, quantizations improve, catalogs churn. What stays useful is the shape of what you log per result.

route and exact model revision
input, cached input, reasoning, output tokens
queue time, first token, completion, timeout
hardware energy and idle allocation
pass, retry, abstain, or human repair
engineering and incident time
privacy, license, fallback constraints

The quality gate is tests, static checks, real diff review, and measured correction time, not a vibe. Keep every failure and label why:

• missing evidence: the model never saw the file
• instruction failure: it ignored something explicit
• malformed structure: output that won't parse
• wrong reasoning: plausible logic, wrong conclusion
• tool error: a call failed or returned garbage
• unsafe action: something you'd never auto-apply

Those labels let you route work instead of picking one winner. A local model can be flawless on repetitive tickets and fall apart once ambiguity crosses a line you can only draw from a stack of labeled failures.

Volume decides more than capability

The commercial route wins when traffic is spiky, a wrong answer is costly, and the provider removes real operational work you'd otherwise carry. Put a cheaper commercial tier in the same comparison and cap output length before assuming it earns its keep. Managed capacity earns its price by removing work you'd otherwise staff, not because a dashboard exists.

The free or open-weight route wins when load is steady, the task fits a smaller model, and you have the hardware and staff to run it. That edge evaporates once the model barely fits in memory, reloads constantly, or depends on a runtime only one person understands, and spare hardware isn't free if another job loses its GPU when your assistant needs it.

The mistake I keep seeing: dividing a GPU's price by some theoretical lifetime token count, or comparing nothing but input price per million tokens. Both are trivia. Tie every figure to a real workload and time period, and the numbers that matter become cost per accepted task, p95 latency, failure severity, and remaining human attention, which is closer to what copilot-token-diet covers.

None of this holds if both routes are welded into the codebase. Put provider quirks behind one adapter, keep prompts and schemas in version control, and store job state outside any vendor's thread object; contract-test streaming, tool calls, and structured output, because that's where just-swap-the-model quietly breaks.

Fallback must be explicit: a local model is a fine degraded mode when the commercial API is down, but if local capacity runs out, silently shipping sensitive data to a remote provider defeats the point of running local. Privacy rules run before capability routing, close to the case for a local-first-cascade rather than one fixed route.

Re-run the comparison whenever traffic, prices, licenses, or review practice moves: a choice right at low volume can be wrong at ten times the volume, and a free tier can vanish overnight with nothing but a changelog entry. There's no universal winner, just the smallest set of routes that clears your quality gate, keeps cost and data movement visible, and fails safe.

So don't start by pricing anything. Pull last month's logs on retries, reviewer time, and failed patches, sort them against that six-label list, and see how many of those mistakes a cheaper route would have caught.