Commercial vs free models for tool-using agents: a hybrid route instead of a winner
Route by task class, not by vendor: cheap and private by default, escalate to a paid API only when a case crosses a threshold you can name. That's the whole rule. Everything else here explains why I stopped trusting "commercial vs free" to answer the only question that matters: whether a tool call finishes correctly and stays inside policy.
People treat "commercial" and "free" as two products. The table actually holds four, wearing two hats. A paid API bundles inference with a managed service around it. A free hosted tier is that same service with a smaller allowance and a shrug where the SLA should be. An open-weight model is an artifact whose license may or may not be open source; the compute is still somebody's line item, moved from a token invoice to a power bill. Local inference trades that invoice for hardware, electricity, patching, and someone on call when a fan won't stop screaming. Flattening all four into "commercial vs free" is how teams pick a vendor instead of a local-first cascade.
The retry tax nobody prices in
For an agent that calls tools, the unit that matters isn't a token count, it's a completed task whose actions stayed inside policy. The input is an objective, tool schemas, the last observation, and the execution state so far. The failure mode isn't subtle: a loop that burns budget without converging, a malformed call the harness rejects, or, worse, a syntactically fine action with no authority behind it. A cheap model that fails validation, needs a retry, or eats ten minutes of a reviewer's time can cost more than the pricier model that finished once. That's the whole economic argument, and most comparisons never get past price per million tokens.
Score the task, not the token
Run the comparison at task level. Label the traffic first, and enforce sensitivity filters before anything gets routed, not after. Give both routes the same evidence and acceptance bar, respecting each side's own prompt template and API. A native tool one side has and the other must reimplement by hand is an integration cost. Count it. Track completion, call validity, step count, recovery from its own mistake, and policy violations. Keep the failures, labeled: missing evidence, an ignored instruction, a malformed structure, bad reasoning, a tool error, or an unauthorized action. A cheap local model will nail the routine case and fall over once ambiguity crosses some line. Use real names and dated prices in the experiment, but keep them out of the permanent rule: catalogs get revised, free allowances vanish, checkpoints get re-quantized past whatever you tested. What should survive isn't the price list, it's the accepted-task math a 99-percent cost architecture runs on.
The routing decision should never be made by a model more expensive than the one you're deciding whether to use.
That's the recurring mistake: reaching for another big LLM before plain rules and a free validation signal get exhausted. Attach every number to a workload and a time window. List price and tokens-per-second are diagnostic, not outcomes. What you want is cost per accepted task, p95 completion time, failure severity, and how much human attention the loop still needs.
Wire in the exit before you need it
Put every provider's request and response shape behind one narrow adapter, and keep prompts and schemas in version control. Don't let job state live inside a vendor's thread object. Contract-test streaming, tool calls, structured output, errors, and token accounting per route. Fallback must be explicit: a degraded local mode is fine when the API is down, as long as the user knows capability changed. Silently routing sensitive data to a remote provider because local capacity ran out is a privacy failure in an outage costume; privacy runs before capability, full stop. Re-run the comparison on a schedule: right at ten thousand tasks a month, wrong at a hundred thousand; a model that choked last year might pass now on a better quant, and a free tier can vanish with a changelog entry.
If I keep exactly one rule, it's the one I opened with: default cheap and private, escalate on a threshold you can name out loud, and never let an expensive model decide whether to spend money on an expensive model.