Commercial vs free models for tool-using agents: privacy and data control

People have shipped tool-using agents that did the right thing at exactly the wrong moment: approved a refund, closed a ticket, sent a message it had no authority to send. Nobody was watching closely enough. Or the loop behind it burned an afternoon of compute deciding whether to bother. That's the failure under this whole commercial-versus-free argument, and it has little to do with price per token.

"Commercial or free" sounds like one choice. It is at least three, and none lines up with privacy, the axis that should decide who touches a workload. A paid API bundles inference with a service around it. A free hosted tier is usually a limited version of that same service, not something else. An open-weight model is an artifact you run yourself, which doesn't make it free: it moves the bill from a token invoice to hardware, electricity, upkeep, capacity planning, and whoever now owns the pager.

the unit that actually matters (commercialmodels)

For an agent that calls tools, the thing worth measuring is a completed task whose actions stayed inside policy. The input is an objective, tool schemas, running observations, and execution state. Typical failure: a loop running long without converging, a malformed call, or a correct step taken without authority. A cheap model that fails validation and eats a reviewer's time can cost more than a pricier model that just finishes, and a local model that nails easy cases can still fail once ambiguity crosses some threshold. I wouldn't bother pinning any of this to a model ID or a dated price: run those in the experiment, then throw them away.

follow the data before you follow the price

The comparison has to trace every copy of the data and every operator who can see it: collection, transit, retention, admin access, backups, and any downstream tool getting a copy it never asked for. Run the workflow with sensitive fields stripped out, on both routes, against the same evidence and acceptance bar, using each side's documented prompt template and API. If one side hands you a native tool the other must build by hand, count it: real cost, not a rounding error.

Score completion, tool-call validity, step count, recovery, and whether policy ever got crossed. When a run fails, write down the mechanism, not just "it broke":

• missing evidence
• an ignored instruction
• a malformed tool call
• wrong reasoning
• a tool error
• an unauthorized action

Track the same fields on every route so the numbers are actually comparable:

route + exact model revision
input, cached, reasoning, output tokens
queue, first token, completion, timeouts
power draw, idle compute charged to the job
outcome: pass, retry, abstain, human repair
integration, engineering, and incident hours
privacy, license, and fallback limits

where local earns its keep, and where it doesn't

Commercial wins when the contract, certifications, residency, and managed security beat what your operation can promise, not because a bigger model exists somewhere. Put a cheaper commercial tier in the comparison too, and cap reasoning effort first. Managed capacity earns its cost only when it removes real work your team would otherwise do, not because a dashboard exists.

Local or open-weight wins when data has to stay on hardware you control and you can secure the whole lifecycle, not just inference. The advantage compounds with steady, narrow, repeatable traffic, and shrinks once the model barely fits, reloads constantly, or depends on one engineer who understands the runtime. Spare hardware isn't free if another workload loses access while yours runs.

The mistake I see most is calling a workflow private just because inference happened on premises, as if that settles the logs or where the metrics end up. List price and tokens per second are useful for tuning, not deciding, the same trap covered in subscription-wrong-for-ai: a rate card is not an outcome. What matters is cost per accepted task, p95 latency, failure severity, and human attention spent, re-measured whenever traffic, prices, licenses, or review practice changes.

design so today's choice isn't permanent

Put every provider-specific request and response behind one narrow adapter, keep prompts and schemas in version control, and never let job state live only inside a vendor's thread object. Contract-test streaming, tool calls, structured output, errors, cancellation, and token accounting on every route you support, so swapping one out doesn't mean rewriting the agent. Routing between local and commercial by load and sensitivity, instead of picking one side forever, is basically local-first-cascade applied to privacy instead of cost.

Fallback has to be explicit. A local model stepping in when the commercial API is down can be a fine degraded mode, but people relying on the output need to know capability changed. What can't happen is the reverse: local capacity runs out and sensitive data quietly heads to a remote provider because nobody wired a hard stop. Privacy and authorization rules run before capability routing, always.

A setup right at ten thousand tasks a month can be wrong at a hundred thousand, and a model that failed last year might pass on a better quant while the free tier you liked has quietly gone away. So here's the tradeoff I make on purpose, named plainly: I accept the friction of running and babysitting local inference for the workload that has to stay on hardware I control, and give up the elasticity a managed API hands you for free. For that workload, elastic was never the property worth buying.