Commercial vs free models for customer-support automation: quality ceiling versus sufficient quality

And that's the piece everyone skips: "commercial" and "free" aren't two models on a shelf, they're three or four procurement deals producing text. A paid API bundles inference with a service, on-call rotation included. A free tier is that same service with a shorter leash. An open-weight model is an artifact, not a service: license and compute are separate bills, and running it yourself trades a token invoice for hardware, power, and capacity planning.

a resolved ticket is the only unit that counts

For support automation, what's worth measuring isn't eloquence, it's a case correctly resolved or safely escalated. Input: ticket history, policy excerpts, account-safe facts, approved actions. The failure that hurts isn't a clumsy sentence, it's a confidently wrong policy statement or an unapproved account action. That reframes cost immediately. A cheap generation that fails validation and eats ten reviewer minutes can end up pricier than the expensive call that finishes once. Judge quality where the application accepts or rejects it, not how the reply reads.

put both routes through the same case, blind

Pick a free-tier or open-weight model behind a constrained workflow, and a commercial API chosen for multilingual reasoning and tool use. Feed both the same evidence and acceptance criteria, but let each keep its own prompt template: pretending the interfaces are identical is where these comparisons quietly cheat. A tool one provider has natively that the other must fake belongs in the count. Score blind, track how often the stronger route changes the outcome:

route: local-7b-support | commercial-tier-a | commercial-tier-b
tokens: input / cached / reasoning / output
latency: queue_ms, first_token_ms, done_ms, timeout
outcome: pass | retry | abstain | human_repair
cost: engineering_hours, incident_hours

Tag every failure by mechanism: missing evidence, an ignored instruction, a malformed reply, wrong reasoning, a broken tool call, an unsafe action. Those labels turn routing into something other than a guess. A local model often nails routine cases and falls over past a threshold of ambiguity you can name.

the point where the expensive route earns its keep

The commercial route wins when hard cases are frequent enough that frontier capability cuts severe errors or review time. Even then, put a cheaper commercial tier, output capped, into the comparison too: "commercial" has more than one price on the menu. Managed capacity earns its keep by removing real work, not by sitting on a dashboard. The free route wins when an evaluated model already clears the bar on most traffic, and the edge compounds with steady utilization and narrow tasks. It evaporates once the model barely fits, keeps reloading, or needs a runtime one engineer understands. Borrowed GPU capacity isn't free either.

Underneath both is the trap of paying for a benchmark lead that never touches a real decision. Price per million tokens and tokens per second are diagnostic, not outcomes, unless attached to a workload and a time window. What matters is cost per accepted case, p95 completion time, failure severity, and the human attention still required after.

build the adapter you'll be glad exists later

Put every provider's request and response handling behind one narrow adapter, keep prompts and schemas in version control, and store job state outside any vendor's thread object. Contract-test streaming, tool calls, structured output, errors, and token accounting: the day you swap is not the day to find mismatches. It's the same instinct behind a local-first cascade, fallback stated rather than assumed. Degrading to local when the API goes down can be fine, but the team should know capability changed. If local capacity runs dry and the system quietly reroutes ticket contents to a remote provider, calling that a fallback is generous; privacy rules run before capability routing.

the trigger I still haven't solved

The honest gap is timing. Traffic, model versions, prices, licenses, and review practice all drift, and a call right at ten thousand tickets a month can be wrong at a hundred thousand. A local model that failed last year might pass now on a better quantization; a free tier that made the plan work might be gone next quarter, one changelog entry nobody reads. What I don't have is a good trigger for re-running this besides something already breaking, so the recheck happens exactly when you have the least patience for it. Calendar reminders don't survive a quiet quarter. I haven't found the fix, and I doubt anyone who claims they have.