Commercial vs free models for customer-support automation: the real cost per completed task
Support tickets don't arrive on a schedule that matches when your GPU is idle. That alone wrecks most back-of-envelope math before picking a model for customer-support automation, commercial or free.
Three prices wearing one label
"Commercial versus free" hides three deals. A paid API bundles inference with a service: rate limits, uptime, somebody else's on-call rotation. A free tier is the same service with a smaller allowance. An open-weight model is different: an artifact whose license may or may not be open source, its compute still somebody's expense. Local inference trades that token invoice for hardware, power, upkeep, and capacity planning, the same mistake behind the subscription framing of AI cost.
For a support desk the unit worth measuring is a case resolved or safely escalated. Not a token. Not a request. Input: ticket history, policy excerpts, account facts safe to expose, and a fixed list of allowed actions. The failure to guard against is a confidently wrong policy statement, or an unapproved account action, and a cheap generation that trips validation and burns ten reviewer minutes can cost more than a pricier call that finishes. Total cost is inference, idle hardware, engineering, retries, validation, and human review, summed. Use real names and prices in the experiment, but keep them out of the permanent rule: catalogs change.
What to log on every ticket
Give both routes the same evidence and acceptance bar, but let each keep its own prompt template and API. If one side ships a native tool the other must hand-roll, count that cost. Per accepted result, log:
- exact model revision and route taken
- input, cached input, reasoning, output: tokens or watts
- queue time, first-token latency, completion time, timeout rate
- pass, retry, abstain, human-repair counts
- engineering time, plus privacy or license constraints
Score against resolution accuracy, escalation quality, handling time, and policy compliance, and label every failure: missing evidence, instruction failure, malformed structure, wrong reasoning, tool error, unsafe action. A local model often nails routine cases, then fails once context crosses some threshold.
Where the two routes actually diverge
Commercial wins when traffic is spiky, a mistake is expensive, and the vendor removes real work. Put a cheaper commercial tier in the comparison too, output capped. Free or open-weight wins when volume is steady, the task fits a smaller model, and staff on hand can run it. That advantage shrinks once the model barely fits or needs a runtime only one person understands; spare hardware isn't free if another workload needed it. Leaning that way, a local-first cascade is the right shape: local first, commercial only once capacity runs out.
The classic mistake is dividing a GPU's price by theoretical lifetime tokens, or comparing list prices. Attach every number to a workload and a time window. Put a narrow adapter in front of both routes, keep prompts and schemas in version control, and make fallback explicit: a local model failing over to a remote one with account data is a privacy bug. Re-run the comparison as traffic, prices, or review practice shifts; a route right at ten thousand tickets a month can be wrong at a hundred thousand.
Before you commit either way, pull last month's ticket-volume curve and check whether it's steady enough for an open-weight model, or bursty enough that only a metered commercial API survives.