Commercial vs free models for customer-support automation: long-context economics

Route by evidence, not size.

That's the rule before any meeting about which model runs support automation: routing decides the cost, not the model card. Evidence that fits a local window after retrieval goes on the cheap or free route. Evidence that needs the full sweep of ticket history and policy goes to the commercial API, and you stop apologizing for the bill. Most arguments for more context mean nobody built retrieval yet.

Commercial versus free hides three comparisons: a paid API bundles inference with service, a free tier is the same thing cheaper, and an open-weight model is an artifact with its own license while the compute is still someone's expense. Local inference swaps a token invoice for hardware, power, and maintenance.

The unit that matters is a case resolved or safely escalated, not a token processed, built from ticket history, policy, account facts, and approved actions. The failure that costs you is a confident wrong policy answer or an unauthorized account action, and a cheap generation that fails validation or eats ten reviewer minutes can cost more than the pricier call that finishes once.

the number that beats price per token

Test complete paths, not price sheets. Bucket prompts by length, measure prefill time and answer quality per bucket, and put retrieval-plus-compression against a route that stuffs the full window and hopes. Give both the same evidence and acceptance bar, and count a tool call one side has to hand-roll as an integration cost, not a footnote.

Track the model revision, the token split across input, cached input, and output, latency from queue to completion, and the outcome: pass, retry, abstain, or human repair, graded against resolution accuracy, escalation quality, and policy compliance, with failures named as missing evidence, wrong reasoning, a broken tool call, or an unsafe action.

The recurring mistake is sending every document because the window will swallow it. Attach every number to a workload and a time period.

Cost per accepted task beats price per million tokens every time you have to defend a budget.

Price per million tokens and tokens per second are diagnostic; what matters is cost per accepted task, p95 latency, and failure severity.

build the off-ramp before you need it (commercialmodels)

Commercial earns its keep when a real slice of tickets needs genuinely extreme context and managed capacity absorbs a rare peak without you owning hardware for it. Put a cheaper vendor tier in the same test; the premium pays off when it removes work your team would otherwise do, not because the sales deck says enterprise-grade.

The free or open-weight route wins when routine tickets fit a controlled local window after retrieval and a stable prompt prefix gets reused. That edge grows with steady utilization and narrow, repeatable ticket types, and shrinks once the model barely fits or depends on a runtime only one engineer understands. I wouldn't bother standing up local inference for a desk closing a handful of tickets a day: hardware and on-call burden eat the savings first.

Build the exit before you commit: one narrow adapter for provider handling, prompts and schemas in version control, and state you own rather than a vendor's thread object, the shape a local-first-cascade takes. Contract-test streaming, tool calls, and token accounting for every route you support.

Fallback must be explicit: say so when a local model becomes the degraded mode, and never let exhausted local capacity quietly push tickets to a remote provider; privacy checks run before capability routing, always.

Re-run this whenever traffic, prices, licenses, or review load moves: a choice that's right at ten thousand tickets a month can be wrong at a hundred thousand, and a free tier you depended on can vanish overnight. Free and commercial are procurement labels, not engineering ones, the same trap subscription-wrong-for-ai describes from the billing side.

So before the next procurement conversation: pull a month of closed tickets, bucket by how much evidence each one needed, and see what share fits a retrieval-fed local window without a retry. That number tells you which side of the rule you're on.