// routing · routing

Building an autorouter: local-first, paid only when it must

$ Jakub Jirák · Jun 15 · 11 min

The local-first cascade is an idea; the autorouter is the code that makes it real. It's the function standing between every task and your wallet: it drains as much as possible to the free local tier, prepares a clean handoff when it must escalate, and steps up the paid ladder (Haiku, then Sonnet, then Opus) only on genuine need. Here's how to build one.

The three decisions a router makes

1. Difficulty pre-classification: where to start. Before wasting a local round-trip on something obviously hard, classify the task and pick a starting tier. A cheap signal does it: task type, prompt size, keywords ("subtle race condition" → start high; "rename this variable" → start local). The classifier can itself be a tiny local call. This is for known difficulty; the ladder below handles the unknown.

2. The verification gate: whether a tier succeeded. A tier produces; an oracle checks. Tests pass, JSON validates, linter is clean, a property holds → done, accept it. Fail → escalate. When you have an oracle this is the whole router; you pay for exactly the capability you provably needed and never a tier more.

3. Confidence, when there's no oracle. Some tasks have no hard check. Then sample the local model a few times. If the answers disagree, that's your escalation signal; agreement is cheap evidence it got it. (Logprob/uncertainty too, where exposed.)

The verifier is the router. With a good oracle, escalation is automatic and you never over-pay. Without one, you're guessing, and the cascade quietly degrades to "hope the cheap model was right."

Preparing the handoff: the step everyone skips

Here's the part that separates a cheap, accurate router from an expensive, sloppy one. When you escalate, do not resend the raw task. Have the local tier prepare the paid call:

Distill the context. Local summarizes the relevant files/logs down to what matters, so the paid model spends its expensive tokens reasoning, not reading. This is context-as-the-scarce-resource applied at the tier boundary.
Carry the failed attempt up. Include what the cheaper tier tried and why it failed: the verifier's output, the failing test, the error. A paid model told "local produced this, it failed like this" solves faster than one starting cold.
Isolate the hard sub-problem. Often only one piece was actually hard. Send that, with the rest pre-solved by local, instead of the whole sprawling task.

This is local-as-preprocessor, and it's a double win: a tight, pre-digested prompt is both cheaper (fewer input tokens) and more accurate (the model isn't distracted by the mess) than handing Opus the raw context. The preparation is the "příprava pro placené modely." The free tier does the cheap context engineering so the paid tier doesn't have to.

The ladder, and what goes where

LOCAL (free)   draft / filter / classify / extract / the verifiable 60–80%
  ▼  escalate on: verify-fail / low confidence
HAIKU ($1/$5)  quick paid reasoning, cleanup of a local draft
  ▼
SONNET ($3/$15) the workhorse: real reasoning, agentic multi-file changes
  ▼
OPUS ($5/$25)   today's ceiling: the subtle bug, the gnarly refactor
  ▼
FABLE           top rung WHEN AVAILABLE, currently withdrawn (see below)

You don't always step one rung at a time. The difficulty classifier can jump straight to Sonnet or Opus for known-hard work. The ladder is for unknown difficulty, the classifier is for known. And note the top rung: Claude Fable 5 is currently withdrawn (why), so Opus is the live ceiling. This is why your router must be provider-abstracted: a tier can vanish overnight by policy, and a well-built router should degrade to the next rung, not break.

The router, in code

TIERS = [local, haiku, sonnet, opus]   # + fable on top when available

def route(task):
    start = classify_difficulty(task)       # skip doomed-cheap attempts
    prior = None
    for model in TIERS[start:]:
        prompt = prepare(task, prior)        # distill context + carry failure up
        result = model.run(prompt)
        if verify(result):                   # tests / schema / linter / property
            log(task, model, "ok")           # feed the escalation log
            return result, model
        prior = result                       # hand the failed attempt to the next tier
    return escalate_to_human(task, prior)    # nothing passed, don't burn frontier forever

Three functions carry the design: classify_difficulty (where to start), prepare (the handoff), verify (the oracle). Everything else is the loop.

Operate it like a system

Measure routing. Escalation rate per tier, cost per task, and which task types fail at which rung, straight from the log (observability). This is the data that tunes classify_difficulty: anything failing local 50% of the time should start at Sonnet.
Mind wasted attempts. A failed local try costs latency you then pay to redo. If a task type fails cheap often enough, classify it past the cheap tiers. Let the router learn its thresholds from the escalation log rather than hard-coding them.
Don't expect cross-tier cache hits. Caches are per-model; the handoff prompt is fresh at each tier. Cache within a tier's repeated prefix, not across the ladder.

The lazy version

You do not start with five tiers, a trained classifier, and a handoff pipeline:

1. Two tiers + a verifier. Local, then one frontier model, escalate on verify-fail. No classifier. This captures most of the win in a dozen lines.

2. Add the handoff (prepare) next: distilling context before escalation is the cheapest accuracy-and-cost win after the basic gate.

3. Add middle tiers and difficulty classification only when the escalation log proves they pay.

That order is the ponytail rule: build the simplest router that moves the bill, measure, and let the log tell you what to add. Most teams find a two-tier router with a good verifier and a decent handoff is 90% of the value. The elaborate ladder is something to grow into, not start with.

#routing#cost#agents