Tool calling on local models: usable, with an asterisk

In the second week of June I gave one small agent three different brains and kept score. The agent babysits my homelab: five tools (disk_free, tail_log, service_status, restart_service, list_backups) wired through Ollama's tool-calling API, plus a script of 40 canned requests pulled from things I actually type, like 'why is jellyfin down' and 'is the array filling up again'. The brains: Qwen3 8B on the 3090, a 30B-class Qwen coder MoE on the same card, and the frontier model my client work already pays for.

The part that surprised me

The 8B picked the right tool on 33 of 40 requests. The 30B managed 38. The frontier model went 40 for 40, which you'd expect, and which is also the least interesting number of the three. The interesting one is 33, because I wrote small-model tool calling off as a toy in 2024 and kept believing that about two years too long. For a narrow agent with a handful of well-named tools, an 8B is now plainly usable.

Arguments are the softer spot. The 8B would choose tail_log correctly, then ask for 10,000 lines, or pass a container name where a systemd unit belongs. Picking a tool is pattern matching. Filling its arguments is where a small model spends its entire IQ budget.

That gap between choosing and filling is most of the story of local tool calling.

The cliff past eight tools

Then I got greedy. I merged my network toolset into the same agent, 13 tools total, and reran the script. The frontier model didn't blink. The 30B slipped to roughly nine in ten. The 8B fell off a cliff: 24 of 40. Most misses were a plausible neighbor, service_status where tail_log should have gone. There's no gentle slope here. Somewhere past eight tools the list stopped reading as a menu and started reading as noise, and three reruns put the knee in the same place every time.

My mistake wore a name tag

A confession before I blame any model: my ugliest early numbers were self-inflicted. The first toolset had get_status and service_status sitting side by side: one for the host, one for a unit, and the 8B alternated between them close to randomly. I spent a weekend convinced the whole feature was a gimmick and drafted most of a grumpy post saying so. Then I renamed get_status to host_health and reran: 29 of 40 became 33, with no other change. The grumpy post stays unpublished.

Design tools for the dumbest caller you'll ever ship, and every smarter model inherits the favor for free.

The asterisk, spelled out

Two behaviors stay stubbornly frontier-shaped. Parallel calls: ask the 8B for disk and services in one turn and you usually get one call, occasionally the same call twice. Long chains: by step five of a diagnose-restart-verify sequence, the small models re-check things they already know or quietly lose the goal. That matches what I found wiring open coding CLIs to local backends in an earlier piece. Single hops fine, ten-step plans no. So my split follows squeeze the local tier: the 8B owns anything under three steps, and real planning goes upstream.

What moved my numbers wasn't swapping models. It was tool design, and the long version lives in designing agent tools. The short version:

• names that share no words with each other
• three parameters at most, enums wherever possible
• descriptions that say when to call the tool, with one example
• errors written as instructions, so a failed call tells the model what to try next

With those rules in place, the five-tool agent on the 8B has run the homelab since early June, and I now open a terminal for routine chores maybe twice a week instead of daily. Usable, with an asterisk. The asterisk is load-bearing: the moment the job stops being narrow (more tools, longer chains, parallel work), the honest move is still a bigger model, and pretending otherwise costs evenings I've already spent for you.