Find the token leaks in an agent loop

Ten to thirty tool calls is a normal life span for one task in a local coding agent, and each one resends the tool schema, the accumulated history, and whatever the last command dumped to stdout. None of that is optional under a stateless API: the model has no memory of its own, so the conversation gets rebuilt on every hop. That rebuild is where the token budget quietly disappears before any real work happens.

The tax you pay before the task starts

Split the input into categories instead of staring at one big number. The schema is fixed and repeats verbatim, tool output balloons with what the command returned, and history grows on its own unless something prunes it. Fix one model, one prompt set, and one exact server launch command before comparing anything, and write the artifact name and prompt template into the record: the two things people forget, and what sinks a comparison later. Time to first token is loading and prompt processing, steady token rate is decoding, completion time is what a person waits for. Keep a short field list next to the numbers:

• artifact, runtime, and launch flags
• the fixed workload and its input set
• cold start, warm start, p50, p95
• peak memory, and wall power when it matters
• the decision this run produced, and who owns it

A benchmark without a decision is trivia. A setting nobody owns becomes folklore.

Nothing throws an error until it's expensive

The common mistake is trimming the user prompt while megabytes of terminal output ride along every turn, because the system still answers and a plausible answer looks like success. Local inference fails softly: a model partly offloads to CPU, a cache misses without complaint, swap creeps upward, a queue keeps a dead request alive, or a fallback quietly changes where a request is allowed to go. None of that throws an error. Watching runtime logs and OS counters while a test runs, not after, is most of what agent observability means once stripped of branding. Change one variable at a time, repeat enough to separate a durable improvement from a lucky sample, and read the output rather than trusting a latency number. If a change makes one task worse, weigh that cost against the win. A fragile optimization needing manual repair after every driver update isn't free. A boring setup rebuilt from a service file beats a clever one you can't, especially once your loop sits inside the wider agentic architecture patterns debate.

Margin is not the same as proof

The rule that has held up for me: measure the loop, not the opening request. Less satisfying than picking the biggest model or newest card, but it leaves a stack whose limits are visible, not discovered at two in the morning. Stop tuning once the workload hits its target with headroom left. That margin isn't waste. It absorbs a longer document, one more user, a hot afternoon, or the next runtime release that moves the baseline under you. What I haven't found a clean answer for is how much history you can drop without the agent losing something it needs three turns later. You find out by watching it get confused, and by then the experiment already cost the turns you meant to save.