Three months of Headroom sitting between me and the model

I wrote about Headroom back when I'd been running it for a week and was still mostly quoting its benchmarks. That post holds up, but a week is a honeymoon. This is what three months looks like.

Quick recap for anyone joining late. Headroom sits between the agent and the model and compresses context before it's sent: type-aware, so JSON gets structurally crushed, code gets AST-compressed, logs get deduplicated. In my setup it runs as an MCP server inside Claude Code, and the agent calls it on fat tool outputs before they land in the conversation.

The distrust week

I didn't trust it, obviously. Compressing input to a language model sounds like the setup for a subtle-wrongness generator, and subtle wrongness is the expensive kind.

So for the first week I ran everything twice. Same task, one session raw, one compressed, then I diffed the answers. Boring work. I did eleven of these A/B runs on real tasks: dependency upgrades, a flaky test hunt, two medium refactors. One divergence. The compressed session solved the flaky test faster, because the raw session had burned four turns re-reading a 3,000-line CI log that Headroom had folded into forty lines of structure plus anomalies.

That was the moment the tool clicked for me. Most of what I was feeding the model wasn't information. It was formatting.

Compression revealed how much of my context was never doing anything.

What the stats actually say

Headroom keeps a running ledger, and checking it became my Friday ritual, the way you'd glance at a build dashboard. As of this morning: 12.4M tokens saved since mid-April, averaging 71% on tool output, 84% on JSON blobs. Prose is less impressive, around 30%, which is fair since prose is already dense.

The subtle win is latency. Smaller prompts move faster and cache better. My long sessions stopped hitting the context ceiling mid-afternoon, which used to force a compact right when the agent had finally accumulated useful state. That alone would justify the setup. I went deeper on that failure mode in the context budget post.

The one real failure

Late May, chasing a race condition in a queue worker. The signal was in the timing of interleaved log lines, not the content, the rhythm. Headroom's log compressor correctly deduplicated the repeats and in doing so flattened the exact pattern I needed. The agent confidently declared the logs unremarkable.

The fix was already built in. I just hadn't internalized it: compression keeps a pointer to the original, and the agent can retrieve the uncompressed version on demand. I added one line to my project instructions: when debugging timing or ordering, retrieve raw logs first. Hasn't bitten me since. But it taught me the honest rule. Compression preserves what its author decided is signal, and once in a while your problem lives in what they classified as noise.

Would I keep it

It's the least glamorous layer of my whole token stack, and probably the one that pays off most. It asks for nothing day to day: no prompt changes, no retraining your habits. That's exactly why it survived three months when flashier tools didn't.

If you run agents on real work, your tool outputs are longer than your prompts. They are, go look. Put a compressor in the path and check the ledger on Fridays. Just remember where the raw-retrieval switch is before you need it at 2am.