Treat summaries as lossy state, not memory

Here's the rule I use: anything that would break the next step if I forgot it goes into a fact list, never into a rolling summary. Summaries exist so a session stays affordable to run, and staying affordable means compressing. Compression drops whatever looked unimportant at the moment it got written, and you can't know in advance which detail that will turn out to be. So I quit trusting summaries with anything load-bearing. They're cheap, lossy state, fine for narrative continuity, useless as a record of anything you'd later cite as fact.

I test this against a long-running agent working a multi-step implementation task. Nothing exotic. No lab bench, no single rig standing in for every machine a reader owns. The point isn't an isolated number, it's a decision you can defend later: keep the setup, change one thing, or throw the run out. That decision needs a job defined before anything else, whether that's interactive chat, code completion, document extraction, or an overnight batch, because a change that helps one of those will quietly punish another.

The compaction tax

Every summarization pass costs something, and the cost is paid in specifics. A model asked to shrink twenty turns into three sentences will keep tone and drop numbers. It keeps "we decided to use the retry queue" and drops the retry count, the backoff curve, the exact file the queue writes to. None of that looks important while the conversation is still live, which is exactly why it goes missing. By the time you need the dropped detail, the only surviving record of it is a paragraph that paraphrased it away three summarization passes back. This isn't a flaw in any particular summarizer. It's what summarization is for, and pretending otherwise is how a working setup quietly stops working for reasons nobody can name.

Keep decisions in their own file

The fix is boring on purpose: store decisions and invariants explicitly, alongside a running list of whatever's still unresolved, kept separate from the narrative summary that carries mood and sequence. This is close to what most serious write-ups on agent memory architecture land on eventually, narrative in one place, facts in another, and frameworks that treat long-term memory as a first-class concept save you from building that split by hand. Use the same inputs for every run and save the launch command next to the result, not buried in a commit message from three days ago. One warm sample tells you nothing. Include a cold start whenever a human will actually hit one, run long enough to expose thermal throttling or queueing, and log task success right alongside the timing numbers. What to log: time to first token, prompt-processing speed, generation speed, peak memory, wall power if that matters for your rig, and whether the task actually passed. Report the median, because that's the case a user lives in day to day, but keep a slow percentile too, since averages erase exactly the pause that makes someone give up on a tool mid-task. A change that shaves time off the median while quietly corrupting the answer isn't a faster version of the old system. It's a different system wearing the old one's name.

"It loaded" is not a finding

The failure mode I run into most is trusting a rolling prose summary to hold onto exact commands and identifiers, plus the acceptance criteria a run has to clear, which is exactly the job it's worst at. Local inference setups are full of configurations that technically run without doing what you asked of them. "It loaded" describes a process, not a result. "The answer looked fine" describes a feeling, not an evaluation. Check the runtime log and the operating system's own metrics instead of trusting whatever flag you believe you passed, because the flag you passed and the flag that actually took effect are not always the same flag. When two runs disagree, change exactly one variable and say out loud what you expect it to do before you run it again. Skip that step and your benchmark stops being a benchmark. It turns into a pile of anecdotes with timestamps attached.

Where the headroom goes

There's a maintenance cost to all of this that's easy to underweight. A fragile five-percent win evaporates the next time a model, driver, or runtime updates, and you won't notice until something downstream breaks in a way that's a pain to trace back. A small script, a handful of representative prompts, and a plain-text result file cover this. I wouldn't build an observability stack for one workstation, and I'd be suspicious of anyone who did. The decision rule underneath all of it is simple: separate durable facts from disposable conversational texture, test the smallest plausible change first, and stop tuning the moment the workflow clears its own latency and quality bar. Leave headroom on purpose, because that spare capacity is what absorbs next week's longer prompt, the background app you forgot was still running, and the model you'll want to try next month.

So next time a run drifts and you're tempted to reread the summary for an explanation, don't. Open the fact file, find the invariant that changed, and check the runtime log before you touch a single setting.