Advanced agent architecture: context is the scarce resource

The foundational post covers the loop, tools, and evals. The patterns post covers topologies. This one is the next layer down: the discipline that separates a demo agent from one that survives a long, real task. One subject, really. The context window is the scarce resource, and managing it is the whole advanced game.

Why context is where it's won or lost

Three things go wrong with a naive, ever-growing context, and all of them get worse the longer the agent runs:

• It fills with stale junk. Old tool outputs, completed sub-tasks, dead-end reasoning: all still in the window, all still being re-read and re-billed every turn.
• The signal gets buried. Models reason best about the start and end of a long window and skim the murky middle ("lost in the middle"). Bury the key function at token 400,000 and it gets skimmed.
• You re-send the world. Every turn re-processes the whole transcript. Without intervention, cost and latency grow with the square of the conversation.

Everything below is a tool for the same job: keep the window small and relevant.

The advanced toolkit

Compaction. When the conversation approaches the window limit, summarize the earlier history (server-side, where supported) into a compact block so the loop can keep going past the raw window. One detail matters here: preserve the compaction block in your message history by appending the full response, not just the text. Skip that and you lose the summarized state without noticing.

Context editing. The complement to compaction: instead of summarizing old content, prune it. Clear stale tool results and completed thinking blocks past a threshold. Compaction condenses; editing removes. Long tool-heavy loops want both: edit the dead weight, compact when you near the limit.

Memory tiers. Be honest about what "memory" you need. Most agents need three plain layers, not a vector database:

• Short-term: the conversation, in the window.
• Scratchpad: a file for the current task's working notes.
• Durable: cross-session facts worth persisting (conventions, decisions). (Software as memory is this layer, governed.)

Reach for a vector "memory" store only when a user complaint (not a roadmap) proves you need it. The classic over-build is a long-term memory layer shipped six months early and maintained forever.

Sub-agent context isolation. The single most powerful advanced move. Fan out work to subagents, each with a clean, scoped context, and have the orchestrator keep only their summaries, not the raw firehose. Searching a big codebase or exploring three approaches in parallel shouldn't pollute the main thread's window. This is the orchestrator-worker topology seen through the context lens: isolation is the point, parallelism is the bonus.

Programmatic tool calling. When the agent needs many sequential tool calls or produces large intermediate results, let it write a script that calls the tools instead of round-tripping each one through its context. The intermediates live in the running code; only the final output returns to the model. A 50-result search that gets filtered down to 3 in a loop costs you 3 results of context, not 50.

Tool search. Don't load 100 tool schemas into every request. Let the agent discover the few relevant tools on demand, and append them rather than swapping so the prompt cache survives. Big fixed tool sets are a context tax you pay on every turn.

Caching is part of the architecture, not an afterthought

The stable prefix (system prompt, tool definitions, the codebase) is a design surface. Keep it byte-stable so it caches: no timestamps in the system prompt, deterministic tool ordering, volatile content after the breakpoint. An advanced agent treats "what is the cacheable prefix" as a first-class architectural question, because it's the difference between linear and near-flat cost (the caching lever).

The lazy advanced stack

Advanced doesn't mean maximal. Turn things on as the task demands them:

• Compaction on for anything that might exceed the window.
• Context editing for long, tool-heavy loops.
• A file scratchpad, and a markdown memory directory for cross-session facts.
• Subagents for fan-out, keeping the orchestrator's window clean.
• Caching protected: the prefix is sacred.
• Add programmatic tool calling, tool search, and a real memory store only when a profiler proves the simpler stack ran out of room.

The throughline: you don't make an agent smarter by giving it more context. You make it smarter by giving it the right context and ruthlessly evicting the rest. The frontier model is rented; the context discipline is the part you own, and it's where the hard problems are actually solved. The natural next step is spending that context budget across model tiers: the local-first cascade.