Architecting for agents that run for minutes, not milliseconds

The first version of every long-running agent is a synchronous endpoint. A request comes in, you run the agent loop inside the handler, and you return the result when it's done. For a quick single-turn assistant this is fine. For an agent that grinds away for ten minutes on a real task, it's a design that's already broken, you just haven't hit the failure yet.

The HTTP request times out long before the agent finishes. The user is staring at a spinner you've asked them to trust for minutes. And the moment the process restarts, deploys, or simply crashes nine minutes into a ten-minute run, all of that work evaporates and the agent starts over from nothing. A long-running agent is a background job that happens to think, and the architecture has to treat it like one.

Off the request path

The first move is to stop running the agent inside the request. The request kicks off a job and returns a handle immediately. The agent runs somewhere else, in a worker, and the client streams or polls for progress against that handle. This sounds obvious written down and is skipped constantly, because the synchronous version is so easy to write and demos so well on a fast task.

Put the jobs on a queue. Submitting an agent task and executing it become two separate things, which buys you a lot at once: bursts of submissions don't overwhelm you because the queue absorbs them, workers pull work at their own pace, and a failed task can be retried by putting it back on the queue rather than being lost. The queue is the spine of the whole thing.

While it runs, the user needs to see life. A ten-minute job behind a silent spinner feels broken even when it's working perfectly, so stream progress, the steps taken, the tools called, partial output, over SSE or websockets. This is half UX and half debugging, because the same stream that reassures the user is the one you'll read when something goes wrong.

Surviving a crash

Here's where it stops being a web pattern and starts being a systems problem. A run that takes minutes will be interrupted eventually, by a deploy, a crash, a preemption, and starting over from the top is both expensive, you pay for all those tokens again, and lossy, the agent may not retrace its steps the same way. So the state of a long run has to be durable. You persist enough of the agent's state, the conversation, the progress, the intermediate results, that a worker can pick it up and resume rather than restart. This is the problem that durable-execution frameworks exist to solve, and even if you don't adopt one, you're solving the same thing they do: making a long computation survivable.

Resuming creates a problem of its own, and it's the one people skip and regret. If the agent crashed right after it sent an email and before it recorded that it sent the email, a naive resume sends the email again. Multiply that across every side-effecting action, every payment, every ticket created, every message posted, and a crash-and-resume turns into a duplicate-everything machine. The defense is idempotency: every tool action with a real-world side effect needs a key that lets you detect "I already did this" so a retry or a resume becomes a no-op instead of a second send. Read-only actions you can retry freely. Side effects you cannot, not without making them idempotent first, and there is no shortcut around that work.

Bounding it, and pausing it

Two more pieces. A long-running agent needs hard bounds, a ceiling on turns, tokens, and wall-clock time, so that a stuck or looping agent fails cheaply instead of running until someone notices the bill. An agent with no budget is an agent that can cost you an unbounded amount of money in a way you'll discover after the fact.

And human-in-the-loop has to be asynchronous to fit this world. When the agent reaches a step that needs approval, it can't block a worker thread for the hour it takes a human to respond. It parks, persists its state, and waits for an event, an approval coming back through a webhook or a queue, that wakes it to continue. This is the same durable-state machinery again, now serving the pause rather than the crash. If your approval flow holds a thread open waiting for a person, you have a timeout waiting to happen.

The reason to take all of this seriously is that none of it is novel. Queues, durable state, idempotent side effects, async human steps, bounded execution, these are the well-worn patterns of any background job system that does real work and has to survive failure. The only thing the agent adds is that the job is non-deterministic and occasionally creative, which makes the resume and idempotency story more important, not less. So the temptation to architect a ten-minute agent like a thirty-millisecond function call is the thing to resist. It's a durable, asynchronous, side-effecting workload. Build it like one from the start, because the synchronous version doesn't fail in the demo. It fails the first time a real run gets interrupted in production, which is to say, soon.