Migrating from chains to graphs without a rewrite weekend

The triage pipeline spent a year as a fat LCEL chain: classify piped into enrich piped into a RunnableBranch that picked one of three reply chains, about 340 lines all told. It worked, mostly; the sharp edges are documented in lcel-in-anger. What finally forced the move wasn't architectural taste. The client wanted refunds above 200 euro held for human approval, and resume-after-approval is precisely the thing a stateless chain can't do.

The obvious plan was a rewrite weekend: freeze the chain, rebuild it as a proper graph, cut over Monday. I've run that play before and it has never once fit in a weekend. So instead we migrated over two weeks in early May and kept shipping the whole time. Eleven deploys, no freeze.

day one: a graph with one node

Step one was almost embarrassing. Define a minimal state, wrap the entire legacy chain in a single function, make it the only node.

g = StateGraph(TriageState)
g.add_node("legacy", run_legacy_chain)
g.add_edge(START, "legacy")
g.add_edge("legacy", END)
app = g.compile()

Functionally it's pure ceremony. But it proved the plumbing end to end (config propagation, tracing, the deploy pipeline) while the risky logic sat untouched inside code with a year of production behind it. It shipped on day one and nobody noticed, which was the point.

state second, checkpointer third

Days two through four went to state. The chain passed dicts whose shape existed mostly in my head, and the graph forced an explicit TypedDict. My first draft mirrored the chain's internal dict exactly, including a key literally named input, which tells you how much design thought that dict had ever received. I widened it slowly instead of inventing the perfect schema up front. The langgraph-mental-model framing of state as a shared whiteboard helped.

The whole trick of an incremental migration is boring: never let the graph refactor and a behavior change land in the same deploy.

The checkpointer went in on day five, sqlite locally and Postgres in staging, and immediately produced the best bug of the migration. I wired the saver but passed a constant thread_id for every request. Every ticket in staging resumed every other ticket's conversation, and for one surreal afternoon our test tickets accreted into a single immortal thread with opinions about forty different customers. Keying threads on ticket ID fixed it in five minutes; noticing it took two hours. The longer version of that lesson lives in langgraph-checkpointing.

unbundling, one seam per deploy

With state and persistence solid, the mega-node got split at natural seams, one seam per deploy. Classify came out first, because routing wanted to be a conditional edge instead of a RunnableBranch. Then enrichment. Then the three reply chains became three nodes behind that edge. Each split was a small diff: cut where the chain already passed a dict, point the edges at the new node, delete that slice of the legacy function. The legacy node hit zero lines on day nine.

The interrupt (the actual reason for the whole project) landed on day ten and took about half a day once the graph existed. That ratio is normal, and nobody believes it up front.

what I refused to do, and what stayed a chain

Around day six the temptation got loud: everything's moving anyway, so why not redesign properly with a supervisor node, subgraphs per ticket category, fan-out enrichment. I wrote all five ideas into a NOTES file instead of the codebase. One of them, fan-out enrichment, got built in June because latency numbers earned it. The other four are still in the file, unmissed. Resisting the mid-migration redesign is most of why we never stopped shipping.

And two pieces never became nodes at all. The reply drafter is still prompt into model into structured-output parser: a chain, living inside a node, doing the one thing chains are genuinely great at. Same for the retriever that enriches tickets with order history. The graph doesn't care what runs inside a node, and exploding a perfectly good chain into micro-nodes is redesign theater.

Two weeks and eleven deploys later, the rewrite weekend had quietly never happened.