LCEL in anger: pipes, parallelism, and the day I over-composed

LCEL is the pipe syntax, prompt | model | parser, and the first one you write feels like a magic trick. The second month, when you own nine of them chained end to end, it feels like a crime scene. Both of those happened to me on the same client project, so this is a report from the middle of it.

what the pipes actually buy you

The value isn't the cute syntax. What you're actually buying is this: once you express a step as a Runnable and compose it with |, you get streaming, batching, and async on the whole composed thing for free, without hand-writing any of it.

That is not nothing. On the ticket-triage pipeline I could take one composed chain and call .invoke for a single ticket in the request path, .batch for the overnight reprocessing run, and .astream when the frontend wanted tokens as they landed. Same object, three execution modes, zero extra plumbing. Every previous time I'd hand-rolled that, the async version and the sync version drifted apart inside a month, and one of them always had the stale prompt.

The other real win is RunnableParallel. My triage step needed three independent lookups before the model ran — customer tier, recent ticket history, the SLA clock. Fanning them out to run at once instead of in sequence dropped the enrichment from about five seconds to a bit under two on the M2 Ultra I prototype on, and the change stayed declarative: name the three branches, let the runtime run them together. Why that shape matters for streaming and latency is a longer story I told separately.

the Friday I over-composed

Here's the mistake, and it was a good one.

I had a triage flow that grew the way these things grow, one clever addition at a time. Classify, enrich, re-rank, draft a reply, critique the draft, revise, extract fields, format, log. I'd expressed every single seam as a Runnable, so the whole thing became one glorious nine-stage expression stitched together with pipes and parallel blocks. I built most of it on a Friday and felt like a wizard.

The next Tuesday a ticket came out misclassified and I could not find where. The data moving between stage four and stage five was some intermediate dict shape that only existed inside the composition, and you can't set a breakpoint on a pipe. I ended up dumping the chain's whole intermediate state to a file and reading it like tea leaves.

A pipe you can't set a breakpoint inside is a pipe that had better be simple enough to never need one.

what I do now

I refactored it down to three composed stages, and the glue between them is plain Python: functions with names, arguments I can actually see, and somewhere to put a breakpoint. Compose within a stage, where the expression stays short and honest and reads the way prompt-as-code wants it to; use ordinary control flow between stages, where you'll be standing at 1am trying to find the bug.

The rule I landed on is boring and it holds. If I can't read the whole composed expression aloud in one breath and know what flows where, it's too long, and I break it with a function that has a name. Three stages I can explain to the junior on the account. Nine stages I had to explain to myself, with a print statement, on a Tuesday.

LCEL earned a permanent spot in how I build (I'm on the record about why LangChain survived its hype), but it earned that spot as a way to express one clean step, not as a stand-in for a program. The pipe makes a great sentence. It makes a terrible paragraph.