with_structured_output is the reason I keep LangChain around

If I had to keep exactly one feature out of the entire LangChain surface and throw the rest in the sea, it'd be with_structured_output. Nearly everything else I could replace in an afternoon. This one quietly does something that's genuinely a pain to do by hand across providers, and it does it the same way every time.

the shape of it

You hand it a Pydantic model. You get back an instance of that model, validated, or you get an error. That's the whole contract from where your code is standing.

class Ticket(BaseModel):
    category: str
    urgency: int
    customer_id: Optional[str] = None

structured = model.with_structured_output(Ticket)

Underneath, it's doing different work on each provider: one backend gets a tool schema and is told to call it, another is dropped into a JSON mode with a response schema, a third gets steered with a grammar. Those mechanisms aren't the same and they don't fail the same way, and the entire point is that I don't have to care which one is running while I write the extraction. My code sees a Ticket with the fields I declared, on every backend.

On the logistics triage work the extraction schema pulled nine fields off each inbound ticket: category, urgency, a customer id when present, a couple of dates, that sort of thing. One Pydantic class, decorated onto whichever chat model I'd built.

the migration that didn't hurt

Here's the part that sold me for good. Halfway through the project we moved the extraction step from one hosted provider to another, mostly for cost and a rate-limit ceiling we kept slamming into on the overnight batch. I changed which chat model object I constructed. I did not touch the schema, the parsing, or the twenty-odd tests asserting on Ticket fields. The seam just held.

The point isn't that structured output is easy on any one model; it's that it stays the same handful of lines in your code while the model underneath is swapped out from under it.

I've written the full provider-swap story elsewhere, but this layer is the reason that swap was a Tuesday-afternoon change instead of a rewrite.

the sharp edges, and there are three

It is not free of teeth. I hit all three of these on the same project.

Optional fields are the first footgun. Mark a field Optional and some models will happily leave it out whenever they're unsure, so your 'optional' field silently becomes 'usually missing,' and the downstream code that assumed it'd mostly be present starts throwing. These days I make fields required with an explicit 'unknown' sentinel far more often than I reach for Optional.

Enums bit me harder. I had a priority enum with four values, and on one weaker model, constraining the output to exactly those four measurably hurt the reasoning that chose the value. The model spent its effort satisfying the schema and picked wrong more often than the same model asked for free text I then mapped myself. Locking the output shape can cost you output quality, and you only learn that by measuring both ways. The Ollama take on this is worth your time if you run local, because the effect gets louder on smaller models.

Third, validation is a promise about shape, not about being right. A Ticket that parses cleanly can still be nonsense. So I wrap the call in a retry-on-validation loop (if Pydantic rejects it, hand the error back and ask again, up to twice), and past that I let it fail loudly instead of papering over it. That pattern, and where it sits alongside real output guardrails, is the gap between 'it validated' and 'it's correct.'

I keep LangChain around for this one feature, and I'd make that trade again tomorrow. The provider abstraction is a nice bonus. This is the part I'd actually grieve.