GLM-5.1: Migrating without changing behavior by accident

Z.AI lists GLM-5.1, not GLM-5.2, as its current top coding model: 200K context, up to 128K output tokens, thinking mode, streaming tool calls, and an OpenAI-compatible endpoint. If you went looking for "5.2" because some announcement thread implied it, it isn't there yet. The identifier to put in config is glm-5.1. Check Z.AI's own docs again before you ship anything built on this, because aliases, prices, and availability move after anyone writes about them.

What the number doesn't tell you

A leaderboard score summarizes someone else's tasks run through someone else's harness. It says nothing about your workload. The number that matters starts from what you're already running: the requests, the deadlines, the evidence a task has to produce, the tools it calls, what a wrong answer costs you. Only then do you ask whether GLM-5.1 changes the outcome enough to earn a new route in the gateway. That gap between benchmark and workload is the whole game, and it doesn't close just because the model got bigger (see model migration reality).

Decide what "still works" means before you touch the router, not after. For extraction that's a schema-valid record whose fields trace back to the source document. For coding it's a patch that passes tests and survives review, not one that merely compiles. For research work it's a claim set built from sources you can click through. Fluency isn't the acceptance criterion for any of the three. A model that writes gorgeous prose around a wrong extraction has still failed, just more persuasively.

The actual test is boring: pin old and new versions side by side, replay a slice of real traffic through both, diff the structured outputs, and read the tool calls, not just the final text. Keep the full request intact: system instructions, tool schemas, attached files, sampling and reasoning settings, the output cap. Skip that and you'll draw the wrong conclusion for a dumb reason, a model looks dramatically better or worse depending on whether the harness hands it native tool calling or makes it parse pasted terminal output. Hold the workflow steady, let each provider use its own documented message format, and only then compare.

A model swap touches more surface than most people check before flipping the switch:

• prompt templates and tool-schema versions the harness quietly assumes
• default sampling and reasoning settings the vendor changed without telling you
• which tools the model reaches for, and how carefully it fills their arguments
• the shape of the output itself, verbosity, formatting, how it handles ambiguity
• the failure modes, specifically what kind of wrong answer survives a skim

Log the boring stuff on every run, not just the ones that go wrong:

model id + dated API version
prompt / tool-schema revision in use
tokens: input, cached, reasoning, output
timing: queue, first token, total completion
outcome: pass, retry, abstain, human fix
provider errors and which fallback fired

The reply that looks fine and isn't

The real danger isn't the migration script, it's your own eyes on the output. Swap the alias in production, skim a handful of responses, and they all read fine, because frontier models are extremely good at reading fine. That's the exact property that lets a regression sit invisible until it's expensive. Measure what ordinary code can check instead of trusting your skim: schema validity, test pass rate, whether a cited source exists. Get a reviewer to label concrete defects instead of rating overall quality. Keep the wrong outputs too, a pile of failures tells you more about where to put guardrails than another folder of successes ever will.

A bigger context window doesn't buy what it sounds like it buys. Two hundred thousand tokens still costs prefill time, still lets retrieval noise in, and still requires keeping trusted instructions separate from whatever untrusted document you pasted in. If the API supports caching, put the stable part of the prompt first and keep anything that changes turn to turn away from that boundary. Don't dump a whole conversation history in just because the window can hold it, a structured checkpoint costs less and the model handles it better anyway. The same restraint applies on the way out: a short decision plus its evidence is cheaper to generate and easier to review than three paragraphs the model volunteered because it had the budget to write them.

Tool calling raises its own kind of risk. The model might get noticeably better at picking the right function and filling its arguments, and that's useful, but it changes nothing about what your application owns. You still validate every argument, authorize the caller, cap what a single call can do, handle a duplicate request without doing the thing twice, write down what happened. Anything the model reads back, a web page, a file, a tool's own output, counts as data, nothing in it carries authority just because a smarter model relayed it.

The rule that survives contact with production is unglamorous: a migration ships once contract tests and your own task evaluations pass, with a rollback kept warm, and that rule lives in configuration rather than someone's memory of which model was fashionable last week. Log model identity, phase timings, token categories, tool outcomes, and eval results, and skip retaining sensitive prompt content by default just because you technically can (the local sampling defaults deserve the same scrutiny, see local sampling settings). Keep a cheap, fast lane for routine work with a clear trigger for escalating, because routing everything to the strongest model isn't generosity, it's a bill with no upside, and a cheap tier only earns its keep when retries and review stay under what it saves on inference.

Treat the whole thing like any other dependency bump. Pin dated versions where it matters, watch the vendor's deprecation notices, canary a new alias on a slice of traffic before it touches everything, keep a known-good fallback within reach. Re-run the evaluation set whenever the model, the prompt, the tool contract, or a reasoning default shifts underneath you, not just on upgrade day.

I'd like to tell you this process guarantees GLM-5.1 won't surprise you. It doesn't. All the pinning and diffing buys you is a cleaner failure: when the behavior changes, and eventually it will, you get a diff to point at instead of a shrug.