GLM-5.1: Long context without the token landfill
GLM-5.1 ships with a 200K token context window and a 128K output cap, and every input token gets scored against the query during prefill whether it turns out to matter or not.
Z.AI's current docs list GLM-5.1, not GLM-5.2, as the lead coding model: agentic coding, long-horizon work, thinking, streaming tool calls, OpenAI-compatible access. The identifier is glm-5.1. Test what's documented, not what's rumored, since limits move.
what 200K tokens buys and what it doesn't
Decide what counts as success before the first prompt. A schema-valid record for extraction. A patch that passes tests and review for coding. A claim set with traceable sources for research. Fluency doesn't count.
Test it: bucket prompts by length, drop known evidence at several positions, measure recall, and compare against loading the full window. Keep the whole request intact, system instructions, tool schemas, files, sampling and reasoning settings, the output cap, since native tool access alone can make a model look sharper than pasted terminal output on the identical prompt.
| what you track | why |
|---|---|
| model ID + dated API version | pins the comparison |
| input, cached, reasoning, output tokens | separates the bill from the work |
| queue, first token, total time | prefill cost hides here |
| pass, retry, abstain, human repair | the real outcome |
| provider error and fallback route | what happened on failure |
when the haystack eats the needle
The expensive mistake: treat the advertised maximum as an architecture, then find the fact you needed between two duplicates the model skimmed past. Hard to catch, since a wrong answer reads as confident as a right one. Measure what code can verify, label the defects by hand, and keep the failures. A page of failures routes traffic better than a page of successes.
Long context doesn't remove prefill time, retrieval noise, or the need to keep trusted instructions separate from documents pulled off the web, which are data, not authority. Cache the stable prefix first, keep volatile metadata off the cache boundary, and send a checkpoint instead of full history when one will do. Keep the output short too: a decision plus evidence beats a tutorial nobody asked for. Better tool selection doesn't excuse you from validating arguments, authorizing the caller, capping side effects, and logging what ran.
put the rule where it survives the release notes
Retrieve the smallest evidence set that answers the question; reserve the full window for tasks that prove they need it. Put that in configuration, not in someone's memory of which model is fashionable. Log model identity, phase timings, token categories, and eval results, and skip the sensitive prompt content by default.
Keep a cheap, fast lane for routine work and an escalation condition. Save the strongest model for tasks that benefit; a discount tier stops being a discount once retries and review eat the savings. Treat migration as routine maintenance: pin dated versions, watch deprecation notices, canary new aliases before full traffic, keep a trusted fallback. Re-run the local eval set whenever the model, prompt, tool contract, or reasoning default changes, which happens more often than the changelog admits.
What I haven't settled: how much recall has to drop, and at what position, before a task earns the full window. I move that line by watching the retry queue, not by any number I can defend. I'd rather have the number.