← all posts
// models · grok

Grok 4.5: A coding workflow that survives the demo

Grok 4.5 will not make your code review better. It might make what shows up at review slightly better, and that is the only claim worth testing before you touch a router config over a new model name.

The identifier to test is grok-4.5: it folds into chat, files, connected tools, and the multimodal, current-information handling every frontier lab now ships by default. Fine. None of that says whether it earns a place in a pipeline that already has a model doing the job. Test it against the requests your team actually gets, not the ones a benchmark author picked.

Grade the patch, not the paragraph

Decide what counts as a win before you send the first prompt. For extraction that is a schema-valid record whose fields trace back to the source document. For coding it is a patch that passes tests and survives a human reviewer. For research it is a claim set you can walk back to citations. Fluency does not qualify for any of the three, and a model that writes confident prose about a bug it never fixed is worse than useless because it reads like it fixed it.

Run a fixture set of real bug fixes, real refactors, and questions about code the model has never seen, with the same tool permissions and acceptance checks every time. Keep the whole request intact: system instructions, tool schemas, files, sampling or reasoning settings, output limit. Give one model native tools and another a pasted terminal transcript and you are comparing harnesses, not models. Track model identity and dated API version, token counts by category, time to first token and to completion, and how often a task passes outright versus needs a retry, an abstain, or a human finish.

The mistake that actually costs money is grading coding ability on an isolated greenfield prompt that never has to open a real file or repair a test it broke. Frontier output is fluent enough that this failure mode hides in plain sight. Keep the wrong answers: they tell you more about where to route work and what guardrails to add than another folder of things that went right. Long context does not fix this either. More tokens does not remove prefill time, retrieval noise, or the job of keeping instructions separate from whatever a fetched web page or email happens to say.

A more capable model is not a substitute for a narrower interface.

The rule goes in config, not conversation

Route coding work by measured outcome on your own repositories, and require passing tests plus a diff a person can actually read before anything ships. Put that rule in configuration, so it survives the next engineer who liked a different model last month. Keep a cheap, fast lane for routine changes and a clear trigger to escalate to the strongest tier: a discount model earns nothing if its retries and review time outrun what it saved on inference.

Treat every model swap as migration, not an upgrade: pin the dated version where behavior matters, canary a new alias on a slice of traffic before it touches everything, and keep a known-good fallback ready to take the load back. I accept the extra weeks that pinning and canarying cost, on purpose, in trade for never debugging a silently reshaped model at two in the morning. That is the trade. Grok 4.5 does not change it.

#grok#xai#api#evaluation