Give an LLM the diff plus just enough neighborhood

Point a review model at a diff alone and it invents problems that were already true last Tuesday, because it never saw the contract those lines must satisfy. Point it at the whole repository and you've burned the context budget before it reaches the line you changed. Treat the diff as an entry point into a small neighborhood of code, never the whole meal, never an excuse to skip retrieval.

Start with the work waiting on the API's other side, not a model name or a throughput figure: changed lines, their contract, callers, plus the regression tests, all in view together or the review degrades. That machine rarely runs alone: a private chat, a coding loop with its own token budget worth watching, plus a nightly extraction job can share one box, wanting different things.

Baseline before you touch a single flag

Before reconfiguring anything, capture one baseline: one named model, one fixed prompt set, the exact server command, logged beside the model artifact and prompt template. Both get lost fast enough to invalidate next month's comparison.

Then build the review pass: start from the diff, retrieve the symbol definitions and tests that touch it, then ask for findings tied to specific lines. A code graph built for exactly this handles retrieval. Measure the whole path: first-token time is loading and prompt work, steady rate is decoding, completion time is what the person waiting feels. Note peak memory, plus queue delay and card power, when either would change the decision.

Log the artifact and runtime, the launch flags, the workload and its fixed inputs; cold and warm p50/p95; peak memory and wall energy; the failures and abstentions you saw; then the decision itself, who owns it, when you'll retest. Skip that part. You'll regret it. A benchmark with no decision is trivia; a setting nobody owns is folklore by next quarter.

Confidently wrong doesn't set off an alarm

The tempting shortcut is dumping every touched file into the prompt for a generic architecture essay. It survives because the output reads like a real review.

Local inference fails quietly: partial CPU offload unnoticed for weeks; a cache that misses without logging; swap that quietly grows; a queue serving a hung-up client; a fallback path that changes where your code gets sent, the one I'd worry about most since running locally was the point. None of it throws an error; all of it changes what your numbers mean.

Watch the logs and OS counters while it runs; change one variable at a time unless comparing whole configurations; run it more than once, since one sample proves nothing. Read the output too: a case made worse costs something, and that belongs beside the gain, not a footnote. Manual repair after every driver update isn't free, just deferred. Favor a boring setup you can reproduce from a service file or script, and keep the hashes and commands that make reruns comparable.

Byte count was never the thing worth optimizing

The rule worth keeping: build review context around what changed in behavior, not how many bytes moved. Less flattering than the biggest model or newest card, but it produces a system whose limits you can see. A visible limit gets routed around or priced in; an invisible one degrades quietly until the false-negative rate gets noticed and called an emergency.

Stop tuning once the workload hits its quality and latency target with headroom left over, and don't apologize for the margin. It absorbs an unusually large diff, a second reviewer, a hot afternoon that throttles the card, or an untested runtime release.

Spend context on what changed, not on what merely exists in the repository.