Give the model a repository budget

Somebody points a fresh coding assistant at the whole repository, context window maxed out, and gets back a function that references a helper deleted three refactors ago. It happens constantly. The assistant had access to everything and used almost none of it well, because every irrelevant file competed for the same fixed budget of attention as the two or three that actually mattered. That's the failure this is about: not the assistant being dumb, but the person operating it mistaking volume of context for quality of context.

I use a local coding model with a 16K practical context as my reference setup, nothing exotic, no claim that this configuration represents every machine on someone's desk. The point of naming it is to make the decision measurable instead of vibes-based. Before touching a setting, write down the actual job: interactive chat, code completion, document extraction, an overnight batch, whatever it is, and decide in advance what a good result looks like. Skip that step and you'll spend an afternoon tuning a number that has nothing to do with what you're trying to ship.

Chase the call graph, not the folder tree

The method that actually works starts at the failing symbol. Pull its callers, pull its tests, and stop adding files the moment you can't state a reason for the next one. That discipline sounds fussy until you compare it to the alternative: dumping a directory listing at the model and hoping something useful floats to the top. Run every comparison against the same fixed input set, save the launch command next to the result, and don't trust a single warm sample. Include a cold start if a human will ever hit one in practice, and repeat the run long enough to expose thermal throttling or queueing delays. Keep a note of output quality right alongside the speed numbers, because a faster wrong answer doesn't count as a win. If shortening the context also changes what the model says, treat that as a different system wearing the old one's name, not a faster version of it.

What actually belongs in the log

Not every number is worth writing down, and most dashboards make that mistake by recording everything. What actually changes the decision is time to first token, prompt-processing speed, generation speed, peak memory, wall power when it's relevant to the setup, and whether the task actually succeeded. Medians tell you about the normal case. A slow percentile, the p95, catches the pause that makes someone stop trusting the tool, the one that never shows up in an average. Notes about noise, about how responsive the thing felt, about how much friction it took to set up: all of that belongs next to the numbers, because it decides whether anyone keeps using the setup once the novelty wears off. A run sheet doesn't need to be fancier than this:

Fill in six fields honestly and you already know more than most benchmark screenshots tell you.

The trap of feeding it more

The common mistake, and I've made it, is reacting to a bad answer by throwing more files at the model and burying the one constraint that actually mattered under a pile of context that didn't. Local inference is full of configurations that technically run. Loading without errors is not a performance result. An answer that looks plausible on a glance is not an evaluation either, it's a guess dressed up as a conclusion. Check the runtime logs and the operating-system metrics instead of trusting whatever flag you think you set. Change one variable at a time and say out loud what mechanism you expect to move, or the whole exercise turns into a pile of anecdotes with timestamps attached.

There's a maintenance cost to all this too, and it's real. A fragile five-percent win evaporates the next time a model, driver, or runtime gets updated, and you won't notice until something quietly regresses. Keep a small script, a handful of representative prompts, and a plain-text result file. That's enough to retest after every upgrade without building yourself an observability platform for what is, after all, one workstation.

Where I actually draw the line

My rule, such as it is: spend tokens on causal proximity, not on directory coverage. Test the smallest plausible change first. Keep headroom in reserve, because that spare capacity is what absorbs a longer prompt next week, a background process you forgot was running, or the next model you'll want to try, whether you're staying on a modest local setup or you've read the case for heavier iron when the workload justifies it and are weighing the jump. Stop tuning the moment the workflow clears its latency and quality bar. The setup worth keeping isn't the one with the flashiest isolated number. It's the one whose behavior you understand well enough to bet on.

And here's the honest undercut: I don't run this whole ritual for a one-off Tuesday-afternoon question, I just paste three functions and hope, and it works often enough that half of what's above is really discipline for the workflow you touch every day, not the debugging session you'll forget by Friday.