Update models across an air gap without improvising

So the air gap doesn't exempt you from the discipline of a normal rollout, it just relocates where that discipline lives. My reference setup is an isolated workstation that only receives GGUF files someone approved, deliberately ordinary, no clean-room conditions. The temptation is to treat "it's offline, it's fine" as a substitute for process. It isn't. Write down the job first, and decide what counts as a good result before you touch a setting.

The manifest is the release, not the courier

Every batch of models that crosses that gap gets handled like a release, not a delivery. A signed manifest goes out before anything leaves its network, and the files get scanned on a staging host before they touch the isolated machine. Checksums get verified after transfer and have to pass before the previous known-good artifact retires. I keep it until the new one earns its place. Same inputs every test run, launch command saved with the result, because nobody remembers which quantization flag they used.

What a single warm run can't tell you

One warm sample tells you nothing worth acting on. If someone hits this model cold, one measurement has to start cold too, and the workload has to run long enough for thermal throttling or queueing to show up. Whatever change you're testing has to leave the answer itself intact, or you haven't made anything faster, you've swapped in a different, wrong system.

Medians describe the ordinary run. The tail is what the user remembers.

What matters: time to first token, prompt-processing speed, generation speed, peak memory, wall power when relevant, and whether the task succeeded.

Where the discipline actually goes missing

The place this falls apart is boring: someone copies unlabeled GGUF files onto a USB stick, and the link between the model, its license, and the prompt template it expects disappears somewhere between machines. Local inference runs configurations that technically work, which is the trap: loading isn't a performance result, and a plausible-looking answer hasn't been evaluated. Check the runtime logs and OS counters instead of trusting the flag you set: the log is what happened, the flag is just a request. When two runs disagree, change one variable and say why, the habit llm-security-red-teaming keeps coming back to.

Small releases, kept small on purpose

A five-percent win tied to this exact model, driver, and runtime build evaporates the moment any one of those changes, so the real deliverable isn't the number. It's a small script and a handful of representative prompts, saved next to a plain-text result file rerun after every upgrade. Treat every offline update as a small release with a rollback path, the same approval-gate instinct behind codex-approval-modes. Test the smallest change first and leave headroom instead of running the machine flat out, then stop tuning once the workflow clears its latency and quality bar.

I'll admit the manifest-and-staging-host routine is more ceremony than a single laptop running one model for yourself needs, so if that's your whole setup, keep two copies of the file and skip the rest.