Resizable BAR and local inference

Somebody flips Resizable BAR on in the BIOS, reruns the same command, times it twice, and gets the same number as before. Snake oil, they decide, and move on. Somebody else does the opposite: sees the driver report the feature as enabled and assumes the win is banked, without ever checking whether the workload touches the larger PCIe mapping. Both are making the identical mistake: treating a platform capability like a benchmark result.

Resizable BAR lets the host address the whole VRAM window at once instead of the old fixed, small mapping window, and that only matters when something crosses the PCIe bus a lot: CPU offload of a few layers, a runtime swapping models in and out of a shared card, host and GPU handing memory back and forth mid-request. Load one model into VRAM and decode for an hour and the bus barely gets touched again, so aperture size is close to irrelevant. What decides whether this matters isn't the chipset spec sheet. It's whether your driver and runtime route the transfer through the bigger mapping at all, and whether your workload is transfer-heavy enough for that to be visible.

pin the run before you touch the BIOS

Pin the model artifact, tokenizer, prompt template, runtime build, launch command, and sampling settings before you change anything else. Pick a small set of inputs from your actual workload, awkward cases included. If users hit a cold-storage, unloaded-model path in production, run that once and record it separately, then repeat warm runs long enough to expose cache effects, queueing, memory pressure, and thermal throttling. A single number from a single run tells you almost nothing.

Raw tokens per second is diagnostic, not the product result. For automation, count valid completed records per hour. For coding, fold in the review time a human still spends afterward. For chat, look at p50 and p95 first-token latency across realistic conversation lengths, not the first exchange in an empty context window. A setup that wins a short, warm, batch-of-one benchmark can lose the day once model swaps (see the VRAM-juggling problem) and long prompts show up, exactly the situation Resizable BAR claims to help with.

If you're offloading layers to fit a model on a card that's too small for it, get the layer-offload math right first. It matters more than any BIOS toggle.

your runtime will not tell you it failed

Local inference runtimes are stubbornly resilient. They offload layers, page memory, eat cache misses, and fall back to generic kernels rather than error out, producing an answer regardless. That's a feature most of the time. It's also why a working ReBAR-enabled run proves nothing on its own: the runtime never tells you which code path it took. Read the startup logs. Check device placement. Watch the OS-level PCIe counters if your platform exposes them. Confirm the optimization is active for your tensor shapes and context length, not someone else's.

Change one variable at a time unless you're explicitly comparing two whole systems. Save outputs, not just metrics: quantization, context compression, and sampling changes can all quietly shift the answer while the speed number climbs, so run the same quality gate on every candidate, no exceptions. If a response comes back shorter, check it's complete. If an extractor comes back faster, count valid records, not parseable braces.

Weigh operational cost alongside the performance number: startup, upgrades, observability, rollback. Could you rebuild this box after a disk failure without tribal knowledge stuck in someone's head? A small gain resting on an undocumented patch, or someone re-warming a cache by hand every morning, is a bad trade for anything you keep running. Once you hit your target, stop and leave headroom: free memory for prompt variance, spare queue capacity so a batch job doesn't starve an interactive user, thermal and power margin for the day the room runs warmer than usual.

Treat any ReBAR result as a platform experiment, not a universal optimization, and write down what should trigger a retest: new model family, driver update, longer contexts, different traffic. Skip that and last quarter's number quietly turns into folklore.

Here's the rule I keep: I don't credit Resizable BAR, or any platform toggle, with anything until I've watched the specific phase it's supposed to move, on my own hardware, running my own workload.