System RAM is not slow VRAM

So no, throwing more system RAM at a CPU-offload setup doesn't make it behave like VRAM. It just changes which failure you hit first.

RAM buys capacity: room for a model too big for your GPU to load at all, sitting mostly idle in system memory while a slice of its layers run on the card. That part works reliably now, so people stop checking. Bandwidth is the part nobody looks at, and bandwidth decides whether the thing stays pleasant to use once it's running.

capacity gets you in the door, bandwidth pays the rent

My reference point is a 16 GB GPU next to 64 GB of dual-channel system memory, an ordinary desktop pairing, not a lab bench. On that hardware an oversized model loads without complaint, because loading only ever asks one question: is there room. Whether it's usable is different, and answering that means treating offload as a dial, not a switch: run the same inputs across several layer splits, log prompt-processing and generation speed separately, and save the launch command beside every result so you can reproduce it without archaeology. Don't trust one warm run. Include a cold start, since that's the one real users hit, and run long enough for thermal throttling or queueing to show up. Keep output quality in the same record as the speed numbers. A faster answer that reads differently isn't a quicker version of the old setup, it's a new one you haven't checked yet.

how I decide whether the offload earns a permanent spot

Numbers worth keeping: time to first token, prompt-processing speed, generation speed, peak memory, wall power if you meter it, task success. Use the median for the typical case, but watch the slow tail, since a p95 stall is what a real person notices and complains about. The trap I see most: treating a successful load as though it were a result.

"It loaded" is not a performance result.

Neither is a quick glance at output that seemed fine. Check runtime logs and memory figures instead of the flag you passed at launch, because the two disagree more than you'd expect. Change one variable at a time and say why you expect it to matter, or the exercise turns into anecdotes wearing a spreadsheet. There's a maintenance tax too: a fragile five-percent win from one layer split evaporates the moment a driver or runtime changes, so keep retesting cheap, a short script and a few representative prompts, the same discipline any decent run-local-llm-guide pushes. My rule: partial offload earns its keep for occasional deep work, not a substitute for a model that fits your card day to day. Once latency and quality clear the bar, stop turning knobs and leave headroom for next month's model. Fitting the daily driver inside VRAM outright still wins, roughly where the hardware for local LLMs math keeps landing anyway.

I'll admit that's easier to preach than practice. I still catch myself tuning a layer split at midnight for a win nothing asked for, so take "stop once it's good enough" from someone who mostly ignores it.