NUMA can make a large CPU model feel broken

Two sockets, a model too large to fit inside one node's local memory, and a RAM total that looks generous on the spec sheet. That's the setup. free -h will tell you there's plenty of headroom and say nothing about where that memory sits relative to the cores using it.

The RAM you have isn't the RAM you're using

Cross-socket access crosses the interconnect linking the two dies, and that path carries less bandwidth and more latency than either socket's route to its own DIMMs. An abundant machine turns bandwidth-starved the moment allocation lands on the wrong side, and nothing you'd normally check will show it. Before touching a setting, write down the job this box runs, chat, code completion, document extraction, overnight batch, and what a good result means for it. Chase a faster isolated number without that anchor and you'll spend a week optimizing the wrong thing.

Binding the run before you believe the run

The only test worth trusting binds a controlled run to each node in turn, inspects where the pages landed, then compares interleave against local allocation. Same inputs, every run. Save the launch command with the result, because in six weeks you won't remember which flags produced which number. One warm sample proves nothing. Include a cold start if a real user will hit one, and repeat long enough to expose thermal throttling or a growing queue.

Record only what could move the decision:

• time to first token and generation speed once warm
• prompt-processing throughput, tracked separately
• peak memory and wall power if you can measure it
• task success, not just whether it produced text

Medians describe an ordinary afternoon. A slow percentile describes the moment it goes wrong in front of someone waiting.

Pages before threads, threads before hardware

The trap I keep watching is changing model formats and thread counts while the weight pages sit on the wrong socket. Loading successfully isn't a performance result, and a response that looks fine hasn't been evaluated. Pull the runtime logs and OS-level memory stats instead of trusting whatever flag you asked for: requested and actual allocation diverge more than the docs admit. Change one variable, say what mechanism you expect it to move, and stay suspicious of a fragile five-percent win: it evaporates on the next driver update and leaves you with no idea why. Fix placement before blaming the CPU or buying faster memory. Test the smallest change first, stop once it clears its latency and quality bar. I ran the same socket-affinity pass in the DL380 host-tuning piece, and the logic carried over. What I still don't have an answer for: what happens when the scheduler migrates a generation thread mid-request to the other socket. The weights stay put, the thread doesn't, and pinning one without fighting the other is still open on my end.