Buying a laptop for local models

People buy the laptop with the biggest number on the spec sheet, get it home, load a 14B-class GGUF, and watch the fans spin up while the model spills into system memory. It still answers, just slower than the demo that sold them, and hot enough they stop trusting it unplugged. Nobody warned them a GPU name tells you almost nothing once it is soldered into a chassis with a battery to protect.

Desktop VRAM advice does not transfer. A 12 GB desktop card and a 12 GB mobile part with the same name can sit at very different sustained power limits, sharing memory with an OS and a browser full of tabs. The workload matters too: a coding assistant off the network, a private chat a few times an hour, and a nightly extraction job are three jobs on one thermal envelope, and none wants the same thing. Buy for one, misjudge the other two.

The five numbers that make it into my notebook

Before I touch a buy button or a config file, I capture a baseline: one named model, one fixed prompt set, the exact server command. Not "a 14B model," the actual artifact and template, the details that get lost first and invalidate the comparison.

Then I look for what the spec sheet will not say:

• exact VRAM, not the figure rounded up in marketing copy
• sustained power under load, not the boost number on the box
• independent cooling reviews from people who ran it hot
• the physical size of the charger it ships with
• how much memory the OS eats at idle before any model loads

A small charger is often a power limit in disguise, and laptop-llm-thermals predicts real behavior better than the marketing page does.

Measurements should follow the request through the whole system. Time to first token catches loading and prompt work; steady token rate is the decode you sit through; completion time is what the waiting person gets end to end. Add peak memory, queue delay, and wall power when the decision turns on them. For the overnight job, count valid jobs per hour; for the chat and coding loop, count the waits a person remembers.

I keep the notebook itself plain:

artifact + runtime + launch flags
workload and fixed input set
cold start / warm start / p50 / p95
peak memory and wall energy
quality failures and abstentions
decision, owner, retest date

A benchmark without a decision is trivia. A setting without an owner turns into folklore.

Failures that never throw an error

The easy mistake is buying off a five-minute benchmark and living with fan noise and throttling for years of real sessions. It survives because the machine keeps producing plausible text throughout. Local inference fails softly: a model partly offloads onto the CPU, a cache starts missing, swap creeps up, a queue holds dead requests, or a fallback quietly sends a query off the machine and moves your privacy boundary. None of that throws an error, it just gets slower and less private on the same screen, and ollama-gpu-offload-math shows how easily the first one happens.

So watch it happen. Pull logs and OS memory and thermal counters while the benchmark runs, not just the final number. Change one variable at a time unless comparing whole configurations. Run it more than once: a single fast pass tells you nothing about a durable gain versus a lucky sample. Read the output too, not just the speed. If a change buys throughput and costs an important task, write down both numbers.

Simplicity earns a column too. An optimization you hand-repair after every driver update was not free, just because it once ran fast in a demo. I would rather keep a boring setup rebuildable from a service file or a short script than a fragile trick to remember. Keep raw model files out of the report; keep the hashes and the command instead.

The rule I keep coming back to: buy for usable memory and sustained comfort, not the best number from a three-minute burst. Less exciting than chasing the biggest accelerator on the shelf, but it leaves a stack whose limits you can see. A visible limit gets scheduled around, or priced, or handed off. An invisible one turns into unexplained waiting and an unbudgeted upgrade. Stop once the workload clears its quality and latency bar with headroom left; that margin absorbs the longer document, the extra job, or the next runtime release.

What I have not solved is battery behavior under sustained inference over the long run. Every laptop I have run this way ages differently after a year near its power ceiling, and I have no benchmark that tells you which one holds up before you have already bought it.