External GPUs and local LLMs: mind the enclosure

An eGPU is a good way to run local models off a laptop, but don't expect it to act like a card in the machine's own slot. Thunderbolt makes capacity portable; the enclosure adds sleep, wake, cable reseat, driver reset, none visible on a spec sheet.

The comparison people reach for first is a peak-throughput number tied to a model name. Wrong place to start. What matters is the work on the other side of the API: a private chat used a couple times an hour, a coding loop, an overnight batch job.

Docked at night, in a bag by morning

Picture the machine: a laptop docked to the eGPU some evenings, undocked next morning for the commute. Baseline that setup before buying or reconfiguring anything: one named model, one prompt set, the launch command, the model artifact, the prompt template. Skip those two and the comparison goes meaningless within weeks.

Test what the eGPU has to survive that a card in a slot doesn't: model load, prompt ingestion, sustained decoding, a Thunderbolt disconnect, a suspend cycle. Time to first token: loading plus prompt work. Steady token rate: decoding. Completion time: what the person feels. Log peak memory, queue delay, and wall power where they matter, jobs per hour for batch work, slow waits for chat. Long runs heat the enclosure as much as the card, worth checking against a laptop under sustained load.

Keep the notebook fields boring: artifact, runtime, launch flags; workload and fixed input; cold-start and warm-start as p50 and p95, not one average; peak memory and wall energy; quality failures or abstentions; a decision with an owner and a retest date. Skip it and a benchmark is trivia, a setting nobody owns is folklore.

Plausible output is not proof of anything

The easy mistake is comparing steady token speed alone while ignoring how often the enclosure or driver resets, since the system keeps producing plausible text through it regardless. Local inference fails softly more than loudly: a model partly falls back to CPU, a cache misses, swap creeps up, a queue holds a disconnected request, or a fallback quietly changes which backend, and which privacy boundary, served the answer. None of it throws an error.

Watch runtime logs and OS counters while the test runs, not just the summary after. Change one variable at a time unless comparing whole configurations, repeat enough to separate a durable win from a lucky sample, and read the outputs: speed alone proves nothing about equivalence. A regression needs its cost written down beside the gain.

Operational simplicity counts too: an optimization needing manual repair after every driver or model update isn't free. Prefer something boring, rebuildable from a service file or script, over something only one person can fix. Keep raw artifacts out of the report; keep the hashes and commands so a rerun means the same thing later.

Buy the headroom, not the leaderboard spot

The rule I'd carry forward: pick an eGPU for flexibility, not the cheapest route to peak throughput. Less exciting than the biggest model or newest accelerator, but it leaves limits you can see. A visible limit gets scheduled, priced, or routed elsewhere; an invisible one turns into an afternoon of unexplained waiting and an unbudgeted emergency upgrade.

An eGPU will not out-bandwidth a card in the machine's own slot. It never will. Not worth an afternoon proving otherwise.

Stop tuning once the workload hits its quality and latency target with headroom left over. That margin isn't waste, it absorbs a longer document, one more user, a warm afternoon, or whatever the next release changes underneath you. Efficient LLM use is mostly spending model capacity only where it changes the outcome.

Next check, before another spec sheet: dock the enclosure, load a model, let the laptop sleep, and see what's still alive on wake. That test says more than any throughput chart.