Reach a home model safely with WireGuard

And no, TLS on the inference port does not make it fit for the open internet. That's the mistake I keep seeing when someone wants a home model reachable from a laptop in a hotel lobby: they treat encryption in transit as the same problem as who gets to knock on the door. It isn't. Once a local model is reachable from anywhere, you've added an identity boundary and a network boundary to what used to be a plain local service.

Most of this discussion also starts in the wrong place, with a model name or a peak number from someone else's benchmark. What matters is the work behind the API once it's reachable from outside the house: a chat session, a coding loop, and a nightly extraction job can share a box and still want different guarantees once the network changes under them.

the tunnel is the boundary, not the cert

The inference port only listens on the VPN interface, never on every address on the box. The one thing exposed to the internet is the WireGuard listener, and its peer list stays short and hand-maintained. A client profile for the traveling laptop looks like this:

[Interface]
Address = 10.66.0.2/32
ListenPort = 51820

[Peer]
PublicKey = <server-pubkey>
AllowedIPs = 10.66.0.0/24
Endpoint = home.example:51820
PersistentKeepalive = 25

Nothing clever there, on purpose. The application on top can still ask for a token if you want defense in depth, though proving it holds is closer to red-teaming the endpoint than to tuning it. The first gate stays: are you a member of this network at all. Test what a bad hotel connection does to that before you rely on it.

what to log before touching a setting

Before touching the network, write a baseline: one named model artifact, one fixed prompt set, the exact server command. Those details vanish first, and losing them is what makes an old comparison meaningless later. Time to first token covers loading and prompt work, steady token rate covers decoding, completion time is what the person on the other end feels and remembers. Add peak memory, queue delay, and wall power when they'd change the decision. For batch, count valid jobs per hour; for chat, count the waits someone notices.

Keep one line per run: artifact, runtime, flags, workload, cold and warm timings with p50 and p95, peak memory, wall energy, quality failures, a decision with an owner and a retest date. Drop that last part and a benchmark turns into trivia.

failures that never throw an error (wireguard)

The failure mode that bites is quiet. A model partly offloads onto the CPU and gets slower. A cache misses and the answer still reads as plausible. Swap creeps up. A queue holds requests from a client that already disconnected. A fallback path serves the request from outside your tunnel, the one part you cared about. None of it raises an exception.

Watch runtime logs and OS counters while the test runs, not just the final number. Change one variable at a time unless you're comparing whole configurations. Run it enough to tell a real improvement from a lucky sample, and read what came out the other end instead of assuming speed means quality. If a change makes something worse, write that cost next to what it saved.

boring wins after the second driver update

One column matters more than the rest: how much babysitting a setting needs. An optimization that has to be hand-repaired after every driver or model update isn't free, whatever it saved the day you measured it. A boring configuration that comes back the same way every time, from a service file or a short script in the repo, beats a clever one only one person can rebuild from memory. Keep raw model artifacts out of the report; keep the hashes and commands, so a run from six weeks ago still means what it meant then.

membership first, login second

Which is the one rule I'd keep if everything else got cut: network membership is the first access check, application authentication is the second. Not the other way round, not either alone. It's a duller decision than picking the biggest model or the newest card, but it leaves a stack whose limits are visible, and a visible limit can be scheduled around. An invisible one turns into a page at 2am.

Network membership first, application login second.