Build a local AI workstation you can live beside

This has already happened to people: they build a capable local rig, hit the benchmark numbers they wanted, then quietly stop running it because it sounds like a hair dryer under load. Nobody officially decides to shelve a machine. It just happens. The daily workload moves elsewhere. That's the actual failure mode with local inference hardware: a correct benchmark attached to a machine nobody wants to sit next to.

My reference setup is a high-airflow case, large intake fans, and a power-limited GPU. Nothing exotic: no water loop, no acoustic foam. I picked it because the tuning choices generalize: case pressure, fan curves, a power cap you set once and forget. Decide what the machine is for before touching any of those. Interactive chat and code completion want near silence and low latency; document extraction and overnight batch work care more about throughput than acoustics, since nobody's sitting there at 3am. Define what good means for your job before you touch a setting, or you'll optimize the wrong thing and feel pleased about it.

measuring from the chair, not the spec sheet

Tune at sustained load, not whatever a launch script produces in the first ten seconds. A GPU that idles quietly and only gets loud two minutes into a long generation passes every quick check and fails every real one. Sit where you'd actually sit, run long enough for heat and queueing to show up, and listen. Tonal whine from one fan or a coil is worse than raw volume: kill that before chasing a lower decibel number, since a broadband hum at a slightly higher level beats a whine that finds one pitch and holds it.

Use the same inputs every run and write the launch command next to the result; you won't remember which flag you changed last week. One warm run tells you almost nothing. Include a cold start when your real usage has one, and run long enough to catch thermal throttling or a queue backing up. A change that makes the model faster and also changes what it says is a different system sharing a name with the old one.

Worth recording: time to first token, prompt-processing speed, generation speed, peak memory, wall power if capped, and whether the task passed. Close to what sits next to my bench:

Medians describe the normal case. The slow percentile explains why a setup feels annoying even when the average looks fine: nobody remembers the median, they remember the run that stalled for what felt like forever. Noise and responsiveness belong next to those numbers, plus startup friction, not off in a separate notebook, since those decide whether the machine gets used tomorrow.

Loading without crashing tells you the process started, nothing more. A plausible-sounding answer needs to be checked, not eyeballed. Check the runtime logs and OS metrics instead of trusting a flag's name: GPU-offload settings are a known place for that to go wrong, and the ollama-gpu-offload-math is worth knowing first. Change one variable at a time and say what mechanism you expect to move before running the test. Otherwise a week of tweaking turns into a pile of anecdotes you can't reconstruct.

There's a maintenance cost too. A fragile five-percent win evaporates the moment a driver or runtime updates. A short script, a few prompts, and a plain-text log is enough to retest after every upgrade. You don't need a dashboard for a machine that lives under a desk.

the trade I keep making

My rule is boring: give up a few percent of throughput for a machine that stays on, stays quiet, and is available all day. Test the smallest plausible change first: a lower power cap, a different fan curve, not anything structural. Stop tuning once the workflow hits its latency and quality target, not a number that could theoretically go higher. Spare thermal and power headroom absorbs longer prompts, a background compile, and next month's model, without a redo.

This matters more on a desk than in a rack. A homelab-llm-server tucked into a closet can run as loud as it wants; a workstation next to your keyboard cannot. The setup that wins isn't the one with the best isolated number. It's the one you actually leave running.

The one thing I still haven't solved is dust. Airflow drops slowly as filters clog, the fan curve compensates by spinning harder, and the room gets louder over months with no single change to blame. My fix is opening the case every so often and listening for the drift, a chore I haven't figured out how to automate.