Active cooling is part of a Raspberry Pi AI build

Here's the rule I use before I trust a single throughput number off a Raspberry Pi: seal the board in its actual enclosure, run the model for half an hour straight, and watch what the temperature does before you write anything down. Skip that step and every benchmark you produce is a photograph of a board that hasn't started sweating yet. The rest of this piece is just me explaining why I insist on it.

A Pi 5 running inference locally, tucked into a compact case next to the rest of your smart-home gear, is not under the same thermal load as a Pi serving the occasional web request. A web hit spikes a core for a few hundred milliseconds and lets it cool. Inference pins cores near their ceiling for the length of the job, and if the job repeats every few seconds, the board never really gets to rest. That's the whole physical reason case and cooler choice matter here in a way they wouldn't for a file server idling in the same box. Physics doesn't negotiate.

Write the job in four lines before you shop for a board

Before any hardware decision, write down the event, the input, the output, and the deadline. "Add AI to the camera" is not a job description, it's a wish. "When a parcel sits in the porch zone for a fixed stretch, fire one local notification with a thumbnail" is testable, and testable is the whole point. Writing it this way also tells you, fast, which parts of the pipeline don't need a model at all: a zone check and a debounce timer handle most of the false triggers before any inference runs. Picking the board comes after this, not before it, and it's worth reading up on hardware for local LLMs once you know the job, because the right board for a bounded four-second decision is not the board you'd want for anything closer to a chat session.

Thirty minutes inside the real box, not on the desk

The failure I see most often is trusting a benchmark run on an open desk with a fan blowing across the board, for about a minute. That tells you almost nothing about the appliance you're shipping. Run the actual workload for thirty minutes with the case closed, log clock speed, temperature, fan or throttle state, and noise, and use real input from the actual room: the evening light, a television in the background, a warm cabinet, a marginal USB cable, a Home Assistant database that's busier than your test rig. Any one of those moves the result more than swapping model quantization does, which is a slightly embarrassing thing to learn after you've spent an afternoon benchmarking. Keep the record boring and consistent:

board=pi5 os=<name> psu=<rating> cooling=<passive|active>
model=<artifact> runtime=<version>
input=<real room sample> expected=<action>
latency_cold=<> latency_warm=<> rate_sustained=<>
mem=<> temp_c=<> watts_wall=<>
false_action=<> missed_action=<> recovery=<manual path>

Once you're logging real numbers, it's worth knowing what to actually measure and why one figure lies more than another, which is most of what measuring local tokens is about.

Don't let the model hold the deadbolt

Generative output is fine at interpreting a request or ranking a few options. It has no business being the only thing standing between a bad interpretation and a door lock, a heater, an alarm, a valve, a battery, or a person. Keep ranges, permissions, timeouts, confirmations, and hardware interlocks in plain deterministic code that a model cannot argue its way around. Treat anything the model reads, camera text, calendar entries, a scraped webpage, an MQTT payload, a voice transcript, as data, never as instructions, because sooner or later one of those fields will contain something that looks like a command. Privacy needs the same discipline end to end: running inference on the box avoids one upload, sure, but the recordings, the thumbnails, the logs, the backups, and whatever remote dashboard you built can still leave copies scattered everywhere. Decide what gets kept, for how long, and who can pull it, before you decide whether the model itself is any good.

The board has to survive being ignored

Run it under a real service manager or a small pinned container, keep configuration separate from the model files so a bad update doesn't require re-deriving both, and write a health check that proves the function actually works rather than just confirming a process is holding a port open. Back up the controller state, record the model hashes you're running, and keep a known-good image for any node you installed somewhere you don't want to visit twice. Heat and power belong in the acceptance test, not as an afterthought: let the enclosure reach equilibrium and measure watts at the wall with the normal peripherals attached, not the bench supply you used for development. If the board throttles, starts swapping, corrupts the storage, or drags down the rest of your home-automation host under that load, it isn't ready, no matter how good the best five-minute run looked.

I run mine with a small fan and a slightly bigger case than I'd like, and I've made peace with that. The fanless, silent build was the one I actually wanted, and I gave it up on purpose, because a board that throttles quietly at three in the morning is worse than one that hums a little and keeps its numbers honest all day.