Put Raspberry Pi AI workloads on NVMe
A Raspberry Pi's microSD card will boot for years, then die within a month of a real workload: a model cache rewritten every restart, a database committing every few seconds, a camera buffer that never stops filling, more writes than a consumer SD controller was built for. Once a board runs unattended, storage matters more than any tokens-per-second number.
A Pi 5 with an M.2 HAT hosting models and Home Assistant history is the real shape here. Calling it a gaming PC, or writing it off as no frontier chatbot, misses the point: it's good at quiet, bounded jobs near a sensor, no cloud account, low power. Real throughput is homelab LLM server territory.
Write the four-line job before you buy the HAT
Write the event, input, output, and deadline in four lines before ordering anything, the way you'd write "notify me if a parcel sits in the porch zone twenty seconds," not "add AI to the camera." Writing it out shows which parts don't need a model.
First experiment: separate the replaceable model cache from state that must survive a reboot, confirm the HAT's PCIe revision matches the board, and pull power mid-write to check recovery. Test with your real room: edge deployments amplify variation more than any model update will.
| Log this | Why |
|---|---|
| Board, OS, power, cooling | baseline config |
| Model and runtime version | what's really running |
| Real inputs, expected action | ground truth |
| Latency, temp, wall power | performance and headroom |
| Missed action, recovery | failure behavior |
Accuracy alone means little. What matters is reboot survival, reconnecting once the network returns, and failing safely when inference is down.
Nothing generative touches the lock, the valve, or the heater
The real failure is moving everything to fast NVMe and forgetting some state has no backup. A model can interpret a request or rank options; it shouldn't guard a lock, a heater, or a person alone. Put guardrails, ranges, permissions, and timeouts in deterministic code, and treat anything the model reads, camera text or an MQTT payload, as data, never instruction.
Privacy needs the same thinking: local inference stops one upload, but recordings and a remote dashboard can still leave copies. Decide what's collected and who can pull it. Safest is the frame never captured; next safest is discarded right after the decision.
Design for the day nobody's watching it
Use a real service manager, keep configuration separate from model files you'll overwrite, and add a health check that proves actual function, not that a process owns a port. Back up controller state and keep a known-good image for hard-to-reach nodes.
Power and heat belong in the acceptance test: run until the enclosure reaches equilibrium, measure at the wall with real peripherals attached. If the board throttles, swaps, or corrupts storage, it isn't done. What's held up: NVMe for anything active, reproducible configuration everywhere else, and when the model is unreachable, automation gets dumber, not stopped.
This was never about proving a cheap board can run a model. The point is one small capability still doing its job after the benchmark window closes and the device disappears into a cupboard. My own weak spot: no clean way to update models on a node I can't reach without pulling the card, and "known-good image" carries more weight than I'd like.