Containerize SBC AI services selectively
Docker is worth running on a Raspberry Pi doing AI work. It won't fix the real problems on that board, and installing it expecting a fix wastes a weekend.
People treat these boards two ways, and both are wrong: tiny gaming PCs pushed toward a bigger model, or toys not worth the effort otherwise. The useful work sits between: bounded jobs near the sensor, working without a cloud account.
What Docker buys you there is a reproducible boundary, not horsepower: pin a multi-architecture image, and declare device access and memory limits. That's roughly the case I made for running Ollama in Docker, and it holds harder on a 64-bit Pi or Orange Pi running MQTT and inference, where you can't add RAM to a mistake.
The four-line spec beats the parts list
Write the job in four lines before buying anything: event, input, output, deadline. Add AI to the camera is a wish, not a job. A parcel in the porch zone twenty seconds, one local notification with a thumbnail, is something you can fail at, which tells you what never needed a model.
Test against the room you're deploying into, not a demo clip: microphone, camera angle, language, network. Evening light, a TV in the background, or a database under load can move the result more than swapping the model does. Keep a short record: board, power supply, model version, latency, and what a false trigger and its recovery look like.
Good accuracy alone isn't enough. A setup fails if it fires twice, forgets its state after a power blip, or goes quiet without saying so. Anyone besides you using it needs visible state and a manual fallback: nobody should need to know which container is misbehaving to turn a light on.
A container boundary is not a safety boundary.
Camera text is not a command
The most common failure isn't a bad model, it's stacking opaque containers until nobody knows which one owns the camera feed. Generative output has no business standing alone between a bad guess and a lock, a heater, or a person: put permissions, timeouts, and interlocks in deterministic code a model can't argue past. Camera text and voice transcripts are untrusted data, never instructions.
Local inference solves one thing: it keeps a raw frame from leaving the building as an upload, not the recordings, logs, or a remote dashboard that might hold a copy later. Decide up front what gets kept, how long, and who can pull it. The safest frame is one the camera never captures.
Health checks that prove the light turns on
Run it with a service manager or a small pinned container definition, configuration kept apart from the model files. Give it a health check that proves the function, not that a process sits on a port, and back up the controller state and model hashes.
Power and heat belong in the acceptance test: run until the enclosure reaches a steady temperature, measure at the wall with peripherals attached. A board that throttles, swaps, or corrupts storage has failed regardless of its best result.
Containerize what's stable, and write down the host-level hardware glue that isn't. Edge AI earns its keep as a thin layer above sensors you trust, below a household policy someone set. Take the model away and the automation should get dumber, not stop.
That's the tradeoff I take on purpose: a locked-down, pinned, boring deployment that won't pick up the next new model on its own. I give up pulling a new image, for a porch light that still works months after I stop watching it. Some days that trade feels stingy. I'd make it again.