AI hardware ROI for an edge or SBC AI fleet: a sensitivity analysis that can change the answer

A scatter of Raspberry Pis and Orange Pis, each bolted into its own weatherproof box with a camera, an accelerator, and a power supply, doing OCR on shipping labels or watching a line for defects, with one management node tying the fleet together.

That's the fleet.

The alternative was never exotic: ship the frames to a cloud endpoint, run one central server in the back office, or skip the model and wire up plain sensors with threshold rules. What you're buying instead is low power draw per node, a decision made right where the camera sits, and video that never has to hit a WAN link.

Nobody selling you accelerators will say this out loud, but the honest answer to "what's the ROI" is a range that moves on six dials: how hard you run the hardware, how long it lasts, what the API alternative costs per call, what power costs locally, whether the cheap route produces usable output, and how fast demand grows once nodes prove out.

The thing you're paying for is one reliable edge decision made at the point it's needed, not a token count or an accelerator-hour. A cost model earns its keep only if it shows which assumption is steering the purchase, so it belongs in a spreadsheet tied to today's power rate and this week's API sheet, not a paragraph written once and never opened again. Prices drift by the month. The shape of the decision barely moves.

separating what you paid once from what you pay per event

The model itself is unglamorous: discounted benefit minus discounted cost, divided by discounted cost, run once per scenario. Vary one input at a time before you combine them, then build a low, a base, and a high case and report where each breaks even, instead of one blended guess dressed up as precision.

Split every line item into fixed and variable before it goes near the model; the two behave very differently once the fleet is live:

Only count equipment the fleet needed. A laptop your developer already owned isn't an AI cost just because it flashed an SD card; a memory upgrade bought solely so a model would fit is.

Then test the workload across local, rented, and API routes, the same triage from local-first-cascade, and log:

• accepted tasks against severe failures
• input, cached input, reasoning, and output volume
• retries, abstentions, human repair minutes
• queue time, first token, completion time, p95 latency
• active, resident-idle, sleep, and unavailable hours
• wall energy plus a cooling allocation
• maintenance and incident time

Quality has to enter the number too, or the exercise lies to you. If the cheap model needs two attempts and five minutes of correction, that correction time belongs to the completed result, not the first pass. Price a local specialist that reliably nails a narrow task against a frontier API cranked to similar effort on the same task, or the comparison isn't fair.

the fleet punishes optimism node by node

Here's where these projects die: maintenance multiplied by node count, a storage card failing on the floor, installation labor nobody scoped, accelerators that don't match across generations, and hardware that quietly never leaves prototype status. Build low, base, and high demand curves and use the daily peak, not the average: a node running flat out has nothing left for an interactive request. Batch work can fill the quiet hours, but only count it if the organization has batch work waiting.

Meter the idle states honestly: a box that sleeps overnight has different economics from a server holding several models resident for instant answers, and shared infrastructure needs queueing and abandonment measured too. For an edge fleet, multiply update, replacement, backup, and travel time by node count; operational labor tends to dominate the electricity bill.

The recurring mistake is publishing one precise payback month from uncertain inputs with no margin. Keep the assumptions visible in a small table, date the volatile ones, and don't let decimal precision hide how little you know. Payback month is the first month cumulative discounted benefit clears cumulative cost; ROI over your horizon is net discounted benefit over discounted cost. Both depend on a conservative useful life and a real residual value, and hardware can stay physically fine while a model's growth or a dropped support contract makes it obsolete anyway.

Productivity gains need a realization factor before they enter the model. Saving ten minutes of a developer's time doesn't hand you ten minutes of billable revenue; count it only if it avoids a hire, cuts outsourced spend, raises deliverable output, or shortens a queue that was binding. Run the model once with that term set to zero, and treat any purchase that only clears the bar through speculative time savings the way you'd treat a vendor quoting you their own ROI.

Compare hardware against renting, not just the API. Hourly GPU rental earns its cost while demand and hardware shape are uncertain, and a short rental benchmark can save you from an expensive wrong purchase, provided you count persistent storage, image prep, transfer, minimum billing, and the idle instance you forgot to shut down. If the honest alternative is a subscription rather than a rack of boxes, that's a different comparison, covered separately in subscription-wrong-for-ai.

Approve the hardware only when the base case clears and the downside is livable, and write the break-even boundary in operational terms you can check: accepted tasks per month, productive GPU hours, maximum API cost per task, or minimum useful life. A boundary like that gets revisited fast; a narrative recommendation gets re-argued from scratch. Put a retest date on it and name the triggers: volume moving materially, a model change shifting the quality gate, API prices moving, power contracts renewing, a component hitting end of support. ROI isn't a property stamped onto the accelerator so much as a relationship between one workload, one alternative, and assumptions with dates attached.

I'd love to tell you the sheet gets revisited on schedule. In my experience it mostly doesn't: the retest date arrives, gets noticed, and gets ignored until something breaks.