AI hardware ROI for an Apple Silicon local-model system: the utilization curve

Someone always buys the maxed-out Mac Studio for local inference, runs one big MLX model once, then watches it idle for weeks while the real work happens in a browser tab against a hosted API. That box didn't stop costing money when the fan spun down. It just stopped being visible.

The pitch for Apple Silicon as an inference box is real: unified memory big enough to run models a discrete GPU can't touch without stacking cards, low idle draw, no second box in the corner. None of that tells you whether it pays for itself.

the box, the job, and what it's replacing

The setup here is a Mac mini, Studio, or MacBook with enough unified memory and storage for the models you want, plus the OS and everyday apps. The workload is unglamorous: local development, document analysis, MLX experiments, and the occasional large model that only earns its keep in a local-first cascade, API as overflow, not default. The alternative isn't hypothetical: a discrete-GPU workstation plus a plain laptop, or paying per token to a hosted API priced the way subscriptions are, a habit bursty workloads punish. What it sells is integration: one quiet system with a shared memory pool instead of two machines and a noise problem. Price the unit that matters: a productive local-model month, absorbing the machine's ordinary non-AI use too. Fixed cost only gets cheap once spread across time the machine does something useful; idle hours still cost capital, often power. Keep the numbers in a worksheet you update, not frozen into prose. The arithmetic ages well. The quotes don't.

counting the hours nobody bills for

Start from cash, not a spec sheet. The whole model collapses to one line:

effective_cost_per_hour = annual_fixed_cost / productive_gpu_hours + variable_cost_per_hour

Everything else fills that in honestly. Track queued, active, resident-idle, asleep, and unavailable hours for a real month. Hardware, installation, and reserved capacity are fixed over a useful range; electricity, paid tokens, transfer, and support time move with the work. A dedicated operator is fixed; someone pulled in only on failure is variable. The bill of materials needs the accelerator, host, memory, storage, networking, power, cooling, spares, tax, shipping, and installation, minus whatever the org would buy anyway: an everyday laptop isn't an AI cost, a memory upgrade bought solely to fit a model is. I wouldn't price cooling separately for a mini under a desk, it's noise next to labor cost. Then measure local, rented, and API routes, and record:

• accepted tasks and severe failures
• input, cached input, reasoning, and output tokens
• retries, abstentions, and human repair minutes
• queue time, first token, completion, p95
• wall energy and a cooling allocation
• maintenance and incident time

Quality adjustment matters too: a cheap model needing two attempts and five minutes of correction should be priced on the finished result, not the first draft, and a local specialist that reliably clears one narrow job shouldn't be judged against a frontier API run at max effort.

why 8,760 hours is a lie you tell yourself

The risk specific to this platform is Apple's own memory: no upgrades after purchase, bigger tiers cost disproportionately more, some runtimes lag CUDA, and resale on an unusual configuration is a guess, not a market. Build low, base, and high demand curves, and respect daily peaks and p95 latency: a machine can't run at full utilization and still absorb an interactive request. Batch work can soak up quiet hours, but only if the org genuinely needs it. Meter idle states separately: a workstation asleep overnight differs economically from a server holding several models resident so nothing waits on a cold load. On shared systems, watch queueing and abandoned requests; on an edge fleet, multiply update, replacement, backup, and travel time by node count, since labor beats electricity. The recurring mistake is dividing by the full 8,760 hours in a year as if demand were constant. Keep every assumption in a small table, sourced and dated, not hidden behind decimal places.

the payback date, and the trade you're making

Payback month is the first month cumulative discounted benefit clears cumulative cost; ROI over your horizon is net discounted benefit divided by discounted cost. Both need a conservative useful life and residual value, since a machine can outlast the models it was bought for, or its software support. Productivity claims need a realization factor: saving ten minutes only counts if it avoids a hire, cuts outsourced spend, or shortens a binding queue. Run it once with productivity value at zero; a purchase surviving only on speculative time savings deserves a second look. Price renting too: hourly rental earns its keep while demand and hardware shape are uncertain, provided you count storage, image prep, transfer, minimum billing, and the instance you forgot was running. A short rental benchmark beats an expensive wrong box. State your break-even in checkable terms: accepted tasks per month, productive GPU hours, a maximum API cost per task, or a minimum useful life. Set a retest date and triggers: volume shifts, a new model resets the quality bar, API prices move, power contracts change, a part reaches end of support. What you're accepting on purpose, buying the box, is less flexibility than renting, and a machine drawing power and losing resale value on the nights nobody touches it, in trade for a quiet system sized to a workload you measured, not guessed at.