AI hardware ROI for a personal AI workstation: depreciation and resale value
A 24GB card that comfortably runs today's mid-size open-weight models will not comfortably run next year's, and no amount of cooling changes that.
Picture a personal AI workstation: a developer-owned desktop with a capable GPU, enough RAM, NVMe fast enough to skip the checkpoint wait, cooling and power sized for sustained load. The job is ordinary: interactive coding, private document work, dead-end experiments, an occasional overnight batch. The alternative is paid API calls plus the machine you already own. Owning the box buys low-latency private access and open-weight models with no per-request approval. Tokens and GPU-hours are the intermediate numbers; what counts is verified developer-months of supported work.
Useful life is a judgment about model growth, platform compatibility, warranty, and the resale market for your exact configuration, not a number on a box. Current quotes, electricity rates, taxes, and API prices belong in an updatable worksheet, not prose you'll reread in eight months: the formula holds, the digits don't.
What goes on the bill, what doesn't
Start from one relationship: installed cost minus conservative resale, divided by useful years, gives annual capital cost. Keep the tax software's depreciation schedule separate from your own conservative estimate of early replacement and resale: only the second should drive a purchase.
Split fixed from variable. Card, host, and reserved capacity are fixed once bought; electricity, paid tokens for whatever the box can't cover, transfer, and some support scale with use. Human review sits in between: a dedicated operator is fixed, someone pulled in only when things fail scales with failure rate.
The bill of materials covers everything incremental: card, host, memory, storage, network, power, cooling, enclosure, spares, tax, shipping, installation, minus whatever you'd have bought anyway. A laptop you already use for ordinary work isn't an AI cost. A RAM upgrade bought purely to fit a model is.
Measure the same task three ways
Run the actual workload, not a benchmark, across the routes you're weighing: local, rented, API, roughly in that order per a local-first cascade. Record the same fields for each:
- accepted tasks against severe failures, not raw completions
- input, cached input, reasoning, and output tokens
- retries, abstentions, human repair minutes
- queue, first-token, completion, and p95 latency
- idle and unavailable hours, wall energy, maintenance time
Quality adjustment matters more than most budgets allow. A cheap route needing two attempts and five minutes of correction owes its real cost to the finished result, not the first draft. Don't benchmark a small local model that reliably nails one task against a frontier API at maximum reasoning effort. Not a fair fight, and the comparison will lie to you.
Idle time is not a rounding error
The biggest risk is low utilization dressed as savings: experimentation-driven upgrades, hobby time counted as business value. Build low, base, and high demand curves and watch daily peaks and p95 latency; a fully-loaded machine has no room for an interactive request right when needed. Batch work can fill quiet hours, but only if genuinely needed.
Meter idle states honestly: a workstation that sleeps nightly differs economically from a server holding models resident for instant answers. Shared systems: watch queueing and abandoned requests. Edge fleets: multiply update, replacement, backup, and travel time by node count; labor usually beats electricity as the real driver.
The recurring mistake is treating a long tax depreciation schedule as proof the accelerator stays competitive. It won't necessarily. Keep assumptions visible somewhere shared, date anything volatile, and don't let decimal precision hide how uncertain the inputs are.
Ten minutes saved is not ten minutes billed
Payback month is the first month cumulative discounted benefit clears cumulative cost; ROI over your chosen horizon is net discounted benefit over discounted cost. Both need a conservative useful life and resale value, because hardware often keeps running fine past the point where model growth or dropped software support makes it economically obsolete.
Productivity gains need a realization factor before they count as money. Ten saved developer minutes doesn't automatically become ten billed minutes; count it only when it avoids a hire, cuts outsourced spend, increases what ships, or shortens a genuinely binding queue. Run the model once with productivity value at zero: a purchase clearing the bar only on speculative savings deserves more scrutiny, not less.
Rent before you buy
Compare against renting too, not just the API. Hourly GPU rental earns its keep while demand and hardware shape are still uncertain. Budget for persistent storage, image prep, transfer, minimum billing, setup automation, and the instance everyone forgets to shut down. A short rental trial beats an expensive wrong purchase.
Base ROI on whichever is shorter: financial life or workload-relevant technical life. State the break-even boundary in operational terms: accepted tasks per month, productive GPU hours, max API cost per task, minimum useful life. That kind of boundary gets revisited; a narrative recommendation just gets re-argued. Retest when volume shifts materially, a model resets the quality gate, API prices move, electricity contracts change, or a component hits end of support.
The rule worth keeping: attach ROI to a workload, an alternative, and a dated set of assumptions, never to the box under the desk. Forget to update the dates and the number goes stale fast.