AI hardware ROI for a used-GPU inference build: depreciation and resale value

A pair of used 24GB cards, a secondhand workstation board, and a case built for someone else's mining rig: that's the build, and it starts losing money before a single fan bearing fails.

The workload behind it is steady, unglamorous local inference, the kind where raw memory capacity matters more than whatever generation of tensor cores shipped last quarter. The alternative is new consumer hardware, a refurbished workstation, or paying per token through an API and skipping ownership entirely. The used-card route wins on purchase price and on VRAM per currency unit, for as long as the cards keep working and the platform keeps supporting them. That "for as long as" is doing more load-bearing than most spreadsheets admit.

Here's the part people skip: what you're buying isn't tokens per second, it's a quality-gated inference hour, delivered reliably, for as long as the hardware stays functional and worth running. Tokens and GPU-hours are just how you get there. The assumption that decides whether the build pays off isn't the purchase price, it's how long the configuration stays relevant, given model growth, driver and framework support, whatever warranty exists, and what the resale market for that exact card will bear. Put current prices, electricity rates, and API rates in a worksheet you update on a schedule, not in prose you'll reread next year and trust by accident.

Count everything that isn't the card

Annual capital cost is installed cost minus a conservative resale estimate, divided by realistic useful years. None of that should be borrowed from the tax code. Model early replacement and a pessimistic resale number separately from whatever the asset is worth on paper under accounting depreciation, because the two rarely agree, and the paper figure is the less useful one.

Split the bill into fixed and variable first.

Labor is the interesting row: a dedicated operator is fixed regardless of load, but time spent fixing bad output scales with how often it happens.

The bill of materials needs every incremental piece: accelerator, host, memory, storage, network, power, cooling, enclosure, spares, tax, shipping, the afternoon lost installing it. Then subtract whatever you'd have bought anyway. A laptop you already use for work isn't an AI cost just because you also run models on it. A memory upgrade bought specifically so a model would fit is.

None of it means much without quality-adjusted measurement across local, rented, and API routes. Track accepted output against severe failures, not raw completions. Track retries, abstentions, and minutes spent repairing bad output. Track queue time, time to first token, completion time, and p95, since averages hide the tail that annoys real users. Track active, resident-idle, sleep, and unavailable hours, plus the energy and cooling load attached to each. If a cheaper model needs two attempts and five minutes of correction, that cost belongs to the finished result, not the first pass. And don't price a local model that reliably clears one narrow job against a frontier API run at maximum reasoning effort: they aren't doing the same job.

Idle time is where the estimate lies to you

The real risk here is unglamorous: wear you can't see, no warranty, a driver update that quietly drops support, higher power draw than the listing implied, downtime, and a resale market thinner than hoped when you need to sell. Model demand at low, base, and high levels, and build daily peaks and p95 latency into all three, because a machine scheduled at full utilization still has to absorb someone who wants an answer now. Batch work can fill the gaps between peaks, but only count it if it's genuinely needed, not invented to flatter the utilization number.

A workstation that sleeps overnight carries a different cost profile from a server holding several models resident for instant answers, and treating the two as equivalent produces a number that's wrong in an avoidable way. On shared systems, measure queueing and abandoned requests: those show when the box is actually the bottleneck, not the model. On a fleet of edge boxes, multiply update, replacement, backup, and travel time by the node count, because operational labor can quietly outweigh the electricity bill.

The recurring mistake is treating a long tax depreciation schedule as evidence the hardware will still be operationally competitive at the end of it. Those two clocks run at different speeds. Keep every assumption in a small table with a source and a date attached to anything that moves, and resist the urge to hide uncertainty behind three decimal places.

Find the month it actually pays for itself

Payback month is the first month cumulative discounted benefit clears cumulative cost. ROI over whatever horizon chosen is net discounted benefit divided by discounted cost. Both numbers are only as honest as the useful-life and residual-value assumptions behind them, and hardware can keep running after model growth or a support cutoff makes it pointless.

Productivity claims need a realization factor before anyone trusts them. Saving a developer ten minutes a day doesn't automatically turn into ten minutes of anything billable. Count that time only if it avoids a hire, cuts outsourced spend, increases what ships, or shortens a queue that was genuinely binding. Run the whole model once with the productivity term set to zero; if the purchase only makes sense because of speculative time savings, that deserves a second, harder look.

Compare renting against buying too. Hourly GPU rental earns its keep when demand or card shape is still uncertain, and a short rental benchmark is often cheaper than an expensive wrong purchase. Price the persistent storage, image setup, transfer, minimum billing increments, and the instance someone forgot to shut down. I've written before about a cost model that survives the next price move for token costs, and the same discipline applies here.

Write the boundary down, then put a date on it

Base the ROI on whichever life is shorter, the financial one from the amortization schedule or the technical one where the workload still fits the hardware. State the break-even boundary in operational terms: accepted tasks per month, productive GPU-hours, the maximum API cost per task that would make renting cheaper, or the minimum useful life needed. A number like that is easy to revisit. A narrative recommendation from months back is not.

Then set a retest date and name what moves it up: volume changes materially, a new model changes what counts as passing, API prices move, the electricity contract renews at a different rate, or a component reaches end of support. None of that is optional bookkeeping. It keeps the rest of the analysis honest.

Someone will eventually suggest a flat-rate plan would have been simpler, and for a workload this uneven, that's worth taking seriously: the same argument behind why a subscription is often the wrong default for uneven AI usage.

Put a date on the assumptions, not just on the invoice.