AI hardware ROI for a used-GPU inference build: electricity and cooling economics

And no, multiplying the GPU's rated draw by however many hours it churns through tokens will not get you an honest number. That shortcut ignores the host board, the fans, the PSU's conversion loss, and the fact that the card sits idle far more than it sits at full tilt. A used-GPU inference build is exactly the purchase where this bites hardest: you bought the card because the price-per-gigabyte of memory looked good, not because the card is efficient, and efficiency is precisely what the electricity bill measures.

count everything that pulls current

Start from a formula you can actually audit:

energy_cost = wall_kW × active_hours × electricity_rate × cooling_factor

Then meter the whole system through sleep, idle, model-resident, and active states, and weight each by the real duty cycle rather than the one you hoped for. Split fixed costs (the card, the board, memory, storage, the case, the PSU, spares, tax, shipping, installation) from variable ones (electricity, any paid API overflow, data transfer). A dedicated operator is a fixed cost; per-exception human repair time is not. Subtract anything the team would have bought regardless of this project, since a general-purpose workstation doesn't become an AI cost just because it occasionally runs a model. Log accepted tasks against severe failures, token counts by type, retries and abstentions and repair minutes, latency at p95, and hours spent in each power state, because a cheaper model that needs two tries and five minutes of human cleanup isn't actually cheaper. It's the same instinct behind 99-percent-cost-architecture: the last mile of the bill is where the real money hides.

the used-card risk nobody prices in

Here's what a spreadsheet won't warn you about: unknown wear, zero warranty, a platform that gets orphaned by a driver update, and a resale market that may not want your card back. Model low, base, and high demand, and remember that a box running at full utilization has no headroom left to absorb an interactive request without a queue forming. Idle metering matters too: a workstation that sleeps overnight is a different animal from a server holding several models resident so it can answer instantly, and if you're weighing that against renting or an API instead, the reasoning is the same one behind local-first-cascade, route by cost and latency, not by habit. Payback month is just the first month cumulative benefit clears cumulative cost, ROI the ratio over your horizon, and both need a residual value that assumes the card outlives its warranty, not its usefulness. Run the productivity side at zero once too, since ten saved developer minutes isn't automatically ten dollars of revenue, and a short rental benchmark on someone else's hardware will tell you more than another spreadsheet tab. Set the break-even in operational terms, accepted tasks per month, max tolerable API cost per task, minimum useful life, and write down a retest date tied to volume, model, price, or component changes.

None of that saves you from buying a lemon off a marketplace listing. It only tells you sooner.