AI hardware ROI for a shared team GPU server: depreciation and resale value

A team buys a shared GPU box, two 24 GB cards at a decent price, and for a year it's the right call. Then a new model needs more VRAM than the cards have. The vendor drops driver support for that GPU family. Or the workload moves from short completions to long-context retrieval that chokes the memory bus, not the compute. The hardware hasn't failed. It still boots, still passes the same benchmark. It's just worth less than the balance sheet claims. The value was never in the metal. It was in the metal's fit to a moving target.

the unit is a finished task, not a token

This is a multi-user Linux host: one or more GPUs, server-grade memory, redundant storage, networking, monitoring, someone's name on the pager. The workload is coding assistants, retrieval-augmented generation, extraction, evaluation, and overnight jobs from several people. The alternative is commercial model APIs or rented inference sized for the same traffic; if you haven't priced that honestly, see why the subscription math for AI spend usually loses.

Owning the box pitches pooled utilization, controlled data placement, stable model versions, and reusable local capacity. Price the right unit: one accepted task at the required p95 latency, not tokens, not accelerator-hours, just readouts. Useful-life assumptions track model growth, platform compatibility, warranty terms, and the resale market for your exact configuration. Keep quotes, electricity rates, and API prices in a worksheet you revisit; prices move faster than the formula does.

what actually goes into the number

Start from the capital side, kept separate from whatever your accountant calls depreciation:

annual_capital_cost = (installed_capex - conservative_resale) / useful_years

Model early replacement and conservative resale separately. Split fixed from variable: GPU, host, installation, and reserved capacity are fixed; electricity, paid tokens for overflow, data transfer, and part of the support contract move with the work. Human review sits in between, a dedicated operator is fixed cost, correction minutes scale with how often the model is wrong.

The bill of materials runs past the GPU line: accelerator, host, memory, storage, network gear, power, cooling, enclosure, spares, tax, shipping, installation. Subtract whatever the org would buy anyway: a laptop already in use isn't an AI cost; a memory upgrade bought solely for a model is.

Quality adjustment matters more than people credit. A cheap model needing two attempts and five minutes of cleanup gets priced on the finished result, not the first draft. A small local model that reliably clears one narrow job all day shouldn't be judged against a frontier API at maximum reasoning effort.

the box that's on but not earning

Main risk: queues, idle capacity, operational labor, single-host outages, demand outgrowing whatever shape you bought. Build low, base, and high demand curves with daily peaks and p95 in them; a machine at full capacity has no room left for an interactive request mid-batch. Batch work fills quiet hours only if the org genuinely needs it done.

Meter idle states separately: a sleeping workstation and a server holding models resident for instant response differ economically. On shared systems, watch queueing and abandoned requests. On edge fleets, multiply update, replacement, backup, and travel time by node count; operational labor can dominate the electricity line.

Log this per route, local box or rental or API:

• accepted tasks and severe failures
• input, cached input, reasoning, and output tokens
• retries, abstentions, human repair minutes
• queue time, time to first token, completion time, p95
• active, resident-idle, sleep, and unavailable hours
• wall energy and cooling allocation
• maintenance and incident time

Recurring mistake: a long tax depreciation schedule taken as proof the card stays competitive in year five. Two clocks, not one. Keep assumptions in a small visible table, a source and date on every number that moves, no hiding uncertainty behind decimal places.

the month the math flips

Payback month: the first month cumulative discounted benefit exceeds cumulative cost. ROI over your horizon: net discounted benefit divided by discounted cost. Both need a conservative useful life and residual value, since hardware can stay functional after a model generation or a dropped support contract makes it economically obsolete.

Productivity benefits need a realization factor: saving ten minutes of a developer's day doesn't automatically become ten minutes of billable revenue. Count time only when it avoids a hire, cuts outsourced spend, adds deliverable output, or shortens a binding queue. Run the model once with that number at zero; hardware justified only by speculative time savings deserves the most scrutiny.

Compare against renting too: hourly rental earns its keep while demand and shape stay uncertain, the same uncertainty behind when staying local is even worth the trouble. Add persistent storage, image prep, transfer, minimum billing, setup automation, the forgotten idle instance. A short rental benchmark beats an expensive wrong guess. I wouldn't bother with a full NPV table for a two-card box: a spreadsheet with a payback month is plenty.

Base it on the shorter of financial life and workload-relevant technical life, stated in operational terms: accepted tasks per month, productive GPU hours, maximum API cost per task, minimum useful life. Put a retest date on it: recalculate when volume changes, a model changes what counts as passing, API prices move, the electricity contract changes, or a component goes end of support.

A GPU doesn't hold its value. A workload, an alternative, and a set of dated assumptions do. Date them, or don't bother running the numbers at all.