// calculator · local-model running cost

Local-model cost calculator

What it costs to run an open model on your own iron or rented in the cloud. Pick a model and concurrent users — it derives tok/s, GPU count, and price per 1M tokens (buy and rent) across ~90 cards, next to the API price. Decode is memory-bandwidth-bound, so everything comes from physics, not a hand-typed matrix.

model users

quant currency ctx/request electricity $/kWh volume M tok/mo

advanced

amortize (years) batch/node KV kB/tok PUE grid g/kWh

GPU	tier	tok/s	GPUs	VRAM	/1M buy▲	/1M rent	/mo HW	kWh/mo

I already own hardware

GPU owned target tok/s

Now actually run it

the 15-minute setup · Ollama in practice · model picks by tier · hardware guide · serving throughput

Methodology & caveats. First-order estimates (~±30%). Decode is memory-bandwidth-bound: tok/s ≈ (bandwidth × efficiency) / (active params × bytes/param). VRAM drives GPU count, batching drives users served. Real throughput depends on kernel, serving stack, and quantization — treat this as a comparison, not a quote. GPU prices are indicative (MSRP / typical street); click buy ↗ for the live Amazon price. Apple unified memory counts as VRAM (shared with the OS). See running models locally and hardware. The “buy” links are Amazon affiliate links — as an Amazon Associate I earn from qualifying purchases, at no extra cost to you.