Overnight agents on local models: cheap, slow, surprisingly useful

The honest knock on local models for agent work is speed. On the 3090 box, a mid-size coder model gives me 25-ish tokens a second, and an agent chain a frontier API finishes in four minutes takes most of an hour. For interactive work that's disqualifying; I won't pretend otherwise.

But at 2 am, nobody's waiting.

Since late April I've run an overnight queue: bounded agent tasks dropped into a folder during the day, drained by a systemd timer at midnight, results sitting as branches when I get up. Latency stopped mattering the moment nobody was around to feel it.

what goes in the queue

Three task families earned permanent slots. CI triage: read the night's test failures, label flaky versus real, draft a one-paragraph diagnosis each. Doc drafts: changelog entries from merged PRs, docstrings for the untyped corners nobody loves. Dataset cleanup: dedupe and outlier-flag a 40k-row internal labeling set. Work nobody would ever do by hand. The shape matters more than the list: each task is bounded and machine-checkable, every result disposable. Each runs on its own branch; anything failing its acceptance check is deleted before I ever see it.

First two weeks, tallied: 31 tasks queued, 19 worth keeping, five self-discarded on failed checks, seven passed technically and were garbage anyway. The ratio has held since, roughly.

Overnight, a slow model and a fast model finish at exactly the same time: before you wake up.

task: ci-triage-nightly
max_turns: 40
timeout_minutes: 25
token_budget: 400000
on_check_fail: discard

the night it looped

Mid-June, a triage task met a test failing for infrastructure reasons (a registry timeout no code change could touch). The agent decided the test itself was wrong, edited it, failed verification, reverted, edited it again. Six hours. I found it at seven in the morning: 41,000 log lines, the same three actions repeating like wallpaper, the GPU pinned all night for nothing.

My first week had a milder case of the same disease: I queued refactor the auth module, felt clever, and woke to a 3,100-line diff that didn't compile. Both failures trace to one mistake: an unbounded task handed to a system with no instinct for giving up.

The fixes are dumb and they work: hard caps of 40 turns, 25 minutes wall clock and a 400k-token budget per task. A loop detector kills any run repeating the same tool call three times. A global kill switch fires at 6:30, whatever the state. The thinking is cribbed from long-running autonomous agents: the difference between an agent that runs long and one you can leave alone is entirely guardrails. One personal rule since the auth-module night: if the acceptance test won't fit in a shell command, the task doesn't queue.

the electricity math

Under load the 3090 pulls about 340 watts. Seven hours is call it 2.4 kWh, which at my Czech tariff is around fifteen crowns a night, well under a euro. The looped night burned north of nine million tokens producing nothing, and the marginal cost of that disaster was a warm closet. Push that volume through a frontier API at current rates and it's an apologetic-expense-note kind of mistake. That asymmetry is the real edge: when failure is nearly free, you can hand a mid model work you'd never pay frontier prices to even attempt.

In June I moved the runner off Ollama onto vLLM: draining four tasks in parallel is exactly the batched shape it's built for, awkward consumer-GPU install and all.

what I refuse to queue

Nothing that touches prod, and nothing that needs product judgment. If I can't verify it mechanically, it doesn't run at night either. Frontier models still win long agentic chains by a margin I can feel. Overnight local doesn't move that ranking, it only changes which tasks are worth attempting at all: the low-value-density work nobody would fund at API rates or engineer-hours. The queue itself is just async agent architecture shrunk to homelab scale. Fire, forget, then review an inbox at breakfast.

The honest limitation sits on my side of the keyboard. Twelve branches on a Monday is its own tax, and twice I've merged something I skimmed too fast and regretted it by Thursday. Cheap generation just relocates the bottleneck to my attention span. I'll take the trade — the model works nights, review became my morning shift, but it is a trade, and nobody warns you about the reading.