Local vision models earn their disk space

For about a year I treated local vision models as a parlor trick: point one at a photo, hear that it's a cat on a windowsill, move on. What changed my mind was paperwork. My accountant wanted receipts for fourteen months of expenses, those receipts were 531 photos scattered through my camera roll, and I wasn't about to upload fourteen months of my financial life to somebody's API to get them sorted.

So on a Sunday in late May I pointed a vision model at the pile: Qwen's VL line, run through my usual Ollama setup on the M2 Ultra.

Expectations were low.

It read crumpled thermal-paper receipts photographed at an angle in restaurant lighting. It read a fuel receipt that had been through the washing machine. Of the 531 photos, 41 came back flagged for a human (unreadable totals, one picture of my thumb), and the rest landed as clean rows. I hand-checked a random hundred and found three wrong amounts, all on faded thermal paper. That's a far better hit rate than I'd have managed myself by the third hour of manual entry.

The piece that made it usable was pairing vision with structured outputs: the overnight batch hit the API with a JSON schema, so vendor, date, amount, and currency arrived as fields instead of chat. For quick spot checks, the CLI takes a file path right in the prompt:

ollama run qwen2.5vl "vendor, date, total, currency as JSON: ./IMG_4471.jpg"

the resolution wall

Then I got cocky and threw the same setup at full-page A4 invoices, and the first overnight run produced confident garbage. A vision model sees your image only after it's been resized and tiled down to what the encoder can chew, and a dense invoice squeezed through that budget turns 9-point line items into gray mush. The model stays confident either way; you only find out by checking.

Too-small text doesn't come back as an error. It comes back as a plausible number that's wrong.

The fix was dumb and effective: split each page into halves, run the halves separately, stitch the fields back together. Usable extractions on the invoice batch went from roughly a third to all but four. It cost me one burned night's run to learn the rule I now follow everywhere: spot-check ten outputs before trusting a thousand.

the three jobs that stuck

Since the receipts, local vision has settled into three standing roles here:

• Screenshot to bug note: paste the broken-UI screenshot, get the visible error text pulled out and a first-draft repro description for the ticket.
• Offline document extraction: receipts, delivery notes, the odd contract page, anything I'd hesitate to upload anywhere.
• Alt text in bulk: 1,850 product photos for a client's shop, drafted overnight on my desk, reviewed by a human in the morning.

The alt-text job is the one that surprised the client. Their unreleased product photos never left my machine, which their legal team cared about considerably more than the writing quality. The quality turned out fine after review too. It's the same shape as running Whisper locally for meeting audio: the pipeline has no upload step for anyone to audit, because there is no upload.

Honest limits, since there are real ones. Blurry input produces confident fiction, so I validate whatever's checkable and sample the rest. Charts and dense layouts still clearly belong to frontier models. Counting objects is a coin flip. So the local model gets the work where volume is high and stakes per item are low. That's also where the data shouldn't travel, which turns out to be more of my work than I'd guessed.

A year ago I'd have called vision the least useful thing a local model does. Now it's half the reason the M2 Ultra earns its desk space — and the receipts never left it.