Vision models for document vision: video through frame sampling
And that's the trap: a resize, a crop, a tile grid, and whatever survived the token budget are what the model actually receives, not the page you scanned. It answers fluently regardless, grounded or not.
The page you send isn't the page it sees
Document vision means scanned forms, invoices, reports, mixed-layout PDFs: text, fields, tables, layout relationships, and source locations, without losing qualifiers. Feed it page images with OCR text, page numbers, and a target schema, and "understand this image" turns into something scorable. Start with OCR plus deterministic parsing; a vision-language model earns its place once that baseline hits layout, ambiguity, relationships, or language fixed rules can't handle economically. Score field accuracy, table structure, source coordinates, abstention, and review time, and load in real unanswerable examples so it can say so.
Sample for the event, not the clock
Video is where teams get lazy: grab a frame every few seconds and call it coverage. Don't. Sample on motion or scene-change signals, keep before-and-after context, and score recall against annotated clips, not a fixed interval. Log the chain per call: dimensions, crop, resize, orientation, count and order, timestamps, OCR alongside the pixels, token accounting, latency, peak memory, and whether it landed grounded, unsupported, abstained, or repaired. If the API exposes an image-detail knob, log what it actually used, not the default, same as with Gemini CLI's multimodal mode. Local setups add one layer: projector, quantization, context length, preprocessing, since mismatched weights and projector fail quietly while the server keeps talking. Ollama's vision path exposes those knobs, worth the setup time.
Where a plausible answer stops being a true one
The failure that survives review is a field value copied from the wrong row, page, or section. The video version is a real event that fell between two sampled frames; a reviewer blames the model for missing it. Demand a region, page, frame, or visible feature per conclusion, and show that provenance, not bury it in a log. Separate observation from association from inference from action: a red indicator, then the network panel, then the connection failed, then restart the gateway. An invented serial number is worse than a blank one; score unsupported claims apart from omissions. With more than one image, label them, state the relationship, drop duplicates, and shuffle the order once as a control.
After the answer ships, the pixels are still your problem
Crop what isn't needed, before and after inference: rooms, faces, screens, addresses. Set retention separately for originals, thumbnails, OCR text, and debug captures; a local model fixes the transfer problem, not an unencrypted disk or a loose account. Test upload time and region for hosted, cold load, heat, and concurrency for local, and compare the whole thing at p95, not tokens per second. Stop once the gate passes at acceptable latency, cost, and abstention. A bigger model isn't automatically safer, just more exposed.
If I keep one rule: build the sampler around the event, not the interval, and stop trusting any answer that can't point to its pixel.