Vision models for camera-event understanding: reasoning across multiple images

Four frames land in the queue: something crosses the driveway in the dark, crosses again a bit later, then an empty frame, then a shape near the fence. Somebody has to turn that into one alert, one label, no false confidence. Hand all four to a vision-language model in one prompt and it will happily invent a story connecting them: fluent and grounded look identical on screen.

That's the job: doorway, workshop, warehouse, garden, whatever cameras you run, classifying a bounded event from a handful of frames without repeat alerts for one visit. Don't reach for a VLM first. Run motion detection and a plain object detector, and escalate only when a fixed rule can't cover the ambiguity, the object relationships, or the language involved. The model's job stays narrower than "look and tell me what happened": stable identifiers per frame, plus a statement of what may be compared across them.

what four frames actually give the model

The model never sees your driveway. It sees a resized, cropped, tiled, token-budgeted stand-in, and that gap is where confident nonsense comes from. Label every image before it goes in, keep timestamps and camera identity attached, then rerun the same prompt on the same frames reordered.

model, runtime, prompt revision
crop, resize, image count
frame timestamps, ordering
grounded / unsupported / abstain

Using Gemini CLI's multimodal handling? Log the image-detail value it used, not the default. On a local vision stack under Ollama, log the projector and quantization: a mismatched pair keeps answering fluently while quietly wrong.

one shaky clue does not make an identity

The failure that survives review: one uncertain visual clue turned into a confident identity, hazard, or security call, because it sounds like what a person would guess anyway. Split conclusions into layers: observation ("a red light, upper right"), association ("belongs to the panel"), inference ("connection failed"), action ("restart it"). None of those chains into the next automatically.

• require a frame or visible feature behind every conclusion, shown to the reviewer
• drop duplicate frames, they burn tokens without adding evidence
• score an invented identity apart from a plain "not enough evidence"
• if shuffling the frame order doesn't change the story, the sequence isn't doing anything

the rule lives in code, not the prompt

Reduce the set, label each view, compare only what shares an identity, and put that in preprocessing and validation code, not editable prompt wording. Crop faces and screens before and after inference, and set retention separately for originals, thumbnails, and debug captures. Local inference cuts external transfer; it does nothing for an unencrypted disk. Stop tuning once precision, the duplicate-alert rate, and abstention clear your bar.

I'd rather ship a system that says "can't tell" too often than one that's fluent and wrong. I'll admit the fluent one is usually the one I've shipped under deadline.