Vision models for camera-event understanding: structured output from images
So no, the vision-language model doesn't go first. Motion detection plus an object detector clears the easy cases; the VLM only gets paged for what a fixed rule can't parse. Schemas constrain an answer's shape, never its source: a VLM never sees your photo, only a resized, cropped, token-budgeted stand-in for it, and a fluent answer can be grounded, half-grounded, or built on a cropped-out detail.
the frame you actually fed it
The workload is doorway, workshop, warehouse, garden, and safety-camera frames: classify a bounded event from motion-gated stills without leaking a face into a log or repeating the same alert. Feed it timestamps, camera-zone metadata, and a narrow event vocabulary. "Describe what's happening" isn't gradable. A narrow question is.
what the eval record has to hold (visionmodels)
A log line with just a model name isn't an eval. Record dimensions, crop, resize, orientation, frame order and timestamps, OCR or metadata beside the pixels, token counts, latency, and whether the answer was grounded, unsupported, an abstain, or a repair, plus the actual image-detail value used. Local, via ollama-vision-local, add the projector, quantization, and preprocessing: a mismatched pair fails quietly while the answer stays fluent and wrong. Score precision, recall, duplicate-alert rate, and correct abstention, and judge unsupported claims apart from missing ones: an omission gets reviewed, an invented serial number gets acted on. Include frames with no real answer, and reward the honest one.
four claims wearing one sentence
The failure I watch for is one shaky visual cue dressed up as a confident identity or hazard call, dropped into a schema-valid field that automation trusts because it looks like what a person expected to see. Every conclusion needs a region or frame attached, visible to the reviewer. Four layers, four kinds of evidence: a red light is an observation, "that's the network panel" is association, "the link is down" is inference, "restart the gateway" is an action no one should trigger off the other three alone. Multiple frames add identity risk: label them, say if it's the same camera or time, and drop duplicates. Shuffle order in testing: a model with the same story after reshuffling isn't reading the sequence.
running it, not just scoring it
Put the provenance check in code, not prompt wording, and version model and pipeline so an upgrade replays against the same frames. Crop what you don't need to see, and set retention separately per artifact: originals, thumbnails, embeddings. Local cuts network exposure, but not an unencrypted disk or an overbroad dashboard account. Hosted, via gemini-cli-multimodal: upload time, regional routing, rate limits. Local: cold-load time, projector memory, heat, concurrency. What matters is the full path at realistic load, not tokens per second. A bigger model and more pixels aren't automatically safer: more room exposed, more room to answer past the evidence. I stop tuning once the gate clears at a latency and abstention rate I can live with, accepting I'll miss a few ambiguous events a looser model might catch. I'd rather run something that knows when it's guessing.