Vision models for product-image analysis: privacy and security for visual inputs

So no, the model never looked at your photo. By the time a product image reaches it, the picture's been resized, cropped, and squeezed into a token budget, and the answer describes that version, not the shelf.

Narrow the job or you can't test it: catalog photos, packaging, defects, visual search, describe attributes, compare items, flag a defect, retrieve similar products, fed by controlled views and catalog metadata.

The privacy call is dull: collect less, redact before storage, cap retention, gate access, treat image text as untrusted input, never an instruction. Try embeddings against deterministic filters first; a vision-language model earns its place once that baseline hits layout, ambiguity, or language it can't handle.

keep a receipt for every pixel

Map every frame you store, crop or redact before it leaves the client, isolate tenants, log per call: model, runtime, prompt revision, dimensions and crop, image count, OCR data, token counts, latency, memory, and whether it landed grounded, unsupported, an abstention, or a repair. Log the real detail-level value; on local models like Ollama, the projector and quantization can drift from the text weights while the server answers fluently anyway.

Grade on attribute precision, retrieval relevance, defect recall, and view consistency. Score an unsupported claim apart from a plain omission: a missed detail earns review, a fabricated attribute drives a wrong action. Include unanswerable cases; reward admitting the pixels don't show it.

local does not mean private

The failure that survives review is inferring material, authenticity, dimensions, or condition the pixels never established, because the answer matches what a person expected. Calling a system private because inference runs locally is the same trap: raw images still sit in logs and backups indefinitely. Require a region or visible feature behind every conclusion, and show it, not bury it in a log.

Separate observation, association, inference, and action: a red indicator is observation, that it's the network panel is association, that the connection failed is inference, restarting the gateway is action; don't let the first jump straight to the last.

Multiple images add identity risk: label each one, drop duplicate frames, state whether they're one object or a sequence, and test a shuffled version if order matters, since matching stories after reordering means it isn't using the sequence.

Capture the smallest region, retain it briefly, crop rooms, faces, screens, addresses, and reflections before they reach the model, never let image text authorize a tool call, enforce it in code, log versions for replay. Retain originals, thumbnails, OCR, embeddings, prompts, and debug captures separately. A local model cuts external transfer. It does nothing for an unencrypted disk or an overbroad dashboard account.

Test a hosted path via a Gemini CLI multimodal flow on upload time, regional routing, and provider errors; a local one on cold loading, projector memory, and heat under concurrency. Compare the whole thing at p50 and p95, not tokens per second. Stop once the gate passes at acceptable latency and cost: bigger and more pixels aren't automatically safer, just more likely to leak. I still don't have an answer for how much resolution is enough before you're handing over somebody's kitchen.