Vision models for product-image analysis: resolution and visual-token budgets
Every vision-language model, hosted or local, resizes, tiles, or crops your product photo into a fixed visual-token budget before your prompt reaches the transformer. Nobody flags this. The picture you uploaded is not the picture the model reasoned over, and a fluent answer can be fully grounded, half grounded, or built around a detail that got downsampled away before inference started.
For product-image work the material is catalog photos, user uploads, packaging shots, defect photos, and visual-search queries. The job: describe an attribute, compare two items, detect a bounded defect, retrieve similar products. Feed it multiple controlled views, catalog metadata, allowed attribute values, and an image-quality signal, and the task becomes something you can measure, not "understand this image."
Useful pixels are set by the smallest evidence the task needs to resolve, not by the camera's maximum output. Start with the boring option: image embeddings plus deterministic catalog filters. A vision-language model earns its place once that baseline hits layout, ambiguity, relationships between items, or language fixed rules can't handle economically.
measure the resize, not the vibe
Sweep image dimensions and crop strategies and log task accuracy, visual-token counts, first response, and total cost. Keep the original asset untouched and record every transformation before the model call: "we resized it" is not a debuggable variable. A useful evaluation record holds more than a model name.
| field | why it matters |
|---|---|
| model ID, API/runtime, prompt revision | pins the exact config |
| dimensions, crop, resize, orientation | what the model received |
| image count, ordering, timestamps | catches sequence bugs |
| OCR or metadata beside pixels | text evidence versus vision evidence |
| visual, input, output token accounting | ties accuracy to budget spent |
| first response, latency, peak memory | decides if this ships |
| grounded pass, unsupported claim, abstain, repair | the outcome that matters |
Log the image-detail or resolution value the provider used, not the default. Locally, record the vision projector, quantization, context length, and preprocessing implementation: text weights from one release paired with a projector from another can fail quietly while the server keeps talking about a picture it mangled.
Score attribute precision, retrieval relevance, defect recall, cross-view consistency, and unsupported-claim rate, and keep that last number separate from plain omissions. A missing detail sends something to review; an invented serial number, UI control, trend, or material call sends the system straight into a wrong action. Put unanswerable examples in the eval set and reward the model for saying so.
the four-word leap from red light to reboot
The failure that bites is inferring material, authenticity, dimensions, or condition the pixels can't establish. The resolution-specific version: downscale a page until the text disappears, or upload a full sensor image to inspect one part the size of a coin. Both survive review because the answer matches expectation.
Require a region, frame, label, or visible feature behind every conclusion that matters, and show it to the reviewer instead of burying it in a log. Split the reasoning into layers: "red indicator, top right" is observation, "on the network panel" is association, "the connection has failed" is inference, "restart the gateway" is action. Each layer needs its own evidence and authority; a plausible observation should never skip straight to an irreversible command.
Multiple images add identity risk. Label every input: same object, a page sequence, one camera, one point in time, and strip duplicate frames before they burn tokens for nothing. If order matters, attach timestamps and run a reordering test; a model producing the same causal story after a shuffle isn't using the sequence.
privacy and latency live in the same budget
The rule that survives production: crop to the evidence, keep enough pixels for that region, cap everything outside it. Put that rule in preprocessing, routing, or authorization code, not prompt prose a future edit breaks, and store the model and pipeline version with every result so an upgrade can be replayed against the same images.
Privacy applies on both sides. Crop irrelevant rooms, faces, screens, addresses, and reflections where you can, and set retention separately for originals, thumbnails, OCR text, embeddings, prompts, and debug captures: they don't share a lifespan. Local cuts external transfer, not an unencrypted disk or an overbroad dashboard account.
Route through a hosted setup like Gemini CLI's multimodal mode and measure upload time, regional routing, retention, rate limits, and provider errors in the same test. Build local the way Ollama handles vision models and the cost moves to cold load time, projector memory, CPU or GPU placement, preprocessing time, heat, and concurrency. Compare the whole path at p50 and p95, not the tokens-per-second figure on a vendor slide.
Stop tuning once the visual gate passes at acceptable latency, cost, and abstention. A bigger model and more pixels don't make this safer by default; they can leak more private detail and tempt the system into answering past what the picture shows. Crop to what the task needs to see, and let the model say it doesn't know.