Metadata filters are cheaper than better embeddings

And that's the argument I keep having with people who want a better embedding model before they've touched their metadata: you're spending effort on the wrong stage of the pipeline. A vector score is a guess dressed up as a number. A tenant ID, a product line, a schema version, a language tag, a document's effective date, those are facts you already had before running a single similarity comparison. Use them first.

Tenant, version, language, in that order

Picture a shared local RAG service backing several products across a handful of release lines. A private chat used a couple of times an hour, a coding assistant loop, and a nightly extraction job can all share the same index. Nothing about that setup says "buy a bigger embedding model." It says most of the corpus is irrelevant to any given query before you've compared a single vector, and you already know which part. Make metadata trustworthy at ingestion, not bolted on afterward, and derive filters straight from the request context.

filter:
  tenant: request.tenant_id
  product: request.product_line
  version: request.schema_version
  language: request.locale
  effective_before: request.now

That block runs before the vector search, not alongside it. Whatever survives is at least eligible to be right, a cheaper problem than making retrieval smarter about documents that were never in scope. Capture a baseline while you're at it: one named model, one fixed prompt set, the exact launch command, because those details are easy to lose. Track time to first token, decode rate, completion time, peak memory, queue delay, and, for the nightly job, jobs finished per hour.

The wrong-version answer that reads perfectly fine

Here's the failure that costs the least to prevent and the most to fix later: you search the whole corpus, the top match belongs to the wrong version or the wrong tenant, and the model writes a fluent answer from it anyway. Nothing errors. Nothing looks wrong in a demo. The text reads fine, and that's the problem, because fluent and correct are different properties and only one is easy to check by eye. Local inference layers its own quiet failures on top: a model partly offloading to CPU, a cache missing without complaint, swap creeping up, a queue holding requests nobody is waiting on anymore, a fallback path that changes where data is allowed to leave the machine. None of it raises an exception. You catch it by watching runtime logs and OS counters while the thing runs, changing one variable at a time, repeating enough to rule out a lucky run, and reading the output instead of trusting that faster means equivalent.

Keep the setup boring, too. A filter layer needing manual repair after every model or driver update isn't saving you anything, it's borrowing time from next month; keep it rebuildable from a short script instead. Stop once the workload meets its quality and latency target with real headroom. That margin isn't wasted capacity, it's what absorbs a longer document, one more user, or the next runtime update.

The rule I'd keep, stated flat: apply hard facts before probabilistic ranking.