Do you actually need a vector database?

The conversation goes the same way almost every time. Someone decides to build a RAG feature, and within the hour there's a line item for Pinecone, or Weaviate, or Qdrant, or whichever vector database is having a moment. Nobody asks whether it's needed. "We're doing RAG" has become synonymous with "we need a vector database," and the two are not the same thing.

Let me make the contrarian case, because I think most teams reach for the dedicated store far too early and pay for it in ways they don't see coming.

What a vector database is actually for

A vector database stores embeddings and does fast approximate nearest-neighbor search over them. That's the job. The word doing the heavy lifting there is approximate: at large scale, finding the exact nearest vectors is slow, so these systems use clever indexes (HNSW and friends) that trade a little accuracy for a lot of speed, plus the machinery to filter on metadata, scale horizontally, and stay fast as the collection grows into the millions.

Read that again and notice what the value actually is. It's scale and index performance. It is not "RAG requires this." If you have fifty thousand chunks, exact nearest-neighbor search is already fast, and the entire reason the dedicated index exists doesn't apply to you yet.

What you can use instead, and probably should

For most projects starting out, the right answer is the database you already have.

pgvector turns Postgres into a perfectly good vector store. You add an extension, you get a vector column, you query it with SQL alongside everything else. The advantages compound fast: it's one system to run instead of two, your embeddings live next to the source documents they came from so they can't drift out of sync behind your back, you get transactions, and (this is the part people miss) you can do hybrid search in a single query, combining vector similarity with Postgres's full-text keyword search, which is exactly the combination good retrieval needs. One system, one query, both signals.

If you're building something local or embedded, SQLite with a vector extension does the same trick at smaller scale. And if you genuinely only have a few thousand vectors, you don't need a database for the search at all. A numpy array and a brute-force dot product against every vector will return results in milliseconds, and you can delete a great deal of infrastructure by admitting that. I've seen "vector database" line items justified by corpora that would fit comfortably in memory on a laptop.

When a dedicated vector database earns it

This isn't "never use one." There's a real threshold, and when you cross it the dedicated store is the right call.

You want one when you're into the millions of vectors and the ANN index performance is the thing standing between you and acceptable latency. You want one when you need serious metadata filtering at scale, or horizontal scaling your transactional database can't or shouldn't do, or operational separation between your retrieval workload and your application database so one can't take down the other. These are real reasons, and at that size pgvector starts to strain and a purpose-built system starts to pay for itself.

Vector databases aren't the problem here. Whether you actually have that much scale, with those performance needs, is the real question, and the honest answer is no far more often than the rush to adopt one suggests.

The cost nobody puts in the proposal

Here's the part that gets left out of the architecture diagram. A dedicated vector database is another distributed system you now operate. You run it, you back it up, you monitor it, you upgrade it. And critically, you own the consistency problem between it and your real source of truth: when a document changes, its embeddings are now stale, and you are the pipeline that has to notice, re-embed, and update the index. Get that pipeline wrong and your beautiful vector search confidently returns answers based on last month's version of the document, which is its own special kind of wrong because it looks completely fine.

With pgvector, that consistency problem mostly evaporates, because the vectors live in the same transactional database as the documents and you can keep them in sync in the same write. Splitting them into two systems is splitting them into two things that can disagree.

The thing this is really about

Step back and the vector store turns out to be the easy, commoditized part of RAG. The hard parts are the ones that actually determine whether your retrieval is any good: chunking on structure, combining vector and keyword search, and reranking the candidates before they reach the model. That's where retrieval quality is won or lost, and it's where teams should be spending their attention. Instead a lot of them pour weeks into choosing and operating a vector database and then wonder why their RAG still surfaces the wrong passages. The database was never the bottleneck.

So the boring, correct path: start with pgvector, or with brute force if your corpus is small enough to laugh at. Build good chunking, hybrid search, and reranking on top of it, because that's the work that matters. Move to a dedicated vector database the day your scale or latency numbers prove you've outgrown Postgres, and not one day sooner. Most projects never reach that day, and the ones that do will know, because the profiler will tell them, not the hype.