The unglamorous half of RAG: getting documents in
RAG has a glamorous half and an unglamorous half, and almost all the attention goes to the wrong one. The glamorous half is retrieval: embeddings, hybrid search, rerankers, the stuff people tune and blog about. The unglamorous half is ingestion, the pipeline that takes your actual documents, in all their messy real-world formats, and turns them into the clean, well-chunked, correctly-tagged, up-to-date vectors that retrieval depends on. People learn this too late: retrieval quality is capped by ingestion quality. You cannot retrieve well from a corpus that was ingested badly, no matter how good your reranker is.
I've seen teams pour weeks into the retrieval side while feeding the system soup, and then wonder why the answers are wrong. The answers were wrong because the text going in was garbage.
Parsing the mess is genuinely hard
Real documents do not arrive as clean text. They're PDFs, which are the worst, plus HTML, wiki pages, Office documents, and scanned images. Extracting usable text from a PDF with two columns, tables, headers and footers, and the occasional image is real work, no library call solves it cleanly. Naive extraction produces a mess: tables flattened into a run of meaningless numbers, page headers spliced into the middle of sentences, reading order scrambled so paragraphs interleave. Whatever the model eventually answers is downstream of this text, so if the extraction produced soup, the answer is built on soup. This is the single most underinvested step in most RAG systems, and it's also the one where a little care pays off most directly in answer quality: better PDF handling, table-aware extraction, dropping boilerplate.
Chunking and tagging happen here, and they matter
The structure-aware chunking that makes retrieval work happens at ingestion time, not at query time. Splitting on real boundaries, keeping a table or a code block intact, keeping a heading with the text it introduces: all of that happens in the pipeline, and getting it wrong means every later query retrieves fragments that were broken before they were ever embedded.
Ingestion is also where you stamp metadata onto each chunk, the source, the section, the date, and critically the tenant and permission information that retrieval will later filter on. This is load-bearing for security: if you don't tag ownership at ingestion, you can't filter by it at retrieval, and you've built a cross-tenant leak into the foundation. The metadata you forget to attach at ingestion is the filter you can't apply later.
Freshness is a pipeline problem, and it's where RAG rots
The naive ingestion pipeline re-processes the entire corpus on a schedule, which is wasteful at any size and, more importantly, leaves a window where the index is stale. The real pipeline does incremental updates: it detects which source documents changed, re-ingests only those, and deletes the vectors for content that was removed or replaced. Doing that requires tracking the state of each source so you know what's new, and it's more work than a nightly dump, but it's the work that keeps the index true.
And keeping the index true is the whole game, because a stale index produces confidently wrong answers, citing a version of a document that stopped being accurate weeks ago, with no error to warn anyone. The ingestion pipeline stands between you and that failure, which means it needs an owner and an alert for when a source goes quiet. Treating ingestion as a one-time setup step, "we loaded the docs, we're done", is how RAG systems silently drift into lying. Documents change. The pipeline that notices is the difference between a RAG system that stays correct and one that slowly becomes a well-indexed archive of things that used to be true.
One more discipline carries over from the async world: ingestion has to be idempotent. Re-running it should update and replace, not duplicate, or you end up with three slightly different copies of the same chunk all competing in retrieval, which degrades results in a way that's annoying to debug.
None of this is exciting, and that's exactly why it gets shortchanged. But the embedding model you chose so carefully and the reranker you tuned are both working on whatever the ingestion pipeline handed them, and they can't recover quality the pipeline threw away. Spend your effort proportionally. The glamorous half is capped by the unglamorous one.