LlamaIndex's Document Parsing Protocol: 140 rules for the part of RAG that fails silently

On October 28 LlamaIndex published working draft 0.1 of the Document Parsing Protocol: 140 rules for what a correctly parsed page should look like, covering text, tables, forms, charts and diagrams. Every rule comes with real example pages beside the output it expects. That last part caught my attention, because most RAG teams I review have no written answer to "what should the parser have produced for this page?", so they cannot tell a parser bug from a retrieval bug.

The failure DPP is aimed at

The post by Gauthami Polasani and Pierre-Loic Doulcet states the problem plainly: two parsers can produce the same words with different meaning. A quarterly revenue table flattened into Markdown can keep every number and still lose which year sits above which quarter. A form with an empty field can come out as nothing at all, and your retriever will happily attach the label to the value on the next line. Nobody gets an error. The answer is simply wrong, with a citation.

The rules grew out of the annotation guidelines LlamaIndex used to label pages for ParseBench, its parsing benchmark. That origin matters to me. These are decisions annotators had to make page by page and then wrote down, not a committee's wish list.

Rules worth adopting even without LlamaIndex

A few rules from the post map directly onto bugs I have shipped.

Tables with a header spanning several columns become HTML tables, because "Markdown tables can't show a header that covers several columns". Headers stay as printed, so the year remains above its quarter columns. Wrapping rows in thead and tbody elements is optional "as long as a document uses one form throughout", the kind of consistency rule that makes downstream chunking predictable.

Forms get explicit placeholders: "an empty field becomes [blank]", and checkboxes stay as [x] and [ ]. For retrieval this is the important one. An explicit [blank] lets a model answer "the applicant left this empty" instead of borrowing the neighbouring value.

Charts turn into a table of their numbers, "one row per category and one column per series". Diagrams become Mermaid "when it can hold every label and arrow", and otherwise a description that keeps every label. Text hidden in the PDF that never shows on the page is left out, and visible but unreadable text becomes [illegible]. Element positions are optional.

A parser that guesses silently is worse than one that writes [illegible], because the guess gets embedded and retrieved with full confidence.

The hidden text rule deserves a second look from a security angle. Invisible text in PDFs is a known carrier for prompt injection into RAG systems, and a parser that follows DPP drops it by default.

What it costs you

HTML tables are heavier than Markdown. A wide financial table with colspan attributes can use noticeably more tokens than its pipe version, and every chunk that carries it pays that cost at embedding time and again at generation time. I would accept that for tables with spanning headers and keep simple grids in Markdown. Your chunker also has to treat an HTML table as one unit; splitting it halfway through the body rows produces chunks that are worse than the flattened Markdown you started with. I covered the rest of the ingestion side in the unglamorous half of RAG.

Then there are the limits of a 0.1 draft. The post describes no formal conformance test suite, so "DPP compliant" means nothing yet as a procurement checkbox. LlamaIndex also sells a commercial parser, and a protocol it authors will naturally fit its own output. That does not make the rules wrong, but read them as a vendor's proposal that is open for review. The repository licenses code under MIT and the protocol and examples under CC BY 4.0, with one Markdown file per rule grouped by chapter, so forking it into an internal standard is both legal and practical. Feedback goes in as GitHub issues or pull requests with the page and the expected output.

How I would use it this month

Treat DPP as an answer key, not a dependency. Pick the twenty ugliest documents in your corpus: scanned forms, multi-level financial tables, a few architecture diagrams. Write the expected output for each by hand using DPP conventions, and run your current parser against them in CI. Diff structure as well as text: count spanning headers preserved, blank fields marked, charts converted to rows. The repository's worked examples include IRS Form 1040, which is public domain and makes a good first fixture; the Apple FY2024 statement of operations is third-party copyright, so check the NOTICE file before you vendor it.

LlamaIndex also ships the rules as a Claude skill and as a ChatGPT reference file, which makes it cheap to try a model-based parser against the same answer key. The next time retrieval quality drops after a parser upgrade, that fixture set will tell you within minutes whether the regression sits in parsing or in retrieval. I learned that lesson the slow way, and wrote it up in your RAG demo lied to you.