Unlimited-OCR-2 in a RAG ingest pipeline: great across pages, weaker on headings

Baidu released Unlimited-OCR-2 on 27 October under the MIT license, an OCR model that parses a long document in one pass instead of page by page. If your RAG pipeline gives wrong answers about tables that cross page breaks, it is aimed at exactly that failure. Most RAG failures I debug start before the embedding model sees a single token. A table splits across pages 14 and 15, the parser emits two half tables, the chunker cuts them apart, and the retriever returns the header row without the numbers. The model was downloaded about 190,000 times in its first month, so I am clearly not the only one who wants that fixed.

What it is, briefly

It is a 3B model with 500M parameters active per token, built on DeepSeek OCR: a DeepEncoder for the page images and a mixture-of-experts decoder, per the paper. The decoder uses Sparse Reference Sliding Window Attention, SR-SWA, which caps the reference tokens it attends to at a fixed budget, so the cost of each decoding step stays flat as the document grows. The authors report a 7.0x per-step speedup over full attention at the hundred-page scale, and with their unlimited-horizon decoding, peak KV memory at batch size 1 falls from 47.00 GB to 1.77 GB. That is context for this article, not the point. The point for ingest is that one request can see page 14 and page 15 together.

Parsing quality: where it wins and where it does not

The paper's numbers split cleanly into two stories. On multi-page parsing it leads by a wide margin. On MPDocBench, documents of dozens of pages, it scores 84.59 overall against 78.30 for Qwen3.8-Max and 71.79 for Gemini-3.1-Pro Preview. On MarathonDocBench, 100 to 300 pages, it scores 77.06 while the best baseline, Kimi-K3, gets 33.74. The cross-page table metric is the one I care about for RAG: 71.11 TEDS on tables that span page breaks, against 26.30 for the best baseline.

Two caveats on those numbers. MarathonDocBench is a benchmark the same authors introduced, so I read the margin as directional until someone else runs it. And on single pages the picture is ordinary: OmniDocBench v1.6 overall is 94.02, barely above its predecessor's 93.92 and behind TeleOCR at 96.91 and OvisOCR2 at 96.47. If your corpus is invoices, receipts and one-page forms, this model buys you little.

The paper also names the limitation that hits retrieval hardest. Sparse attention does not preserve global document information, and the authors say this hurts document-level structure such as heading hierarchies. For RAG that is far from cosmetic. Section-aware chunking, breadcrumbs in chunk metadata and "chapter 4, section 2" citations all depend on a correct heading tree.

The cross-page tables are the reason to adopt it, and the heading tree is the reason you cannot trust it alone.

How I would wire it into an ingest pipeline

I would route by document shape rather than replace the existing parser. Single-page and short documents keep going through whatever you use today, and I still run classic OCR before a vision LLM when the layout is simple. Long reports, contracts and manuals with tables crossing pages go to Unlimited-OCR-2.

Then repair the structure separately, because a wrong heading tree fails quietly. If the model files section 4.2 under chapter 3, every chunk from that section carries the wrong breadcrumb, and your assistant cites a chapter the answer never came from. Where the PDF has an outline or table of contents, rebuild the heading hierarchy from it and ignore the levels the model emits. Where it has none, run a cheap heading detector over the parsed text and compare the two trees. For tables, count the columns in rows that came from adjacent pages: a merged table that keeps the same column count across the break is a good sign, and a ragged one goes to a review queue before it reaches the index.

The word unlimited needs a footnote. For very long documents the SGLang setup on the model card reads windows of 10 pages, and the card warns that smaller windows hurt cross-page merging, because a block that crosses a window boundary loses its context. If a table starts on page 10, ends on page 11 and a window boundary falls between them, you are back to two half tables, the very failure you adopted the model to fix. So the window size is a quality setting as much as a memory setting, and the token caps matter too: per-page output is capped at 20,000 tokens through an environment variable, and the example client budgets 8,000 per page. This is a GPU server job, not a laptop one. Serving needs a custom SGLang wheel or Transformers with trust_remote_code, and the tested setup was an A800 80GB. The number that decides a backfill is in the paper: at batch size 16, unlimited-horizon decoding reaches 1,315 tokens per second against 360 for standard decoding, about 3.7 times the throughput, which is what makes reparsing a whole archive on one GPU realistic.

The rest of my RAG ingestion notes still applies: keep the raw parse, version it, and make reparsing cheap, because you will swap parsers again within a year.

What to do first: take twenty of your own long documents that contain tables crossing page breaks, parse them with your current tool and with Unlimited-OCR-2, and score only two things, table integrity and heading levels. That afternoon tells you more than either benchmark.