pdf-inspector: Stop Sending Every PDF Through OCR

Mon, 03 Aug 2026 00:00:00 +0000

Ever watched your RAG pipeline burn $3 and 9 seconds on OCR for a PDF that was never scanned in the first place? Yeah, me too. Every PDF hits the OCR service by default because nobody knows what’s actually inside until it’s too late. So when Firecrawl shipped pdf-inspector — a Rust classifier that tells you in ~10ms whether a PDF even needs OCR — it hit Trending for a reason. 6,633 stars and climbing fast.

What pdf-inspector actually does

The short version: it’s a two-stage tool built in pure Rust (MIT, single lopdf dependency, zero ML models). First it classifies — TextBased, Scanned, ImageBased, or Mixed — by sampling content streams for text operators. Then, if it’s text-based, it extracts position-aware text and converts it to clean Markdown locally. The whole point is the ~54% of PDFs that don’t need OCR get handled in under 200ms instead of being shipped off to a slow, paid service.

What sold me is the routing logic. It returns a per-page list of pages that genuinely need OCR, not a blanket yes/no. So a 40-page report where page 17 is a scan doesn’t get dumped wholesale into OCR — only page 17 does. That’s the kind of granularity most “PDF parsers” don’t bother with. And that’s the difference between a tool and a routing decision. It’s the exact thing your knowledge-graph memory layer depends on — garbage text in means garbage retrieval out.

My hands-on test

I installed it via pip in about 30 seconds (pip install pdf-inspector, prebuilt wheel, 2.6MB — no compiling, no Rust toolchain needed). Then I threw three real documents at it on my workstation.

Here’s what came back:

Document	Detected type	Confidence	Detect time	Full process	Markdown out
Hermes kanban spec (spec doc)	text_based	1.0	14.6ms	116.9ms	54,891 chars
COLM 2025 conference paper	text_based	1.0	2.0ms	15.0ms	10,687 chars
ICLR 2026 conference paper	text_based	1.0	3.0ms	25.2ms	14,686 chars
Image-only 2-page PDF (my test)	scanned	0.95	2.0ms	—	pages 0,1 → OCR

The conference papers came out clean — headings mapped to H1/H2, italics preserved, page numbers stripped, even the “Under review” lines handled. Then I built a fake scanned document (image XObjects, no text stream). It correctly flagged it scanned at 0.95 confidence and handed back [0, 1] as pages needing OCR. In 2 milliseconds. That’s the exact behavior the README promises, and it held up on my machine.

Where the benchmarks land

They benchmarked it on the opendataloader corpus (200 PDFs, no OCR, local engines only). pdf-inspector scored 0.875 overall — top of the field, and it finished all 200 docs in 0.470s. The table structure score (0.814) is the standout. PyMuPDF4LLM, the usual go-to, scored 0.735 overall and took 17 seconds. For a local Rust tool that’s a real gap, not a rounding error. Yet the honest caveat is these are their own published numbers, so treat them as direction, not gospel.

What to watch out for

It’s not magic. Classification reads the content stream, so a PDF that rasterized text into curves or images gets sent to OCR — that’s by design, but don’t expect it to rescue bad exports. The Markdown is built for structure, and while the table detection is good, complex multi-page financial tables still need a human glance. And the CLI (pdf2md) is solid, but for heavy production use you’ll want the Python or Node bindings, which are the actively maintained path.

Bottom line