PDFree › Blog › PDF to Markdown Benchmark

PDF to Markdown for AI & RAG
— a Real Benchmark, Not Marketing Copy

7-minute read · Published August 2026 · Updated Aug 25 — re-ran after real bug fixes (hyphen-repair, Markdown-escaping), average rose from 91.6 to 92.6

Feeding a PDF to ChatGPT or Claude, or chunking it for a RAG pipeline, works far better as clean Markdown than as raw extracted text — headings stay headings, tables stay tables, reading order stays intact. We tested pdfree's PDF-to-Markdown tool against three widely-used converters — pymupdf4llm, Docling, and Marker — on 7 real documents, scored by the same independent evaluator all three were already benchmarked with. Here are the actual numbers, including the ones we don't win.

Methodology — apples to apples, not vibes

We didn't invent our own scoring to flatter the result. We used an existing PDF-to-Markdown evaluation harness that pymupdf4llm, Docling, and Marker had already been scored against on the same 7-document corpus — real papers and reports covering a two-column academic layout, a formula-heavy paper, a multi-page financial ledger, a legal brief with footnotes, book front matter, a columned scientific paper, and a mixed-layout journal page. Each output is scored 0–100 (we call it ARI) across five channels: heading structure, reading order, text flow, table fidelity, and formula handling.

Seven documents is a real but modest corpus — enough to catch genuine structural failure modes (a table that gets flattened to prose, a heading that gets missed), not enough to claim statistical significance the way a thousand-document benchmark could. We're publishing the exact numbers rather than a summary claim so you can judge that for yourself.

The headline number

pdfree: 92.6 / 100 average — within 0.1 points of Docling (92.7), essentially a statistical tie, clearly ahead of pymupdf4llm (80.2) on every single document, behind Marker (95.1), which uses OCR-based LaTeX formula reconstruction.

This is up from 91.6 at first publication — not a re-scoring, a real gain from two bug fixes shipped since: broken words at the end of a line ("informa-" / "tion") are now correctly rejoined instead of left as two garbage tokens, and literal */_/backtick characters pulled from the source PDF (e.g. a footnote marker) are now escaped instead of being emitted raw into the Markdown, where they could silently corrupt nearby formatting. Both were found by running our own documents through a structural quality checker, not by chasing this benchmark's number directly.

Where we actually win

On the financial ledger, pdfree scores a clean 100 — beating all three other converters, including Marker. The document has two mutually-exclusive columns (debit/credit), which trips up naive table detectors; ours handles it correctly. On the mixed-layout journal page, pdfree beats both Docling and Marker. On the legal brief, we tie Docling exactly.

Where we lose — and why, honestly

Docling and Marker both pull ahead on the formula-heavy paper and the columned scientific paper. By default, pdfree's formula handling is still honest flattening — a formula is preserved as readable text rather than silently dropped, but no LaTeX reconstruction is attempted. The numbers above measure that default path, the same way the original benchmark did — apples to apples with the first run, not a moving target.

That said, this is no longer a hard architectural wall: pdfree now ships an opt-in, beta Formula OCR (Texo/FormulaNet, running entirely in-browser via WASM — no server, no upload, same privacy guarantee as everything else here). It's off by default because it's honest about its own limits — around 71% accuracy on our own real-formula testing, and every recognized formula carries a visible "AI-recognized, not verified" disclosure rather than silently presenting itself as ground truth. We didn't re-run this benchmark with it enabled — measuring a toggle a reader has to turn on themselves would muddy an apples-to-apples comparison — but if formula fidelity matters more to you than the default's guaranteed-correct flattening, it's there to try.

On the two-column paper, Docling's column detection is currently more robust in one specific edge case (a page that mixes a full-width title block with two columns below it). We know exactly why — it's a documented, open item, not a mystery — and it's a smaller gap than it was a few weeks ago, not a static number.

Why this matters for AI/RAG use specifically

Searching for "best PDF to Markdown for RAG" today surfaces almost entirely developer-facing tools — Marker, Docling, HURIDOCS, nanonets — things you self-host or call via an API. There's real value in that segment, but it locks out anyone who just wants to paste a clean, well-structured document into ChatGPT or Claude right now, from a browser, without installing Python or standing up a GPU. That's the audience pdfree's PDF-to-Markdown tool is actually built for: open the tool, drop a PDF, get Markdown with headings, tables, and (flattened, readable) formulas intact — nothing leaves your device.

Use it outside the browser

The browser tool is the whole product for most people, but the exact same extraction engine benchmarked above also runs as a plain Node library/CLI, and as a self-hosted Docker REST API — for anyone who wants this in a script, a cron job, or an actual RAG pipeline instead of a browser tab. Same guarantee both ways: nothing is uploaded to pdfree.io or anywhere else — the code just runs on your own machine or your own server instead of a browser tab.

Not published to a package registry yet, but the source is real, tested, and buildable today:

git clone https://github.com/mahmudovbahrom555-lab/pdfree33
cd pdfree33/packages/pdf2md-core && npm install --omit=optional
node bin/pdf2md.js report.pdf --out report.md          # CLI

# or the self-hosted REST API (from the repo root):
docker build -f packages/pdf2md-server/Dockerfile -t pdf2md-server .
docker run -p 8080:8080 pdf2md-server
curl --data-binary @report.pdf -H "Content-Type: application/pdf" \
  http://localhost:8080/convert -o report.md

Full docs, environment variables, and the REST API's security notes (it ships with no built-in auth by design, matching how self-hosted tools like this are normally deployed — put it behind your own reverse proxy if it's reachable from anywhere untrusted) are in each package's own README: pdf2md-core and pdf2md-server.

Reproduce this yourself

We're not asking you to take our word for it. pdfree's PDF-to-Markdown tool is free and runs in your browser — take any of your own real documents (a two-column paper, a table-heavy report, a legal document with footnotes) and compare the output against whatever you're using today. The whole point of a client-side tool is that there's nothing to sign up for before you can check.