davanstrien
diff --git a/‎ocr/CLAUDE.md‎
Lines changed: 93 additions & 0 deletions b/‎ocr/CLAUDE.md‎
Lines changed: 93 additions & 0 deletions
diff --git a/‎ocr/README.md‎
Lines changed: 4 additions & 2 deletions b/‎ocr/README.md‎
Lines changed: 4 additions & 2 deletions
@@ -196,6 +196,99 @@ hf jobs uv run --flavor l4x1 \
 
 ## Other OCR Scripts
 
+### Unlimited-OCR (`unlimited-ocr-vllm.py`)
+✅ **Production Ready — single-image** (added + validated 2026-06-28)
+
+Baidu's `baidu/Unlimited-OCR` (3.3B, MIT, DeepSeek-OCR / DeepSeek-OCR-2 descendant). Offline vLLM
+batch recipe adapted from `deepseek-ocr-vllm.py` — `llm.generate()` with PIL images +
+`NGramPerReqLogitsProcessor` (imported from `vllm.model_executor.models.unlimited_ocr`), prompt
+`<image>document parsing.`, `SamplingParams(temperature=0, skip_special_tokens=False,
+extra_args=dict(ngram_size=35, window_size=128))`, `limit_mm_per_prompt={"image": 1}`. One image per
+row → one markdown. `--strip-grounding` drops `<|det|>`/`<|ref|>` tags (verified locally on real
+output: removes boxes, keeps inner text + LaTeX).
+
+**⚠️ Dedicated image, not the standard one.** The arch is NOT in any stable vLLM pip wheel — must run
+on Baidu's `vllm/vllm-openai:unlimited-ocr` (CUDA 13.0; `:unlimited-ocr-cu129` on Hopper). So `vllm`
+and `torch` are **omitted from the PEP 723 deps** and come from the image via `PYTHONPATH`. The image
+uses the **standard** layout: `--python /usr/bin/python3 -e PYTHONPATH=/usr/local/lib/python3.12/dist-packages`
+(vLLM `0.23.1rc1.dev541` lives there; probed 2026-06-28). The `unlimited_ocr` module re-exports
+`deepseek_ocr.NGramPerReqLogitsProcessor`. Recipe: https://recipes.vllm.ai/baidu/Unlimited-OCR
+
+**Smoke tests (2026-06-28):**
+- **ufo-ColPali** (5, l4x1): 5/5 OK, 2.3 min, ~200 tok/s. Clean layout-grounded markdown — accurate
+  text, `<|det|>` bboxes (0–1000), multilingual (Spanish), LaTeX. Output `davanstrien/unlimited-ocr-smoke`.
+- **encyclopaedia-britannica-1771** (8, l4x1, `--strip-grounding`): 6/6 content pages produced clean
+  text matching the dataset's own `ocr_text` length almost exactly (e.g. row 1: md 5811 vs ocr_text
+  5752), period-accurate 1771 OCR (long-ſ, archaic spelling). The 2 "empty" rows are genuinely blank
+  pages (ground-truth `ocr_text` 3–24 chars). Output `davanstrien/unlimited-ocr-britannica-smoke`.
+
+**Multi-page: BOTH engines work on clean docs; robustness differs on hard scans. (Corrected
+2026-06-29 — earlier "vLLM multi-page is broken" was an input-difficulty artifact.)**
+- **Control test that overturned the first read:** ran the SAME clean synthetic 2-page doc through the
+  **vLLM server** that SGLang had aced. vLLM returned **`<PAGE>=2`, both pages, real text** (`Chapter
+  One The Harbor` + lines / `Chapter Two The Market` + lines), with minor body-OCR slips ("early oakh",
+  "Guile covered") — i.e. the model *misreading*, not the engine hallucinating. Worked with both 1×
+  and 2× `<image>` prompt forms + `vllm_xargs.window_size=1024`. So **vLLM multi-page works**.
+- **What the earlier garbling actually was:** my first vLLM multi-page tests used **hard** inputs —
+  `unlimited-ocr-pdf-test` (blank + dense 1771 Britannica) and ufo newspaper clippings. On those, vLLM
+  multi-page degraded to hallucination (counting garbage "SIGILLUM. 17. 96…", `2017年1月1日` loops,
+  content in neither input). SGLang read the *same* hard ufo input as real content → **SGLang is more
+  robust on hard/degraded scans**, but neither engine is "broken."
+- **Offline `LLM().generate()`** still needs one `<image>` per image (single placeholder → assertion);
+  offline multi-page was only tested on the hard Britannica PDF (garbled) — not re-tested on clean, so
+  the recipe stays single-image (multi-page belongs to serving).
+- `images_config`/`image_mode` are **SGLang-only** params (vLLM ignores them); on vLLM use one
+  `<image>` per page + `window_size=1024` in `vllm_xargs`.
+- **Upstream check (vllm-project/vllm#46564, "Support Unlimited OCR", merged 2026-06-28):** confirms
+  this. Multi-image IS implemented (crop/gundam auto-disabled → base mode; one `<image>` placeholder
+  per image). R-SWA needs the **FlexAttention** backend (auto on non-FA4 GPUs like L4) or FA4 on
+  H20/H100 — our run correctly used FlexAttention. BUT: the PR's only benchmark is **single-page
+  OmniDocBench** (FA4 92.12 / Flex 92.38); there is **no multi-page test, no `examples/`, no canonical
+  multi-page prompt** in the merged code. PR-author comment: multi-page needs **V1 + NGramPerReq-
+  LogitsProcessor** (V2 lacks custom logits processors), and their "14-page PDF merge" smoke test only
+  confirmed "**R-SWA itself works**" (mechanism runs on long seqs) — *not* OCR quality. So nobody
+  upstream has shown multi-page OCR quality; the tweet's "40+ pages, low edit distance" is ahead of the
+  merged evidence. (Our own clean-doc control test later showed vLLM multi-page DOES read correctly —
+  see the corrected block above; the earlier garbling was hard-input degradation, not an engine break.)
+- **Conclusion:** the **batch recipe stays single-image** (offline multi-page is finicky and untested
+  on clean; `--pdf-column` removed). For multi-page, **serve** the model — both engines read clean
+  multi-page docs; route hard/degraded scans to **SGLang** (more robust; authors' `images_config` path;
+  serving-unlimited-ocr.md Option B + §3). Image probed: `vllm 0.23.1rc1.dev541` (docs say "0.25.0+").
+- **SGLang multi-page — ✅ FIXED + validated working (2026-06-28).** Multi-page is the model's headline
+  feature and **SGLang delivers it robustly** (vLLM multi-page also works on clean docs but hallucinated
+  on hard scans — see corrected block above). Two pins were needed:
+  1. **Image `lmsysorg/sglang:v0.5.10.post1`** (not `:latest`). `:latest` drifted to sglang 0.5.14 /
+     torch 2.11 / cu130; the wheel (`dev11416`) needs torch 2.9.1 / cuda-python 12.9 / flashinfer 0.6.7 /
+     xgrammar 0.1.32 / transformers 5.3.0. Found v0.5.10.post1 by bisecting sglang release pyproject
+     pins — the **last** release before the torch-2.11 bump; matches the wheel exactly.
+  2. **`a100-large` + `--attention-backend flashinfer`** (not `h200`/`fa3`). `fa3` needs Hopper, but
+     HF's `h200` nodes **fail GPU init with `CUDA error 802: system not yet initialized`, 3/3** (infra /
+     Fabric-Manager — *all* working jobs this session were l4x1/a100, never h200). The version pin alone
+     did NOT fix 802; the 802 is purely the h200 node. a100+flashinfer dodges it.
+  - **Result:** server up; clean 2-page synthetic doc → **both pages read verbatim, `<PAGE>`-separated**
+    (`Chapter One: The Harbor…` / `Chapter Two: The Market…`); ufo pages → **real content**
+    (`OUT OF THIS WORLD / UFO FlyBys…`), *not* vLLM's hallucinated garbage. Client: OpenAI API,
+    `images_config:{image_mode:base}` + `Multi page parsing.`; no per-request NGram processor (so harder
+    scans show minor page-merge/OCR slips — fa3 + the custom logit processor would tighten quality; the
+    mechanism works). Working command lives in `serving-unlimited-ocr.md` Option B; switch back to
+    `fa3`/`h200` for exact R-SWA once the h200 802 infra issue clears.
+
+**Example usage:**
+```bash
+hf jobs uv run --flavor l4x1 -s HF_TOKEN \
+    --image vllm/vllm-openai:unlimited-ocr --python /usr/bin/python3 \
+    -e PYTHONPATH=/usr/local/lib/python3.12/dist-packages \
+    ./ocr/unlimited-ocr-vllm.py davanstrien/ufo-ColPali output-dataset --max-samples 10 --shuffle
+```
+
+**Deferred follow-up (captured, not built):** a *multi-page batch* recipe that drives the **SGLang
+server** in-job (server lifecycle + `ThreadPoolExecutor` over multi-page docs, like Baidu's `infer.py`,
+→ Hub) — the only way to get robust multi-page at corpus scale, since SGLang offline-Engine is
+non-viable (server-only, custom-logit-processor/R-SWA are server-side, `fa3` Hopper-only) and vLLM
+offline needs one `<image>` per page and degrades on hard scans. Gate: a real corpus-scale multi-page
+need **+** the h200/`fa3` infra fix (for exact R-SWA quality). Single-image vLLM (this recipe) stays
+the batch default.
+
 ### Nanonets OCR (`nanonets-ocr.py`, `nanonets-ocr2.py`)
 ✅ Both versions working
 
 
@@ -33,7 +33,7 @@ This will:
 
 ## Serve a model as a live endpoint
 
-The recipes here run as batch jobs. To call a model interactively, from an agent, or with concurrent ad-hoc requests, you can instead run it as a temporary endpoint: [HF Jobs serving](https://huggingface.co/docs/hub/jobs-serving) exposes a port on a GPU Job, giving an OpenAI-compatible endpoint that runs until the job is cancelled or its `--timeout` is reached. See [serving-unlimited-ocr.md](serving-unlimited-ocr.md) for a worked example serving Baidu's [Unlimited-OCR](https://huggingface.co/baidu/Unlimited-OCR) with SGLang.
+The recipes here run as batch jobs. To call a model interactively, from an agent, or with concurrent ad-hoc requests, you can instead run it as a temporary endpoint: [HF Jobs serving](https://huggingface.co/docs/hub/jobs-serving) exposes a port on a GPU Job, giving an OpenAI-compatible endpoint that runs until the job is cancelled or its `--timeout` is reached. See [serving-unlimited-ocr.md](serving-unlimited-ocr.md) for a worked example serving Baidu's [Unlimited-OCR](https://huggingface.co/baidu/Unlimited-OCR) — with vLLM (official image) or SGLang. To OCR a whole corpus of single-page images instead, the batch recipe `unlimited-ocr-vllm.py` is the better fit (it's single-image only). **Multi-page** documents need a server: both vLLM and SGLang read clean multi-page docs, but **SGLang is the more robust** — on hard/degraded scans vLLM multi-page hallucinated in our tests while SGLang held up.
 
 ## Models at a glance
 
@@ -69,6 +69,7 @@ _Sorted by model size:_
 | `deepseek-ocr-vllm.py` | [DeepSeek-OCR](https://huggingface.co/deepseek-ai/DeepSeek-OCR) | 4B | vLLM | 5 resolution + 5 prompt modes |
 | `deepseek-ocr.py` | [DeepSeek-OCR](https://huggingface.co/deepseek-ai/DeepSeek-OCR) | 4B | Transformers | Same model, Transformers backend |
 | `deepseek-ocr2-vllm.py` | [DeepSeek-OCR-2](https://huggingface.co/deepseek-ai/DeepSeek-OCR-2) | 3B | vLLM | Newer; needs nightly vLLM **+ the `vllm/vllm-openai` image** ([why](#if-a-vllm-script-crashes-at-startup-the-nvcc--nvrtc-error)) |
+| `unlimited-ocr-vllm.py` | [Unlimited-OCR](https://huggingface.co/baidu/Unlimited-OCR) | 3.3B | vLLM | DeepSeek-OCR-based; layout-grounded markdown (`--strip-grounding` for clean text). Single-image batch — needs Baidu's **dedicated `vllm/vllm-openai:unlimited-ocr` image** (`-cu129` on Hopper). Multi-page "long-horizon" parsing → serve it ([doc](serving-unlimited-ocr.md)); both engines do clean docs, **SGLang more robust** on hard scans. MIT |
 | `nuextract3.py` | [NuExtract3](https://huggingface.co/numind/NuExtract3) | 4B | vLLM | Markdown OCR **+ schema-guided JSON extraction** (template/Pydantic). Needs `vllm/vllm-openai` image |
 | `qianfan-ocr.py` | [Qianfan-OCR](https://huggingface.co/baidu/Qianfan-OCR) | 4.7B | vLLM | #1 OmniDocBench v1.5 (93.12), Layout-as-Thought, 192 languages |
 | `olmocr2-vllm.py` | [olmOCR-2-7B](https://huggingface.co/allenai/olmOCR-2-7B-1025-FP8) | 7B | vLLM | 82.4% olmOCR-Bench |
@@ -213,10 +214,11 @@ Beyond the shared flags, some models add their own. Run `--help` on any script f
 | `dots-ocr.py` | `--prompt-mode ocr\|layout-all\|layout-only` |
 | `dots-mocr.py` | `--prompt-mode` (8: ocr, layout-all, layout-only, web-parsing, scene-spotting, grounding-ocr, svg, general); SVG: `--model rednote-hilab/dots.mocr-svg --prompt-mode svg` |
 | `qianfan-ocr.py` | `--prompt-mode ocr\|table\|formula\|chart\|scene\|kie`, `--think` (Layout-as-Thought); `kie` needs `--custom-prompt` |
+| `unlimited-ocr-vllm.py` | `--strip-grounding` (drop `<\|det\|>`/`<\|ref\|>` grounding tags); needs the **`vllm/vllm-openai:unlimited-ocr`** image |
 | `numarkdown-ocr.py` | `--include-thinking` (store the reasoning trace) |
 | `nuextract3.py` | `--template` / `--schema` / `--enable-thinking` — see the NuExtract3 section above |
 
-**Image-mode models** — `abot-ocr.py` and `nuextract3.py` (Qwen3.5 architecture) need the `vllm/vllm-openai` image because the default uv-script image lacks `nvcc`. Add `--image vllm/vllm-openai:latest --python /usr/bin/python3 -e PYTHONPATH=/usr/local/lib/python3.12/dist-packages` (see the NuExtract3 example above for the full command).
+**Image-mode models** — `abot-ocr.py` and `nuextract3.py` (Qwen3.5 architecture) need the `vllm/vllm-openai` image because the default uv-script image lacks `nvcc`. Add `--image vllm/vllm-openai:latest --python /usr/bin/python3 -e PYTHONPATH=/usr/local/lib/python3.12/dist-packages` (see the NuExtract3 example above for the full command). `unlimited-ocr-vllm.py` is a special case — its architecture isn't in any stable vLLM wheel, so it needs Baidu's **dedicated** `vllm/vllm-openai:unlimited-ocr` image (tag `:unlimited-ocr-cu129` on Hopper), e.g. `--image vllm/vllm-openai:unlimited-ocr --python /usr/bin/python3 -e PYTHONPATH=/usr/local/lib/python3.12/dist-packages` (its docstring has the full command).
 
 ## Output & features