Summary
Cold-testing the README quickstart before going public, the flagship command fails on the default Jobs image:
hf jobs uv run --flavor l4x1 --secrets HF_TOKEN \
https://huggingface.co/datasets/uv-scripts/ocr/raw/main/glm-ocr.py \
davanstrien/ufo-ColPali your-username/ufo-ocr
vLLM's engine core crashes at warmup:
RuntimeError: Could not find nvcc and default cuda_home='/usr/local/cuda' doesn't exist
(Stack: _dummy_sampler_run → topk_topp_sampler → FlashInfer.) Same class of issue as the ABot/NuExtract image-mode problem in huggingface/hub-docs#2517.
Root cause
glm-ocr.py pins vLLM nightly (wheels.vllm.ai/nightly/cu129). The current nightly (v0.22.1rc1.dev188, 2026-06-02) defaults to the FlashInfer top-k/top-p sampler, which JIT-compiles at engine init and needs nvcc — absent from the default ghcr.io/astral-sh/uv:python3.12-bookworm image. So the command that worked earlier in the year silently broke when the nightly advanced. This is the concrete cost of pinning nightly for the flagship recipe (reproducibility).
Fixes
(a) Band-aid — validated end-to-end. Set VLLM_USE_FLASHINFER_SAMPLER=0 (greedy OCR doesn't use the sampler; negligible cost; harmless once nightly stops defaulting to FlashInfer). With it, the engine inits, OCR runs, and the dataset pushes:
Done! GLM-OCR processing complete.
Dataset available at: https://huggingface.co/datasets/davanstrien/ufo-ocr-coldtest3
Implement by baking os.environ.setdefault("VLLM_USE_FLASHINFER_SAMPLER", "0") into glm-ocr.py before the LLM(...) is constructed, so the plain README command works without users passing the flag.
(b) Robust — preferred long-term. Move glm-ocr.py off vLLM nightly onto a stable vLLM that includes GLM-OCR (the script header notes it was "added in v0.16.0"). Fixes the reproducibility problem at the source. Needs a version check + a re-test.
Broader: smoke-test the headline recipes before launch
This isn't only glm-ocr — deepseek-ocr2-vllm.py also requires nightly, and other recipes may have drifted. A quick run of the top ~5 recipes (e.g. against davanstrien/ufo-ColPali --max-samples 2) before flipping the repo public would catch the rest. Ties into the "OCR smoke-test dataset" idea in ocr/CLAUDE.md.
Related
Launch blocker: the flagship quickstart must run clean before the repo goes public.
Summary
Cold-testing the README quickstart before going public, the flagship command fails on the default Jobs image:
hf jobs uv run --flavor l4x1 --secrets HF_TOKEN \ https://huggingface.co/datasets/uv-scripts/ocr/raw/main/glm-ocr.py \ davanstrien/ufo-ColPali your-username/ufo-ocrvLLM's engine core crashes at warmup:
(Stack:
_dummy_sampler_run→topk_topp_sampler→ FlashInfer.) Same class of issue as the ABot/NuExtract image-mode problem in huggingface/hub-docs#2517.Root cause
glm-ocr.pypins vLLM nightly (wheels.vllm.ai/nightly/cu129). The current nightly (v0.22.1rc1.dev188, 2026-06-02) defaults to the FlashInfer top-k/top-p sampler, which JIT-compiles at engine init and needsnvcc— absent from the defaultghcr.io/astral-sh/uv:python3.12-bookwormimage. So the command that worked earlier in the year silently broke when the nightly advanced. This is the concrete cost of pinning nightly for the flagship recipe (reproducibility).Fixes
(a) Band-aid — validated end-to-end. Set
VLLM_USE_FLASHINFER_SAMPLER=0(greedy OCR doesn't use the sampler; negligible cost; harmless once nightly stops defaulting to FlashInfer). With it, the engine inits, OCR runs, and the dataset pushes:Implement by baking
os.environ.setdefault("VLLM_USE_FLASHINFER_SAMPLER", "0")intoglm-ocr.pybefore theLLM(...)is constructed, so the plain README command works without users passing the flag.(b) Robust — preferred long-term. Move
glm-ocr.pyoff vLLM nightly onto a stable vLLM that includes GLM-OCR (the script header notes it was "added in v0.16.0"). Fixes the reproducibility problem at the source. Needs a version check + a re-test.Broader: smoke-test the headline recipes before launch
This isn't only glm-ocr —
deepseek-ocr2-vllm.pyalso requires nightly, and other recipes may have drifted. A quick run of the top ~5 recipes (e.g. againstdavanstrien/ufo-ColPali --max-samples 2) before flipping the repo public would catch the rest. Ties into the "OCR smoke-test dataset" idea inocr/CLAUDE.md.Related
--secrets HF_TOKEN(401 on push) and over-promised local runs for vLLM scripts — both fixed in Reframe 'run anywhere' → 'portable'; fix the local-run overclaim #8.Launch blocker: the flagship quickstart must run clean before the repo goes public.