Document reusing image-baked libraries with hf jobs uv run#2517
Merged
Conversation
Add a "Reusing image-baked libraries with uv run" section to the Popular Images page. By default `hf jobs uv run` resolves a script's dependencies from PyPI and does not use the libraries already in the image; for images that ship prebuilt, CUDA-matched builds (vLLM, PyTorch, FlashInfer) you usually want to reuse them via `--python` + `PYTHONPATH`. The section covers: - the two-flag recipe, with vLLM as the worked example - a cpu-basic probe to discover the interpreter / site-packages paths per image - the `RuntimeError: Could not find nvcc` symptom (FlashInfer JIT) as one example - that this applies only to `uv run`, not generic `hf jobs run`, plus the upstream uv issue (astral-sh/uv#7999) Also adds a short cross-link tip in the vLLM section.
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
lhoestq
approved these changes
Jun 3, 2026
lhoestq
left a comment
lhoestq
left a comment
Member
There was a problem hiding this comment.
lgtm ! I hope they can ship uv run --system-siye-packages soon
Address review: relocate the detailed "Reusing image-baked libraries with uv run" section to the end of the page, and add a warning callout near the top so the naive `hf jobs uv run` failure mode is flagged before the framework list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GPU-tested the claims on HF Jobs (l4x1/a100, Qwen2.5-0.5B): - Reproduced `Could not find nvcc` ONLY on the default uv image (no CUDA toolkit). With --image vllm/vllm-openai it works, because the image provides nvcc + the CUDA system stack. - So the real fix is using the framework image, not the --python + PYTHONPATH recipe; "usually all you need" is just --image. - Reuse via PYTHONPATH does not skip the PyPI install (same ~3.5GB downloaded), so it's not a speed-up — demoted to an optional sub-section for ABI/CUDA-mismatch cases. Lead the section with the system-stack point and the nvcc repro; keep the probe (paths/uv vary per image, verified across vLLM, sglang, unsloth, trl, transformers, axolotl). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
davanstrien
commented
Jun 4, 2026
davanstrien
commented
Jun 4, 2026
Member
Author
|
Will merge for now. can iterate more on this / hopefully simplify approach if uv add supports for using system packages in scripts |
pcuenca
reviewed
Jun 4, 2026
Comment on lines
+42
to
+47
| GPU libraries like vLLM need more than their Python package — they need a matching system | ||
| environment: the CUDA toolkit (including `nvcc`), system libraries like NCCL and cuDNN, and so | ||
| on. If you omit `--image`, `hf jobs uv run` uses the default uv image | ||
| (`ghcr.io/astral-sh/uv:python3.12-bookworm`), a bare Python base with no CUDA toolkit. Your | ||
| dependencies still install from PyPI, but at runtime a library that needs the toolkit can fail — | ||
| for example FlashInfer's sampler JIT-compiles a kernel and aborts with: |
Member
There was a problem hiding this comment.
But passing --image is what we already do in the existing version of the page, isn't it?
Member
There was a problem hiding this comment.
arg too late lol; no worries though.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Popular Images page shows how to point Jobs at the vLLM/TRL images, but not how
to actually reuse the libraries baked into them.
hf jobs uv runresolves a script'sdeps from PyPI by default, so a script importing
vllmgets a fresh PyPI install ratherthan the image's prebuilt, CUDA-matched build — which can fail at startup with
RuntimeError: Could not find nvcc(FlashInfer trying to JIT-compile a kernel).This adds a short "Reusing image-baked libraries with uv run" section covering:
--python+PYTHONPATHrecipe, with vLLM as the worked examplecpu-basicprobe to find the interpreter / site-packages paths for any imageuv run, not generichf jobs runNote
Low Risk
Documentation-only change to the Hub Jobs popular images page; no runtime or product code affected.
Overview
Expands Popular Images with guidance for GPU Jobs: new tips on using
--imagefor a CUDA-matched stack, and a section Using framework images for GPU libraries that explains why the defaultuvimage can hitnvcc/ FlashInfer failures and how framework images fix that.Adds an optional workflow to reuse image-baked Python builds via
--pythonandPYTHONPATH, acpu-basicprobe command to discover paths per image, and notes on layout differences (e.g. vLLM vs Unsloth) plus upstreamuvlimitations.Reviewed by Cursor Bugbot for commit 1191e76. Bugbot is set up for automated code reviews on this repo. Configure here.