Document reusing image-baked libraries with hf jobs uv run (#2517)

* Document reusing image-baked libraries with `hf jobs uv run`

Add a "Reusing image-baked libraries with uv run" section to the Popular Images
page. By default `hf jobs uv run` resolves a script's dependencies from PyPI and
does not use the libraries already in the image; for images that ship prebuilt,
CUDA-matched builds (vLLM, PyTorch, FlashInfer) you usually want to reuse them
via `--python` + `PYTHONPATH`.

The section covers:
- the two-flag recipe, with vLLM as the worked example
- a cpu-basic probe to discover the interpreter / site-packages paths per image
- the `RuntimeError: Could not find nvcc` symptom (FlashInfer JIT) as one example
- that this applies only to `uv run`, not generic `hf jobs run`, plus the upstream
  uv issue (astral-sh/uv#7999)

Also adds a short cross-link tip in the vLLM section.

* Move image-baked libraries section to bottom, add top warning

Address review: relocate the detailed "Reusing image-baked libraries
with uv run" section to the end of the page, and add a warning callout
near the top so the naive `hf jobs uv run` failure mode is flagged
before the framework list.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Reframe around using a framework image; demote Python-package reuse

GPU-tested the claims on HF Jobs (l4x1/a100, Qwen2.5-0.5B):
- Reproduced `Could not find nvcc` ONLY on the default uv image
  (no CUDA toolkit). With --image vllm/vllm-openai it works, because
  the image provides nvcc + the CUDA system stack.
- So the real fix is using the framework image, not the --python +
  PYTHONPATH recipe; "usually all you need" is just --image.
- Reuse via PYTHONPATH does not skip the PyPI install (same ~3.5GB
  downloaded), so it's not a speed-up — demoted to an optional
  sub-section for ABI/CUDA-mismatch cases.

Lead the section with the system-stack point and the nvcc repro;
keep the probe (paths/uv vary per image, verified across vLLM, sglang,
unsloth, trl, transformers, axolotl).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Apply suggestion from @davanstrien

* Apply suggestion from @davanstrien

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: davanstrien

docs/hub/jobs-popular-images.md

@@ -4,6 +4,11 @@ Here is the list of ready-to-use Docker images from popular frameworks that you
 
 These Docker images already have uv installed but if you want to use an image + uv for an image without uv installed you’ll need to make sure uv is installed first. This will work well in many cases but for LLM inference libraries which can have quite specific requirements, it can be useful to use a specific image that has the library installed.
 
+> [!TIP]
+> For GPU inference libraries, pass `--image` so the run gets a matching CUDA system stack
+> (toolkit, `nvcc`, libraries) — see [Using framework images for GPU
+> libraries](#using-framework-images-for-gpu-libraries) below.
+
 ## vLLM
 
 vLLM is a very well known and heavily used inference engine. It is known for its ability to scale inference for LLMs.
@@ -15,6 +20,11 @@ Use the `--image` argument to use this Docker image:
 >>> hf jobs uv run --image vllm/vllm-openai --flavor l4x4 generate-responses.py
 ```
 
+> [!TIP]
+> The `vllm/vllm-openai` image bundles the CUDA toolkit and prebuilt vLLM/FlashInfer kernels.
+> Using it is usually all you need; you can also reuse its prebuilt Python builds — see [Using
+> framework images for GPU libraries](#using-framework-images-for-gpu-libraries) below.
+
 You can find more information on vLLM batch inference on Jobs in [Daniel Van Strien's blog post](https://danielvanstrien.xyz/posts/2025/hf-jobs/vllm-batch-inference.html).
 
 ## TRL
@@ -26,3 +36,67 @@ Use the `--image` argument to use this Docker image:
 ```bash
 >>> hf jobs uv run --image huggingface/trl --flavor a100-large -s HF_TOKEN train.py
 ```
+
+## Using framework images for GPU libraries
+
+GPU libraries like vLLM need more than their Python package — they need a matching system
+environment: the CUDA toolkit (including `nvcc`), system libraries like NCCL and cuDNN, and so
+on. If you omit `--image`, `hf jobs uv run` uses the default uv image
+(`ghcr.io/astral-sh/uv:python3.12-bookworm`), a bare Python base with no CUDA toolkit. Your
+dependencies still install from PyPI, but at runtime a library that needs the toolkit can fail —
+for example FlashInfer's sampler JIT-compiles a kernel and aborts with:
+
+```text
+RuntimeError: Could not find nvcc and default cuda_home='/usr/local/cuda' doesn't exist
+```
+
+Passing the framework image fixes this — it provides the CUDA toolkit, `nvcc`, and matched
+libraries — and this is usually all you need:
+
+```bash
+hf jobs uv run --image vllm/vllm-openai --flavor l4x4 -s HF_TOKEN generate-responses.py
+```
+
+UV still reinstalls your script dependencies from PyPI, but they now run against the
+image's system stack, so the framework works.
+
+### Optionally: reuse the image's prebuilt Python builds
+
+These images also ship prebuilt, CUDA-matched builds of the framework itself. To import those
+instead of UV's fresh PyPI install — handy if a PyPI build and the image's CUDA stack disagree
+(an ABI mismatch, or a kernel that won't build) — point UV at the image's interpreter and add
+its site-packages to the import path:
+
+```bash
+hf jobs uv run \
+    --image vllm/vllm-openai \
+    --flavor l4x4 \
+    --python /usr/bin/python3 \
+    -e PYTHONPATH=/usr/local/lib/python3.12/dist-packages \
+    -s HF_TOKEN \
+    generate-responses.py
+```
+
+- `--python` uses the **image's** interpreter, keeping ABI compatibility with its compiled extensions.
+- `-e PYTHONPATH=...` makes `import vllm` resolve to the image's prebuilt build for that run.
+
+Paths differ per image, so probe them on `cpu-basic` rather than hardcoding:
+
+```bash
+hf jobs run --flavor cpu-basic vllm/vllm-openai bash -c 'which python3; which uv; python3 -m pip show vllm | grep Location'
+```
+
+```text
+/usr/bin/python3                              # pass to --python
+/usr/local/bin/uv                             # uv is present, so `uv run` works
+Location: /usr/local/lib/python3.12/dist-packages   # pass to PYTHONPATH
+```
+
+Swap `vllm` for whichever library you're reusing. Layouts vary — `vllm/vllm-openai` and
+`lmsysorg/sglang` use the system `dist-packages` above, while `unsloth/unsloth` uses a
+virtualenv (`/opt/venv/...`).
+
+> [!TIP]
+> This pins imports to the image's build; UV still installs your declared dependencies, so it
+> isn't a speed-up. A `uv run --system-site-packages` that would reuse the image's packages and
+> skip the reinstall is [requested upstream](https://github.com/astral-sh/uv/issues/7999).