Document reusing image-baked libraries with hf jobs uv run

docs/hub/jobs-popular-images.md

@@ -4,6 +4,11 @@ Here is the list of ready-to-use Docker images from popular frameworks that you
 
 These Docker images already have uv installed but if you want to use an image + uv for an image without uv installed you’ll need to make sure uv is installed first. This will work well in many cases but for LLM inference libraries which can have quite specific requirements, it can be useful to use a specific image that has the library installed.
 
+> [!TIP]
+> For GPU inference libraries, pass `--image` so the run gets a matching CUDA system stack
+> (toolkit, `nvcc`, libraries) — see [Using framework images for GPU
+> libraries](#using-framework-images-for-gpu-libraries) below.
+
 ## vLLM
 
 vLLM is a very well known and heavily used inference engine. It is known for its ability to scale inference for LLMs.
@@ -15,6 +20,11 @@ Use the `--image` argument to use this Docker image:
 >>> hf jobs uv run --image vllm/vllm-openai --flavor l4x4 generate-responses.py
 ```
 
+> [!TIP]
+> The `vllm/vllm-openai` image bundles the CUDA toolkit and prebuilt vLLM/FlashInfer kernels.
+> Using it is usually all you need; you can also reuse its prebuilt Python builds — see [Using
+> framework images for GPU libraries](#using-framework-images-for-gpu-libraries) below.
+
 You can find more information on vLLM batch inference on Jobs in [Daniel Van Strien's blog post](https://danielvanstrien.xyz/posts/2025/hf-jobs/vllm-batch-inference.html).
 
 ## TRL
@@ -26,3 +36,67 @@ Use the `--image` argument to use this Docker image:
 ```bash
 >>> hf jobs uv run --image huggingface/trl --flavor a100-large -s HF_TOKEN train.py
 ```
+
+## Using framework images for GPU libraries
+
+GPU libraries like vLLM need more than their Python package — they need a matching system
+environment: the CUDA toolkit (including `nvcc`), system libraries like NCCL and cuDNN, and so
+on. If you omit `--image`, `hf jobs uv run` uses the default uv image
+(`ghcr.io/astral-sh/uv:python3.12-bookworm`), a bare Python base with no CUDA toolkit. Your
+dependencies still install from PyPI, but at runtime a library that needs the toolkit can fail —
+for example FlashInfer's sampler JIT-compiles a kernel and aborts with:
+
+```text
+RuntimeError: Could not find nvcc and default cuda_home='/usr/local/cuda' doesn't exist
+```
+
+Passing the framework image fixes this — it provides the CUDA toolkit, `nvcc`, and matched
+libraries — and this is usually all you need:
+
+```bash
+hf jobs uv run --image vllm/vllm-openai --flavor l4x4 -s HF_TOKEN generate-responses.py
+```
+
+UV still reinstalls your script dependencies from PyPI, but they now run against the
+image's system stack, so the framework works.
+
+### Optionally: reuse the image's prebuilt Python builds
+
+These images also ship prebuilt, CUDA-matched builds of the framework itself. To import those
+instead of UV's fresh PyPI install — handy if a PyPI build and the image's CUDA stack disagree
+(an ABI mismatch, or a kernel that won't build) — point UV at the image's interpreter and add
+its site-packages to the import path:
+
+```bash
+hf jobs uv run \
+    --image vllm/vllm-openai \
+    --flavor l4x4 \
+    --python /usr/bin/python3 \
+    -e PYTHONPATH=/usr/local/lib/python3.12/dist-packages \
+    -s HF_TOKEN \
+    generate-responses.py
+```
+
+- `--python` uses the **image's** interpreter, keeping ABI compatibility with its compiled extensions.
+- `-e PYTHONPATH=...` makes `import vllm` resolve to the image's prebuilt build for that run.
+
+Paths differ per image, so probe them on `cpu-basic` rather than hardcoding:
+
+```bash
+hf jobs run --flavor cpu-basic vllm/vllm-openai bash -c 'which python3; which uv; python3 -m pip show vllm | grep Location'
+```
+
+```text
+/usr/bin/python3                              # pass to --python
+/usr/local/bin/uv                             # uv is present, so `uv run` works
+Location: /usr/local/lib/python3.12/dist-packages   # pass to PYTHONPATH
+```
+
+Swap `vllm` for whichever library you're reusing. Layouts vary — `vllm/vllm-openai` and
+`lmsysorg/sglang` use the system `dist-packages` above, while `unsloth/unsloth` uses a
+virtualenv (`/opt/venv/...`).
+
+> [!TIP]
+> This pins imports to the image's build; UV still installs your declared dependencies, so it
+> isn't a speed-up. A `uv run --system-site-packages` that would reuse the image's packages and
+> skip the reinstall is [requested upstream](https://github.com/astral-sh/uv/issues/7999).