Skip to content

Commit ba3f046

Browse files
davanstrienclaude
andauthored
Jobs: add "Persist your results" section (#2650)
* Jobs: add 'Persist your results' section Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Drop bucket example duplicated from the Volumes section Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 6cfbe03 commit ba3f046

2 files changed

Lines changed: 13 additions & 1 deletion

File tree

docs/hub/jobs-manage.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,18 @@ Because `hf jobs wait` returns a non-zero exit code when a Job fails, you can ch
185185
>>> hf jobs wait 693994e21a39f67af5a41ad0 && echo "job completed successfully"
186186
```
187187

188+
## Persist your results
189+
190+
A Job's filesystem is deleted when the Job ends. Write anything you want to keep somewhere durable before the Job exits:
191+
192+
- **Intermediate artifacts, checkpoints, and logs → a Storage Bucket volume.** Mount a bucket and write outputs under the mount path — see [Volumes](./jobs-configuration#volumes) for examples. Volume mounts are authorized with your Hugging Face identity when the Job is created, so your script doesn't need a token to write to them.
193+
194+
- **Final models and datasets → push to a Hub repo.** Jobs have no Hugging Face token unless you pass one, e.g. `--secrets HF_TOKEN` (the bare form resolves to your logged-in token automatically). If your script calls `push_to_hub()` or `create_repo()`, make sure the token has write access (fine-grained tokens need repo write and create permissions). A common failure mode is a Job that completes hours of compute and then errors on the final upload because the token can't write — the compute is done, but the results aren't saved.
195+
196+
- **Print key results to the logs.** Job logs are retained after the Job ends and can be fetched anytime with `hf jobs logs <job-id>`. Printing final metrics to stdout keeps them recoverable even if an upload step fails.
197+
198+
After a Job completes, check that your outputs actually landed (for example with `hf buckets list`) — a `COMPLETED` status means the command exited successfully, not that files were persisted where you expected.
199+
188200
## Debug a Job
189201

190202
If a Job has an error, you can see it in on the Job page

docs/hub/jobs-quickstart.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@ Save this script as `train.py`, and we can now run it with UV on Hugging Face Jo
100100

101101
## Run the training job
102102

103-
`hf jobs` takes several arguments: select the hardware with `--flavor`, choose a maximum duration with `--timeout`, and pass environment variable with `--env` and `--secrets`. Here we use the A100 Large GPU flavor with `--flavor a100-large` and pass your Hugging Face token as a secret with `--secrets HF_TOKEN` in order to be able to push the resulting model to your account.
103+
`hf jobs` takes several arguments: select the hardware with `--flavor`, choose a maximum duration with `--timeout`, and pass environment variable with `--env` and `--secrets`. Here we use the A100 Large GPU flavor with `--flavor a100-large` and pass your Hugging Face token as a secret with `--secrets HF_TOKEN` in order to be able to push the resulting model to your account. See [Persist your results](./jobs-manage#persist-your-results) for how to make sure your Job's outputs survive after it finishes.
104104

105105
Moreover, UV accepts the `--with` argument to define python dependencies, so we use `--with trl` to have the `trl` library available.
106106

0 commit comments

Comments
 (0)