Add SkyPilot to storage bucket integrations#2644
Conversation
Add a SkyPilot section to storage-buckets-integrations.md (store: hf file mounts for buckets and repos) and a SkyPilot row to the access-method table in storage-buckets-access.md.
julien-c
left a comment
julien-c
left a comment
There was a problem hiding this comment.
cool! sorry about the review delay. let's wait for @davanstrien or @lhoestq to take a super quick look too
| |--------|----------|---------| | ||
| | **hf-mount** | Mount as local filesystem — any tool works | [See below](#mount-as-a-local-filesystem) | | ||
| | **Volume mounts** | HF Jobs & Spaces (same idea, managed for you) | [See below](#volume-mounts-in-jobs-and-spaces) | | ||
| | **SkyPilot** | Cloud training/serving jobs across 20+ clouds | [Bucket Integrations](./storage-buckets-integrations#skypilot) | |
There was a problem hiding this comment.
more natural to move this at the end of the list imo
|
|
||
| See [Inspect's eval logs guide](https://inspect.aisi.org.uk/eval-logs.html#sec-hugging-face-storage-buckets) for details. | ||
|
|
||
| ## SkyPilot |
There was a problem hiding this comment.
Might make sense to move this to its own page, but cc @davanstrien wdyt
There was a problem hiding this comment.
Let's merge as-is for now but since the third-party list is going to grow I'll do a proper reorg of these pages soon
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
lhoestq
left a comment
lhoestq
left a comment
There was a problem hiding this comment.
yes ok to merge there fore now and reorg later
next step is skypilot for hf jobs ? haha
5 participants
Adds SkyPilot to the Storage Buckets docs:
storage-buckets-integrations.md- new## SkyPilotsection showingstore: hffile mounts (read-only model/dataset repos + a read-writebucket for checkpoints), the auth flow, and a note on hf-mount's base-image
requirements.
storage-buckets-access.md- a SkyPilot row in the access-method table,linking to the new section.
Context: https://huggingface.co/blog/skypilot-hf-storage
Note
Low Risk
Documentation-only changes with no runtime, API, or security behavior impact.
Overview
Documents using SkyPilot with Hugging Face storage buckets and repos via the integrations guide.
storage-buckets-integrations.mdadds a SkyPilot section:store: hfonfile_mountsfor read-only model/dataset repos and read-write buckets (example training YAML), auth withhf auth login/HF_TOKENandskypilot[huggingface], optional--secret HF_TOKENfor gated repos, a tip on hf-mount glibc/FUSE image requirements on Kubernetes, and a link to the SkyPilot + HF blog post.storage-buckets-access.mdadds a short pointer under the access-method table to the integrations page (SkyPilot alongside Inspect).Reviewed by Cursor Bugbot for commit bf096e7. Bugbot is set up for automated code reviews on this repo. Configure here.