Skip to content

Update models-downloading with downloading behind a proxy or firewall…#2628

Merged
Michellehbn merged 3 commits into
mainfrom
Update-models-downloading
Jul 8, 2026
Merged

Update models-downloading with downloading behind a proxy or firewall…#2628
Michellehbn merged 3 commits into
mainfrom
Update-models-downloading

Conversation

@Michellehbn

@Michellehbn Michellehbn commented Jul 7, 2026

Copy link
Copy Markdown
Member

Adds a "Downloading behind a proxy or firewall" section to the model downloading guide.

Downloads redirect from huggingface.co to separate storage, Xet, and CDN hosts, so allowlisting huggingface.co alone isn't enough; from_pretrained and hf download fail (typically a ReadTimeoutError mid-download) when those hosts are blocked. This list wasn't documented anywhere, so users in firewalled/proxied environments were reverse-engineering it from forum threads and third-party posts.

The new section lists the required hostnames (Hub API, Xet US/EU, CDN edges, and legacy LFS) with their purpose, plus wildcard guidance (*.hf.co / *.xethub.hf.co / *.huggingface.co) and a note on the legacy huggingface.co LFS hosts that *.hf.co doesn't cover.


Note

Low Risk
Documentation-only change to the Hub model downloading guide; no application code, auth, or infrastructure behavior is modified.

Overview
Adds a Downloading behind a proxy or firewall section to models-downloading.md after the hf-mount section.

It explains that downloads redirect from huggingface.co to separate Xet, storage, and CDN hosts, so allowlisting only the Hub domain is insufficient and from_pretrained / hf download can fail (often with ReadTimeoutError mid-download). The section documents an explicit HTTPS allowlist table (Hub API, Xet US/EU, CDN edges, legacy LFS hosts) and tips on suffix vs single-label wildcard rules, plus a warning that endpoint hostnames may change over time.

Reviewed by Cursor Bugbot for commit 6a9e062. Bugbot is set up for automated code reviews on this repo. Configure here.

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Pierrci Pierrci left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks for adding this

@rajatarya rajatarya left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @michelle — this is a genuinely useful addition. The exact set of hosts that downloads redirect to (Xet, CDN, legacy LFS) has never been documented, and firewalled/proxied users have been reverse-engineering it from forum threads, so closing that gap is great. A few things I especially liked: the Purpose column on the table, the ReadTimeoutError-as-a-symptom tip (that's exactly the signal people hit when a storage/CDN host is blocked mid-download), the explicit callout that legacy cdn-lfs*.huggingface.co isn't under *.hf.co, and the WARNING that the concrete list can drift. Style-wise it's consistent with the rest of docs/hub too — ## heading level matches the sibling sections, and > [!TIP]/> [!WARNING] are the prevailing callout syntax in this repo. No new links are introduced, so nothing to break there.

A few things to consider before merging:

  1. Wildcard coverage of the CDN edges is a bit optimistic. The reason *.xethub.hf.co is listed separately is that *.hf.co only matches a single label — but by that same logic *.hf.co also won't cover the CDN edge hosts (us.aws.cdn.hf.co, us.gcp.cdn.hf.co) since those sit under cdn.hf.co. Worth either clarifying that coverage depends on whether the proxy does single-label vs. suffix matching, or recommending *.cdn.hf.co alongside *.xethub.hf.co. (inline)
  2. The CDN edge rows look US-only while the Xet and LFS rows have explicit EU entries — worth confirming whether there are EU CDN edges to list. (inline)
  3. Routing through a proxy vs. allowlisting in a firewall. This section covers allowlisting hosts; consider a one-liner pointing users who route outbound traffic through a proxy at the standard HTTP_PROXY/HTTPS_PROXY environment variables (honored by the underlying HTTP clients), since that's the adjacent question this audience tends to have.
  4. Discoverability for datasets/Spaces. The intro mentions "models and datasets," and the same hosts apply, but this lives only in models-downloading.md. Consider surfacing it in datasets-downloading.md as well (or a shared page linked from both) so dataset users can find it. (inline)
  5. nit: small comma splice in the wildcard tip. (inline)

Nothing blocking — these are all clarity/completeness polish on a solid, needed doc.

Comment thread docs/hub/models-downloading.md
Comment thread docs/hub/models-downloading.md Outdated
Comment thread docs/hub/models-downloading.md Outdated
Comment thread docs/hub/models-downloading.md Outdated
Comment thread docs/hub/models-downloading.md Outdated

@rajatarya rajatarya left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My comments are minor. Thanks for adding this.

Addresses comments from @rajatarya 💪
Comment thread docs/hub/models-downloading.md Outdated
Co-authored-by: Hugo Larcher <hugo.larcher@huggingface.co>
@Michellehbn
Michellehbn merged commit 68cf2ff into main Jul 8, 2026
3 checks passed
@Michellehbn
Michellehbn deleted the Update-models-downloading branch July 8, 2026 09:03

@julien-c julien-c left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's either add to this one as well (they have some content) https://huggingface.co/docs/hub/main/en/datasets-downloading#faster-downloads

OR split this into a dedicated page

@julien-c

Copy link
Copy Markdown
Member

Also i'm not sure i would document the legacy ones (they're not used anymore no?)

@Michellehbn

Copy link
Copy Markdown
Member Author

Tadah! #2647

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants