feat: resolve repo type and fetch accordingly by drbh · Pull Request #435 · huggingface/kernels

drbh · 2026-04-09T08:00:43Z

This PR adds support for fetching from kernel repo types and falls back to fetching from model repo types if the kernel repo does not exist

HuggingFaceDocBuilderDev · 2026-04-09T08:02:13Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Wauplin

Thanks for the ping! I've mostly checked the huggingface_hub integration

danieldk

Great PR! Added two comments.

My main worry is that some public functions (install_kernel, get_variants, may have missed some) get a public repo_type argument, which we'll have to remove again if we switch fully to kernel-repos only. Is there some way we can get rid of them? (e.g. by doing automatic detection inside the functions)

danieldk · 2026-04-13T17:07:51Z

    is_new_branch = False
    if branch is not None:
-        is_new_branch = not _branch_exists(api, repo_id, branch)
+        is_new_branch = not _branch_exists(api, repo_id, branch, repo_type="model")


Fixed to model because kernels upload is deprecated?

yea exactly, this is because the python upload only support uploading model repo prior to being deprecated, so now all of the call explicitly use repo_type="model" instead of updating the deprecated path for kernel types. happy to change if it makes more sense to upload to kernel repos, but intentionally avoided adding the new functionality for now.

Yeah no point updating deprecated functionality.

sayakpaul

Thanks for the PR!

Main comment is using _resolve_repo_type() for fetching the repo type inline instead of calling it as a part of other functions such as select_revision_or_version. This feels more idiomatic to me.

Also, I think we should add the deprecation now. If we do so, we could also add a test to make sure deprecation warnings are raised accordingly.

sayakpaul · 2026-04-14T03:34:19Z

    is_new_branch = False
    if branch is not None:
-        is_new_branch = not _branch_exists(api, repo_id, branch)
+        is_new_branch = not _branch_exists(api, repo_id, branch, repo_type="model")


Yeah no point updating deprecated functionality.

sayakpaul · 2026-04-14T03:37:04Z

+    repo_type = _resolve_repo_type(repo_id)
    versions = {}
-    for tag in _get_hf_api().list_repo_refs(repo_id).tags:
+    for tag in _get_hf_api().list_repo_refs(repo_id, repo_type=repo_type).tags:


(nit): Similar to the above function, do we want?

tags = _get_hf_api().list_repo_refs(repo_id, repo_type=repo_type).tags

sayakpaul · 2026-04-14T03:39:00Z

    """
    if isinstance(version_spec, int):
-        versions = _get_available_versions(repo_id)
+        versions, repo_type = _get_available_versions(repo_id)


_get_available_versions() returning versions as well as repo_type looks confusing to me. Isn't it better to just call _resolve_repo_type(repo_id) to determine the repo_type?

sayakpaul · 2026-04-14T03:47:07Z

+    # TODO: add back the deprecation warning when we remove support for the "model" repo type.
+    # we should also add a removal date in the warning message for a final migration date.
+    # warnings.warn(
+    #     f"Repository '{repo_id}' uses deprecated repo type 'model'; "
+    #     "use 'kernel' instead.",
+    #     DeprecationWarning,
+    #     stacklevel=2,
+    # )


Should we not add it right now? If so, we could add a deprecation date (maybe two release cycles) and instructions to do it the proper way (re-uploading the repo as a kernel type)?

IIUC, after the model repos are removed, the above LOC will definitely fail. Deprecation shouldn't mean complete removal; it should indicate that removal is coming and change is needed.

Or am I misunderstanding something?

Wauplin

Made a new pass on the PR and dropping "model" indeed made the implementation simpler ^^

I've left some comments, mostly to know if you need new things to be updated on the huggingface_hub side or not.

(also a nit, the PR title and description are outdated now)

Wauplin · 2026-04-15T07:23:41Z

            repo_id=repo_id,
            operations=list(chunk),
            revision=revision,
-            repo_type="model",


no repo_type="kernel" here? (same above)

no repo_type="kernel" in this case since we plan to fully remove this upload file and all of the upload logic on the python cli in a following PR.

I removed repo_type="model" but realize that was unneeded - however in this case I don't think is technically changes any of the logic since if I understand correctly the default functionality is to push to model repos.

however as mentioned this will all be removed in the next PR I open since we are cleaning up deprecated paths for the v0.14.0 release

I removed repo_type="model" but realize that was unneeded - however in this case I don't think is technically changes any of the logic since if I understand correctly the default functionality is to push to model repos.

yes indeed, "model" is the default repo type

no repo_type="kernel" in this case since we plan to fully remove this upload file and all of the upload logic on the python cli in a following PR.

Got it!

Wauplin · 2026-04-15T07:26:44Z

    # Ensure that we are testing with a non-existing kernel, so that we know
    # that the kernel must be local.
-    with pytest.raises(RepositoryNotFoundError):
+    with pytest.raises(HfHubHTTPError):


RepositoryNotFoundError was more specific. Did it change? Or it is that RepositoryNotFoundError is not raised for kernel repos (and in such a case we should fix it on huggingface_hub side)

oh thanks good catch, RepositoryNotFoundError is a better error but I think throw kernel repos throw a 401 instead of a 404 when a repo cannot be found (example in this error). We may also need a update on the server side to improve the error.

not sure if we should block this PR with that update, maybe it can be a small followup PR? what do you think?

I think throw kernel repos throw a 401 instead of a 404 when a repo cannot be found

That depends whether user is authenticated or not. It's not specific to kernels. You can test that by opening

https://huggingface.co/api/models/huggingface/doesnt_exist

https://huggingface.co/api/kernels/huggingface/doesnt_exist

in your browser. In normal mode you get a 404 and in incognito (i.e. logged out) you get a 401. This is because repo might exist and be private (so unauthorized). The theory behind it is debatable, but low prio as we could argue for both 401 and 404 .

So here I would advice to either make authenticated calls or catch HfHubHTTPError, it's fine

ah okay that makes sense thanks for the info! will keep HfHubHTTPError for now

Wauplin · 2026-04-15T07:29:40Z

        assert version() == "1"


+@pytest.mark.skip(reason="Tags are not supported on kernel repos")


"Tags are not supported on kernel repos"

Is this something that you need resolved on huggingface_hub side? If yes I can work on a PR, if not maybe just remove the test/feature if it doesn't exist anymore

I'm not sure we need any changes on the huggingface_hub side, this test is skipped because it tests the backward compatible path where we used tags instead of branches.

this test and the _old locking path is also going to be removed in a follow up PR as this path is deprecated an we are removing support in v0.14.0

Wauplin · 2026-04-15T07:30:15Z

        assert linear.n_calls == 2


+@pytest.mark.skip(reason="Tags are not supported on kernel repos")


same questions here for tags and RepositoryNotFoundError

tried to use RepositoryNotFoundError but ran into the 401 issue noted above. I've reverted back to HfHubHTTPError so CI passes but also this test will be removed in a follow up PR

thanks for pointing these things out!

danieldk · 2026-04-15T13:48:20Z

 def install_kernel_all_variants(
    repo_id: str,
    revision: str,
+    *,


same as above

danieldk · 2026-04-15T13:51:51Z

    package_name, variant_path = install_kernel(
-        repo_id, revision=revision, backend=backend, user_agent=user_agent
+        repo_id,
+        revision,


Not using a kwarg for revision seems like a regression. Also see the other comments about kwargs.

thanks for pointing out, updated as mentioned above

drbh had a problem deploying to testpypi April 9, 2026 12:28 — with GitHub Actions Failure

drbh had a problem deploying to testpypi April 9, 2026 12:38 — with GitHub Actions Failure

drbh temporarily deployed to testpypi April 9, 2026 13:17 — with GitHub Actions Inactive

drbh mentioned this pull request Apr 9, 2026

Replace huggingface-hub git dependency from kernel-builder #404

Closed

drbh temporarily deployed to testpypi April 9, 2026 13:22 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 9, 2026 13:37 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 9, 2026 13:48 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 9, 2026 13:54 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 9, 2026 14:02 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 9, 2026 14:35 — with GitHub Actions Inactive

Wauplin reviewed Apr 9, 2026

View reviewed changes

Comment thread kernels/src/kernels/utils.py Outdated

Comment thread kernels/src/kernels/lockfile.py Outdated

drbh temporarily deployed to testpypi April 9, 2026 14:53 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 9, 2026 15:10 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 13, 2026 13:47 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 13, 2026 13:49 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 13, 2026 15:39 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 13, 2026 16:11 — with GitHub Actions Inactive

danieldk reviewed Apr 13, 2026

View reviewed changes

drbh temporarily deployed to testpypi April 13, 2026 17:34 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 13, 2026 17:55 — with GitHub Actions Inactive

sayakpaul reviewed Apr 14, 2026

View reviewed changes

sayakpaul mentioned this pull request Apr 14, 2026

Support the new kernel repo type #441

Closed

drbh temporarily deployed to testpypi April 14, 2026 14:49 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 14, 2026 16:29 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 14, 2026 17:04 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 14, 2026 18:20 — with GitHub Actions Inactive

drbh temporarily deployed to testpypi April 14, 2026 18:48 — with GitHub Actions Inactive

drbh added 2 commits April 14, 2026 16:06

feat: resolve repo type and fetch accordingly

191a5be

fix: lint and format and prefer pinning hfh commit

030145f

drbh added 10 commits April 14, 2026 16:06

feat: prefer locking kernel repo hashes

3e7dc01

fix: avoid deprecation warning for now

4a9ea8a

feat: make repo type optional and comment for siblings

6f71720

fix: remove repo_type from all public interfaces

2ab8262

fix: remove backward compat of model repo types

b26b7f6

fix: improve error cases

d1722af

fix: bump lock files for kernel repos

9c48e38

fix: avoid non version locking tests

db72e5d

fix: avoid upload changes

1a15029

fix: apply lints after rebase

e8095a9

drbh force-pushed the support-kernel-repo-fetching branch from 0135bc0 to e8095a9 Compare April 14, 2026 20:16

drbh temporarily deployed to testpypi April 14, 2026 20:17 — with GitHub Actions Inactive

Wauplin reviewed Apr 15, 2026

View reviewed changes

fix: adjust repo not found error regression

c7ef910

drbh temporarily deployed to testpypi April 15, 2026 12:38 — with GitHub Actions Inactive

fix: adjust repo not found error regression in layer test too

ef80f07

drbh temporarily deployed to testpypi April 15, 2026 12:40 — with GitHub Actions Inactive

fix: revert back to hfhub error until 401 resolved

ef46c95

drbh temporarily deployed to testpypi April 15, 2026 13:37 — with GitHub Actions Inactive

fix: avoid the unneeded removal of repo type

e553a8b

drbh temporarily deployed to testpypi April 15, 2026 13:50 — with GitHub Actions Inactive

danieldk reviewed Apr 15, 2026

View reviewed changes

feat: enforce kwargs

5f5926f

drbh temporarily deployed to testpypi April 15, 2026 14:09 — with GitHub Actions Inactive

fix: adjust for kwarg

9448ba3

drbh temporarily deployed to testpypi April 15, 2026 14:23 — with GitHub Actions Inactive

fix: update test for kwarg changes

95a0316

drbh temporarily deployed to testpypi April 15, 2026 14:34 — with GitHub Actions Inactive

danieldk approved these changes Apr 15, 2026

View reviewed changes

drbh merged commit 84629d2 into main Apr 15, 2026
39 checks passed

		assert version() == "1"


		@pytest.mark.skip(reason="Tags are not supported on kernel repos")

		assert linear.n_calls == 2


		@pytest.mark.skip(reason="Tags are not supported on kernel repos")

Conversation

drbh commented Apr 9, 2026

Uh oh!

HuggingFaceDocBuilderDev commented Apr 9, 2026

Uh oh!

Wauplin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

danieldk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Wauplin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Wauplin Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Wauplin Apr 15, 2026 •

edited

Loading