feat(extensions): authenticate GitHub-hosted catalog and download requests with GITHUB_TOKEN/GH_TOKEN by anasseth · Pull Request #2087 · github/spec-kit

anasseth · 2026-04-04T04:11:29Z

Description

Fixes #2037. Closes the authentication gap introduced when multi-catalog support landed in #1707.

Before this change, all network requests in ExtensionCatalog used bare urllib.request.urlopen(url) with no headers. Any catalog or extension ZIP hosted in a private GitHub repository would silently fail with HTTP 404, regardless of whether GITHUB_TOKEN or GH_TOKEN was set in the environment.

This PR adds a _make_request(url) helper on ExtensionCatalog that attaches an Authorization: token <value> header when:

GITHUB_TOKEN or GH_TOKEN is present in the environment, and
the target URL is a GitHub-hosted domain (raw.githubusercontent.com,
github.com, or api.github.com)

Non-GitHub URLs are always fetched without credentials to prevent token leakage to third-party hosts.

The three affected call sites are:

_fetch_single_catalog — fetches catalog JSON from a configured catalog URL
fetch_catalog — legacy single-catalog path used when SPECKIT_CATALOG_URL is set
download_extension — downloads extension ZIP from a release asset URL

No behavior change for users without a token set — the code path is identical to before.

Documentation in EXTENSION-USER-GUIDE.md has been updated: the existing GH_TOKEN/GITHUB_TOKEN table entry (which described the token as "for downloads" only) now accurately reflects that it covers catalog fetches as well, and a private-catalog usage example has been added.

Testing

Ran uv run specify --help — CLI loads correctly, all commands present
Ran full test suite: 1131 passed, 5 skipped, 2 failed
- Both failures are pre-existing on main before this change:
- TestManifestPathTraversal::test_record_file_rejects_absolute_path
- TestCommandRegistrar::test_codex_skill_registration_uses_fallback_script_variant_without_init_options
8 new tests added to TestExtensionCatalog in tests/test_extensions.py:
- 6 unit tests for _make_request: no-token path, GITHUB_TOKEN, GH_TOKEN fallback, precedence when both are set, non-GitHub URL never gets header (security), api.github.com domain
- 2 integration tests that mock urlopen and assert the captured Request object carries the auth header — one for _fetch_single_catalog, one for download_extension

AI Disclosure

I did use AI assistance (describe below)
This PR was implemented with AI assistance via Claude Code.

…uests with GITHUB_TOKEN/GH_TOKEN

Copilot

Pull request overview

Adds GitHub-token authentication to extension catalog fetching and extension ZIP downloads so catalogs/assets hosted in private GitHub repos work when GITHUB_TOKEN/GH_TOKEN is set, while aiming to avoid leaking credentials to non-GitHub hosts.

Changes:

Introduces ExtensionCatalog._make_request(url) to attach an Authorization header for GitHub-hosted URLs when a token is available.
Updates all urllib.request.urlopen(...) call sites in ExtensionCatalog to use the new request builder.
Adds unit/integration tests for the request/header behavior and updates user docs to reflect token usage for both catalogs and downloads.

Show a summary per file

File	Description
`src/specify_cli/extensions.py`	Adds `_make_request` and routes catalog/download `urlopen` calls through it to support authenticated GitHub fetches.
`tests/test_extensions.py`	Adds tests validating auth header behavior and that `urlopen` receives a `Request` containing the header.
`extensions/EXTENSION-USER-GUIDE.md`	Updates env var documentation and adds an example for private GitHub-hosted catalogs.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 3/3 changed files
Comments generated: 2

Copilot · 2026-04-06T13:02:16Z

src/specify_cli/extensions.py

+
+        headers: Dict[str, str] = {}
+        token = os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN")
+        if token and any(
+            host in url
+            for host in ("raw.githubusercontent.com", "github.com", "api.github.com")
+        ):


The GitHub-hosted URL check uses substring matching (host in url), which can incorrectly attach the token to non-GitHub hosts (e.g., https://github.com.evil.com/... or https://internal.example.com/path/github.com/...) and violates the stated goal of preventing credential leakage. Parse the URL and compare urlparse(url).hostname (lowercased) against an allowlist of exact hostnames instead of scanning the full URL string.

Suggested change

headers: Dict[str, str] = {}

token = os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN")

if token and any(

host in url

for host in ("raw.githubusercontent.com", "github.com", "api.github.com")

):

from urllib.parse import urlparse

headers: Dict[str, str] = {}

token = os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN")

hostname = (urlparse(url).hostname or "").lower()

github_hosts = {"raw.githubusercontent.com", "github.com", "api.github.com"}

if token and hostname in github_hosts:

Copilot · 2026-04-06T13:02:17Z

tests/test_extensions.py

+        catalog = self._make_catalog(temp_dir)
+        req = catalog._make_request("https://internal.example.com/catalog.json")
+        assert "Authorization" not in req.headers
+


Current tests cover a generic non-GitHub domain, but they don't cover common spoofing cases that would slip through the current substring-based domain check (e.g., https://github.com.evil.com/... or a non-GitHub host whose path/query contains github.com). Add negative tests for these URL shapes to ensure the auth header is never attached outside the intended allowlist.

Suggested change

def test_make_request_token_not_added_for_github_lookalike_host(self, temp_dir, monkeypatch):

"""Auth header is not attached to non-GitHub hosts that only contain github.com in the hostname."""

monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")

catalog = self._make_catalog(temp_dir)

req = catalog._make_request("https://github.com.evil.com/org/repo/releases/download/v1/ext.zip")

assert "Authorization" not in req.headers

def test_make_request_token_not_added_for_non_github_host_with_github_in_path(self, temp_dir, monkeypatch):

"""Auth header is not attached when a non-GitHub host includes github.com only in the URL path."""

monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")

catalog = self._make_catalog(temp_dir)

req = catalog._make_request("https://evil.example.com/github.com/org/repo/releases/download/v1/ext.zip")

assert "Authorization" not in req.headers

def test_make_request_token_not_added_for_non_github_host_with_github_in_query(self, temp_dir, monkeypatch):

"""Auth header is not attached when a non-GitHub host includes github.com only in the query string."""

monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")

catalog = self._make_catalog(temp_dir)

req = catalog._make_request("https://evil.example.com/download?source=https://github.com/org/repo/releases/download/v1/ext.zip")

assert "Authorization" not in req.headers

feat(extensions): authenticate GitHub-hosted catalog and download req…

147a0af

…uests with GITHUB_TOKEN/GH_TOKEN

anasseth requested a review from mnriem as a code owner April 4, 2026 04:11

mnriem requested a review from Copilot April 6, 2026 12:59

Copilot started reviewing on behalf of mnriem April 6, 2026 12:59 View session

Copilot AI reviewed Apr 6, 2026

View reviewed changes

samdamazio approved these changes Apr 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(extensions): authenticate GitHub-hosted catalog and download requests with GITHUB_TOKEN/GH_TOKEN#2087

feat(extensions): authenticate GitHub-hosted catalog and download requests with GITHUB_TOKEN/GH_TOKEN#2087
anasseth wants to merge 1 commit intogithub:mainfrom
anasseth:feat/github-token-auth-private-catalogs

anasseth commented Apr 4, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 6, 2026

Uh oh!

Copilot AI Apr 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

+    def test_make_request_token_not_added_for_github_lookalike_host(self, temp_dir, monkeypatch):
+        """Auth header is not attached to non-GitHub hosts that only contain github.com in the hostname."""
+        monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
+        catalog = self._make_catalog(temp_dir)
+        req = catalog._make_request("https://github.com.evil.com/org/repo/releases/download/v1/ext.zip")
+        assert "Authorization" not in req.headers
+    def test_make_request_token_not_added_for_non_github_host_with_github_in_path(self, temp_dir, monkeypatch):
+        """Auth header is not attached when a non-GitHub host includes github.com only in the URL path."""
+        monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
+        catalog = self._make_catalog(temp_dir)
+        req = catalog._make_request("https://evil.example.com/github.com/org/repo/releases/download/v1/ext.zip")
+        assert "Authorization" not in req.headers
+    def test_make_request_token_not_added_for_non_github_host_with_github_in_query(self, temp_dir, monkeypatch):
+        """Auth header is not attached when a non-GitHub host includes github.com only in the query string."""
+        monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
+        catalog = self._make_catalog(temp_dir)
+        req = catalog._make_request("https://evil.example.com/download?source=https://github.com/org/repo/releases/download/v1/ext.zip")
+        assert "Authorization" not in req.headers

Conversation

anasseth commented Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

AI Disclosure

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

anasseth commented Apr 4, 2026 •

edited

Loading