Skip to content

Commit ee922cb

Browse files
Copilotmnriemgithub-code-quality[bot]
authored
feat(extensions): support multiple active catalogs simultaneously (#1720)
* Initial plan * feat(extensions): implement multi-catalog stack support - Add CatalogEntry dataclass to represent catalog entries - Add get_active_catalogs() reading SPECKIT_CATALOG_URL, project config, user config, or built-in default stack (org-approved + community) - Add _load_catalog_config() to parse .specify/extension-catalogs.yml - Add _validate_catalog_url() HTTPS validation helper - Add _fetch_single_catalog() with per-URL caching, backward-compat for DEFAULT_CATALOG_URL - Add _get_merged_extensions() that merges all catalogs (priority wins on conflict) - Update search() and get_extension_info() to use merged results annotated with _catalog_name and _install_allowed - Update clear_cache() to also remove per-URL hash cache files - Add extension_catalogs CLI command to list active catalogs - Add catalog add/remove sub-commands for .specify/extension-catalogs.yml - Update extension_add to enforce install_allowed=false policy - Update extension_search to show source catalog per result - Update extension_info to show source catalog with install_allowed status - Add 13 new tests covering catalog stack, merge conflict resolution, install_allowed enforcement, and catalog metadata Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> * docs: update RFC, user guide, and API reference for multi-catalog support - RFC: replace FUTURE FEATURE section with full implementation docs, add catalog stack resolution order, config file examples, merge conflict resolution, and install_allowed behavior - EXTENSION-USER-GUIDE.md: add multi-catalog section with CLI examples for catalogs/catalog-add/catalog-remove, update catalog config docs - EXTENSION-API-REFERENCE.md: add CatalogEntry class docs, update ExtensionCatalog docs with new methods and result annotations, add catalog CLI commands (catalogs, catalog add, catalog remove) Also fix extension_catalogs command to correctly show "Using built-in default catalog stack" when config file exists but has empty catalogs Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> * Potential fix for pull request finding 'Empty except' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * fix: remove extraneous f-string prefixes (ruff F541) Remove f-prefix from strings with no placeholders in catalog_remove and extension_search commands. Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> * fix: address PR review feedback for multi-catalog support - Rename 'org-approved' catalog to 'default' - Move 'catalogs' command to 'catalog list' for consistency - Add 'description' field to CatalogEntry dataclass - Add --description option to 'catalog add' CLI command - Align install_allowed default to False in _load_catalog_config - Add user-level config detection in catalog list footer - Fix _load_catalog_config docstring (document ValidationError) - Fix test isolation for test_search_by_tag, test_search_by_query, test_search_verified_only, test_get_extension_info - Update version to 0.1.14 and CHANGELOG - Update all docs (RFC, User Guide, API Reference) * fix: wrap _load_catalog_config() calls in catalog_list with try/except - Check SPECKIT_CATALOG_URL first (matching get_active_catalogs() resolution order) - Wrap both _load_catalog_config() calls in try/except ValidationError so a malformed config file cannot crash `specify extension catalog list` after the active catalogs have already been printed successfully Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
1 parent 1df24f1 commit ee922cb

File tree

7 files changed

+1280
-89
lines changed

7 files changed

+1280
-89
lines changed

CHANGELOG.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1212
### Added
1313

1414
- feat: add Tabnine CLI agent support
15+
- **Multi-Catalog Support (#1707)**: Extension catalog system now supports multiple active catalogs simultaneously via a catalog stack
16+
- New `specify extension catalog list` command lists all active catalogs with name, URL, priority, and `install_allowed` status
17+
- New `specify extension catalog add` and `specify extension catalog remove` commands for project-scoped catalog management
18+
- Default built-in stack includes `catalog.json` (default, installable) and `catalog.community.json` (community, discovery only) — community extensions are now surfaced in search results out of the box
19+
- `specify extension search` aggregates results across all active catalogs, annotating each result with source catalog
20+
- `specify extension add` enforces `install_allowed` policy — extensions from discovery-only catalogs cannot be installed directly
21+
- Project-level `.specify/extension-catalogs.yml` and user-level `~/.specify/extension-catalogs.yml` config files supported, with project-level taking precedence
22+
- `SPECKIT_CATALOG_URL` environment variable still works for backward compatibility (replaces full stack with single catalog)
23+
- All catalog URLs require HTTPS (HTTP allowed for localhost development)
24+
- New `CatalogEntry` dataclass in `extensions.py` for catalog stack representation
25+
- Per-URL hash-based caching for non-default catalogs; legacy cache preserved for default catalog
26+
- Higher-priority catalogs win on merge conflicts (same extension id in multiple catalogs)
27+
- 13 new tests covering catalog stack resolution, merge conflicts, URL validation, and `install_allowed` enforcement
28+
- Updated RFC, Extension User Guide, and Extension API Reference documentation
1529

1630
## [0.1.13] - 2026-03-03
1731

extensions/EXTENSION-API-REFERENCE.md

Lines changed: 110 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,34 @@ manager.check_compatibility(
243243
) # Raises: CompatibilityError if incompatible
244244
```
245245

246+
### CatalogEntry
247+
248+
**Module**: `specify_cli.extensions`
249+
250+
Represents a single catalog in the active catalog stack.
251+
252+
```python
253+
from specify_cli.extensions import CatalogEntry
254+
255+
entry = CatalogEntry(
256+
url="https://example.com/catalog.json",
257+
name="default",
258+
priority=1,
259+
install_allowed=True,
260+
description="Built-in catalog of installable extensions",
261+
)
262+
```
263+
264+
**Fields**:
265+
266+
| Field | Type | Description |
267+
|-------|------|-------------|
268+
| `url` | `str` | Catalog URL (must use HTTPS, or HTTP for localhost) |
269+
| `name` | `str` | Human-readable catalog name |
270+
| `priority` | `int` | Sort order (lower = higher priority, wins on conflicts) |
271+
| `install_allowed` | `bool` | Whether extensions from this catalog can be installed |
272+
| `description` | `str` | Optional human-readable description of the catalog (default: empty) |
273+
246274
### ExtensionCatalog
247275

248276
**Module**: `specify_cli.extensions`
@@ -253,30 +281,67 @@ from specify_cli.extensions import ExtensionCatalog
253281
catalog = ExtensionCatalog(project_root)
254282
```
255283

284+
**Class attributes**:
285+
286+
```python
287+
ExtensionCatalog.DEFAULT_CATALOG_URL # default catalog URL
288+
ExtensionCatalog.COMMUNITY_CATALOG_URL # community catalog URL
289+
```
290+
256291
**Methods**:
257292

258293
```python
259-
# Fetch catalog
294+
# Get the ordered list of active catalogs
295+
entries = catalog.get_active_catalogs() # List[CatalogEntry]
296+
297+
# Fetch catalog (primary catalog, backward compat)
260298
catalog_data = catalog.fetch_catalog(force_refresh: bool = False) # Dict
261299
262-
# Search extensions
300+
# Search extensions across all active catalogs
301+
# Each result includes _catalog_name and _install_allowed
263302
results = catalog.search(
264303
query: Optional[str] = None,
265304
tag: Optional[str] = None,
266305
author: Optional[str] = None,
267306
verified_only: bool = False
268-
) # Returns: List[Dict]
307+
) # Returns: List[Dict] — each dict includes _catalog_name, _install_allowed
269308
270-
# Get extension info
309+
# Get extension info (searches all active catalogs)
310+
# Returns None if not found; includes _catalog_name and _install_allowed
271311
ext_info = catalog.get_extension_info(extension_id: str) # Optional[Dict]
272312
273-
# Check cache validity
313+
# Check cache validity (primary catalog)
274314
is_valid = catalog.is_cache_valid() # bool
275315
276-
# Clear cache
316+
# Clear all catalog caches
277317
catalog.clear_cache()
278318
```
279319

320+
**Result annotation fields**:
321+
322+
Each extension dict returned by `search()` and `get_extension_info()` includes:
323+
324+
| Field | Type | Description |
325+
|-------|------|-------------|
326+
| `_catalog_name` | `str` | Name of the source catalog |
327+
| `_install_allowed` | `bool` | Whether installation is allowed from this catalog |
328+
329+
**Catalog config file** (`.specify/extension-catalogs.yml`):
330+
331+
```yaml
332+
catalogs:
333+
- name: "default"
334+
url: "https://raw.githubusercontent.com/github/spec-kit/main/extensions/catalog.json"
335+
priority: 1
336+
install_allowed: true
337+
description: "Built-in catalog of installable extensions"
338+
- name: "community"
339+
url: "https://raw.githubusercontent.com/github/spec-kit/main/extensions/catalog.community.json"
340+
priority: 2
341+
install_allowed: false
342+
description: "Community-contributed extensions (discovery only)"
343+
```
344+
280345
### HookExecutor
281346

282347
**Module**: `specify_cli.extensions`
@@ -543,6 +608,39 @@ EXECUTE_COMMAND: {command}
543608

544609
**Output**: List of installed extensions with metadata
545610

611+
### extension catalog list
612+
613+
**Usage**: `specify extension catalog list`
614+
615+
Lists all active catalogs in the current catalog stack, showing name, description, URL, priority, and `install_allowed` status.
616+
617+
### extension catalog add
618+
619+
**Usage**: `specify extension catalog add URL [OPTIONS]`
620+
621+
**Options**:
622+
623+
- `--name NAME` - Catalog name (required)
624+
- `--priority INT` - Priority (lower = higher priority, default: 10)
625+
- `--install-allowed / --no-install-allowed` - Allow installs from this catalog (default: false)
626+
- `--description TEXT` - Optional description of the catalog
627+
628+
**Arguments**:
629+
630+
- `URL` - Catalog URL (must use HTTPS)
631+
632+
Adds a catalog entry to `.specify/extension-catalogs.yml`.
633+
634+
### extension catalog remove
635+
636+
**Usage**: `specify extension catalog remove NAME`
637+
638+
**Arguments**:
639+
640+
- `NAME` - Catalog name to remove
641+
642+
Removes a catalog entry from `.specify/extension-catalogs.yml`.
643+
546644
### extension add
547645

548646
**Usage**: `specify extension add EXTENSION [OPTIONS]`
@@ -551,13 +649,13 @@ EXECUTE_COMMAND: {command}
551649

552650
- `--from URL` - Install from custom URL
553651
- `--dev PATH` - Install from local directory
554-
- `--version VERSION` - Install specific version
555-
- `--no-register` - Skip command registration
556652

557653
**Arguments**:
558654

559655
- `EXTENSION` - Extension name or URL
560656

657+
**Note**: Extensions from catalogs with `install_allowed: false` cannot be installed via this command.
658+
561659
### extension remove
562660

563661
**Usage**: `specify extension remove EXTENSION [OPTIONS]`
@@ -575,6 +673,8 @@ EXECUTE_COMMAND: {command}
575673

576674
**Usage**: `specify extension search [QUERY] [OPTIONS]`
577675

676+
Searches all active catalogs simultaneously. Results include source catalog name and install_allowed status.
677+
578678
**Options**:
579679

580680
- `--tag TAG` - Filter by tag
@@ -589,6 +689,8 @@ EXECUTE_COMMAND: {command}
589689

590690
**Usage**: `specify extension info EXTENSION`
591691

692+
Shows source catalog and install_allowed status.
693+
592694
**Arguments**:
593695

594696
- `EXTENSION` - Extension ID

extensions/EXTENSION-USER-GUIDE.md

Lines changed: 92 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -76,15 +76,15 @@ vim .specify/extensions/jira/jira-config.yml
7676

7777
## Finding Extensions
7878

79-
**Note**: By default, `specify extension search` uses your organization's catalog (`catalog.json`). If the catalog is empty, you won't see any results. See [Extension Catalogs](#extension-catalogs) to learn how to populate your catalog from the community reference catalog.
79+
`specify extension search` searches **all active catalogs** simultaneously, including the community catalog by default. Results are annotated with their source catalog and install status.
8080

8181
### Browse All Extensions
8282

8383
```bash
8484
specify extension search
8585
```
8686

87-
Shows all extensions in your organization's catalog.
87+
Shows all extensions across all active catalogs (default and community by default).
8888

8989
### Search by Keyword
9090

@@ -402,13 +402,13 @@ In addition to extension-specific environment variables (`SPECKIT_{EXT_ID}_*`),
402402

403403
| Variable | Description | Default |
404404
|----------|-------------|---------|
405-
| `SPECKIT_CATALOG_URL` | Override the extension catalog URL | GitHub-hosted catalog |
405+
| `SPECKIT_CATALOG_URL` | Override the full catalog stack with a single URL (backward compat) | Built-in default stack |
406406
| `GH_TOKEN` / `GITHUB_TOKEN` | GitHub API token for downloads | None |
407407

408408
#### Example: Using a custom catalog for testing
409409

410410
```bash
411-
# Point to a local or alternative catalog
411+
# Point to a local or alternative catalog (replaces the full stack)
412412
export SPECKIT_CATALOG_URL="http://localhost:8000/catalog.json"
413413
414414
# Or use a staging catalog
@@ -419,13 +419,76 @@ export SPECKIT_CATALOG_URL="https://example.com/staging/catalog.json"
419419

420420
## Extension Catalogs
421421

422-
For information about how Spec Kit's dual-catalog system works (`catalog.json` vs `catalog.community.json`), see the main [Extensions README](README.md#extension-catalogs).
422+
Spec Kit uses a **catalog stack** — an ordered list of catalogs searched simultaneously. By default, two catalogs are active:
423+
424+
| Priority | Catalog | Install Allowed | Purpose |
425+
|----------|---------|-----------------|---------|
426+
| 1 | `catalog.json` (default) | ✅ Yes | Curated extensions available for installation |
427+
| 2 | `catalog.community.json` (community) | ❌ No (discovery only) | Browse community extensions |
428+
429+
### Listing Active Catalogs
430+
431+
```bash
432+
specify extension catalog list
433+
```
434+
435+
### Adding a Catalog (Project-scoped)
436+
437+
```bash
438+
# Add an internal catalog that allows installs
439+
specify extension catalog add \
440+
--name "internal" \
441+
--priority 2 \
442+
--install-allowed \
443+
https://internal.company.com/spec-kit/catalog.json
444+
445+
# Add a discovery-only catalog
446+
specify extension catalog add \
447+
--name "partner" \
448+
--priority 5 \
449+
https://partner.example.com/spec-kit/catalog.json
450+
```
451+
452+
This creates or updates `.specify/extension-catalogs.yml`.
453+
454+
### Removing a Catalog
455+
456+
```bash
457+
specify extension catalog remove internal
458+
```
459+
460+
### Manual Config File
461+
462+
You can also edit `.specify/extension-catalogs.yml` directly:
463+
464+
```yaml
465+
catalogs:
466+
- name: "default"
467+
url: "https://raw.githubusercontent.com/github/spec-kit/main/extensions/catalog.json"
468+
priority: 1
469+
install_allowed: true
470+
description: "Built-in catalog of installable extensions"
471+
472+
- name: "internal"
473+
url: "https://internal.company.com/spec-kit/catalog.json"
474+
priority: 2
475+
install_allowed: true
476+
description: "Internal company extensions"
477+
478+
- name: "community"
479+
url: "https://raw.githubusercontent.com/github/spec-kit/main/extensions/catalog.community.json"
480+
priority: 3
481+
install_allowed: false
482+
description: "Community-contributed extensions (discovery only)"
483+
```
484+
485+
A user-level equivalent lives at `~/.specify/extension-catalogs.yml`. Project-level config takes full precedence when it contains one or more catalog entries. An empty `catalogs: []` list falls back to built-in defaults.
423486

424487
## Organization Catalog Customization
425488

426489
### Why Customize Your Catalog
427490

428-
Organizations customize their `catalog.json` to:
491+
Organizations customize their catalogs to:
429492

430493
- **Control available extensions** - Curate which extensions your team can install
431494
- **Host private extensions** - Internal tools that shouldn't be public
@@ -503,24 +566,40 @@ Options for hosting your catalog:
503566

504567
#### 3. Configure Your Environment
505568

506-
##### Option A: Environment variable (recommended for CI/CD)
569+
##### Option A: Catalog stack config file (recommended)
507570

508-
```bash
509-
# In ~/.bashrc, ~/.zshrc, or CI pipeline
510-
export SPECKIT_CATALOG_URL="https://your-org.com/spec-kit/catalog.json"
571+
Add to `.specify/extension-catalogs.yml` in your project:
572+
573+
```yaml
574+
catalogs:
575+
- name: "my-org"
576+
url: "https://your-org.com/spec-kit/catalog.json"
577+
priority: 1
578+
install_allowed: true
511579
```
512580

513-
##### Option B: Per-project configuration
581+
Or use the CLI:
582+
583+
```bash
584+
specify extension catalog add \
585+
--name "my-org" \
586+
--install-allowed \
587+
https://your-org.com/spec-kit/catalog.json
588+
```
514589

515-
Create `.env` or set in your shell before running spec-kit commands:
590+
##### Option B: Environment variable (recommended for CI/CD, single-catalog)
516591

517592
```bash
518-
SPECKIT_CATALOG_URL="https://your-org.com/spec-kit/catalog.json" specify extension search
593+
# In ~/.bashrc, ~/.zshrc, or CI pipeline
594+
export SPECKIT_CATALOG_URL="https://your-org.com/spec-kit/catalog.json"
519595
```
520596

521597
#### 4. Verify Configuration
522598

523599
```bash
600+
# List active catalogs
601+
specify extension catalog list
602+
524603
# Search should now show your catalog's extensions
525604
specify extension search
526605

0 commit comments

Comments
 (0)