Skip to content

Add workflow step catalog — community-installable step types#2394

Draft
Copilot wants to merge 3 commits intomainfrom
copilot/add-community-installable-steps
Draft

Add workflow step catalog — community-installable step types#2394
Copilot wants to merge 3 commits intomainfrom
copilot/add-community-installable-steps

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 28, 2026

The workflow engine shipped with a dynamic STEP_REGISTRY but no distribution mechanism for community-authored step types. This adds a full catalog system for discovering, installing, and managing custom step types, following the same patterns as the workflow/extension catalogs.

New classes (workflows/catalog.py)

  • StepRegistry — persists installed custom steps in .specify/workflows/steps/step-registry.json
  • StepCatalog — multi-source catalog stack with SHA256-based caching; resolves SPECKIT_STEP_CATALOG_URL env var → .specify/step-catalogs.yml~/.specify/step-catalogs.yml → built-in defaults (step-catalog.json + step-catalog.community.json)
  • StepCatalogError / StepValidationError / StepCatalogEntry supporting types

Dynamic step loading (workflows/__init__.py)

load_custom_steps(project_root) scans .specify/workflows/steps/, dynamically imports each package's __init__.py, finds the StepBase subclass matching the declared type_key, and registers it into STEP_REGISTRY. Broken packages are silently skipped.

CLI surface (specify workflow step …)

specify workflow step list              # built-in + installed custom types
specify workflow step add <id>          # fetch step.yml + __init__.py, validate, install
specify workflow step remove <id>       # delete package dir + registry entry
specify workflow step search [query]    # search across all configured catalogs
specify workflow step info <id>         # detail view (built-in or custom)
specify workflow step catalog list      # show active catalog sources
specify workflow step catalog add <url> # add catalog to .specify/step-catalogs.yml
specify workflow step catalog remove <n># remove by index

add validates that the downloaded step.yml's type_key matches the catalog ID and that all fetches use HTTPS before writing anything to disk.

Catalog files

  • workflows/step-catalog.json — official catalog (empty, ready for entries)
  • workflows/step-catalog.community.json — community catalog (empty, ready for entries)

Tests

27 new tests across TestStepRegistryCustom, TestStepCatalog, and TestLoadCustomSteps covering CRUD, catalog resolution (env var / project / user / default), URL validation, search, and dynamic loading edge cases (missing files, broken imports, already-registered keys).

Copilot AI requested review from Copilot and removed request for Copilot April 28, 2026 17:48
Copilot AI linked an issue Apr 28, 2026 that may be closed by this pull request
Copilot AI requested review from Copilot and removed request for Copilot April 28, 2026 17:59
Comment thread tests/test_workflows.py Fixed
Copilot AI changed the title [WIP] Add step catalog and CLI for community-installable step types Add workflow step catalog — community-installable step types Apr 28, 2026
Copilot AI requested a review from mnriem April 28, 2026 18:01
…e-effect'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 28, 2026 19:18
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a catalog/registry system for community-installable workflow step types, plus CLI commands to discover, install, and manage them alongside built-in steps.

Changes:

  • Introduces StepRegistry and StepCatalog (multi-source resolution + SHA256 cache) for step type distribution/management.
  • Adds dynamic filesystem-based loading of installed custom step packages into STEP_REGISTRY.
  • Expands CLI with specify workflow step … and adds tests and initial (empty) catalog JSON files.
Show a summary per file
File Description
workflows/step-catalog.json Adds the built-in “official” step catalog scaffold (currently empty).
workflows/step-catalog.community.json Adds the built-in “community” step catalog scaffold (currently empty).
src/specify_cli/workflows/catalog.py Implements StepRegistry + StepCatalog with config resolution and caching.
src/specify_cli/workflows/init.py Adds load_custom_steps(project_root) dynamic import/registration for installed step packages.
src/specify_cli/init.py Adds Typer subcommands for listing/searching/installing/removing steps and managing step catalogs.
tests/test_workflows.py Adds unit tests covering registry CRUD, catalog resolution/validation, search/info, and dynamic loading behavior.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

src/specify_cli/init.py:5555

  • workflow_step_remove() builds step_dir from unvalidated step_id and then shutil.rmtree(step_dir). A malicious value like ../... could delete arbitrary directories outside .specify/workflows/steps. Add the same resolved-path relative_to() guard used by workflow_remove/workflow_add before performing deletions.
    step_dir = project_root / ".specify" / "workflows" / "steps" / step_id
    if step_dir.exists():
        import shutil
        shutil.rmtree(step_dir)

  • Files reviewed: 6/6 changed files
  • Comments generated: 4

Comment on lines +5419 to +5428
if not info:
console.print(f"[red]Error:[/red] Step type '{step_id}' not found in catalog")
raise typer.Exit(1)

if not info.get("_install_allowed", True):
console.print(
f"[yellow]Warning:[/yellow] Step type '{step_id}' is from a discovery-only catalog"
)
console.print("Direct installation is not enabled for this catalog source.")
raise typer.Exit(1)
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This command allows installing a step whose step_id collides with an already-registered built-in step type. In that case the install will succeed, but load_custom_steps() will later skip loading it because the key is already in STEP_REGISTRY, leaving the user with an unusable “installed” step. Add an explicit check that step_id is not already present in STEP_REGISTRY (and likely not already installed in StepRegistry) and fail fast with a clear error.

Copilot uses AI. Check for mistakes.
Comment on lines +5364 to +5369
for key in sorted(STEP_REGISTRY.keys()):
if key in custom_keys:
custom.append(key)
else:
built_in.append(key)

Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

workflow_step_list() only iterates over STEP_REGISTRY.keys(), so custom steps that are recorded as installed in StepRegistry but fail to import/validate (and therefore aren’t registered) won’t be shown at all. Since the help text says it lists installed custom types, consider also listing registry entries that are not currently registered (e.g., under a “Custom (installed, failed to load)” section) so users can discover and remove/repair them.

Copilot uses AI. Check for mistakes.
Comment on lines +823 to +825
fetched_at = meta.get("fetched_at", 0)
return (time.time() - fetched_at) < self.CACHE_DURATION
except (json.JSONDecodeError, OSError):
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_is_url_cache_valid() assumes meta['fetched_at'] is numeric; if the meta file is corrupted and fetched_at is a string/null, (time.time() - fetched_at) will raise TypeError and break catalog operations. Coerce fetched_at to a float (or return False when it’s not an int/float) and include TypeError in the handled exceptions. (The same pattern exists in WorkflowCatalog._is_url_cache_valid, so consider updating both for consistency.)

Suggested change
fetched_at = meta.get("fetched_at", 0)
return (time.time() - fetched_at) < self.CACHE_DURATION
except (json.JSONDecodeError, OSError):
fetched_at = float(meta.get("fetched_at", 0))
return (time.time() - fetched_at) < self.CACHE_DURATION
except (json.JSONDecodeError, OSError, TypeError, ValueError):

Copilot uses AI. Check for mistakes.
raise ValueError(f"Redirect to non-HTTPS URL: {final_url}")
return resp.read()

step_dir = project_root / ".specify" / "workflows" / "steps" / step_id
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

step_id is used directly to build step_dir under .specify/workflows/steps/. Without a safety check, values like ../x or absolute paths can escape the intended directory and lead to arbitrary file writes/deletes (especially since failure cleanup rmtree() follows). Validate that the resolved step_dir stays within the intended base directory (similar to the workflow_add path-traversal guard) before creating the directory.

This issue also appears on line 5551 of the same file.

Suggested change
step_dir = project_root / ".specify" / "workflows" / "steps" / step_id
steps_base_dir = (project_root / ".specify" / "workflows" / "steps").resolve()
step_dir = (steps_base_dir / step_id).resolve()
try:
step_dir.relative_to(steps_base_dir)
except ValueError:
console.print(f"[red]Error:[/red] Invalid step id '{step_id}'")
raise typer.Exit(1)

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Workflow Step Catalog — community-installable step types

3 participants