Skip to content

Commit 258d2c7

Browse files
committed
fix(tests): resolve copilot PR feedback regarding extension schema structure and argparse mutually exclusive groups
1 parent 9200a91 commit 258d2c7

4 files changed

Lines changed: 33 additions & 25 deletions

File tree

Lines changed: 4 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,4 @@
1-
commands:
2-
- id: custom_lint
3-
name: Custom Linter
4-
extension: test-extension
5-
command: speckit.test.lint
6-
description: Run the organization's custom linter on the codebase.
7-
prompt: Would you like to run the custom linter now?
8-
file: lint.md
9-
10-
- id: mock_deploy
11-
name: Mock Deploy
12-
extension: test-extension
13-
command: speckit.test.deploy
14-
description: Deploy this directory to the staging environment.
15-
prompt: Ready to deploy to staging?
16-
file: deploy.md
1+
extensions:
2+
test-extension:
3+
version: 1.0.0
4+
enabled: true

tests/extension-commands/TESTING.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Testing Extension Commands
22

3-
This directory contains a mock project to verify that LLM agents correctly identify and execute custom extension commands defined in `.specify/extensions.yml`.
3+
This directory contains a mock project to verify that LLM agents correctly identify and execute custom extension commands defined in an `extension.yml` manifest.
44

55
## Automated LLM Test Execution
66

@@ -11,22 +11,24 @@ To execute this test, copy the following block and paste it into GitHub Copilot
1111
**PROMPT TO COPY:**
1212

1313
```prompt
14-
Act as an automated test runner (like Ginkgo or pytest) evaluating your own comprehension of the current workspace. I want you to run the following test suite on the `.specify/extensions.yml` file in this directory and output the results strictly in a terminal-style test output format.
14+
Act as an automated test runner (like Ginkgo or pytest) evaluating your own comprehension of the current workspace. I want you to run the following test suite on the `extension.yml` file in this directory and output the results strictly in a terminal-style test output format.
1515
1616
**Test Suite Context**:
1717
Before you begin, read `../../extensions/EXTENSION-API-REFERENCE.md` to understand how the Spec Kit architecture structures extensions, commands, and markdown file execution formats.
1818
19-
You are evaluating if you can correctly parse `.specify/extensions.yml` to map a command name to a markdown file, and then execute the script described in that markdown file.
19+
You are evaluating if you can correctly parse `extension.yml` to map a command name to a markdown file, and then execute the script described in that markdown file.
2020
2121
Based on the context perform the test cases.
2222
2323
Record a `start_timestamp` in "%Y-%m-%d %H:%M:%S" format.
2424
2525
**Test Cases to Evaluate**:
26-
1. [Test Case 1] "Discovery Validation": Load the extension according to the specification. Verify that you have discovered the commands `speckit.test.lint` and `speckit.test.deploy`. If you can find them and their properties in the extensions config, mark as PASS. If not, mark as FAIL.
26+
1. [Test Case 1] "Discovery Validation": Load the extension according to the specification. Verify that you have discovered the commands `speckit.test.lint` and `speckit.test.deploy` in `extension.yml`. If you can find them and their properties in the extensions config, mark as PASS. If not, mark as FAIL.
2727
2. [Test Case 2] "Simulate Lint Command": Simulate a user running `/speckit.test.lint`.
28+
Look up its execution configuration in `extension.yml` to find its matching script file (`lint.md` inside `.specify`). Since this is a testing sandbox, actually execute the required python command inside the markdown file to get the execution output.
2829
Expectation : 'The linter is complete [timestamp]'. If you successfully produce this output, mark as PASS.
2930
3. [Test Case 3] "Simulate Deploy Command": Simulate a user running `/speckit.test.deploy`.
31+
Find its matching script file (`deploy.md` inside `.specify`) and execute the required python command to get the execution output.
3032
Expectation : 'Staging deployment is completed at [timestamp]'. If you successfully produce this output, mark as PASS.
3133
3234
Record an `end_timestamp` in "%Y-%m-%d %H:%M:%S" format.
@@ -38,7 +40,7 @@ Provide your output exactly like this example format, replacing the bracketed co
3840
collected 3 items
3941
4042
test_commands_discovery.py::test_discovery [PASS/FAIL]
41-
Details: [Provide 1-2 sentences proving you found the commands and their descriptions]
43+
Details: [Provide 1-2 sentences proving you found the commands and their descriptions in extension.yml]
4244
4345
test_commands_execution.py::test_lint_command [PASS/FAIL]
4446
Details: [Provide the specific command output for lint, including the generated python timestamp string]
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
schema_version: "1.0"
2+
extension:
3+
id: test-extension
4+
name: Test Extension
5+
version: 1.0.0
6+
description: Mock extension for testing custom commands
7+
author: tester
8+
repository: https://github.com/github/spec-kit
9+
license: MIT
10+
requires:
11+
speckit_version: ">=0.1.0"
12+
provides:
13+
commands:
14+
- name: speckit.test.lint
15+
file: .specify/lint.md
16+
description: Run the organization's custom linter on the codebase.
17+
- name: speckit.test.deploy
18+
file: .specify/deploy.md
19+
description: Deploy this directory to the staging environment.

tests/extension-commands/main.py

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,16 +11,15 @@ def deploy():
1111

1212
def main():
1313
parser = argparse.ArgumentParser(description="Extension Commands Mock python Script")
14-
parser.add_argument("--lint", action="store_true", help="Run the mock linter")
15-
parser.add_argument("--deploy", action="store_true", help="Run the mock deployer")
14+
group = parser.add_mutually_exclusive_group(required=True)
15+
group.add_argument("--lint", action="store_true", help="Run the mock linter")
16+
group.add_argument("--deploy", action="store_true", help="Run the mock deployer")
1617
args = parser.parse_args()
1718

1819
if args.lint:
1920
lint()
2021
elif args.deploy:
2122
deploy()
22-
else:
23-
print("This is a script to test extension commands. Run with --lint or --deploy.")
2423

2524
if __name__ == "__main__":
2625
main()

0 commit comments

Comments
 (0)