Skip to content

Commit 9e34f1a

Browse files
locus313Copilot
andcommitted
docs: mandate bats tests for all new scripts in AGENTS.md and copilot-instructions.md
- Replaced the placeholder Testing section in AGENTS.md with full bats suite documentation: test files table, what to test per script, and the mock_curl.sh pattern with a code example - Added step 7 (write tests) to the Adding a New Script checklist in AGENTS.md; renumbered subsequent steps - Updated CI/CD bullet to mention the bats test job - Updated Maintenance Matrix 'Add a new script' row to include tests/test_script_validation.bats - Mirrored all changes in .github/copilot-instructions.md: expanded Adding New Scripts checklist with mandatory test step, replaced single-line Testing Approach with table + requirement prose Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 8b2549d commit 9e34f1a

2 files changed

Lines changed: 76 additions & 13 deletions

File tree

.github/copilot-instructions.md

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -227,15 +227,33 @@ Uses API version `2026-03-10` and the new usage-metrics NDJSON endpoints (signed
227227
3. Source `lib/github-common.sh` for validation and output helpers
228228
4. Start with the standard boilerplate (see Script Anatomy above)
229229
5. Create `action.yml` in the same directory — expose every env var as a named input (required inputs without defaults, optional inputs with sensible defaults); map CLI flags such as `--dry-run` and `--type` to boolean/string inputs and build an `ARGS` array in the `run:` step. Mirror the pattern of any existing `action.yml`.
230-
6. Document in README.md following existing format:
230+
6. **Add tests to `tests/test_script_validation.bats`** — mandatory. Add a labelled section with tests for every required env var missing (exit 1), unknown CLI args (exit 1), `--help` exits 0, and script-specific validation guards. See existing sections for the pattern.
231+
7. Document in README.md following existing format:
231232
- Use case description
232233
- Required variables table
233234
- Usage example with exports
234235
- Output format (if applicable)
235236
- Add a row to the Available Actions table in the "Using Scripts in GitHub Actions" section
236237

237238
### Testing Approach
238-
- **Always test on a test organization first**
239+
240+
The project has a bats unit-test suite in `tests/`. Run the full suite with:
241+
242+
```bash
243+
bats tests/
244+
```
245+
246+
| File | What it covers |
247+
|------|----------------|
248+
| `tests/test_common.bats` | `lib/github-common.sh` pure-logic functions and API helpers |
249+
| `tests/test_script_validation.bats` | Every script — missing env vars, invalid args, `--help`, script-specific guards |
250+
| `tests/mock_curl.sh` | Universal curl mock used by both test files |
251+
252+
Every new script **must** include a test section in `tests/test_script_validation.bats` before it is merged. At minimum test: each required env var missing exits 1, unknown CLI args exit 1, and any enum or allowlist validation specific to the script.
253+
254+
Additional testing approaches:
255+
- **Always test on a test organization first** before running against production
256+
- **Dry-run flags** — several scripts support `--dry-run` to preview changes without applying them
239257

240258
### Commit Messages — Conventional Commits (required)
241259

@@ -285,7 +303,7 @@ When you change one of these files, you must also update the files in the "Also
285303
| `README.md` — script documentation | Verify the script's `# ===` header comment still matches (env vars, options, requirements) |
286304
| `.githooks/pre-commit` | `install-hooks.sh` if hook path or installation instructions change; README.md Best Practices section |
287305
| `install-hooks.sh` | README.md Installation section |
288-
| Add a new script | `action.yml` in the same directory; `README.md` (add use case, env var table, usage example, Available Actions table row) |
306+
| Add a new script | `tests/test_script_validation.bats` (add a test section for the new script); `action.yml` in the same directory; `README.md` (add use case, env var table, usage example, Available Actions table row) |
289307
| Add a new domain folder | `README.md` top-level structure description; `AGENTS.md` Repository Structure section |
290308
| `.github/workflows/ci.yml` — shellcheck flags | `.githooks/pre-commit` shellcheck invocation (keep them in sync) |
291309
| `.github/workflows/copilot-setup-steps.yml` — tool versions | `AGENTS.md` Tech Stack table |

AGENTS.md

Lines changed: 55 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -79,14 +79,58 @@ find . -name "*.sh" | xargs shellcheck --severity=warning --exclude=SC2034,SC109
7979

8080
## Testing
8181

82-
There is no automated test suite. The validation approach is:
82+
The project has a bats unit-test suite in `tests/`. **Every new script must ship with tests.**
8383

84-
1. **Pre-commit hook** — shellcheck on every staged `.sh` file; gitleaks secret scan
85-
2. **Dry-run flags** — several scripts support `--dry-run` to preview changes without applying them:
86-
- `github-close-archived-repo-security-alerts`
87-
- `github-enable-issues`
88-
- `github-organize-stars`
89-
3. **Test org first** — always run against a non-production GitHub org before production
84+
```bash
85+
# Run all tests
86+
bats tests/
87+
88+
# Run a single file
89+
bats tests/test_common.bats
90+
```
91+
92+
### Test files
93+
94+
| File | What it covers |
95+
|------|----------------|
96+
| `tests/test_common.bats` | `lib/github-common.sh` — pure-logic functions (`validate_slug`, `require_env_var`, `require_command`, `err`, `configure_gh_auth`, `validate_token`, `get_repo_page_count`) and API helpers (`gh_api` sentinels, `gh_api_paginate`) |
97+
| `tests/test_script_validation.bats` | Every script — missing required env vars exit 1, invalid CLI args exit 1, `--help` exits 0, script-specific enum/allowlist validation |
98+
| `tests/mock_curl.sh` | Universal drop-in curl mock (used by both test files); response data via env vars `MOCK_CURL_CODE`, `MOCK_CURL_BODY`, `MOCK_CURL_LINK` |
99+
100+
### What to test for every new script
101+
102+
1. **Missing required env vars** — one `@test` per required variable, in the order the script checks them. Each test asserts `status -eq 1`.
103+
2. **Invalid CLI args** — unknown flag exits 1 with an "Unknown" message.
104+
3. **`--help` flag** — exits 0 (for scripts that implement it).
105+
4. **Recognised flags**`--dry-run` and other known flags do not trigger the unknown-arg error (test by asserting output does *not* contain "Unknown").
106+
5. **Script-specific guards** — enum validation (`--type`, `DEPENDABOT_REASON`), URL allowlists (`GIT_URL_PREFIX`), required positional args.
107+
108+
### Mocking pattern
109+
110+
Tests shadow real binaries by prepending a `MOCK_BIN` directory to `PATH`:
111+
112+
```bash
113+
setup() {
114+
MOCK_BIN="$(mktemp -d)"
115+
# Fail gh auth so GITHUB_TOKEN is never auto-resolved from a session
116+
printf '#!/bin/sh\nexit 1\n' > "$MOCK_BIN/gh"
117+
chmod +x "$MOCK_BIN/gh"
118+
}
119+
120+
teardown() { rm -rf "$MOCK_BIN"; }
121+
122+
@test "my-script: exits 1 when GITHUB_TOKEN is not set" {
123+
run bash -c "export PATH='${MOCK_BIN}:${PATH}'; unset GITHUB_TOKEN; bash '${REPO_ROOT}/domain/my-script/my-script.sh'"
124+
[ "$status" -eq 1 ]
125+
}
126+
```
127+
128+
Use `_mock_curl_200` (defined in `test_script_validation.bats`) when a test must reach code that runs after `validate_github_token`.
129+
130+
### Dry-run flags and other testing approaches
131+
132+
- **`--dry-run`** — several scripts support it to preview changes without applying them.
133+
- **Test org first** — always run against a non-production GitHub org before production.
90134

91135
---
92136

@@ -185,8 +229,9 @@ done
185229
4. **Source the shared library** using `SCRIPT_DIR`
186230
5. **Validate all inputs** before any API calls
187231
6. **Create `action.yml`** in the same directory — expose every env var as an input (required inputs first, optional inputs with defaults); map CLI flags (`--dry-run`, `--type`, etc.) to boolean/string inputs and construct the `ARGS` array in the `run:` step. See existing `action.yml` files for the pattern.
188-
7. **Add to README.md** — follow the existing format: use case, env var table, usage example, output format; add a row to the Available Actions table in the "Using Scripts in GitHub Actions" section
189-
8. Place in the correct domain:
232+
7. **Add tests to `tests/test_script_validation.bats`** — add a labelled section (`# ═══ github-<name> ═══`) with tests for: every required env var missing (exit 1), unknown CLI args (exit 1), `--help` exits 0, and any script-specific validation (enum guards, URL allowlists, positional args). See existing sections for the pattern.
233+
8. **Add to README.md** — follow the existing format: use case, env var table, usage example, output format; add a row to the Available Actions table in the "Using Scripts in GitHub Actions" section
234+
9. Place in the correct domain:
190235
- `org-admin/` — organization-level operations (repos, teams, members)
191236
- `enterprise/` — enterprise-level operations (licenses, org enumeration)
192237
- `reporting/` — read-only reports and audits
@@ -199,7 +244,7 @@ done
199244
- **Pre-commit hook:** `.githooks/pre-commit` — runs gitleaks + shellcheck on staged `.sh` files
200245
- **Install:** `./install-hooks.sh` or `git config core.hooksPath .githooks`
201246
- **Bypass (emergency only):** `git commit --no-verify`
202-
- **CI:** shellcheck runs on all `.sh` files on every PR (`.github/workflows/ci.yml`)
247+
- **CI:** shellcheck runs on all `.sh` files on every PR (`.github/workflows/ci.yml`); bats unit tests run in a dedicated `test` job (`bats tests/`)
203248
- **Releases:** automated by Release Please (`.github/workflows/release.yml`) — pushes to `main` trigger a release PR; merging it publishes the GitHub Release and tag
204249

205250
## Commit Messages — Conventional Commits (required)

0 commit comments

Comments
 (0)