|
| 1 | +# PR Manual Testing |
| 2 | + |
| 3 | +Some skern changes — CLI behavior, skill management, platform adapters, registry logic — require manual verification that CI cannot provide. This page documents the lightweight manual test verification step required on PRs before merge. |
| 4 | + |
| 5 | +## When Is It Required? |
| 6 | + |
| 7 | +Manual test verification is required for PRs that touch: |
| 8 | + |
| 9 | +- CLI commands or flags (`internal/cli/`) |
| 10 | +- Skill management logic (`internal/skill/`, `internal/registry/`) |
| 11 | +- Platform adapters (`internal/platform/`) |
| 12 | +- Overlap detection (`internal/overlap/`) |
| 13 | +- Output formatting (`internal/output/`) |
| 14 | + |
| 15 | +### Exemptions |
| 16 | + |
| 17 | +The following PRs are exempt from manual test verification: |
| 18 | + |
| 19 | +- Documentation-only changes (`docs/`, `*.md` outside `internal/`) |
| 20 | +- CI/CD configuration changes (`.github/workflows/`) |
| 21 | +- Trivially mechanical changes (typo fixes, import reordering, comment edits) |
| 22 | + |
| 23 | +Mark exempt PRs with a comment noting the exemption reason, then check the template checkbox. |
| 24 | + |
| 25 | +## Comment Format |
| 26 | + |
| 27 | +A maintainer posts a PR comment with the following table before merge: |
| 28 | + |
| 29 | +````markdown |
| 30 | +### Manual Test Verification |
| 31 | + |
| 32 | +| # | Scenario | Result | Notes | |
| 33 | +|---|----------|--------|-------| |
| 34 | +| 1 | `skern skill list` shows expected skills | :white_check_mark: | | |
| 35 | +| 2 | `skern skill install <name> --platform claude-code` works | :white_check_mark: | | |
| 36 | +| 3 | `skern platform status` reflects installed skill | :white_check_mark: | | |
| 37 | +| 4 | *<add rows relevant to the PR>* | | | |
| 38 | + |
| 39 | +**Tested at:** `<commit SHA>` |
| 40 | +**Platform:** `<OS / shell>` |
| 41 | +```` |
| 42 | + |
| 43 | +Adapt the rows to the specific PR — only test what the PR changes. Keep it brief. |
| 44 | + |
| 45 | +## Rules |
| 46 | + |
| 47 | +1. **Pass / Fail / Skip** — use :white_check_mark: (pass), :x: (fail), or :fast_forward: (skip with reason in Notes). |
| 48 | +2. **All rows must pass** before merge. If any row fails, the PR must be updated and re-tested. |
| 49 | +3. **Pin the commit SHA** — record which commit was tested. If new commits are pushed after the verification comment, the test must be re-run. |
| 50 | +4. **Brevity** — test only what the PR changes. Don't re-run the full agent test harness (that's a separate concern, see below). |
| 51 | + |
| 52 | +## Labels |
| 53 | + |
| 54 | +Two GitHub labels track manual test status: |
| 55 | + |
| 56 | +| Label | Color | Meaning | |
| 57 | +|-------|-------|---------| |
| 58 | +| `needs-manual-test` | yellow | PR requires manual test verification before merge | |
| 59 | +| `manual-test-verified` | green | Manual test verification has been posted and passes | |
| 60 | + |
| 61 | +Maintainers apply `needs-manual-test` when opening or reviewing a non-exempt PR, and replace it with `manual-test-verified` after a passing verification comment. |
| 62 | + |
| 63 | +## Relationship to the Agent Test Harness |
| 64 | + |
| 65 | +The [agent test harness](/contributing/manual-testing) (`tests/manual/`) is a separate, more comprehensive concern — it covers 10 end-to-end scenarios for pre-release validation. PR manual testing is lighter: it verifies only the specific behavior changed by the PR. The two complement each other but are independent processes. |
0 commit comments