|
| 1 | +## Summary |
| 2 | + |
| 3 | +Adds **`testsprite test failure triage --project <id>`** — groups all failed tests in a project into root-cause clusters using existing M2.1 analysis fields, without downloading failure bundles. |
| 4 | + |
| 5 | +Closes #[ISSUE_NUMBER] |
| 6 | + |
| 7 | +## Motivation |
| 8 | + |
| 9 | +When a batch run produces many failures, agents currently see a flat list and must investigate each test independently. This command collapses duplicate root causes into a few clusters with a representative test, confidence score, and fix priority — reducing bundle downloads, AI tokens, and debug time. |
| 10 | + |
| 11 | +Mentor-approved as CLI Phase-0 triage ahead of native backend clustering. |
| 12 | + |
| 13 | +## What changed |
| 14 | + |
| 15 | +### New command |
| 16 | +```bash |
| 17 | +testsprite test failure triage --project proj_xxx --output json |
| 18 | +testsprite test failure triage --project proj_xxx --type backend --filter checkout --dry-run |
| 19 | +``` |
| 20 | + |
| 21 | +### Grouping heuristics (deterministic, no LLM) |
| 22 | +1. Shared `recommendedFixTarget.reference` |
| 23 | +2. Env-wide `failureKind` (`infra`, `network`, `network_timeout`, `routing_404`) |
| 24 | +3. Normalized `rootCauseHypothesis` prefix |
| 25 | +4. Singleton fallback |
| 26 | + |
| 27 | +### Files |
| 28 | +| File | Change | |
| 29 | +|------|--------| |
| 30 | +| `src/lib/failure-triage.ts` | Grouping logic, confidence/priority scoring, text renderer | |
| 31 | +| `src/lib/failure-triage.test.ts` | 11 unit tests | |
| 32 | +| `src/commands/test.ts` | `runFailureTriage`, Commander wiring | |
| 33 | +| `src/commands/test.test.ts` | 7 integration tests | |
| 34 | +| `DOCUMENTATION.md`, `README.md`, `CHANGELOG.md` | Docs | |
| 35 | +| `skills/testsprite-verify.skill.md` | Agent triage playbook | |
| 36 | +| `test/help.snapshot.test.ts.snap` | Help snapshot | |
| 37 | + |
| 38 | +## Example JSON output |
| 39 | +```json |
| 40 | +{ |
| 41 | + "projectId": "proj_abc", |
| 42 | + "clusters": [ |
| 43 | + { |
| 44 | + "clusterId": "cluster_kind_network_timeout", |
| 45 | + "label": "Environment issue (network_timeout)", |
| 46 | + "groupReason": "failure_kind", |
| 47 | + "representativeTestId": "test_a", |
| 48 | + "memberTestIds": ["test_a", "test_b"], |
| 49 | + "confidence": 0.88, |
| 50 | + "fixPriority": 1 |
| 51 | + } |
| 52 | + ], |
| 53 | + "summary": { "totalFailed": 2, "clusterCount": 1, "skipped": 0 } |
| 54 | +} |
| 55 | +``` |
| 56 | + |
| 57 | +## Test plan |
| 58 | + |
| 59 | +- [x] `npm run typecheck` — pass |
| 60 | +- [x] `npm run lint` — pass |
| 61 | +- [x] `npx vitest run src/lib/failure-triage.test.ts` — 11/11 pass |
| 62 | +- [x] `npx vitest run src/commands/test.test.ts -t runFailureTriage` — 7/7 pass |
| 63 | +- [x] `npm run build` — pass |
| 64 | +- [x] Help snapshot added for `test failure triage --help` |
| 65 | + |
| 66 | +## Notes |
| 67 | + |
| 68 | +- Client-side Phase-0 only; when backend ships native clustering, this command can become a thin wrapper over a new read API. |
| 69 | +- Skips tests listed as `failed` but with no `failure/summary` (stale status race) — reported in `summary.skipped`. |
0 commit comments