From 2b00730ed9094ae1769570b177d539737826a746 Mon Sep 17 00:00:00 2001 From: Burner Date: Thu, 26 Mar 2026 21:35:21 +0100 Subject: [PATCH] docs(testing): clarify integration vs acceptance policy --- AGENTS.md | 28 ++++++++++++++++++++++++++++ docs/PLANS.md | 13 ++++++++++--- docs/exec-plans/active/README.md | 6 ++++++ 3 files changed, 44 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index e6bc024..4b1cc5b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -54,6 +54,8 @@ This repository is agent-first and documentation-indexed for `docctl`. - If checks fail, MUST use failure output to revise and rerun the loop. - If checks pass, MAY perform behavior-preserving refactors; MUST rerun checks after refactors. +- During interactive loops, run fast tests first. Run longer acceptance/E2E + checks at the end of implementation before handoff. - Python test framework in this repository is `pytest`. - Commits intended for `main` MUST use the Conventional Commits format (`type(scope): summary` or `type: summary`) so semantic-release can derive @@ -61,6 +63,32 @@ This repository is agent-first and documentation-indexed for `docctl`. - All programming artifacts MUST be in English: identifiers, modules, classes, functions, comments, log/error messages, CLI flags, and JSON keys. +### Integration Test Requirement Policy +- Scope definitions to avoid ambiguity: + - Integration tests (`tests/integration/`) verify interfaces and interactions + between integrated components/systems for a workflow slice. + - Acceptance/E2E smoke tests (`tests/acceptance/`) verify the system is + acceptable for a small set of top-level user/CLI journeys. +- At least one integration test MUST be added or updated when a change has any + of these impact triggers: + - new user-visible behavior, + - CLI contract change, + - cross-module workflow/orchestration change, + - persistence or serialization contract change (`--json` payloads, NDJSON + session behavior, snapshot/import-export behavior, manifest shape, or exit + code mapping), + - significant bug fix that changes runtime behavior across module boundaries. +- Pure internal refactors that preserve externally observable behavior do not + require new integration tests. +- Temporary exceptions are allowed only when both are present: + - explicit rationale in the active execution plan, + - tracked follow-up item in `docs/exec-plans/tech-debt-tracker.md` (or an + equivalent plan artifact) to add the missing integration coverage. +- Default validation order for feature work: + - run fast unit tests first during implementation loops, + - run impacted integration tests before handoff, + - run the small acceptance/E2E smoke suite at the end. + ### Docstring Quality Policy - Public Python modules, classes, functions, and methods MUST include Google-style docstrings that describe intent and usage, not line-by-line diff --git a/docs/PLANS.md b/docs/PLANS.md index e3d2d39..ccc4da5 100644 --- a/docs/PLANS.md +++ b/docs/PLANS.md @@ -2,9 +2,16 @@ Planning workflow: 1. Capture active plan in `docs/exec-plans/active/`. -2. Define explicit acceptance checks per bounded validate loop. -3. Move completed plans to `docs/exec-plans/completed/`. -4. Track leftovers in `docs/exec-plans/tech-debt-tracker.md`. +2. For every new feature plan, ask explicit acceptance-test questions before + implementation and record the agreed scenarios in the active plan. +3. Define explicit acceptance checks per bounded validate loop. +4. Classify planned test coverage by scope to avoid overlap: + - `tests/integration/` is the default for feature workflow behavior across + module/service boundaries and contracts, + - `tests/acceptance/` is only for a small set of top-level CLI/E2E smoke + scenarios that prove releasability. +5. Move completed plans to `docs/exec-plans/completed/`. +6. Track leftovers in `docs/exec-plans/tech-debt-tracker.md`. Related: - [Active plans](exec-plans/active/README.md) diff --git a/docs/exec-plans/active/README.md b/docs/exec-plans/active/README.md index 84e2a03..55cb078 100644 --- a/docs/exec-plans/active/README.md +++ b/docs/exec-plans/active/README.md @@ -7,6 +7,12 @@ Recommended filename pattern: Each active plan should include: - objective, - acceptance criteria, +- acceptance-test questions asked during planning, +- agreed acceptance scenarios, +- planned `tests/integration/` cases for feature behavior (or a documented + exception with rationale and a linked follow-up item), +- planned `tests/acceptance/` CLI/E2E smoke cases only when releasability + journeys are impacted, - bounded validation loop checkpoints, - decision log, - current status.