Skip to content

Commit d7aefc2

Browse files
chore: add mechanical doc gardening checks
1 parent d6b8ef3 commit d7aefc2

8 files changed

Lines changed: 332 additions & 1 deletion

File tree

opencode/docs/ai/evolution/evolution_history.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,3 +62,22 @@ Evidence:
6262
- Temporary positive and negative lifecycle fixtures passed/fail as expected.
6363
- `change_evaluation.json` records `keep` for the approved scope and documents
6464
the remaining transcript replay limitation for `/evolve`.
65+
66+
## Iteration 013 - Doc Gardening Mechanical Checks
67+
68+
Status: `keep`.
69+
70+
Changes:
71+
72+
- `scripts/check-harness.mjs` validates that `AGENTS.md` stays a short index.
73+
- The checker requires documentation coverage for visible agents and commands.
74+
- The checker validates local paths referenced by manifests and evaluations.
75+
- `docs/ai/harness/checks.md` documents these rules as mechanical doc gardening.
76+
77+
Evidence:
78+
79+
- `node --check scripts/check-harness.mjs` passed.
80+
- `node scripts/check-harness.mjs` passed.
81+
- A temporary negative control confirmed the expected failure when a command is
82+
not documented.
83+
- `./scripts/check.sh` passed from the public repository root.
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Iteration 013 Analysis
2+
3+
## Evidence reviewed
4+
5+
- `scripts/check-harness.mjs`
6+
- `docs/ai/harness/checks.md`
7+
- `docs/ai/evolution/runs/iteration-013-doc-gardening-mechanical-checks/evaluation.md`
8+
9+
## Root cause
10+
11+
The harness already documented doc gardening as a practice, but several cheap
12+
rules were still manual: keeping `AGENTS.md` short, documenting all visible
13+
agents and commands, and avoiding broken local evidence paths in AHE JSON.
14+
15+
## Chosen component
16+
17+
The narrowest fix is tooling plus docs:
18+
19+
- `scripts/check-harness.mjs` enforces the rules locally.
20+
- `docs/ai/harness/checks.md` declares the rules as mechanical doc gardening.
21+
22+
No agent contract, command flow, provider, model, or permission change is needed.
23+
24+
## Attribution expectation
25+
26+
Future harness edits should fail earlier when they add agents, commands, or AHE
27+
evidence without updating the local knowledge map.
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{
2+
"iteration": 13,
3+
"evaluates_iteration": 13,
4+
"results": [
5+
{
6+
"change_id": "chg-1-mechanical-doc-gardening-checks",
7+
"predicted_fixes_confirmed": [
8+
"The packaged checker now validates AGENTS.md as a short index with references to docs/ai/harness/ and docs/ai/evolution/.",
9+
"The packaged checker now validates coverage for agents/*.md in docs/ai/harness/agents.md.",
10+
"The packaged checker now validates coverage for commands/*.md in docs/ai/harness/commands.md.",
11+
"The packaged checker now validates files and evidence paths in AHE JSON without treating arbitrary prose as a path."
12+
],
13+
"predicted_fixes_not_confirmed": [],
14+
"risk_tasks_regressed": [],
15+
"risk_tasks_not_regressed": [
16+
"No workflow phase was added for normal feature work.",
17+
"No routing, permission, model, provider, or agent behavior changed.",
18+
"The path parser is limited to files and evidence fields so prose is not treated as a path."
19+
],
20+
"unpredicted_regressions": [],
21+
"decision": "keep",
22+
"evidence": [
23+
"docs/ai/evolution/runs/iteration-013-doc-gardening-mechanical-checks/evaluation.md",
24+
"docs/ai/evolution/runs/iteration-013-doc-gardening-mechanical-checks/analysis/overview.md",
25+
"docs/ai/evolution/runs/iteration-013-doc-gardening-mechanical-checks/change_manifest.json",
26+
"scripts/check-harness.mjs",
27+
"docs/ai/harness/checks.md"
28+
],
29+
"notes": "This is a static-contract/tooling iteration. Transcript replay is intentionally not required because the change does not modify routing or runtime agent behavior."
30+
}
31+
]
32+
}
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
{
2+
"iteration": 13,
3+
"changes": [
4+
{
5+
"id": "chg-1-mechanical-doc-gardening-checks",
6+
"type": "improvement",
7+
"description": "Extend the packaged harness checker so cheap doc-gardening rules are enforced mechanically instead of staying only as manual review guidance.",
8+
"files": [
9+
"scripts/check-harness.mjs",
10+
"docs/ai/harness/checks.md"
11+
],
12+
"failure_pattern": "Harness documentation can drift when AGENTS.md stops acting as a short index, new agents or commands are not reflected in docs, or AHE JSON references local artifacts that no longer exist.",
13+
"evidence": [
14+
"docs/ai/evolution/runs/iteration-013-doc-gardening-mechanical-checks/evaluation.md",
15+
"docs/ai/evolution/runs/iteration-013-doc-gardening-mechanical-checks/analysis/overview.md",
16+
"docs/ai/harness/checks.md",
17+
"scripts/check-harness.mjs"
18+
],
19+
"root_cause": "Doc gardening was documented as a lightweight practice, but the checker only validated a subset of the rules, leaving avoidable drift to manual review.",
20+
"component_touched": "tooling",
21+
"predicted_fixes": [
22+
"The checker fails when AGENTS.md no longer references the harness and evolution docs or grows beyond the short-index threshold.",
23+
"The checker fails when an agent markdown file is not documented in docs/ai/harness/agents.md.",
24+
"The checker fails when a command markdown file is not documented in docs/ai/harness/commands.md.",
25+
"The checker fails when files or evidence fields in AHE JSON reference missing local repo paths."
26+
],
27+
"risk_tasks": [
28+
"Do not add new mandatory workflow phases for normal feature work.",
29+
"Do not change routing, permissions, models, providers, or agent behavior.",
30+
"Avoid an over-broad path parser that treats prose as a missing local path.",
31+
"Keep error messages specific enough for agent remediation."
32+
],
33+
"constraint_level": "tool",
34+
"why_this_component": "The failure pattern is repository drift in local docs and AHE artifacts, so a local checker is the smallest enforceable boundary. Agent prompts would be weaker and runtime replays are unnecessary because no routing behavior changes.",
35+
"post_change_validation": [
36+
"Run `node --check scripts/check-harness.mjs`.",
37+
"Run `node scripts/check-harness.mjs` and require `Harness check passed.`",
38+
"Run a temporary negative control that removes one command documentation marker and require a missing documented command failure.",
39+
"Run `./scripts/check.sh` from the public repository root.",
40+
"Run `git diff --check`."
41+
],
42+
"decision_criteria": {
43+
"keep": "All static checks pass and the negative control fails with a clear doc coverage message.",
44+
"improve": "The main check passes but the negative control message is unclear or too broad.",
45+
"rollback+pivot": "The checker rejects valid historical artifacts or requires behavioral/routing changes to pass."
46+
}
47+
}
48+
]
49+
}
Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# Iteration 013 - Doc Gardening Mechanical Checks
2+
3+
## Objective evaluated
4+
5+
Convert cheap doc-gardening rules into local mechanical checks without changing
6+
agent routing, permissions, models, providers, or workflow behavior.
7+
8+
## Scenarios executed
9+
10+
### Scenario 1 - Script syntax
11+
12+
- Evidence type: `static_contract`
13+
- Command:
14+
15+
```bash
16+
node --check scripts/check-harness.mjs
17+
```
18+
19+
- Expected: syntax check exits successfully.
20+
- Result: pass.
21+
22+
### Scenario 2 - Harness contract check
23+
24+
- Evidence type: `static_contract`
25+
- Command:
26+
27+
```bash
28+
node scripts/check-harness.mjs
29+
```
30+
31+
- Expected: checker accepts the packaged harness and reports
32+
`Harness check passed.`
33+
- Result: pass.
34+
35+
### Scenario 3 - Negative doc coverage control
36+
37+
- Evidence type: `static_contract`
38+
- Command:
39+
40+
```bash
41+
tmpdir="$(mktemp -d)"
42+
cp -R . "$tmpdir/opencode"
43+
python3 - "$tmpdir/opencode/docs/ai/harness/commands.md" <<'PY'
44+
from pathlib import Path
45+
path = Path(__import__("sys").argv[1])
46+
text = path.read_text()
47+
path.write_text(text.replace("### `/test`", "### `test`", 1))
48+
PY
49+
(cd "$tmpdir/opencode" && node scripts/check-harness.mjs)
50+
```
51+
52+
- Expected: checker fails with a clear missing documented command message.
53+
- Result: pass.
54+
55+
### Scenario 4 - Public repository check
56+
57+
- Evidence type: `static_contract`
58+
- Command from repository root:
59+
60+
```bash
61+
./scripts/check.sh
62+
```
63+
64+
- Expected: required files, packaged harness checks, and private-data scan pass.
65+
- Result: pass.
66+
67+
### Scenario 5 - Whitespace safety
68+
69+
- Evidence type: `static_contract`
70+
- Command from repository root:
71+
72+
```bash
73+
git diff --check
74+
```
75+
76+
- Expected: no whitespace errors.
77+
- Result: pass.
78+
79+
## Results summary
80+
81+
| Scenario | Type | Status |
82+
| --- | --- | --- |
83+
| Script syntax | static_contract | pass |
84+
| Harness contract check | static_contract | pass |
85+
| Negative doc coverage control | static_contract | pass |
86+
| Public repository check | static_contract | pass |
87+
| Whitespace safety | static_contract | pass |
88+
89+
## Residual risk
90+
91+
This iteration proves the checker catches local documentation drift. It does not
92+
prove any runtime routing change, and none was intended.

opencode/docs/ai/harness/checks.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,13 +18,21 @@ The harness check validates:
1818

1919
- `opencode.json`;
2020
- `default_agent: lead`;
21+
- `AGENTS.md` as a short index pointing to `docs/ai/harness/` and
22+
`docs/ai/evolution/`;
2123
- minimum frontmatter in `agents/*.md` and `commands/*.md`;
24+
- documentation coverage for every `agents/*.md` file in
25+
`docs/ai/harness/agents.md`;
26+
- documentation coverage for every `commands/*.md` file in
27+
`docs/ai/harness/commands.md`;
2228
- local `/feature` contract;
2329
- local `/plan` contract;
2430
- main docs in `docs/ai/harness/`;
2531
- benchmark references to replay and evidence taxonomy;
2632
- AHE run lifecycle under `docs/ai/evolution/runs/`;
2733
- AHE manifests when present.
34+
- local paths referenced by `change_manifest.json` and
35+
`change_evaluation.json`.
2836

2937
## AHE Run Lifecycle
3038

@@ -45,4 +53,12 @@ Before closing an AHE iteration:
4553
- verify every manifest has predicted fixes, risk tasks, and component level;
4654
- add `change_evaluation.json` when evaluating a previous change.
4755

56+
## Mechanical Doc Gardening
57+
58+
The local check turns the cheapest maintenance rules into mechanical checks:
59+
60+
- `AGENTS.md` must stay a short map, not a long manual;
61+
- new agents and commands must appear in the harness docs;
62+
- manifests and evaluations must not point to missing local artifacts.
63+
4864
Do not make doc gardening mandatory for normal features.

opencode/docs/ai/harness/commands.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,17 @@ Criteria:
6868
- `debugger` enters only for traces, results, or concrete previous evidence.
6969
- Output is scoped specs, atomic tasks, and validation.
7070

71+
## `/mvp-spec`
72+
73+
Contract: `scoper -> researcher -> scoper synthesis -> specifier`.
74+
75+
Criteria:
76+
77+
- Same no-implementation boundary as `/scope`.
78+
- Produces a strict MVP spec with small 1-2 hour tasks.
79+
- Makes out of scope explicit.
80+
- Stops if research is not ready for spec.
81+
7182
## Visible Auxiliary/Subtask Commands
7283

7384
These commands are visible slash commands, but they do not add mandatory phases

0 commit comments

Comments
 (0)