Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions opencode/agents/developer.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ You are the senior developer.

When `lead` delegates direct mode for a small, clear, low-risk change without using a slash command, treat it as an approved implementation task.

Later adjustments for that same implementation go back to `developer`; keep continuity and do not expect `lead` to implement them.

Before editing, identify:

- objective;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Overview - Iteration 016

## Failure Pattern

The harness checker had coverage gaps for command and agent prompt drift. It
verified that agents and commands were documented, but not enough of the
contract-to-prompt alignment was enforced mechanically.

The follow-up review found three concrete risks:

- prose evidence containing markdown filenames could be misread as a missing path;
- ordered flows could pass when the required agent names appeared in the wrong order;
- cross-agent checks could pass when a prompt matched only one invariant.

## Root Cause

The checker grew incrementally. Existing checks protected the lead router,
`/feature`, and `/plan`, but `/evolve`, `/scope`, `/design`, and broader
contract-to-prompt alignment were not covered consistently.

## Applied Fix

The checker now adds bounded checks for `/evolve`, `/scope`, `/design`, and
agent-prompt invariants. The path parser now accepts only strings that look like
standalone repository-relative file paths. Regression tests cover the previously
observed false positive and false negatives.

## Risk

These checks are intentionally bounded regex checks, not a natural-language
parser. Future prompt rewrites may need checker pattern updates when wording
changes but the contract remains valid.
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"iteration": 16,
"evaluates_iteration": 16,
"results": [
{
"change_id": "chg-16-1",
"predicted_fixes_confirmed": [
"evolve-order-drift",
"scope-order-drift",
"agent-invariant-drift",
"prose-evidence-path-false-positive"
],
"predicted_fixes_not_confirmed": [],
"risk_tasks_regressed": [],
"risk_tasks_not_regressed": [
"public-checker-patterns-match-english-prompts",
"no-private-config-or-paths-in-public-artifacts",
"existing-feature-plan-checks-unchanged"
],
"unpredicted_regressions": [],
"decision": "keep",
"evidence": [
"node --test scripts/check-harness.test.mjs passed 4/4 tests.",
"node scripts/check-harness.mjs passed.",
"Repository-level scripts/check.sh passed.",
"Public artifact leak check found no private paths, local MCP config, providers, credentials, or local search wiring."
],
"notes": "The public port keeps only harness checker, prompt-contract, docs, and summary AHE evidence. Private opencode.json, MCP servers, provider config, local memory data, and raw transcripts are intentionally excluded."
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"iteration": 16,
"changes": [
{
"id": "chg-16-1",
"type": "new",
"description": "Add mechanical checker coverage for /evolve ordering, cross-agent prompt invariants, and /scope and /design command contracts.",
"files": [
"scripts/check-harness.mjs",
"scripts/check-harness.test.mjs",
"agents/developer.md",
"docs/ai/harness/checks.md"
],
"failure_pattern": "Harness prompts and command contracts could drift without detection, and prior path validation treated prose evidence containing markdown filenames as a missing local path.",
"evidence": [
"docs/ai/evolution/runs/iteration-016-contract-prompt-alignment/evaluation.md",
"docs/ai/evolution/runs/iteration-016-contract-prompt-alignment/analysis/overview.md"
],
"root_cause": "The checker validated coverage and a few critical flows, but did not enforce several command contracts or each configured agent invariant.",
"predicted_fixes": [
"evolve-order-drift",
"scope-order-drift",
"agent-invariant-drift",
"prose-evidence-path-false-positive"
],
"risk_tasks": [
"public-checker-patterns-match-english-prompts",
"no-private-config-or-paths-in-public-artifacts",
"existing-feature-plan-checks-unchanged"
],
"constraint_level": "tool",
"why_this_component": "The failure is mechanical checker coverage, so scripts/check-harness.mjs is the narrowest component. The developer prompt receives one explicit continuity rule to keep the documented contract reflected in the prompt."
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Iteration 016 - Contract/Prompt Alignment

## Result

Decision: `improve`.

The checker now covers additional harness contracts:

- `/evolve` must preserve the `evaluator -> debugger -> evolver` ordering.
- `/scope` must preserve `researcher -> specifier` ordering.
- `/design` must keep its Open Design contract.
- Agent prompts must keep each configured invariant from `docs/ai/harness/agents.md`.
- AHE JSON evidence prose may mention markdown filenames without being treated as a path.

## Scenarios

| Scenario | Expected | Actual |
| --- | --- | --- |
| `node --check scripts/check-harness.mjs` | pass | pass |
| `node scripts/check-harness.mjs` | pass | pass |
| Prose evidence mentions `PRODUCT.md/DESIGN.md` | pass | pass |
| `/evolve` order is inverted | fail | fail |
| `/scope` order is inverted | fail | fail |
| `lead` prompt keeps only one invariant | fail | fail |
| `git diff --check` | pass | pass |

## Notes

- This public artifact contains summary evidence only.
- No raw transcripts, private providers, MCP configuration, credentials, or local
machine paths are included.
3 changes: 3 additions & 0 deletions opencode/docs/ai/harness/checks.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ The harness check validates:
`docs/ai/harness/commands.md`;
- local `/feature` contract;
- local `/plan` contract;
- local `/evolve` contract;
- minimum consistency between agent contracts and prompts in `agents/*.md`;
- local `/scope` and `/design` contracts;
- main docs in `docs/ai/harness/`;
- benchmark references to replay and evidence taxonomy;
- AHE run lifecycle under `docs/ai/evolution/runs/`;
Expand Down
Loading
Loading