Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
1ffee4e
Document coordinated batch QA lane
justin808 Jun 20, 2026
8c76863
Clarify batch QA evidence expectations
justin808 Jun 20, 2026
cc15c03
Separate mixed batch QA guidance
justin808 Jun 20, 2026
96ea239
Docs: sync batch QA lane audit guidance
justin808 Jun 20, 2026
b9826d0
Docs: clarify batch QA evidence fields
justin808 Jun 20, 2026
ffa9f7e
Docs: confirm QA scope in batch audits
justin808 Jun 20, 2026
8c6735b
Docs: wire QA evidence through batch planning
justin808 Jun 20, 2026
34f9ea1
Clarify QA lane readiness evidence semantics
justin808 Jun 21, 2026
2beb173
Docs: address QA lane review nits
justin808 Jun 21, 2026
f3d06dd
Docs: align QA lane status vocabulary
justin808 Jun 21, 2026
9441175
Docs: clarify QA coverage fallback states
justin808 Jun 21, 2026
b5cde7a
Docs: tighten batch QA prompt wording
justin808 Jun 21, 2026
f0c8621
Docs: disambiguate QA handoff prompt fields
justin808 Jun 21, 2026
e3835e2
Docs: add QA evidence handoff examples
justin808 Jun 21, 2026
80568da
Docs: clarify QA fallback audit handling
justin808 Jun 21, 2026
7b08ce2
Docs: route QA audit follow-ups consistently
justin808 Jun 21, 2026
4bcd3ab
Merge remote-tracking branch 'origin/main' into jg-codex/4127-batch-q…
justin808 Jun 22, 2026
ad8230c
Address batch QA lane review follow-ups
justin808 Jun 22, 2026
0fb275a
Polish batch QA lane status wording
justin808 Jun 22, 2026
5d9e3c6
Merge main and polish QA lane docs
justin808 Jun 22, 2026
306c3f1
Clarify batch QA review follow-ups
justin808 Jun 22, 2026
81cb604
Gate QA evidence in batch goal prompt
justin808 Jun 22, 2026
2fe35d2
Round out batch QA examples
justin808 Jun 22, 2026
c2f1315
Clarify QA closeout evidence mapping
justin808 Jun 22, 2026
4a70bc9
Clarify QA readiness review follow-ups
justin808 Jun 22, 2026
6e883e2
Clarify QA evidence edge cases
justin808 Jun 22, 2026
f10e37e
Clean up QA handoff examples
justin808 Jun 22, 2026
9e630f6
Merge origin/main into batch QA lane
justin808 Jun 22, 2026
6a9fc35
Merge remote-tracking branch 'origin/main' into jg-codex/4127-batch-q…
justin808 Jun 23, 2026
dd87719
Merge remote-tracking branch 'origin/main' into jg-codex/4127-batch-q…
justin808 Jun 23, 2026
a27ad6c
docs: address batch QA review feedback
justin808 Jun 23, 2026
01671ca
Merge remote-tracking branch 'origin/main' into jg-codex/4127-batch-q…
justin808 Jun 23, 2026
b84b85d
docs: tighten QA lane review guidance
justin808 Jun 23, 2026
89b1e42
docs: mirror required QA handoff scope
justin808 Jun 23, 2026
163e87f
docs: clarify QA handoff fields
justin808 Jun 23, 2026
4caa3c1
docs: block unknown QA status in gates
justin808 Jun 23, 2026
a10c72c
docs: fix QA status list wrapping
justin808 Jun 23, 2026
e48a4aa
docs: clarify QA lane boundaries
justin808 Jun 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 26 additions & 14 deletions .agents/skills/plan-pr-batch/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,21 @@ Plan a PR batch
`batches/<batch-id>.json` with those lane refs before dependent workers
start; otherwise `agent-coord status --batch-id <batch-id> --json` cannot report
`blocked_on` lanes.
- Apply `.agents/workflows/pr-processing.md` under **Batch QA Lane**. Declare
whether QA is required, which subset qualifies, and the planned QA owner.
When QA is required, plan the `qa` lane name, stable owner/heartbeat
expectations, and private-state representation for the launched coordinator
to create when the backend is available. If private state will be
unavailable, require the final handoff to record QA claim/heartbeat state as
`UNKNOWN` and include allowed fallback evidence (see
`.agents/workflows/pr-processing.md`, the "Allowed fallback evidence:"
label in the **Batch QA Lane** section) instead of downgrading QA to
Comment thread
justin808 marked this conversation as resolved.
`not required`.
Use that section's capitalization convention: uppercase `UNKNOWN` means
coordination/backend state, and lowercase `unknown` is the QA lane status.
When QA is omitted for low-risk work, record `not required` plus the
rationale. Include the final QA Evidence expectations in the Batch Plan and
generated goal prompt.
- Build a File-touch map for the batch: list the paths each item changes or
intends to affect, including creates, deletes, and renames. Never guess
paths.
Expand Down Expand Up @@ -157,6 +172,7 @@ Plan a PR batch
- `merge_authority`:
- Concurrent activity and dependency status:
- Coordination hooks, including backend claim exclusions:
- Batch QA Lane decision and QA Evidence expectations:
Comment thread
justin808 marked this conversation as resolved.
- Verification expectations:
- Prompt sizing: `Goal prompt character count: N characters`; note any split fallback and keep omitted item
details here, not in the goal prompt.
Expand All @@ -175,10 +191,10 @@ Preflight first: if this session cannot run workers without blocking approval pr
Repository: OWNER/REPO
Batch objective: ...
merge_authority: <none | ask | auto_merge_when_gates_pass>.
Scope summary: [one paragraph: compact titles, sequencing, dependencies, exclusions, and path ownership for this batch. Keep bulky evidence, long validation notes, and later-batch details outside this prompt.]
Batch QA Lane: <required: lane/owner/scope/private-state or UNKNOWN fallback | not required: rationale>.
Scope summary: [one paragraph: compact titles, sequencing, dependencies, exclusions, path ownership; keep bulky evidence, validation notes, and later-batch details outside.]
File-touch map (one line per item; pick the applicable format):
- PR/Issue #N -> changed/affected paths, including create/delete/rename (owner: lane/name)
- PR/Issue #N -> summarized path pattern(s) plus collision-relevant exact paths/renames/deletes (owner: lane/name)
- PR/Issue #N -> exact paths or summarized patterns, including creates/deletes/renames (owner: lane/name)
- PR/Issue #N -> UNKNOWN (paths not determinable from issue body/design notes; treat as serial)
Batch-level reservations, not tied to a single item:
- Deferred/reserved paths -> path(s) (reason: ... / later owner: lane/name)
Expand All @@ -187,29 +203,25 @@ Items:
- PR #N: URL
Goal: one-line outcome.
Worker notes: short scope, branch, or dependency note.
Done when: final state is reported using the requested `merge_authority` and the split states from pr-batch.
Done when: final state satisfies requested `merge_authority` and matches a pr-batch split state.
- Issue #N: URL
Goal: one-line outcome.
Worker notes: short scope, branch, or dependency note.
Done when: final state is reported using the requested `merge_authority` and the split states from pr-batch, with PR/no-PR evidence or documented no-fix rationale.
Done when: final state satisfies requested `merge_authority`, with PR/no-PR evidence or no-fix rationale.

Execution rules:
- Run `git fetch --prune origin main` first. Verify repo-local `.agents/skills/pr-batch/SKILL.md` and `.agents/workflows/pr-processing.md` exist before launching workers. If either is missing in the checkout but present on `origin/main`, update the worktree before continuing; if still missing, stop and report repo workflow state as `UNKNOWN`.
- Follow `.agents/skills/pr-batch/SKILL.md` "Goal Prompt Template"; if skill autoloading is unavailable, copy its safety, review, /simplify, CI, and readiness gates before running.
- Dispatch one subagent per independent item; group dependent items only when shared context is required. Dispatch only the current file-disjoint wave. Hold serial and `UNKNOWN`
discovery lanes until no active editor lane can collide with them.
- Workers edit only owned File-touch map paths; this map is how the batch makes
pr-batch's "disjoint write scopes" concrete, since pr-batch's own template has
no File-touch map slot. If an `UNKNOWN`, unlisted, or other-lane path is
needed, stop, report discovered paths, and wait for an updated map or explicit
coordinator confirmation before editing.
- Run `git fetch --prune origin main` first. Verify repo-local `.agents/skills/pr-batch/SKILL.md` and `.agents/workflows/pr-processing.md` before editing. If a required file is missing locally but present on `origin/main`, update that specific file before continuing; if it is still missing, report repo workflow state as `UNKNOWN`.
- Follow `.agents/skills/pr-batch/SKILL.md`; if autoloading is unavailable, copy its safety/review/simplify/CI/readiness gates.
- Dispatch one subagent per independent item, current file-disjoint wave only. Hold serial and `UNKNOWN` discovery lanes until no active editor lane can collide.
- Workers edit only owned File-touch map paths. If an `UNKNOWN`, unlisted, or other-lane path is needed, stop before editing it and report discovered paths for coordinator confirmation.
- Sequenced lanes may share declared files only in the stated order.
- Each subagent must verify current GitHub state before edits and report UNKNOWN for unverifiable facts.
- For coordination, respect coordination claims and dependencies:
`agent-coord doctor --json`, then
`agent-coord status --repo <repo> --target <issue-or-pr> --json` or
`agent-coord status --batch-id <batch-id> --json` per `AGENTS.md`; claim,
heartbeat, and stop/report UNKNOWN.
- Apply Batch QA Lane in `.agents/workflows/pr-processing.md`: declare required/not required, use private `qa` lane when available, `UNKNOWN` fallback evidence when not, and include QA Evidence in final handoff.
- Use local validation, self-review, review-comment, CI, and readiness gates from the repo workflow. For PRs, merge only when `merge_authority` is `auto_merge_when_gates_pass` or a later explicit approval exists, current release mode permits it, and confidence/readiness gates pass; document confidence data in the PR description.
- Final handoff must include links, tests, blockers, next action, confidence or UNKNOWN facts, `merge_authority`, and explicit final-state sections: `merged`, `ready-gates-clean`, `ready-no-merge-authority`, `waiting-on-checks-or-review`, `external-gate-failing`, `blocked-user-input`, or `no-pr-evidence`.
```
Expand Down
95 changes: 68 additions & 27 deletions .agents/skills/post-merge-audit/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,15 @@ The resolver is read-only. It resolves the default release-candidate base, the h
Keep PR-range inclusion separate from worked-issue coverage so no-PR,
blocked, parked, and unmerged lanes are still evaluated.

Show included worked issues, included PRs, excluded near-matches, base/head SHAs, coordination status evidence, and assumptions. Ask for confirmation before deep audit unless the user explicitly asks to proceed without confirmation.
After the scope algorithm identifies the batch or reports an `UNKNOWN` scope,
collect any QA lane and QA Evidence block for that batch. Do not use missing QA
state to shrink the worked-issue scope; report it as a QA coverage finding or
`UNKNOWN` fact instead.
Use the capitalization convention from `.agents/workflows/pr-processing.md`:
uppercase `UNKNOWN` means coordination/backend state, and lowercase `unknown`
is the QA lane status.

Show included worked issues, included PRs, collected QA lanes and QA Evidence blocks, excluded near-matches, base/head SHAs, coordination status evidence, and assumptions. Ask for confirmation before deep audit unless the user explicitly asks to proceed without confirmation.

## Audit Checks

Expand All @@ -114,31 +122,60 @@ For each included PR:
reproducible artifact or justified missing-artifact caveat, internal consistency, production-environment
caveats, and refutable-conclusion handling.
- Validation: compare changed areas with the validation evidence in the PR body or comments.
- QA evidence: verify required QA Evidence exists, records `Tested at` with the
Comment thread
justin808 marked this conversation as resolved.
PR/head SHA or audited range it applies to, is current for that head/range, covers the changed
surfaces, and does not leave release-blocking findings untriaged. If private
coordination claim/heartbeat state is `UNKNOWN`, verify the documented
fallback evidence is complete and names a concrete QA owner and branch/worktree
before treating QA coverage as satisfied. Only private claim/heartbeat
sub-values may be `UNKNOWN` in fallback mode. If fallback evidence is absent or
incomplete, classify the QA lane as `unknown` and surface it as a readiness
blocker. Verify `Release-blocking status` is derived from QA lane status:
`satisfied` -> `clear`, `blocked` -> `blocked`, `waived` -> `waived`,
`not_applicable` -> `not_applicable`, and `in_progress` / `unknown` -> `blocked`.
Comment thread
justin808 marked this conversation as resolved.
- Cross-PR interactions: compare changed files, shared behavior, assumptions, and release-sensitive areas across the batch.
- Decision log: inspect any `Codex Decision Log` or equivalent section and verify the decisions still hold after the merge.

For each worked issue from coordination state or advisory `codex-claim`
recovery rows, including no-PR, blocked, parked, done-unmerged, or still-open
lanes:
For each worked issue, QA lane, or advisory `codex-claim` recovery row from
coordination state, including no-PR, blocked, parked, done-unmerged, or
still-open lanes:

- Intent coverage: compare the issue intent and acceptance criteria with the PR
diff, no-PR evidence comment, branch state, or blocker note.
- Intent coverage: compare the issue or QA-lane intent with the PR diff, no-PR
evidence comment, QA evidence, branch state, or blocker note.
- Final state: verify whether the issue was merged, closed, parked, blocked,
left open intentionally, or remains `UNKNOWN`.
left open intentionally, or remains `UNKNOWN`; for QA lanes, verify whether
the QA coverage status is `satisfied`, `blocked`, `waived`, healthy
`in_progress`, `not_applicable` when QA was not required, or `unknown`.
- Handoff expectations: check validation evidence, decision-point count,
confidence notes, review/comment triage, and any Process Gap Disposition
fields required by `.agents/workflows/pr-processing.md`.
confidence notes, QA evidence, review/comment triage, and any Process Gap
Disposition fields required by `.agents/workflows/pr-processing.md`.
- Classification: reuse the intent-achievement classes from
`.agents/workflows/continuous-evaluation-loop.md` (`in_progress`,
`realized`, `partial`, `missed`, `regressed`, `stalled`, or `unknown`) and
explain any `UNKNOWN` evidence needed to resolve the issue outcome.
- Post-merge intake: record healthy `in_progress` lanes and evidenced
`realized` outcomes in the worked-issue table as no-action items; route
explain any `UNKNOWN` evidence needed to resolve the issue outcome. For QA
lanes, use the QA-coverage result `satisfied`, `blocked`, `waived`,
`in_progress`, `not_applicable`, or `unknown`. Use `satisfied` when required
QA evidence is current, adequately scoped, and has no untriaged
release-blocking finding; `blocked` when a release-blocking QA finding still
needs a fix or waiver; `waived` when an explicit waiver exists and the auditor
verifies a maintainer comment URL, issue link, or PR body entry names the
finding, scope, and reason; `in_progress` when required QA is not complete;
`not_applicable` when QA was correctly omitted with `QA required: no` and a
documented rationale; and `unknown` when evidence is missing, stale, or
incomplete.
- Post-merge intake: record healthy `in_progress` worked-issue lanes and
evidenced `realized` worked-issue outcomes, `satisfied` or `waived` QA lanes,
and `not_applicable` QA omissions in the coverage table as no-action items
during active batch phase;
treat required QA lanes still `in_progress` during readiness/release audits as
QA coverage findings; route
Comment thread
justin808 marked this conversation as resolved.
`stalled` lanes back to the batch coordinator as resume/reassign/drop
decisions unless the user explicitly approves tracking the stalled lane as an
issue; route every other non-OK worked-issue class (`partial`, `missed`,
Comment thread
justin808 marked this conversation as resolved.
`regressed`, or `unknown`), merged or not, into the issue plan or an explicit
coordinator action that names the missing evidence or decision.
`regressed`, or `unknown`), merged or not, and every non-OK QA coverage
outcome (`blocked`, `unknown`, or release-audit `in_progress`) into the issue
plan or an explicit coordinator action that names the missing evidence or
decision.

## Codex And Claude Coordination

Expand Down Expand Up @@ -185,11 +222,13 @@ lane was evaluated, even when the issue produced no merged PR:
The audit should usually produce an issue plan for non-OK findings, but not create issues until approval.

- **No issue**: for `OK`, duplicate findings, findings fully resolved by the
audit evidence, evidenced `realized` lanes, or healthy `in_progress` lanes;
include `realized` and `in_progress` lanes in the worked-issue coverage table
so the coordinator can see they were checked.
audit evidence, evidenced `realized` lanes, healthy `in_progress` worked-issue
lanes, evidenced `satisfied` or `waived` QA lanes, or evidenced QA omissions
marked `not_applicable`; include `realized`, worked-issue `in_progress`,
`satisfied`, `waived`, and `not_applicable` rows in the worked-issue/QA-lane
coverage table so the coordinator can see they were checked.
- **Changelog only**: for missing changelog entries; prefer one bundled changelog issue or a recommendation to run `/update-changelog`, not one issue per entry.
- **One child issue**: for each independently actionable fix PR, revert consideration, maintainer question, follow-up task, or non-OK worked-issue outcome (`partial`, `missed`, `regressed`, or `unknown`) that needs follow-up.
- **One child issue**: for each independently actionable fix PR, revert consideration, maintainer question, follow-up task, non-OK worked-issue outcome (`partial`, `missed`, `regressed`, or `unknown`), or non-OK QA coverage outcome (`blocked`, `unknown`, or release-audit `in_progress`) that needs follow-up.
- **Parent issue**: create one parent issue only to group two or more related
_child fix_ issues from the same audit. Do **not** create a standalone
audit-snapshot tracker (a `Post-<range> audit` / `Post-rc.N catch-up audit`
Expand Down Expand Up @@ -234,15 +273,17 @@ Only the coordinator should create issues. Independent Codex and Claude audits s
Return high-risk findings first, then:

1. Review-gate violations, including PRs merged before requested reviews finished, before actionable review findings were triaged, or with AI review systems incorrectly counted as approval gates.
2. Missing changelog candidates, with a single recommendation to run `/update-changelog` when any are found.
3. Cross-PR interaction risks.
4. A deduped issue plan with parent/child recommendations and fingerprints.
5. A worked-issue coverage table with issue number, coordination lane/branch,
linked PR or no-PR/blocker evidence, final state, intent-achievement
classification, and `UNKNOWN` facts (see the example in
`.agents/workflows/post-merge-audit.md`).
6. A PR-by-PR table.
7. Exact commands and data sources used, including bounded `agent-coord status`
2. QA coverage findings, including missing, stale, still-`UNKNOWN` coverage/scope,
or insufficient required QA evidence.
3. Missing changelog candidates, with a single recommendation to run `/update-changelog` when any are found.
4. Cross-PR interaction risks.
5. A deduped issue plan with parent/child recommendations and fingerprints.
6. A worked-issue/QA-lane coverage table with issue number or QA lane id,
coordination lane/branch, linked PR or no-PR/blocker/QA evidence, final
state, issue intent-achievement or QA-coverage classification, and `UNKNOWN`
facts (see the example in `.agents/workflows/post-merge-audit.md`).
7. A PR-by-PR table.
8. Exact commands and data sources used, including bounded `agent-coord status`
output for the named batch or the exact reason coordination state was
`UNKNOWN`.

Expand Down
Loading
Loading