feat(agent-docs-audit): delta-only high-confidence CI gate#3301
Conversation
Fails the workflow when a PR introduces new high-confidence findings vs base, while leaving existing baseline debt and heuristic classes warning- only. High-confidence classes that block CI: - broken @imports - broken symlink targets - linked-inverted pairs - unexpected-duplicate pairs Heuristic/advisory classes (still warning-only via the comment): - broken path refs (backtick regex, known false-positive prone) - budget warnings - unresolved pnpm commands Mechanics: a separate gate script worktree-scans origin/$BASE_REF and diffs high-confidence finding identities against the PR head, scoped to files (or pair-dirs) the PR actually changed. Result is written to /tmp/agent-docs-gate.json; the comment script reads it and prepends a 'Blocking' header so reviewers see why CI is red without opening logs.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c7e928e53a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const scoped = newFindings.filter((f) => { | ||
| if (f.type === 'pair') return dirSet.has(f.dir); | ||
| return changedSet.has(f.relPath); | ||
| }); |
There was a problem hiding this comment.
When a PR deletes or renames an imported agent-doc target such as .claude/rules/foo.md without editing the importing AGENTS.md, highConfidenceFindings() correctly reports a new broken import on the importer, but this filter drops it because changedSet only contains the changed/deleted target path. Since .claude/rules/** changes trigger this workflow, those PRs can introduce a new broken @import and still pass the new gate; the import case needs to also scope against changed import targets, not only f.relPath.
Useful? React with 👍 / 👎.
| @@ -0,0 +1,172 @@ | |||
| #!/usr/bin/env node | |||
There was a problem hiding this comment.
This new script is invoked by agent-docs-audit.yml, but the workflow's pull_request.paths list still matches the audit, L1, and PR-comment scripts only, not .github/scripts/agent-docs-pr-gate.mjs. As a result, a PR that changes only the blocking gate logic will not run this audit workflow at all, so regressions in the new CI gate can be merged without the workflow validating them.
Useful? React with 👍 / 👎.
Adds a fifth high-confidence blocking class: when a PR transitions a dir from a paired classification (linked, linked-inverted, unexpected-duplicate, intentional-different) to 'single', the gate fails. Legitimate-single dirs are unaffected since this check is delta-only. Also fixes a gap where the comment script took the 'resolved' path when the PR deleted an agent-doc file, hiding the gate's Blocking banner. The banner now renders in both the findings and clean paths whenever the gate result file marks blocking=true.
1 participant
Adds a delta-only CI gate to the agent-docs audit. The workflow now fails only when a PR introduces new high-confidence findings vs base; existing baseline debt and heuristic classes stay warning-only.
@imports, broken symlink targets,linked-invertedandunexpected-duplicatepairs.origin/$BASE_REFin a tmp worktree, set-diffs high-confidence finding identities vs PR head, scopes to files/dirs the PR actually changed.#3300 (test PR with an intentional broken
@import) stays open as the first live test. Once this lands, the next run there should turn red for the right reason.