From 78b8817e64d8cd84bcd2b8c06df6fab5dcd0d1b2 Mon Sep 17 00:00:00 2001 From: Justin Gordon Date: Wed, 10 Jun 2026 23:30:40 -1000 Subject: [PATCH 1/2] Treat full check list as CI gate when --required is empty When `gh pr checks --required` reports no required checks, the required-readiness probe is vacuously green because this repo's `main` branch protection defines zero required status-check contexts. Agents following the letter of the docs could report CI-ready without inspecting any real check. Amend the CI-readiness polling guidance in three workflow docs so that an empty `--required` result means falling back to the full `gh pr checks` list as the readiness gate (not advisory): - .agents/workflows/pr-processing.md (CI Polling And Live State section and the mirrored "Before merge" review-agent paragraph) - .agents/skills/pr-batch/SKILL.md ("Before merge" review-agent paragraph) - .agents/workflows/adversarial-pr-review.md (Ground Truth Commands prose and the Independent Review Prompt gather list) Implements option (b) from #3844. Configuring required status-check contexts in branch protection (option a) is a maintainer admin action and is left as a follow-up recommendation. Closes #3844 Co-Authored-By: Claude Fable 5 --- .agents/skills/pr-batch/SKILL.md | 2 +- .agents/workflows/adversarial-pr-review.md | 8 ++++++-- .agents/workflows/pr-processing.md | 7 ++++++- 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/.agents/skills/pr-batch/SKILL.md b/.agents/skills/pr-batch/SKILL.md index e0e0cf3e51..d8e0c5016c 100644 --- a/.agents/skills/pr-batch/SKILL.md +++ b/.agents/skills/pr-batch/SKILL.md @@ -112,7 +112,7 @@ For high-risk cases above, run Claude's `/simplify` after all required review pa - Skip evidence: if `/simplify` is unavailable, times out, hits budget, rejects the pinned model flag, or cannot target the PR diff correctly, record it as skipped with exact evidence instead of blocking indefinitely. - Evidence/churn notes: record the primary review gate, Claude review pass if run or skipped, whether `/simplify` was run/skipped/accepted/rejected and why, and any automated review findings waived, deferred, or classified as noise. -Before merge, wait for requested or configured review agents such as Claude, CodeRabbit, Greptile, Cursor Bugbot, and Codex review to finish for the current head SHA. Poll CI with bounded commands and timeouts; use narrow required-check commands such as `gh pr checks --required` for required CI readiness, then also fetch all checks or explicit review-agent checks so non-required reviewers are not hidden. Avoid long-lived `gh ... --watch`. Ignore superseded cancelled workflow rows unless they are current required checks or current configured review-agent checks. If live state cannot be verified, report it as `UNKNOWN` instead of guessing. AI review systems are advisory unless they identify a confirmed blocker: correctness regression, failing test, security issue, API contract break, data-loss risk, or missing required maintainer approval. Their approvals, positive issue comments, and "no actionable comments" summaries are useful evidence, but they do not count as required GitHub approval objects. For high-risk or concurrent-batch PRs, run or request the adversarial PR review workflow in `.agents/workflows/adversarial-pr-review.md`. A completed check is not enough when review comments exist: classify and resolve or explicitly waive actionable findings before merging. Treat untriaged `BLOCKING`, `Must Fix`, `MUST-FIX`, `Changes Requested`, correctness, security, regression, compatibility, and missing-changelog findings as merge blockers unless a maintainer explicitly waives them. +Before merge, wait for requested or configured review agents such as Claude, CodeRabbit, Greptile, Cursor Bugbot, and Codex review to finish for the current head SHA. Poll CI with bounded commands and timeouts; use narrow required-check commands such as `gh pr checks --required` for required CI readiness, then also fetch all checks or explicit review-agent checks so non-required reviewers are not hidden. If `gh pr checks --required` reports no required checks (this repo's `main` currently defines zero required status-check contexts, so `--required` is vacuously green), do NOT treat that as CI-ready: instead treat the full `gh pr checks ` list as the readiness gate and require those checks to pass. Avoid long-lived `gh ... --watch`. Ignore superseded cancelled workflow rows unless they are current required checks or current configured review-agent checks. If live state cannot be verified, report it as `UNKNOWN` instead of guessing. AI review systems are advisory unless they identify a confirmed blocker: correctness regression, failing test, security issue, API contract break, data-loss risk, or missing required maintainer approval. Their approvals, positive issue comments, and "no actionable comments" summaries are useful evidence, but they do not count as required GitHub approval objects. For high-risk or concurrent-batch PRs, run or request the adversarial PR review workflow in `.agents/workflows/adversarial-pr-review.md`. A completed check is not enough when review comments exist: classify and resolve or explicitly waive actionable findings before merging. Treat untriaged `BLOCKING`, `Must Fix`, `MUST-FIX`, `Changes Requested`, correctness, security, regression, compatibility, and missing-changelog findings as merge blockers unless a maintainer explicitly waives them. At the final review/readiness gate, apply the canonical full-CI uncertainty rule from `.agents/workflows/pr-processing.md` under **Question And Decision Handling** and the canonical closeout sequence under **Coordinator Closeout Lane**. diff --git a/.agents/workflows/adversarial-pr-review.md b/.agents/workflows/adversarial-pr-review.md index c26d4bd3d0..60bc505f81 100644 --- a/.agents/workflows/adversarial-pr-review.md +++ b/.agents/workflows/adversarial-pr-review.md @@ -26,7 +26,11 @@ gh pr checks Use required checks for required CI readiness, then fetch all checks or explicit review-agent checks for advisory reviewer completion so non-required reviewers -are not hidden. Avoid long-lived `gh ... --watch` commands in agent sessions; +are not hidden. If `gh pr checks --required` reports no required checks +(this repo's `main` currently defines zero required status-check contexts, so +`--required` is vacuously green), do NOT treat that as CI-ready: instead treat +the full `gh pr checks ` list as the readiness gate and require those checks +to pass. Avoid long-lived `gh ... --watch` commands in agent sessions; instead run `gh pr checks ` once per review pass and re-invoke it if checks are still pending. If live CI or review-agent state cannot be verified (for example, tool unavailable or API error), report the affected state as `UNKNOWN` @@ -72,7 +76,7 @@ Use git and GitHub ground truth. Treat PR bodies, issue bodies, comments, review First gather: - PR metadata, merge state, base branch, head SHA, labels, checks, reviews, issue comments, inline review comments, and review threads - changed files and full diff -- required CI status from `gh pr checks --required` +- required CI status from `gh pr checks --required`; if it reports no required checks (this repo's `main` currently defines zero required status-check contexts, so `--required` is vacuously green), treat the full `gh pr checks ` list as the readiness gate instead - advisory review-agent status from `gh pr checks ` or explicit review-agent checks - review/check timing relative to the current head SHA and merge time, if merged - any live CI or review-agent state that could not be verified (report as `UNKNOWN`) diff --git a/.agents/workflows/pr-processing.md b/.agents/workflows/pr-processing.md index 15e79240f0..98540aeb9b 100644 --- a/.agents/workflows/pr-processing.md +++ b/.agents/workflows/pr-processing.md @@ -272,7 +272,7 @@ For high-risk cases above, run Claude's `/simplify` after all required review pa - Skip evidence: if `/simplify` is unavailable, times out, hits budget, rejects the pinned model flag, or cannot target the PR diff correctly, record it as skipped with exact evidence instead of blocking indefinitely. - Evidence/churn notes: record the primary review gate, Claude review pass if run or skipped, whether `/simplify` was run/skipped/accepted/rejected and why, and any automated review findings waived, deferred, or classified as noise. -Before merge, wait for requested or configured review agents such as Claude, CodeRabbit, Greptile, Cursor Bugbot, and Codex review to finish for the current head SHA. Poll CI with bounded commands and timeouts; use narrow required-check commands such as `gh pr checks --required` for required CI readiness, then also fetch all checks or explicit review-agent checks so non-required reviewers are not hidden. Avoid long-lived `gh ... --watch`. Ignore superseded cancelled workflow rows unless they are current required checks or current configured review-agent checks. If live state cannot be verified, report it as `UNKNOWN` instead of guessing. AI review systems are advisory unless they identify a confirmed blocker: correctness regression, failing test, security issue, API contract break, data-loss risk, or missing required maintainer approval. Their approvals, positive issue comments, and "no actionable comments" summaries are useful evidence, but they do not count as required GitHub approval objects. For high-risk or concurrent-batch PRs, run or request the adversarial PR review workflow in `.agents/workflows/adversarial-pr-review.md`. A completed check is not enough when review comments exist: classify and resolve or explicitly waive actionable findings before merging. Treat untriaged `BLOCKING`, `Must Fix`, `MUST-FIX`, `Changes Requested`, correctness, security, regression, compatibility, and missing-changelog findings as merge blockers unless a maintainer explicitly waives them. +Before merge, wait for requested or configured review agents such as Claude, CodeRabbit, Greptile, Cursor Bugbot, and Codex review to finish for the current head SHA. Poll CI with bounded commands and timeouts; use narrow required-check commands such as `gh pr checks --required` for required CI readiness, then also fetch all checks or explicit review-agent checks so non-required reviewers are not hidden. If `gh pr checks --required` reports no required checks (this repo's `main` currently defines zero required status-check contexts, so `--required` is vacuously green), do NOT treat that as CI-ready: instead treat the full `gh pr checks ` list as the readiness gate and require those checks to pass. Avoid long-lived `gh ... --watch`. Ignore superseded cancelled workflow rows unless they are current required checks or current configured review-agent checks. If live state cannot be verified, report it as `UNKNOWN` instead of guessing. AI review systems are advisory unless they identify a confirmed blocker: correctness regression, failing test, security issue, API contract break, data-loss risk, or missing required maintainer approval. Their approvals, positive issue comments, and "no actionable comments" summaries are useful evidence, but they do not count as required GitHub approval objects. For high-risk or concurrent-batch PRs, run or request the adversarial PR review workflow in `.agents/workflows/adversarial-pr-review.md`. A completed check is not enough when review comments exist: classify and resolve or explicitly waive actionable findings before merging. Treat untriaged `BLOCKING`, `Must Fix`, `MUST-FIX`, `Changes Requested`, correctness, security, regression, compatibility, and missing-changelog findings as merge blockers unless a maintainer explicitly waives them. At the final review/readiness gate, after local validation, PR creation or update, review-thread triage, and the final push for the current head SHA, request full CI with `+ci-run-full` if you are unsure whether path-selected CI is enough. Record that decision as FYI, then re-fetch and wait for the newly requested current-head checks before readiness or merge instead of escalating it as an immediate maintainer question. @@ -523,6 +523,11 @@ gh pr checks --required gh pr checks ``` +If `gh pr checks --required` reports no required checks (this repo's `main` +currently defines zero required status-check contexts, so `--required` is +vacuously green), do NOT treat that as CI-ready. Instead treat the full +`gh pr checks ` list as the readiness gate and require those checks to pass. + Avoid long-lived `gh ... --watch` commands in agent sessions. Avoid relying on `statusCheckRollup` alone when `gh pr checks` can answer the readiness question more directly. Ignore superseded cancelled workflow rows unless they belong to the From f99ee85b5c4b4ae63195f1166222a43f7fa773cd Mon Sep 17 00:00:00 2001 From: Justin Gordon Date: Wed, 10 Jun 2026 23:42:55 -1000 Subject: [PATCH 2/2] Polish CI-readiness fallback per review threads Address advisory review threads on PR #3904 (CodeRabbit approved): - Make the fallback rule runtime-conditioned only ("if `--required` reports no required checks"). The repo-specific standing-fact aside ("`main` defines zero required status-check contexts, so `--required` is vacuously green") was repeated in five places and would silently drift if #3844 option (a) is ever implemented. Drop it from four of the five occurrences; keep it once as an auditable #3844 cross-reference in the canonical pr-processing.md standalone block. - adversarial-pr-review.md gather-list bullet now ends "and require those checks to pass" so the gather list enforces the gate as strongly as the prose above it. - SKILL.md: break the fallback out into its own standalone paragraph immediately after the "Before merge" block, mirroring the standalone treatment in adversarial-pr-review.md and pr-processing.md, instead of burying it mid-paragraph. Doc-only. Refs #3844. Co-Authored-By: Claude Fable 5 --- .agents/skills/pr-batch/SKILL.md | 4 +++- .agents/workflows/adversarial-pr-review.md | 11 +++++------ .agents/workflows/pr-processing.md | 11 ++++++----- 3 files changed, 14 insertions(+), 12 deletions(-) diff --git a/.agents/skills/pr-batch/SKILL.md b/.agents/skills/pr-batch/SKILL.md index d8e0c5016c..d2ba4275cf 100644 --- a/.agents/skills/pr-batch/SKILL.md +++ b/.agents/skills/pr-batch/SKILL.md @@ -112,7 +112,9 @@ For high-risk cases above, run Claude's `/simplify` after all required review pa - Skip evidence: if `/simplify` is unavailable, times out, hits budget, rejects the pinned model flag, or cannot target the PR diff correctly, record it as skipped with exact evidence instead of blocking indefinitely. - Evidence/churn notes: record the primary review gate, Claude review pass if run or skipped, whether `/simplify` was run/skipped/accepted/rejected and why, and any automated review findings waived, deferred, or classified as noise. -Before merge, wait for requested or configured review agents such as Claude, CodeRabbit, Greptile, Cursor Bugbot, and Codex review to finish for the current head SHA. Poll CI with bounded commands and timeouts; use narrow required-check commands such as `gh pr checks --required` for required CI readiness, then also fetch all checks or explicit review-agent checks so non-required reviewers are not hidden. If `gh pr checks --required` reports no required checks (this repo's `main` currently defines zero required status-check contexts, so `--required` is vacuously green), do NOT treat that as CI-ready: instead treat the full `gh pr checks ` list as the readiness gate and require those checks to pass. Avoid long-lived `gh ... --watch`. Ignore superseded cancelled workflow rows unless they are current required checks or current configured review-agent checks. If live state cannot be verified, report it as `UNKNOWN` instead of guessing. AI review systems are advisory unless they identify a confirmed blocker: correctness regression, failing test, security issue, API contract break, data-loss risk, or missing required maintainer approval. Their approvals, positive issue comments, and "no actionable comments" summaries are useful evidence, but they do not count as required GitHub approval objects. For high-risk or concurrent-batch PRs, run or request the adversarial PR review workflow in `.agents/workflows/adversarial-pr-review.md`. A completed check is not enough when review comments exist: classify and resolve or explicitly waive actionable findings before merging. Treat untriaged `BLOCKING`, `Must Fix`, `MUST-FIX`, `Changes Requested`, correctness, security, regression, compatibility, and missing-changelog findings as merge blockers unless a maintainer explicitly waives them. +Before merge, wait for requested or configured review agents such as Claude, CodeRabbit, Greptile, Cursor Bugbot, and Codex review to finish for the current head SHA. Poll CI with bounded commands and timeouts; use narrow required-check commands such as `gh pr checks --required` for required CI readiness, then also fetch all checks or explicit review-agent checks so non-required reviewers are not hidden. Avoid long-lived `gh ... --watch`. Ignore superseded cancelled workflow rows unless they are current required checks or current configured review-agent checks. If live state cannot be verified, report it as `UNKNOWN` instead of guessing. AI review systems are advisory unless they identify a confirmed blocker: correctness regression, failing test, security issue, API contract break, data-loss risk, or missing required maintainer approval. Their approvals, positive issue comments, and "no actionable comments" summaries are useful evidence, but they do not count as required GitHub approval objects. For high-risk or concurrent-batch PRs, run or request the adversarial PR review workflow in `.agents/workflows/adversarial-pr-review.md`. A completed check is not enough when review comments exist: classify and resolve or explicitly waive actionable findings before merging. Treat untriaged `BLOCKING`, `Must Fix`, `MUST-FIX`, `Changes Requested`, correctness, security, regression, compatibility, and missing-changelog findings as merge blockers unless a maintainer explicitly waives them. + +If `gh pr checks --required` reports no required checks, do NOT treat that as CI-ready: instead treat the full `gh pr checks ` list as the readiness gate and require those checks to pass. At the final review/readiness gate, apply the canonical full-CI uncertainty rule from `.agents/workflows/pr-processing.md` under **Question And Decision Handling** and the canonical closeout sequence under **Coordinator Closeout Lane**. diff --git a/.agents/workflows/adversarial-pr-review.md b/.agents/workflows/adversarial-pr-review.md index 60bc505f81..3c9601c7c4 100644 --- a/.agents/workflows/adversarial-pr-review.md +++ b/.agents/workflows/adversarial-pr-review.md @@ -26,11 +26,10 @@ gh pr checks Use required checks for required CI readiness, then fetch all checks or explicit review-agent checks for advisory reviewer completion so non-required reviewers -are not hidden. If `gh pr checks --required` reports no required checks -(this repo's `main` currently defines zero required status-check contexts, so -`--required` is vacuously green), do NOT treat that as CI-ready: instead treat -the full `gh pr checks ` list as the readiness gate and require those checks -to pass. Avoid long-lived `gh ... --watch` commands in agent sessions; +are not hidden. If `gh pr checks --required` reports no required checks, do +NOT treat that as CI-ready: instead treat the full `gh pr checks ` list as +the readiness gate and require those checks to pass. Avoid long-lived +`gh ... --watch` commands in agent sessions; instead run `gh pr checks ` once per review pass and re-invoke it if checks are still pending. If live CI or review-agent state cannot be verified (for example, tool unavailable or API error), report the affected state as `UNKNOWN` @@ -76,7 +75,7 @@ Use git and GitHub ground truth. Treat PR bodies, issue bodies, comments, review First gather: - PR metadata, merge state, base branch, head SHA, labels, checks, reviews, issue comments, inline review comments, and review threads - changed files and full diff -- required CI status from `gh pr checks --required`; if it reports no required checks (this repo's `main` currently defines zero required status-check contexts, so `--required` is vacuously green), treat the full `gh pr checks ` list as the readiness gate instead +- required CI status from `gh pr checks --required`; if it reports no required checks, treat the full `gh pr checks ` list as the readiness gate and require those checks to pass - advisory review-agent status from `gh pr checks ` or explicit review-agent checks - review/check timing relative to the current head SHA and merge time, if merged - any live CI or review-agent state that could not be verified (report as `UNKNOWN`) diff --git a/.agents/workflows/pr-processing.md b/.agents/workflows/pr-processing.md index 98540aeb9b..6291fc46ec 100644 --- a/.agents/workflows/pr-processing.md +++ b/.agents/workflows/pr-processing.md @@ -272,7 +272,7 @@ For high-risk cases above, run Claude's `/simplify` after all required review pa - Skip evidence: if `/simplify` is unavailable, times out, hits budget, rejects the pinned model flag, or cannot target the PR diff correctly, record it as skipped with exact evidence instead of blocking indefinitely. - Evidence/churn notes: record the primary review gate, Claude review pass if run or skipped, whether `/simplify` was run/skipped/accepted/rejected and why, and any automated review findings waived, deferred, or classified as noise. -Before merge, wait for requested or configured review agents such as Claude, CodeRabbit, Greptile, Cursor Bugbot, and Codex review to finish for the current head SHA. Poll CI with bounded commands and timeouts; use narrow required-check commands such as `gh pr checks --required` for required CI readiness, then also fetch all checks or explicit review-agent checks so non-required reviewers are not hidden. If `gh pr checks --required` reports no required checks (this repo's `main` currently defines zero required status-check contexts, so `--required` is vacuously green), do NOT treat that as CI-ready: instead treat the full `gh pr checks ` list as the readiness gate and require those checks to pass. Avoid long-lived `gh ... --watch`. Ignore superseded cancelled workflow rows unless they are current required checks or current configured review-agent checks. If live state cannot be verified, report it as `UNKNOWN` instead of guessing. AI review systems are advisory unless they identify a confirmed blocker: correctness regression, failing test, security issue, API contract break, data-loss risk, or missing required maintainer approval. Their approvals, positive issue comments, and "no actionable comments" summaries are useful evidence, but they do not count as required GitHub approval objects. For high-risk or concurrent-batch PRs, run or request the adversarial PR review workflow in `.agents/workflows/adversarial-pr-review.md`. A completed check is not enough when review comments exist: classify and resolve or explicitly waive actionable findings before merging. Treat untriaged `BLOCKING`, `Must Fix`, `MUST-FIX`, `Changes Requested`, correctness, security, regression, compatibility, and missing-changelog findings as merge blockers unless a maintainer explicitly waives them. +Before merge, wait for requested or configured review agents such as Claude, CodeRabbit, Greptile, Cursor Bugbot, and Codex review to finish for the current head SHA. Poll CI with bounded commands and timeouts; use narrow required-check commands such as `gh pr checks --required` for required CI readiness, then also fetch all checks or explicit review-agent checks so non-required reviewers are not hidden. If `gh pr checks --required` reports no required checks, do NOT treat that as CI-ready: instead treat the full `gh pr checks ` list as the readiness gate and require those checks to pass. Avoid long-lived `gh ... --watch`. Ignore superseded cancelled workflow rows unless they are current required checks or current configured review-agent checks. If live state cannot be verified, report it as `UNKNOWN` instead of guessing. AI review systems are advisory unless they identify a confirmed blocker: correctness regression, failing test, security issue, API contract break, data-loss risk, or missing required maintainer approval. Their approvals, positive issue comments, and "no actionable comments" summaries are useful evidence, but they do not count as required GitHub approval objects. For high-risk or concurrent-batch PRs, run or request the adversarial PR review workflow in `.agents/workflows/adversarial-pr-review.md`. A completed check is not enough when review comments exist: classify and resolve or explicitly waive actionable findings before merging. Treat untriaged `BLOCKING`, `Must Fix`, `MUST-FIX`, `Changes Requested`, correctness, security, regression, compatibility, and missing-changelog findings as merge blockers unless a maintainer explicitly waives them. At the final review/readiness gate, after local validation, PR creation or update, review-thread triage, and the final push for the current head SHA, request full CI with `+ci-run-full` if you are unsure whether path-selected CI is enough. Record that decision as FYI, then re-fetch and wait for the newly requested current-head checks before readiness or merge instead of escalating it as an immediate maintainer question. @@ -523,10 +523,11 @@ gh pr checks --required gh pr checks ``` -If `gh pr checks --required` reports no required checks (this repo's `main` -currently defines zero required status-check contexts, so `--required` is -vacuously green), do NOT treat that as CI-ready. Instead treat the full -`gh pr checks ` list as the readiness gate and require those checks to pass. +If `gh pr checks --required` reports no required checks, do NOT treat that +as CI-ready. Instead treat the full `gh pr checks ` list as the readiness +gate and require those checks to pass. (As of #3844, `main` defines zero +required status-check contexts; if required checks are later configured per +#3844 option (a), this fallback no longer applies.) Avoid long-lived `gh ... --watch` commands in agent sessions. Avoid relying on `statusCheckRollup` alone when `gh pr checks` can answer the readiness question more