Skip to content

ci: stop PR bot over-closing on missing checklist + fix reopen loop#4581

Merged
mmabrouk merged 3 commits into
release/v0.102.0from
ci/pr-contribution-relax
Jun 8, 2026
Merged

ci: stop PR bot over-closing on missing checklist + fix reopen loop#4581
mmabrouk merged 3 commits into
release/v0.102.0from
ci/pr-contribution-relax

Conversation

@mmabrouk
Copy link
Copy Markdown
Member

@mmabrouk mmabrouk commented Jun 8, 2026

Summary

Follow-up to #4579 / #4580. I ran the contribution bot in live mode against all 18 open non-draft PRs and graded each decision. It got 17 of 18 right: it correctly skipped all internal members (including private-membership ones), passed the two compliant external PRs, and closed 8 genuine low-quality PRs. But it wrongly closed one good PR and revealed a reopen loop. This PR fixes both.

Bug 1 — over-strict template check. PR #4551 included a YouTube demo and detailed testing notes but was auto-closed only because it lacked the ## Checklist section. The bot required all four template sections and read the demo only from the ## Demo section.

Fix: require just a non-empty ## Summary plus a demo for functional changes. Missing Testing/Checklist sections no longer close a PR, and the demo is now detected anywhere in the body.

Bug 2 — reopen loop. When a maintainer manually reopened a flagged PR, the reopened event re-ran the bot and it immediately re-closed the PR (observed live on #4551).

Fix: drop reopened from the triggers. A maintainer reopen now wins. Auto-reopen on a fixed description still works via edited / synchronize.

Demo policy stays strict: every functional change (including SDK and config) still needs a demo, so #4519 and #4391 remain correctly closed.

Testing

Verified locally

Re-ran the decision logic against the real bodies and file lists of all 13 PRs of interest:

  • Swap sendgrid by SMTP #4551 now passes (has a demo); the 10 genuine closures still close; the 2 compliant externals still pass.
  • Confirmed reopened is no longer in the trigger list and the YAML parses.

Added or updated tests

N/A. CI workflow.

QA follow-up

After merge: re-enable the workflow (it is disabled right now), then dispatch a dry-run against #4551 to confirm it now passes, and against a no-demo external PR to confirm it still flags.

Demo

N/A. Touches only .github/**.

Checklist

  • I have included a video or screen recording for UI changes, or marked Demo as N/A
  • Relevant tests pass locally
  • Relevant linting and formatting pass locally
  • I have signed the CLA, or I will sign it when the bot prompts me

mmabrouk added 3 commits June 8, 2026 12:46
Add a pull_request_target workflow that checks PRs from external
contributors (org members and bots are exempt via author_association):
the PR template must be filled in, and any PR touching functional code
(SDK, API, or frontend) must include a demo recording. Non-compliant PRs
get a sticky comment, an incomplete-pr label, and are closed, then
auto-reopened once fixed. Adds workflow_dispatch dry_run/force_external
inputs for testing and registers the new label in .labels.yml.
Two fixes found while auditing the bot against all open PRs:

- Only a non-empty Summary plus a demo (for functional changes) are required.
  Missing Testing/Checklist sections no longer close a PR. The demo is now
  detected anywhere in the body, not just the Demo section. This fixes a PR
  that had a YouTube demo and full testing notes but was closed for lacking
  the checklist section.
- Drop the 'reopened' trigger so a maintainer who manually reopens a flagged
  PR wins, instead of the bot immediately re-closing it. Auto-reopen on a
  fixed description still works via 'edited'/'synchronize'.
@dosubot dosubot Bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Jun 8, 2026
@vercel
Copy link
Copy Markdown

vercel Bot commented Jun 8, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Building Building Preview, Comment Jun 8, 2026 11:57am

Request Review

@dosubot dosubot Bot added the ci/cd label Jun 8, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 8, 2026

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 29c7a2d8-747d-41ae-83f0-47e3699bb590

📥 Commits

Reviewing files that changed from the base of the PR and between 1c2402c and a2dc2f3.

📒 Files selected for processing (1)
  • .github/workflows/13-check-pr-contribution.yml

Disabled knowledge base sources:

  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.


📝 Walkthrough

Summary by CodeRabbit

  • Changes to PR Requirements
    • PR template simplified to require only a non-empty Summary section; Testing, Demo, and Checklist sections are now optional
    • Demo content detection enhanced to scan entire PR body instead of requiring a dedicated Demo section
    • Updated pull request workflow triggers to prevent unintended automatic re-closure of manually reopened PRs

Walkthrough

This PR relaxes PR contribution checks in a GitHub Actions workflow by removing the reopened trigger event and simplifying template validation requirements. Template checks now enforce only a non-empty description and Summary section, and demo detection scans the entire PR body instead of a dedicated Demo section.

Changes

PR Validation Workflow Updates

Layer / File(s) Summary
Trigger event configuration
.github/workflows/13-check-pr-contribution.yml
The pull_request_target trigger removes the reopened event type, leaving only opened, edited, synchronize, and ready_for_review to prevent automatic re-close on manual reopen.
Template validation and demo detection
.github/workflows/13-check-pr-contribution.yml
Required template sections are reduced to description and Summary only; Testing, Demo, and Checklist sections are no longer mandatory. Demo content detection expands to scan the full PR body instead of only the Demo section.

Possibly related PRs

  • Agenta-AI/agenta#4579: Both PRs directly modify the same .github/workflows/13-check-pr-contribution.yml workflow and its PR-template validation and demo-detection logic.

🎯 2 (Simple) | ⏱️ ~8 minutes

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/pr-contribution-relax

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@mmabrouk mmabrouk changed the base branch from main to release/v0.102.0 June 8, 2026 11:57
@mmabrouk mmabrouk merged commit 52d6389 into release/v0.102.0 Jun 8, 2026
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant