Skip to content

Add GitHub Action to validate issue/PR template compliance#12541

Open
rodrigobnogueira wants to merge 1 commit into
aio-libs:masterfrom
rodrigobnogueira:check-template-12163-v2
Open

Add GitHub Action to validate issue/PR template compliance#12541
rodrigobnogueira wants to merge 1 commit into
aio-libs:masterfrom
rodrigobnogueira:check-template-12163-v2

Conversation

@rodrigobnogueira
Copy link
Copy Markdown
Member

What do these changes do?

Adds a Check Template GitHub Actions workflow (.github/workflows/check-template.yml) that automatically validates issue and PR descriptions against the repository templates.

For issues (bug reports):

  • Detects version fields left at the default placeholder (e.g. $ python --version with no actual output)
  • Detects empty required sections (Describe the bug, To Reproduce, Expected behavior)
  • Handles the legacy markdown issue template (ISSUE_TEMPLATE.md)

For PRs:

  • Checks that the template sections are present ("What do these changes do?", "Checklist")
  • Detects a blank "What do these changes do?" section
  • Rejects PRs opened from the fork's default branch (inspired by bdraco's suggestion)

When violations are found, the workflow:

  1. Applies the invalid label for triage
  2. Posts an explanatory comment listing all problems
  3. Closes the issue/PR

Safeguards:

  • Skips bot-authored submissions (dependabot, pre-commit-ci, etc.)
  • Avoids duplicate comments on re-edits (updates the existing comment instead)
  • Maintainers can bypass validation by applying bot:chronographer:skip or backport:skip labels

Prior validation on fork:

Are there changes in behavior for the user?

No changes to the library itself. This only affects the GitHub repository workflow for contributors: issues and PRs that do not follow the templates will now be automatically flagged and closed with a comment explaining what needs to be fixed.

Is it a substantial burden for the maintainers to support this?

No — this is a standalone GitHub Actions workflow with no dependencies on the codebase. It can be disabled by simply deleting the file or adding a workflow_dispatch condition. The label-based bypass mechanism means maintainers can override it at any time.

Related issue number

Fixes #12163

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt
  • Add a new news fragment into the CHANGES/ folder

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label May 14, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 14, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.94%. Comparing base (7eb0e80) to head (b836e74).
⚠️ Report is 2 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #12541   +/-   ##
=======================================
  Coverage   98.94%   98.94%           
=======================================
  Files         131      131           
  Lines       46622    46622           
  Branches     2414     2414           
=======================================
  Hits        46132    46132           
  Misses        367      367           
  Partials      123      123           
Flag Coverage Δ
Autobahn 22.44% <ø> (ø)
CI-GHA 98.91% <ø> (ø)
OS-Linux 98.67% <ø> (+<0.01%) ⬆️
OS-Windows 97.03% <ø> (ø)
OS-macOS 97.93% <ø> (-0.01%) ⬇️
Py-3.10.11 97.42% <ø> (-0.01%) ⬇️
Py-3.10.20 97.91% <ø> (ø)
Py-3.11.15 98.15% <ø> (-0.01%) ⬇️
Py-3.11.9 97.68% <ø> (+<0.01%) ⬆️
Py-3.12.10 97.76% <ø> (ø)
Py-3.12.13 98.25% <ø> (ø)
Py-3.13.13 98.47% <ø> (+<0.01%) ⬆️
Py-3.14.4 97.78% <ø> (-0.66%) ⬇️
Py-3.14.5 98.05% <ø> (?)
Py-3.14.5t 97.54% <ø> (+<0.01%) ⬆️
Py-pypy3.11.15-7.3.21 97.41% <ø> (-0.02%) ⬇️
VM-macos 97.93% <ø> (-0.01%) ⬇️
VM-ubuntu 98.67% <ø> (+<0.01%) ⬆️
VM-windows 97.03% <ø> (ø)
cython-coverage 37.94% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Adds a Check Template workflow triggered on issue and PR open/edit
events.  It validates that:

- Bug report version fields contain actual output, not just the
  default placeholder commands
- Required issue sections (Describe the bug, To Reproduce,
  Expected behavior) are not left empty
- PR descriptions include the template sections and have content
  in the 'What do these changes do?' section
- PRs are not opened from the fork's default branch

When violations are detected, the workflow applies the 'invalid'
label, posts an explanatory comment, and closes the submission.
It skips bot-authored items and avoids duplicate comments on
re-edits.

Maintainers can bypass validation by adding bot:chronographer:skip
or backport:skip labels before the check runs.

Fixes aio-libs#12163
@rodrigobnogueira rodrigobnogueira force-pushed the check-template-12163-v2 branch from e2f15c0 to b836e74 Compare May 14, 2026 21:06
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 14, 2026

Merging this PR will not alter performance

✅ 72 untouched benchmarks
⏩ 69 skipped benchmarks1


Comparing rodrigobnogueira:check-template-12163-v2 (b836e74) with master (7eb0e80)2

Open in CodSpeed

Footnotes

  1. 69 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on master (125dfae) during the generation of this report, so 7eb0e80 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@bdraco bdraco added the backport:skip Skip backport bot label May 15, 2026
@Dreamsorcerer
Copy link
Copy Markdown
Member

Dreamsorcerer commented May 19, 2026

I'm even more concerned about this now, as we have AGENTS.md ensuring that AI contributions follow the template now, and this week I've seen a couple of obviously human-written PRs that didn't follow the template, and I'm grateful they didn't make me read through the whole template for what could be said in 1 line.

This will now only close human PRs. I've also now unsubscribed from PR notifications, so I have no way to audit this workflow and know that legitimate PRs are getting closed incorrectly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip Skip backport bot bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement automation closing PRs and issues where the OP does not follow the expected template

3 participants