chore(ci): add PR Sentinel compliance checks and grace-period auto-close by rafaeltonholo · Pull Request #11254 · thunderbird/thunderbird-android

rafaeltonholo · 2026-07-09T17:54:47Z

Contribution Summary

Linked Issue/Ticket: Closes #11158

Description

Adds PR Sentinel, an automated bot that validates every pull request against the project's contribution guidelines, gives contributors a single, actionable feedback comment, and enforces the rules with a grace period before any auto-close.

Mandatory checks (per PR):

A linked issue via a closing keyword (e.g. Closes #123, Part of #123), ignoring keywords inside code so the template placeholder doesn't count.
A Conventional Commit PR title.
Conventional Commit messages on every commit (description ≤ 70 chars), with no Co-authored-by: trailers.
No merge commits (rebase onto the base branch instead).
A completed AI Disclosure (exactly one option selected).

This Pull request also includes:

Updates in the Pull Request template
Updates in the Contribution Workflow
Updates in the Git Commit Guidelines

Testing

To test this PR, you can run the ./scripts/ci/pr-sentinel/tests/all_tests.sh, which will run the script unit tests or run the ./scripts/ci/pr-sentinel/report-pr.sh <pr-number> --dry-run.

You can also see the PR Sentinel outputs in the following PRs, inside my personal fork:

The above-mentioned PRs will still be open until the PR Sentinel auto-closes them.

AI Disclosure

Select one of the following (mandatory)

This contribution does not include any changes created or assisted by AI.
This contribution includes changes assisted by AI.
Disclosure: Used AI to help with shell script language, script test setup and script organization
This contribution includes changes created by AI.

rafaeltonholo · 2026-07-10T11:52:51Z

@kryoseu and @wmontwe, if it feels better, I can convert the sh scripts to Python. That might be easier to review/maintain in the future.

I could also write it as a CLI in Kotlin. Just let me know your preference. I don't think it would take too long to convert them.

…t script

- Add CI enforcement notice for PR titles and commit subjects - Increase description limit from 50 to 70 characters - Add `perf`, `build`, `ci`, and `deps` commit types

- Add mandatory PR description fields (linked issue, description, AI disclosure, checklist) - Document PR Sentinel automated validation rules and timeline - Clarify PR description guidelines with updated template sections - Add examples using closing keywords for issue linking

…ucture - Add automated PR Sentinel checks notice with compliance requirements and 3-day auto-close timeline - Restructure template sections with better formatting and clearer guidance - Add Testing section for reproducible steps - Improve placeholder text with closing keyword examples - Minor wording improvements in checklist items

- Add Conventional Commits validation for PR titles and commit subjects - Add linked issue check requiring closing keywords - Add AI disclosure section validation - Add commit description length check (max 70 chars) - Add Co-authored-by trailer check - Add bot exemption and escalation logic

- Add tests for PR title, linked issue, and AI disclosure validation - Add tests for commit subject length and Co-authored-by trailer checks - Add tests for bot exemption and escalation logic - Add tests for date helper functions (ISO/epoch conversion) - Add shared assertion helpers for test suites

…tration - Add constants for label, markers, and dry-run mode - Add gh CLI wrapper with dry-run support - Add functions to find, create, update, and delete status comments - Add functions to add and remove PR Sentinel label - Add status comment body renderer with contextual "How to fix" links

- Add evaluate-pr.sh to run all PR Sentinel checks and output JSON verdict - Check PR title, linked issue, AI disclosure, commit subjects, and Co-authored-by trailers - Detect merge commits and flag for rebase - Support bot exemption and draft detection - Add test suite covering compliant/non-compliant PRs, topic filtering, and edge cases

- Add report-pr.sh to evaluate PRs and apply status comment/label - Skip draft PRs and clear prior artifacts when converted to draft - Post/update consolidated status comment for non-compliant PRs - Apply 'pr-sentinel: needs updates' label when non-compliant and not exempt - Exit non-zero for non-compliant, non-exempt PRs to fail CI checks

- Add escalate-pr.sh to escalate or close non-compliant PRs based on status comment age - Escalate after 24 hours by adding close-countdown banner with 3-day deadline - Auto-close after 72 hours if issues remain unresolved - Skip draft PRs and already-escalated comments to avoid duplicate notifications - Add tests for escalation marker detection and age-based decision logic

…rcement - Run on pull_request_target with permissions to comment and label - Support manual dispatch for batch evaluation or dry-run mode - Evaluate PRs for compliance with title, issue link, AI disclosure, and commit requirements - Post consolidated status comments and apply 'pr-sentinel: needs updates' label when non-compliant - Fail CI checks for non-compliant PRs to block merge

This will avoid the issue 1234 to be linked

The check now ignores closing keywords inside inline code (`Closes thunderbird#1234`) or fenced code blocks, matching GitHub's actual issue-linking behavior where code-enclosed keywords do not create links.

This will allow GitHub to auto-link the commit

…el labels - Add `pr-sentinel: ready for review` label for compliant PRs - Implement mutually exclusive label transitions via mark_ready_for_review, mark_needs_updates, and clear_sentinel_labels - Add tests for label state transitions and mutual exclusivity - Update documentation to reflect new ready-for-review label behaviour

…lose workflow

- Add app token generation step with fallback to github.token - Set botmobile environment for both sentinel workflows - Move GH_TOKEN from workflow-level to step-level env vars

wmontwe

This feels a bit over-scoped for the initial Sentinel rollout. That said, shell scripts are acceptable for this scope, and I’d prefer to keep Python usage low in the repository unless there’s a strong reason to add it.

A Kotlin CLI would probably be the preferred option long term for maintainability and testing, but converting this now would delay the rollout and restart a lot of the review.

I’d rather keep this PR as is and then learn how sentinel behaves in real use before deciding whether a Kotlin CLI is a worthy follow-up.

wmontwe · 2026-07-10T12:35:34Z


-if [[ $# -ne 4 ]]; then
-    echo "Usage: $0 <pr-number> <identifier> <message> <status>"
+if [[ $# -lt 4 || $# -gt 5 ]]; then


- Move build and ci from commit types to scopes in documentation - Update regex to exclude build and ci as standalone types - Add test coverage for invalid ci type usage

wmontwe

LGTM

rafaeltonholo requested review from a team as code owners July 9, 2026 17:54

rafaeltonholo added the report: exclude Exclude changes from user-facing reports (internal, minor, or not relevant to users). label Jul 9, 2026

rafaeltonholo requested a review from wmontwe July 9, 2026 17:54

rafaeltonholo temporarily deployed to review July 9, 2026 17:54 — with GitHub Actions Inactive

rafaeltonholo temporarily deployed to botmobile July 9, 2026 17:54 — with GitHub Actions Inactive

github-actions Bot added the tb-team Tasks and features handled by project maintainers label Jul 9, 2026

thunderbird-botmobile Bot assigned wmontwe Jul 9, 2026

kryoseu reviewed Jul 9, 2026

View reviewed changes

Comment thread scripts/ci/pr-sentinel/lib.sh

wmontwe reviewed Jul 10, 2026

View reviewed changes

Comment thread docs/contributing/git-commit-guide.md Outdated

rafaeltonholo added 17 commits July 10, 2026 09:36

feat(ci): add optional resolved message parameter to manage-pr-commen…

40b3320

…t script

docs: update commit message guide with CI enforcement and new types

791c847

- Add CI enforcement notice for PR titles and commit subjects - Increase description limit from 50 to 70 characters - Add `perf`, `build`, `ci`, and `deps` commit types

test(ci): add tests for dry-run mode in PR Sentinel gh wrapper

9f53c46

test(ci): add test runner for PR Sentinel test suites

7678220

chore: add markdown code formatting to PR template example

30268fd

This will avoid the issue 1234 to be linked

fix(ci): strip markdown code blocks from linked issue check

65206fc

The check now ignores closing keywords inside inline code (`Closes thunderbird#1234`) or fenced code blocks, matching GitHub's actual issue-linking behavior where code-enclosed keywords do not create links.

chore(ci): remove backticks from PR Sentinel check error messages

c9d7bcd

This will allow GitHub to auto-link the commit

rafaeltonholo added 6 commits July 10, 2026 09:36

feat(ci): support "Part of #N" as valid issue link in PR

f001245

fix(ci): rename require_label to require_label_needs_updates in autoc…

eb43208

…lose workflow

feat(ci): use GitHub App token for PR Sentinel workflows

b03df90

- Add app token generation step with fallback to github.token - Set botmobile environment for both sentinel workflows - Move GH_TOKEN from workflow-level to step-level env vars

chore(ci): add safety flags to pr-sentinel lib.sh

acd0ed1

chore(ci): remove perf type from commit message validation

812774f

rafaeltonholo force-pushed the chore/11158/ci-pr-sentinel branch from 6ff232b to 812774f Compare July 10, 2026 12:37

wmontwe reviewed Jul 10, 2026

View reviewed changes

rafaeltonholo changed the title ~~ci: add PR Sentinel compliance checks and grace-period auto-close~~ chore(ci): add PR Sentinel compliance checks and grace-period auto-close Jul 10, 2026

rafaeltonholo added 3 commits July 10, 2026 11:27

chore(ci): remove build and ci types from allowed commit message types

7d2ada0

- Move build and ci from commit types to scopes in documentation - Update regex to exclude build and ci as standalone types - Add test coverage for invalid ci type usage

chore(ci): remove shellcheck directive comments from pr-sentinel scripts

411332a

chore(ci): add safety flags to pr-sentinel checks.sh

ff78851

wmontwe approved these changes Jul 10, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

chore(ci): add PR Sentinel compliance checks and grace-period auto-close #11254

chore(ci): add PR Sentinel compliance checks and grace-period auto-close #11254
rafaeltonholo wants to merge 26 commits into
thunderbird:mainfrom
rafaeltonholo:chore/11158/ci-pr-sentinel

rafaeltonholo commented Jul 9, 2026

Uh oh!

Uh oh!

rafaeltonholo commented Jul 10, 2026

Uh oh!

Uh oh!

wmontwe left a comment

Uh oh!

Uh oh!

wmontwe Jul 10, 2026

Uh oh!

Uh oh!

wmontwe left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

rafaeltonholo commented Jul 9, 2026

Contribution Summary

Description

Testing

AI Disclosure

Uh oh!

Uh oh!

rafaeltonholo commented Jul 10, 2026

Uh oh!

Uh oh!

wmontwe left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wmontwe Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wmontwe left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants