Skip to content

Commit e2f15c0

Browse files
rodrigobnogueirarodrigobnogueira
committed
Add GitHub Action to validate issue/PR template compliance
Adds a Check Template workflow triggered on issue and PR open/edit events. It validates that: - Bug report version fields contain actual output, not just the default placeholder commands - Required issue sections (Describe the bug, To Reproduce, Expected behavior) are not left empty - PR descriptions include the template sections and have content in the 'What do these changes do?' section - PRs are not opened from the fork's default branch When violations are detected, the workflow applies the 'invalid' label, posts an explanatory comment, and closes the submission. It skips bot-authored items and avoids duplicate comments on re-edits. Maintainers can bypass validation by adding bot:chronographer:skip or backport:skip labels before the check runs. Fixes #12163
1 parent 125dfae commit e2f15c0

2 files changed

Lines changed: 251 additions & 0 deletions

File tree

‎.github/workflows/check-template.yml‎

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
name: Check Template
2+
3+
on:
4+
issues:
5+
types: [opened, edited]
6+
pull_request_target:
7+
types: [opened, edited]
8+
9+
permissions:
10+
issues: write
11+
pull-requests: write
12+
13+
jobs:
14+
check-template:
15+
name: Check Template
16+
runs-on: ubuntu-latest
17+
# Skip bots (dependabot, pre-commit-ci, etc.)
18+
if: >-
19+
(github.event.issue.user.type || 'User') != 'Bot'
20+
&& (github.event.pull_request.user.type || 'User') != 'Bot'
21+
steps:
22+
- name: Validate submission against template
23+
uses: actions/github-script@v8
24+
with:
25+
script: |
26+
const isIssue = !!context.payload.issue && !context.payload.pull_request;
27+
const isPR = !!context.payload.pull_request;
28+
const item = isPR ? context.payload.pull_request : context.payload.issue;
29+
const body = (item.body || '').trim();
30+
const itemNumber = isPR
31+
? context.payload.pull_request.number
32+
: context.payload.issue.number;
33+
34+
// --- Maintainer bypass ---
35+
// If a maintainer has already triaged the item by applying
36+
// a label, skip validation so human decisions are not overridden.
37+
const bypassLabels = new Set([
38+
'bot:chronographer:skip',
39+
'backport:skip',
40+
]);
41+
const currentLabels = (item.labels || []).map(l => l.name);
42+
if (currentLabels.some(l => bypassLabels.has(l))) {
43+
core.info('Maintainer bypass label found — skipping validation.');
44+
return;
45+
}
46+
47+
let problems = [];
48+
49+
if (isIssue) {
50+
// --- Bug report (Issue Forms YAML) validation ---
51+
// GitHub Issue Forms render `textarea` fields with
52+
// `render: console` as fenced code blocks. An unfilled
53+
// field looks like:
54+
//
55+
// ### Python Version
56+
//
57+
// ```console
58+
// $ python --version
59+
// ```
60+
//
61+
// A properly filled field has extra lines between the
62+
// command and the closing fence.
63+
64+
const versionChecks = [
65+
{
66+
name: 'Python Version',
67+
regex: /### Python Version\s*```console\s*\$ python --version\s*```/,
68+
},
69+
{
70+
name: 'aiohttp Version',
71+
regex: /### aiohttp Version\s*```console\s*\$ python -m pip show aiohttp\s*```/,
72+
},
73+
{
74+
name: 'multidict Version',
75+
regex: /### multidict Version\s*```console\s*\$ python -m pip show multidict\s*```/,
76+
},
77+
{
78+
name: 'propcache Version',
79+
regex: /### propcache Version\s*```console\s*\$ python -m pip show propcache\s*```/,
80+
},
81+
{
82+
name: 'yarl Version',
83+
regex: /### yarl Version\s*```console\s*\$ python -m pip show yarl\s*```/,
84+
},
85+
];
86+
87+
for (const { name, regex } of versionChecks) {
88+
if (regex.test(body)) {
89+
problems.push(
90+
`The **${name}** field still contains only the default ` +
91+
`placeholder command. Please paste the actual output.`
92+
);
93+
}
94+
}
95+
96+
// Detect required textarea sections that are completely empty
97+
// (heading followed immediately by the next heading).
98+
const requiredSections = [
99+
'Describe the bug',
100+
'To Reproduce',
101+
'Expected behavior',
102+
];
103+
for (const section of requiredSections) {
104+
const emptyPattern = new RegExp(
105+
`### ${section}\\s*(?:###|$)`
106+
);
107+
if (emptyPattern.test(body)) {
108+
problems.push(
109+
`The **${section}** section appears to be empty. ` +
110+
`Please provide the requested information.`
111+
);
112+
}
113+
}
114+
115+
// --- Legacy markdown template (ISSUE_TEMPLATE.md) ---
116+
const oldTemplateBlank =
117+
/## Long story short\s*(?:<!--[\s\S]*?-->\s*)*## Expected behaviour/;
118+
if (oldTemplateBlank.test(body)) {
119+
problems.push(
120+
'The **Long story short** section is empty. ' +
121+
'Please describe your problem.'
122+
);
123+
}
124+
} else if (isPR) {
125+
// --- Pull Request template validation ---
126+
127+
if (!body) {
128+
problems.push(
129+
'The PR description is completely empty. ' +
130+
'Please use the provided PR template.'
131+
);
132+
} else {
133+
if (!body.includes('## What do these changes do?')) {
134+
problems.push(
135+
'The PR description is missing the ' +
136+
'"What do these changes do?" section from the template.'
137+
);
138+
}
139+
if (!body.includes('## Checklist')) {
140+
problems.push(
141+
'The PR description is missing the ' +
142+
'"Checklist" section from the template.'
143+
);
144+
}
145+
146+
// Detect a blank "What do these changes do?" section
147+
// (only HTML comments between the heading and the next one).
148+
const emptyBrief =
149+
/## What do these changes do\?\s*(?:<!--[\s\S]*?-->\s*)*## Are there changes in behavior for the user\?/;
150+
if (emptyBrief.test(body)) {
151+
problems.push(
152+
'The **What do these changes do?** section is blank. ' +
153+
'Please describe your changes.'
154+
);
155+
}
156+
}
157+
158+
// --- Reject PRs opened from the fork's default branch ---
159+
const head = context.payload.pull_request.head;
160+
const base = context.payload.pull_request.base;
161+
if (
162+
head.repo.full_name !== base.repo.full_name
163+
&& head.ref === context.payload.repository.default_branch
164+
) {
165+
problems.push(
166+
`This PR was opened from your fork's \`${head.ref}\` ` +
167+
`branch. Please create a dedicated feature branch instead ` +
168+
`(e.g. \`git checkout -b my-feature\`).`
169+
);
170+
}
171+
}
172+
173+
if (problems.length === 0) {
174+
core.info('Template validation passed.');
175+
return;
176+
}
177+
178+
// Build the comment
179+
const itemType = isIssue ? 'issue' : 'pull request';
180+
const lines = [
181+
`👋 Thanks for your submission!`,
182+
``,
183+
`However, it looks like the ${itemType} description does not ` +
184+
`fully follow the expected template:`,
185+
``,
186+
...problems.map(p => `- ${p}`),
187+
``,
188+
`Please update the description to address the above and ` +
189+
`reopen the ${itemType}.`,
190+
];
191+
const message = lines.join('\n');
192+
193+
// Apply a label for easier triage
194+
const labelName = 'invalid';
195+
try {
196+
await github.rest.issues.addLabels({
197+
owner: context.repo.owner,
198+
repo: context.repo.repo,
199+
issue_number: itemNumber,
200+
labels: [labelName],
201+
});
202+
} catch (e) {
203+
core.warning(`Could not add label "${labelName}": ${e.message}`);
204+
}
205+
206+
// Avoid duplicate bot comments on re-edits
207+
const comments = await github.rest.issues.listComments({
208+
owner: context.repo.owner,
209+
repo: context.repo.repo,
210+
issue_number: itemNumber,
211+
});
212+
const botLogin = 'github-actions[bot]';
213+
const existing = comments.data.find(
214+
c => c.user.login === botLogin
215+
&& c.body.includes('does not fully follow the expected template')
216+
);
217+
218+
if (existing) {
219+
await github.rest.issues.updateComment({
220+
owner: context.repo.owner,
221+
repo: context.repo.repo,
222+
comment_id: existing.id,
223+
body: message,
224+
});
225+
} else {
226+
await github.rest.issues.createComment({
227+
owner: context.repo.owner,
228+
repo: context.repo.repo,
229+
issue_number: itemNumber,
230+
body: message,
231+
});
232+
}
233+
234+
// Close the issue/PR
235+
if (isIssue && item.state === 'open') {
236+
await github.rest.issues.update({
237+
owner: context.repo.owner,
238+
repo: context.repo.repo,
239+
issue_number: itemNumber,
240+
state: 'closed',
241+
state_reason: 'not_planned',
242+
});
243+
} else if (isPR && item.state === 'open') {
244+
await github.rest.pulls.update({
245+
owner: context.repo.owner,
246+
repo: context.repo.repo,
247+
pull_number: itemNumber,
248+
state: 'closed',
249+
});
250+
}

‎CHANGES/12163.contrib.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Added a GitHub Actions workflow to validate that issues and PRs follow the expected templates, closing non-compliant submissions with an explanatory comment -- by :user:`rodrigobnogueira`.

0 commit comments

Comments
 (0)