Skip to content

Commit b19e80d

Browse files
trasktrask
committed
Replace test-* labels with /test comment commands
Adds .github/workflows/test-on-comment.yml that handles three slash-commands on PR comments from OWNER/MEMBER/COLLABORATOR users: /test native -- force-run native tests /test openj9 -- run the openj9 test variants /test windows -- run the windows smoke tests These replace the previous label-driven design (test native / test openj9 / test windows labels). Labels were removed because adding any label fires a pull_request labeled event, which entered the build's per-PR concurrency group BEFORE the job-level if: filter ran, cancelling the in-progress real build whenever any non-test label was added by code-review-sweep, dependabot, the labeler, or a human (see PR #18441). Even after a conditional concurrency-group fix, the labeled run's check_suite shadowed the in-progress real build's check_suite in the PR Checks UI, since GitHub keys that view by workflow file with the latest check_suite winning. The new design: * build-pull-request.yml only triggers on opened / synchronize / reopened. Its concurrency group is the simple per-PR one with cancel-in-progress, restoring the pre-bug push semantics for the regular event flow. * The native-tests path filter formerly in .github/labeler.yml is moved inline as a resolve-native job that diffs the PR head against the base, eliminating the labeler-vs-build race entirely. .github/labeler.yml and .github/workflows/label.yml are deleted (the labeler rule was the only content in those files). * build-pull-request.yml additionally accepts workflow_dispatch with explicit run-{native,openj9,windows-smoke}-tests inputs; test-on-comment.yml validates the commenter's association and dispatches it against the PR's head branch, so the resulting check_runs surface in the PR Checks tab and supersede prior runs in the same per-PR concurrency group. Limitations: * /test commands aren't supported for PRs from forks (workflow_dispatch can only target refs in the base repo). The comment workflow detects this and posts a brief explanation back. * Native tests now run for any PR whose paths match the historical glob, regardless of whether a test native label was applied. This was already the intended behavior; the label was just an artifact of how the labeler implemented the path filter.
1 parent f0833f0 commit b19e80d

4 files changed

Lines changed: 219 additions & 49 deletions

File tree

‎.github/labeler.yml‎

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎.github/workflows/build-pull-request.yml‎

Lines changed: 67 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -6,32 +6,81 @@ on:
66
- opened
77
- synchronize
88
- reopened
9-
- labeled
9+
# Triggered by `.github/workflows/test-on-comment.yml` to enable optional
10+
# tests (openj9, windows, or to force-run native) on a specific PR. The
11+
# comment workflow validates the commenter's association before dispatching.
12+
workflow_dispatch:
13+
inputs:
14+
pr-number:
15+
description: "PR number to build"
16+
type: string
17+
required: true
18+
run-native-tests:
19+
description: "Force-enable native tests"
20+
type: boolean
21+
default: false
22+
run-openj9-tests:
23+
description: "Enable openj9 tests"
24+
type: boolean
25+
default: false
26+
run-windows-smoke-tests:
27+
description: "Enable windows smoke tests"
28+
type: boolean
29+
default: false
1030

1131
concurrency:
12-
group: build-pull-request-${{ github.event.pull_request.number }}
32+
# Per-PR group across both event types so a new push or a new
33+
# workflow_dispatch from a /test command supersedes any in-progress build.
34+
group: build-pull-request-${{ github.event.pull_request.number || inputs.pr-number }}
1335
cancel-in-progress: true
1436

1537
permissions:
1638
contents: read
1739

1840
jobs:
41+
# Detect whether native tests should run based on the PR's changed file
42+
# paths (formerly handled by `.github/labeler.yml`'s `test native` rule).
43+
# Runs for both pull_request and workflow_dispatch events so that a
44+
# `/test openj9` (or similar) dispatched build still auto-runs native
45+
# tests if the PR's diff warrants them.
46+
resolve-native:
47+
runs-on: ubuntu-latest
48+
outputs:
49+
run-native-tests: ${{ steps.filter.outputs.native }}
50+
steps:
51+
- name: Detect native-relevant changes
52+
id: filter
53+
env:
54+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
55+
PR_NUMBER: ${{ github.event.pull_request.number || inputs.pr-number }}
56+
run: |
57+
files=$(
58+
gh api --paginate "/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files" \
59+
--jq '.[].filename'
60+
)
61+
include='^instrumentation/logback/logback-appender-1\.0/library/'
62+
include+='|^instrumentation/jdbc/library/'
63+
include+='|^instrumentation/spring/'
64+
include+='|^smoke-tests-otel-starter/'
65+
include+='|^dependencyManagement/build\.gradle\.kts$'
66+
include+='|^settings\.gradle\.kts$'
67+
exclude='^instrumentation/spring/.+/javaagent/'
68+
if grep -Eq "$include" <<<"$files" && ! grep -Eq "$exclude" <<<"$files"; then
69+
echo "native=true" >> "$GITHUB_OUTPUT"
70+
else
71+
echo "native=false" >> "$GITHUB_OUTPUT"
72+
fi
73+
1974
build:
20-
# On `labeled` events, only proceed when the added label affects what we
21-
# build. All other event types (opened, synchronize, reopened) always
22-
# proceed.
23-
if: |
24-
github.event.action != 'labeled' ||
25-
github.event.label.name == 'test native' ||
26-
github.event.label.name == 'test openj9' ||
27-
github.event.label.name == 'test windows'
75+
needs: [resolve-native]
2876
uses: ./.github/workflows/reusable-pr-build.yml
2977
with:
30-
# it's rare for only the openj9 tests, openj9 smoke variants, the windows
31-
# smoke tests, or the native tests to break, so they are gated by labels.
32-
# `test native` is applied automatically by .github/labeler.yml when the
33-
# PR diff touches native-relevant paths; `test openj9` and `test windows`
34-
# are applied manually.
35-
skip-native-tests: ${{ !contains(github.event.pull_request.labels.*.name, 'test native') }}
36-
skip-openj9-tests: ${{ !contains(github.event.pull_request.labels.*.name, 'test openj9') }}
37-
skip-windows-smoke-tests: ${{ !contains(github.event.pull_request.labels.*.name, 'test windows') }}
78+
# Native tests run when either:
79+
# * a /test native comment dispatched this build (run-native-tests=true), or
80+
# * the changed paths match the legacy `test native` glob (resolve-native).
81+
skip-native-tests: ${{ !(inputs.run-native-tests || needs.resolve-native.outputs.run-native-tests == 'true') }}
82+
# openj9 / windows tests are opt-in via /test openj9 or /test windows
83+
# comments handled by `.github/workflows/test-on-comment.yml`.
84+
skip-openj9-tests: ${{ !inputs.run-openj9-tests }}
85+
skip-windows-smoke-tests: ${{ !inputs.run-windows-smoke-tests }}
86+

‎.github/workflows/label.yml‎

Lines changed: 0 additions & 20 deletions
This file was deleted.

‎.github/workflows/test-on-comment.yml‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
name: Test on comment
2+
3+
# Lets a maintainer enable optional tests on a PR by commenting one of:
4+
#
5+
# /test native -- force-run native tests (also auto-detected from paths)
6+
# /test openj9 -- run the openj9 test variants
7+
# /test windows -- run the windows smoke tests
8+
#
9+
# Multiple words can be combined, e.g. `/test openj9 windows`.
10+
#
11+
# Replaces the previous label-driven design (`test native` / `test openj9` /
12+
# `test windows` labels). Labels were removed because adding a label fires a
13+
# `pull_request labeled` event, which entered the build's per-PR concurrency
14+
# group BEFORE the job-level `if:` filter ran, cancelling the in-progress
15+
# real build whenever any non-test label was added (see PR #18441).
16+
#
17+
# Comment commands sidestep that entirely: this workflow lives in its own
18+
# `issue_comment` event scope and dispatches `build-pull-request.yml` via
19+
# `workflow_dispatch` against the PR's head branch, so its check_runs
20+
# surface in the PR Checks tab and supersede prior runs in the same per-PR
21+
# concurrency group (just like a fresh push would).
22+
23+
on:
24+
issue_comment:
25+
types: [created]
26+
27+
permissions:
28+
contents: read
29+
30+
jobs:
31+
dispatch:
32+
# Only run on PR comments containing a /test command from a maintainer.
33+
if: |
34+
github.event.issue.pull_request != null &&
35+
startsWith(github.event.comment.body, '/test ') &&
36+
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association)
37+
runs-on: ubuntu-latest
38+
permissions:
39+
actions: write # to dispatch build-pull-request.yml
40+
pull-requests: write # to react to the comment
41+
steps:
42+
- name: Acknowledge comment with eyes reaction
43+
env:
44+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
45+
run: |
46+
gh api -X POST -H "Accept: application/vnd.github+json" \
47+
"/repos/${GITHUB_REPOSITORY}/issues/comments/${{ github.event.comment.id }}/reactions" \
48+
-f content=eyes
49+
50+
- name: Parse /test command
51+
id: parse
52+
env:
53+
BODY: ${{ github.event.comment.body }}
54+
run: |
55+
set -euo pipefail
56+
# Drop the leading "/test " then split remaining words.
57+
rest="${BODY#'/test '}"
58+
# Keep only the first line; ignore anything after a newline.
59+
rest="${rest%%$'\n'*}"
60+
run_native=false
61+
run_openj9=false
62+
run_windows=false
63+
for word in $rest; do
64+
case "$word" in
65+
native) run_native=true ;;
66+
openj9) run_openj9=true ;;
67+
windows) run_windows=true ;;
68+
esac
69+
done
70+
if ! $run_native && ! $run_openj9 && ! $run_windows; then
71+
echo "Refusing to dispatch: no recognized test variants in /test command." >&2
72+
echo "valid=false" >> "$GITHUB_OUTPUT"
73+
else
74+
echo "valid=true" >> "$GITHUB_OUTPUT"
75+
echo "run-native=$run_native" >> "$GITHUB_OUTPUT"
76+
echo "run-openj9=$run_openj9" >> "$GITHUB_OUTPUT"
77+
echo "run-windows=$run_windows" >> "$GITHUB_OUTPUT"
78+
fi
79+
80+
- name: Resolve PR head ref
81+
if: steps.parse.outputs.valid == 'true'
82+
id: pr
83+
env:
84+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
85+
run: |
86+
set -euo pipefail
87+
# We need the head branch name for `gh workflow run --ref`
88+
# (workflow_dispatch only accepts branches/tags, not SHAs).
89+
IFS=$'\t' read -r head_ref head_repo < <(
90+
gh api "/repos/${GITHUB_REPOSITORY}/pulls/${{ github.event.issue.number }}" \
91+
--jq '[.head.ref, .head.repo.full_name] | @tsv'
92+
)
93+
if [[ "$head_repo" != "$GITHUB_REPOSITORY" ]]; then
94+
echo "Refusing to dispatch: PR is from a fork ($head_repo); /test commands are not supported for fork PRs." >&2
95+
echo "supported=false" >> "$GITHUB_OUTPUT"
96+
else
97+
echo "supported=true" >> "$GITHUB_OUTPUT"
98+
echo "head-ref=$head_ref" >> "$GITHUB_OUTPUT"
99+
fi
100+
101+
- name: Dispatch build-pull-request.yml
102+
if: steps.parse.outputs.valid == 'true' && steps.pr.outputs.supported == 'true'
103+
env:
104+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
105+
# Pass templated values through env so attacker-controlled branch
106+
# names (e.g. containing $, `, ") cannot escape the shell command.
107+
HEAD_REF: ${{ steps.pr.outputs.head-ref }}
108+
PR_NUMBER: ${{ github.event.issue.number }}
109+
RUN_NATIVE: ${{ steps.parse.outputs.run-native }}
110+
RUN_OPENJ9: ${{ steps.parse.outputs.run-openj9 }}
111+
RUN_WINDOWS: ${{ steps.parse.outputs.run-windows }}
112+
run: |
113+
set -euo pipefail
114+
# Dispatch against the PR's head branch so the resulting check_runs
115+
# attach to the PR's HEAD commit and surface in the PR's Checks
116+
# tab. The workflow file used is the one at that ref; this is the
117+
# same trust model as the regular `pull_request` trigger.
118+
gh workflow run build-pull-request.yml \
119+
--ref "$HEAD_REF" \
120+
-f pr-number="$PR_NUMBER" \
121+
-f run-native-tests="$RUN_NATIVE" \
122+
-f run-openj9-tests="$RUN_OPENJ9" \
123+
-f run-windows-smoke-tests="$RUN_WINDOWS"
124+
125+
- name: React with rocket on success
126+
if: steps.parse.outputs.valid == 'true' && steps.pr.outputs.supported == 'true' && success()
127+
env:
128+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
129+
run: |
130+
gh api -X POST -H "Accept: application/vnd.github+json" \
131+
"/repos/${GITHUB_REPOSITORY}/issues/comments/${{ github.event.comment.id }}/reactions" \
132+
-f content=rocket
133+
134+
- name: Comment back when PR is from a fork
135+
if: steps.parse.outputs.valid == 'true' && steps.pr.outputs.supported == 'false'
136+
env:
137+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
138+
BODY: |
139+
@${{ github.event.comment.user.login }} `/test` commands aren't supported for PRs from forks.
140+
141+
A maintainer can pull the branch into this repo and re-run the command, or push an empty commit to retrigger the build with the requested tests.
142+
run: |
143+
gh pr comment "${{ github.event.issue.number }}" --body "$BODY"
144+
145+
- name: React with confused on parse failure
146+
if: steps.parse.outputs.valid != 'true'
147+
env:
148+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
149+
run: |
150+
gh api -X POST -H "Accept: application/vnd.github+json" \
151+
"/repos/${GITHUB_REPOSITORY}/issues/comments/${{ github.event.comment.id }}/reactions" \
152+
-f content=confused

0 commit comments

Comments
 (0)