Skip to content

Commit b64786a

Browse files
trasktrask
committed
Replace test-* labels with /test comment commands
Adds .github/workflows/test-on-comment.yml that handles three slash-commands on PR comments from OWNER/MEMBER/COLLABORATOR users: /test native -- force-run native tests /test openj9 -- run the openj9 test variants /test windows -- run the windows smoke tests These replace the previous label-driven design (test native / test openj9 / test windows labels). Labels were removed because adding any label fires a pull_request labeled event, which entered the build's per-PR concurrency group BEFORE the job-level if: filter ran, cancelling the in-progress real build whenever any non-test label was added by code-review-sweep, dependabot, the labeler, or a human (see PR #18441). Even after a conditional concurrency-group fix, the labeled run's check_suite shadowed the in-progress real build's check_suite in the PR Checks UI, since GitHub keys that view by workflow file with the latest check_suite winning. The new design: * build-pull-request.yml only triggers on opened / synchronize / reopened. Its concurrency group is the simple per-PR one with cancel-in-progress, restoring the pre-bug push semantics for the regular event flow. * The native-tests path filter formerly in .github/labeler.yml is moved inline as a resolve-native job that diffs the PR head against the base, eliminating the labeler-vs-build race entirely. .github/labeler.yml and .github/workflows/label.yml are deleted (the labeler rule was the only content in those files). * build-pull-request.yml additionally accepts workflow_dispatch with explicit run-{native,openj9,windows-smoke}-tests inputs; test-on-comment.yml validates the commenter's association and dispatches it against the PR's head branch, so the resulting check_runs surface in the PR Checks tab and supersede prior runs in the same per-PR concurrency group. Limitations: * /test commands aren't supported for PRs from forks (workflow_dispatch can only target refs in the base repo). The comment workflow detects this and posts a brief explanation back. * Native tests now run for any PR whose paths match the historical glob, regardless of whether a test native label was applied. This was already the intended behavior; the label was just an artifact of how the labeler implemented the path filter.
1 parent f0833f0 commit b64786a

4 files changed

Lines changed: 251 additions & 49 deletions

File tree

‎.github/labeler.yml‎

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎.github/workflows/build-pull-request.yml‎

Lines changed: 99 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -6,32 +6,113 @@ on:
66
- opened
77
- synchronize
88
- reopened
9-
- labeled
9+
# Triggered by `.github/workflows/test-on-comment.yml` to enable optional
10+
# tests (openj9, windows, or to force-run native) on a specific PR. The
11+
# comment workflow validates the commenter's association before
12+
# dispatching.
13+
workflow_dispatch:
14+
inputs:
15+
pr-number:
16+
description: "PR number to build"
17+
type: string
18+
required: true
19+
head-sha:
20+
description: "PR head SHA to build"
21+
type: string
22+
required: true
23+
run-native-tests:
24+
description: "Force-enable native tests"
25+
type: boolean
26+
default: false
27+
run-openj9-tests:
28+
description: "Enable openj9 tests"
29+
type: boolean
30+
default: false
31+
run-windows-smoke-tests:
32+
description: "Enable windows smoke tests"
33+
type: boolean
34+
default: false
1035

1136
concurrency:
12-
group: build-pull-request-${{ github.event.pull_request.number }}
37+
# Per-PR group across both event types so a new push or a new
38+
# workflow_dispatch from a /test command supersedes any in-progress build.
39+
group: build-pull-request-${{ github.event.pull_request.number || inputs.pr-number }}
1340
cancel-in-progress: true
1441

1542
permissions:
1643
contents: read
1744

1845
jobs:
46+
# Detect whether native tests should run based on the PR's changed file
47+
# paths (formerly handled by `.github/labeler.yml`'s `test native` rule).
48+
# Runs for both pull_request and workflow_dispatch events so that a
49+
# `/test openj9` (or similar) dispatched build still auto-runs native
50+
# tests if the PR's diff warrants them.
51+
resolve-native:
52+
runs-on: ubuntu-latest
53+
outputs:
54+
run-native-tests: ${{ steps.filter.outputs.native }}
55+
steps:
56+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
57+
with:
58+
fetch-depth: 0
59+
# On workflow_dispatch we explicitly check out the PR's HEAD SHA
60+
# so the diff matches what the dispatcher told us to build.
61+
ref: ${{ github.event_name == 'workflow_dispatch' && inputs.head-sha || github.event.pull_request.head.sha }}
62+
- name: Resolve base SHA
63+
id: base
64+
env:
65+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
66+
run: |
67+
set -euo pipefail
68+
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
69+
echo "sha=${{ github.event.pull_request.base.sha }}" >> "$GITHUB_OUTPUT"
70+
else
71+
base_sha=$(gh api "/repos/${GITHUB_REPOSITORY}/pulls/${{ inputs.pr-number }}" --jq '.base.sha')
72+
echo "sha=$base_sha" >> "$GITHUB_OUTPUT"
73+
fi
74+
- name: Detect native-relevant changes
75+
id: filter
76+
env:
77+
BASE_SHA: ${{ steps.base.outputs.sha }}
78+
HEAD_SHA: ${{ github.event_name == 'workflow_dispatch' && inputs.head-sha || github.event.pull_request.head.sha }}
79+
run: |
80+
set -euo pipefail
81+
# Mirrors the former `.github/labeler.yml` `test native` rule:
82+
# match if any changed file is in the positive set AND no changed
83+
# file is under any `instrumentation/spring/**/javaagent/**` path.
84+
mapfile -t changed < <(git diff --name-only "$BASE_SHA" "$HEAD_SHA")
85+
has_positive=false
86+
has_javaagent_spring=false
87+
for f in "${changed[@]}"; do
88+
if [[ "$f" == instrumentation/spring/* && "$f" == */javaagent/* ]]; then
89+
has_javaagent_spring=true
90+
fi
91+
if [[ "$f" == instrumentation/logback/logback-appender-1.0/library/* ]] || \
92+
[[ "$f" == instrumentation/jdbc/library/* ]] || \
93+
[[ "$f" == instrumentation/spring/* ]] || \
94+
[[ "$f" == smoke-tests-otel-starter/* ]] || \
95+
[[ "$f" == dependencyManagement/build.gradle.kts ]] || \
96+
[[ "$f" == settings.gradle.kts ]]; then
97+
has_positive=true
98+
fi
99+
done
100+
if $has_positive && ! $has_javaagent_spring; then
101+
echo "native=true" >> "$GITHUB_OUTPUT"
102+
else
103+
echo "native=false" >> "$GITHUB_OUTPUT"
104+
fi
105+
19106
build:
20-
# On `labeled` events, only proceed when the added label affects what we
21-
# build. All other event types (opened, synchronize, reopened) always
22-
# proceed.
23-
if: |
24-
github.event.action != 'labeled' ||
25-
github.event.label.name == 'test native' ||
26-
github.event.label.name == 'test openj9' ||
27-
github.event.label.name == 'test windows'
107+
needs: [resolve-native]
28108
uses: ./.github/workflows/reusable-pr-build.yml
29109
with:
30-
# it's rare for only the openj9 tests, openj9 smoke variants, the windows
31-
# smoke tests, or the native tests to break, so they are gated by labels.
32-
# `test native` is applied automatically by .github/labeler.yml when the
33-
# PR diff touches native-relevant paths; `test openj9` and `test windows`
34-
# are applied manually.
35-
skip-native-tests: ${{ !contains(github.event.pull_request.labels.*.name, 'test native') }}
36-
skip-openj9-tests: ${{ !contains(github.event.pull_request.labels.*.name, 'test openj9') }}
37-
skip-windows-smoke-tests: ${{ !contains(github.event.pull_request.labels.*.name, 'test windows') }}
110+
# Native tests run when either:
111+
# * a /test native comment dispatched this build (run-native-tests=true), or
112+
# * the changed paths match the legacy `test native` glob (resolve-native).
113+
skip-native-tests: ${{ !(inputs.run-native-tests || needs.resolve-native.outputs.run-native-tests == 'true') }}
114+
# openj9 / windows tests are opt-in via /test openj9 or /test windows
115+
# comments handled by `.github/workflows/test-on-comment.yml`.
116+
skip-openj9-tests: ${{ !inputs.run-openj9-tests }}
117+
skip-windows-smoke-tests: ${{ !inputs.run-windows-smoke-tests }}
118+

‎.github/workflows/label.yml‎

Lines changed: 0 additions & 20 deletions
This file was deleted.

‎.github/workflows/test-on-comment.yml‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
name: Test on comment
2+
3+
# Lets a maintainer enable optional tests on a PR by commenting one of:
4+
#
5+
# /test native -- force-run native tests (also auto-detected from paths)
6+
# /test openj9 -- run the openj9 test variants
7+
# /test windows -- run the windows smoke tests
8+
#
9+
# Multiple words can be combined, e.g. `/test openj9 windows`.
10+
#
11+
# Replaces the previous label-driven design (`test native` / `test openj9` /
12+
# `test windows` labels). Labels were removed because adding a label fires a
13+
# `pull_request labeled` event, which entered the build's per-PR concurrency
14+
# group BEFORE the job-level `if:` filter ran, cancelling the in-progress
15+
# real build whenever any non-test label was added (see PR #18441).
16+
#
17+
# Comment commands sidestep that entirely: this workflow lives in its own
18+
# `issue_comment` event scope and dispatches `build-pull-request.yml` via
19+
# `workflow_dispatch` against the PR's HEAD SHA, so its check_runs surface
20+
# in the PR Checks tab and supersede prior runs in the same per-PR
21+
# concurrency group (just like a fresh push would).
22+
23+
on:
24+
issue_comment:
25+
types: [created]
26+
27+
permissions:
28+
contents: read
29+
30+
jobs:
31+
dispatch:
32+
# Only run on PR comments containing a /test command from a maintainer.
33+
if: |
34+
github.event.issue.pull_request != null &&
35+
startsWith(github.event.comment.body, '/test ') &&
36+
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association)
37+
runs-on: ubuntu-latest
38+
permissions:
39+
actions: write # to dispatch build-pull-request.yml
40+
pull-requests: write # to react to the comment
41+
steps:
42+
- name: Acknowledge comment with eyes reaction
43+
env:
44+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
45+
run: |
46+
gh api -X POST -H "Accept: application/vnd.github+json" \
47+
"/repos/${GITHUB_REPOSITORY}/issues/comments/${{ github.event.comment.id }}/reactions" \
48+
-f content=eyes
49+
50+
- name: Parse /test command
51+
id: parse
52+
env:
53+
BODY: ${{ github.event.comment.body }}
54+
run: |
55+
set -euo pipefail
56+
# Drop the leading "/test " then split remaining words.
57+
rest="${BODY#'/test '}"
58+
# Keep only the first line; ignore anything after a newline.
59+
rest="${rest%%$'\n'*}"
60+
run_native=false
61+
run_openj9=false
62+
run_windows=false
63+
unknown=()
64+
for word in $rest; do
65+
case "$word" in
66+
native) run_native=true ;;
67+
openj9) run_openj9=true ;;
68+
windows) run_windows=true ;;
69+
*) unknown+=("$word") ;;
70+
esac
71+
done
72+
if (( ${#unknown[@]} > 0 )); then
73+
printf 'unknown=%s\n' "${unknown[*]}" >> "$GITHUB_OUTPUT"
74+
fi
75+
if ! $run_native && ! $run_openj9 && ! $run_windows; then
76+
echo "Refusing to dispatch: no recognized test variants in /test command." >&2
77+
echo "valid=false" >> "$GITHUB_OUTPUT"
78+
else
79+
echo "valid=true" >> "$GITHUB_OUTPUT"
80+
echo "run-native=$run_native" >> "$GITHUB_OUTPUT"
81+
echo "run-openj9=$run_openj9" >> "$GITHUB_OUTPUT"
82+
echo "run-windows=$run_windows" >> "$GITHUB_OUTPUT"
83+
fi
84+
85+
- name: Resolve PR head ref
86+
if: steps.parse.outputs.valid == 'true'
87+
id: pr
88+
env:
89+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
90+
run: |
91+
set -euo pipefail
92+
# We need the head branch name for `gh workflow run --ref`
93+
# (workflow_dispatch only accepts branches/tags, not SHAs).
94+
pr_json=$(gh api "/repos/${GITHUB_REPOSITORY}/pulls/${{ github.event.issue.number }}")
95+
head_ref=$(echo "$pr_json" | jq -r '.head.ref')
96+
head_sha=$(echo "$pr_json" | jq -r '.head.sha')
97+
head_repo=$(echo "$pr_json" | jq -r '.head.repo.full_name')
98+
if [[ "$head_repo" != "$GITHUB_REPOSITORY" ]]; then
99+
echo "Refusing to dispatch: PR is from a fork ($head_repo); /test commands are not supported for fork PRs." >&2
100+
echo "supported=false" >> "$GITHUB_OUTPUT"
101+
else
102+
echo "supported=true" >> "$GITHUB_OUTPUT"
103+
echo "head-ref=$head_ref" >> "$GITHUB_OUTPUT"
104+
echo "head-sha=$head_sha" >> "$GITHUB_OUTPUT"
105+
fi
106+
107+
- name: Dispatch build-pull-request.yml
108+
if: steps.parse.outputs.valid == 'true' && steps.pr.outputs.supported == 'true'
109+
env:
110+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
111+
run: |
112+
set -euo pipefail
113+
# Dispatch against the PR's head branch so the resulting check_runs
114+
# attach to the PR's HEAD commit and surface in the PR's Checks
115+
# tab. The workflow file used is the one at that ref; this is the
116+
# same trust model as the regular `pull_request` trigger.
117+
gh workflow run build-pull-request.yml \
118+
--ref "${{ steps.pr.outputs.head-ref }}" \
119+
-f pr-number="${{ github.event.issue.number }}" \
120+
-f head-sha="${{ steps.pr.outputs.head-sha }}" \
121+
-f run-native-tests="${{ steps.parse.outputs.run-native }}" \
122+
-f run-openj9-tests="${{ steps.parse.outputs.run-openj9 }}" \
123+
-f run-windows-smoke-tests="${{ steps.parse.outputs.run-windows }}"
124+
125+
- name: React with rocket on success
126+
if: steps.parse.outputs.valid == 'true' && steps.pr.outputs.supported == 'true' && success()
127+
env:
128+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
129+
run: |
130+
gh api -X POST -H "Accept: application/vnd.github+json" \
131+
"/repos/${GITHUB_REPOSITORY}/issues/comments/${{ github.event.comment.id }}/reactions" \
132+
-f content=rocket
133+
134+
- name: Comment back when PR is from a fork
135+
if: steps.parse.outputs.valid == 'true' && steps.pr.outputs.supported == 'false'
136+
env:
137+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
138+
run: |
139+
gh pr comment "${{ github.event.issue.number }}" \
140+
--body "$(printf '%s\n' \
141+
"@${{ github.event.comment.user.login }} \`/test\` commands aren't supported for PRs from forks." \
142+
"" \
143+
"A maintainer can pull the branch into this repo and re-run the command, or push an empty commit to retrigger the build with the requested tests.")"
144+
145+
- name: React with confused on parse failure
146+
if: steps.parse.outputs.valid != 'true'
147+
env:
148+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
149+
run: |
150+
gh api -X POST -H "Accept: application/vnd.github+json" \
151+
"/repos/${GITHUB_REPOSITORY}/issues/comments/${{ github.event.comment.id }}/reactions" \
152+
-f content=confused

0 commit comments

Comments
 (0)