Skip to content

Commit 7c7f79d

Browse files
trasktrask
committed
Replace test-* labels with /test comment commands
Adds .github/workflows/test-on-comment.yml that handles three slash-commands on PR comments from OWNER/MEMBER/COLLABORATOR users: /test native -- force-run native tests /test openj9 -- run the openj9 test variants /test windows -- run the windows smoke tests These replace the previous label-driven design (test native / test openj9 / test windows labels). Labels were removed because adding any label fires a pull_request labeled event, which entered the build's per-PR concurrency group BEFORE the job-level if: filter ran, cancelling the in-progress real build whenever any non-test label was added by code-review-sweep, dependabot, the labeler, or a human (see PR open-telemetry#18441). Even after a conditional concurrency-group fix, the labeled run's check_suite shadowed the in-progress real build's check_suite in the PR Checks UI, since GitHub keys that view by workflow file with the latest check_suite winning. The new design: * build-pull-request.yml only triggers on opened / synchronize / reopened. Its concurrency group is the simple per-PR one with cancel-in-progress, restoring the pre-bug push semantics for the regular event flow. * The native-tests path filter formerly in .github/labeler.yml is moved inline as a resolve-native job that diffs the PR head against the base, eliminating the labeler-vs-build race entirely. .github/labeler.yml and .github/workflows/label.yml are deleted (the labeler rule was the only content in those files). * build-pull-request.yml additionally accepts workflow_dispatch with explicit run-{native,openj9,windows-smoke}-tests inputs; test-on-comment.yml validates the commenter's association and dispatches it against the PR's head branch, so the resulting check_runs surface in the PR Checks tab and supersede prior runs in the same per-PR concurrency group. Limitations: * /test commands aren't supported for PRs from forks (workflow_dispatch can only target refs in the base repo). The comment workflow detects this and posts a brief explanation back. * Native tests now run for any PR whose paths match the historical glob, regardless of whether a test native label was applied. This was already the intended behavior; the label was just an artifact of how the labeler implemented the path filter.
1 parent f0833f0 commit 7c7f79d

4 files changed

Lines changed: 212 additions & 49 deletions

File tree

‎.github/labeler.yml‎

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎.github/workflows/build-pull-request.yml‎

Lines changed: 67 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -6,32 +6,81 @@ on:
66
- opened
77
- synchronize
88
- reopened
9-
- labeled
9+
# Triggered by `.github/workflows/test-on-comment.yml` to enable optional
10+
# tests (openj9, windows, or to force-run native) on a specific PR. The
11+
# comment workflow validates the commenter's association before
12+
# dispatching.
13+
workflow_dispatch:
14+
inputs:
15+
pr-number:
16+
description: "PR number to build"
17+
type: string
18+
required: true
19+
run-native-tests:
20+
description: "Force-enable native tests"
21+
type: boolean
22+
default: false
23+
run-openj9-tests:
24+
description: "Enable openj9 tests"
25+
type: boolean
26+
default: false
27+
run-windows-smoke-tests:
28+
description: "Enable windows smoke tests"
29+
type: boolean
30+
default: false
1031

1132
concurrency:
12-
group: build-pull-request-${{ github.event.pull_request.number }}
33+
# Per-PR group across both event types so a new push or a new
34+
# workflow_dispatch from a /test command supersedes any in-progress build.
35+
group: build-pull-request-${{ github.event.pull_request.number || inputs.pr-number }}
1336
cancel-in-progress: true
1437

1538
permissions:
1639
contents: read
1740

1841
jobs:
42+
# Detect whether native tests should run based on the PR's changed file
43+
# paths (formerly handled by `.github/labeler.yml`'s `test native` rule).
44+
# Runs for both pull_request and workflow_dispatch events so that a
45+
# `/test openj9` (or similar) dispatched build still auto-runs native
46+
# tests if the PR's diff warrants them.
47+
resolve-native:
48+
runs-on: ubuntu-latest
49+
outputs:
50+
run-native-tests: ${{ steps.filter.outputs.native }}
51+
steps:
52+
- name: Detect native-relevant changes
53+
id: filter
54+
env:
55+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
56+
PR_NUMBER: ${{ github.event.pull_request.number || inputs.pr-number }}
57+
run: |
58+
set -euo pipefail
59+
# Mirrors the former `.github/labeler.yml` `test native` rule:
60+
# match if any changed file is in the positive set AND no changed
61+
# file is under any `instrumentation/spring/**/javaagent/**` path.
62+
files=$(
63+
gh api --paginate "/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files" \
64+
--jq '.[].filename'
65+
)
66+
positive='^(instrumentation/logback/logback-appender-1\.0/library/|instrumentation/jdbc/library/|instrumentation/spring/|smoke-tests-otel-starter/|dependencyManagement/build\.gradle\.kts$|settings\.gradle\.kts$)'
67+
exclude='^instrumentation/spring/.+/javaagent/'
68+
if grep -Eq "$positive" <<<"$files" && ! grep -Eq "$exclude" <<<"$files"; then
69+
echo "native=true" >> "$GITHUB_OUTPUT"
70+
else
71+
echo "native=false" >> "$GITHUB_OUTPUT"
72+
fi
73+
1974
build:
20-
# On `labeled` events, only proceed when the added label affects what we
21-
# build. All other event types (opened, synchronize, reopened) always
22-
# proceed.
23-
if: |
24-
github.event.action != 'labeled' ||
25-
github.event.label.name == 'test native' ||
26-
github.event.label.name == 'test openj9' ||
27-
github.event.label.name == 'test windows'
75+
needs: [resolve-native]
2876
uses: ./.github/workflows/reusable-pr-build.yml
2977
with:
30-
# it's rare for only the openj9 tests, openj9 smoke variants, the windows
31-
# smoke tests, or the native tests to break, so they are gated by labels.
32-
# `test native` is applied automatically by .github/labeler.yml when the
33-
# PR diff touches native-relevant paths; `test openj9` and `test windows`
34-
# are applied manually.
35-
skip-native-tests: ${{ !contains(github.event.pull_request.labels.*.name, 'test native') }}
36-
skip-openj9-tests: ${{ !contains(github.event.pull_request.labels.*.name, 'test openj9') }}
37-
skip-windows-smoke-tests: ${{ !contains(github.event.pull_request.labels.*.name, 'test windows') }}
78+
# Native tests run when either:
79+
# * a /test native comment dispatched this build (run-native-tests=true), or
80+
# * the changed paths match the legacy `test native` glob (resolve-native).
81+
skip-native-tests: ${{ !(inputs.run-native-tests || needs.resolve-native.outputs.run-native-tests == 'true') }}
82+
# openj9 / windows tests are opt-in via /test openj9 or /test windows
83+
# comments handled by `.github/workflows/test-on-comment.yml`.
84+
skip-openj9-tests: ${{ !inputs.run-openj9-tests }}
85+
skip-windows-smoke-tests: ${{ !inputs.run-windows-smoke-tests }}
86+

‎.github/workflows/label.yml‎

Lines changed: 0 additions & 20 deletions
This file was deleted.

‎.github/workflows/test-on-comment.yml‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
name: Test on comment
2+
3+
# Lets a maintainer enable optional tests on a PR by commenting one of:
4+
#
5+
# /test native -- force-run native tests (also auto-detected from paths)
6+
# /test openj9 -- run the openj9 test variants
7+
# /test windows -- run the windows smoke tests
8+
#
9+
# Multiple words can be combined, e.g. `/test openj9 windows`.
10+
#
11+
# Replaces the previous label-driven design (`test native` / `test openj9` /
12+
# `test windows` labels). Labels were removed because adding a label fires a
13+
# `pull_request labeled` event, which entered the build's per-PR concurrency
14+
# group BEFORE the job-level `if:` filter ran, cancelling the in-progress
15+
# real build whenever any non-test label was added (see PR #18441).
16+
#
17+
# Comment commands sidestep that entirely: this workflow lives in its own
18+
# `issue_comment` event scope and dispatches `build-pull-request.yml` via
19+
# `workflow_dispatch` against the PR's HEAD SHA, so its check_runs surface
20+
# in the PR Checks tab and supersede prior runs in the same per-PR
21+
# concurrency group (just like a fresh push would).
22+
23+
on:
24+
issue_comment:
25+
types: [created]
26+
27+
permissions:
28+
contents: read
29+
30+
jobs:
31+
dispatch:
32+
# Only run on PR comments containing a /test command from a maintainer.
33+
if: |
34+
github.event.issue.pull_request != null &&
35+
startsWith(github.event.comment.body, '/test ') &&
36+
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association)
37+
runs-on: ubuntu-latest
38+
permissions:
39+
actions: write # to dispatch build-pull-request.yml
40+
pull-requests: write # to react to the comment
41+
steps:
42+
- name: Acknowledge comment with eyes reaction
43+
env:
44+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
45+
run: |
46+
gh api -X POST -H "Accept: application/vnd.github+json" \
47+
"/repos/${GITHUB_REPOSITORY}/issues/comments/${{ github.event.comment.id }}/reactions" \
48+
-f content=eyes
49+
50+
- name: Parse /test command
51+
id: parse
52+
env:
53+
BODY: ${{ github.event.comment.body }}
54+
run: |
55+
set -euo pipefail
56+
# Drop the leading "/test " then split remaining words.
57+
rest="${BODY#'/test '}"
58+
# Keep only the first line; ignore anything after a newline.
59+
rest="${rest%%$'\n'*}"
60+
run_native=false
61+
run_openj9=false
62+
run_windows=false
63+
for word in $rest; do
64+
case "$word" in
65+
native) run_native=true ;;
66+
openj9) run_openj9=true ;;
67+
windows) run_windows=true ;;
68+
esac
69+
done
70+
if ! $run_native && ! $run_openj9 && ! $run_windows; then
71+
echo "Refusing to dispatch: no recognized test variants in /test command." >&2
72+
echo "valid=false" >> "$GITHUB_OUTPUT"
73+
else
74+
echo "valid=true" >> "$GITHUB_OUTPUT"
75+
echo "run-native=$run_native" >> "$GITHUB_OUTPUT"
76+
echo "run-openj9=$run_openj9" >> "$GITHUB_OUTPUT"
77+
echo "run-windows=$run_windows" >> "$GITHUB_OUTPUT"
78+
fi
79+
80+
- name: Resolve PR head ref
81+
if: steps.parse.outputs.valid == 'true'
82+
id: pr
83+
env:
84+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
85+
run: |
86+
set -euo pipefail
87+
# We need the head branch name for `gh workflow run --ref`
88+
# (workflow_dispatch only accepts branches/tags, not SHAs).
89+
IFS=$'\t' read -r head_ref head_repo < <(
90+
gh api "/repos/${GITHUB_REPOSITORY}/pulls/${{ github.event.issue.number }}" \
91+
--jq '[.head.ref, .head.repo.full_name] | @tsv'
92+
)
93+
if [[ "$head_repo" != "$GITHUB_REPOSITORY" ]]; then
94+
echo "Refusing to dispatch: PR is from a fork ($head_repo); /test commands are not supported for fork PRs." >&2
95+
echo "supported=false" >> "$GITHUB_OUTPUT"
96+
else
97+
echo "supported=true" >> "$GITHUB_OUTPUT"
98+
echo "head-ref=$head_ref" >> "$GITHUB_OUTPUT"
99+
fi
100+
101+
- name: Dispatch build-pull-request.yml
102+
if: steps.parse.outputs.valid == 'true' && steps.pr.outputs.supported == 'true'
103+
env:
104+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
105+
run: |
106+
set -euo pipefail
107+
# Dispatch against the PR's head branch so the resulting check_runs
108+
# attach to the PR's HEAD commit and surface in the PR's Checks
109+
# tab. The workflow file used is the one at that ref; this is the
110+
# same trust model as the regular `pull_request` trigger.
111+
gh workflow run build-pull-request.yml \
112+
--ref "${{ steps.pr.outputs.head-ref }}" \
113+
-f pr-number="${{ github.event.issue.number }}" \
114+
-f run-native-tests="${{ steps.parse.outputs.run-native }}" \
115+
-f run-openj9-tests="${{ steps.parse.outputs.run-openj9 }}" \
116+
-f run-windows-smoke-tests="${{ steps.parse.outputs.run-windows }}"
117+
118+
- name: React with rocket on success
119+
if: steps.parse.outputs.valid == 'true' && steps.pr.outputs.supported == 'true' && success()
120+
env:
121+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
122+
run: |
123+
gh api -X POST -H "Accept: application/vnd.github+json" \
124+
"/repos/${GITHUB_REPOSITORY}/issues/comments/${{ github.event.comment.id }}/reactions" \
125+
-f content=rocket
126+
127+
- name: Comment back when PR is from a fork
128+
if: steps.parse.outputs.valid == 'true' && steps.pr.outputs.supported == 'false'
129+
env:
130+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
131+
BODY: |
132+
@${{ github.event.comment.user.login }} `/test` commands aren't supported for PRs from forks.
133+
134+
A maintainer can pull the branch into this repo and re-run the command, or push an empty commit to retrigger the build with the requested tests.
135+
run: |
136+
gh pr comment "${{ github.event.issue.number }}" --body "$BODY"
137+
138+
- name: React with confused on parse failure
139+
if: steps.parse.outputs.valid != 'true'
140+
env:
141+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
142+
run: |
143+
gh api -X POST -H "Accept: application/vnd.github+json" \
144+
"/repos/${GITHUB_REPOSITORY}/issues/comments/${{ github.event.comment.id }}/reactions" \
145+
-f content=confused

0 commit comments

Comments
 (0)