Stop using pull_request: labeled to gate optional tests#18446
Merged
Conversation
trask
changed the title
trask
added
test native
trask
force-pushed
the
fix-build-pr-label-concurrency
branch
from
23fd63b to
f5f7388
Compare
trask
changed the title
trask
force-pushed
the
fix-build-pr-label-concurrency
branch
from
f5f7388 to
52e5a6c
Compare
trask
changed the title
trask
added
area:tests
and removed
test native
trask
force-pushed
the
fix-build-pr-label-concurrency
branch
from
52e5a6c to
bab64fd
Compare
trask
changed the title
trask
added
area:build
trask
force-pushed
the
fix-build-pr-label-concurrency
branch
from
bab64fd to
b64786a
Compare
trask
changed the title
trask
force-pushed
the
fix-build-pr-label-concurrency
branch
2 times, most recently
from
7c7f79d to
b19e80d
Compare
trask
changed the title
pull_request: labeled to gate optional tests
trask
force-pushed
the
fix-build-pr-label-concurrency
branch
from
b19e80d to
b0b027c
Compare
The labeled trigger caused two issues that combined into recurring CI breakage on PRs that received any label (renovate, dependabot, code-review-sweep, the labeler, or a human applied one): * GitHub evaluates a workflow's `concurrency:` group before its job-level `if:` filter, so a labeled run cancelled the in-progress real build before being filtered out. * Even after a conditional concurrency-group fix, the labeled run's check_suite displaced the in-progress real build's check_suite in the PR Checks UI (GitHub keys that view by workflow file, latest check_suite wins). Concrete failure: PR open-telemetry#18441, run https://github.com/open-telemetry/opentelemetry-java-instrumentation/actions/runs/25156724872 — every job ended within ~2 seconds with no steps executed. Fix: drop `labeled` from the trigger entirely. Read labels from `github.event.pull_request.labels` (which is present on every `pull_request` event) and gate optional jobs by `contains(...)`. Skipped jobs that are listed as required satisfy branch protection, so auto-merge still works. * `test openj9` / `test windows` labels: still consulted, but only at the moments the build naturally runs (opened / synchronize / reopened). Applying a label after the last push no longer re-triggers the build; to pick up the label, close/reopen the PR or push another commit. This trade-off avoids the labeled-event race and the wasted reruns from non-test labels. * `test native` is now detected inline from the PR's changed paths via a new `resolve-native` job that mirrors the former `.github/labeler.yml` `test native` rule. The label is no longer auto-applied; the manual label is still honored as a force-enable. * `.github/labeler.yml` and `.github/workflows/label.yml` are deleted (the labeler's only rule was the `test native` one).
trask
force-pushed
the
fix-build-pr-label-concurrency
branch
from
b0b027c to
b5f15aa
Compare
zeitlinger
approved these changes
May 1, 2026
zeitlinger
left a comment
zeitlinger
left a comment
Member
There was a problem hiding this comment.
I wish that could be simpler - but I can't think of anything
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Sorry for the wall of text below, but I went down a lot of wrong paths here and ended up with this compromise, and leaving this comment for myself when I think I can do better later on.
Why
Optional test variants (
test openj9,test windows,test native) were gated by GitHub PR labels viapull_request: [labeled]. We tried three progressively more elaborate ways to make the labeled trigger work, and all of them broke:rebuild-pull-request-on-label.ymlthat re-invoked the build viaworkflow_callontest *label adds. Two workflows, two concurrency groups, full build ran twice in parallel every time the labeler auto-appliedtest native(Remove deprecated redact-query-parameters and db-sqlcommenter configurations #18229).pull_request: [labeled]plus a job-levelif:filtering totest *labels (Unify PR build into a single workflow with a dispatcher #18295). Butconcurrency:is evaluated before the job-levelif:filter, so adding any label (renovate, dependabot, code-review-sweep, the labeler, a human) entered the per-PR group and cancelled the in-progress real build (Code review sweep (run 25154254051) #18441). Concrete failure: PR Code review sweep (run 25154254051) #18441, run 25156724872 — every job ended within ~2 seconds with no steps executed. And even after a conditional concurrency group keyed off the label, the labeled run's check_suite still displaced the real build's check_suite in the PR Checks UI (GitHub keys that view by workflow file, latest check_suite wins).All three failures share a root cause:
pull_request: [labeled]was never meant to be a build-configuration trigger.What
Drop
labeledfrom the trigger entirely. Read labels fromgithub.event.pull_request.labels(present on everypull_requestevent) and gate optional jobs withcontains(...). Skipped required jobs satisfy branch protection so auto-merge still gates correctly.test nativedetection (formerly the labeler's glob) moves inline as aresolve-nativejob in the build.Trade-off
Adding a
test openj9/test windowslabel doesn't retrigger the build. Push another commit or close/reopen the PR to pick it up. A newcomment-on-test-label.ymlworkflow posts a reminder comment when one of those labels is added so this isn't surprising. (Safe underpull_request_targetbecause that workflow doesn't check out or run any PR code — onlygh pr comment.)