Skip to content

fix: proper handling of bazel test runner suite events#11322

Draft
daniel-mohedano wants to merge 1 commit intomasterfrom
daniel.mohedano/bazel-run-notifier-instrumentation
Draft

fix: proper handling of bazel test runner suite events#11322
daniel-mohedano wants to merge 1 commit intomasterfrom
daniel.mohedano/bazel-run-notifier-instrumentation

Conversation

@daniel-mohedano
Copy link
Copy Markdown
Contributor

@daniel-mohedano daniel-mohedano commented May 8, 2026

What Does This Do

  • Introduces proper handling for Bazel's JUnit tests runner suite events by:
    • Removing the previous temporary solution of lazy starting suites.
    • Introduces a new instrumentation for fireTestSuiteStarted and fireTestSuiteFinished methods to unwrap the notifiers and propagate the firing to our tracing listener.

Motivation

Bazel's default JUnit4 configuration for running test suites introduces changes that mess with our instrumentation:

  1. Bazel's JUnit entry-point is through BazelTestRunner.main(), which builds a JUnit4Runner (does not inherit org.junit.runner.Runner) and calls run().
  2. JUnit4Runner.run() instantiates JUnitCore, registers its run listeners and calls core.run(cancellableRequest).
  3. JUnitCore creates a fresh RunNotifier (from now on A) and registers the listeners on it.
  4. The request is run with a custom CancellableRunner (extends org.junit.runner.Runner). Its run(A) does currentNotifier = new ThreadSafeRunNotifier(A); delegate.run(currentNotifier);.
  5. ThreadSafeRunNotifier extends RunNotifierWrapper extends RunNotifier, so currentNotifier (from now on B) has its own private inherited listeners field.
  6. The real runner.run(B) executes.
  7. Our instrumentation modifies the call to do B.addListener(tracingListener), but Bs implementation overrides the method to call delegate.addListener(tracingListener); so our listener lands on A.listeners.
  8. The real runner fires events such as test suite starts with B.fireTestSuiteStarted(...).
  9. The wrapper does not override test suite methods, so it is resolved as RunNotifier.fireTestSuiteStarted(...) invoked with this = B. This iterates over B.listeners, which is empty.
  10. The wrapper does forward regular test events with delegate.fireTest...(...) by overriding the methods

All this caused test suite events to be dropped.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@daniel-mohedano daniel-mohedano added type: bug Bug report and fix comp: ci visibility Continuous Integration Visibility labels May 8, 2026
@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - sbt-scalatest

Job Status: success

Scenario Overhead (%)
agent 54.74
agentEvpProxy 54.68

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - nebula-release-plugin

Job Status: success

Scenario Overhead (%)
agent 35.84
agentless 35.35
agentlessCodeCoverage 42.97
agentlessLineCoverage 73.76

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - netflix-zuul

Job Status: success

Scenario Overhead (%)
agent 86.27
agentless 80.29
agentlessCodeCoverage 95.07
agentlessLineCoverage 111.19

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - pass4s

Job Status: success

Scenario Overhead (%)
agent 14.32
agentless 11.41
agentlessCodeCoverage 22.27

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - reactive-streams-jvm

Job Status: success

Scenario Overhead (%)
agent 23.84
agentless 19.96
agentlessCodeCoverage 20.69
agentlessLineCoverage 30.66

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - sonar-kotlin

Job Status: success

Scenario Overhead (%)
agent 14.02
agentless 13.08
agentlessCodeCoverage 16.14
agentlessLineCoverage 20.21

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - jolokia

Job Status: success

Scenario Overhead (%)
agent 95.52
agentless 89.64
agentlessCodeCoverage 100.66
agentlessLineCoverage 100.73

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - okhttp

Job Status: success

Scenario Overhead (%)
agent 18.70
agentless 18.76
agentlessCodeCoverage 21.52
agentlessLineCoverage 43.44

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - spring_boot

Job Status: success

Scenario Overhead (%)
agent 16.66
agentless 9.94
agentlessCodeCoverage 13.61
agentlessLineCoverage 33.42

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Test Environment - sonar-java

Job Status: success

Scenario Overhead (%)
agent 29.18
agentless 15.26
agentlessCodeCoverage 133.84
agentlessLineCoverage 143.73

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility type: bug Bug report and fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant