Commit c7c78e0
committed
[MINOR][CORE][TESTS] Fix flaky DAGSchedulerSuite test for push-based shuffle
### What changes were proposed in this pull request?
Add `sc.listenerBus.waitUntilEmpty()` before the `completedStage` assertion in the push-based shuffle test in `DAGSchedulerSuite`.
### Why are the changes needed?
The test was intermittently failing due to a race condition. `SparkListenerStageCompleted` events are delivered asynchronously through the
listener bus, so the `completedStage` assertion could execute before the event was processed, causing a spurious failure. Waiting for the
listener bus to drain before asserting eliminates the race.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing test in `DAGSchedulerSuite`. The fix removes a race condition rather than adding new test logic.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Sonnet 4.6 (Claude Code)
Closes #55221 from dbtsai/dagscheduler-flaky-fix.
Authored-by: DB Tsai <dbtsai@dbtsai.com>
Signed-off-by: DB Tsai <dbtsai@dbtsai.com>1 parent e8b5cb8 commit c7c78e0
1 file changed
Lines changed: 1 addition & 0 deletions
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5827 | 5827 | | |
5828 | 5828 | | |
5829 | 5829 | | |
| 5830 | + | |
5830 | 5831 | | |
5831 | 5832 | | |
5832 | 5833 | | |
| |||
0 commit comments