Skip to content

Commit 02e7b98

Browse files
committed
Fix StatefulStreamingParDoEvaluatorTest window assertions
The createStreamingSource helper method generates events across 4 different 1-second windows when iterCount=2, because it advances the watermark after every single batch (even between Key 1 and Key 2 batches). Previously, VerifyStatefulOutput incorrectly assumed Key 1 and Key 2 would be emitted in the same windows (Window 1 and Window 2). This caused an AssertionError because Key 2 actually had 0 elements in Window 1. This commit updates the assertions in shouldProcessWindowedStatefulParDo to verify the 4 distinct windows individually, matching the generated timestamps.
1 parent 7676575 commit 02e7b98

1 file changed

Lines changed: 12 additions & 2 deletions

File tree

runners/spark/src/test/java/org/apache/beam/runners/spark/translation/streaming/StatefulStreamingParDoEvaluatorTest.java

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -372,14 +372,24 @@ public void shouldProcessWindowedStatefulParDo() {
372372

373373
IntervalWindow window1 = new IntervalWindow(new Instant(0), Duration.standardSeconds(1L));
374374
IntervalWindow window2 = new IntervalWindow(new Instant(1000), Duration.standardSeconds(1L));
375+
IntervalWindow window3 = new IntervalWindow(new Instant(2000), Duration.standardSeconds(1L));
376+
IntervalWindow window4 = new IntervalWindow(new Instant(3000), Duration.standardSeconds(1L));
375377

376378
PAssert.that(result)
377379
.inWindow(window1)
378-
.satisfies(new VerifyStatefulOutput(Arrays.asList(1, 2, 3), Arrays.asList(4, 5, 6)));
380+
.satisfies(new VerifyStatefulOutput(Arrays.asList(1, 2, 3), Collections.emptyList()));
379381

380382
PAssert.that(result)
381383
.inWindow(window2)
382-
.satisfies(new VerifyStatefulOutput(Arrays.asList(7, 8, 9), Arrays.asList(10, 11, 12)));
384+
.satisfies(new VerifyStatefulOutput(Collections.emptyList(), Arrays.asList(4, 5, 6)));
385+
386+
PAssert.that(result)
387+
.inWindow(window3)
388+
.satisfies(new VerifyStatefulOutput(Arrays.asList(7, 8, 9), Collections.emptyList()));
389+
390+
PAssert.that(result)
391+
.inWindow(window4)
392+
.satisfies(new VerifyStatefulOutput(Collections.emptyList(), Arrays.asList(10, 11, 12)));
383393

384394
p.run().waitUntilFinish();
385395
}

0 commit comments

Comments
 (0)