[AUTOCUT] Gradle Check Flaky Test Report for FullRollingRestartIT

[opensearch-ci-bot]

Flaky Test Report for FullRollingRestartIT

Noticed the FullRollingRestartIT has some flaky, failing tests that failed during post-merge actions and timer trigerred run on main branch.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
main	null	60843 71542 42 60063 60317 60324 60337 60345 60450 60482 60567 60733 60753 60763 60793 60807 60831 60890 60925 60949 60982 61009 61012 61022 61075 61082 61090 61095 61098 61261 61276 61322 61331 61426 61528 61529 61553 61623 61644 61689 61759 61830 61838 61892 71099 71188 71259 71351 71403 71409 71508 71554 71585 71597 71632 71673 71714 71724 71925 71995 72302 72775 72957 73375 73652 74203 74235 75143 75436	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}} org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart_withNoRecoveryPayloadAndSource {p0={"cluster.indices.replication.strategy":"SEGMENT"}} org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"DOCUMENT"}}
ac0f565	21425	75424	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart_withNoRecoveryPayloadAndSource {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
d56b643	21306	75002	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart_withNoRecoveryPayloadAndSource {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
fc04bed	21044	74460	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
6806ff9	21005	73917	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
d2f0e48	21070	73792	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
c94203d	20587	71279	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart_withNoRecoveryPayloadAndSource {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
e0e7f29	20589	71205	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart_withNoRecoveryPayloadAndSource {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
09b6a36	20551	71085	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart_withNoRecoveryPayloadAndSource {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
c5d26f7	18857	61858	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart_withNoRecoveryPayloadAndSource {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
b55b140	18904	61695	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
b0417f3	18511	61205	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
36844f5	18754	61117	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
c4d9de6	18767	60884	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
6119faa	18749	60845	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
1352627	18707	60548	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
cb22cb0	18661	60520	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart_withNoRecoveryPayloadAndSource {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
624f85a	18743	60468	org.opensearch.recovery.FullRollingRestartIT.testFullRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
f7ebe22	18488	59157	org.opensearch.recovery.FullRollingRestartIT.classMethod org.opensearch.recovery.FullRollingRestartIT.testDerivedSourceWithConcurrentUpdatesRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}
8a59ca1	18277	59141	org.opensearch.recovery.FullRollingRestartIT.classMethod org.opensearch.recovery.FullRollingRestartIT.testDerivedSourceWithConcurrentUpdatesRollingRestart {p0={"cluster.indices.replication.strategy":"DOCUMENT"}}
c92b8ea	18435	59128	org.opensearch.recovery.FullRollingRestartIT.testDerivedSourceWithMixedVersionRollingRestart {p0={"cluster.indices.replication.strategy":"DOCUMENT"}}
d0c2e39	17797	59086	org.opensearch.recovery.FullRollingRestartIT.classMethod org.opensearch.recovery.FullRollingRestartIT.testDerivedSourceWithConcurrentUpdatesRollingRestart {p0={"cluster.indices.replication.strategy":"SEGMENT"}}

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the FullRollingRestartIT class are:

• 18504
• 18479
• 18467
• 18497
• 18517
• 18070
• 18195
• 18229
• 18351
• 18375
• 18664
• 18685
• 18887
• 18896
• 20503
• 18672
• 18702
• 18724
• 18805
• 18936
• 20775
• 21247
• 21311
• 18333
• 18424
• 18451
• 18565
• 18656
• 18706
• 18777
• 18800
• 18877
• 18890
• 20858
• 21178
• 16291
• 17741
• 18066
• 18491
• 18531
• 18550
• 18568
• 18639
• 18668
• 18671
• 18691
• 18697
• 18712
• 18720
• 18732
• 18735
• 18741
• 18746
• 18748
• 18752
• 18766
• 18779
• 18803
• 18814
• 18825
• 18837
• 18842
• 18843
• 18844
• 18874
• 18885
• 18889
• 18901
• 18914
• 18929
• 18933
• 19783
• 20150
• 20274
• 20542
• 20558
• 20566
• 20585
• 20606
• 20607
• 20623
• 20632
• 20637
• 20648
• 20675
• 20682
• 20713
• 20727
• 20729
• 20802
• 20854
• 20857
• 20909
• 20935
• 20943
• 20978
• 20996
• 21017
• 21055
• 21122
• 21137
• 21171
• 21217
• 21225
• 21253
• 21285
• 21418

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[tanik98]

Concurrent update on same doc which is causing sequence number conflict, and while waiting on this, test suite is timing out.

java.lang.Exception: Test abandoned because suite timeout was reached.

Бэс 10, 2025 9:11:07 ЭК com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
WARNING: Uncaught exception in thread: Thread[#4762,opensearch[node_t0][generic][T#1],5,TGRP-FullRollingRestartIT]
java.lang.AssertionError: seqNo [100] was processed twice in generation [5], with different data. prvOp [Index{id='76', seqNo=100, primaryTerm=2, version=3, autoGeneratedIdTimestamp=-1}], newOp [Index{id='76', seqNo=100, primaryTerm=2, version=3, autoGeneratedIdTimestamp=-1}]
	at __randomizedtesting.SeedInfo.seed([33B7FED8D60FC88D]:0)
	at org.opensearch.index.translog.TranslogWriter.assertNoSeqNumberConflict(TranslogWriter.java:341)
	at org.opensearch.index.translog.TranslogWriter.add(TranslogWriter.java:289)
	at org.opensearch.index.translog.Translog.add(Translog.java:579)
	at org.opensearch.index.translog.InternalTranslogManager.add(InternalTranslogManager.java:338)
	at org.opensearch.index.engine.NRTReplicationEngine.index(NRTReplicationEngine.java:235)
	at org.opensearch.index.shard.IndexShard.applyIndexOperation(IndexShard.java:1176)
	at org.opensearch.index.shard.IndexShard.applyTranslogOperation(IndexShard.java:2472)
	at org.opensearch.index.shard.IndexShard.applyTranslogOperation(IndexShard.java:2459)
	at org.opensearch.indices.recovery.RecoveryTarget.lambda$indexTranslogOperations$4(RecoveryTarget.java:317)
	at org.opensearch.core.action.ActionListener.completeWith(ActionListener.java:344)
	at org.opensearch.indices.recovery.RecoveryTarget.indexTranslogOperations(RecoveryTarget.java:292)
	at 

[andrross]

Catch All Triage - 1 2

@tanik98 Is this still a problem?

[tanik98]

Catch All Triage - 1 2

@tanik98 Is this still a problem?

It's addressed as part of this PR, though not merged yet: #18565
But it shouldn't be causing the issue currently as, previous PR was reverted.