[AUTOCUT] Gradle Check Flaky Test Report for FlightMetricsTests

[opensearch-ci-bot]

Flaky Test Report for FlightMetricsTests

Noticed the FlightMetricsTests has some flaky, failing tests that failed during post-merge actions and timer trigerred run on main branch.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
main	null	61789 61849 63619 63754 63999 64967 65370 65858 67367 67765 68286 68946 69296 69727 71382 71926 72254 72821 74408 75022	org.opensearch.arrow.flight.stats.FlightMetricsTests.testComprehensiveMetrics
54d6d1f	20897	72846	org.opensearch.arrow.flight.stats.FlightMetricsTests.testComprehensiveMetrics
1a53317	18966	62203	org.opensearch.arrow.flight.stats.FlightMetricsTests.testComprehensiveMetrics
02d7608	18968	61959	org.opensearch.arrow.flight.stats.FlightMetricsTests.testComprehensiveMetrics
6468464	18896	61864	org.opensearch.arrow.flight.stats.FlightMetricsTests.testComprehensiveMetrics

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the FlightMetricsTests class are:

• 18424
• 18874
• 19178
• 21242
• 18568
• 18936
• 19101
• 19229
• 19332
• 19352
• 19467
• 19648
• 19704
• 19734
• 19907
• 19976
• 20055
• 20391
• 20407
• 20438
• 20536
• 20570
• 20625
• 20672
• 20816
• 20900
• 20908
• 20930
• 21024
• 21048
• 21079
• 21106
• 21206
• 21211
• 21444

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[rishabhmaurya]

I'm taking a look

[andrross]

@rishabhmaurya Looks like this is still failing:

https://build.ci.opensearch.org/job/gradle-check/69798/consoleText

REPRODUCE WITH: ./gradlew ':plugins:arrow-flight-rpc:test' --tests 'org.opensearch.arrow.flight.stats.FlightMetricsTests.testComprehensiveMetrics' -Dtests.seed=8A2810F3A5BF6F74 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=sr-Latn-RS -Dtests.timezone=Etc/GMT+10 -Druntime.java=25

FlightMetricsTests > testComprehensiveMetrics FAILED
    BindTransportException[Failed to bind to [/0:0:0:0:0:0:0:1%lo, /127.0.0.1]:PortsRange{portRange='10601'}]; nested: IOException[Failed to bind to address /127.0.0.1:10601]; nested: BindException[Address already in use];
        at __randomizedtesting.SeedInfo.seed([8A2810F3A5BF6F74:A4537445BE734CCA]:0)
        at app//org.opensearch.arrow.flight.transport.FlightTransport.bindToPort(FlightTransport.java:261)
        at app//org.opensearch.arrow.flight.transport.FlightTransport.bindServer(FlightTransport.java:182)
        at app//org.opensearch.arrow.flight.transport.FlightTransport.doStart(FlightTransport.java:162)
        at app//org.opensearch.common.lifecycle.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:77)
        at app//org.opensearch.arrow.flight.transport.FlightTransportTestBase.setUp(FlightTransportTestBase.java:104)
        at app//org.opensearch.arrow.flight.stats.FlightMetricsTests.setUp(FlightMetricsTests.java:31)

        Caused by:
        java.io.IOException: Failed to bind to address /127.0.0.1:10601
            at io.grpc.netty.NettyServer.start(NettyServer.java:341)
            at io.grpc.internal.ServerImpl.start(ServerImpl.java:185)
            at io.grpc.internal.ServerImpl.start(ServerImpl.java:94)
            at org.apache.arrow.flight.FlightServer.start(FlightServer.java:90)
            at org.opensearch.arrow.flight.transport.FlightTransport.lambda$bindToPort$0(FlightTransport.java:250)
            at org.opensearch.common.transport.PortsRange.iterate(PortsRange.java:73)
            at org.opensearch.arrow.flight.transport.FlightTransport.bindToPort(FlightTransport.java:216)
            ... 5 more

[cwperks]

Can we create a port tracking mechanism to check if a port was previously allocated?

Not sure if its useful, but the security plugin does have a PortAllocator which keeps track of which ports have been allocated when starting up nodes in testclusters.