Investigate the reassure test flakiness

[mountiny]

The reassure tests are flaky in the onyx repo https://github.com/Expensify/react-native-onyx/actions/runs/15822875174/job/44595958067?pr=642 investigate the root cause and fix it

Issue Owner

Current Issue Owner: @fabioh8010

[melvin-bot]

Triggered auto assignment to @alexpensify (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details. Please add this bug to a GH project, as outlined in the SO.

[fabioh8010]

Updates:

• Performing tests here. It seems that the flakiness come from OnyxUtils.ts. I suspect some tests there are leaving leftovers that are impacting itself and the other files too.

[fabioh8010]

Updates:

Spent the last days trying different approaches:

• Clearing some OnyxUtils internals during tests
• Changing the workflows to have less steps during the tests and use the same runner we have in E/App
• Making workflows output the Reassure data so I could inspect

None of them seem to have definitive fix over the flakiness – it will still happen randomly, although way more often on CI than locally, which is very weird.

I have the impression the order of the test files executed somehow matter a lot – I could observe that the flakiness seem to happen when the baseline and delta tests are ran following a specific file order.

I will continue investigating next week.

[fabioh8010]

Updates:

• Created a PR with some improvements I believe will reduce flakiness in CI -> [NoQA] Investigating Reassure flakiness react-native-onyx#646

[fabioh8010]

Updates:

• Although I can observe some improvements, flakiness is still consistently present on CI. Performing new tests here.

[fabioh8010]

Updates:

• Came up with a specific workflow config that seems much more stable, will continue performing tests next week.

[fabioh8010]

Updates:

• After more testing I'm really confident this configuration is stable enough for us, I will perform cleanup in the PR and open it for review.

[melvin-bot]

@alexpensify @fabioh8010 @mountiny this issue was created 2 weeks ago. Are we close to a solution? Let's make sure we're treating this as a top priority. Don't hesitate to create a thread in #expensify-open-source to align faster in real time. Thanks!