Fixing flaky tests#5555
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I set Claude loose on three flaky tests I observed in CI. Hoping this squashes them!
Summary
Three independent flaky-test fixes, each with a different root cause but the same symptom: passes locally, fails intermittently on CI.
1. Inventory event replay tiebreaker (
app/models/event.rb)Event.for_organizationordered by(event_time, updated_at). When tests usetravel_toto freeze the clock, multiple events get identicalevent_time,created_at, andupdated_at. Postgres tiebreaks at its discretion — sometimes aDistributionEventsorts before theAuditEventit depends on, soInventoryAggregate.inventory_forreplays them in the wrong order andvalidate_inventoryfails withCould not reduce quantity by N - current quantity is 0.Changed the secondary sort from
:updated_atto:id. Insertion order is the only stable, never-changing tiebreak;idis unique and monotonic. Snapshot-correctness logic inmost_recent_snapshotstill usesupdated_at(for detecting retroactive edits) and is unchanged.Symptomatic example:
spec/system/request_system_spec.rb:223("should change request to fulfilled").2. Date Range Picker keyboard submit (
spec/support/date_range_picker_shared_example.rb)The shared example did:
fill_intypes character-by-character, and each keystroke fires Litepicker.js handlers that can re-render the input. The.nativereference grabs a Ferrum node id at one point in time; by the timesend_keys(:enter)runs, the underlying DOM node has been replaced and Cuprite raisesObsoleteNode/No node with given id found.Replaced the three uses with
click_on "Filter", which re-resolves the Filter button at click time. All six consumers of the shared example (Donation, Transfer, Purchase, Distribution, Request, Adjustment) have a Filter button via thefilter_buttonUI helper.3. Partner CSV export spec — duplicate profiles (
spec/services/exports/export_partners_csv_service_spec.rb)The "should handle a partner with missing profile info" test did:
The
partner_profilefactory defaultspartner { Partner.first || create(:partner) }, so the new profile silently attached to the same partner that already had one — leaving twopartner_profilesrows with the samepartner_id. There is no DB-level unique constraint on that column, so Rails'has_one :profilelookup (WHERE partner_id = X LIMIT 1, noORDER BY) returns whichever row Postgres picks. About half the time on CI, Rails saw the new profile as already-attached, no-op'd the assignment, and left the old populated profile in place — so the export contained the old profile's data instead of the empty one the test expected.Destroyed the existing profile first and passed the partner explicitly to the new one, so there's only ever one row per partner.
Note: not bundled in this PR, but worth considering as a follow-up — adding a unique index on
partner_profiles.partner_idwould enforce thehas_oneinvariant at the DB level and prevent this whole class of bug.Test plan
spec/system/request_system_spec.rb:223runs green repeatedly (5/5 locally)spec/services/exports/export_partners_csv_service_spec.rbpasses (11/11 locally)