acc: fix flaky grants-remove-principal plan golden#5945
Merged
Conversation
The testserver deliberately randomizes grant-assignment order (it iterates a Go map, mirroring the Azure backend; see libs/testserver/grants.go), but this test dumped remote_state.__embed__ straight into out.plan.grants.json without normalizing the order. The golden happened to match on most runs and failed whenever map iteration emitted the principals in the other order (it surfaced on linux/direct on main). Sort __embed__ by principal in the jq step, matching how the sibling grants/schemas/out_of_band_principal test handles the same array. Co-authored-by: Isaac
janniklasrose
enabled auto-merge
July 16, 2026 11:44
pietern
approved these changes
Jul 16, 2026
andrewnester
approved these changes
Jul 16, 2026
janniklasrose
disabled auto-merge
July 16, 2026 13:42
Collaborator
Integration test reportCommit: 10e54e2
20 interesting tests: 11 MISS, 4 SKIP, 3 RECOVERED, 1 PANIC, 1 flaky
Top 10 slowest tests (at least 2 minutes):
|
Collaborator
Integration test reportCommit: 84a9fe7
38 interesting tests: 20 FAIL, 12 flaky, 3 RECOVERED, 2 SKIP, 1 KNOWN
Top 50 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
acceptance/bundle/deploy/readplan/grants-remove-principalwas failing intermittently onmain(seen on thelinux, directjob of the post-mergebuildrun for dce783d). The diff was a pure reordering of the two entries inremote_state.__embed__:Why
The testserver deliberately randomizes grant-assignment order — it builds the list by iterating a Go map (
libs/testserver/grants.go), with a comment noting this is intentional because the Azure backend behaves the same way. The test wroteremote_state.__embed__straight into the golden without normalizing that order, so the golden matched only when map iteration happened to emit the principals in the checked-in order.This is a flaky test, not a product regression:
macos, directpassed on the same commit, and the test passes locally on repeated runs.Fix
Sort
__embed__by principal in thejqstep, matching how the siblingacceptance/bundle/resources/grants/schemas/out_of_band_principaltest handles the same non-deterministic array. Verified with 40 consecutive local runs of bothEnvMatrixvariants.This pull request and its description were written by Isaac.