test(shardservice): make remote compatibility preflight deterministic#26063
Merged
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
XuPeng-SH
approved these changes
Jul 23, 2026
XuPeng-SH
left a comment
XuPeng-SH
left a comment
Contributor
There was a problem hiding this comment.
Reviewed the complete test-only diff and the production path it exercises. The deterministic cache setup preserves the intended service.Read preflight-ordering contract while removing unrelated heartbeat/rebalance timing. Resource cleanup is safe, the focused race/stress and full package tests pass, and a mutation that sends after the first compatible target is correctly caught by the zero-send assertion. No blocking findings.
Contributor
Merge Queue Status
This pull request spent 5 minutes 15 seconds in the queue, with no time running CI. Waiting for
All conditions
ReasonPull request #26063 has been merged manually at 67adb75 HintYou were too fast! Tick the box to put this pull request back in the merge queue (same as
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Which issue(s) this PR fixes:
Fixes #26045
What this PR does / why we need it:
TestReadValidatesAllRemoteShardsBeforeSendingwas added with #25872 to protect a specific ordering contract: when a binary prepared-parameter read targets multiple remote shards, every selected target must pass commit compatibility validation before the first RPC is sent.The observed failure was initially reported as a remote-read timeout, but no remote read had started: the counting client remained at zero
AsyncSendcalls. The roughly 60-second delay came from the test setup. It created a real three-CN shard topology through asynchronous heartbeat, allocation, rebalance, and read-cache refresh. If allocation started before all CNs were visible, replicas first accumulated on one CN; subsequent balancing could wait for the one-minute CN freeze interval. The test then chose an incompatible target from asynchronously maintained state, whileReadcould observe a different target ordering and return after the first incompatible shard, leaving only oneAdjustobservation.#26051 improved that setup by waiting for all CNs and separating setup/read contexts. This PR narrows the test further to the contract it owns:
service.Read, replica selection, cluster compatibility lookup, and counting RPC client;AsyncSendcalls.This deliberately does not increase timeouts, add retries, or change production behavior. Scheduler, heartbeat, rebalance, and real remote-handler behavior remain covered by their owning tests, including
TestBinaryReadUsesVersionedRemoteHandler; they are not prerequisites for this preflight-ordering regression. A regression that validates and sends in the same loop will send after the first compatible shard and fail the zero-send assertion.Validation after the final edit:
-count=100passed in 4.407s;-race -count=20passed;./pkg/shardservicepackage passed in 17.895s;go build ./pkg/shardservicepassed;go vet ./pkg/shardservicepassed;git diff --checkpassed.For comparison, current
mainwith the asynchronous setup passed 100 repetitions locally but took 109.544s.