perf: drop unused ORDER BY in old-user-record pruning queries#2302
Open
taddes wants to merge 2 commits into
Open
perf: drop unused ORDER BY in old-user-record pruning queries#2302taddes wants to merge 2 commits into
taddes wants to merge 2 commits into
Conversation
Collaborator
|
The fetched records are checked in syncstorage-rs/tools/tokenserver/purge_old_records.py Lines 81 to 82 in 004d027 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The two
_GET_OLD_USER_RECORDS_FOR_SERVICEqueries in database.py are used bypurge_old_records.pysort their entire matching set by(replaced_at DESC, uid DESC)before applying LIMIT/OFFSET. In productionthe matching set is a lot of rows, so the planner must sort them
all to hand back 10, repeated on every iteration of the purge loop, which is very much suboptimal.
This removes that
ORDER BYfrom both queries.Also had to update relevant tests in
test_purge_old_recordssince the test enumerated service_requests by index and compared each fxa_kid positionally, relying on the dropped ORDER BY replaced_at DESC to make the order deterministic. The test's actual intent is "both replaced records were purged with valid HAWK-signed DELETErequests against the storage node" order isn't part of it. Comparing as a set makes that intent explicit and keeps the test stable regardless of scan order.
Testing
How should reviewers test?
Issue(s)
Closes STOR-117.