Skip to content

perf: drop unused ORDER BY in old-user-record pruning queries#2302

Open
taddes wants to merge 2 commits into
masterfrom
perf/unneeded-order-by-STOR-117
Open

perf: drop unused ORDER BY in old-user-record pruning queries#2302
taddes wants to merge 2 commits into
masterfrom
perf/unneeded-order-by-STOR-117

Conversation

@taddes
Copy link
Copy Markdown
Collaborator

@taddes taddes commented May 12, 2026

Description

The two _GET_OLD_USER_RECORDS_FOR_SERVICE queries in database.py are used by
purge_old_records.py sort their entire matching set by
(replaced_at DESC, uid DESC) before applying LIMIT/OFFSET. In production
the matching set is a lot of rows, so the planner must sort them
all to hand back 10, repeated on every iteration of the purge loop, which is very much suboptimal.

This removes that ORDER BY from both queries.

Also had to update relevant tests in test_purge_old_recordssince the test enumerated service_requests by index and compared each fxa_kid positionally, relying on the dropped ORDER BY replaced_at DESC to make the order deterministic. The test's actual intent is "both replaced records were purged with valid HAWK-signed DELETE
requests against the storage node" order isn't part of it. Comparing as a set makes that intent explicit and keeps the test stable regardless of scan order.

Testing

How should reviewers test?

Issue(s)

Closes STOR-117.

@taddes taddes self-assigned this May 12, 2026
@taddes taddes requested review from chenba and pjenvey May 12, 2026 17:45
@chenba
Copy link
Copy Markdown
Collaborator

chenba commented May 12, 2026

The fetched records are checked in

if rows == previous_list:
raise Exception("Loop detected")
. Without sorting, that check works differently; the same set but in different order is not considered a "loop". Might be okay in practice though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants