fix(query): push down indexed equality + IN filters and surface DB errors#224
Open
jxom wants to merge 1 commit into
Open
fix(query): push down indexed equality + IN filters and surface DB errors#224jxom wants to merge 1 commit into
jxom wants to merge 1 commit into
Conversation
…rors
`WHERE "orderId" = X` (unquoted), `WHERE "orderId" IN (X, Y)`, and
similar multi-row joins against decoded indexed columns previously fell
through to `abi_uint(topic1) = N` predicates with no pushdown, forcing
Postgres to compute `abi_uint` over every selector-matching row in
`logs` and routinely tripping the per-query `statement_timeout`.
The single-row `= X LIMIT 1` shape worked only because the planner
streamed enough rows before timing out. Cadent (and other downstream
clients) had to fan multi-row joins out as N parallel single-row
queries to work around the failure.
This patch:
- Splits `rewrite_filters_for_pushdown` into two passes — `=` and
`IN (...)` — that each rewrite a decoded-indexed-column predicate
into a raw `topicN = '\\x...'` (or `topicN IN ('\\x...', ...)`) form
Postgres can resolve via the logs index.
- Accepts both quoted (`'17486013'`) and unquoted (`17486013`)
numeric literals on the equality form.
- Leaves `IN (SELECT ...)` subqueries untouched (the regex's inner
paren-free restriction skips them).
- Surfaces the actual Postgres `DbError` (severity + message) via a
new `format_error_chain` helper instead of the opaque "db error"
head string from `tokio_postgres::Error::Display`. Without this the
underlying timeout was invisible to API clients.
Amp-Thread-ID: https://ampcode.com/threads/T-019e929b-cab4-745f-804a-8c5e03ae80dc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WHERE "orderId" = X(unquoted),WHERE "orderId" IN (X, Y), and similar multi-row joins against decoded indexed columns previously fell through toabi_uint(topic1) = Npredicates with no pushdown, forcing Postgres to computeabi_uintover every selector-matching row inlogsand routinely tripping the per-querystatement_timeout.The single-row
= X LIMIT 1shape worked only because the planner streamed enough rows before timing out. Cadent (and other downstream clients) had to fan multi-row joins out as N parallel single-row queries to work around the failure.What this PR does
rewrite_filters_for_pushdowninto two passes —=andIN (...)— that each rewrite a decoded-indexed-column predicate into a rawtopicN = '\\x...'(ortopicN IN ('\\x...', ...)) form Postgres can resolve via the logs index.'17486013') and unquoted (17486013) numeric literals on the equality form.IN (SELECT ...)subqueries untouched (the regex's inner paren-free restriction skips them).DbError(severity + message) via a newformat_error_chainhelper instead of the opaque"db error"head string fromtokio_postgres::Error::Display. Without this the underlying timeout was invisible to API clients.Live verification (testnet)
Before:
After (same testnet, same data):
Test impact
src/query/parser.rs: addedrewrite_eq_for_pushdown/rewrite_in_for_pushdownhelpers + snapshots covering quoted addresses, IN-list, and unquoted numeric equality.src/service/mod.rs:format_error_chain+ threading throughsanitize_db_error.Downstream
Unblocks cadent#53 (
GET /exchanges/:tokenA/:tokenB/orders), which joinsOrderCancelledandOrderFilledbyorderId IN (...)against anOrderPlacedscan window. Also cleans up the per-orderIdsingle-row workaround currently used by the swap-feed and OHLC routes.Amp-Thread-ID: https://ampcode.com/threads/T-019e929b-cab4-745f-804a-8c5e03ae80dc