Skip to content

perf(postgres:subaccounts): add partial indices and optimize query fo… #3318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
UnbornAztecKing merged 15 commits into from
Jun 23, 2026
Merged

perf(postgres:subaccounts): add partial indices and optimize query fo… #3318

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
f3ad822
perf(postgres:subaccounts): add partial indices and optimize query fo…
UnbornAztecKing Jan 16, 2026
c99dddf
updated test to expect results in any order
UnbornAztecKing Jan 16, 2026
b39486d
feat(postgres): admit a ValueDict as bindings, aligned with knex
UnbornAztecKing Jan 16, 2026
04c39a5
feat(postgres:transfer, subaccount): parameterize queries to permit S…
UnbornAztecKing Jan 16, 2026
3819467
lint
UnbornAztecKing Jan 16, 2026
9cb7ebe
change index operations to concurrently
UnbornAztecKing Jan 16, 2026
994087d
typo
UnbornAztecKing Jan 16, 2026
02600f0
fix knex multi-statement migrations
UnbornAztecKing Jan 16, 2026
b52f00e
lint
UnbornAztecKing Jan 16, 2026
c510cf6
synchronous drop index
UnbornAztecKing Jan 16, 2026
b0a98b9
set work_mem for transfers query
UnbornAztecKing Jan 17, 2026
1f1b68a
typo
UnbornAztecKing Jan 17, 2026
3129539
lint
UnbornAztecKing Jan 17, 2026
229f75f
perf(ender): raise work_mem for block processing and candles
UnbornAztecKing Jan 17, 2026
1f219fa
parameterize queries for planner
UnbornAztecKing Jan 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions indexer/packages/postgres/__tests__/stores/funding-payments-table.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,16 @@ describe('funding payments store', () => {
});

it('Successfully finds all FundingPayments', async () => {
await Promise.all([
FundingPaymentsTable.create(defaultFundingPayment),
FundingPaymentsTable.create(defaultFundingPayment2),
]);
// funding payment insertion order is expected
await FundingPaymentsTable.create(defaultFundingPayment);
await FundingPaymentsTable.create(defaultFundingPayment2);

const { results: fundingPayments } = await FundingPaymentsTable.findAll({}, [], {});

expect(fundingPayments.length).toEqual(2);
expect(fundingPayments[0]).toEqual(expect.objectContaining(defaultFundingPayment));
expect(fundingPayments[1]).toEqual(expect.objectContaining(defaultFundingPayment2));

});

it('Successfully finds FundingPayments with createdAtHeight', async () => {
Expand Down Expand Up @@ -125,10 +125,9 @@ describe('funding payments store', () => {
});

it('supports pagination', async () => {
await Promise.all([
FundingPaymentsTable.create(defaultFundingPayment),
FundingPaymentsTable.create(defaultFundingPayment2),
]);
// funding payment insertion order is expected
await FundingPaymentsTable.create(defaultFundingPayment);
await FundingPaymentsTable.create(defaultFundingPayment2);

const { results: fundingPayments } = await FundingPaymentsTable.findAll(
{
Expand Down
47 changes: 47 additions & 0 deletions ...res/src/db/migrations/migration_files/20260116114440_create_indices_transfers_not_null.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
import * as Knex from 'knex';

export async function up(knex: Knex): Promise<void> {
await knex.raw(
'DROP INDEX IF EXISTS "transfers_sendersubaccountid_createdatheight_index";',
);

await knex.raw(
`CREATE INDEX CONCURRENTLY IF NOT EXISTS transfers_sender_id_height_nn
ON transfers("senderSubaccountId", "createdAtHeight")
WHERE "senderSubaccountId" IS NOT NULL;`,
);

await knex.raw(
'DROP INDEX IF EXISTS "transfers_recipientsubaccountid_createdatheight_index";',
);

await knex.raw(
`CREATE INDEX CONCURRENTLY IF NOT EXISTS transfers_recipient_id_height_nn
ON transfers("recipientSubaccountId", "createdAtHeight")
WHERE "recipientSubaccountId" IS NOT NULL;`,
);
}

export async function down(knex: Knex): Promise<void> {
await knex.raw(
'DROP INDEX CONCURRENTLY IF EXISTS "transfers_sender_id_height_nn";',
);

await knex.raw(
`CREATE INDEX CONCURRENTLY IF NOT EXISTS transfers_sendersubaccountid_createdatheight_index
ON transfers("senderSubaccountId", "createdAtHeight");`,
);

await knex.raw(
'DROP INDEX CONCURRENTLY IF EXISTS "transfers_recipient_id_height_nn";',
);

await knex.raw(
`CREATE INDEX CONCURRENTLY IF NOT EXISTS transfers_recipientsubaccountid_createdatheight_index
ON transfers("recipientSubaccountId", "createdAtHeight");`,
);
}

export const config = {
transaction: false,
};
4 changes: 2 additions & 2 deletions indexer/packages/postgres/src/stores/candle-table.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ export async function findCandlesMap(
SELECT DISTINCT ON (
ticker,
resolution
) candles.* FROM
) candles.* FROM
candles
WHERE
"ticker" IN (${tickers.map((ticker) => { return `'${ticker}'`; }).join(',')}) AND
Expand All @@ -215,7 +215,7 @@ export async function findCandlesMap(
SELECT DISTINCT ON (
ticker,
resolution
) candles.* FROM
) candles.* FROM
candles
WHERE
"ticker" IN (${tickers.map((ticker) => { return `'${ticker}'`; }).join(',')}) AND
Expand Down
12 changes: 6 additions & 6 deletions indexer/packages/postgres/src/stores/fill-table.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -340,11 +340,11 @@ export async function getCostOfFills(
.getConnection()
.raw(
`
SELECT SUM(CASE
WHEN side = 'SELL' THEN price * size
ELSE -1 * price * size
SELECT SUM(CASE
WHEN side = 'SELL' THEN price * size
ELSE -1 * price * size
END) AS "cost"
FROM fills
FROM fills
WHERE "subaccountId" = ?
AND "createdAtHeight" <= ?;
`,
Expand Down Expand Up @@ -519,7 +519,7 @@ export async function getOpenSizeWithFundingIndex(
.raw(
`
WITH input AS (
SELECT
SELECT
f."clobPairId",
SUM(CASE
WHEN side = 'SELL' THEN -1 * size
Expand All @@ -534,7 +534,7 @@ export async function getOpenSizeWithFundingIndex(
GROUP BY
f."clobPairId"
)
SELECT
SELECT
input.*,
fiu."fundingIndex",
fiu."effectiveAtHeight" as "fundingIndexHeight"
Expand Down
63 changes: 42 additions & 21 deletions indexer/packages/postgres/src/stores/subaccount-table.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ import Transaction from '../helpers/transaction';
import { getUuid } from '../helpers/uuid';
import SubaccountModel from '../models/subaccount-model';
import {
QueryConfig,
SubaccountFromDatabase,
SubaccountQueryConfig,
SubaccountColumns,
SubaccountCreateObject,
Options,
Ordering,
QueryableField,
QueryConfig,
SubaccountColumns,
SubaccountCreateObject,
SubaccountFromDatabase,
SubaccountQueryConfig,
SubaccountUpdateObject,
} from '../types';

Expand Down Expand Up @@ -107,23 +107,44 @@ export async function getSubaccountsWithTransfers(
createdBeforeOrAtHeight: string,
options: Options = DEFAULT_POSTGRES_OPTIONS,
): Promise<SubaccountFromDatabase[]> {
const queryString: string = `
SELECT *
FROM subaccounts
WHERE id IN (
SELECT "senderSubaccountId" FROM transfers
WHERE "createdAtHeight" <= '${createdBeforeOrAtHeight}'
UNION
SELECT "recipientSubaccountId" FROM transfers
WHERE "createdAtHeight" <= '${createdBeforeOrAtHeight}'
)
`;

const result: {
rows: SubaccountFromDatabase[],
} = await rawQuery(queryString, options);
const txId: number = await Transaction.start();
const txOptions: Options = {
...options,
txId,
};
Comment on lines +111 to +115

@coderabbitai coderabbitai Bot Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Inspect transaction startup and readReplica+txId usage to verify
# getSubaccountsWithTransfers preserves read-replica routing.

set -euo pipefail

echo "### Transaction helper implementation"
fd -t f 'transaction\.ts$' indexer/packages/postgres/src --exec sh -c '
  echo "## $1"
  sed -n "1,220p" "$1"
' sh {}

echo
echo "### Call sites that combine Transaction.start, txId, and readReplica"
rg -n -C4 'Transaction\.start\(|readReplica:\s*true|txId' indexer/packages/postgres/src indexer/services

Repository: dydxprotocol/v4-chain

Length of output: 50378

🏁 Script executed:

# Find how readReplica option is used in queries
rg -n "readReplica" indexer/packages/postgres/src --type ts -A 2 | head -100

# Check if there's special handling for transactions + read-replicas
rg -n "Model\.startTransaction|setupBaseQuery" indexer/packages/postgres/src/stores -A 5 | head -80

# Look for rawQuery implementation to see how readReplica is handled
fd -t f 'raw-query|rawQuery' indexer/packages/postgres/src --exec head -50 {} \;

Repository: dydxprotocol/v4-chain

Length of output: 9236

🏁 Script executed:

# Check the full setupBaseQuery implementation
rg -n "setupBaseQuery" indexer/packages/postgres/src/helpers/stores-helpers.ts -A 15

# Check the full rawQuery implementation  
rg -n "rawQuery" indexer/packages/postgres/src/helpers/stores-helpers.ts -B 5 -A 15

# See if there's any conflict detection or special handling when txId + readReplica are both present
rg -n "txId.*readReplica|readReplica.*txId" indexer/packages/postgres/src -B 3 -A 3

Repository: dydxprotocol/v4-chain

Length of output: 1392

🏁 Script executed:

# Check the full rawQuery implementation to see if there's error handling
cat indexer/packages/postgres/src/helpers/stores-helpers.ts | head -100

# Check what Transaction.start() does and if it can be configured
cat indexer/packages/postgres/src/helpers/transaction.ts

# Find if getSubaccountsWithTransfers is actually called with readReplica option
rg -n "getSubaccountsWithTransfers" indexer --type ts -B 2 -A 2

Repository: dydxprotocol/v4-chain

Length of output: 9162

🏁 Script executed:

# Check what happens in the pnl-ticks-helper when it passes both readReplica and txId
sed -n '60,85p' indexer/services/roundtable/src/helpers/pnl-ticks-helper.ts

# Look at the full getSubaccountsWithTransfers function to understand the transaction lifecycle
sed -n '106,148p' indexer/packages/postgres/src/stores/subaccount-table.ts

Repository: dydxprotocol/v4-chain

Length of output: 2120

🏁 Script executed:

# Check if Model.startTransaction() has any connection configuration
rg -n "Model.startTransaction|knexPrimary.*startTransaction" indexer/packages/postgres/src -B 3 -A 3

# Check what knexReadReplica.getConnection() returns and if it supports transactions
fd -t f 'knex\.ts$' indexer/packages/postgres/src --exec cat {} \;

Repository: dydxprotocol/v4-chain

Length of output: 1926

Transaction started on primary connection conflicts with caller's read-replica routing preference.

Line 111 unconditionally starts a transaction on the primary connection using Transaction.start(), which calls Model.startTransaction() (bound globally to knexPrimary). When callers pass { readReplica: true } in options (as seen at indexer/services/roundtable/src/helpers/pnl-ticks-helper.ts:73), the resulting txOptions contains both txId (primary transaction) and readReplica: true.

In rawQuery(), this creates a routing mismatch: the connection is selected as the read replica, but the transaction object is bound to the primary. This prevents proper transaction isolation on the replica or forces the query to the primary despite the caller's read-replica intent.

Either:

  1. Reject readReplica: true when a transaction ID is provided, or
  2. Do not start a transaction if readReplica: true is passed and no txId is already present.
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@indexer/packages/postgres/src/stores/subaccount-table.ts` around lines 111 -
115, In the subaccount-table.ts file, the code unconditionally calls
Transaction.start() to create a txId bound to the primary connection, then
merges it into txOptions which may already contain readReplica: true from the
caller. This creates a routing conflict. Before calling Transaction.start(),
check if the incoming options parameter contains readReplica: true. If it does,
either throw an error indicating that transactions cannot be used with read
replicas, or skip starting the transaction entirely. Ensure that txOptions never
contains both a txId (primary transaction) and readReplica: true simultaneously.


try {
await rawQuery('SET LOCAL work_mem = \'64MB\';', txOptions);

const result: { rows: SubaccountFromDatabase[] } = await rawQuery(`
SELECT s.*
FROM subaccounts s
WHERE EXISTS (
SELECT 1
FROM transfers t
WHERE t."senderSubaccountId" = s.id
AND t."createdAtHeight" <= :createdBeforeOrAtHeight::bigint
)
OR EXISTS (
SELECT 1
FROM transfers t
WHERE t."recipientSubaccountId" = s.id
AND t."createdAtHeight" <= :createdBeforeOrAtHeight::bigint
)`,
{
...txOptions,
bindings: { createdBeforeOrAtHeight },
});

await Transaction.commit(txId);

return result.rows;
return result.rows;

} catch (error) {
await Transaction.rollback(txId);
throw error;
}
}
Comment on lines +111 to 148

@coderabbitai coderabbitai Bot Jan 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

rg -n -B 2 -A 5 "getSubaccountsWithTransfers" --type ts

Repository: dydxprotocol/v4-chain

Length of output: 7682

🏁 Script executed:

rg -n -B 10 -A 10 'getSubaccountsWithTransfers.*txId' indexer/services/roundtable/src/helpers/pnl-ticks-helper.ts

Repository: dydxprotocol/v4-chain

Length of output: 843

🏁 Script executed:

rg -n -B 30 'getMostRecentPnlTicksForEachAccount\(\)' indexer/services/roundtable/src/helpers/pnl-ticks-helper.ts | head -60

Repository: dydxprotocol/v4-chain

Length of output: 2107

🏁 Script executed:

rg -n -B 5 -A 10 'function getPnlTicksCreateObjects' indexer/services/roundtable/src/helpers/pnl-ticks-helper.ts | head -40

Repository: dydxprotocol/v4-chain

Length of output: 700

🏁 Script executed:

# Check how BlockTable.getLatest handles txId
rg -n -A 15 'export async function getLatest' indexer/packages/postgres/src/stores/block-table.ts

Repository: dydxprotocol/v4-chain

Length of output: 609

🏁 Script executed:

# Check the Transaction class to understand txId behavior
rg -n -A 20 'class Transaction|export.*function start' indexer/packages/postgres/src/utils/transaction.ts | head -50

Repository: dydxprotocol/v4-chain

Length of output: 155

🏁 Script executed:

# Check how setupBaseQuery uses txId to verify if it's reused or created
rg -n -A 15 'setupBaseQuery' indexer/packages/postgres/src/utils/helpers.ts | head -40

Repository: dydxprotocol/v4-chain

Length of output: 151

🏁 Script executed:

# Find Transaction-related files
fd -t f 'transaction' indexer/packages/postgres/src/

Repository: dydxprotocol/v4-chain

Length of output: 393

🏁 Script executed:

# Find helper files
fd -t f 'helper' indexer/packages/postgres/src/ | head -20

Repository: dydxprotocol/v4-chain

Length of output: 484

🏁 Script executed:

# Check the Transaction class implementation
rg -n -A 20 'export.*class Transaction|export.*function start' indexer/packages/postgres/src/helpers/transaction.ts | head -80

Repository: dydxprotocol/v4-chain

Length of output: 913

🏁 Script executed:

# Check how other similar functions in subaccount-table.ts handle the txId parameter
rg -n -B 3 -A 10 'export async function' indexer/packages/postgres/src/stores/subaccount-table.ts | head -100

Repository: dydxprotocol/v4-chain

Length of output: 2900

🏁 Script executed:

# Check how getPnlTicksCreateObjects is called
rg -n -B 5 -A 5 'getPnlTicksCreateObjects' indexer/services/roundtable/src/tasks/pnl-instrumentation.ts

Repository: dydxprotocol/v4-chain

Length of output: 47

🏁 Script executed:

# Search for calls to getPnlTicksCreateObjects across the codebase
rg -n 'getPnlTicksCreateObjects' indexer/

Repository: dydxprotocol/v4-chain

Length of output: 1556

🏁 Script executed:

# Check the create-pnl-ticks.ts to understand the transaction context
rg -n -B 10 -A 10 'getPnlTicksCreateObjects\(txId\)' indexer/services/roundtable/src/tasks/create-pnl-ticks.ts

Repository: dydxprotocol/v4-chain

Length of output: 830

Fix transaction handling: reuse existing txId instead of creating a new transaction.

The function creates a new transaction and ignores any txId provided in options. When called from getPnlTicksCreateObjects(txId) (which is initiated with REPEATABLE_READ isolation), this breaks transaction consistency by starting a separate transaction instead of reusing the caller's context. Other functions in the same file (create, update, findById) correctly use Transaction.get(options.txId) to reuse the transaction.

Update the function to check if options.txId is provided and reuse it; only start a new transaction if none is provided.

🤖 Prompt for AI Agents 
In `@indexer/packages/postgres/src/stores/subaccount-table.ts` around lines 112 -
149, The function currently always calls Transaction.start() and commits/rolls
back that txId which breaks callers that pass an active transaction; change it
to reuse an existing transaction when options.txId is provided: set const
externalTxId = options?.txId; const startedHere = !externalTxId; const txId =
externalTxId ?? await Transaction.start(); build txOptions = { ...options, txId
} and use that for rawQuery, and only call Transaction.commit(txId) or
Transaction.rollback(txId) when startedHere is true (if reusing options.txId, do
not commit/rollback). Keep the existing use of rawQuery and bindings unchanged.


export async function create(
Expand Down Expand Up @@ -229,7 +250,7 @@ export async function findIdsForParentSubaccount(
// (subaccountNumber - parentSubaccountNumber) % MAX_PARENT_SUBACCOUNTS = 0
return subaccounts
.filter((subaccount) => (subaccount.subaccountNumber - parentSubaccount.subaccountNumber) %
MAX_PARENT_SUBACCOUNTS === 0,
MAX_PARENT_SUBACCOUNTS === 0,
)
.map((subaccount) => subaccount.id);
}
Loading
Loading