Skip to content

Commit 7e77c5c

Browse files
authored
perf(ensindexer): unblock Ponder prefetch on hot tables (#2016)
1 parent 0476de9 commit 7e77c5c

18 files changed

Lines changed: 345 additions & 143 deletions

File tree

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@ensnode/ensdb-sdk": minor
3+
---
4+
5+
`migrated_nodes` renamed to `migrated_nodes_by_parent` and re-keyed by composite `(parentNode, labelHash)` to match the payload of `ENSv1Registry(Old)#NewOwner` events. New sibling `migrated_nodes_by_node` keyed solely by `node` for the three `ENSv1RegistryOld` handlers (`Transfer` / `NewTTL` / `NewResolver`) that emit only `node`. Both rows are written together by the migration helper so each read site addresses whichever key matches its event payload. Schema definitions live in a new `migrated-nodes.schema.ts`.
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
"enssdk": minor
3+
---
4+
5+
Switch composite ids to dash-delimited tuples so Ponder's profile-pattern matcher can decompose them and prefetch hot tables.
6+
7+
Every id constructor (`makeENSv1RegistryId`, `makeENSv2RegistryId`, `makeENSv1VirtualRegistryId`, `makeConcreteRegistryId`, `makeResolverId`, `makeENSv1DomainId`, `makeENSv2DomainId`, `makePermissionsId`, `makePermissionsResourceId`, `makePermissionsUserId`, `makeResolverRecordsId`, `makeRegistrationId`, `makeRenewalId`) now joins its components with `-` instead of CAIP-style mixed `:` / `/` delimiters. `makeENSv2DomainId` no longer wraps the registry contract in CAIP-19 ERC1155 form since the registry already namespaces it. Ponder's matcher only does single-level string-delimiter splits, so the unified `-` tuple is the shape it can decompose to derive prefetch lookup keys from event args.

apps/ensindexer/src/lib/get-this-account-id.ts

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import type { AccountId } from "enssdk";
1+
import type { AccountId, NormalizedAddress } from "enssdk";
22

33
import type { IndexingEngineContext } from "@/lib/indexing-engines/ponder";
44
import type { LogEventBase } from "@/lib/ponder-helpers";
@@ -12,4 +12,9 @@ import type { LogEventBase } from "@/lib/ponder-helpers";
1212
export const getThisAccountId = (
1313
context: IndexingEngineContext,
1414
event: Pick<LogEventBase, "log">,
15-
) => ({ chainId: context.chain.id, address: event.log.address }) satisfies AccountId;
15+
) =>
16+
({
17+
chainId: context.chain.id,
18+
// Ponder provides us a NormalizedAddress, cast here to avoid the minor overhead of (as|to)NormalizedAddress
19+
address: event.log.address as NormalizedAddress,
20+
}) satisfies AccountId;
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
import config from "@/config";
2+
3+
import { type LabelHash, makeSubdomainNode, type Node } from "enssdk";
4+
5+
import { getENSRootChainId } from "@ensnode/datasources";
6+
7+
import { ensIndexerSchema, type IndexingEngineContext } from "@/lib/indexing-engines/ponder";
8+
9+
/**
10+
* Why two tables for one logical "is this node migrated?" check.
11+
*
12+
* The check fires from many Registry handlers, but the event payload differs between them:
13+
* - ENSv1Registry(Old)#NewOwner emits `parentNode` and `labelHash` as separate args.
14+
* - ENSv1RegistryOld#Transfer / NewTTL / NewResolver emit only the post-namehash `node`
15+
*
16+
* Ponder's indexing-cache prefetch path predicts hot-table reads ahead of each event by deriving
17+
* the lookup key from the event's args — but its profile-pattern matcher can only do direct equality
18+
* and single-level string-delimiter splits. It can NOT invert keccak. So a table keyed by the
19+
* post-namehash `node` is unprofileable from a NewOwner event (where `node` is a computed namehash
20+
* of `(parentNode, labelHash)`), and a table keyed by `(parentNode, labelHash)` is unprofileable
21+
* from a Transfer/NewTTL/NewResolver event (which doesn't carry those fields).
22+
*
23+
* Either single-table choice surrenders prefetch on other handlers. Keying solely by
24+
* `(parentNode, labelHash)` would help the NewOwner hot path but disable prefetching on the other
25+
* three handlers, which can't reconstruct that pair from `node` without a reverse-index whose lookup
26+
* key is itself a un-prefetchable namehash.
27+
*
28+
* The two-table layout sidesteps both problems: write _both_ rows on every migration, then have each
29+
* read site address the table whose key matches its event payload. Both reads stay on the prefetch
30+
* hot-path. The cost is one extra "insert on conflict do nothing" per migration, and the storage of
31+
* that information, naturally, doubles. As of 2026-04-29, the size of the migrated_nodes_by_parent
32+
* table is ~1GB, meaning that this optimization will consume an additional ~1GB of storage but
33+
* will result in significantly faster indexing for the ENSv1Registry(Old) events.
34+
*
35+
* See {@link migratedNodeByParent} and {@link migratedNodeByNode} in the ensdb-sdk schema.
36+
*/
37+
38+
const invariant_isENSRootChain = (context: IndexingEngineContext) => {
39+
if (context.chain.id === getENSRootChainId(config.namespace)) return;
40+
41+
throw new Error(
42+
`Invariant: Node migration status is only relevant on the ENS Root Chain, and this function was called in the context of ${context.chain.id}.`,
43+
);
44+
};
45+
46+
/**
47+
* Returns whether `(parentNode, labelHash)` has migrated to the new Registry contract. Used by
48+
* ENSv1RegistryOld#NewOwner where both fields are emitted as event args directly — keyed access
49+
* keeps the read on Ponder's prefetch hot-path.
50+
*/
51+
export async function nodeIsMigratedByParentAndLabel(
52+
context: IndexingEngineContext,
53+
parentNode: Node,
54+
labelHash: LabelHash,
55+
) {
56+
invariant_isENSRootChain(context);
57+
58+
const record = await context.ensDb.find(ensIndexerSchema.migratedNodeByParent, {
59+
parentNode,
60+
labelHash,
61+
});
62+
return record !== null;
63+
}
64+
65+
/**
66+
* Returns whether `node` has migrated to the new Registry contract. Used by
67+
* ENSv1RegistryOld#Transfer/NewTTL/NewResolver where only `node` is emitted as an event arg —
68+
* keyed access on the sibling {@link migratedNodeByNode} table keeps the read on the prefetch
69+
* hot-path even though the composite-key {@link migratedNodeByParent} table can't be addressed
70+
* without a reverse lookup.
71+
*/
72+
export async function nodeIsMigrated(context: IndexingEngineContext, node: Node) {
73+
invariant_isENSRootChain(context);
74+
75+
const record = await context.ensDb.find(ensIndexerSchema.migratedNodeByNode, { node });
76+
return record !== null;
77+
}
78+
79+
/**
80+
* Record that `(parentNode, labelHash)` has migrated to the new Registry contract. Writes both
81+
* the composite-key {@link migratedNodeByParent} row and its sibling {@link migratedNodeByNode}
82+
* index so each downstream read site can address whichever key it can profile against event args.
83+
*/
84+
export async function migrateNode(
85+
context: IndexingEngineContext,
86+
parentNode: Node,
87+
labelHash: LabelHash,
88+
) {
89+
invariant_isENSRootChain(context);
90+
91+
await context.ensDb
92+
.insert(ensIndexerSchema.migratedNodeByParent)
93+
.values({ parentNode, labelHash })
94+
.onConflictDoNothing();
95+
96+
const node = makeSubdomainNode(labelHash, parentNode);
97+
await context.ensDb
98+
.insert(ensIndexerSchema.migratedNodeByNode)
99+
.values({ node })
100+
.onConflictDoNothing();
101+
}

apps/ensindexer/src/lib/protocol-acceleration/registry-migration-status.ts

Lines changed: 0 additions & 36 deletions
This file was deleted.

apps/ensindexer/src/plugins/ensv2/handlers/ensv1/ENSv1Registry.ts

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,10 @@ import {
2929
import { getManagedName } from "@/lib/managed-names";
3030
import { namespaceContract } from "@/lib/plugin-helpers";
3131
import type { EventWithArgs } from "@/lib/ponder-helpers";
32-
import { nodeIsMigrated } from "@/lib/protocol-acceleration/registry-migration-status";
32+
import {
33+
nodeIsMigrated,
34+
nodeIsMigratedByParentAndLabel,
35+
} from "@/lib/protocol-acceleration/migrated-node-db-helpers";
3336

3437
const pluginName = PluginName.ENSv2;
3538

@@ -250,8 +253,11 @@ export default function () {
250253
const { label: labelHash, node: parentNode } = event.args;
251254

252255
// ignore the event on ENSv1RegistryOld if node is migrated to new Registry
253-
const node = makeSubdomainNode(labelHash, parentNode);
254-
const shouldIgnoreEvent = await nodeIsMigrated(context, node);
256+
const shouldIgnoreEvent = await nodeIsMigratedByParentAndLabel(
257+
context,
258+
parentNode,
259+
labelHash,
260+
);
255261
if (shouldIgnoreEvent) return;
256262

257263
return handleNewOwner({ context, event });

apps/ensindexer/src/plugins/ensv2/handlers/ensv2/ETHRegistrar.ts

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ import {
55
makeStorageId,
66
type NormalizedAddress,
77
type TokenId,
8+
toNormalizedAddress,
89
type UnixTimestampBigInt,
910
type Wei,
1011
} from "enssdk";
@@ -35,11 +36,14 @@ async function getRegistrarAndRegistry(context: IndexingEngineContext, event: Lo
3536
const registry: AccountId = {
3637
chainId: context.chain.id,
3738
// ETHRegistrar (this contract) provides a handle to its backing Registry
38-
address: await context.client.readContract({
39-
abi: context.contracts[namespaceContract(pluginName, "ETHRegistrar")].abi,
40-
address: event.log.address,
41-
functionName: "REGISTRY",
42-
}),
39+
// NOTE: viem returns checksummed addresses, need to normalize
40+
address: toNormalizedAddress(
41+
await context.client.readContract({
42+
abi: context.contracts[namespaceContract(pluginName, "ETHRegistrar")].abi,
43+
address: event.log.address,
44+
functionName: "REGISTRY",
45+
}),
46+
),
4347
};
4448

4549
return { registrar, registry };

apps/ensindexer/src/plugins/protocol-acceleration/handlers/ENSv1Registry.ts

Lines changed: 3 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,6 @@
11
import config from "@/config";
22

3-
import {
4-
type LabelHash,
5-
makeENSv1DomainId,
6-
makeSubdomainNode,
7-
type Node,
8-
type NormalizedAddress,
9-
} from "enssdk";
3+
import { type LabelHash, makeENSv1DomainId, type Node, type NormalizedAddress } from "enssdk";
104

115
import { getENSRootChainId } from "@ensnode/datasources";
126
import { PluginName } from "@ensnode/ensnode-sdk";
@@ -17,7 +11,7 @@ import { getManagedName } from "@/lib/managed-names";
1711
import { namespaceContract } from "@/lib/plugin-helpers";
1812
import type { EventWithArgs } from "@/lib/ponder-helpers";
1913
import { ensureDomainResolverRelation } from "@/lib/protocol-acceleration/domain-resolver-relationship-db-helpers";
20-
import { migrateNode, nodeIsMigrated } from "@/lib/protocol-acceleration/registry-migration-status";
14+
import { migrateNode, nodeIsMigrated } from "@/lib/protocol-acceleration/migrated-node-db-helpers";
2115

2216
const ensRootChainId = getENSRootChainId(config.namespace);
2317

@@ -69,8 +63,7 @@ export default function () {
6963
if (context.chain.id !== ensRootChainId) return;
7064

7165
const { label: labelHash, node: parentNode } = event.args;
72-
const node = makeSubdomainNode(labelHash, parentNode);
73-
await migrateNode(context, node);
66+
await migrateNode(context, parentNode, labelHash);
7467
},
7568
);
7669

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,37 @@
11
# Minimal compose for CI integration tests.
22
# Provides only the infrastructure services needed by orchestrator.ts:
33
# devnet (local EVM) and ensdb (database).
4+
#
5+
# NOTE: not using container_name so testcontainers gives it a unique one and avoids collisions
6+
#
7+
# NOTE: ensdb is inlined (not `extends`-ing services/ensdb.yml) so we can override its host
8+
# port to ephemeral without using docker-compose-specific !override syntax. The shared
9+
# services/ensdb.yml binds 5432:5432, which collides with any host-native postgres on a developer
10+
# machine and silently routes orchestrator connections to that native postgres instead of the
11+
# docker container — leading to schema-collision errors. Using "0:5432" lets docker pick an ephemeral
12+
# host port; orchestrator.ts reads it via testcontainers' getMappedPort()
413
services:
514
devnet:
615
extends:
716
file: services/devnet.yml
817
service: devnet
918

1019
ensdb:
11-
extends:
12-
file: services/ensdb.yml
13-
service: ensdb
20+
image: postgres:17
21+
ports:
22+
- "0:5432"
23+
volumes:
24+
- ensdb_data:/var/lib/postgresql/data
1425
env_file:
1526
- path: envs/.env.docker.common
1627
required: true
28+
healthcheck:
29+
test: [ "CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}" ]
30+
interval: 5s
31+
timeout: 5s
32+
retries: 5
33+
start_period: 10s
1734

1835
volumes:
19-
# Docker Compose requires volumes used by services to be declared in each
20-
# compose file that references them — they cannot be inherited via `extends`.
2136
ensdb_data:
2237
driver: local

docs/ensnode.io/src/content/docs/ensdb/concepts/database-schemas.mdx

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -527,16 +527,29 @@ Keyed by `(chainId, resolver, node, key)`, where the composite key segment `(cha
527527

528528
**Relations:** belongs to one `resolver_records` via `(chainId, address, node)`.
529529

530-
#### `migrated_nodes`
530+
#### `migrated_nodes_by_parent`
531531

532-
Tracks the migration status of a node. Due to a security issue, ENS migrated from the `RegistryOld` contract to a new Registry contract. When indexing events, the indexer must ignore any events on `RegistryOld` for domains that have since been migrated to the new Registry.
532+
Tracks the migration status of a node, keyed by `(parentNode, labelHash)`. Due to a security issue, ENS migrated from the `RegistryOld` contract to a new Registry contract. When indexing events, the indexer must ignore any events on `RegistryOld` for domains that have since been migrated to the new Registry.
533533

534-
The set of nodes registered in the (new) Registry contract on the ENS Root Chain is stored here. When an event is encountered on the `RegistryOld` contract, if the relevant node exists in this set, the event should be ignored, as the node is considered migrated.
534+
The set of nodes registered in the (new) Registry contract on the ENS Root Chain is stored here. When a `RegistryOld#NewOwner` event is encountered (which emits both `parentNode` and `labelHash` directly), the relevant row is looked up here; if it exists, the event is ignored.
535535

536536
:::note
537537
This logic is only necessary for the ENS Root Chain — the only chain that includes the Registry migration. This Registry migration tracking is isolated to the Protocol Acceleration plugin. The subgraph plugin implements its own Registry migration logic. By isolating this logic here, the Protocol Acceleration plugin can be run independently of other plugins. The ENSv2 plugin depends on the Protocol Acceleration plugin in order to piggyback on this Registry migration logic.
538538
:::
539539

540+
The composite key is chosen so that Ponder's profile-pattern matcher can decompose it from event args directly, keeping the read on the indexing-cache prefetch hot-path.
541+
542+
| Column | Type | Nullable |
543+
|--------|------|----------|
544+
| `parentNode` | `text` | no |
545+
| `labelHash` | `text` | no |
546+
547+
**Primary key:** `(parentNode, labelHash)`.
548+
549+
#### `migrated_nodes_by_node`
550+
551+
Sibling lookup-by-namehash table for `migrated_nodes_by_parent`, keyed by `node`. The three `RegistryOld` handlers (`Transfer` / `NewTTL` / `NewResolver`) emit only the post-namehash `node` and cannot reconstruct the `(parentNode, labelHash)` pair without an unprofileable reverse lookup. Existence in this table is equivalent to existence in `migrated_nodes_by_parent`; both rows are written together by the migration helper. See `apps/ensindexer/src/lib/protocol-acceleration/migrated-node-db-helpers.ts` for the full rationale.
552+
540553
| Column | Type | Nullable |
541554
|--------|------|----------|
542555
| `node` | `text` | no |

0 commit comments

Comments
 (0)