Skip to content

Commit 8f53b82

Browse files
feat(eth-indexer): event-driven AUDIO balance indexer (#849)
## Summary - New `eth-indexer` entrypoint that mirrors `solana-indexer`'s shape: `New(cfg)` + `Start(ctx) error` + `Close()` + `GetHealth()`, plus a sibling fiber `Server` on `:1325` exposing `GET /eth/health`. - Event-driven instead of polling: opens a persistent WSS subscription via `eth_subscribe` to the AUDIO Transfer log topic. On each event, joins from/to against `users.wallet ∪ chain='eth' associated_wallets`, fans out `balanceOf` + `totalStakedFor` + `getTotalDelegatorStake` in parallel for matches, and upserts the sum into the new `eth_wallet_balances` table. - Resumable via `eth_indexer_checkpoints` — on (re)connect we `eth_getLogs` the gap (9K-block chunks) and replay. This is the api-side replacement for the discovery-provider's `cache_user_balance` job that currently feeds `associated_wallets_balance` on `GET /v1/users/handle/...`. Same three contract reads, same result, but live via WS instead of 30s polling. ## Why event-driven For a baseline of ~50K AUDIO transfers/day across ~5M tracked wallets, this approach issues roughly one balance read per actual on-chain transfer — vs. hundreds of thousands of speculative `eth_getBalance` calls per day for a periodic poll-everyone approach. RPC cost scales with on-chain activity, not user count. ## Files - [eth/indexer/eth_indexer.go](https://github.com/AudiusProject/api/blob/api/eth-balance-indexer/eth/indexer/eth_indexer.go) — `EthIndexer` struct with WS subscription loop, backfill, fan-out balance reads, upsert, checkpoint - [eth/indexer/server.go](https://github.com/AudiusProject/api/blob/api/eth-balance-indexer/eth/indexer/server.go) — fiber app on `:1325`, `GET /eth/health` - [ddl/migrations/0203_eth_wallet_balances.sql](https://github.com/AudiusProject/api/blob/api/eth-balance-indexer/ddl/migrations/0203_eth_wallet_balances.sql) — `eth_wallet_balances` + `eth_indexer_checkpoints` - [config/config.go](https://github.com/AudiusProject/api/blob/api/eth-balance-indexer/config/config.go) — env wiring + mainnet defaults - [main.go](https://github.com/AudiusProject/api/blob/api/eth-balance-indexer/main.go) — `case "eth-indexer":` parallels `case "solana-indexer":` - [cmd/eth_smoke/main.go](https://github.com/AudiusProject/api/blob/api/eth-balance-indexer/cmd/eth_smoke/main.go) — one-shot CLI that runs the same three contract reads for a given holder, for ops debugging ## Config | env var | required | default | |---|---|---| | `ethRpcUrl` | yes | — | | `ethWsUrl` | no | auto-derived from `ethRpcUrl` (https→wss) | | `ethAudioContractAddress` | no | `0x18aAA7115705e8be94bfFEbDE57Af9BFc265B998` | | `ethStakingContractAddress` | no | `0xe6D97B2099F142513be7A2a068bE040656Ae4591` | | `ethDelegateManagerContractAddress` | no | `0x4d7968ebfD390D5E7926Cb3587C39eFf2F9FB225` | If `ethRpcUrl`/`ethWsUrl` are unset the indexer logs a warning and idles until shutdown — safe to deploy without a provider key. ## Test plan - [x] `go build ./...` clean - [x] `go vet ./...` clean - [x] Smoke test of the 3 contract reads via `cmd/eth_smoke` against mainnet: - rayjacobson primary (`0x7d273…b060`) → 0/0/0 ✓ (matches discovery-provider `balance: "0"`) - Staking contract self (`0xe6D9…4591`) → 247,024,527 AUDIO ✓ - [x] End-to-end local run: - Seeded a known active AUDIO holder as a tracked associated_wallet - Pre-set checkpoint 9000 blocks back to force backfill - Indexer found a Transfer involving the seed wallet, called all 3 contracts, upserted `80975640000000000000000` wei - Cross-checked the persisted value with `cmd/eth_smoke` against the same holder — bytes match exactly - `GET /eth/health` returned `connected: true`, advanced checkpoint, correct tracked/cached counts - [ ] Deploy plan: set `ethRpcUrl` (and optionally `ethWsUrl`) in stage; let it run for ~24h and confirm `eth_wallet_balances` populates as AUDIO transfers occur ## Out of scope / follow-ups - This populates `eth_wallet_balances` but does not yet roll it up into a `/v1/users/...` API response field. That can be a follow-up PR (or a SQL view) once the data is flowing. - No on-demand "force refresh wallet X" endpoint yet — every refresh is event-triggered. We can add a manual hook if support tickets demand it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 94100cf commit 8f53b82

5 files changed

Lines changed: 735 additions & 9 deletions

File tree

config/config.go

Lines changed: 51 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -37,15 +37,31 @@ type Config struct {
3737
OpenAudioURLs []string
3838
ChainId string
3939
BirdeyeToken string
40-
SolanaIndexerWorkers int
41-
SolanaIndexerRetryInterval time.Duration
42-
CommsMessagePush bool
43-
AudiusdChainID uint
44-
AudiusdEntityManagerAddress string
45-
AudiusAppUrl string
46-
RewardCodeAuthorizedKeys []string
47-
LaunchpadDeterministicSecret string
48-
UnsplashKeys []string
40+
// HTTP(S) JSON-RPC endpoint for the Ethereum mainnet provider (e.g. an
41+
// Alchemy URL). Used by the eth-indexer for backfill `eth_getLogs` and
42+
// targeted `balanceOf` reads. If empty, the indexer is a no-op.
43+
EthRpcUrl string
44+
// WebSocket JSON-RPC endpoint used by the eth-indexer for live
45+
// subscriptions to AUDIO Transfer events. Auto-derived from EthRpcUrl
46+
// (https:// -> wss://) if left unset.
47+
EthWsUrl string
48+
// AUDIO ERC-20 contract address on Ethereum mainnet. Override only when
49+
// pointing at a non-mainnet deployment.
50+
EthAudioContractAddress string
51+
// Audius Staking proxy address — used to read totalStakedFor(holder).
52+
EthStakingContractAddress string
53+
// Audius DelegateManager address — used to read
54+
// getTotalDelegatorStake(holder).
55+
EthDelegateManagerContractAddress string
56+
SolanaIndexerWorkers int
57+
SolanaIndexerRetryInterval time.Duration
58+
CommsMessagePush bool
59+
AudiusdChainID uint
60+
AudiusdEntityManagerAddress string
61+
AudiusAppUrl string
62+
RewardCodeAuthorizedKeys []string
63+
LaunchpadDeterministicSecret string
64+
UnsplashKeys []string
4965
// Nodes that volunteer as STORE_ALL nodes and are always included in mirrors lists
5066
StoreAllNodes []string
5167
// Nodes that are truly dead and should not be included in rendezvous
@@ -81,6 +97,11 @@ var Cfg = Config{
8197
AudiusdURL: os.Getenv("audiusdUrl"),
8298
OpenAudioURLs: []string{},
8399
BirdeyeToken: os.Getenv("birdeyeToken"),
100+
EthRpcUrl: os.Getenv("ethRpcUrl"),
101+
EthWsUrl: os.Getenv("ethWsUrl"),
102+
EthAudioContractAddress: os.Getenv("ethAudioContractAddress"),
103+
EthStakingContractAddress: os.Getenv("ethStakingContractAddress"),
104+
EthDelegateManagerContractAddress: os.Getenv("ethDelegateManagerContractAddress"),
84105
SolanaIndexerWorkers: 50,
85106
SolanaIndexerRetryInterval: 5 * time.Minute,
86107
CommsMessagePush: true,
@@ -100,6 +121,27 @@ func init() {
100121

101122
Cfg.SolanaConfig = NewSolanaConfig()
102123

124+
// Default AUDIO ERC-20 + staking + delegate manager to mainnet addresses
125+
// (from packages/sdk/src/sdk/config/production.ts).
126+
if Cfg.EthAudioContractAddress == "" {
127+
Cfg.EthAudioContractAddress = "0x18aAA7115705e8be94bfFEbDE57Af9BFc265B998"
128+
}
129+
if Cfg.EthStakingContractAddress == "" {
130+
Cfg.EthStakingContractAddress = "0xe6D97B2099F142513be7A2a068bE040656Ae4591"
131+
}
132+
if Cfg.EthDelegateManagerContractAddress == "" {
133+
Cfg.EthDelegateManagerContractAddress = "0x4d7968ebfD390D5E7926Cb3587C39eFf2F9FB225"
134+
}
135+
// Derive WS endpoint from the HTTP endpoint if not set explicitly.
136+
if Cfg.EthWsUrl == "" && Cfg.EthRpcUrl != "" {
137+
switch {
138+
case strings.HasPrefix(Cfg.EthRpcUrl, "https://"):
139+
Cfg.EthWsUrl = "wss://" + strings.TrimPrefix(Cfg.EthRpcUrl, "https://")
140+
case strings.HasPrefix(Cfg.EthRpcUrl, "http://"):
141+
Cfg.EthWsUrl = "ws://" + strings.TrimPrefix(Cfg.EthRpcUrl, "http://")
142+
}
143+
}
144+
103145
switch env := os.Getenv("ENV"); env {
104146
case "dev":
105147
fallthrough
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
BEGIN;
2+
3+
CREATE TABLE IF NOT EXISTS eth_wallet_balances (
4+
wallet TEXT PRIMARY KEY,
5+
balance NUMERIC NOT NULL DEFAULT 0,
6+
blocknumber BIGINT,
7+
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
8+
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
9+
);
10+
COMMENT ON TABLE eth_wallet_balances IS 'AUDIO ERC-20 balances (in wei) for tracked Ethereum wallets — primary users.wallet and chain=eth associated_wallets. Maintained event-driven by the eth-indexer (WebSocket subscription to the AUDIO Transfer topic, targeted balanceOf reads).';
11+
12+
CREATE INDEX IF NOT EXISTS eth_wallet_balances_updated_at_idx ON eth_wallet_balances (updated_at);
13+
COMMENT ON INDEX eth_wallet_balances_updated_at_idx IS 'Supports staleness queries / catch-up sweeps.';
14+
15+
CREATE TABLE IF NOT EXISTS eth_indexer_checkpoints (
16+
name TEXT PRIMARY KEY,
17+
last_block BIGINT NOT NULL,
18+
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
19+
);
20+
COMMENT ON TABLE eth_indexer_checkpoints IS 'Resumable backfill checkpoints for the eth-indexer (last block whose Transfer events have been processed, keyed by subscription name).';
21+
22+
COMMIT;

0 commit comments

Comments
 (0)