feat: index MerkleDistributor pools, roots, and claims#186
Open
zer0stars wants to merge 11 commits into
Open
Conversation
- Guard the HTTP tree fetcher against SSRF via redirects by re-validating every redirect target against the HTTPS scheme and host allowlist - Apply both after and before cursors in GetMerkleRewards instead of dropping before when after is set - Reject MerkleDistributor event poolId/week arguments that overflow int64 - Put the MERKLE_DISTRIBUTOR_ADDR TODO comment on its own line in settings.sample.yaml
Review fixes for the Merkle claims indexing feature: - Replace the per-leaf claim Upsert loop in handleRootSet with a single INSERT ... SELECT unnest(...) ON CONFLICT statement; claimed_at and claim_tx are still preserved on conflict since they are not in the SET list. Covered by a new 150-leaf handler test that also exercises redelivery through the conflict path. - Replace the single-column merkle_claims account index with a composite (account, pool_id, epoch DESC) index matching the merkleRewards query and cursor order, and add a partial index on account WHERE claimed_at IS NULL for the unclaimed-by-account hot path. Indexes do not affect SQLBoiler models, so no regen needed. - Resolve MerkleReward.pool through a new MerklePoolByID dataloader instead of one query per reward row. - Log fetchDurationMs and dbWriteDurationMs when a root is set, and document why the tree-file/event root mismatch path intentionally returns an error (Kafka redelivery is the operator alert). - Hoist the tree fetcher's 10s timeout into merkle.FetchTimeout with a comment noting it spans the full body read of up to 50 MiB. - Document and pin the RootSet root wire format: even as an indexed parameter, bytes32 decodes via abi.ParseTopicsIntoMap/toGoType to a [32]byte and serializes as a JSON number array (verified against go-ethereum v1.15.11, contract-event-processor's version), so the Root field stays [32]byte and the consumer needs no wire change.
rewards(user).totalTokens now merges legacy per-device reward sums with merkle_claims amounts for the account, so user-level earnings keep growing after the merkle cutover. Merkle amounts count as earned at RootSet, claimed or not. History edges and totalCount stay legacy-only; per-epoch merkle data lives in the merkleRewards query.
Member
Author
|
Added in latest push: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
proofsURI(HTTPS-only, host allowlist, redirect-guarded, 50MB cap), recomputes the merkle root from leaves and verifies it byte-equal to the on-chain root before serving anything, then upserts all leaves+proofs in one transaction (claim state preserved on root overwrite).SELECT FOR UPDATE(Kafka redelivery safe).merklePools,merklePool(poolId),merkleRewards(account, poolId, claimed)— Relay connections; serves everything a client needs to callclaim()/claimBatch().pkg/merkletreecopied from rewards-api (swap to module import once published).Companion PRs: dimo-rewards (contract), rewards-api (producer), contract-event-processor (config), transactions (SDK), dimo-driver (UI).
Test plan
go test ./...exit 0MERKLE_DISTRIBUTOR_ADDR+ tree host after deployment (placeholders in chart values)