Skip to content

Commit 6a90020

Browse files
thevaizmanclaude
thevaizman
and
claude
committed
feat(dex): add 0x Settler RFQ fills to dex.trades (robust echo-ported decoder, 16 chains)
Promotes 0x Settler plain-RFQ (action selector 0xd92aadfb) maker fills into dex.trades as a new PMM venue (project '0x API', version 'settler') across 16 chains. They were previously absent from dex.trades (which only carried the legacy ExchangeProxy decodes). Method, ported from Dune's echo indexer (zero_ex_settler.rs): - Token identities (maker, makerAsset, takerToken) come from the signed RFQ action's calldata static head; amounts come from the real ERC20 Transfer logs pivoted on the maker, with an exactly-one-per-leg validity gate (drops ambiguous / native-ETH legs). - Identity-grouped maker pivot so distinct fills sharing a byte offset across multiple settler traces in one tx are not collapsed (the naive (tx_hash,p) grouping silently dropped/aliased such fills via arbitrary()). Files: - new macro zeroex_settler_rfq + per-chain zeroex_settler_<chain>_base_trades (16 chains), registered in dex_<chain>_base_trades; dex_info + ethereum seed test added. - settler-txs staging extended to emit tx_from + rfq_input (RFQ-bearing rows only). Verified on-chain (Ethereum + sampled chains): decoder reproduces real fills exactly; 0 overlap with the legacy native 0x venue (no double counting); AMM-routed settler trades correctly excluded (their underlying pool venues already represent them). mode excluded (no dex pipeline). dex_aggregator path unchanged in this PR. CUR2-2843 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 85b4e89 commit 6a90020

53 files changed

Lines changed: 845 additions & 2 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎dbt_subprojects/dex/macros/models/_project/zeroex/zeroex_settler_rfq.sql‎

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
{% macro zeroex_settler_rfq(blockchain, project='0x API', version='settler', start_date='2024-07-15') %}
2+
{#- Intentionally NO CI date-floor here (unlike the settler-txs staging): the check_dex_base_trades_seed
3+
test validates fixed-date rows, so this model needs a full-window build for the seed to resolve. It reads
4+
the already-materialized staging + a partition-pruned logs join bounded to RFQ-bearing txs, so a
5+
full-window CI build stays cheap. (The expensive non-pushable traces scan is floored once, in the staging.) -#}
6+
7+
-- Robust 0x Settler plain-RFQ decoder (action selector 0xd92aadfb), ported from Dune's echo indexer
8+
-- (crates/dex-trades-indexer/src/modules/zero_ex_settler.rs).
9+
-- Token identities come from the signed RFQ action's calldata static head; amounts come from the real
10+
-- ERC20 Transfer logs pivoted on the maker (the counterparty guaranteed to move both legs). A leg without
11+
-- exactly one non-zero matching transfer is dropped (ambiguous / non-RFQ false positive), as is native-ETH.
12+
-- Only the plain RFQ action is a 0x-native maker fill with no underlying venue, so only it belongs in
13+
-- dex.trades; AMM/VIP-routed settler actions are represented by their underlying pool venues.
14+
15+
{#- RFQ action static-head layout: byte offset (after the 0xd92aadfb selector at position P, 1-indexed) of
16+
each 32-byte ABI word. The address is the word's last 20 bytes; its leading 12 bytes must be zero. -#}
17+
{%- set off_maker_asset = 36 -%}
18+
{%- set off_maker = 164 -%}
19+
{%- set off_taker_token = 228 -%}
20+
{%- set off_metatxn_taker = 177 -%} {#- executeMetaTxn msgSender address, offset into the top-level input -#}
21+
{%- set zero_word = '0x000000000000000000000000' -%}
22+
{%- set zero_addr = '0x0000000000000000000000000000000000000000' -%}
23+
{%- set native_tokens = '(0x0000000000000000000000000000000000000000, 0xeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee)' -%}
24+
{%- set erc20_transfer_topic = '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef' -%}
25+
26+
WITH settler_rfq_txs AS (
27+
-- RFQ-bearing settler calls only: the staging persists rfq_input exactly when the calldata carries 0xd92aadfb.
28+
SELECT
29+
tx_hash,
30+
block_time,
31+
block_number,
32+
method_id,
33+
settler_address,
34+
rfq_input AS input
35+
FROM {{ ref('zeroex_v2_' ~ blockchain ~ '_settler_txs') }}
36+
WHERE rfq_input IS NOT NULL
37+
{% if is_incremental() %}
38+
AND {{ incremental_predicate('block_time') }}
39+
{% else %}
40+
AND block_time >= DATE '{{ start_date }}'
41+
{% endif %}
42+
),
43+
44+
-- Distinct partition key, reused by the transactions + logs joins below so each prunes by partition
45+
-- (block_time + block_number + tx_hash), per zeroex_v2.sql's norm.
46+
rfq_tx_keys AS (
47+
SELECT DISTINCT block_time, block_number, tx_hash FROM settler_rfq_txs
48+
),
49+
50+
-- Root transaction sender. The settler trace's own caller is an intermediary — for plain `execute` it is
51+
-- almost always the AllowanceHolder permit-forwarder (~96% of RFQ execute fills on ethereum), not the user
52+
-- — so the taker must come from the tx-level `from`. Matches zeroex_settler_agg's receiver derivation.
53+
txs AS (
54+
SELECT t.hash AS tx_hash, t."from" AS tx_from
55+
FROM {{ source(blockchain, 'transactions') }} t
56+
JOIN rfq_tx_keys k
57+
ON k.block_time = t.block_time
58+
AND k.block_number = t.block_number
59+
AND k.tx_hash = t.hash
60+
{% if is_incremental() %}
61+
WHERE {{ incremental_predicate('t.block_time') }}
62+
{% else %}
63+
WHERE t.block_time >= DATE '{{ start_date }}'
64+
{% endif %}
65+
),
66+
67+
-- Every byte offset of the plain-RFQ selector in each call (a call can bundle multiple RFQ actions).
68+
rfq_positions AS (
69+
SELECT t.*, pos.p
70+
FROM settler_rfq_txs t
71+
CROSS JOIN UNNEST(sequence(1, varbinary_length(t.input) - 3)) AS pos(p)
72+
WHERE varbinary_substring(t.input, pos.p, 4) = 0xd92aadfb
73+
),
74+
75+
-- Slice the three static-head ABI words once (offset arithmetic lives here only).
76+
rfq_words AS (
77+
SELECT
78+
rp.tx_hash, rp.block_time, rp.block_number, rp.settler_address, rp.p,
79+
CASE
80+
WHEN rp.method_id = 0xfd3ad6d4 THEN varbinary_substring(rp.input, {{ off_metatxn_taker }}, 20) -- executeMetaTxn: msgSender (signer), relayer pays gas
81+
ELSE x.tx_from -- execute: tx-level sender, not the AllowanceHolder caller
82+
END AS taker,
83+
varbinary_substring(rp.input, rp.p + {{ off_maker_asset }}, 32) AS maker_asset_word,
84+
varbinary_substring(rp.input, rp.p + {{ off_maker }}, 32) AS maker_word,
85+
varbinary_substring(rp.input, rp.p + {{ off_taker_token }}, 32) AS taker_token_word
86+
FROM rfq_positions rp
87+
JOIN txs x ON x.tx_hash = rp.tx_hash
88+
),
89+
90+
-- Address = the word's last 20 bytes. Reject words whose leading 12 bytes are non-zero (selector
91+
-- collisions inside other actions' payloads) and native-ETH / zero token legs.
92+
rfq_actions AS (
93+
SELECT
94+
tx_hash, block_time, block_number, settler_address, p, taker,
95+
varbinary_substring(maker_asset_word, 13, 20) AS maker_asset,
96+
varbinary_substring(maker_word, 13, 20) AS maker,
97+
varbinary_substring(taker_token_word, 13, 20) AS taker_token
98+
FROM rfq_words
99+
WHERE varbinary_substring(maker_asset_word, 1, 12) = {{ zero_word }}
100+
AND varbinary_substring(maker_word, 1, 12) = {{ zero_word }}
101+
AND varbinary_substring(taker_token_word, 1, 12) = {{ zero_word }}
102+
AND varbinary_substring(maker_word, 13, 20) <> {{ zero_addr }}
103+
AND varbinary_substring(maker_asset_word, 13, 20) NOT IN {{ native_tokens }}
104+
AND varbinary_substring(taker_token_word, 13, 20) NOT IN {{ native_tokens }}
105+
),
106+
107+
-- ERC20 Transfer logs for the RFQ txs, partition-aligned to the settler tx set
108+
-- (block_time + block_number + tx_hash) so the logs scan prunes by partition, per zeroex_v2.sql's norm.
109+
transfers AS (
110+
SELECT
111+
logs.block_number,
112+
logs.tx_hash,
113+
logs.index AS evt_index,
114+
logs.contract_address AS token,
115+
varbinary_substring(logs.topic1, 13, 20) AS transfer_from,
116+
varbinary_substring(logs.topic2, 13, 20) AS transfer_to,
117+
bytearray_to_uint256(logs.data) AS amount
118+
FROM {{ source(blockchain, 'logs') }} AS logs
119+
JOIN rfq_tx_keys k
120+
ON k.block_time = logs.block_time
121+
AND k.block_number = logs.block_number
122+
AND k.tx_hash = logs.tx_hash
123+
WHERE logs.topic0 = {{ erc20_transfer_topic }}
124+
{% if is_incremental() %}
125+
AND {{ incremental_predicate('logs.block_time') }}
126+
{% else %}
127+
AND logs.block_time >= DATE '{{ start_date }}'
128+
{% endif %}
129+
),
130+
131+
-- Maker-pivot match in a single pass over the transfers, keyed on the full RFQ action identity
132+
-- (maker, maker_asset, taker_token) so distinct fills sharing a byte offset across multiple settler
133+
-- traces in one tx are not collapsed. Validity gate: EXACTLY ONE non-zero transfer per leg
134+
-- (makerAsset out of the maker; takerToken in to the maker). evt_index = the maker-leg log index.
135+
legs AS (
136+
SELECT
137+
a.tx_hash, a.p,
138+
a.block_time, a.block_number, a.settler_address, a.taker, a.maker_asset, a.taker_token,
139+
count(*) FILTER (WHERE t.token = a.maker_asset AND t.transfer_from = a.maker) AS maker_n,
140+
count(*) FILTER (WHERE t.token = a.taker_token AND t.transfer_to = a.maker) AS taker_n,
141+
arbitrary(t.amount) FILTER (WHERE t.token = a.maker_asset AND t.transfer_from = a.maker) AS maker_amount,
142+
arbitrary(t.evt_index) FILTER (WHERE t.token = a.maker_asset AND t.transfer_from = a.maker) AS maker_evt_index,
143+
arbitrary(t.amount) FILTER (WHERE t.token = a.taker_token AND t.transfer_to = a.maker) AS taker_amount
144+
FROM rfq_actions a
145+
JOIN transfers t
146+
ON t.tx_hash = a.tx_hash
147+
AND t.block_number = a.block_number
148+
AND t.amount > UINT256 '0'
149+
AND (
150+
(t.token = a.maker_asset AND t.transfer_from = a.maker)
151+
OR (t.token = a.taker_token AND t.transfer_to = a.maker)
152+
)
153+
GROUP BY a.tx_hash, a.p, a.block_time, a.block_number, a.settler_address, a.taker, a.maker_asset, a.taker_token, a.maker
154+
)
155+
156+
SELECT
157+
'{{ blockchain }}' AS blockchain,
158+
'{{ project }}' AS project,
159+
'{{ version }}' AS version,
160+
CAST(date_trunc('month', block_time) AS date) AS block_month,
161+
CAST(date_trunc('day', block_time) AS date) AS block_date,
162+
block_time,
163+
block_number,
164+
maker_amount AS token_bought_amount_raw, -- taker receives the maker's asset
165+
taker_amount AS token_sold_amount_raw, -- taker gives the taker token
166+
maker_asset AS token_bought_address,
167+
taker_token AS token_sold_address,
168+
taker,
169+
CAST(NULL AS varbinary) AS maker, -- PMM venue: settler-side maker not emitted (matches native/clipper)
170+
settler_address AS project_contract_address,
171+
tx_hash,
172+
maker_evt_index AS evt_index
173+
FROM legs
174+
WHERE maker_n = 1
175+
AND taker_n = 1
176+
AND maker_asset <> taker_token
177+
178+
{% endmacro %}

‎dbt_subprojects/dex/macros/models/_project/zeroex/zeroex_settler_txs_cte.sql‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -106,8 +106,11 @@ settler_txs AS (
106106
0x000000000000175a8b9bC6d539B3708EEd92EA6c
107107
)
108108
THEN varbinary_substring(input, varbinary_length(input) - 19, 20)
109-
ELSE taker
110-
END AS taker
109+
ELSE taker
110+
END AS taker,
111+
-- Keep raw calldata only for RFQ-bearing settler calls (plain RFQ action 0xd92aadfb);
112+
-- consumed by the zeroex_settler_rfq macro. NULL otherwise to avoid bloating the staging table.
113+
CASE WHEN varbinary_position(input, 0xd92aadfb) <> 0 THEN input END AS rfq_input
111114
FROM
112115
settler_trace_data
113116
WHERE

‎dbt_subprojects/dex/models/dex_info.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ FROM (VALUES
7070
, ('1inch-LOP', '1inch Limit Order Protocol', 'Aggregator', '1inch')
7171
, ('zeroex', '0x', 'Aggregator', '0xProject')
7272
, ('0x-API', '0x API', 'Aggregator', '0xProject')
73+
, ('0x API', '0x API Settler', 'Direct & Aggregator', '0xProject') -- 0x Settler RFQ PMM venue (version 'settler')
7374
, ('paraswap', 'ParaSwap', 'Aggregator', 'paraswap')
7475
, ('cow_protocol', 'CoW Swap', 'Aggregator', 'CoWSwap')
7576
, ('openocean', 'OpenOcean', 'Aggregator', 'OpenOceanGlobal')

‎dbt_subprojects/dex/models/trades/arbitrum/_schema.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1081,3 +1081,25 @@ models:
10811081
- blockchain
10821082
- token_address
10831083
- block_date
1084+
1085+
- name: zeroex_settler_arbitrum_base_trades
1086+
meta:
1087+
blockchain: arbitrum
1088+
sector: dex
1089+
project: 0x API
1090+
config:
1091+
tags: ["arbitrum", "dex", "trades", "zeroex", "settler"]
1092+
description: "0x Settler plain-RFQ (action 0xd92aadfb) PMM base trades on arbitrum"
1093+
data_tests:
1094+
- dbt_utils.unique_combination_of_columns:
1095+
arguments:
1096+
combination_of_columns:
1097+
- tx_hash
1098+
- evt_index
1099+
columns:
1100+
- name: tx_hash
1101+
data_tests:
1102+
- not_null
1103+
- name: evt_index
1104+
data_tests:
1105+
- not_null

‎dbt_subprojects/dex/models/trades/arbitrum/dex_arbitrum_base_trades.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@
6464
, ref('native_arbitrum_base_trades')
6565
, ref('eulerswap_arbitrum_base_trades')
6666
, ref('zeroex_arbitrum_base_trades')
67+
, ref('zeroex_settler_arbitrum_base_trades')
6768
] %}
6869

6970
{{ dex_base_trades_macro(

‎dbt_subprojects/dex/models/trades/arbitrum/platforms/zeroex_settler_arbitrum_base_trades.sql‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{{
2+
config(
3+
schema = 'zeroex_settler_arbitrum',
4+
alias = 'base_trades',
5+
materialized = 'incremental',
6+
file_format = 'delta',
7+
incremental_strategy = 'merge',
8+
unique_key = ['tx_hash', 'evt_index'],
9+
incremental_predicates = [incremental_predicate('DBT_INTERNAL_DEST.block_time')]
10+
)
11+
}}
12+
13+
{{
14+
zeroex_settler_rfq(
15+
blockchain = 'arbitrum'
16+
)
17+
}}

‎dbt_subprojects/dex/models/trades/avalanche_c/_schema.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -571,3 +571,25 @@ models:
571571
- blockchain
572572
- token_address
573573
- block_date
574+
575+
- name: zeroex_settler_avalanche_c_base_trades
576+
meta:
577+
blockchain: avalanche_c
578+
sector: dex
579+
project: 0x API
580+
config:
581+
tags: ["avalanche_c", "dex", "trades", "zeroex", "settler"]
582+
description: "0x Settler plain-RFQ (action 0xd92aadfb) PMM base trades on avalanche_c"
583+
data_tests:
584+
- dbt_utils.unique_combination_of_columns:
585+
arguments:
586+
combination_of_columns:
587+
- tx_hash
588+
- evt_index
589+
columns:
590+
- name: tx_hash
591+
data_tests:
592+
- not_null
593+
- name: evt_index
594+
data_tests:
595+
- not_null

‎dbt_subprojects/dex/models/trades/avalanche_c/dex_avalanche_c_base_trades.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@
3838
, ref('blackhole_v3_avalanche_c_base_trades')
3939
, ref('pharaoh_v3_legacy_avalanche_c_base_trades')
4040
, ref('pharaoh_v3_cl_avalanche_c_base_trades')
41+
, ref('zeroex_settler_avalanche_c_base_trades')
4142
] %}
4243

4344
{{ dex_base_trades_macro(

‎dbt_subprojects/dex/models/trades/avalanche_c/platforms/zeroex_settler_avalanche_c_base_trades.sql‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{{
2+
config(
3+
schema = 'zeroex_settler_avalanche_c',
4+
alias = 'base_trades',
5+
materialized = 'incremental',
6+
file_format = 'delta',
7+
incremental_strategy = 'merge',
8+
unique_key = ['tx_hash', 'evt_index'],
9+
incremental_predicates = [incremental_predicate('DBT_INTERNAL_DEST.block_time')]
10+
)
11+
}}
12+
13+
{{
14+
zeroex_settler_rfq(
15+
blockchain = 'avalanche_c'
16+
)
17+
}}

‎dbt_subprojects/dex/models/trades/base/_schema.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1402,3 +1402,25 @@ models:
14021402
- check_dex_base_trades_seed:
14031403
arguments:
14041404
seed_file: ref('elfomofi_base_base_trades_seed')
1405+
1406+
- name: zeroex_settler_base_base_trades
1407+
meta:
1408+
blockchain: base
1409+
sector: dex
1410+
project: 0x API
1411+
config:
1412+
tags: ["base", "dex", "trades", "zeroex", "settler"]
1413+
description: "0x Settler plain-RFQ (action 0xd92aadfb) PMM base trades on base"
1414+
data_tests:
1415+
- dbt_utils.unique_combination_of_columns:
1416+
arguments:
1417+
combination_of_columns:
1418+
- tx_hash
1419+
- evt_index
1420+
columns:
1421+
- name: tx_hash
1422+
data_tests:
1423+
- not_null
1424+
- name: evt_index
1425+
data_tests:
1426+
- not_null

0 commit comments

Comments
 (0)