Skip to content

Commit 45615c7

Browse files
thevaizmanclaude
thevaizman
and
claude
committed
feat(dex): add 0x Settler RFQ fills to dex.trades (robust echo-ported decoder, 16 chains)
Promotes 0x Settler plain-RFQ (action selector 0xd92aadfb) maker fills into dex.trades as a new PMM venue (project '0x API', version 'settler') across 16 chains. They were previously absent from dex.trades (which only carried the legacy ExchangeProxy decodes). Method, ported from Dune's echo indexer (zero_ex_settler.rs): - Token identities (maker, makerAsset, takerToken) come from the signed RFQ action's calldata static head; amounts come from the real ERC20 Transfer logs pivoted on the maker, with an exactly-one-per-leg validity gate (drops ambiguous / native-ETH legs). - Identity-grouped maker pivot so distinct fills sharing a byte offset across multiple settler traces in one tx are not collapsed (the naive (tx_hash,p) grouping silently dropped/aliased such fills via arbitrary()). Files: - new macro zeroex_settler_rfq + per-chain zeroex_settler_<chain>_base_trades (16 chains), registered in dex_<chain>_base_trades; dex_info + ethereum seed test added. - settler-txs staging extended to emit tx_from + rfq_input (RFQ-bearing rows only). Verified on-chain (Ethereum + sampled chains): decoder reproduces real fills exactly; 0 overlap with the legacy native 0x venue (no double counting); AMM-routed settler trades correctly excluded (their underlying pool venues already represent them). mode excluded (no dex pipeline). dex_aggregator path unchanged in this PR. CUR2-2843 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 85b4e89 commit 45615c7

53 files changed

Lines changed: 714 additions & 2 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎dbt_subprojects/dex/macros/models/_project/zeroex/zeroex_settler_rfq.sql‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
{% macro zeroex_settler_rfq(blockchain, project='0x API', version='settler', start_date='2024-07-15') %}
2+
{%- if target.name == 'ci' -%}
3+
{%- set start_date = (modules.datetime.date.today() - modules.datetime.timedelta(days=14)).strftime('%Y-%m-%d') -%}
4+
{%- endif -%}
5+
6+
-- Robust 0x Settler plain-RFQ decoder (action selector 0xd92aadfb), ported from Dune's echo indexer
7+
-- (crates/dex-trades-indexer/src/modules/zero_ex_settler.rs).
8+
-- Token identities come from the signed RFQ action's calldata static head; amounts come from the real
9+
-- ERC20 Transfer logs pivoted on the maker (the counterparty guaranteed to move both legs). A leg without
10+
-- exactly one non-zero matching transfer is dropped (ambiguous / non-RFQ false positive), as is native-ETH.
11+
-- Only the plain RFQ action is a 0x-native maker fill with no underlying venue, so only it belongs in
12+
-- dex.trades; AMM/VIP-routed settler actions are represented by their underlying pool venues.
13+
14+
{#- RFQ action static-head layout: byte offset (after the 0xd92aadfb selector at position P, 1-indexed) of
15+
each 32-byte ABI word. The address is the word's last 20 bytes; its leading 12 bytes must be zero. -#}
16+
{%- set off_maker_asset = 36 -%}
17+
{%- set off_maker = 164 -%}
18+
{%- set off_taker_token = 228 -%}
19+
{%- set off_metatxn_taker = 177 -%} {#- executeMetaTxn msgSender address, offset into the top-level input -#}
20+
{%- set zero_word = '0x000000000000000000000000' -%}
21+
{%- set zero_addr = '0x0000000000000000000000000000000000000000' -%}
22+
{%- set native_tokens = '(0x0000000000000000000000000000000000000000, 0xeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee)' -%}
23+
{%- set erc20_transfer_topic = '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef' -%}
24+
25+
WITH settler_rfq_txs AS (
26+
-- RFQ-bearing settler calls only: the staging persists rfq_input exactly when the calldata carries 0xd92aadfb.
27+
SELECT
28+
tx_hash,
29+
block_time,
30+
block_number,
31+
method_id,
32+
settler_address,
33+
tx_from,
34+
rfq_input AS input
35+
FROM {{ ref('zeroex_v2_' ~ blockchain ~ '_settler_txs') }}
36+
WHERE rfq_input IS NOT NULL
37+
{% if is_incremental() %}
38+
AND {{ incremental_predicate('block_time') }}
39+
{% else %}
40+
AND block_time >= DATE '{{ start_date }}'
41+
{% endif %}
42+
),
43+
44+
-- Every byte offset of the plain-RFQ selector in each call (a call can bundle multiple RFQ actions).
45+
rfq_positions AS (
46+
SELECT t.*, pos.p
47+
FROM settler_rfq_txs t
48+
CROSS JOIN UNNEST(sequence(1, varbinary_length(t.input) - 3)) AS pos(p)
49+
WHERE varbinary_substring(t.input, pos.p, 4) = 0xd92aadfb
50+
),
51+
52+
-- Slice the three static-head ABI words once (offset arithmetic lives here only).
53+
rfq_words AS (
54+
SELECT
55+
tx_hash, block_time, block_number, settler_address, p,
56+
CASE
57+
WHEN method_id = 0xfd3ad6d4 THEN varbinary_substring(input, {{ off_metatxn_taker }}, 20) -- executeMetaTxn: msgSender
58+
ELSE tx_from -- execute: tx sender
59+
END AS taker,
60+
varbinary_substring(input, p + {{ off_maker_asset }}, 32) AS maker_asset_word,
61+
varbinary_substring(input, p + {{ off_maker }}, 32) AS maker_word,
62+
varbinary_substring(input, p + {{ off_taker_token }}, 32) AS taker_token_word
63+
FROM rfq_positions
64+
),
65+
66+
-- Address = the word's last 20 bytes. Reject words whose leading 12 bytes are non-zero (selector
67+
-- collisions inside other actions' payloads) and native-ETH / zero token legs.
68+
rfq_actions AS (
69+
SELECT
70+
tx_hash, block_time, block_number, settler_address, p, taker,
71+
varbinary_substring(maker_asset_word, 13, 20) AS maker_asset,
72+
varbinary_substring(maker_word, 13, 20) AS maker,
73+
varbinary_substring(taker_token_word, 13, 20) AS taker_token
74+
FROM rfq_words
75+
WHERE varbinary_substring(maker_asset_word, 1, 12) = {{ zero_word }}
76+
AND varbinary_substring(maker_word, 1, 12) = {{ zero_word }}
77+
AND varbinary_substring(taker_token_word, 1, 12) = {{ zero_word }}
78+
AND varbinary_substring(maker_word, 13, 20) <> {{ zero_addr }}
79+
AND varbinary_substring(maker_asset_word, 13, 20) NOT IN {{ native_tokens }}
80+
AND varbinary_substring(taker_token_word, 13, 20) NOT IN {{ native_tokens }}
81+
),
82+
83+
-- ERC20 Transfer logs for the RFQ txs, partition-aligned to the settler tx set
84+
-- (block_time + block_number + tx_hash) so the logs scan prunes by partition, per zeroex_v2.sql's norm.
85+
rfq_tx_keys AS (
86+
SELECT DISTINCT block_time, block_number, tx_hash FROM settler_rfq_txs
87+
),
88+
transfers AS (
89+
SELECT
90+
logs.block_number,
91+
logs.tx_hash,
92+
logs.index AS evt_index,
93+
logs.contract_address AS token,
94+
varbinary_substring(logs.topic1, 13, 20) AS transfer_from,
95+
varbinary_substring(logs.topic2, 13, 20) AS transfer_to,
96+
bytearray_to_uint256(logs.data) AS amount
97+
FROM {{ source(blockchain, 'logs') }} AS logs
98+
JOIN rfq_tx_keys k
99+
ON k.block_time = logs.block_time
100+
AND k.block_number = logs.block_number
101+
AND k.tx_hash = logs.tx_hash
102+
WHERE logs.topic0 = {{ erc20_transfer_topic }}
103+
{% if is_incremental() %}
104+
AND {{ incremental_predicate('logs.block_time') }}
105+
{% else %}
106+
AND logs.block_time >= DATE '{{ start_date }}'
107+
{% endif %}
108+
),
109+
110+
-- Maker-pivot match in a single pass over the transfers, keyed on the full RFQ action identity
111+
-- (maker, maker_asset, taker_token) so distinct fills sharing a byte offset across multiple settler
112+
-- traces in one tx are not collapsed. Validity gate: EXACTLY ONE non-zero transfer per leg
113+
-- (makerAsset out of the maker; takerToken in to the maker). evt_index = the maker-leg log index.
114+
legs AS (
115+
SELECT
116+
a.tx_hash, a.p,
117+
a.block_time, a.block_number, a.settler_address, a.taker, a.maker_asset, a.taker_token,
118+
count(*) FILTER (WHERE t.token = a.maker_asset AND t.transfer_from = a.maker) AS maker_n,
119+
count(*) FILTER (WHERE t.token = a.taker_token AND t.transfer_to = a.maker) AS taker_n,
120+
arbitrary(t.amount) FILTER (WHERE t.token = a.maker_asset AND t.transfer_from = a.maker) AS maker_amount,
121+
arbitrary(t.evt_index) FILTER (WHERE t.token = a.maker_asset AND t.transfer_from = a.maker) AS maker_evt_index,
122+
arbitrary(t.amount) FILTER (WHERE t.token = a.taker_token AND t.transfer_to = a.maker) AS taker_amount
123+
FROM rfq_actions a
124+
JOIN transfers t
125+
ON t.tx_hash = a.tx_hash
126+
AND t.block_number = a.block_number
127+
AND t.amount > UINT256 '0'
128+
AND (
129+
(t.token = a.maker_asset AND t.transfer_from = a.maker)
130+
OR (t.token = a.taker_token AND t.transfer_to = a.maker)
131+
)
132+
GROUP BY a.tx_hash, a.p, a.block_time, a.block_number, a.settler_address, a.taker, a.maker_asset, a.taker_token, a.maker
133+
)
134+
135+
SELECT
136+
'{{ blockchain }}' AS blockchain,
137+
'{{ project }}' AS project,
138+
'{{ version }}' AS version,
139+
CAST(date_trunc('month', block_time) AS date) AS block_month,
140+
CAST(date_trunc('day', block_time) AS date) AS block_date,
141+
block_time,
142+
block_number,
143+
maker_amount AS token_bought_amount_raw, -- taker receives the maker's asset
144+
taker_amount AS token_sold_amount_raw, -- taker gives the taker token
145+
maker_asset AS token_bought_address,
146+
taker_token AS token_sold_address,
147+
taker,
148+
CAST(NULL AS varbinary) AS maker, -- PMM venue: settler-side maker not emitted (matches native/clipper)
149+
settler_address AS project_contract_address,
150+
tx_hash,
151+
maker_evt_index AS evt_index
152+
FROM legs
153+
WHERE maker_n = 1
154+
AND taker_n = 1
155+
AND maker_asset <> taker_token
156+
157+
{% endmacro %}

‎dbt_subprojects/dex/macros/models/_project/zeroex/zeroex_settler_txs_cte.sql‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ settler_trace_data AS (
4848
tr.block_number,
4949
tr.block_time,
5050
tr."to" AS contract_address,
51+
tr."from" AS tx_from,
5152
tr.method_id,
5253
-- Extract tracker information from input data
5354
varbinary_substring(tr.input,varbinary_position(tr.input,0xfd3ad6d4)+132,32) AS tracker,
@@ -106,8 +107,12 @@ settler_txs AS (
106107
0x000000000000175a8b9bC6d539B3708EEd92EA6c
107108
)
108109
THEN varbinary_substring(input, varbinary_length(input) - 19, 20)
109-
ELSE taker
110-
END AS taker
110+
ELSE taker
111+
END AS taker,
112+
tx_from,
113+
-- Keep raw calldata only for RFQ-bearing settler calls (plain RFQ action 0xd92aadfb);
114+
-- consumed by the zeroex_settler_rfq macro. NULL otherwise to avoid bloating the staging table.
115+
CASE WHEN varbinary_position(input, 0xd92aadfb) <> 0 THEN input END AS rfq_input
111116
FROM
112117
settler_trace_data
113118
WHERE

‎dbt_subprojects/dex/models/dex_info.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ FROM (VALUES
7070
, ('1inch-LOP', '1inch Limit Order Protocol', 'Aggregator', '1inch')
7171
, ('zeroex', '0x', 'Aggregator', '0xProject')
7272
, ('0x-API', '0x API', 'Aggregator', '0xProject')
73+
, ('0x API', '0x API Settler', 'Direct & Aggregator', '0xProject') -- 0x Settler RFQ PMM venue (version 'settler')
7374
, ('paraswap', 'ParaSwap', 'Aggregator', 'paraswap')
7475
, ('cow_protocol', 'CoW Swap', 'Aggregator', 'CoWSwap')
7576
, ('openocean', 'OpenOcean', 'Aggregator', 'OpenOceanGlobal')

‎dbt_subprojects/dex/models/trades/arbitrum/_schema.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1081,3 +1081,18 @@ models:
10811081
- blockchain
10821082
- token_address
10831083
- block_date
1084+
1085+
- name: zeroex_settler_arbitrum_base_trades
1086+
meta:
1087+
blockchain: arbitrum
1088+
sector: dex
1089+
project: 0x API
1090+
config:
1091+
tags: ["arbitrum", "dex", "trades", "zeroex", "settler"]
1092+
description: "0x Settler plain-RFQ (action 0xd92aadfb) PMM base trades on arbitrum"
1093+
data_tests:
1094+
- dbt_utils.unique_combination_of_columns:
1095+
arguments:
1096+
combination_of_columns:
1097+
- tx_hash
1098+
- evt_index

‎dbt_subprojects/dex/models/trades/arbitrum/dex_arbitrum_base_trades.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@
6464
, ref('native_arbitrum_base_trades')
6565
, ref('eulerswap_arbitrum_base_trades')
6666
, ref('zeroex_arbitrum_base_trades')
67+
, ref('zeroex_settler_arbitrum_base_trades')
6768
] %}
6869

6970
{{ dex_base_trades_macro(

‎dbt_subprojects/dex/models/trades/arbitrum/platforms/zeroex_settler_arbitrum_base_trades.sql‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{{
2+
config(
3+
schema = 'zeroex_settler_arbitrum',
4+
alias = 'base_trades',
5+
materialized = 'incremental',
6+
file_format = 'delta',
7+
incremental_strategy = 'merge',
8+
unique_key = ['tx_hash', 'evt_index'],
9+
incremental_predicates = [incremental_predicate('DBT_INTERNAL_DEST.block_time')]
10+
)
11+
}}
12+
13+
{{
14+
zeroex_settler_rfq(
15+
blockchain = 'arbitrum'
16+
)
17+
}}

‎dbt_subprojects/dex/models/trades/avalanche_c/_schema.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -571,3 +571,18 @@ models:
571571
- blockchain
572572
- token_address
573573
- block_date
574+
575+
- name: zeroex_settler_avalanche_c_base_trades
576+
meta:
577+
blockchain: avalanche_c
578+
sector: dex
579+
project: 0x API
580+
config:
581+
tags: ["avalanche_c", "dex", "trades", "zeroex", "settler"]
582+
description: "0x Settler plain-RFQ (action 0xd92aadfb) PMM base trades on avalanche_c"
583+
data_tests:
584+
- dbt_utils.unique_combination_of_columns:
585+
arguments:
586+
combination_of_columns:
587+
- tx_hash
588+
- evt_index

‎dbt_subprojects/dex/models/trades/avalanche_c/dex_avalanche_c_base_trades.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@
3838
, ref('blackhole_v3_avalanche_c_base_trades')
3939
, ref('pharaoh_v3_legacy_avalanche_c_base_trades')
4040
, ref('pharaoh_v3_cl_avalanche_c_base_trades')
41+
, ref('zeroex_settler_avalanche_c_base_trades')
4142
] %}
4243

4344
{{ dex_base_trades_macro(

‎dbt_subprojects/dex/models/trades/avalanche_c/platforms/zeroex_settler_avalanche_c_base_trades.sql‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{{
2+
config(
3+
schema = 'zeroex_settler_avalanche_c',
4+
alias = 'base_trades',
5+
materialized = 'incremental',
6+
file_format = 'delta',
7+
incremental_strategy = 'merge',
8+
unique_key = ['tx_hash', 'evt_index'],
9+
incremental_predicates = [incremental_predicate('DBT_INTERNAL_DEST.block_time')]
10+
)
11+
}}
12+
13+
{{
14+
zeroex_settler_rfq(
15+
blockchain = 'avalanche_c'
16+
)
17+
}}

‎dbt_subprojects/dex/models/trades/base/_schema.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1402,3 +1402,18 @@ models:
14021402
- check_dex_base_trades_seed:
14031403
arguments:
14041404
seed_file: ref('elfomofi_base_base_trades_seed')
1405+
1406+
- name: zeroex_settler_base_base_trades
1407+
meta:
1408+
blockchain: base
1409+
sector: dex
1410+
project: 0x API
1411+
config:
1412+
tags: ["base", "dex", "trades", "zeroex", "settler"]
1413+
description: "0x Settler plain-RFQ (action 0xd92aadfb) PMM base trades on base"
1414+
data_tests:
1415+
- dbt_utils.unique_combination_of_columns:
1416+
arguments:
1417+
combination_of_columns:
1418+
- tx_hash
1419+
- evt_index

0 commit comments

Comments
 (0)