Skip to content

Commit a00011e

Browse files
thevaizmanclaude
thevaizman
and
claude
committed
feat(dex): add 0x Settler RFQ fills to dex.trades (robust echo-ported decoder, 16 chains)
Promotes 0x Settler plain-RFQ (action selector 0xd92aadfb) maker fills into dex.trades as a new PMM venue (project '0x API', version 'settler') across 16 chains. They were previously absent from dex.trades (which only carried the legacy ExchangeProxy decodes). Method, ported from Dune's echo indexer (zero_ex_settler.rs): - Token identities (maker, makerAsset, takerToken) come from the signed RFQ action's calldata static head; amounts come from the real ERC20 Transfer logs pivoted on the maker, with an exactly-one-per-leg validity gate (drops ambiguous / native-ETH legs). - Identity-grouped maker pivot so distinct fills sharing a byte offset across multiple settler traces in one tx are not collapsed (the naive (tx_hash,p) grouping silently dropped/aliased such fills via arbitrary()). Files: - new macro zeroex_settler_rfq + per-chain zeroex_settler_<chain>_base_trades (16 chains), registered in dex_<chain>_base_trades; dex_info + ethereum seed test added. - settler-txs staging extended to emit tx_from + rfq_input (RFQ-bearing rows only). Verified on-chain (Ethereum + sampled chains): decoder reproduces real fills exactly; 0 overlap with the legacy native 0x venue (no double counting); AMM-routed settler trades correctly excluded (their underlying pool venues already represent them). mode excluded (no dex pipeline). dex_aggregator path unchanged in this PR. CUR2-2843 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 85b4e89 commit a00011e

53 files changed

Lines changed: 857 additions & 2 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎dbt_subprojects/dex/macros/models/_project/zeroex/zeroex_settler_rfq.sql‎

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
{% macro zeroex_settler_rfq(blockchain, project='0x API', version='settler', start_date='2024-07-15') %}
2+
{#- In CI, floor the full-refresh window to the last 14 days, matching the settler-txs staging's own CI floor
3+
(and zeroex_settler_agg) so the logs/transactions joins stay cheap. The staging only carries 14 days in CI,
4+
so this is the binding window regardless; the seed test's fixed-date rows must fall inside it (keep fresh). -#}
5+
{%- if target.name == 'ci' -%}
6+
{%- set start_date = (modules.datetime.date.today() - modules.datetime.timedelta(days=14)).strftime('%Y-%m-%d') -%}
7+
{%- endif -%}
8+
9+
-- Robust 0x Settler plain-RFQ decoder (action selector 0xd92aadfb), ported from Dune's echo indexer
10+
-- (crates/dex-trades-indexer/src/modules/zero_ex_settler.rs).
11+
-- Token identities come from the signed RFQ action's calldata static head; amounts come from the real
12+
-- ERC20 Transfer logs pivoted on the maker (the counterparty guaranteed to move both legs). A leg without
13+
-- exactly one non-zero matching transfer is dropped (ambiguous / non-RFQ false positive), as is native-ETH.
14+
-- Only the plain RFQ action is a 0x-native maker fill with no underlying venue, so only it belongs in
15+
-- dex.trades; AMM/VIP-routed settler actions are represented by their underlying pool venues.
16+
17+
{#- RFQ action static-head layout: byte offset (after the 0xd92aadfb selector at position P, 1-indexed) of
18+
each 32-byte ABI word. The address is the word's last 20 bytes; its leading 12 bytes must be zero. -#}
19+
{%- set off_maker_asset = 36 -%}
20+
{%- set off_maker = 164 -%}
21+
{%- set off_taker_token = 228 -%}
22+
{%- set off_metatxn_taker = 177 -%} {#- executeMetaTxn msgSender address, offset into the top-level input -#}
23+
{%- set zero_word = '0x000000000000000000000000' -%}
24+
{%- set zero_addr = '0x0000000000000000000000000000000000000000' -%}
25+
{%- set native_tokens = '(0x0000000000000000000000000000000000000000, 0xeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee)' -%}
26+
{%- set erc20_transfer_topic = '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef' %}
27+
28+
WITH settler_rfq_txs AS (
29+
-- RFQ-bearing settler calls only: the staging persists rfq_input exactly when the calldata carries 0xd92aadfb.
30+
SELECT
31+
tx_hash,
32+
block_time,
33+
block_number,
34+
method_id,
35+
settler_address,
36+
rfq_input AS input
37+
FROM {{ ref('zeroex_v2_' ~ blockchain ~ '_settler_txs') }}
38+
WHERE rfq_input IS NOT NULL
39+
{% if is_incremental() %}
40+
AND {{ incremental_predicate('block_time') }}
41+
{% else %}
42+
AND block_time >= DATE '{{ start_date }}'
43+
{% endif %}
44+
),
45+
46+
-- Distinct partition key, reused by the transactions + logs joins below so each prunes by partition
47+
-- (block_time + block_number + tx_hash), per zeroex_v2.sql's norm.
48+
rfq_tx_keys AS (
49+
SELECT DISTINCT block_time, block_number, tx_hash FROM settler_rfq_txs
50+
),
51+
52+
-- Root transaction sender. The settler trace's own caller is an intermediary — for plain `execute` it is
53+
-- almost always the AllowanceHolder permit-forwarder (~96% of RFQ execute fills on ethereum), not the user
54+
-- — so the taker must come from the tx-level `from`. Matches zeroex_settler_agg's receiver derivation.
55+
txs AS (
56+
SELECT t.hash AS tx_hash, t."from" AS tx_from
57+
FROM {{ source(blockchain, 'transactions') }} t
58+
JOIN rfq_tx_keys k
59+
ON k.block_time = t.block_time
60+
AND k.block_number = t.block_number
61+
AND k.tx_hash = t.hash
62+
{% if is_incremental() %}
63+
WHERE {{ incremental_predicate('t.block_time') }}
64+
{% else %}
65+
WHERE t.block_time >= DATE '{{ start_date }}'
66+
{% endif %}
67+
),
68+
69+
-- Every byte offset of the plain-RFQ selector in each call (a call can bundle multiple RFQ actions).
70+
-- The position scan is chunked into <=10k-entry windows: Trino caps sequence() at 10,000 entries and
71+
-- settler calldata can exceed 10kB (max observed ~24kB). Chunking is lossless (no position cap) and scans
72+
-- the same total positions as a single sequence — the second UNNEST is sized to each chunk's actual span.
73+
rfq_positions AS (
74+
SELECT t.*, pos.p
75+
FROM settler_rfq_txs t
76+
CROSS JOIN UNNEST(sequence(0, (varbinary_length(t.input) - 4) / 10000)) AS chunk(c)
77+
CROSS JOIN UNNEST(sequence(chunk.c * 10000 + 1, least((chunk.c + 1) * 10000, varbinary_length(t.input) - 3))) AS pos(p)
78+
WHERE varbinary_substring(t.input, pos.p, 4) = 0xd92aadfb
79+
),
80+
81+
-- Slice the three static-head ABI words once (offset arithmetic lives here only).
82+
rfq_words AS (
83+
SELECT
84+
rp.tx_hash, rp.block_time, rp.block_number, rp.settler_address, rp.p,
85+
CASE
86+
WHEN rp.method_id = 0xfd3ad6d4 THEN varbinary_substring(rp.input, {{ off_metatxn_taker }}, 20) -- executeMetaTxn: msgSender (signer), relayer pays gas
87+
ELSE x.tx_from -- execute: tx-level sender, not the AllowanceHolder caller
88+
END AS taker,
89+
varbinary_substring(rp.input, rp.p + {{ off_maker_asset }}, 32) AS maker_asset_word,
90+
varbinary_substring(rp.input, rp.p + {{ off_maker }}, 32) AS maker_word,
91+
varbinary_substring(rp.input, rp.p + {{ off_taker_token }}, 32) AS taker_token_word
92+
FROM rfq_positions rp
93+
JOIN txs x ON x.tx_hash = rp.tx_hash
94+
),
95+
96+
-- Address = the word's last 20 bytes. Reject words whose leading 12 bytes are non-zero (selector
97+
-- collisions inside other actions' payloads) and native-ETH / zero token legs.
98+
rfq_actions AS (
99+
SELECT
100+
tx_hash, block_time, block_number, settler_address, p, taker,
101+
varbinary_substring(maker_asset_word, 13, 20) AS maker_asset,
102+
varbinary_substring(maker_word, 13, 20) AS maker,
103+
varbinary_substring(taker_token_word, 13, 20) AS taker_token
104+
FROM rfq_words
105+
WHERE varbinary_substring(maker_asset_word, 1, 12) = {{ zero_word }}
106+
AND varbinary_substring(maker_word, 1, 12) = {{ zero_word }}
107+
AND varbinary_substring(taker_token_word, 1, 12) = {{ zero_word }}
108+
AND varbinary_substring(maker_word, 13, 20) <> {{ zero_addr }}
109+
AND varbinary_substring(maker_asset_word, 13, 20) NOT IN {{ native_tokens }}
110+
AND varbinary_substring(taker_token_word, 13, 20) NOT IN {{ native_tokens }}
111+
),
112+
113+
-- ERC20 Transfer logs for the RFQ txs, partition-aligned to the settler tx set
114+
-- (block_time + block_number + tx_hash) so the logs scan prunes by partition, per zeroex_v2.sql's norm.
115+
transfers AS (
116+
SELECT
117+
logs.block_number,
118+
logs.tx_hash,
119+
logs.index AS evt_index,
120+
logs.contract_address AS token,
121+
varbinary_substring(logs.topic1, 13, 20) AS transfer_from,
122+
varbinary_substring(logs.topic2, 13, 20) AS transfer_to,
123+
-- CASE-guard the conversion (Trino only evaluates the THEN branch when the WHEN holds): a non-standard
124+
-- Transfer-topic log with >32-byte data would otherwise overflow bytearray_to_uint256. The WHERE filter
125+
-- below is not sufficient on its own — Trino may evaluate this projection before applying it.
126+
CASE WHEN varbinary_length(logs.data) = 32 THEN bytearray_to_uint256(logs.data) END AS amount
127+
FROM {{ source(blockchain, 'logs') }} AS logs
128+
JOIN rfq_tx_keys k
129+
ON k.block_time = logs.block_time
130+
AND k.block_number = logs.block_number
131+
AND k.tx_hash = logs.tx_hash
132+
WHERE logs.topic0 = {{ erc20_transfer_topic }}
133+
-- standard ERC20 value transfers only (uint256 amount is exactly 32 bytes): drops NFT/ERC721 (0-byte
134+
-- data) and non-standard >32-byte Transfer-topic logs. Row-reducer; the overflow guard is the CASE above.
135+
AND varbinary_length(logs.data) = 32
136+
{% if is_incremental() %}
137+
AND {{ incremental_predicate('logs.block_time') }}
138+
{% else %}
139+
AND logs.block_time >= DATE '{{ start_date }}'
140+
{% endif %}
141+
),
142+
143+
-- Maker-pivot match in a single pass over the transfers, keyed on the full RFQ action identity
144+
-- (maker, maker_asset, taker_token) so distinct fills sharing a byte offset across multiple settler
145+
-- traces in one tx are not collapsed. Validity gate: EXACTLY ONE non-zero transfer per leg
146+
-- (makerAsset out of the maker; takerToken in to the maker). evt_index = the maker-leg log index.
147+
legs AS (
148+
SELECT
149+
a.tx_hash, a.p,
150+
a.block_time, a.block_number, a.settler_address, a.taker, a.maker_asset, a.taker_token,
151+
count(*) FILTER (WHERE t.token = a.maker_asset AND t.transfer_from = a.maker) AS maker_n,
152+
count(*) FILTER (WHERE t.token = a.taker_token AND t.transfer_to = a.maker) AS taker_n,
153+
arbitrary(t.amount) FILTER (WHERE t.token = a.maker_asset AND t.transfer_from = a.maker) AS maker_amount,
154+
arbitrary(t.evt_index) FILTER (WHERE t.token = a.maker_asset AND t.transfer_from = a.maker) AS maker_evt_index,
155+
arbitrary(t.amount) FILTER (WHERE t.token = a.taker_token AND t.transfer_to = a.maker) AS taker_amount
156+
FROM rfq_actions a
157+
JOIN transfers t
158+
ON t.tx_hash = a.tx_hash
159+
AND t.block_number = a.block_number
160+
AND t.amount > UINT256 '0'
161+
AND (
162+
(t.token = a.maker_asset AND t.transfer_from = a.maker)
163+
OR (t.token = a.taker_token AND t.transfer_to = a.maker)
164+
)
165+
GROUP BY a.tx_hash, a.p, a.block_time, a.block_number, a.settler_address, a.taker, a.maker_asset, a.taker_token, a.maker
166+
)
167+
168+
SELECT
169+
'{{ blockchain }}' AS blockchain,
170+
'{{ project }}' AS project,
171+
'{{ version }}' AS version,
172+
CAST(date_trunc('month', block_time) AS date) AS block_month,
173+
CAST(date_trunc('day', block_time) AS date) AS block_date,
174+
block_time,
175+
block_number,
176+
maker_amount AS token_bought_amount_raw, -- taker receives the maker's asset
177+
taker_amount AS token_sold_amount_raw, -- taker gives the taker token
178+
maker_asset AS token_bought_address,
179+
taker_token AS token_sold_address,
180+
taker,
181+
CAST(NULL AS varbinary) AS maker, -- PMM venue: settler-side maker not emitted (matches native/clipper)
182+
settler_address AS project_contract_address,
183+
tx_hash,
184+
maker_evt_index AS evt_index
185+
FROM legs
186+
WHERE maker_n = 1
187+
AND taker_n = 1
188+
AND maker_asset <> taker_token
189+
190+
{% endmacro %}

‎dbt_subprojects/dex/macros/models/_project/zeroex/zeroex_settler_txs_cte.sql‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -106,8 +106,11 @@ settler_txs AS (
106106
0x000000000000175a8b9bC6d539B3708EEd92EA6c
107107
)
108108
THEN varbinary_substring(input, varbinary_length(input) - 19, 20)
109-
ELSE taker
110-
END AS taker
109+
ELSE taker
110+
END AS taker,
111+
-- Keep raw calldata only for RFQ-bearing settler calls (plain RFQ action 0xd92aadfb);
112+
-- consumed by the zeroex_settler_rfq macro. NULL otherwise to avoid bloating the staging table.
113+
CASE WHEN varbinary_position(input, 0xd92aadfb) <> 0 THEN input END AS rfq_input
111114
FROM
112115
settler_trace_data
113116
WHERE

‎dbt_subprojects/dex/models/dex_info.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ FROM (VALUES
7070
, ('1inch-LOP', '1inch Limit Order Protocol', 'Aggregator', '1inch')
7171
, ('zeroex', '0x', 'Aggregator', '0xProject')
7272
, ('0x-API', '0x API', 'Aggregator', '0xProject')
73+
, ('0x API', '0x API Settler', 'Direct & Aggregator', '0xProject') -- 0x Settler RFQ PMM venue (version 'settler')
7374
, ('paraswap', 'ParaSwap', 'Aggregator', 'paraswap')
7475
, ('cow_protocol', 'CoW Swap', 'Aggregator', 'CoWSwap')
7576
, ('openocean', 'OpenOcean', 'Aggregator', 'OpenOceanGlobal')

‎dbt_subprojects/dex/models/trades/arbitrum/_schema.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1081,3 +1081,25 @@ models:
10811081
- blockchain
10821082
- token_address
10831083
- block_date
1084+
1085+
- name: zeroex_settler_arbitrum_base_trades
1086+
meta:
1087+
blockchain: arbitrum
1088+
sector: dex
1089+
project: 0x API
1090+
config:
1091+
tags: ["arbitrum", "dex", "trades", "zeroex", "settler"]
1092+
description: "0x Settler plain-RFQ (action 0xd92aadfb) PMM base trades on arbitrum"
1093+
data_tests:
1094+
- dbt_utils.unique_combination_of_columns:
1095+
arguments:
1096+
combination_of_columns:
1097+
- tx_hash
1098+
- evt_index
1099+
columns:
1100+
- name: tx_hash
1101+
data_tests:
1102+
- not_null
1103+
- name: evt_index
1104+
data_tests:
1105+
- not_null

‎dbt_subprojects/dex/models/trades/arbitrum/dex_arbitrum_base_trades.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@
6464
, ref('native_arbitrum_base_trades')
6565
, ref('eulerswap_arbitrum_base_trades')
6666
, ref('zeroex_arbitrum_base_trades')
67+
, ref('zeroex_settler_arbitrum_base_trades')
6768
] %}
6869

6970
{{ dex_base_trades_macro(

‎dbt_subprojects/dex/models/trades/arbitrum/platforms/zeroex_settler_arbitrum_base_trades.sql‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{{
2+
config(
3+
schema = 'zeroex_settler_arbitrum',
4+
alias = 'base_trades',
5+
materialized = 'incremental',
6+
file_format = 'delta',
7+
incremental_strategy = 'merge',
8+
unique_key = ['tx_hash', 'evt_index'],
9+
incremental_predicates = [incremental_predicate('DBT_INTERNAL_DEST.block_time')]
10+
)
11+
}}
12+
13+
{{
14+
zeroex_settler_rfq(
15+
blockchain = 'arbitrum'
16+
)
17+
}}

‎dbt_subprojects/dex/models/trades/avalanche_c/_schema.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -571,3 +571,25 @@ models:
571571
- blockchain
572572
- token_address
573573
- block_date
574+
575+
- name: zeroex_settler_avalanche_c_base_trades
576+
meta:
577+
blockchain: avalanche_c
578+
sector: dex
579+
project: 0x API
580+
config:
581+
tags: ["avalanche_c", "dex", "trades", "zeroex", "settler"]
582+
description: "0x Settler plain-RFQ (action 0xd92aadfb) PMM base trades on avalanche_c"
583+
data_tests:
584+
- dbt_utils.unique_combination_of_columns:
585+
arguments:
586+
combination_of_columns:
587+
- tx_hash
588+
- evt_index
589+
columns:
590+
- name: tx_hash
591+
data_tests:
592+
- not_null
593+
- name: evt_index
594+
data_tests:
595+
- not_null

‎dbt_subprojects/dex/models/trades/avalanche_c/dex_avalanche_c_base_trades.sql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@
3838
, ref('blackhole_v3_avalanche_c_base_trades')
3939
, ref('pharaoh_v3_legacy_avalanche_c_base_trades')
4040
, ref('pharaoh_v3_cl_avalanche_c_base_trades')
41+
, ref('zeroex_settler_avalanche_c_base_trades')
4142
] %}
4243

4344
{{ dex_base_trades_macro(

‎dbt_subprojects/dex/models/trades/avalanche_c/platforms/zeroex_settler_avalanche_c_base_trades.sql‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{{
2+
config(
3+
schema = 'zeroex_settler_avalanche_c',
4+
alias = 'base_trades',
5+
materialized = 'incremental',
6+
file_format = 'delta',
7+
incremental_strategy = 'merge',
8+
unique_key = ['tx_hash', 'evt_index'],
9+
incremental_predicates = [incremental_predicate('DBT_INTERNAL_DEST.block_time')]
10+
)
11+
}}
12+
13+
{{
14+
zeroex_settler_rfq(
15+
blockchain = 'avalanche_c'
16+
)
17+
}}

‎dbt_subprojects/dex/models/trades/base/_schema.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1402,3 +1402,25 @@ models:
14021402
- check_dex_base_trades_seed:
14031403
arguments:
14041404
seed_file: ref('elfomofi_base_base_trades_seed')
1405+
1406+
- name: zeroex_settler_base_base_trades
1407+
meta:
1408+
blockchain: base
1409+
sector: dex
1410+
project: 0x API
1411+
config:
1412+
tags: ["base", "dex", "trades", "zeroex", "settler"]
1413+
description: "0x Settler plain-RFQ (action 0xd92aadfb) PMM base trades on base"
1414+
data_tests:
1415+
- dbt_utils.unique_combination_of_columns:
1416+
arguments:
1417+
combination_of_columns:
1418+
- tx_hash
1419+
- evt_index
1420+
columns:
1421+
- name: tx_hash
1422+
data_tests:
1423+
- not_null
1424+
- name: evt_index
1425+
data_tests:
1426+
- not_null

0 commit comments

Comments
 (0)