Skip to content

Commit bb35fe2

Browse files
sirandreww-starkwaresirandreww-starkware
committed
apollo_consensus_orchestrator: add SNIP-35 integration TODO doc
1 parent d4d6b73 commit bb35fe2

1 file changed

Lines changed: 174 additions & 0 deletions

File tree

‎crates/apollo_consensus_orchestrator/src/snip35/INTEGRATION_TODO.md‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
# SNIP-35 — centralized integration TODO
2+
3+
> **Status:** Working checklist. Capture of the plan to wire `fee_proposal_fri` from the centralized Python feeder gateway into the Rust sequencer.
4+
>
5+
> **Verified against:** `starkware/` synced as of 2026-05-05.
6+
7+
## Context
8+
9+
The Rust SNIP-35 stack already adds `fee_proposal_fri: Option<GasPrice>` everywhere it needs to live (block header, storage, sync proto, consensus proto, sliding window). Today the centralized feeder gateway doesn't emit this field, so the Rust client at `apollo_starknet_client/src/reader/objects/block.rs:345` hardcodes `fee_proposal_fri: None` on the conversion from gateway JSON to `BlockHeaderWithoutHash`.
10+
11+
To close the loop, the centralized Python feeder gateway needs to start emitting `fee_proposal_fri` in its block JSON, and the Rust client needs to start reading it.
12+
13+
**Hash invariant:** `fee_proposal_fri` must NOT enter `BlockHash` or `PartialBlockHash`. It only enters `ProposalCommitment` (the consensus-voted hash) via `proposal_commitment_from`. On the centralized side, this means the field must NOT be propagated into `BlockHashHeader`.
14+
15+
**Backward-compat invariant:** existing requests to the feeder gateway must continue to receive the exact same JSON shape they receive today. New SNIP-35 data is gated behind a new opt-in toggle `withSnip35Info`. Existing callers see no change unless they explicitly opt in.
16+
17+
**Architectural decision: independence.** SNIP-35 data is stored, served, and toggled **independently** of `FeeMarketInfo`. They're related (SNIP-35 uses `next_l2_gas_price` as a floor, etc.) but they're produced by different mechanisms (EIP-1559 measurement vs. proposer declaration) and consumed by different flows. Keeping them separate avoids touching `FeeMarketInfo`'s production schema and keeps strip-logic clean (two independent toggles, no chained logic). A new `Snip35Info` dataclass holds `fee_proposal_fri`; `FeeMarketInfo` is left untouched.
18+
19+
| Query params | Response includes |
20+
|---|---|
21+
| (default — no flags) | none of the new data (existing default, unchanged) |
22+
| `withFeeMarketInfo=true` | `l2_gas_consumed`, `next_l2_gas_price` (existing behavior, unchanged) |
23+
| `withSnip35Info=true` | `fee_proposal_fri` |
24+
| both `=true` | all three |
25+
26+
## Where the value comes from
27+
28+
`fee_proposal_fri` is *stated* by the proposer at the moment of proposing a block — not independently derived from chain state. The SNIP-35 clamp formula (`compute_fee_proposal` in `crates/apollo_consensus_orchestrator/src/snip35/mod.rs`) runs only in the active proposer.
29+
30+
The proposer is **always an Apollo (Rust consensus) node** — round-robin selected. The centralized Python is never the proposer; it's record-only. So the centralized Python side **does not compute anything**. It just receives the stated value through the cende-blob pipeline and serves it via the feeder gateway.
31+
32+
End-to-end data flow:
33+
34+
```
35+
Apollo (Rust): compute_fee_proposal → states fee_proposal_fri in ProposalInit
36+
└─ also writes Snip35Info { fee_proposal_fri } into the cende blob
37+
↓
38+
[centralized storage]
39+
↓
40+
CENDE recorder (Python): reads blob, persists Snip35Info to its own storage
41+
↓
42+
Feeder gateway (Python): on withSnip35Info=true, includes fee_proposal_fri in JSON
43+
↓
44+
apollo_starknet_client (Rust consumer): reads JSON into BlockHeaderWithoutHash
45+
```
46+
47+
Independence means there are now two parallel data structures (`FeeMarketInfo`, `Snip35Info`) with parallel writers, parallel storage, and parallel response handling. Not more complex per-thing — just symmetric copies of the existing fee-market plumbing for the new data.
48+
49+
## The two repos
50+
51+
### 1. `starkware/` — centralized Python monorepo
52+
53+
Path: `/home/andrew/workspace/starkware/`. Files that need to change:
54+
55+
| Path | What |
56+
|---|---|
57+
| `src/starkware/starknet/services/batcher/shared_objects.py` (near line 757) | New `Snip35Info` dataclass alongside `FeeMarketInfo`. New `IndexedDBObject` with its own storage namespace. `FeeMarketInfo` itself is **unchanged**. |
58+
| `src/starkware/starknet/services/batcher/shared_objects.py` (near line 361) | New `BatchCreated.get_snip35_info(...)` method mirroring the existing `get_fee_market_info`. |
59+
| `src/starkware/starknet/services/api/feeder_gateway/response_objects.py` (near line 988) | Add `fee_proposal_fri: Optional[int]` field to `StarknetBlock` (top-level, parallel to `next_l2_gas_price`). Thread through `StarknetBlock.create()`. |
60+
| `src/starkware/starknet/services/api/feeder_gateway/response_objects.py` (new function) | New `remove_snip35_info_from_block_json(block_json)` that pops `fee_proposal_fri`. The existing `remove_fee_market_info_from_block_json` is **not modified**. |
61+
| `src/starkware/starknet/services/feeder_gateway/feeder_gateway.py` | Add `WITH_SNIP35_INFO = "withSnip35Info"` constant + allowlist + parse + apply strip. Two independent toggles (no chained `elif`). |
62+
| `src/starkware/starknet/services/feeder_gateway/feeder_gateway_impl.py:1315` | Pending block path — also read `Snip35Info` and pass `fee_proposal_fri` to `StarknetBlock.create()`. |
63+
| `src/starkware/starknet/services/utils/sequencer_internal_api_utils.py:521` | Same plumbing for the non-pending path. |
64+
| `src/starkware/starknet/services/cende/cende_recorder.py` | Update the `Blob` dataclass to include `snip35_info`. Add `_write_snip35_info` mirror of `_write_fee_market_info`. Wire into the writer driver. |
65+
| `src/starkware/starknet/services/block_hash_calculator/shared_objects.py:332-364` | `BlockHashHeader` and its `new()` — **do NOT touch**. Block hash must remain unaffected by SNIP-35. |
66+
67+
### 2. `sequencer/` — Rust repo (this one)
68+
69+
Path: `/home/andrew/workspace/sequencer/`. Two independent edits:
70+
71+
**Outbound (write side, Apollo → cende blob):**
72+
73+
| Path | What |
74+
|---|---|
75+
| `crates/apollo_consensus_orchestrator/src/snip35/mod.rs` (or sibling module) | Define `Snip35Info { fee_proposal_fri: Option<GasPrice> }`. |
76+
| `crates/apollo_consensus_orchestrator/src/cende/mod.rs:364` | Add `snip35_info: Snip35Info` to `BlobParameters`. |
77+
| `crates/apollo_consensus_orchestrator/src/cende/central_objects.rs:88` | Add `CentralSnip35Info` type alias. |
78+
| `crates/apollo_consensus_orchestrator/src/sequencer_consensus_context.rs:548` | Construct `Snip35Info` from `init.fee_proposal_fri` and pass it via `BlobParameters`. |
79+
80+
**Inbound (read side, feeder gateway JSON → Rust types):**
81+
82+
| Path | What |
83+
|---|---|
84+
| `crates/apollo_starknet_client/src/reader/objects/block.rs` | Add `fee_proposal_fri: Option<GasPrice>` to `BlockPostV0_13_1` (~line 89), accessor on `Block` enum (~line 200), change converter at line 345. |
85+
| `crates/apollo_starknet_client/src/reader/mod.rs:157, 239` | Add `SNIP35_INFO_QUERY` constant; append `withSnip35Info=true` to the get_block URL. |
86+
| `crates/apollo_starknet_client/resources/reader/block_post_0_14_3.json` | New JSON test fixture with the field populated. |
87+
| `crates/apollo_starknet_client/src/reader/objects/block_test.rs` | New tests + register fixture. |
88+
| `crates/apollo_starknet_client/tests/feeder_gateway_integration_test.rs` | New live integration test, gated until Python deploys. |
89+
90+
## TODO list (in dependency order)
91+
92+
### Prerequisite
93+
94+
- [ ] **Task #1 — Confirm scope with manager.** Validate the design (independent `Snip35Info`, opt-in toggle, no Python-side computation).
95+
96+
### Phase A — Python (`starkware/`)
97+
98+
- [ ] **Task #2 — Add new `Snip35Info` dataclass.** New `IndexedDBObject` in `shared_objects.py`. Don't touch `FeeMarketInfo`. Also add `BatchCreated.get_snip35_info(...)` as a sibling of `get_fee_market_info`.
99+
100+
- [ ] **Task #3 — Confirm centralized side is record-only.** No SNIP-35 computation in Python; just plumbing. Verify with manager that no Python-as-proposer mode exists for V0_14_3+ blocks.
101+
102+
- [ ] **Task #4 — Add `fee_proposal_fri` to `StarknetBlock` response object** (top-level field, parallel to `next_l2_gas_price`). Thread through `StarknetBlock.create()`. Add `remove_snip35_info_from_block_json` helper. **Do NOT touch** `remove_fee_market_info_from_block_json`.
103+
104+
- [ ] **Task #15 — Add `withSnip35Info` query toggle in `feeder_gateway.py`.** Independent of `withFeeMarketInfo`. Two parallel strip operations, no chained logic. Backward-compat: existing `withFeeMarketInfo=true` callers see no change in JSON shape.
105+
106+
- [ ] **Task #5 — Update two `StarknetBlock.create()` call sites** (`feeder_gateway_impl.py:1315`, `sequencer_internal_api_utils.py:521`) to also read `Snip35Info` and pass `fee_proposal_fri`. Do NOT change `BlockHashHeader.new()`.
107+
108+
- [ ] **Task #16 — Add CENDE recorder writer for `Snip35Info` blobs.** Update `Blob` dataclass, add `_write_snip35_info`, wire into the writer driver. Backward-compat: tolerate missing `snip35_info` in old blobs.
109+
110+
- [ ] **Task #6 — Update Python tests.** Tests for the new `Snip35Info`, the new toggle, and CENDE recorder writes. `FeeMarketInfo` tests are unchanged.
111+
112+
- [ ] **Task #7 — Land Python PR and deploy to staging feeder gateway.** Coordinate with the deployment-owning team.
113+
114+
### Phase B — Rust (`sequencer/`) — runs in parallel with Phase A
115+
116+
- [ ] **Task #14 — Add `Snip35Info` struct (independent of `FeeMarketInfo`) and wire into cende blob.** Outbound write side. Define `Snip35Info { fee_proposal_fri }`, add to `BlobParameters` alongside the existing `fee_market_info`, populate at `sequencer_consensus_context.rs:548`. Add a serialization round-trip test.
117+
118+
- [ ] **Task #8 — Insert PR: `apollo_starknet_client: read fee_proposal_fri from feeder gateway`.** Inbound read side. Adds the field to `BlockPostV0_13_1`, the accessor, the converter call, and `withSnip35Info=true` to the request URL. **Blocked by Task #7** — cannot be deployed until Python knows about the new toggle.
119+
120+
- [ ] **Task #9 — Add JSON test fixture `block_post_0_14_3.json`.**
121+
122+
- [ ] **Task #10 — Add unit/conversion/round-trip tests.**
123+
124+
- [ ] **Task #11 — Add live integration test, marked `#[ignore]`.** Remove the ignore once Task #7 is on staging.
125+
126+
### Phase C — coordination
127+
128+
- [ ] **Task #12 — Investigate `pending_data.rs:214` TODO.** Pending blocks don't currently include fee market info. If proposers need `fee_proposal_fri` for blocks built on top of pending state, this needs the same treatment.
129+
130+
- [ ] **Task #13 — Verify cross-repo wire compatibility and roll out.** Confirm Python emits the new field, Rust reads it, backward compat holds (old gateway responses still parse), `BlockHash` is unchanged.
131+
132+
## When to do what
133+
134+
```
135+
┌─ Task #1 — Confirm scope ─┐
136+
│ │
137+
↓ ↓
138+
[Python track] [Rust track]
139+
│ │
140+
├─ Task #2 (Snip35Info) ├─ Task #14 (outbound: cende blob)
141+
├─ Task #4 (response) ├─ Task #8 (inbound: starknet_client) ← blocked by Task #7
142+
├─ Task #15 (toggle) ├─ Tasks #9, #10
143+
├─ Task #5 (callsites) ├─ Task #11 (gated integration test)
144+
├─ Task #16 (CENDE) └─ Task #12 (followup)
145+
├─ Task #6 (tests)
146+
├─ Task #7 (deploy staging)
147+
│ │
148+
└────── Task #13 (verify integration) ──────┘
149+
```
150+
151+
The two tracks are mostly independent. The exception: Task #8 (Rust inbound) cannot deploy against any environment until Task #7 (Python) has reached that environment, because sending `withSnip35Info=true` to a Python that doesn't allowlist the flag causes the request to be rejected.
152+
153+
## What about the existing Rust stack
154+
155+
**Stack stays as-is.** No PR in the current stack needs to be amended for this work.
156+
157+
- PR 1 (`starknet_api: add fee_proposal_fri to BlockHeaderWithoutHash`) and everything above continues unchanged. The hardcoded `fee_proposal_fri: None` at `block.rs:345` is correct *until* the new PR (Task #8) replaces it with the real accessor.
158+
- Task #8 inserts above PR 1 in the stack, separating "introduce the type field" from "wire the source."
159+
- Task #14 is a separate Rust PR (write side) that can land anywhere in the stack — it's independent of the inbound read path.
160+
- Until Python deploys, the field defaults to `None` from gateway responses (because the JSON doesn't carry it and `#[serde(default)]` resolves to `None`). This means **the entire SNIP-35 stack can land in this Rust repo on its own schedule.**
161+
162+
## Testing strategy
163+
164+
| Layer | Requires Python deploy? | Catches |
165+
|---|---|---|
166+
| 1. JSON deserialization unit test (Task #10) | No | Schema parsing, optional defaulting. |
167+
| 2. Conversion test (Task #10) | No | Field plumbed through `to_starknet_api_block_and_events()`. |
168+
| 3. Backwards-compat test (Task #10) | No | Old fixtures (no `fee_proposal_fri`) still parse with `None`. |
169+
| 4. Round-trip test (Task #10) | No | `Block → JSON → Block` preserves the field. |
170+
| 5. Cende blob round-trip test (Task #14) | No | Apollo's `Snip35Info` JSON-serializes to a shape the Python recorder accepts. |
171+
| 6. Live integration test (Task #11) | **Yes** | End-to-end against a real feeder gateway. Gated `#[ignore]` until staging deploy. |
172+
| 7. Production rollout (Task #13) | Yes | Mainnet/testnet behavior. |
173+
| 8. Block hash invariance check (Task #13) | Yes | `BlockHash` unchanged before vs. after the new field is emitted. Sanity-check that `BlockHashHeader.new()` doesn't accidentally include `fee_proposal_fri`. |
174+
| 9. Backward-compat shape check (Task #13) | Yes | A request with only `withFeeMarketInfo=true` (no `withSnip35Info`) returns byte-identical JSON before vs. after the deploy. |

0 commit comments

Comments
 (0)