Commit 760d927
committed
feat(fast-inbox): flip consensus to streaming inbox (A-1384)
Coordinated cutover of the Fast Inbox (AZIP-22) project (issue A-1384, FI-14). Flips the authoritative path across L1, circuits, and the node from the legacy `inHash`/frontier-tree consumption to the streaming rolling-hash Inbox.
Stacked on the node phase: #24784 (A-1379) -> #24785 (A-1380) -> #24786 (A-1381) -> #24787 (A-1382) -> #24788 (A-1383). Base is `spl/a-1383-validator-streaming`.
- **A-1431 (domain-separation decision) must resolve before this merges.** It is deliberately deferred and nothing is adopted here. If it adopts a separator, the rolling-hash link-encoding change must be inserted below this PR in the stack.
- `ProposeLib.propose` calls `validateInboxConsumption` against the parent checkpoint's consumed position (read from the parent temp-log record) instead of `inbox.consume()` + the `Rollup__InvalidInHash` check; the returned consumed total is stored in the new record.
- `ProposeArgs` gains an unsigned `bucketHint` calldata field (out of the attested payload digest).
- `TempCheckpointLog` / `CompressedTempCheckpointLog` carry `{inboxRollingHash, inboxMsgTotal, inboxConsumedBucket}`; genesis is `{0,0,0}`.
- `EpochProofLib.getEpochProofPublicInputs` anchors the rolling-hash chain start to the record of checkpoint `start - 1` (`Rollup__InvalidPreviousInboxRollingHash`).
- `Inbox.sendL2Message` returns and emits the compact cumulative message index.
- `MAX_L1_TO_L2_MSGS_PER_BLOCK` 1024 -> 256.
- `L1ToL2MessageBundle` drops `num_real_msgs`; a single `num_msgs` drives the compact tree append and the message-sponge absorb.
- `SpongeBlob::absorb_block_end_data` absorbs the L1-to-L2 tree root for every block.
- **Streaming is the only path**: the `streamingInbox` flag is removed across foundation/stdlib config + the sequencer/validator plumbing.
- **Sequencer**: sources the parent bucket from the fork's L1-to-L2 leaf count + `getInboxBucketByTotalMsgCount` (cross-checkpoint); feeds inHash zero; relaxes `waitForMinTxs` for message-only blocks; threads the consumed bucket seq as the propose `bucketHint`. Automine mirrors the selection.
- **Validator**: per-block acceptance + checkpoint re-execution always run; the checkpoint rebuild derives the consumed message list from the buckets between the parent checkpoint's position and the last block.
- **World state**: `appendL1ToL2MessagesToTree` / `handleL2BlockAndMessages` append real leaves at compact indices; the synchronizer derives each block's bundle from its leaf-index range.
- **Blob / constants / bundle**: per-block blob root, `MAX_L1_TO_L2_MSGS_PER_BLOCK = 256`, `L1ToL2MessageBundle` drops `numRealMsgs`; provable per-checkpoint ceiling 2457 -> 2234.
- **Archiver / TXE**: the dead inHash cross-check is removed; TXE appends unpadded compact leaves.
- **L1 ABI**: `@aztec/l1-artifacts` regenerated for the new `ProposeArgs` (gitignored generated output; the `ethereum` package builds against it).
- L1 `forge test`: full propose/inbox suite green (ProposeInboxConsumption, Inbox, InboxBuckets, Rollup incl. a new anchoring negative, RollupFieldRange, FeeRollup, ValidatorSelection, escape-hatch, tmnt207/419).
- Circuits `nargo test`: parity, block_root structure, msgs_only green on touched crates.
- Node jest (runnable locally): validator proposal_handler (34) + streaming checks (14), sequencer inbox selector (9), world-state native (56), blob-lib + stdlib. tsc for the broken-build-zone packages (prover-client, sequencer-client, validator-client, etc.) is CI's job of record.
- Codex (gpt-5.6-terra) reviewed both the L1/circuits layer and the node wiring: no consensus-critical issue; bucket cursor/range `(fromExclusive, toInclusive]` and pipelining-parent sourcing verified correct.
VK regen and `Prover.toml` sample-input regen ride CI (local `bb write_vk` OOMs on `rollup_root`).
- **Prover per-block message split**: the checkpoint's messages are sliced per block from the blocks' L1-to-L2 leaf-count ranges; each block root appends its own real slice at compact indices with per-block start/end snapshots and a full-height frontier hint. `startCheckpoint` no longer inserts messages up front.
- **Checkpoint keying from records**: `getL1ToL2MessageCheckpoint` binary-searches the block records for the first block whose L1-to-L2 leaf count exceeds the message index (portal claim helpers depend on this); the prover-node derives the checkpoint's consumed messages from the Inbox buckets.
Codex (gpt-5.6-terra) reviewed the prover/keying rework: no consensus defect; slice partitioning, bucket-range agreement, fork ordering, frontier height, and the checkpoint boundary rule all verified.
The `message_store` `InboxLeaf` index-formula methods (`smallestIndexForCheckpoint` etc.) have no correctness-critical live callers post-flip; the public-simulator's next-checkpoint fetch degrades safely without them (simulates without the not-yet-consumed messages). Deleted in FI-18.
A-1390 (unconsumed-bucket overwrite protection) stays post-flip: a consumption backlog older than the Inbox ring can overwrite unconsumed buckets and halt proposals until an upgrade. Acceptable for non-production lines.
A final review pass appended `fix(fast-inbox): thread per-block message sponges and wire the msgs-only block root in the prover`:
- The prover supplied the checkpoint-wide message sponge as every non-first block root's start sponge, which the block-merge/checkpoint-root continuity asserts reject whenever a non-first block carries messages. The sponge is now threaded per block.
- A zero-tx non-first block (the message-only shape this PR lets the sequencer produce) was rejected by `BlockProvingState` and the lightweight builder, and the orchestrator never selected the msgs-only block root. Both now accept it and route it through `BlockRootMsgsOnlyRollupPrivateInputs`.
- The required `bucketHint` parameter added to `enqueueProposeCheckpoint` here is now passed by the publisher unit/integration tests and the e2e synching test (the integration suite's consumption model is reworked for streaming semantics in #24793).
The `cross_chain_messages` e2e ("builds multiple blocks per slot with L1 to L2 messages") caught a product regression: when a checkpoint's first block is message-only (consuming a bucket) and the final block fails to build, the `bucketHint` on the propose payload was reconstructed from the last successfully-built block's streaming cursor and lost the already-consumed bucket, so L1 `validateInboxConsumption` rejected the checkpoint. `CheckpointProposalBroadcast` now carries `bucketHint` explicitly, sourced from the streaming state's `lastBucketRef.bucketSeq` at both propose sites (fisherman and main), so the consumed bucket survives a final-block build failure. A `checkpoint_proposal_job.test.ts` regression case drives message-only-first-block + final-block-failure and asserts the enqueued hint is the consumed bucket (RED reverting the fix: `Expected 2n, Received 0n`).
The shared cross-chain e2e helper (`message_test_helpers.ts`) evaluated `isL1ToL2MessageReady` against the default `'latest'` tip, while these suites anchor the PXE to `syncChainTip: 'checkpointed'`. Pre-flip every message entered at the first block of the next checkpoint, so `'latest'` readiness coincided with checkpointed availability; post-flip per-block insertion lets the proposed chain reach a message's consume-checkpoint a full checkpoint before it is published on L1, so readiness flipped true while the PXE had not yet synced the message and the private consume simulated against a missing membership witness (`No L1 to L2 message found`). This deterministically failed `l1_to_l2.test.ts` from this branch up. The helper now gates on the configured PXE sync tip via a new `CrossChainMessagingTest.pxeSyncChainTip` getter (defaulting to `'latest'`, so suites that do not pin a tip keep their semantics). Test-support only; box-verified green on all three cross-chain suites (`l1_to_l2`, `streaming_inbox`, `l1_to_l2_inbox_drift`).
The archiver's checkpoint reconstruction (`retrievedToPublishedCheckpoint` in `data_retrieval.ts`) read the L1→L2 message tree root once from the checkpoint's **first** block and applied it to every block — a stale pre-flip per-checkpoint assumption. Post-flip the blob carries a **per-block** root (any block in a checkpoint can insert messages). A follower node (prover / sync-only) that rebuilds a non-first message-inserting block therefore got the first block's root, so its world-state synchronizer inserted the correct message, recomputed the real root, and mismatched the reconstructed header (`block state does not match world state`) — forking the node from the sequencer on the first message-consuming block. In production this would fork every prover/follower on the first cross-chain-consuming block; it surfaced here as `cross_chain_public_message` and `token_bridge` failing (only follower nodes reconstruct via the archiver, which is why proposal validation — building from each validator's own world-state fork — never caught it). Reconstruction now uses each block's own `l1ToL2MessageRoot` from the blob. A `data_retrieval.test.ts` unit case asserts each reconstructed block's tree root matches its own blob root (RED: non-first blocks got the first block's root); both e2es green at a-1384.
`world-state/src/test/integration.test.ts` drives the streaming synchronizer through `MockPrefilledArchiver`, but `setPrefilled` only seeded the legacy per-checkpoint message map and never registered Inbox buckets. Post-flip the synchronizer reconstructs each block's consumed L1→L2 bundle from buckets (`getInboxBucketByTotalMsgCount`), which returned undefined against the empty bucket map, so every block synced an empty bundle and the reconstructed state diverged from the header (`block state does not match world state`, then a 600s timeout). `setPrefilled` now registers a genesis sentinel plus one bucket per message-carrying checkpoint (cumulative `totalMsgCount` = the block's post-insertion leaf count), rebuilt from the full prefilled chain so reorg re-prefills stay aligned. Box: 12/12 (reorg 32s→2.4s), verified at a-1384 and top.
- Removed the inert `isFirstBlock` parameter from the blob-data test fixtures (`makeBlockEndBlobData` / `makeBlockBlobData` and the archiver `makeBlockBlobDataFromBody` helper). Post-flip every block carries its own L1-to-L2 root, so the flag had no effect on output; the earlier lint fix had only underscore-bound it "for call-site compatibility". The two now-identical blob-data tests are collapsed into one.
Replaces #24789.1 parent cd4bd52 commit 760d927
126 files changed
Lines changed: 1666 additions & 1387 deletions
File tree
- l1-contracts
- src/core
- libraries
- compressed-data
- rollup
- messagebridge
- test
- base
- benchmark
- compression
- escape-hatch/integration
- fee_portal
- fees
- harnesses
- portals
- rollup/libraries/rewardlib
- validator-selection
- noir-projects/fnd/noir-protocol-circuits/crates
- rollup-lib/src
- abis
- block_root
- components
- tests
- parity/tests
- types/src
- blob_data
- yarn-project
- archiver/src
- l1
- modules
- store
- test
- aztec-node/src
- aztec-node
- modules
- blob-lib/src/encoding
- end-to-end/src
- single-node
- cross-chain
- sync
- ethereum/src/contracts
- foundation/src/config
- noir-protocol-circuits-types/src/conversion
- prover-client/src
- light
- mocks
- orchestrator
- test
- prover-node/src
- job
- sequencer-client/src
- publisher
- sequencer
- automine
- test
- stdlib/src
- block
- checkpoint
- config
- deserialization
- gas
- interfaces
- messaging
- tests
- tx
- txe/src
- oracle
- state_machine
- utils
- validator-client/src
- world-state/src
- native
- synchronizer
- test
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| 44 | + | |
44 | 45 | | |
45 | 46 | | |
46 | 47 | | |
| |||
300 | 301 | | |
301 | 302 | | |
302 | 303 | | |
303 | | - | |
| 304 | + | |
304 | 305 | | |
305 | 306 | | |
306 | 307 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
| |||
466 | 467 | | |
467 | 468 | | |
468 | 469 | | |
469 | | - | |
| 470 | + | |
470 | 471 | | |
471 | 472 | | |
472 | 473 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
| 63 | + | |
| 64 | + | |
63 | 65 | | |
64 | 66 | | |
65 | 67 | | |
| |||
Lines changed: 20 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
37 | 43 | | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
38 | 47 | | |
39 | 48 | | |
40 | 49 | | |
| |||
44 | 53 | | |
45 | 54 | | |
46 | 55 | | |
| 56 | + | |
| 57 | + | |
47 | 58 | | |
| 59 | + | |
48 | 60 | | |
49 | 61 | | |
50 | 62 | | |
| |||
61 | 73 | | |
62 | 74 | | |
63 | 75 | | |
64 | | - | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
65 | 80 | | |
66 | 81 | | |
67 | 82 | | |
| |||
77 | 92 | | |
78 | 93 | | |
79 | 94 | | |
80 | | - | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
81 | 99 | | |
82 | 100 | | |
83 | 101 | | |
Lines changed: 36 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
289 | 289 | | |
290 | 290 | | |
291 | 291 | | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
292 | 316 | | |
293 | 317 | | |
294 | 318 | | |
| |||
319 | 343 | | |
320 | 344 | | |
321 | 345 | | |
322 | | - | |
323 | | - | |
324 | | - | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
325 | 349 | | |
326 | 350 | | |
327 | 351 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
| |||
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
36 | 41 | | |
37 | 42 | | |
38 | 43 | | |
| |||
46 | 51 | | |
47 | 52 | | |
48 | 53 | | |
49 | | - | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
50 | 57 | | |
51 | 58 | | |
52 | 59 | | |
| |||
121 | 128 | | |
122 | 129 | | |
123 | 130 | | |
| 131 | + | |
124 | 132 | | |
125 | 133 | | |
126 | 134 | | |
| |||
261 | 269 | | |
262 | 270 | | |
263 | 271 | | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
264 | 286 | | |
265 | 287 | | |
266 | 288 | | |
267 | 289 | | |
268 | 290 | | |
269 | | - | |
| 291 | + | |
270 | 292 | | |
271 | 293 | | |
272 | 294 | | |
273 | 295 | | |
274 | | - | |
| 296 | + | |
275 | 297 | | |
276 | 298 | | |
277 | 299 | | |
| |||
289 | 311 | | |
290 | 312 | | |
291 | 313 | | |
292 | | - | |
| 314 | + | |
293 | 315 | | |
294 | 316 | | |
295 | 317 | | |
296 | 318 | | |
297 | | - | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
298 | 323 | | |
299 | 324 | | |
300 | 325 | | |
301 | | - | |
302 | | - | |
303 | | - | |
304 | | - | |
305 | | - | |
306 | 326 | | |
307 | 327 | | |
308 | 328 | | |
| |||
Lines changed: 4 additions & 21 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
8 | 7 | | |
9 | 8 | | |
10 | 9 | | |
11 | | - | |
12 | | - | |
13 | 10 | | |
14 | 11 | | |
15 | 12 | | |
| |||
21 | 18 | | |
22 | 19 | | |
23 | 20 | | |
24 | | - | |
| 21 | + | |
25 | 22 | | |
26 | 23 | | |
27 | 24 | | |
28 | 25 | | |
29 | 26 | | |
30 | | - | |
31 | | - | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
32 | 30 | | |
33 | | - | |
34 | 31 | | |
35 | 32 | | |
36 | 33 | | |
| |||
39 | 36 | | |
40 | 37 | | |
41 | 38 | | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | 39 | | |
47 | 40 | | |
48 | 41 | | |
| |||
78 | 71 | | |
79 | 72 | | |
80 | 73 | | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | 74 | | |
92 | 75 | | |
93 | 76 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
132 | 132 | | |
133 | 133 | | |
134 | 134 | | |
135 | | - | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
136 | 141 | | |
137 | 142 | | |
138 | 143 | | |
| |||
305 | 310 | | |
306 | 311 | | |
307 | 312 | | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
308 | 333 | | |
309 | 334 | | |
310 | 335 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
130 | 130 | | |
131 | 131 | | |
132 | 132 | | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
137 | 137 | | |
138 | 138 | | |
139 | 139 | | |
| |||
0 commit comments