test(fast-inbox): streaming inbox e2e latency and mid-checkpoint inclusion (A-1385) by spalladino · Pull Request #24790 · AztecProtocol/aztec-packages

spalladino · 2026-07-18T16:04:37Z

Test-only coverage for the Fast Inbox (AZIP-22) streaming L1→L2 message path (issue A-1385, FI-15) — the behaviours the legacy suite could not express, because pre-flip every message entered at the first block of the next checkpoint.

Stacked on the node/flip phase: #24784 (A-1379) → #24785 (A-1380) → #24786 (A-1381) → #24787 (A-1382) → #24788 (A-1383) → #24789 (A-1384). Base is spl/a-1384-flip-streaming-inbox.

e2e (`end-to-end`)

New single-node/cross-chain/streaming_inbox.test.ts, on the proven CrossChainMessagingTest fixture (production pipelining sequencer) with a widened slot (36s, 6s blocks → up to ~4 blocks per checkpoint) and minTxsPerBlock: 0:

Mid-checkpoint inclusion — times a send so the message ages past INBOX_LAG_SECONDS partway through a checkpoint's build, then asserts the inserting block has indexWithinCheckpoint > 0 and that the immediately preceding block did not yet carry the message. Retries with fresh messages if a message happens to age exactly at a checkpoint boundary.
Latency bound — asserts, slot-denominated, that includingBlockTimestamp − messageL1Timestamp ≤ INBOX_LAG_SECONDS + 2·slotDuration (derived from the deployed constants, not hardcoded), and that it is positive. Wall-clock latency is logged only (A-1178 style), never asserted.
Message-only block — on a drained pool, asserts the block that consumes the message carries zero tx effects (the FI-05 zero-tx / non-empty-bundle shape) and that the chain keeps proving past it.
Send-then-consume on the streaming path — consumes a streaming-inserted message with a public tx by its compact leaf index and asserts a second consume reverts (double-spend protection); the insert/consume block relationship is logged.

prover-client

New checkpoint-sub-tree-orchestrator test for a checkpoint whose L1→L2 messages span multiple blocks (a non-first block carries a bundle). Asserts per-block start/end L1→L2 tree-snapshot continuity and slice partitioning (no gap/overlap; block slice = [prevBlockLeafCount, blockLeafCount)) and the isFirstBlock flag. Adds TestContext.makeCheckpointWithMessagesPerBlock to distribute a bundle across a checkpoint's blocks (the single-block-per-checkpoint makeCheckpoint puts every message in the first block).

Validation

The e2e suite and the prover-client orchestrator test could not be run locally — the local build is broken in noir-protocol-circuits-types/simulator/prover-client from stale circuit artifacts, and end-to-end sits downstream. CI validates both. Every helper, fixture, and API call is grounded on an existing green suite (l1_to_l2.test.ts, cross_chain_public_message.test.ts) and verified against the current interfaces. No packages that build locally (stdlib, blob-lib, ethereum, foundation) are touched.
Reviewed by codex (gpt-5.6-terra): it caught an incorrect message-sponge-continuity assumption in the prover test (non-first block roots inherit the checkpoint-wide sponge, not the prior block's end sponge); those sponge assertions were removed, leaving the correct L1→L2 tree-snapshot partitioning checks.

Review follow-up (phase-2 final review)

The multi-block-slice orchestrator test now also asserts per-block message-sponge continuity (start empty, each block absorbs exactly its slice, last end equals the InboxParity sponge) and turns its last block into a zero-tx message-only block, exercising the msgs-only block-root wiring fixed in #24789. A stray async in the e2e suite was dropped (lint).

…age (A-1385) Add a checkpoint-sub-tree orchestrator test for a checkpoint whose L1-to-L2 messages span multiple blocks, including a non-first block carrying a bundle. Asserts per-block start/end L1-to-L2 snapshot continuity and slice partitioning (no gap/overlap; block slice = [prevBlockLeafCount, blockLeafCount)). Adds TestContext.makeCheckpointWithMessagesPerBlock to distribute a bundle across a checkpoint's blocks (the single-block-per-checkpoint makeCheckpoint puts every message in the first block).

…usion (A-1385) New end-to-end coverage the legacy suite could not express (streaming Inbox, AZIP-22 / FI-15): a message included in a non-first block of a checkpoint (mid-checkpoint), the L1-inclusion to L2-availability latency within the streaming bound (slot-denominated), a message-only zero-tx block on an empty pool, and a public consume of a streaming-inserted message by compact index with double-spend protection.

…only block root (A-1385) Extends the multi-block-slice orchestrator test to assert per-block message sponge continuity (first block starts empty, each block absorbs exactly its own slice, last end sponge equals the InboxParity sponge) and turns its last block into a zero-tx message-only block, exercising the msgs-only block root selection wired in the flip. Also drops a stray async on a helper with no await in the streaming inbox e2e (lint).

…s-only block (A-1385) Drive the streaming Inbox shapes through the entire proving DAG (block roots incl. msgs-only, block merge, checkpoint roots, checkpoint merge, root rollup) at simulated-circuit fidelity in the top-tree orchestrator suite: a checkpoint whose messages land in a non-first block plus a zero-tx message-only block, followed by message-carrying and message-less checkpoints. Asserts the root rollup's inbox rolling-hash range matches the accumulation over the epoch's messages in block order. TestContext now threads the inbox rolling hash across the checkpoints it builds (previously hardcoded to zero), so a checkpoint built after a message-carrying one commits the correct continuation in its header and the checkpoint merge continuity assert can be exercised.

…tputs (A-1385) The sub-tree result surfaces at most two top-level nodes for the binary checkpoint root, so a three-block checkpoint yields [block-merge(blocks 0-1), block 2] rather than one output per block. The slice/sponge continuity walk now runs over expected per-output block ranges; merge public inputs span their range (is_first_block from the left child, start sponge/state from the left, end sponge/state from the right).

…block checkpoint fixture (A-1385) The per-block message driver still passed the checkpoint's slices through `addBlock`'s options bag and asked `startNewCheckpoint` for per-block insertion, which is now the only behaviour.

…n or spec references (A-1385)

…ts (A-1385)

spalladino · 2026-07-28T22:38:52Z

Superseded by #25037.

The Fast Inbox stack has been regrouped from 20 per-issue PRs (plus 2 umbrellas) down to 3 area PRs plus an umbrella, and rebased onto merge-train/spartan including #25007 (the noir-projects fnd//labs/ split). This PR's diff is preserved verbatim as a single squashed commit in #25037, and this description is preserved in that commit's message.

New structure: #25036 (circuits + L1) → #25037 (node + flip) → #25038 (cleanup), with #25039 as the full-stack umbrella. The original branch for this PR is left on the remote as a recovery point.

Closing here to cut the rebase and review overhead of maintaining 22 PRs; the work is not abandoned.

Deletes the legacy L1 Inbox path now that `propose` enforces streaming-inbox consumption (AZIP-22 Fast Inbox, FI-16). Stacked on `spl/a-1385-streaming-e2e`; part of the Fast Inbox stack (#24784..#24790 below this one). ## What is deleted - **Frontier trees**: the `trees` mapping, `forest`, `HEIGHT`/`SIZE`/`EMPTY_ROOT`, the per-message tree insert, `getRoot()`, and the whole `consume()` flow. - **LAG / inboxLag**: the `LAG` immutable and `_lag`/`_height` constructor args (the Inbox constructor is now `(rollup, feeAsset, version, bucketRingSize)`), `RollupConfigInput.inboxLag` and its `RollupCore` pass-through, and `Constants.L1_TO_L2_MSG_SUBTREE_HEIGHT` from the Inbox construction. - **Dead event field**: the `MessageSent.checkpointNumber` (former `inProgress`) argument, which became meaningless once `consume()` stopped advancing it. - **Orphaned errors**: `Inbox__Unauthorized`, `Inbox__MustBuildBeforeConsume`, `Rollup__InvalidInHash`. ## What is re-homed / kept - The compact message index now reads the current bucket's running total (`totalMsgCount`) instead of a parallel counter; `InboxState.inProgress` and `getInProgress()` are gone. - The 128-bit `rollingHash`, `InboxState.{rollingHash, totalMessagesInserted}`, `getState()`, and `getTotalMessagesInserted()` are **kept**: the node still consumes them for message sync and L1-reorg detection. Their removal is the node cleanup's job (FI-18). - `FrontierLib` is **kept** — it still backs the base-parity (`test/Parity.t.sol`) and merkle (`test/merkle/Frontier.t.sol`) tests, which are unrelated to the Inbox (parity-circuit territory, FI-17). The plan's "delete the file" assumption is contradicted by grep. ## TS follow-through - `ethereum` InboxContract: drop `getLag()`, drop the `MessageSent.checkpointNumber` decode, stop reading `inProgress`; `InboxContractState.treeInProgress` is now optional (no longer tracked on-chain). - Stop threading `inboxLag` through `queries.getL1ContractsConfig` and the L1 deploy env. The broader `AZTEC_INBOX_LAG` config sweep (`ethereum/src/config.ts`, `foundation`, `stdlib`, ~30 e2e configs) is deferred to FI-18, which explicitly owns it. - Archiver decode derives the message's checkpoint number from the compact index (the event no longer carries it), keeping the legacy per-checkpoint store shape untouched for FI-18. - Removed the obsolete `advanceInboxInProgress` cheat code + its inbox-drift tests, and the orphaned archiver `inHash`-mismatch sync test (the cross-check was removed at the flip). ## Gas Deleting the frontier insert is a large `sendL2Message` win. Whole-test gas (`forge test` on `InboxBuckets.t.sol`), before → after: | Case | Before | After | | --- | --- | --- | | First-ever message | 175,251 | 116,835 | | First message of a new L1 block | 329,460 | 195,555 | | Absorb into an existing bucket | 286,673 | 149,968 | | Rollover mid-block (256 messages) | 18,923,988 | 3,068,480 | Per-call `sendL2Message` gas after cleanup: first-ever 99,526; first-of-new-block 53,763; existing-bucket absorb 9,273; rollover 53,801. ## Testing - `forge build` + `forge test` green: 887 passed, 0 failed. This fixes 4 pre-existing baseline failures the flip stranded (`fee_portal`/`TokenPortal` deposit tests using tree-relative indices + the old event shape). - `@aztec/ethereum` builds clean; `config`/`queries` unit tests green. - `@aztec/archiver` has no self-owned type errors; `message_store` (36), `archiver-sync` (59), and the decode/struct suites (58) all pass. (Pre-existing `noir-protocol-circuits-types` stale-artifact errors are unaffected.) - e2e not run locally. The stability gate ("a few days of green e2e/networks before deleting the fallback") applies at merge time for this stacked line, not at PR creation. ## Review follow-up (phase-2 final review) Deleting the inbox-drift bot test left `bot.test.ts`'s `cheatCodes` variable unused (lint error); a follow-up commit removes it. ## Leftover sweep (post-review) - Corrected the `PublicInputArgs` natspec in `IRollup.sol`: it still described `previousInboxRollingHash` / `endInboxRollingHash` as deliberately unvalidated "until the Fast Inbox flip", but the flip (#24789, below this branch) added exactly that validation — `EpochProofLib` anchors the start boundary against the propose-time record, and the end boundary is covered transitively by the stored checkpoint header hashes. The comment now describes the implemented behavior. - The 128-bit rolling hash this PR deliberately kept (see "What is re-homed / kept") is now removed at the node-cleanup PR #24793, where its last TS readers disappear. Replaces #24791.

…387) FI-17 of the Fast Inbox (AZIP-22) stack. Stacked on #24791 (A-1386); it removes the legacy `inHash` from the checkpoint header now that the consensus flip (#24789, A-1384) has switched validation to the streaming rolling hash. After this PR, `inboxRollingHash` is the checkpoint's only inbox commitment. ## Consensus-format change - The checkpoint header loses its `inHash` field across all three preimage layers in lockstep: noir `CheckpointHeader` (`checkpoint_header.nr`), TS `CheckpointHeader` (`stdlib/src/rollup/checkpoint_header.ts`), and Solidity `ProposedHeaderLib` (struct + hash packing). The field ordering is otherwise unchanged; the header shrinks by one field (32 bytes: `CHECKPOINT_HEADER_SIZE_IN_BYTES` 380 to 348, `CHECKPOINT_HEADER_LENGTH` 14 to 13). - The header-hash fixtures were regenerated from `checkpoint_header.test.ts` (the established workflow) and pasted into the noir tests; the p2p golden-byte fixture (`wire_compat_fixtures.ts`) was refreshed for the checkpoint-proposal serialization, which shrinks by exactly the one removed field. The block-proposal wire fixture is unchanged (block-level `inHash` is out of scope, see A-1388). ## Circuits and constants - Strips the dead unconstrained `in_hash` pass-through from the sized `InboxParity<S>` circuit and `ParityPublicInputs` (noir + TS + the hand-written `noir-protocol-circuits-types` conversion wrappers). The circuit is kept (sized-parity, per FI-06's topology decision), so no VK-tree indices move. - Retires `NUMBER_OF_L1_L2_MESSAGES_PER_ROLLUP` (replaced everywhere by `MAX_L1_TO_L2_MSGS_PER_CHECKPOINT`, same value 1024) and the now-dead `NUM_MSGS_PER_BASE_PARITY`, `NUM_BASE_PARITY_PER_ROOT_PARITY`, and `L1_TO_L2_MSG_SUBTREE_ROOT_SIBLING_PATH_LENGTH`. Generated constants (`constants.gen.ts`, `ConstantsGen.sol`) were regenerated via `remake-constants`. - Deletes `FrontierLib.sol` and its tests (`Parity.t.sol`, the `Frontier` harness and merkle test) now that their last users are gone. ## Verification - Noir: `types::checkpoint_header` (5/5), `rollup_lib::parity` and `checkpoint_root::parity_tests` (15/15), and the checkpoint-root header-assembly path pass locally. The full `checkpoint_root` suite (68 tests) is too slow to run in full locally and rides CI. - Solidity: `forge test` green across `Rollup`, `RollupFieldRange`, `Inbox`, `InboxBuckets`, `RollupGetters` (76/76), covering header hashing and the regenerated checkpoint fixtures. - Jest: `checkpoint_header`, the p2p wire-compat suites, and the parity/messaging suites pass (43 tests), confirming the fixture regeneration is byte-consistent. - VK/artifact/Prover.toml regeneration and the generated Noir ABI types ride CI (local `yarn build` in `noir-protocol-circuits-types` / `simulator` / `prover-client` is expected to fail on stale circuit artifacts until then); e2e and the broken-zone typecheck are the CI-of-record for those. ## Stack Part of the Fast Inbox stack #24784 through #24791 (umbrella #24774): A-1379 (#24784), A-1380 (#24785), A-1381 (#24786), A-1382 (#24787), A-1383 (#24788), A-1384 flip (#24789), A-1385 (#24790), A-1386 (#24791, base of this PR). Resolves A-1387. ## l1 publisher inbox consumption via the streaming selector (CI fix, pass-8) The A-1387 inHash-removal rewrite left `l1_publisher.integration.test.ts` hardcoding `previousInboxRollingHash = Fr.ZERO` and `bucketHint = 0n` while consuming real L1→L2 messages, so message-consuming checkpoints reverted on L1 with `Rollup__InvalidInboxRollingHash` / `UnconsumedInboxMessages` (masked every prior CI round behind earlier failures). `INBOX_LAG_SECONDS` is an L1-time censorship cutoff, not whole checkpoints, so no checkpoint-depth shift register reproduces the aged bucket set. The test now mirrors the real Inbox buckets into its `MockL1ToL2MessageSource` and reuses the production `selectInboxBucketForBlock` (which mirrors `ProposeLib.validateInboxConsumption`) to pick exactly the buckets each checkpoint must consume, deriving the consumed bundle, the propose bucket hint, and the header rolling hash from that one selection. Box: 13/13, verified at a-1387 and a-1388. Replaces #24792.

This was referenced Jul 18, 2026

chore(fast-inbox): delete legacy L1 inbox path (A-1386) #24791

Closed

chore(fast-inbox): circuits cleanup and checkpoint header format (A-1387) #24792

Closed

spalladino force-pushed the spl/a-1385-streaming-e2e branch from 651d40c to 14003e9 Compare July 19, 2026 01:25

spalladino mentioned this pull request Jul 19, 2026

feat(fast-inbox): streaming inbox node integration, flip, and cleanup (umbrella) #24796

Closed

spalladino force-pushed the spl/a-1385-streaming-e2e branch 3 times, most recently from e896f31 to a1f18f3 Compare July 19, 2026 17:57

spalladino force-pushed the spl/a-1384-flip-streaming-inbox branch from 97fce26 to 4d7c269 Compare July 19, 2026 17:57

spalladino force-pushed the spl/a-1385-streaming-e2e branch 2 times, most recently from 7d0b8da to 6599148 Compare July 19, 2026 20:48

spalladino force-pushed the spl/a-1384-flip-streaming-inbox branch from 73e288c to 4c6d86b Compare July 19, 2026 20:48

spalladino force-pushed the spl/a-1385-streaming-e2e branch 3 times, most recently from 7c61dba to 6057ab5 Compare July 20, 2026 14:07

spalladino force-pushed the spl/a-1384-flip-streaming-inbox branch from 0a0d72d to 976062a Compare July 20, 2026 15:30

spalladino force-pushed the spl/a-1385-streaming-e2e branch 2 times, most recently from 36db4d0 to e971f40 Compare July 20, 2026 17:28

spalladino force-pushed the spl/a-1384-flip-streaming-inbox branch 2 times, most recently from de56dbe to 66aff9d Compare July 20, 2026 21:21

spalladino force-pushed the spl/a-1385-streaming-e2e branch 3 times, most recently from fb159da to faaf5c2 Compare July 21, 2026 03:39

spalladino force-pushed the spl/a-1384-flip-streaming-inbox branch from 1740bc9 to a17dce8 Compare July 21, 2026 03:39

spalladino force-pushed the spl/a-1385-streaming-e2e branch 5 times, most recently from 1a51d95 to 6b97921 Compare July 27, 2026 20:53

spalladino requested review from LeilaWang and just-mitch as code owners July 27, 2026 20:53

spalladino force-pushed the spl/a-1384-flip-streaming-inbox branch from d97760c to 1cc2caa Compare July 27, 2026 20:53

spalladino added 8 commits July 28, 2026 09:24

docs(fast-inbox): describe streaming inbox behavior without transitio…

9371d6b

…n or spec references (A-1385)

docs(fast-inbox): drop issue references from the streaming e2e commen…

49f78a2

…ts (A-1385)

spalladino force-pushed the spl/a-1384-flip-streaming-inbox branch from 1cc2caa to bcc37cf Compare July 28, 2026 12:37

spalladino force-pushed the spl/a-1385-streaming-e2e branch from 6b97921 to 49f78a2 Compare July 28, 2026 12:38

spalladino mentioned this pull request Jul 28, 2026

feat(fast-inbox): node integration and the consensus flip (part 2/3) #25037

Open

spalladino closed this Jul 28, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test(fast-inbox): streaming inbox e2e latency and mid-checkpoint inclusion (A-1385) - #24790

test(fast-inbox): streaming inbox e2e latency and mid-checkpoint inclusion (A-1385)#24790
spalladino wants to merge 8 commits into
spl/a-1384-flip-streaming-inboxfrom
spl/a-1385-streaming-e2e

spalladino commented Jul 18, 2026 •

edited

Loading

Uh oh!

spalladino commented Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

spalladino commented Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

e2e (end-to-end)

prover-client

Validation

Review follow-up (phase-2 final review)

Uh oh!

spalladino commented Jul 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

spalladino commented Jul 18, 2026 •

edited

Loading

e2e (`end-to-end`)