Skip to content

feat(fast-inbox): sequencer streaming message selection, flag off (A-1382) - #24787

Closed
spalladino wants to merge 4 commits into
spl/a-1381-proposal-bucket-reffrom
spl/a-1382-sequencer-streaming
Closed

feat(fast-inbox): sequencer streaming message selection, flag off (A-1382)#24787
spalladino wants to merge 4 commits into
spl/a-1381-proposal-bucket-reffrom
spl/a-1382-sequencer-streaming

Conversation

@spalladino

Copy link
Copy Markdown
Contributor

Implements A-1382 (FI-12): sequencer streaming Inbox message selection behind a streamingInbox flag, default off. Part of the Fast Inbox stack (AZIP-22).

Flag semantics

  • One shared env var STREAMING_INBOX -> streamingInbox: boolean (default false), defined once in stdlib's sharedSequencerConfigMappings so the sequencer reads it now and the validator (A-1383) maps the same env. Flag off is byte-identical: the new paths are gated at checkpoint start and per block.
  • With the flag on, checkpoints are expected to fail L1 submission (on-chain propose still enforces the legacy per-checkpoint consumption until the flip, A-1384). The streaming path is exercised with unit/mock tests only.

Selection policy (mirrors ProposeLib.validateInboxConsumption)

New pure selector sequencer-client/src/sequencer/inbox_bucket_selector.ts, run per block:

  • Pick the newest lag-eligible bucket (getLatestInboxBucketAtOrBefore(now - INBOX_LAG_SECONDS)); on the checkpoint's final block, also consider the cutoff bucket and take the newer, so consumption reaches the censorship floor.
  • Walk back from the candidate to the newest bucket that fits both the per-block cap (bucket.totalMsgCount - parent.totalMsgCount) and the per-checkpoint cap (bucket.totalMsgCount - checkpointStartTotalMsgCount). If even the first forward bucket overshoots the per-checkpoint cap, consume nothing — matching L1's cap-escape (next.totalMsgCount - parentTotal > MAX_L1_TO_L2_MSGS_PER_CHECKPOINT).
  • The inclusive <= comparisons make a bucket exactly INBOX_LAG_SECONDS old eligible and a bucket exactly at the cutoff mandatory, matching L1's strict next.timestamp > cutoff.

Cutoff is computed as getTimestampForSlot(slot - 1) - INBOX_LAG_SECONDS, matching ProposeLib exactly (not the consensus timetable's getBuildFrameStart, which subtracts an extra ethereum-slot). Boundary vectors pinned against A-1371 resolution section 13.

Cutoff floor

On the checkpoint's final block (including the block that reaches the per-checkpoint block cap) the cutoff is a consumption floor, so the checkpoint's own header would pass the L1 censorship assert post-flip.

Builder interface change

  • ICheckpointBlockBuilder.buildBlock gains an optional per-block l1ToL2Messages bundle; ICheckpointsBuilder.startCheckpoint gains insertMessagesPerBlock. Threaded through FullNodeCheckpointsBuilder/CheckpointBuilder and the lightweight builder (first-in-checkpoint bundle padded, non-first compact; the inHash/rolling hash recompute over the accumulated logical messages).
  • An optional signed bucket reference is threaded onto block proposals (Validator.createBlockProposal -> ValidationService -> BlockProposal.createProposalFromSigner).

Constant

INBOX_LAG_SECONDS = 12 added to constants.nr and regenerated into the TS constants (constants.gen.ts). Not emitted to ConstantsGen.sol (Solidity whitelist); ProposeLib keeps its local copy pending consolidation (A-1434).

Open item / to verify on CI

  • Non-genesis cross-checkpoint parent-bucket sourcing is not wired: there is no by-rolling-hash archiver lookup and legacy parents do not sit on a bucket boundary, so only the genesis base case is resolved and non-genesis throws (safely caught -> skipped proposal). This is a flip-time concern.
  • Local build/typecheck of sequencer-client, validator-client, prover-client is blocked by the known stale-artifact/@aztec/bb-avm-sim breakage, so CI is the typecheck of record for the job/builder/proposal edits. The pure selector unit tests and the stdlib config changes were verified locally.

@spalladino
spalladino requested a review from LeilaWang as a code owner July 18, 2026 08:15
@spalladino spalladino changed the title feat: sequencer streaming message selection, flag off (A-1382) feat(fast-inbox): sequencer streaming message selection, flag off (A-1382) Jul 19, 2026
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from 8cc06b5 to 2e9f13e Compare July 19, 2026 14:05
@spalladino
spalladino force-pushed the spl/a-1381-proposal-bucket-ref branch from 9eb8813 to 123af00 Compare July 19, 2026 14:05
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from 2e9f13e to d06df3f Compare July 19, 2026 17:57
@spalladino
spalladino force-pushed the spl/a-1381-proposal-bucket-ref branch from 123af00 to 9bdee0d Compare July 19, 2026 17:57
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from d06df3f to a88a509 Compare July 19, 2026 20:48
@spalladino
spalladino force-pushed the spl/a-1381-proposal-bucket-ref branch 2 times, most recently from 9973f4f to 3b13708 Compare July 20, 2026 15:30
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch 2 times, most recently from 19e9567 to 896ea89 Compare July 20, 2026 17:28
@spalladino
spalladino force-pushed the spl/a-1381-proposal-bucket-ref branch from 3b13708 to 4d4af43 Compare July 20, 2026 17:28
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from 896ea89 to a1feb7d Compare July 20, 2026 21:21
@spalladino
spalladino force-pushed the spl/a-1381-proposal-bucket-ref branch 2 times, most recently from 4e21b51 to 8700011 Compare July 21, 2026 02:58
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from a1feb7d to 4bbd6cd Compare July 21, 2026 02:59
@spalladino
spalladino force-pushed the spl/a-1381-proposal-bucket-ref branch from 8700011 to 7f36f7b Compare July 21, 2026 03:39
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from 4bbd6cd to 00a7d98 Compare July 21, 2026 03:39
@spalladino
spalladino force-pushed the spl/a-1381-proposal-bucket-ref branch from 7f36f7b to e9ea2cf Compare July 27, 2026 20:53
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from 00a7d98 to f0cccf4 Compare July 27, 2026 20:53
…ING_INBOX (A-1382)

Behind the shared `streamingInbox` flag (env `STREAMING_INBOX`, default off), the
sequencer selects L1-to-L2 messages from the Inbox rolling-hash buckets per block
instead of consuming the whole checkpoint's messages up front.

- Adds a pure `inbox_bucket_selector` that mirrors `ProposeLib.validateInboxConsumption`:
  newest lag-eligible bucket, per-block/per-checkpoint cap walk-back with cap-escape,
  and a last-block cutoff floor (cutoff = toTimestamp(slot-1) - INBOX_LAG_SECONDS).
- Adds `INBOX_LAG_SECONDS` (12) to constants.nr, regenerated into the TS constants.
- Threads a per-block message bundle through the checkpoint builder interface and an
  optional signed bucket reference onto block proposals.
- Flag off is byte-identical: the new paths are gated at checkpoint start and per block.
…inal block and defer bundle accumulation (A-1382)

- The censorship cutoff floor now applies on whichever block ends the checkpoint,
  including the block that reaches the per-checkpoint block cap, not only the
  timetable's last sub-slot, so the checkpoint always consumes through the cutoff.
- The lightweight builder accumulates a block's streaming message bundle only after
  the block is fully built, so a mid-build failure leaves the checkpoint's message
  list (and its inHash/rolling hash) consistent with the blocks actually built.
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from f0cccf4 to 8b430fa Compare July 28, 2026 12:37
@spalladino

Copy link
Copy Markdown
Contributor Author

Superseded by #25037.

The Fast Inbox stack has been regrouped from 20 per-issue PRs (plus 2 umbrellas) down to 3 area PRs plus an umbrella, and rebased onto merge-train/spartan including #25007 (the noir-projects fnd//labs/ split). This PR's diff is preserved verbatim as a single squashed commit in #25037, and this description is preserved in that commit's message.

New structure: #25036 (circuits + L1) → #25037 (node + flip) → #25038 (cleanup), with #25039 as the full-stack umbrella. The original branch for this PR is left on the remote as a recovery point.

Closing here to cut the rebase and review overhead of maintaining 22 PRs; the work is not abandoned.

@spalladino spalladino closed this Jul 28, 2026
spalladino added a commit that referenced this pull request Jul 28, 2026
…387)

FI-17 of the Fast Inbox (AZIP-22) stack. Stacked on #24791 (A-1386); it removes the legacy `inHash` from the checkpoint header now that the consensus flip (#24789, A-1384) has switched validation to the streaming rolling hash. After this PR, `inboxRollingHash` is the checkpoint's only inbox commitment.

## Consensus-format change

- The checkpoint header loses its `inHash` field across all three preimage layers in lockstep: noir `CheckpointHeader` (`checkpoint_header.nr`), TS `CheckpointHeader` (`stdlib/src/rollup/checkpoint_header.ts`), and Solidity `ProposedHeaderLib` (struct + hash packing). The field ordering is otherwise unchanged; the header shrinks by one field (32 bytes: `CHECKPOINT_HEADER_SIZE_IN_BYTES` 380 to 348, `CHECKPOINT_HEADER_LENGTH` 14 to 13).
- The header-hash fixtures were regenerated from `checkpoint_header.test.ts` (the established workflow) and pasted into the noir tests; the p2p golden-byte fixture (`wire_compat_fixtures.ts`) was refreshed for the checkpoint-proposal serialization, which shrinks by exactly the one removed field. The block-proposal wire fixture is unchanged (block-level `inHash` is out of scope, see A-1388).

## Circuits and constants

- Strips the dead unconstrained `in_hash` pass-through from the sized `InboxParity<S>` circuit and `ParityPublicInputs` (noir + TS + the hand-written `noir-protocol-circuits-types` conversion wrappers). The circuit is kept (sized-parity, per FI-06's topology decision), so no VK-tree indices move.
- Retires `NUMBER_OF_L1_L2_MESSAGES_PER_ROLLUP` (replaced everywhere by `MAX_L1_TO_L2_MSGS_PER_CHECKPOINT`, same value 1024) and the now-dead `NUM_MSGS_PER_BASE_PARITY`, `NUM_BASE_PARITY_PER_ROOT_PARITY`, and `L1_TO_L2_MSG_SUBTREE_ROOT_SIBLING_PATH_LENGTH`. Generated constants (`constants.gen.ts`, `ConstantsGen.sol`) were regenerated via `remake-constants`.
- Deletes `FrontierLib.sol` and its tests (`Parity.t.sol`, the `Frontier` harness and merkle test) now that their last users are gone.

## Verification

- Noir: `types::checkpoint_header` (5/5), `rollup_lib::parity` and `checkpoint_root::parity_tests` (15/15), and the checkpoint-root header-assembly path pass locally. The full `checkpoint_root` suite (68 tests) is too slow to run in full locally and rides CI.
- Solidity: `forge test` green across `Rollup`, `RollupFieldRange`, `Inbox`, `InboxBuckets`, `RollupGetters` (76/76), covering header hashing and the regenerated checkpoint fixtures.
- Jest: `checkpoint_header`, the p2p wire-compat suites, and the parity/messaging suites pass (43 tests), confirming the fixture regeneration is byte-consistent.
- VK/artifact/Prover.toml regeneration and the generated Noir ABI types ride CI (local `yarn build` in `noir-protocol-circuits-types` / `simulator` / `prover-client` is expected to fail on stale circuit artifacts until then); e2e and the broken-zone typecheck are the CI-of-record for those.

## Stack

Part of the Fast Inbox stack #24784 through #24791 (umbrella #24774): A-1379 (#24784), A-1380 (#24785), A-1381 (#24786), A-1382 (#24787), A-1383 (#24788), A-1384 flip (#24789), A-1385 (#24790), A-1386 (#24791, base of this PR).

Resolves A-1387.


## l1 publisher inbox consumption via the streaming selector (CI fix, pass-8)

The A-1387 inHash-removal rewrite left `l1_publisher.integration.test.ts` hardcoding `previousInboxRollingHash = Fr.ZERO` and `bucketHint = 0n` while consuming real L1→L2 messages, so message-consuming checkpoints reverted on L1 with `Rollup__InvalidInboxRollingHash` / `UnconsumedInboxMessages` (masked every prior CI round behind earlier failures). `INBOX_LAG_SECONDS` is an L1-time censorship cutoff, not whole checkpoints, so no checkpoint-depth shift register reproduces the aged bucket set. The test now mirrors the real Inbox buckets into its `MockL1ToL2MessageSource` and reuses the production `selectInboxBucketForBlock` (which mirrors `ProposeLib.validateInboxConsumption`) to pick exactly the buckets each checkpoint must consume, deriving the consumed bundle, the propose bucket hint, and the header rolling hash from that one selection. Box: 13/13, verified at a-1387 and a-1388.


Replaces #24792.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant