Skip to content

feat(fast-inbox): validator streaming acceptance conditions, flag off (A-1383) - #24788

Closed
spalladino wants to merge 7 commits into
spl/a-1382-sequencer-streamingfrom
spl/a-1383-validator-streaming
Closed

feat(fast-inbox): validator streaming acceptance conditions, flag off (A-1383)#24788
spalladino wants to merge 7 commits into
spl/a-1382-sequencer-streamingfrom
spl/a-1383-validator-streaming

Conversation

@spalladino

Copy link
Copy Markdown
Contributor

Implements A-1383: the validator's streaming-Inbox acceptance conditions (AZIP-22 Fast Inbox), behind the shared streamingInbox flag (default off). Flag off ⇒ byte-identical behavior. Stacked on #24787 (A-1382).

The four block-proposal checks (validator-client/src/streaming_inbox_checks.ts, wired in proposal_handler.handleBlockProposal)

  • Exists: the referenced bucket resolves in the node's own Inbox view and its consensus rolling hash matches the reference. An unknown bucket is an immediate reject (bucket_unknown); the bounded-wait soft path is A-1393. The reference is trusted only as a bucketSeq lookup hint — timestamp/counts are read from the locally resolved bucket.
  • Moves forward: the bucket's cumulative total is at least the parent block's (equal ⇒ empty bundle).
  • Not too new: the bucket is at least INBOX_LAG_SECONDS old at validation time (dateProvider.now(), inclusive boundary).
  • Caps: the per-block count and the running per-checkpoint total fit MAX_L1_TO_L2_MSGS_PER_BLOCK / MAX_L1_TO_L2_MSGS_PER_CHECKPOINT.

The derived bundle is fed into per-block re-execution (insertMessagesPerBlock threaded through openCheckpointreexecuteTransactions); the existing state-ref comparison after re-execution stays the final arbiter.

Last-block censorship check (validateCheckpointProposal)

Before attesting, the minimum-consumption rule runs via the shared isInboxConsumptionSufficient predicate: the first unconsumed bucket must be absent, past the cutoff, or a cap-escape. A mandatory unconsumed bucket rejects the checkpoint (no attestation, inbox_consumption_insufficient).

Shared-predicate extraction (refactor commit)

The cutoff formula and the minimum-consumption/cap-escape rule moved to stdlib/messaging/inbox_consumption, so the sequencer's bucket selection and the validator share one source of truth. The sequencer's selectStreamingBundle now calls getInboxCutoffTimestamp instead of inlining it — behavior unchanged (selector suite still green).

Corrected cutoff anchor

The cutoff is getTimestampForSlot(slot - 1) - INBOX_LAG_SECONDS (mirroring ProposeLib.validateInboxConsumption), not getBuildFrameStart. The A-1371 §13 cross-layer vectors are pinned in inbox_consumption.test.ts.

Parent-ref derivation

Parent/last-consumed buckets are resolved from L1-to-L2 tree leaf counts (compact post-flip indexing == bucket cumulative total) via a new getInboxBucketByTotalMsgCount archiver lookup. A leaf count that does not land on a bucket boundary (a pre-flip padded parent) is rejected/deferred; the flip (A-1384) closes that gap.

CI must typecheck what can't be verified locally

tsc -b cannot complete locally due to pre-existing noir-protocol-circuits-types breakage (missing artifacts / stale generated types), which cascades to archiver/validator-client. stdlib builds clean; no errors in the changed files outside the broken zone. Runnable unit suites are green: streaming_inbox_checks (14), inbox_consumption (10), proposal_handler (34, incl. flag-off unchanged + new streaming/censorship tests), message_store (35), stdlib archiver interface (RPC schema). CI is the typecheck of record for the unbuildable zone.

Out of scope: bounded-wait (A-1393), the flip (A-1384), reorg rejection (A-1389), the full streaming checkpoint rebuild (A-1385).

@spalladino spalladino changed the title feat: validator streaming acceptance conditions, flag off (A-1383) feat(fast-inbox): validator streaming acceptance conditions, flag off (A-1383) Jul 19, 2026
@spalladino
spalladino force-pushed the spl/a-1383-validator-streaming branch from a102839 to e73dd44 Compare July 19, 2026 14:05
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from 8cc06b5 to 2e9f13e Compare July 19, 2026 14:05
@spalladino
spalladino force-pushed the spl/a-1383-validator-streaming branch from e73dd44 to ee895d3 Compare July 19, 2026 17:57
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from 2e9f13e to d06df3f Compare July 19, 2026 17:57
@spalladino
spalladino force-pushed the spl/a-1383-validator-streaming branch from ee895d3 to 8c8328f Compare July 19, 2026 20:48
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from d06df3f to a88a509 Compare July 19, 2026 20:48
@spalladino
spalladino force-pushed the spl/a-1383-validator-streaming branch from 8c8328f to 5d841f5 Compare July 20, 2026 15:30
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch 2 times, most recently from 19e9567 to 896ea89 Compare July 20, 2026 17:28
@spalladino
spalladino force-pushed the spl/a-1383-validator-streaming branch from 5d841f5 to 38daa76 Compare July 20, 2026 17:28
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from 896ea89 to a1feb7d Compare July 20, 2026 21:21
@spalladino
spalladino force-pushed the spl/a-1383-validator-streaming branch 2 times, most recently from 28efaeb to 7b89eb4 Compare July 21, 2026 02:58
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch from a1feb7d to 4bbd6cd Compare July 21, 2026 02:59
@spalladino
spalladino force-pushed the spl/a-1383-validator-streaming branch from 7b89eb4 to 7e29709 Compare July 21, 2026 03:39
@spalladino
spalladino force-pushed the spl/a-1382-sequencer-streaming branch 2 times, most recently from 00a7d98 to f0cccf4 Compare July 27, 2026 20:53
@spalladino
spalladino requested a review from LeilaWang as a code owner July 27, 2026 20:53
@spalladino
spalladino force-pushed the spl/a-1383-validator-streaming branch from 7e29709 to 812d923 Compare July 27, 2026 20:53
…383)

Move the streaming-Inbox cutoff formula and the mandatory-consumption / cap-escape
rule into a shared `stdlib/messaging/inbox_consumption` module so the sequencer's
bucket selection and the validator's last-block censorship check share one source of
truth for the boundary semantics.

- `getInboxCutoffTimestamp(slot, l1Constants, lagSeconds)`: buildFrameStart(slot) -
  lagSeconds, mirroring `ProposeLib.validateInboxConsumption`. The sequencer's
  `selectStreamingBundle` now calls it instead of inlining the computation (behavior
  unchanged).
- `isInboxConsumptionSufficient(...)`: the none / past-cutoff (strict >) / cap-escape
  predicate.
- Tests pin the A-1371 section-13 cross-layer vectors (genesisTime=100000,
  slotDuration=36): the cutoff table and the S=10 mandatory-consumption boundary.
…REAMING_INBOX (A-1383)

Behind the shared `streamingInbox` flag (default off), the validator applies the AZIP-22
Fast Inbox acceptance conditions in place of the legacy per-checkpoint inHash comparison.
Flag off, behavior is byte-identical.

Block proposals (`proposal_handler.handleBlockProposal`), via a new pure
`streaming_inbox_checks` module:
- exists: the referenced bucket resolves in the node's own Inbox view and its rolling
  hash matches the reference (unknown bucket is an immediate reject; the bounded wait is
  A-1393);
- moves forward: the bucket's cumulative total is at least the parent block's;
- not too new: the bucket is at least INBOX_LAG_SECONDS old at validation time
  (dateProvider.now, inclusive boundary);
- caps: the per-block count and the running per-checkpoint total fit their caps.
The derived bundle is fed into per-block re-execution (insertMessagesPerBlock threaded
through openCheckpoint/reexecuteTransactions), whose state-ref comparison stays the final
arbiter.

Checkpoint proposals (`validateCheckpointProposal`): the last-block minimum-consumption
(censorship) rule runs before attesting, via the shared `isInboxConsumptionSufficient`
predicate and the corrected cutoff anchor `getTimestampForSlot(slot - 1) -
INBOX_LAG_SECONDS`; a mandatory unconsumed bucket rejects the checkpoint (no attestation).

Parent/last-consumed buckets are resolved from L1-to-L2 tree leaf counts via a new
`getInboxBucketByTotalMsgCount` archiver lookup (compact post-flip indexing); a leaf count
that does not land on a bucket boundary (a pre-flip padded parent) is rejected/deferred,
the flip (A-1384) closes that gap.

The flag is picked up on the validator config via pickConfigMappings of the shared
`streamingInbox` key. New reasons are non-slashable while the path lands. Full tsc -b
cannot run locally (pre-existing noir-protocol-circuits-types breakage); CI is the
typecheck of record for validator-client. Unit tests cover each check, the caps and lag
boundaries, running-total accumulation, the censorship predicate, and the handler wiring.
…lidator (A-1383)

Consolidate the three `BigInt(block.header.state.l1ToL2MessageTree.nextAvailableLeafIndex)`
reads into a single `blockLeafCount` helper.
The archiver only writes bucket snapshots for ingested messages, whose first bucket is
sequence 1, so `getInboxBucket(0)` / `getInboxBucketByTotalMsgCount(0)` returned undefined.
That broke streaming resolution of a genesis parent (first block would reject
`parent_bucket_unresolved`), of an empty genesis block's own bucket reference, and let an
empty checkpoint skip the last-block censorship enforcement. Synthesize the sequence-0
sentinel (rolling hash 0, total 0) on read, mirroring the on-chain Inbox's base case, with
a store-level test exercising it through a populated MessageStore.
Consumers of the L1-to-L2 message source hold block or checkpoint leaf counts,
not Inbox bucket sequences, so each of them resolved the boundary buckets itself
before asking for the message range. Add
`getL1ToL2MessagesBetweenLeafCounts(start, end)`, which resolves both boundaries
internally and throws `InboxBucketBoundaryNotSyncedError` when a count does not
land on a bucket boundary this archiver has synced.
@spalladino

Copy link
Copy Markdown
Contributor Author

Superseded by #25037.

The Fast Inbox stack has been regrouped from 20 per-issue PRs (plus 2 umbrellas) down to 3 area PRs plus an umbrella, and rebased onto merge-train/spartan including #25007 (the noir-projects fnd//labs/ split). This PR's diff is preserved verbatim as a single squashed commit in #25037, and this description is preserved in that commit's message.

New structure: #25036 (circuits + L1) → #25037 (node + flip) → #25038 (cleanup), with #25039 as the full-stack umbrella. The original branch for this PR is left on the remote as a recovery point.

Closing here to cut the rebase and review overhead of maintaining 22 PRs; the work is not abandoned.

@spalladino spalladino closed this Jul 28, 2026
spalladino added a commit that referenced this pull request Jul 28, 2026
…387)

FI-17 of the Fast Inbox (AZIP-22) stack. Stacked on #24791 (A-1386); it removes the legacy `inHash` from the checkpoint header now that the consensus flip (#24789, A-1384) has switched validation to the streaming rolling hash. After this PR, `inboxRollingHash` is the checkpoint's only inbox commitment.

## Consensus-format change

- The checkpoint header loses its `inHash` field across all three preimage layers in lockstep: noir `CheckpointHeader` (`checkpoint_header.nr`), TS `CheckpointHeader` (`stdlib/src/rollup/checkpoint_header.ts`), and Solidity `ProposedHeaderLib` (struct + hash packing). The field ordering is otherwise unchanged; the header shrinks by one field (32 bytes: `CHECKPOINT_HEADER_SIZE_IN_BYTES` 380 to 348, `CHECKPOINT_HEADER_LENGTH` 14 to 13).
- The header-hash fixtures were regenerated from `checkpoint_header.test.ts` (the established workflow) and pasted into the noir tests; the p2p golden-byte fixture (`wire_compat_fixtures.ts`) was refreshed for the checkpoint-proposal serialization, which shrinks by exactly the one removed field. The block-proposal wire fixture is unchanged (block-level `inHash` is out of scope, see A-1388).

## Circuits and constants

- Strips the dead unconstrained `in_hash` pass-through from the sized `InboxParity<S>` circuit and `ParityPublicInputs` (noir + TS + the hand-written `noir-protocol-circuits-types` conversion wrappers). The circuit is kept (sized-parity, per FI-06's topology decision), so no VK-tree indices move.
- Retires `NUMBER_OF_L1_L2_MESSAGES_PER_ROLLUP` (replaced everywhere by `MAX_L1_TO_L2_MSGS_PER_CHECKPOINT`, same value 1024) and the now-dead `NUM_MSGS_PER_BASE_PARITY`, `NUM_BASE_PARITY_PER_ROOT_PARITY`, and `L1_TO_L2_MSG_SUBTREE_ROOT_SIBLING_PATH_LENGTH`. Generated constants (`constants.gen.ts`, `ConstantsGen.sol`) were regenerated via `remake-constants`.
- Deletes `FrontierLib.sol` and its tests (`Parity.t.sol`, the `Frontier` harness and merkle test) now that their last users are gone.

## Verification

- Noir: `types::checkpoint_header` (5/5), `rollup_lib::parity` and `checkpoint_root::parity_tests` (15/15), and the checkpoint-root header-assembly path pass locally. The full `checkpoint_root` suite (68 tests) is too slow to run in full locally and rides CI.
- Solidity: `forge test` green across `Rollup`, `RollupFieldRange`, `Inbox`, `InboxBuckets`, `RollupGetters` (76/76), covering header hashing and the regenerated checkpoint fixtures.
- Jest: `checkpoint_header`, the p2p wire-compat suites, and the parity/messaging suites pass (43 tests), confirming the fixture regeneration is byte-consistent.
- VK/artifact/Prover.toml regeneration and the generated Noir ABI types ride CI (local `yarn build` in `noir-protocol-circuits-types` / `simulator` / `prover-client` is expected to fail on stale circuit artifacts until then); e2e and the broken-zone typecheck are the CI-of-record for those.

## Stack

Part of the Fast Inbox stack #24784 through #24791 (umbrella #24774): A-1379 (#24784), A-1380 (#24785), A-1381 (#24786), A-1382 (#24787), A-1383 (#24788), A-1384 flip (#24789), A-1385 (#24790), A-1386 (#24791, base of this PR).

Resolves A-1387.


## l1 publisher inbox consumption via the streaming selector (CI fix, pass-8)

The A-1387 inHash-removal rewrite left `l1_publisher.integration.test.ts` hardcoding `previousInboxRollingHash = Fr.ZERO` and `bucketHint = 0n` while consuming real L1→L2 messages, so message-consuming checkpoints reverted on L1 with `Rollup__InvalidInboxRollingHash` / `UnconsumedInboxMessages` (masked every prior CI round behind earlier failures). `INBOX_LAG_SECONDS` is an L1-time censorship cutoff, not whole checkpoints, so no checkpoint-depth shift register reproduces the aged bucket set. The test now mirrors the real Inbox buckets into its `MockL1ToL2MessageSource` and reuses the production `selectInboxBucketForBlock` (which mirrors `ProposeLib.validateInboxConsumption`) to pick exactly the buckets each checkpoint must consume, deriving the consumed bundle, the propose bucket hint, and the header rolling hash from that one selection. Box: 13/13, verified at a-1387 and a-1388.


Replaces #24792.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant