From 8ef226b13c0965712266a666ae7d12c69af2f1c7 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Fri, 8 May 2026 10:50:54 +0200 Subject: [PATCH 1/9] engine: add Rest-SSZ spec --- src/engine/refactor-ssz.md | 484 +++++++++++++++++++ src/engine/refactor.md | 960 +++++++++++++++++++++++++++++++++++++ 2 files changed, 1444 insertions(+) create mode 100644 src/engine/refactor-ssz.md create mode 100644 src/engine/refactor.md diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md new file mode 100644 index 000000000..101ef5042 --- /dev/null +++ b/src/engine/refactor-ssz.md @@ -0,0 +1,484 @@ +# Engine API v2 -- SSZ Container Sketches (Amsterdam) + +> **Status:** Sketch. This is a working draft of the concrete SSZ +> container definitions referenced by the Engine API v2 spec +> ([refactor.md](./refactor.md)). Field types, names, and `MAX_*` +> constants are placeholders and need a final review before +> publication. +> +> All conventions in this document follow +> [refactor.md § SSZ encoding conventions](./refactor.md#ssz-encoding-conventions): +> +> - `Optional[T]` ≡ `List[T, 1]` (length 0 = absent, length 1 = present) +> - `String` ≡ `List[byte, MAX_ERROR_BYTES]`, `MAX_ERROR_BYTES = 1024` +> - `ByteList[N]` ≡ `List[byte, N]` +> - `ByteVector[N]` is fixed-size, `Bytes32` etc. are aliases +> - All uints are little-endian + +--- + +## Table of contents + +- [Primitive aliases](#primitive-aliases) +- [`MAX_*` constants](#max-constants) +- [Shared structures](#shared-structures) + - [`Withdrawal`](#withdrawal) + - [`ExecutionPayload` (Amsterdam)](#executionpayload-amsterdam) + - [`PayloadAttributes` (Amsterdam)](#payloadattributes-amsterdam) + - [`ForkchoiceState`](#forkchoicestate) + - [`PayloadStatus`](#payloadstatus) +- [Endpoint containers](#endpoint-containers) + - [`POST /amsterdam/payloads`](#post-amsterdampayloads) + - [`POST /amsterdam/forkchoice`](#post-amsterdamforkchoice) + - [`GET /amsterdam/payloads/{payloadId}`](#get-amsterdampayloadspayloadid) + - [`POST /amsterdam/bodies/hash` and `GET /amsterdam/bodies?...`](#post-amsterdambodieshash-and-get-amsterdambodies) + - [`POST /blobs/v1`](#post-blobsv1) + - [`POST /blobs/v2`](#post-blobsv2) + - [`POST /blobs/v3`](#post-blobsv3) + - [`POST /blobs/v4`](#post-blobsv4) +- [Open sketch questions](#open-sketch-questions) + +--- + +## Primitive aliases + +| Alias | SSZ type | Notes | +| - | - | - | +| `Hash32` | `ByteVector[32]` | block / payload hashes | +| `Root` | `ByteVector[32]` | beacon-block roots, merkle roots | +| `Address` | `ByteVector[20]` | execution-layer 160-bit address | +| `Bloom` | `ByteVector[256]` | logs bloom filter | +| `VersionedHash` | `ByteVector[32]` | EIP-4844 versioned blob hash | +| `Bytes8` | `ByteVector[8]` | `payload_id` | +| `Bytes32` | `ByteVector[32]` | `prevRandao`, generic 32-byte values | +| `Bytes48` | `ByteVector[48]` | KZG commitments and proofs | +| `Uint64` | `uint64` | LE on the wire | +| `Uint256` | `uint256` | LE on the wire (`block_value`, `base_fee_per_gas`) | +| `Boolean` | `bool` | one byte, `0x00` / `0x01` | + +## `MAX_*` constants + +These are sketch values — final values come from a follow-up that +matches the consensus-specs `Amsterdam` preset. They are listed here +for completeness so readers can size the on-wire bounds. + +| Constant | Sketch value | Where it's used | +| - | - | - | +| `MAX_TXS_PER_PAYLOAD` | `1048576` | `ExecutionPayload.transactions` | +| `MAX_BYTES_PER_TX` | `1073741824` | element bound inside `transactions` | +| `MAX_WITHDRAWALS_PER_PAYLOAD` | `16` | `ExecutionPayload.withdrawals`, `PayloadAttributes.withdrawals` | +| `MAX_EXTRA_DATA_BYTES` | `32` | `ExecutionPayload.extra_data` | +| `MAX_BAL_BYTES` | TBD (EIP-7928) | `ExecutionPayload.block_access_list` | +| `MAX_EXECUTION_REQUESTS_PER_PAYLOAD` | TBD (EIP-7685) | `ExecutionPayloadEnvelope.execution_requests` | +| `MAX_BYTES_PER_EXECUTION_REQUEST` | TBD | element bound inside `execution_requests` | +| `MAX_VERSIONED_HASHES_PER_REQUEST` | `128` | `BlobsRequest.versioned_hashes` | +| `MAX_BODIES_REQUEST` | `128` | bodies request and response lists | +| `MAX_BLOBS_REQUEST` | `128` | blobs request and response lists | +| `MAX_BLOBS_PER_PAYLOAD` | `MAX_VERSIONED_HASHES_PER_REQUEST` | `BlobsBundle.commitments`, `.blobs` | +| `CELLS_PER_EXT_BLOB` | `128` (EIP-7594) | cell-proof and custody bitvectors | +| `BYTES_PER_BLOB` | `131072` | one blob (`4096 * 32`) | +| `MAX_ERROR_BYTES` | `1024` | `validation_error`, JSON error `detail` | + +--- + +## Shared structures + +These containers are used by multiple endpoints. They map directly +onto today's JSON-RPC structures with field renaming +(`camelCase` → `snake_case`) and the type changes that follow from +the SSZ encoding conventions. + +### `Withdrawal` + +Same as the consensus-specs `Withdrawal` container. The `amount` +field is now natively LE in SSZ; the `withdrawals.amount` LE-vs-BE +note in shanghai.md goes away. + +``` +Withdrawal { + index: Uint64 + validator_index: Uint64 + address: Address + amount: Uint64 # gwei +} +``` + +### `ExecutionPayload` (Amsterdam) + +Reflects today's [`ExecutionPayloadV4`](./amsterdam.md#executionpayloadv4). +`block_access_list` is a fixed field for Amsterdam (no `Optional[T]` +here — that's only used for cross-fork `BodyEntry` responses; the +Amsterdam `ExecutionPayload` always carries the BAL). + +``` +ExecutionPayload { + parent_hash: Hash32 + fee_recipient: Address + state_root: Hash32 + receipts_root: Hash32 + logs_bloom: Bloom + prev_randao: Bytes32 + block_number: Uint64 + gas_limit: Uint64 + gas_used: Uint64 + timestamp: Uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: Uint256 + block_hash: Hash32 + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] + blob_gas_used: Uint64 + excess_blob_gas: Uint64 + block_access_list: ByteList[MAX_BAL_BYTES] # RLP-encoded EIP-7928 BAL + slot_number: Uint64 +} +``` + +Notes: + +- `block_access_list` is RLP-encoded inside an SSZ `ByteList`. EIP-7928's + encoding is RLP and we don't try to re-encode it as SSZ — the EL + treats it as opaque bytes for transport, decodes it as RLP for + validation. Same pattern as `transactions`. +- `transactions` elements remain RLP-encoded `TransactionType || + TransactionPayload` per EIP-2718. Receiver-side rule: each element + MUST be ≥ 1 byte (see refactor.md § Payload submission). + +### `PayloadAttributes` (Amsterdam) + +Reflects today's [`PayloadAttributesV4`](./amsterdam.md#payloadattributesv4). + +``` +PayloadAttributes { + timestamp: Uint64 + prev_randao: Bytes32 + suggested_fee_recipient: Address + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] + parent_beacon_block_root: Root + slot_number: Uint64 +} +``` + +### `ForkchoiceState` + +Same fields as today's [`ForkchoiceStateV1`](./paris.md#forkchoicestatev1). + +``` +ForkchoiceState { + head_block_hash: Hash32 + safe_block_hash: Hash32 + finalized_block_hash: Hash32 +} +``` + +### `PayloadStatus` + +Used by `POST /payloads` (full enum) and `POST /forkchoice` +(restricted enum — `ACCEPTED` not allowed). + +``` +PayloadStatus { + status: uint8 # see enum below + latest_valid_hash: Optional[Hash32] + validation_error: Optional[String] +} +``` + +Status enum: + +| Value | Name | Used by | +| - | - | - | +| `1` | `VALID` | both | +| `2` | `INVALID` | both | +| `3` | `SYNCING` | both | +| `4` | `ACCEPTED` | `POST /payloads` only | + +`INVALID_BLOCK_HASH` is removed (already supplanted by `INVALID`). +`POST /forkchoice` MUST return `1`/`2`/`3` only; CLs MUST treat a +`4` from `/forkchoice` as a protocol error. + +`Optional[String]` resolves to `List[List[byte, MAX_ERROR_BYTES], 1]`. + +--- + +## Endpoint containers + +### `POST /amsterdam/payloads` + +Replaces `engine_newPayloadV5`. + +#### Request + +``` +ExecutionPayloadEnvelope { + payload: ExecutionPayload + parent_beacon_block_root: Root + execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] +} +``` + +`expected_blob_versioned_hashes` is removed (the EL recomputes it +from `payload.transactions`). + +#### Response + +`PayloadStatus` (full enum, `1`/`2`/`3`/`4`). + +### `POST /amsterdam/forkchoice` + +Replaces `engine_forkchoiceUpdatedV4`. + +#### Request + +``` +ForkchoiceUpdate { + forkchoice_state: ForkchoiceState + payload_attributes: Optional[PayloadAttributes] + custody_columns: Optional[Bitvector[CELLS_PER_EXT_BLOB]] +} +``` + +#### Response + +``` +ForkchoiceUpdateResponse { + payload_status: PayloadStatus # restricted: VALID | INVALID | SYNCING + payload_id: Optional[Bytes8] +} +``` + +### `GET /amsterdam/payloads/{payloadId}` + +Replaces `engine_getPayloadV6`. + +#### Response + +``` +BuiltPayload { + payload: ExecutionPayload + block_value: Uint256 + blobs_bundle: BlobsBundleV2 # see consensus-specs Osaka + execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] + should_override_builder: Boolean +} + +BlobsBundleV2 { + commitments: List[Bytes48, MAX_BLOBS_PER_PAYLOAD] + proofs: List[Bytes48, MAX_BLOBS_PER_PAYLOAD * CELLS_PER_EXT_BLOB] + blobs: List[ByteVector[BYTES_PER_BLOB], MAX_BLOBS_PER_PAYLOAD] +} +``` + +`commitments` and `blobs` MUST have equal length; `proofs` MUST +have length `len(blobs) * CELLS_PER_EXT_BLOB` (mirrors the +`engine_getPayloadV5` rule from osaka.md). + +### `POST /amsterdam/bodies/hash` and `GET /amsterdam/bodies?...` + +Replace `engine_getPayloadBodiesByHashV2` and +`engine_getPayloadBodiesByRangeV2`. Both return the same response +container. + +#### Request — `/bodies/hash` + +``` +BodiesByHashRequest { + block_hashes: List[Hash32, MAX_BODIES_REQUEST] +} +``` + +#### Request — `/bodies?from=N&count=M` + +URL query parameters; no SSZ request body. + +#### Response + +``` +BodiesResponse { + entries: List[BodyEntry, MAX_BODIES_REQUEST] +} + +BodyEntry { + available: Boolean + body: ExecutionPayloadBody +} + +# /amsterdam/bodies/... uses this Amsterdam-fork ExecutionPayloadBody +ExecutionPayloadBody { + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] + withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]] # [] pre-Shanghai + block_access_list: Optional[ByteList[MAX_BAL_BYTES]] # [] pre-Amsterdam or pruned +} +``` + +A CL on the Cancun schema would call `/cancun/bodies/...` and receive +a Cancun-shaped `ExecutionPayloadBody` (no `block_access_list` field +at all). The Cancun-fork variant is sketched here for clarity: + +``` +# /cancun/bodies/... ExecutionPayloadBody (for reference) +ExecutionPayloadBody { + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] + withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]] # [] pre-Shanghai +} +``` + +### `POST /blobs/v1` + +Replaces `engine_getBlobsV1` (Cancun whole-blob). + +#### Request + +``` +BlobsV1Request { + versioned_hashes: List[VersionedHash, MAX_BLOBS_REQUEST] +} +``` + +#### Response + +``` +BlobsV1Response = Optional[List[BlobV1Entry, MAX_BLOBS_REQUEST]] + +BlobV1Entry { + available: Boolean + contents: BlobAndProofV1 +} + +BlobAndProofV1 { + blob: ByteVector[BYTES_PER_BLOB] + proof: Bytes48 +} +``` + +When `available == false`, `contents` carries zero-valued bytes (a +`BYTES_PER_BLOB`-byte zero blob and a 48-byte zero proof). The outer +`Optional` returns `[]` when the EL cannot serve the request at all. + +### `POST /blobs/v2` + +Replaces `engine_getBlobsV2` (Osaka all-or-nothing cell proofs). + +#### Request — same as `/v1` + +``` +BlobsV2Request { + versioned_hashes: List[VersionedHash, MAX_BLOBS_REQUEST] +} +``` + +#### Response + +``` +BlobsV2Response = Optional[List[BlobV2Entry, MAX_BLOBS_REQUEST]] + +BlobV2Entry { + available: Boolean # always true for /v2 (all-or-nothing); included for shape symmetry + contents: BlobAndProofV2 +} + +BlobAndProofV2 { + blob: ByteVector[BYTES_PER_BLOB] + proofs: List[Bytes48, CELLS_PER_EXT_BLOB] +} +``` + +All-or-nothing: if any requested blob is missing, the outer +`Optional` returns `[]` and no per-entry data is sent. CLs that need +partial responses use `/v3`. + +### `POST /blobs/v3` + +Replaces `engine_getBlobsV3` (Osaka partial responses with cell +proofs). + +#### Request — same as `/v2` + +#### Response + +Same shape as `/v2` (`BlobV2Entry` reused), but missing blobs +surface as `available=false` per entry rather than collapsing the +whole response to `[]`. Outer `Optional` returns `[]` only when the +EL cannot serve the request at all (e.g. syncing). + +``` +BlobsV3Response = Optional[List[BlobV2Entry, MAX_BLOBS_REQUEST]] +``` + +### `POST /blobs/v4` + +Replaces `engine_getBlobsV4` (Amsterdam cell-range selection). + +#### Request + +``` +BlobsV4Request { + versioned_hashes: List[VersionedHash, MAX_BLOBS_REQUEST] + indices_bitarray: Bitvector[CELLS_PER_EXT_BLOB] +} +``` + +#### Response + +``` +BlobsV4Response = Optional[List[BlobV4Entry, MAX_BLOBS_REQUEST]] + +BlobV4Entry { + available: Boolean + contents: BlobCellsAndProofs +} + +BlobCellsAndProofs { + blob_cells: List[Optional[ByteVector[BYTES_PER_CELL]], CELLS_PER_EXT_BLOB] + proofs: List[Optional[Bytes48], CELLS_PER_EXT_BLOB] +} +``` + +Per the Amsterdam spec: only the indices set in the request's +`indices_bitarray` carry a value; all other indices are `[]`. Within +the requested indices, individual missing cells are also `[]`, and +the corresponding `proofs` entry MUST also be `[]` (`null` in the +old spec). + +`BYTES_PER_CELL` = `BYTES_PER_BLOB / CELLS_PER_EXT_BLOB` = `1024` +(EIP-7594). + +--- + +## Open sketch questions + +These are the items left to decide before promoting this sketch to +the canonical Amsterdam SSZ schema: + +1. **`MAX_*` placeholder values.** Several constants above are + `TBD` or sketch-only. They need to be pinned to the + consensus-specs `Amsterdam` preset values once those land. +2. **`MAX_BAL_BYTES`.** EIP-7928 defines the BAL but doesn't yet + pin a numeric upper bound that's friendly for SSZ. We need a + concrete number; otherwise the SSZ schema can't round-trip. +3. **`Bitvector` SSZ encoding for `indices_bitarray` and + `custody_columns`.** Both are `Bitvector[CELLS_PER_EXT_BLOB]` + = `Bitvector[128]` = 16 bytes packed. Double-check that's the + reading the Amsterdam spec wants (it currently describes it as + "16 bytes interpreted as a bitarray"). +4. **`should_override_builder` typing.** SSZ has `bool` but it's + a 1-byte field. Keeping it inside `BuiltPayload` (rather than + moving to a header) was the [refactor.md](./refactor.md) + decision; this sketch follows that. +5. **`PayloadStatus` enum encoding.** A `uint8` with sentinel + values matches the JSON-RPC enum; SSZ has no native enum type + so this is the cleanest mapping. Alternative: `Container { ... }` + wrapping a `uint8`. Open for discussion. +6. **`ExecutionPayloadBody` shared definition.** Today every fork + redefines `ExecutionPayloadBody` from scratch. The new spec + would benefit from a small set of fork-named containers + (`ExecutionPayloadBodyParis`, `ExecutionPayloadBodyShanghai`, + `ExecutionPayloadBodyAmsterdam`, …) with the URL `{fork}` + selecting which one. Not worked out here. +7. **Naming convention.** The legacy spec used `camelCase`; this + sketch uses `snake_case` to match consensus-specs. Worth + confirming. +8. **`ByteVector[BYTES_PER_BLOB]` vs `ByteList[BYTES_PER_BLOB]`.** + A blob is fixed-size (131072 bytes), so `ByteVector` is the + correct typing. Verify against consensus-specs to keep + alignment. diff --git a/src/engine/refactor.md b/src/engine/refactor.md new file mode 100644 index 000000000..f0aa6ac14 --- /dev/null +++ b/src/engine/refactor.md @@ -0,0 +1,960 @@ +# Engine API -- Refactor Proposal (REST + SSZ) + +> **Status:** Draft / discussion document. This file proposes a v2 of the +> Engine API that moves from JSON-RPC over a single endpoint to a +> resource-oriented HTTP/REST API where request and response bodies are +> SSZ-encoded. It also takes the opportunity to simplify the surface that +> has accumulated since Paris. +> +> **Target fork:** Amsterdam. The new API ships *as* the Amsterdam Engine +> API; clients implement it instead of `engine_*` JSON-RPC at the +> Amsterdam activation timestamp. + +This document is meant to be read alongside the existing fork-scoped specs +([Paris](./paris.md), [Shanghai](./shanghai.md), [Cancun](./cancun.md), +[Prague](./prague.md), [Osaka](./osaka.md), [Amsterdam](./amsterdam.md)). +Concrete byte-level structures are deferred to a later iteration; the goal +here is to align on the *shape* of the new API. + +--- + +## Table of contents + +- [Mapping from old → new](#mapping-from-old--new) +- [Resource model (overview)](#resource-model-overview) +- [Endpoints](#endpoints) + - [Payload submission](#payload-submission) + - [Forkchoice update](#forkchoice-update) + - [Payload retrieval](#payload-retrieval) + - [Historical bodies](#historical-bodies) + - [Blob pool](#blob-pool) + - [Capabilities & identification](#capabilities--identification) +- [Error model](#error-model) +- [Versioning model](#versioning-model) +- [Authentication](#authentication) +- [Transport & framing](#transport--framing) +- [SSZ encoding conventions](#ssz-encoding-conventions) +- [Message ordering & idempotency](#message-ordering--idempotency) +- [Motivation](#motivation) + - [Goals & non-goals](#goals--non-goals) + - [Why move away from JSON-RPC?](#why-move-away-from-json-rpc) + - [Why SSZ?](#why-ssz) + - [Simplifications & removed concepts](#simplifications--removed-concepts) + - [Summary of design decisions](#summary-of-design-decisions) + +> **Reading order note.** The endpoint sketches reference SSZ types +> like `Optional[T]`, `BodyEntry`, and `BlobEntry`. If a definition +> isn't immediately clear, jump to +> [SSZ encoding conventions](#ssz-encoding-conventions) and +> [Message ordering & idempotency](#message-ordering--idempotency) +> further down — they fully define the wire-level details. + +--- + +## Mapping from old → new + +If you're migrating from the JSON-RPC engine API, this is the lookup +table. Detail on each new endpoint follows in the sections below. + +| Old method | New endpoint | Notes | +| - | - | - | +| `engine_newPayloadV{1..5}` | `POST /{fork}/payloads` | `parentBeaconBlockRoot` and `executionRequests` folded into the SSZ envelope; `expectedBlobVersionedHashes` removed; `INVALID_BLOCK_HASH` removed from the status enum | +| `engine_forkchoiceUpdatedV{1..4}` | `POST /{fork}/forkchoice` | one atomic call; carries forkchoice state, optional `payload_attributes`, and (Amsterdam+) optional `custody_columns` | +| `engine_getPayloadV{1..6}` | `GET /{fork}/payloads/{id}` | poll-style, same semantics as today | +| `engine_getPayloadBodiesByHashV{1,2}` | `POST /{fork}/bodies/hash` | `{fork}` selects the response *schema* (not the era of requested blocks); `POST` because hash lists are too large for URLs | +| `engine_getPayloadBodiesByRangeV{1,2}` | `GET /{fork}/bodies?from=...&count=...` | `{fork}` selects the response schema | +| `engine_getBlobsV1` | `POST /blobs/v1` | independently versioned; legacy version numbers carry forward | +| `engine_getBlobsV2` | `POST /blobs/v2` | all-or-nothing cell proofs | +| `engine_getBlobsV3` | `POST /blobs/v3` | partial-response cell proofs | +| `engine_getBlobsV4` | `POST /blobs/v4` | cell-range selection | +| `engine_getClientVersionV1` | `GET /identity` + `X-Engine-Client-Version` request header | unscoped | +| `engine_exchangeCapabilities` | `GET /capabilities` | unscoped | +| `engine_exchangeTransitionConfigurationV1` | *removed* | already deprecated since Cancun | + +--- + +## Resource model (overview) + +Hot-path endpoints are scoped under `/engine/v2/{fork}/...`. Diagnostic +endpoints are unscoped. + +| Resource | Endpoint | Purpose | +| - | - | - | +| Payload | `POST /engine/v2/{fork}/payloads` | Submit a payload received from the CL gossip network for the EL to validate / import. Replaces `engine_newPayload`. | +| Payload | `GET /engine/v2/{fork}/payloads/{payloadId}` | Retrieve a built payload by id. Replaces `engine_getPayload`. CL polls when it wants a fresher snapshot. | +| Forkchoice | `POST /engine/v2/{fork}/forkchoice` | Atomic forkchoice update: update head/safe/finalized, optionally start a payload build, optionally update custody set. Replaces `engine_forkchoiceUpdated`. | +| Bodies | `POST /engine/v2/{fork}/bodies/hash` | Replaces `engine_getPayloadBodiesByHash`. Fork-scoped: `{fork}` selects the *response schema*, not the fork of the requested blocks. | +| Bodies | `GET /engine/v2/{fork}/bodies?from=N&count=M` | Replaces `engine_getPayloadBodiesByRange`. Fork-scoped on response shape. | +| Blob pool | `POST /engine/v2/blobs/v{1..4}` | Replaces `engine_getBlobsV{1..4}`. The `vN` segment carries forward the legacy version numbers; `/v4` is the Amsterdam cell-range variant, `/v1` is the original Cancun whole-blob variant, and intermediate revisions live alongside. ELs MUST serve at least the current-fork revision (`/v4` for Amsterdam) and MAY serve older revisions alongside. | +| Capabilities | `GET /engine/v2/capabilities` | Replaces `engine_exchangeCapabilities`. Unscoped; advertises supported forks, `/blobs/vN` revisions, and per-endpoint request-size limits. | +| Identity | `GET /engine/v2/identity` | Replaces `engine_getClientVersion`. Unscoped. | + +Every hot-path body uses SSZ; every metadata endpoint uses JSON. + +--- + +## Endpoints + +### Payload submission + +#### `POST /engine/v2/{fork}/payloads` + +Replaces `engine_newPayloadV{1..5}`. + +- **Request body:** SSZ-encoded `ExecutionPayloadEnvelope`, a container + that bundles together everything that today travels alongside the + payload as separate JSON-RPC params: + + ``` + ExecutionPayloadEnvelope { + payload: ExecutionPayload # the fork's payload SSZ container + parent_beacon_block_root: Root # was a separate param since Cancun + execution_requests: List[Bytes, MAX_REQUESTS] # was a separate param since Prague + } + ``` + + `expected_blob_versioned_hashes` is **removed**: it was a + defense-in-depth cross-check, but the block-hash check already covers + the transactions, so the EL recomputes the array from + `payload.transactions` during validation and a mismatch between CL + and EL views surfaces as `INVALID` exactly as before. + +- **Response body:** SSZ-encoded `PayloadStatus`: + + ``` + PayloadStatus { + status: uint8 # VALID=1, INVALID=2, SYNCING=3, ACCEPTED=4 + latest_valid_hash: Optional[Hash32] + validation_error: Optional[String] + } + ``` + + `INVALID_BLOCK_HASH` is dropped (already supplanted by `INVALID`). + `ACCEPTED` is **kept** — CLs rely on it during sync to acknowledge + side-branch payloads that are well-formed but don't extend the + canonical chain. + +- **HTTP status:** `200 OK` for any of the four validation outcomes. + Validation results are not transport errors. + +### Forkchoice update + +#### `POST /engine/v2/{fork}/forkchoice` + +Replaces `engine_forkchoiceUpdatedV{1..4}`. This is the **single +atomic** call that updates the EL's forkchoice state, optionally +triggers a payload build, and (post-Amsterdam) optionally updates the +CL's custody set. Atomicity matters: the CL relies on the EL having +applied the new head before — and only if — the build is started, and +on the build being keyed against the freshly-applied head. + +- **Request body:** SSZ-encoded `ForkchoiceUpdate`: + + ``` + ForkchoiceUpdate { + forkchoice_state: ForkchoiceState # head / safe / finalized + payload_attributes: Optional[PayloadAttributes] # if present, start a build on top of head + custody_columns: Optional[Bitvector[CELLS_PER_EXT_BLOB]] # Amsterdam+, optional + } + ``` + + All three fields are processed in one transaction: the EL MUST apply + the forkchoice state, then (if `payload_attributes` is present and + the new head is `VALID`) start the build, then (if `custody_columns` + is present) update the custody set, all before returning. If the + forkchoice update fails, no build is started and no custody change + is applied. + +- **Response body:** SSZ-encoded `ForkchoiceUpdateResponse`: + + ``` + ForkchoiceUpdateResponse { + payload_status: PayloadStatus # VALID | INVALID | SYNCING + payload_id: Optional[Bytes8] # server-assigned opaque token; set iff a build was started + } + ``` + + The `payload_id` is an **opaque server-assigned token**. The EL + chooses how to mint it (counter, random, hash-tree-root over the + attributes — anything). CLs MUST treat it as opaque bytes and MUST + NOT recompute or validate its contents. This is a change from + today's behavior where both sides derived an 8-byte hash over + `(headBlockHash, payloadAttributes)`. + +- **HTTP status:** `200 OK` for all three payload-status outcomes. + `409 Conflict` is returned for an inconsistent forkchoice state + (today's `-38002`); `422 Unprocessable Entity` for invalid + `payload_attributes` (today's `-38003`); `409 Conflict` for a too-deep + reorg (today's `-38006`). + +- **Skip-allowed semantics:** the EL MAY skip applying the forkchoice + state and instead return `{VALID, latest_valid_hash: head}` if the + new `head` is a `VALID` ancestor of the latest known finalized block. + This preserves the existing Paris-spec rule (point 2 of the + `engine_forkchoiceUpdated` specification) and is deliberate: a CL + that emits a malformed or stale FCU referencing a head behind + finalization should not be able to roll the EL back. We keep the + behaviour that has caught buggy CLs in the past. + +- **Stale-fork URL:** an FCU at `/engine/v2/{fork}/forkchoice` + referencing a `head` from an earlier fork is **allowed**, *as long + as `payload_attributes` is absent*. The CL needs to update head / + safe / finalized across fork boundaries during sync and reorg + recovery, and the URL fork has no bearing on which historical + block can be referenced. + + If `payload_attributes` is present, the URL `{fork}` MUST match + the fork that the new payload would belong to (i.e. the fork + determined by `payload_attributes.timestamp`). Mismatch returns + `400 unsupported-fork`. Building a payload is the only operation + where the URL fork is load-bearing on shape, so it's the only one + we strictly police. + +- **Custody-set semantics** (Amsterdam+): the custody update runs + independently of the forkchoice processing flow, matching the + Amsterdam spec's "MUST run custody set update independently to the + fork choice update". An execution-time custody-set error MUST NOT + affect the `payload_status` returned for the forkchoice update. + A `custody_columns` value, once accepted, remains in effect until + the next `POST /forkchoice` whose body *also* contains a + `custody_columns` field. FCUs that omit the field leave the + custody set unchanged. + +- **No body cap.** `POST /forkchoice` bodies are bounded by the SSZ + schema's `MAX_*` constants (small for `ForkchoiceState` and + `PayloadAttributes`, fixed for `custody_columns`). No additional + HTTP-layer cap is imposed. + +### Payload retrieval + +#### `GET /engine/v2/{fork}/payloads/{payloadId}` + +Replaces `engine_getPayloadV{1..6}`. + +- **Response body:** SSZ-encoded `BuiltPayload`: + + ``` + BuiltPayload { + payload: ExecutionPayload + block_value: Uint256 + blobs_bundle: BlobsBundle + execution_requests: List[Bytes, MAX_REQUESTS] + should_override_builder: bool + } + ``` +- **404** if `payloadId` is unknown or expired. + +Polling semantics are unchanged from `engine_getPayload`: the CL calls +`GET /{fork}/payloads/{payloadId}` whenever it wants the latest +snapshot of the build. Each call returns the most recent version +available at the time of receipt; the EL MAY stop the build process +after serving a call. `payloadId` values are opaque server-assigned +tokens issued by `POST /forkchoice`. + +**Token TTL.** A `payloadId` is valid for **at least 10 minutes** +after its issuing `POST /forkchoice` returns. After 10 minutes the +EL MAY garbage-collect the token and respond `404 unknown-payload` +to subsequent `GET`s. ELs MUST NOT recycle a token within its TTL +(no collisions); after expiry the token namespace is free to reuse. +A CL that needs a fresh `payloadId` after expiry simply issues a new +`POST /forkchoice` with the same attributes. + +### Historical bodies + +These endpoints are **fork-scoped on the response schema, not on the +era of the requested blocks**. The `{fork}` segment tells the EL which +`ExecutionPayloadBody` shape to use when serialising the response. +A CL that has just upgraded to the Amsterdam schema can ask for +`/amsterdam/bodies/hash` and receive `block_access_list` populated +for Amsterdam blocks and `[]` (the SSZ optional sentinel — see +[SSZ encoding conventions](#ssz-encoding-conventions)) for older +blocks; a CL still on Cancun asks `/cancun/bodies/hash` and +gets responses serialised against the Cancun container, never seeing +the trailing `block_access_list` field at all. + +This is different from the `/payloads` and `/forkchoice` `{fork}` +segments, where the URL fork *must* match the timestamp of the +referenced block. For `/bodies` the URL fork is purely a schema +selector and the requester chooses freely. + +The blob endpoint takes yet another approach: it carries a `/vN` +revision instead of a `{fork}` segment, because blob protocol +evolution has historically not aligned with fork activations. See +the [Blob pool](#blob-pool) section. + +#### `POST /engine/v2/{fork}/bodies/hash` + +Replaces `engine_getPayloadBodiesByHashV{1,2}`. Uses `POST` so that +large hash lists travel in the request body rather than the URL. + +- **Request body:** SSZ-encoded `List[Hash32, MAX_BODIES_REQUEST]`. + +#### `GET /engine/v2/{fork}/bodies?from=N&count=M` + +Replaces `engine_getPayloadBodiesByRangeV{1,2}`. Range fits comfortably +in the URL. + +- **Response body** (both endpoints): SSZ-encoded + `List[BodyEntry, MAX_BODIES_REQUEST]`. Each `BodyEntry` carries an + `available: boolean` flag (false for unavailable / pruned blocks, + matching today's `null` semantics) and an `ExecutionPayloadBody` + serialised against the **`{fork}` schema from the URL**. Fields + introduced in `{fork}` or earlier are present (with `Optional[T]` + set to `None` for blocks predating the field's introduction); fields + introduced in forks newer than `{fork}` are absent from the + container entirely. See + [SSZ encoding conventions](#ssz-encoding-conventions). + +### Blob pool + +The blob endpoint is **independently versioned**: blobs are looked up +by versioned hash (not by fork), so the `{fork}` URL segment doesn't +help. But the blob protocol *has* evolved on its own clock — four +distinct semantics across two forks (V1 single proof in Cancun, V2 +cell proofs in Osaka, V3 partial responses, V4 cell-range selection +in Amsterdam). The new spec carries those legacy version numbers +forward onto the URL: `engine_getBlobsVN` becomes `POST /blobs/vN`. +ELs **MUST** serve at least the revision matching their current fork +(`/blobs/v4` for Amsterdam) and **MAY** serve any subset of older +revisions alongside; `GET /capabilities` advertises the actual list. + +This is a different versioning axis from the fork-scoped endpoints +(`/{fork}/payloads`, `/{fork}/forkchoice`, `/{fork}/bodies`). Those +track *consensus protocol* changes coupled to fork activations. +`/blobs/vN` tracks *engine-API blob protocol* changes that have +historically not aligned with fork activations. + +All revisions use `POST` so that 128 versioned hashes (8 KiB hex) +don't have to fit in the URL. All revisions return SSZ +`Optional[List[BlobEntry, MAX_BLOBS_REQUEST]]`, where the outer +`Optional` is the "all-or-nothing"/syncing channel (`None` = +"cannot serve this request, retry later or fall back") and each +`BlobEntry` carries an `available: boolean` per-entry flag for +per-blob misses on revisions that support partial responses. +Revision-specific contents live inside `BlobEntry.contents`. + +#### `POST /engine/v2/blobs/v1` + +Replaces `engine_getBlobsV1` (Cancun, single-proof whole-blob). + +- **Request body:** SSZ `List[VersionedHash, MAX_BLOBS_REQUEST]`. +- **Response `BlobEntry.contents`:** `BlobAndProofV1 { blob, proof }` + (one blob, one 48-byte KZG proof). +- Partial responses supported: missing blobs surface as + `available=false` per entry. The outer `Optional` returns `None` + only if the EL cannot serve the request at all (e.g. syncing). + +#### `POST /engine/v2/blobs/v2` + +Replaces `engine_getBlobsV2` (Osaka, all-or-nothing cell proofs). + +- **Request body:** SSZ `List[VersionedHash, MAX_BLOBS_REQUEST]`. +- **Response `BlobEntry.contents`:** `BlobAndProofV2 { blob, proofs }` + (one blob plus `CELLS_PER_EXT_BLOB` cell proofs). +- **All-or-nothing:** if any requested blob is missing, the outer + `Optional[List[...]]` returns `None`. Otherwise all entries have + `available=true`. This matches today's V2 semantics. + +#### `POST /engine/v2/blobs/v3` + +Replaces `engine_getBlobsV3` (Osaka, partial responses with cell +proofs). + +- **Request body:** same as `/v2`. +- **Response:** same `BlobEntry.contents` shape as `/v2`, but missing + blobs surface as `available=false` per entry rather than collapsing + the whole response to `None`. The outer `Optional` returns `None` + only when the EL cannot serve the request at all. + +#### `POST /engine/v2/blobs/v4` + +Replaces `engine_getBlobsV4` (Amsterdam, cell-range selection). + +- **Request body:** SSZ `BlobsV4Request`: + ``` + BlobsV4Request { + versioned_hashes: List[VersionedHash, MAX_BLOBS_REQUEST] + indices_bitarray: Bitvector[CELLS_PER_EXT_BLOB] # which cells to return + } + ``` +- **Response `BlobEntry.contents`:** `BlobCellsAndProofsV1` + (per-cell `blob_cells` and `proofs` arrays, with `Optional[T]` = + `[]` at indices where individual cells are unavailable). + +### Capabilities & identification + +#### `GET /engine/v2/capabilities` + +Returns JSON. The advertisement includes per-endpoint maximum request +sizes so the CL knows how many block-bodies / blob-cells / payloads +the server is willing to serve in one request: + +```json +{ + "supported_forks": ["paris", "shanghai", "cancun", "prague", "osaka", "amsterdam"], + "fork_scoped_endpoints": ["payloads", "forkchoice", "bodies"], + "independently_versioned": { "blobs": ["v1", "v2", "v3", "v4"] }, + "unscoped_endpoints": ["capabilities", "identity"], + "limits": { + "bodies.max_count": 128, + "blobs.max_versioned_hashes": 128, + "payload.max_bytes": 67108864 + } +} +``` + +The `independently_versioned` map advertises endpoints whose URL +carries an explicit `/vN` revision. ELs MAY support multiple +revisions concurrently (e.g. `["v1", "v2"]`); CLs pick whichever they +implement. + +#### `GET /engine/v2/identity` + +Returns JSON `ClientVersion[]` (same shape as today's +`engine_getClientVersionV1`). The CL identifies itself with a +`X-Engine-Client-Version` header on every request, removing the +mutual-exchange handshake. + +--- + +## Error model + +Errors are signalled by HTTP status code and an +`application/problem+json` body (RFC 7807). To keep responses compact, +we use only **two** of the RFC 7807 fields: + +- **`type`** (required) — relative URI identifying the problem class. + Stable across releases. CLs branch on this string. +- **`detail`** (optional) — human-readable, instance-specific message. + Omitted when the EL has nothing more to say than the `type` already + conveys (e.g. canned SSZ-decode failures). + +We deliberately drop the other RFC 7807 fields: + +- `title` would just duplicate `type` (RFC 7807 says it SHOULD NOT + vary between occurrences of the same `type`); CLs can render their + own from a static `type → title` map. +- `status` duplicates the HTTP status line. +- `instance` adds a per-request URI; operators get correlation from + logs already. + +There is **no** legacy `engine_code` extension. CLs migrating from +the JSON-RPC API map old codes to new `type` strings via the table +below; after migration the codes are gone. + +| HTTP status | `type` | Old JSON-RPC code | When | +| - | - | - | - | +| 400 Bad Request | `/engine-api/errors/parse-error` | -32700 | Body is not valid JSON / SSZ | +| 400 Bad Request | `/engine-api/errors/invalid-request` | -32600 | Request shape is wrong (missing required field, etc.) | +| 400 Bad Request | `/engine-api/errors/ssz-decode-error` | (new) | SSZ decode failed; canned error, no `detail` | +| 400 Bad Request | `/engine-api/errors/unsupported-fork` | -38005 | URL `{fork}` is not supported by this EL | +| 404 Not Found | `/engine-api/errors/method-not-found` | -32601 | URL does not match any endpoint | +| 404 Not Found | `/engine-api/errors/unknown-payload` | -38001 | `payloadId` does not exist | +| 409 Conflict | `/engine-api/errors/invalid-forkchoice` | -38002 | Forkchoice state is inconsistent (e.g. finalized not ancestor of head) | +| 409 Conflict | `/engine-api/errors/reorg-too-deep` | -38006 | Reorg depth exceeds the EL's limit | +| 413 Payload Too Large | `/engine-api/errors/request-too-large` | -38004 | Body exceeds an advertised `limits.*` value | +| 422 Unprocessable Entity | `/engine-api/errors/invalid-body` | -32602 | Body decoded fine but has invalid values | +| 422 Unprocessable Entity | `/engine-api/errors/invalid-attributes` | -38003 | `payload_attributes` validation failed | +| 500 Internal Server Error | `/engine-api/errors/internal` | -32603 / -32000 | Unrecoverable server error; `detail` carries the message | + +`type` URIs are written as **relative references** rooted at +`/engine-api/errors/...`. RFC 7807 allows relative URIs, and the +short form keeps error bodies small without losing identifier +stability. CLs MUST treat them as opaque strings — they MUST NOT +attempt to dereference them. + +Example error body: + +```json +{ + "type": "/engine-api/errors/invalid-forkchoice", + "detail": "finalized 0xab.. is not an ancestor of head 0xcd.." +} +``` + +Canned error (no `detail`): + +```json +{ "type": "/engine-api/errors/ssz-decode-error" } +``` + +Validation outcomes for a payload (`VALID`, `INVALID`, `SYNCING`, +`ACCEPTED`) are **not** errors — they remain part of the response +body with HTTP `200 OK`. HTTP errors are reserved for transport, +format, and authentication problems. + +--- + +## Versioning model + +Three layers: + +1. **Major (`/v2`)** — bumped only for breaking transport changes + (e.g. moving away from REST, swapping SSZ for something else). +2. **Per-fork body schema** — selected via the `{fork}` URL segment + on hot-path endpoints (`/{fork}/payloads`, `/{fork}/forkchoice`, + `/{fork}/bodies`). Tracks consensus-protocol changes that ride + along with fork activations. +3. **Per-endpoint revisions** — selected via a `/vN` URL segment on + endpoints whose protocol evolves independently of the fork + schedule (currently just `/blobs/vN`). Tracks engine-API protocol + changes that don't align with fork activations. + +The server advertises which forks and which `/vN` revisions it +understands via `GET /engine/v2/capabilities`. + +`engine_exchangeCapabilities` is **removed**. Instead the server lists +its supported fork schemas and endpoint set in a single JSON document +at `/engine/v2/capabilities`. + +--- + +## Authentication + +Unchanged in spirit: JWT (HS256, 256-bit shared secret). Differences: + +- The token MUST be presented as `Authorization: Bearer ` on every + request. The HTTP/2 connection itself is not authenticated; each + request stream carries its own bearer token. This means a single + long-lived h2 connection between CL and EL is fine — token rotation + happens per-request, not per-connection. +- IPC (UNIX socket) authentication remains optional, as today. +- JWT claims: + - `iat` (required, unchanged from today: ±60s window) + - `id` (optional, unchanged) + - `clv` is **removed** — the CL version travels in the + `X-Engine-Client-Version` request header instead. Keeping it in + two places caused drift; the header is structured, cheap, and + surfaces in normal HTTP logs. +- **Trace propagation:** CLs MAY include a W3C `traceparent` header + on each request. ELs that record a `traceparent` SHOULD propagate + it into their own logs / spans so a slot-level trace can cross the + CL→EL boundary. Not required, not authenticated, purely diagnostic. + +--- + +## Transport & framing + +- **Protocol:** HTTP/2 is **required**. Both TCP and IPC transports + use **h2c** (HTTP/2 cleartext); JWT-on-every-request provides + authentication, so TLS termination is left to a reverse proxy if + the operator wants it. HTTP/2 multiplexing means a single CL→EL + connection can carry the full request mix (forkchoice, payload + submission, blob fetches, body fetches) without head-of-line + blocking. HTTP/1.1 is not supported. +- **Default port:** `8551`, shared with the legacy JSON-RPC engine API. + The two surfaces are distinguished by path: legacy JSON-RPC remains + at `/` (and accepts JSON-RPC method calls), the new API lives under + `/engine/v2/...`. The same JWT secret authenticates both. +- **Base path:** `/engine/v2/{fork}/...`. The `/v2` segment is the + major-protocol version; the `{fork}` segment selects the fork-scoped + body schema (`paris`, `shanghai`, `cancun`, `prague`, `osaka`, + `amsterdam`, …). Adding a fork = adding one path prefix and one set + of SSZ schemas. See [Versioning](#versioning-model). +- **Trailing slashes are forbidden.** `/engine/v2/payloads` is the + canonical form; `/engine/v2/payloads/` MUST return + `404 method-not-found`. No automatic redirect. +- **Request body encoding:** `application/octet-stream` carrying SSZ + bytes for hot-path endpoints. JSON for diagnostic / metadata + endpoints (capabilities, identity, error bodies). +- **Response body encoding:** SSZ for hot-path data, JSON + (`application/json`) for diagnostics and error bodies. +- **Compression:** Servers MAY support `Accept-Encoding: zstd, gzip`. + Not required to implement; CLs MUST tolerate uncompressed responses. + Blob bundles compress well, so operators are encouraged to enable + `zstd` where available. +- **Flow-control window:** servers and CLs **SHOULD** set HTTP/2 + `INITIAL_WINDOW_SIZE` to at least 1 MiB. Default 64 KiB causes + excessive flow-control round-trips for blob bundles and large + `getPayload` responses. `MAX_FRAME_SIZE` and `MAX_HEADER_LIST_SIZE` + use HTTP/2 defaults — not pinned by this spec. +- **Connection lifecycle:** CLs MAY open fresh h2 connections per + request or reuse a long-lived connection. JWT is per-request so + token rotation works the same way in both patterns. + +### Why fork-in-URL instead of method versioning? + +Today every change of a single field bumps the method version +(`engine_newPayloadV1..V5`). The new API puts the fork in the URL: + +``` +POST /engine/v2/amsterdam/payloads +Content-Type: application/octet-stream +Authorization: Bearer + + +``` + +The EL routes by fork segment, parses the body according to that fork's +SSZ schema, and returns a fork-shaped response. Adding a fork = adding +one path prefix and one set of SSZ schemas. URLs stay greppable and +discoverable in logs. + +--- + +## SSZ encoding conventions + +- **`Optional[T]` ≡ `List[T, 1]`.** SSZ has no native optional type; + we use a length-0-or-1 list as the convention (`[]` = absent, + `[t]` = present). The notation `Optional[T]` in this document is + syntactic sugar for `List[T, 1]`. We picked this over + `Union[None, T]` because `List` is universally supported across + SSZ libraries. +- **`String` ≡ `List[byte, MAX_ERROR_BYTES]`** (UTF-8). Empty list + is the empty string; use `Optional[String]` if absence must be + distinguishable from empty. +- **Endianness:** SSZ uints are **little-endian**. The JSON-RPC API + encoded `QUANTITY` values as big-endian hex, so anything that + carries a uint (`block_value`, `gas_used`, `gas_limit`, `timestamp`, + `base_fee_per_gas`, `excess_blob_gas`, `blob_gas_used`, + `block_number`, the `index`/`validatorIndex`/`amount` triple in + `Withdrawal`) flips byte order on the wire. +- **`MAX_*` constants** live in the fork-scoped SSZ schema files + (e.g. `MAX_TXS_PER_PAYLOAD`, `MAX_WITHDRAWALS_PER_PAYLOAD`, + `MAX_BAL_BYTES`, `MAX_VERSIONED_HASHES_PER_REQUEST`). + `MAX_ERROR_BYTES` is global and pinned at `1024` here. + +### Cross-fork response containers + +Endpoints that return data spanning multiple block-eras come in two +flavours: + +1. **Fork-scoped** (e.g. `/bodies`): the URL `{fork}` selects the + container schema. Within that schema, fields that didn't exist in + earlier block-eras are `Optional[T]` (= `[]` for those blocks). + The outer entry carries an explicit `available` flag so + "pruned / unavailable" stays distinct from "field-not-applicable": + + ``` + # /amsterdam/bodies/hash response + BodyEntry { + available: boolean + body: ExecutionPayloadBody + } + + ExecutionPayloadBody { + transactions: List[Transaction, MAX_TXS] + withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS]] # [] pre-Shanghai + block_access_list: Optional[ByteList[MAX_BAL_BYTES]] # [] pre-Amsterdam or if pruned + } + ``` + + A CL on the Cancun schema calls `/cancun/bodies/hash` and + receives the Cancun container (no `block_access_list` field at + all). Old CLs never see schemas they don't know. + +2. **Independently versioned** (e.g. `/blobs/vN`): each revision is + its own container, no nullable optionals across revisions. Old + CLs keep using `/blobs/v1`; new shapes ship as `/blobs/vN+1` + alongside. + +Per-entry fork tags (a `Union` of fork-shaped variants) were +rejected: every fork would bump the union and break old decoders. + +--- + +## Message ordering & idempotency + +HTTP/2 multiplexes streams over a single connection and a server +handler may complete in any order. The Engine API is sensitive to +ordering, so we pin two rules explicitly: + +- **CL-driven ordering.** The CL is responsible for serialising + dependent requests. In particular: + - Only one `POST /forkchoice` may be in flight at a time. + - If a `POST /payloads` is logically before a `POST /forkchoice` + (or vice versa), the CL MUST wait for the first response before + issuing the second. + - The EL processes streams in receive order. h2 multiplexing + across independent CL→EL flows is fine; the CL MUST NOT rely on + the EL to reorder its own dependent requests. + + This matches today's [`common.md`](./common.md) "Message ordering" + guarantee in spirit; it makes explicit that h2 multiplexing does + not relax it. There is **no sequence number on the wire** — the + protocol stays simple and CL bugs that break ordering are CL bugs. + +- **Idempotency, narrowly defined.** Today's + [`paris.md`](./paris.md) #4 specifies idempotency only with respect + to `VALID | INVALID`: once a payload is decided one way, it cannot + flip. But `SYNCING → VALID`, `SYNCING → INVALID`, and + `ACCEPTED → VALID/INVALID` transitions are explicitly allowed — + the same payload submitted twice can return different statuses if + the EL has acquired more state in between. The new spec preserves + this: an EL MUST NOT short-circuit a retry by returning the cached + status, and a CL MUST NOT assume two responses to the same envelope + match. The only invariant is the `VALID ↔ INVALID` boundary. + +--- + +## Motivation + +The remainder of this document is rationale and reference material: +why we made the choices the spec encodes above, plus a consolidated +decision log for quick scanning. + +### Goals & non-goals + +#### Goals + +1. **Reduce wire size and parse cost.** SSZ-encoded bodies are 30–50% + smaller than hex-JSON for payload-shaped data and parse in linear time + without nibble decoding. This matters most for blob bundles (multi-MB + per slot) and the new `blockAccessList`. +2. **Stop the version sprawl.** Today every fork bumps every method that + touches a changed structure (`engine_newPayload` is at V5, + `engine_getPayload` at V6, etc.). The new API puts the fork in the + URL (`/engine/v2/{fork}/...`) so a single endpoint accepts whatever + schema that fork mandates; adding a fork = adding one path prefix + plus one set of SSZ schemas, not bumping every method name. +3. **Self-contained requests.** No more side-channel parameters + (`expectedBlobVersionedHashes`, `parentBeaconBlockRoot`, + `executionRequests`) that travel beside the payload — they live + inside the payload envelope or are unnecessary. +4. **Idiomatic HTTP.** Use HTTP status codes for transport-level outcomes, + `Content-Type` for negotiation, and a small problem-detail JSON body + for errors. + +#### Non-goals + +- Dropping the EL/CL split, changing trust boundaries, or moving CL state + into the EL (or vice versa). +- Removing JWT. Authentication is unchanged; only the *transport* of the + bearer token differs (HTTP `Authorization` header, same as today). +- Replacing `eth_*` JSON-RPC. The `eth` namespace stays JSON-RPC. This + document only refactors the `engine_*` namespace. +- Wire-perfect SSZ container definitions. The encoding *conventions* + are pinned in this document; the concrete field-by-field SSZ + containers per fork (e.g. the Amsterdam `ExecutionPayload` schema) + are deferred to a follow-up. + +### Why move away from JSON-RPC? + +JSON-RPC over HTTP has served the Engine API since Paris. The pain points +that prompt this refactor: + +- **Encoding overhead.** Every `DATA` field is a `0x`-prefixed lowercase + hex string. A 128 KiB blob becomes a 256 KiB+ string. With Osaka / + Fulu blob counts and the Amsterdam `blockAccessList`, payloads are + routinely multi-megabyte. +- **No content negotiation.** A new fork structure forces a new method + name (`engine_newPayloadV5`), even when the only change is one added + field. With a REST endpoint, the fork is part of the URL + (`/engine/v2/amsterdam/payloads`) and the body schema is selected by + routing, not by method-name suffix. +- **Side-channel params.** JSON-RPC's positional params encourage + bolting on extras like `parentBeaconBlockRoot` and + `executionRequests` next to the payload, instead of inside it. +- **Errors are non-standard.** `-38001..-38006` are bespoke and require + client-side mapping. HTTP status codes + a typed problem body are + universally understood. + +JSON-RPC is fine for the casual `eth_*` query API. For the hot path +between CL and EL, we want something denser and more disciplined. + +### Why SSZ? + +- The CL already speaks SSZ natively for its block, attestation, blobs, + KZG, and request structures. The CL today **converts SSZ → JSON → + hex-strings** when it forwards a payload, then the EL parses hex-JSON + back to bytes. This conversion is pure overhead and has been a + recurring source of subtle field-encoding bugs (e.g. the + `withdrawals.amount` LE-vs-BE note in shanghai.md). +- SSZ's fixed/variable-length distinction lets us validate sizes + cheaply at the transport layer. +- It's already what consensus-specs uses to define `ExecutionPayload`, + `Withdrawal`, `BlobsBundle`, etc. We'd be aligning, not inventing. + +We keep JSON available for **error bodies, capability discovery, and +client identification** because those are ergonomic to debug with `curl` +and not on the hot path. + +### Simplifications & removed concepts + +1. **`expectedBlobVersionedHashes`** — **removed**. The block-hash check + already covers the transactions, so the EL recomputes the array + from `payload.transactions` during validation and surfaces a + mismatch as `INVALID`. The CL no longer sends a redundant copy. +2. **`INVALID_BLOCK_HASH`** — **removed** from the enum. Already + supplanted by `INVALID` since Shanghai. +3. **`ACCEPTED`** — **kept**. CLs use this status during sync to + acknowledge well-formed side-branch payloads that don't extend the + canonical chain. +4. **`shouldOverrideBuilder`** — kept, lives inside the SSZ + `BuiltPayload` body. (Considered moving to a response header but it + complicates the SSZ canonicalisation; better inside the body.) +5. **`engine_exchangeCapabilities`** as a polling handshake — replaced + by a single `GET /capabilities`. +6. **`engine_exchangeTransitionConfigurationV1`** — dropped. Already + deprecated since Cancun. +7. **`payloadId` derivation** — today both sides recompute an 8-byte + hash over `(headBlockHash, payloadAttributes)`. The new + `POST /forkchoice` returns `payload_id` directly in the response; + it is an **opaque server-assigned token**. The EL chooses how to + mint it; CLs MUST treat it as opaque bytes. +8. **The split between `engine_*` namespace and the `eth_*` subset + the EL must expose** — out of scope for this refactor; the `eth_*` + namespace stays JSON-RPC. +9. **Per-method `timeout` SHOULDs** — replaced with HTTP-standard + request timeouts and `Retry-After` semantics on 503. + +### Summary of design decisions + +This is the consolidated decision log. Every item below is normative +and is also detailed in the relevant section earlier in the document; +the summary exists for quick scanning. + +#### Scope + +- **Target fork:** Amsterdam. The new API ships *as* the Amsterdam + Engine API. Pre-Amsterdam timestamps continue to be served by the + legacy JSON-RPC API on the same port; clients run both surfaces. +- **Backwards compatibility** is out of scope. The legacy JSON-RPC + engine API is left in place by clients; this spec does not require + or forbid sunset. +- **`eth_*` JSON-RPC subset** (`eth_blockNumber`, `eth_call`, + `eth_chainId`, `eth_getCode`, `eth_getBlockByHash`, + `eth_getBlockByNumber`, `eth_getLogs`, `eth_sendRawTransaction`, + `eth_syncing`) is **not** mirrored under `/engine/v2/...`. CLs that + need state / log access continue to call them via the legacy + JSON-RPC root. + +#### Transport + +- **HTTP/2 required**, h2c (cleartext) for both TCP and IPC. No + HTTP/1.1 fallback. JWT-on-every-request authenticates; TLS + termination is left to a reverse proxy. +- **IPC** is h2c over UNIX socket — same paths and headers as TCP, + single code path. +- **Default port `8551`**, shared with the legacy JSON-RPC API + (distinguished by path). +- **Trailing slashes are forbidden** — return `404 method-not-found`. +- **Flow-control:** SHOULD set `INITIAL_WINDOW_SIZE` ≥ 1 MiB. + `MAX_FRAME_SIZE` and `MAX_HEADER_LIST_SIZE` use HTTP/2 defaults. +- **Connection lifecycle:** CLs MAY open fresh h2 connections per + request or reuse a long-lived connection. +- **Compression:** `zstd` and `gzip` MAY be implemented. CLs MUST + tolerate uncompressed responses. + +#### Versioning + +- **Fork-scoped endpoints:** `/{fork}/payloads`, `/{fork}/forkchoice`, + `/{fork}/bodies`. Fork in the URL, no `Eth-Consensus-Version` + header. +- **Independently versioned endpoints:** `/blobs/vN`. Legacy + `engine_getBlobsVN` numbers carry forward onto the URL. ELs MUST + serve at least the revision matching their current fork + (`/blobs/v4` for Amsterdam) and MAY serve older revisions + alongside. Future blob-shape changes ship as `/blobs/v5`, `/v6`, + etc. +- **Unscoped endpoints:** `/capabilities`, `/identity`. +- **Major version `/v2`** is bumped only for breaking transport + changes (e.g. dropping REST or SSZ). + +#### Encoding + +- **Hot-path bodies use SSZ.** Diagnostic / metadata endpoints + (`/capabilities`, `/identity`, error bodies) use JSON. +- **`Optional[T]` ≡ `List[T, 1]`** (length 0 = absent, length 1 = + present). Universally supported by SSZ libraries. +- **Strings ≡ `List[byte, MAX_ERROR_BYTES]`**, `MAX_ERROR_BYTES = 1024`. +- **Endianness:** SSZ uints are little-endian. This flips byte order + vs the JSON-RPC `QUANTITY` type for `block_value`, `gas_used`, + `timestamp`, `base_fee_per_gas`, `excess_blob_gas`, + `blob_gas_used`, `block_number`, and the + `index`/`validatorIndex`/`amount` triple in `Withdrawal`. +- **`MAX_*` constants** are defined in fork-scoped SSZ schema files; + `MAX_ERROR_BYTES` is global. +- **Cross-fork response containers** come in two flavours: + fork-scoped (`/bodies`) uses the URL `{fork}` to pick a schema, + with `Optional[T]` for fields absent in pre-fork blocks; + independently versioned (`/blobs/vN`) gives each revision its own + dedicated container. Both wrap their entries in + `BodyEntry { available, body }` / `BlobEntry { available, contents }` + with an outer `Optional[List[...]]` for the syncing / + all-or-nothing channel. Per-entry fork tags were rejected. + +#### Error model + +- **RFC 7807 with two fields:** `type` (required, relative URI rooted + at `/engine-api/errors/...`) and `detail` (optional). Drop `title`, + `status`, `instance`, `engine_code`. +- **CLs MUST NOT dereference `type`** — opaque strings. +- **SSZ-decode failures** are a canned `400 Bad Request` with + `type=/engine-api/errors/ssz-decode-error`, no `detail`. + +#### Ordering & idempotency + +- **CL-driven ordering.** Only one `POST /forkchoice` in flight at a + time; `POST /payloads` ordered with respect to surrounding FCUs by + the CL. No sequence number on the wire. +- **Idempotency is narrow.** `VALID ↔ INVALID` cannot flip. All + other transitions (`SYNCING → VALID/INVALID`, + `ACCEPTED → VALID/INVALID`) are allowed; ELs MUST NOT short-circuit + retries. + +#### Forkchoice update (`POST /{fork}/forkchoice`) + +- **Single atomic call** carrying forkchoice state, optional + `payload_attributes`, and optional `custody_columns`. +- **Skip-allowed semantics:** EL MAY skip applying state when the + new `head` is a `VALID` ancestor of the latest finalized block, + guarding against malformed CL FCUs. +- **Stale-fork URL** is allowed when `payload_attributes` is absent; + with `payload_attributes` present, URL `{fork}` MUST match the + timestamp's fork (otherwise `400 unsupported-fork`). +- **No HTTP-layer body cap** beyond SSZ `MAX_*` constants. +- **Custody-set updates** run independently of the forkchoice flow; + custody errors do not affect `payload_status`. +- **Custody-set lifetime:** set until the next FCU that includes a + `custody_columns` field. FCUs that omit it leave the set unchanged. + +#### Payload submission (`POST /{fork}/payloads`) + +- **`expectedBlobVersionedHashes` removed.** EL recomputes from + `payload.transactions`; block-hash check covers transactions. +- **`INVALID_BLOCK_HASH` removed** from the status enum. +- **`ACCEPTED` kept** — CLs use it during sync. +- **Transaction min-length** ("at least 1 byte") remains a + receiver-side validation rule, not an SSZ schema invariant. + +#### Payload retrieval (`GET /{fork}/payloads/{payloadId}`) + +- **Poll-only**, same semantics as today's `engine_getPayload`. No + SSE / long-poll. +- **`payload_id` is an opaque server-assigned token** issued by + `POST /forkchoice`. CLs MUST NOT recompute or validate it. +- **`payload_id` TTL ≥ 10 minutes.** After expiry the EL MAY GC and + reuse the token namespace; within the TTL no collisions. +- **`shouldOverrideBuilder`** lives inside the SSZ `BuiltPayload` + body. + +#### Authentication & telemetry + +- **JWT (HS256, 256-bit secret)** unchanged in spirit, presented as + `Authorization: Bearer ` on every request. +- **JWT claims:** `iat` required (±60s), `id` optional, **`clv` + removed**. +- **`X-Engine-Client-Version`** is the canonical CL version channel. +- **`traceparent`** (W3C trace context) is supported but optional. + +#### Operations + +- **Multi-CL setups** are operator-managed. The spec does not track + CL identity or restrict who calls `POST /forkchoice`. Today's "one + writer, many readers" pattern carries forward unchanged. +- **`GET /capabilities`** advertises supported forks, fork-scoped + endpoints, independently-versioned endpoints (with the available + `/vN` list), unscoped endpoints, and per-endpoint maximum request + sizes. + +#### Removed concepts + +- `engine_exchangeCapabilities` — replaced by `GET /capabilities`. +- `engine_exchangeTransitionConfigurationV1` — already deprecated + since Cancun. +- Per-method `timeout` SHOULDs — replaced by HTTP-standard request + timeouts and `Retry-After` semantics on 503. +- The mutual-exchange handshake of `engine_getClientVersionV1` — + replaced by one-way `GET /identity` plus the + `X-Engine-Client-Version` request header. From e1861e48788a0f249cbcf23e954f4ddc47fa4991 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Fri, 8 May 2026 11:31:49 +0200 Subject: [PATCH 2/9] engine: cleanup --- src/engine/refactor-ssz.md | 35 +++++-- src/engine/refactor.md | 183 +++++++++++++------------------------ 2 files changed, 90 insertions(+), 128 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index 101ef5042..f9c2d3dc4 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -302,27 +302,46 @@ BodyEntry { available: Boolean body: ExecutionPayloadBody } +``` + +`available` is `false` when the requested block is unavailable / +pruned, **or** when the block's timestamp falls outside the URL +fork's active range, **or** for range queries when the block number +is past the latest known block. When `available=false`, `body` is +zero-valued and CLs MUST ignore its contents. + +Each fork URL pairs with its own `ExecutionPayloadBody` schema. The +Amsterdam variant carries every field unconditionally: -# /amsterdam/bodies/... uses this Amsterdam-fork ExecutionPayloadBody +``` +# Amsterdam ExecutionPayloadBody (used by /amsterdam/bodies/...) ExecutionPayloadBody { transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] - withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]] # [] pre-Shanghai - block_access_list: Optional[ByteList[MAX_BAL_BYTES]] # [] pre-Amsterdam or pruned + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] + block_access_list: ByteList[MAX_BAL_BYTES] } ``` -A CL on the Cancun schema would call `/cancun/bodies/...` and receive -a Cancun-shaped `ExecutionPayloadBody` (no `block_access_list` field -at all). The Cancun-fork variant is sketched here for clarity: +Earlier-fork variants drop the fields their fork didn't have. For +reference: ``` -# /cancun/bodies/... ExecutionPayloadBody (for reference) +# Cancun ExecutionPayloadBody (used by /cancun/bodies/...) +ExecutionPayloadBody { + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] +} + +# Paris ExecutionPayloadBody (used by /paris/bodies/...) ExecutionPayloadBody { transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] - withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]] # [] pre-Shanghai } ``` +No `Optional[T]` cross-fork nullability anywhere — each fork URL +returns only blocks from its own era, so every field is always +present. + ### `POST /blobs/v1` Replaces `engine_getBlobsV1` (Cancun whole-blob). diff --git a/src/engine/refactor.md b/src/engine/refactor.md index f0aa6ac14..52d9d7d99 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -9,13 +9,6 @@ > **Target fork:** Amsterdam. The new API ships *as* the Amsterdam Engine > API; clients implement it instead of `engine_*` JSON-RPC at the > Amsterdam activation timestamp. - -This document is meant to be read alongside the existing fork-scoped specs -([Paris](./paris.md), [Shanghai](./shanghai.md), [Cancun](./cancun.md), -[Prague](./prague.md), [Osaka](./osaka.md), [Amsterdam](./amsterdam.md)). -Concrete byte-level structures are deferred to a later iteration; the goal -here is to align on the *shape* of the new API. - --- ## Table of contents @@ -85,12 +78,11 @@ endpoints are unscoped. | Forkchoice | `POST /engine/v2/{fork}/forkchoice` | Atomic forkchoice update: update head/safe/finalized, optionally start a payload build, optionally update custody set. Replaces `engine_forkchoiceUpdated`. | | Bodies | `POST /engine/v2/{fork}/bodies/hash` | Replaces `engine_getPayloadBodiesByHash`. Fork-scoped: `{fork}` selects the *response schema*, not the fork of the requested blocks. | | Bodies | `GET /engine/v2/{fork}/bodies?from=N&count=M` | Replaces `engine_getPayloadBodiesByRange`. Fork-scoped on response shape. | -| Blob pool | `POST /engine/v2/blobs/v{1..4}` | Replaces `engine_getBlobsV{1..4}`. The `vN` segment carries forward the legacy version numbers; `/v4` is the Amsterdam cell-range variant, `/v1` is the original Cancun whole-blob variant, and intermediate revisions live alongside. ELs MUST serve at least the current-fork revision (`/v4` for Amsterdam) and MAY serve older revisions alongside. | +| Blob pool | `POST /engine/v2/blobs/v{1..4}` | Replaces `engine_getBlobsV{1..4}`. The `vN` segment carries forward the legacy version numbers; `/v4` is the Amsterdam cell-range variant. | | Capabilities | `GET /engine/v2/capabilities` | Replaces `engine_exchangeCapabilities`. Unscoped; advertises supported forks, `/blobs/vN` revisions, and per-endpoint request-size limits. | | Identity | `GET /engine/v2/identity` | Replaces `engine_getClientVersion`. Unscoped. | Every hot-path body uses SSZ; every metadata endpoint uses JSON. - --- ## Endpoints @@ -101,9 +93,7 @@ Every hot-path body uses SSZ; every metadata endpoint uses JSON. Replaces `engine_newPayloadV{1..5}`. -- **Request body:** SSZ-encoded `ExecutionPayloadEnvelope`, a container - that bundles together everything that today travels alongside the - payload as separate JSON-RPC params: +- **Request body:** SSZ-encoded `ExecutionPayloadEnvelope` ``` ExecutionPayloadEnvelope { @@ -130,9 +120,6 @@ Replaces `engine_newPayloadV{1..5}`. ``` `INVALID_BLOCK_HASH` is dropped (already supplanted by `INVALID`). - `ACCEPTED` is **kept** — CLs rely on it during sync to acknowledge - side-branch payloads that are well-formed but don't extend the - canonical chain. - **HTTP status:** `200 OK` for any of the four validation outcomes. Validation results are not transport errors. @@ -141,19 +128,14 @@ Replaces `engine_newPayloadV{1..5}`. #### `POST /engine/v2/{fork}/forkchoice` -Replaces `engine_forkchoiceUpdatedV{1..4}`. This is the **single -atomic** call that updates the EL's forkchoice state, optionally -triggers a payload build, and (post-Amsterdam) optionally updates the -CL's custody set. Atomicity matters: the CL relies on the EL having -applied the new head before — and only if — the build is started, and -on the build being keyed against the freshly-applied head. +Replaces `engine_forkchoiceUpdatedV{1..4}`. - **Request body:** SSZ-encoded `ForkchoiceUpdate`: ``` ForkchoiceUpdate { forkchoice_state: ForkchoiceState # head / safe / finalized - payload_attributes: Optional[PayloadAttributes] # if present, start a build on top of head + payload_attributes: Optional[PayloadAttributes] # if present, start a build custody_columns: Optional[Bitvector[CELLS_PER_EXT_BLOB]] # Amsterdam+, optional } ``` @@ -177,9 +159,7 @@ on the build being keyed against the freshly-applied head. The `payload_id` is an **opaque server-assigned token**. The EL chooses how to mint it (counter, random, hash-tree-root over the attributes — anything). CLs MUST treat it as opaque bytes and MUST - NOT recompute or validate its contents. This is a change from - today's behavior where both sides derived an 8-byte hash over - `(headBlockHash, payloadAttributes)`. + NOT recompute or validate its contents. - **HTTP status:** `200 OK` for all three payload-status outcomes. `409 Conflict` is returned for an inconsistent forkchoice state @@ -202,6 +182,7 @@ on the build being keyed against the freshly-applied head. safe / finalized across fork boundaries during sync and reorg recovery, and the URL fork has no bearing on which historical block can be referenced. + TODO(MariusVanDerWijden) Is that really the case? If `payload_attributes` is present, the URL `{fork}` MUST match the fork that the new payload would belong to (i.e. the fork @@ -211,20 +192,14 @@ on the build being keyed against the freshly-applied head. we strictly police. - **Custody-set semantics** (Amsterdam+): the custody update runs - independently of the forkchoice processing flow, matching the - Amsterdam spec's "MUST run custody set update independently to the - fork choice update". An execution-time custody-set error MUST NOT - affect the `payload_status` returned for the forkchoice update. + independently of the forkchoice processing flow. An execution-time + custody-set error MUST NOT affect the `payload_status` returned for + the forkchoice update. A `custody_columns` value, once accepted, remains in effect until the next `POST /forkchoice` whose body *also* contains a `custody_columns` field. FCUs that omit the field leave the custody set unchanged. -- **No body cap.** `POST /forkchoice` bodies are bounded by the SSZ - schema's `MAX_*` constants (small for `ForkchoiceState` and - `PayloadAttributes`, fixed for `custody_columns`). No additional - HTTP-layer cap is imposed. - ### Payload retrieval #### `GET /engine/v2/{fork}/payloads/{payloadId}` @@ -251,36 +226,28 @@ available at the time of receipt; the EL MAY stop the build process after serving a call. `payloadId` values are opaque server-assigned tokens issued by `POST /forkchoice`. -**Token TTL.** A `payloadId` is valid for **at least 10 minutes** -after its issuing `POST /forkchoice` returns. After 10 minutes the -EL MAY garbage-collect the token and respond `404 unknown-payload` -to subsequent `GET`s. ELs MUST NOT recycle a token within its TTL -(no collisions); after expiry the token namespace is free to reuse. -A CL that needs a fresh `payloadId` after expiry simply issues a new -`POST /forkchoice` with the same attributes. +**Token TTL.** A `payloadId` is valid until either the payload was +retrieved by `GET /{fork}/payloads/{payloadId}` or another payload +was built via a forkchoice with payload attributes. ### Historical bodies -These endpoints are **fork-scoped on the response schema, not on the -era of the requested blocks**. The `{fork}` segment tells the EL which -`ExecutionPayloadBody` shape to use when serialising the response. -A CL that has just upgraded to the Amsterdam schema can ask for -`/amsterdam/bodies/hash` and receive `block_access_list` populated -for Amsterdam blocks and `[]` (the SSZ optional sentinel — see -[SSZ encoding conventions](#ssz-encoding-conventions)) for older -blocks; a CL still on Cancun asks `/cancun/bodies/hash` and -gets responses serialised against the Cancun container, never seeing -the trailing `block_access_list` field at all. - -This is different from the `/payloads` and `/forkchoice` `{fork}` -segments, where the URL fork *must* match the timestamp of the -referenced block. For `/bodies` the URL fork is purely a schema -selector and the requester chooses freely. - -The blob endpoint takes yet another approach: it carries a `/vN` -revision instead of a `{fork}` segment, because blob protocol -evolution has historically not aligned with fork activations. See -the [Blob pool](#blob-pool) section. +These endpoints are **fork-scoped on both the response schema and the +era of the returned blocks.** The `{fork}` segment tells the EL which +`ExecutionPayloadBody` schema to use, *and* limits the response to +blocks whose timestamp falls in `{fork}`'s active time range. A CL +fetching bodies that span a fork boundary issues separate requests +against each fork URL. + +Concretely: + +- `/cancun/bodies/hash` returns bodies *only* for blocks in the + Cancun time range. Requesting a Shanghai or Amsterdam hash yields + `available=false` for that entry. +- `/amsterdam/bodies/hash` returns bodies *only* for Amsterdam blocks. + All fields (including `block_access_list`) are unconditionally + present; older blocks the CL accidentally requested come back as + `available=false`. #### `POST /engine/v2/{fork}/bodies/hash` @@ -292,43 +259,38 @@ large hash lists travel in the request body rather than the URL. #### `GET /engine/v2/{fork}/bodies?from=N&count=M` Replaces `engine_getPayloadBodiesByRangeV{1,2}`. Range fits comfortably -in the URL. +in the URL. Block numbers outside the URL fork's active range come +back as `available=false`; if the requested range straddles a fork +boundary the CL re-issues against the next fork URL for the unfilled +suffix. - **Response body** (both endpoints): SSZ-encoded `List[BodyEntry, MAX_BODIES_REQUEST]`. Each `BodyEntry` carries an - `available: boolean` flag (false for unavailable / pruned blocks, - matching today's `null` semantics) and an `ExecutionPayloadBody` - serialised against the **`{fork}` schema from the URL**. Fields - introduced in `{fork}` or earlier are present (with `Optional[T]` - set to `None` for blocks predating the field's introduction); fields - introduced in forks newer than `{fork}` are absent from the - container entirely. See - [SSZ encoding conventions](#ssz-encoding-conventions). + `available: boolean` flag and an `ExecutionPayloadBody` serialised + against the **`{fork}` schema from the URL**. `available` is false + in any of the following cases: + - the block is unavailable / pruned, + - the block's timestamp falls outside the URL fork's active range, + - or for range queries, the block number is past the latest known + block. + + When `available=false`, the `body` field is zero-valued and CLs + MUST ignore its contents. See + [SSZ encoding conventions](#ssz-encoding-conventions) for the + `BodyEntry` wrapper definition. ### Blob pool -The blob endpoint is **independently versioned**: blobs are looked up -by versioned hash (not by fork), so the `{fork}` URL segment doesn't -help. But the blob protocol *has* evolved on its own clock — four -distinct semantics across two forks (V1 single proof in Cancun, V2 -cell proofs in Osaka, V3 partial responses, V4 cell-range selection -in Amsterdam). The new spec carries those legacy version numbers +The blob endpoint is **independently versioned**. +The new spec carries those legacy version numbers forward onto the URL: `engine_getBlobsVN` becomes `POST /blobs/vN`. ELs **MUST** serve at least the revision matching their current fork (`/blobs/v4` for Amsterdam) and **MAY** serve any subset of older revisions alongside; `GET /capabilities` advertises the actual list. -This is a different versioning axis from the fork-scoped endpoints -(`/{fork}/payloads`, `/{fork}/forkchoice`, `/{fork}/bodies`). Those -track *consensus protocol* changes coupled to fork activations. -`/blobs/vN` tracks *engine-API blob protocol* changes that have -historically not aligned with fork activations. - All revisions use `POST` so that 128 versioned hashes (8 KiB hex) don't have to fit in the URL. All revisions return SSZ -`Optional[List[BlobEntry, MAX_BLOBS_REQUEST]]`, where the outer -`Optional` is the "all-or-nothing"/syncing channel (`None` = -"cannot serve this request, retry later or fall back") and each +`Optional[List[BlobEntry, MAX_BLOBS_REQUEST]]`, where each `BlobEntry` carries an `available: boolean` per-entry flag for per-blob misses on revisions that support partial responses. Revision-specific contents live inside `BlobEntry.contents`. @@ -429,19 +391,6 @@ we use only **two** of the RFC 7807 fields: Omitted when the EL has nothing more to say than the `type` already conveys (e.g. canned SSZ-decode failures). -We deliberately drop the other RFC 7807 fields: - -- `title` would just duplicate `type` (RFC 7807 says it SHOULD NOT - vary between occurrences of the same `type`); CLs can render their - own from a static `type → title` map. -- `status` duplicates the HTTP status line. -- `instance` adds a per-request URI; operators get correlation from - logs already. - -There is **no** legacy `engine_code` extension. CLs migrating from -the JSON-RPC API map old codes to new `type` strings via the table -below; after migration the codes are gone. - | HTTP status | `type` | Old JSON-RPC code | When | | - | - | - | - | | 400 Bad Request | `/engine-api/errors/parse-error` | -32700 | Body is not valid JSON / SSZ | @@ -458,10 +407,7 @@ below; after migration the codes are gone. | 500 Internal Server Error | `/engine-api/errors/internal` | -32603 / -32000 | Unrecoverable server error; `detail` carries the message | `type` URIs are written as **relative references** rooted at -`/engine-api/errors/...`. RFC 7807 allows relative URIs, and the -short form keeps error bodies small without losing identifier -stability. CLs MUST treat them as opaque strings — they MUST NOT -attempt to dereference them. +`/engine-api/errors/...`. Example error body: @@ -620,10 +566,12 @@ Endpoints that return data spanning multiple block-eras come in two flavours: 1. **Fork-scoped** (e.g. `/bodies`): the URL `{fork}` selects the - container schema. Within that schema, fields that didn't exist in - earlier block-eras are `Optional[T]` (= `[]` for those blocks). - The outer entry carries an explicit `available` flag so - "pruned / unavailable" stays distinct from "field-not-applicable": + container schema *and* limits the response to blocks from that + fork's time range. Every field in the fork's body container is + unconditionally present (no `Optional[T]` for cross-fork + nullability); blocks outside the fork's range come back as + `available=false` on the outer entry instead of as a + zero-valued body: ``` # /amsterdam/bodies/hash response @@ -632,25 +580,24 @@ flavours: body: ExecutionPayloadBody } + # Amsterdam ExecutionPayloadBody — every field always present ExecutionPayloadBody { transactions: List[Transaction, MAX_TXS] - withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS]] # [] pre-Shanghai - block_access_list: Optional[ByteList[MAX_BAL_BYTES]] # [] pre-Amsterdam or if pruned + withdrawals: List[Withdrawal, MAX_WITHDRAWALS] + block_access_list: ByteList[MAX_BAL_BYTES] } ``` - A CL on the Cancun schema calls `/cancun/bodies/hash` and + A CL fetching a Cancun-era block calls `/cancun/bodies/hash` and receives the Cancun container (no `block_access_list` field at - all). Old CLs never see schemas they don't know. + all, and no `Optional` wrapper on `withdrawals`). Cross-fork + ranges require multiple requests, one per fork URL. 2. **Independently versioned** (e.g. `/blobs/vN`): each revision is its own container, no nullable optionals across revisions. Old CLs keep using `/blobs/v1`; new shapes ship as `/blobs/vN+1` alongside. -Per-entry fork tags (a `Union` of fork-shaped variants) were -rejected: every fork would bump the union and break old decoders. - --- ## Message ordering & idempotency @@ -669,11 +616,6 @@ ordering, so we pin two rules explicitly: across independent CL→EL flows is fine; the CL MUST NOT rely on the EL to reorder its own dependent requests. - This matches today's [`common.md`](./common.md) "Message ordering" - guarantee in spirit; it makes explicit that h2 multiplexing does - not relax it. There is **no sequence number on the wire** — the - protocol stays simple and CL bugs that break ordering are CL bugs. - - **Idempotency, narrowly defined.** Today's [`paris.md`](./paris.md) #4 specifies idempotency only with respect to `VALID | INVALID`: once a payload is decided one way, it cannot @@ -866,8 +808,9 @@ the summary exists for quick scanning. - **`MAX_*` constants** are defined in fork-scoped SSZ schema files; `MAX_ERROR_BYTES` is global. - **Cross-fork response containers** come in two flavours: - fork-scoped (`/bodies`) uses the URL `{fork}` to pick a schema, - with `Optional[T]` for fields absent in pre-fork blocks; + fork-scoped (`/bodies`) uses the URL `{fork}` to pick *both* the + schema and the era of returned blocks (every body field always + present; out-of-era blocks come back as `available=false`); independently versioned (`/blobs/vN`) gives each revision its own dedicated container. Both wrap their entries in `BodyEntry { available, body }` / `BlobEntry { available, contents }` From 9dbd160d1fbd6df01ddf0c29647c03a8df16cfa7 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Fri, 8 May 2026 11:39:19 +0200 Subject: [PATCH 3/9] engine: cleanup --- src/engine/refactor.md | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/src/engine/refactor.md b/src/engine/refactor.md index 52d9d7d99..03bc010a7 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -9,6 +9,7 @@ > **Target fork:** Amsterdam. The new API ships *as* the Amsterdam Engine > API; clients implement it instead of `engine_*` JSON-RPC at the > Amsterdam activation timestamp. + --- ## Table of contents @@ -54,8 +55,8 @@ table. Detail on each new endpoint follows in the sections below. | `engine_newPayloadV{1..5}` | `POST /{fork}/payloads` | `parentBeaconBlockRoot` and `executionRequests` folded into the SSZ envelope; `expectedBlobVersionedHashes` removed; `INVALID_BLOCK_HASH` removed from the status enum | | `engine_forkchoiceUpdatedV{1..4}` | `POST /{fork}/forkchoice` | one atomic call; carries forkchoice state, optional `payload_attributes`, and (Amsterdam+) optional `custody_columns` | | `engine_getPayloadV{1..6}` | `GET /{fork}/payloads/{id}` | poll-style, same semantics as today | -| `engine_getPayloadBodiesByHashV{1,2}` | `POST /{fork}/bodies/hash` | `{fork}` selects the response *schema* (not the era of requested blocks); `POST` because hash lists are too large for URLs | -| `engine_getPayloadBodiesByRangeV{1,2}` | `GET /{fork}/bodies?from=...&count=...` | `{fork}` selects the response schema | +| `engine_getPayloadBodiesByHashV{1,2}` | `POST /{fork}/bodies/hash` | `{fork}` selects both the response schema and the era of returned blocks; `POST` because hash lists are too large for URLs | +| `engine_getPayloadBodiesByRangeV{1,2}` | `GET /{fork}/bodies?from=...&count=...` | `{fork}` selects both the response schema and the era of returned blocks | | `engine_getBlobsV1` | `POST /blobs/v1` | independently versioned; legacy version numbers carry forward | | `engine_getBlobsV2` | `POST /blobs/v2` | all-or-nothing cell proofs | | `engine_getBlobsV3` | `POST /blobs/v3` | partial-response cell proofs | @@ -76,13 +77,14 @@ endpoints are unscoped. | Payload | `POST /engine/v2/{fork}/payloads` | Submit a payload received from the CL gossip network for the EL to validate / import. Replaces `engine_newPayload`. | | Payload | `GET /engine/v2/{fork}/payloads/{payloadId}` | Retrieve a built payload by id. Replaces `engine_getPayload`. CL polls when it wants a fresher snapshot. | | Forkchoice | `POST /engine/v2/{fork}/forkchoice` | Atomic forkchoice update: update head/safe/finalized, optionally start a payload build, optionally update custody set. Replaces `engine_forkchoiceUpdated`. | -| Bodies | `POST /engine/v2/{fork}/bodies/hash` | Replaces `engine_getPayloadBodiesByHash`. Fork-scoped: `{fork}` selects the *response schema*, not the fork of the requested blocks. | -| Bodies | `GET /engine/v2/{fork}/bodies?from=N&count=M` | Replaces `engine_getPayloadBodiesByRange`. Fork-scoped on response shape. | +| Bodies | `POST /engine/v2/{fork}/bodies/hash` | Replaces `engine_getPayloadBodiesByHash`. `{fork}` selects both the response schema *and* the era of returned blocks; out-of-era blocks come back as `available=false`. | +| Bodies | `GET /engine/v2/{fork}/bodies?from=N&count=M` | Replaces `engine_getPayloadBodiesByRange`. Same fork scoping as `/bodies/hash`. | | Blob pool | `POST /engine/v2/blobs/v{1..4}` | Replaces `engine_getBlobsV{1..4}`. The `vN` segment carries forward the legacy version numbers; `/v4` is the Amsterdam cell-range variant. | | Capabilities | `GET /engine/v2/capabilities` | Replaces `engine_exchangeCapabilities`. Unscoped; advertises supported forks, `/blobs/vN` revisions, and per-endpoint request-size limits. | | Identity | `GET /engine/v2/identity` | Replaces `engine_getClientVersion`. Unscoped. | Every hot-path body uses SSZ; every metadata endpoint uses JSON. + --- ## Endpoints @@ -281,12 +283,12 @@ suffix. ### Blob pool -The blob endpoint is **independently versioned**. -The new spec carries those legacy version numbers -forward onto the URL: `engine_getBlobsVN` becomes `POST /blobs/vN`. -ELs **MUST** serve at least the revision matching their current fork -(`/blobs/v4` for Amsterdam) and **MAY** serve any subset of older -revisions alongside; `GET /capabilities` advertises the actual list. +The blob endpoint is **independently versioned**: legacy +`engine_getBlobsVN` numbers carry forward onto the URL, so +`engine_getBlobsVN` becomes `POST /blobs/vN`. ELs **MUST** serve at +least the revision matching their current fork (`/blobs/v4` for +Amsterdam) and **MAY** serve any subset of older revisions alongside; +`GET /capabilities` advertises the actual list. All revisions use `POST` so that 128 versioned hashes (8 KiB hex) don't have to fit in the URL. All revisions return SSZ @@ -822,7 +824,6 @@ the summary exists for quick scanning. - **RFC 7807 with two fields:** `type` (required, relative URI rooted at `/engine-api/errors/...`) and `detail` (optional). Drop `title`, `status`, `instance`, `engine_code`. -- **CLs MUST NOT dereference `type`** — opaque strings. - **SSZ-decode failures** are a canned `400 Bad Request` with `type=/engine-api/errors/ssz-decode-error`, no `detail`. @@ -867,8 +868,10 @@ the summary exists for quick scanning. SSE / long-poll. - **`payload_id` is an opaque server-assigned token** issued by `POST /forkchoice`. CLs MUST NOT recompute or validate it. -- **`payload_id` TTL ≥ 10 minutes.** After expiry the EL MAY GC and - reuse the token namespace; within the TTL no collisions. +- **`payload_id` lifetime is build-bound, not time-bound.** A token + remains valid until either the payload was retrieved by + `GET /{fork}/payloads/{payloadId}` or another payload was built + via a forkchoice with payload attributes. - **`shouldOverrideBuilder`** lives inside the SSZ `BuiltPayload` body. From 094ec192d7a2571acdb36bb64936c06ef4d688cd Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Fri, 8 May 2026 11:54:33 +0200 Subject: [PATCH 4/9] engine: cleanup --- src/engine/refactor-ssz.md | 15 ++++++---- src/engine/refactor.md | 60 +++++++++++++++++++++++++++++++++++++- 2 files changed, 68 insertions(+), 7 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index f9c2d3dc4..24fdbc451 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -188,14 +188,17 @@ Status enum: | Value | Name | Used by | | - | - | - | -| `1` | `VALID` | both | -| `2` | `INVALID` | both | -| `3` | `SYNCING` | both | -| `4` | `ACCEPTED` | `POST /payloads` only | +| `0` | `VALID` | both | +| `1` | `INVALID` | both | +| `2` | `SYNCING` | both | +| `3` | `ACCEPTED` | `POST /payloads` only | + +Numbering starts at `0` so a default-initialised SSZ `PayloadStatus` +deserialises as `VALID` rather than as a reserved sentinel. `INVALID_BLOCK_HASH` is removed (already supplanted by `INVALID`). -`POST /forkchoice` MUST return `1`/`2`/`3` only; CLs MUST treat a -`4` from `/forkchoice` as a protocol error. +`POST /forkchoice` MUST return `0`/`1`/`2` only; CLs MUST treat a +`3` from `/forkchoice` as a protocol error. `Optional[String]` resolves to `List[List[byte, MAX_ERROR_BYTES], 1]`. diff --git a/src/engine/refactor.md b/src/engine/refactor.md index 03bc010a7..af62ab16b 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -29,6 +29,7 @@ - [Transport & framing](#transport--framing) - [SSZ encoding conventions](#ssz-encoding-conventions) - [Message ordering & idempotency](#message-ordering--idempotency) +- [Security considerations](#security-considerations) - [Motivation](#motivation) - [Goals & non-goals](#goals--non-goals) - [Why move away from JSON-RPC?](#why-move-away-from-json-rpc) @@ -115,7 +116,7 @@ Replaces `engine_newPayloadV{1..5}`. ``` PayloadStatus { - status: uint8 # VALID=1, INVALID=2, SYNCING=3, ACCEPTED=4 + status: uint8 # VALID=0, INVALID=1, SYNCING=2, ACCEPTED=3 latest_valid_hash: Optional[Hash32] validation_error: Optional[String] } @@ -228,6 +229,17 @@ available at the time of receipt; the EL MAY stop the build process after serving a call. `payloadId` values are opaque server-assigned tokens issued by `POST /forkchoice`. +The EL keeps optimising the payload until the slot deadline, so +successive `GET`s against the same `{payloadId}` may return different +bytes. The EL **MUST** include `Cache-Control: no-store` on the +response, and intermediaries **MUST NOT** cache or revalidate this +resource. CLs **MUST NOT** treat the response as cacheable. + +**Path validation.** `{payloadId}` is a path segment carrying a hex- +encoded `Bytes8`. The EL **MUST** validate that the path segment is +well-formed (8 bytes, hex) before dispatching to lookup logic; a +malformed segment returns `400 invalid-request`. + **Token TTL.** A `payloadId` is valid until either the payload was retrieved by `GET /{fork}/payloads/{payloadId}` or another payload was built via a forkchoice with payload attributes. @@ -562,6 +574,27 @@ discoverable in logs. `MAX_BAL_BYTES`, `MAX_VERSIONED_HASHES_PER_REQUEST`). `MAX_ERROR_BYTES` is global and pinned at `1024` here. +### JSON-RPC type → SSZ type mapping + +For implementers porting from the JSON-RPC API, the legacy openrpc +base types map onto SSZ as follows: + +| JSON-RPC type | SSZ type | +| - | - | +| `address` (20 bytes) | `Bytes20` | +| `hash32` (32 bytes) | `Bytes32` | +| `bytes8` (8 bytes) | `Bytes8` | +| `bytes32` (32 bytes) | `Bytes32` | +| `bytes48` (48 bytes) | `Bytes48` | +| `bytes256` (256 bytes) | `ByteVector[256]` | +| `bytesMax32` (0–32 bytes) | `ByteList[32]` | +| `bytes` (variable-length) | `ByteList[MAX_*]` (context-dependent) | +| `uint64` | `uint64` | +| `uint256` | `uint256` | +| `BOOLEAN` | `boolean` | +| `Array of T` | `List[T, MAX_*]` (context-dependent) | +| `T \| null` | `Optional[T]` (= `List[T, 1]`) | + ### Cross-fork response containers Endpoints that return data spanning multiple block-eras come in two @@ -631,6 +664,31 @@ ordering, so we pin two rules explicitly: --- +## Security considerations + +SSZ `MAX_*` constants bound *on-chain validity*, not per-request +resource use. A naive decoder facing a crafted `Content-Length`, +length prefix, or offset can be coerced into large allocations or +scans before any semantic rejection. ELs implementing this API +**MUST**: + +- **Cap by `Content-Length`** against an endpoint-specific maximum + *before* reading the body when the header is present, and cap the + bytes read from the body in all cases. +- **Validate SSZ length prefixes and offsets** against the remaining + buffer size *before* allocating backing storage for variable-length + fields. +- **Apply per-endpoint operational caps** (reverse proxy, + server config) in addition to library-level checks. The advertised + `limits.*` values in `GET /capabilities` are an upper bound, not a + target — operators are encouraged to reject earlier. + +ELs **SHOULD** use well-tested SSZ libraries and fuzz-test SSZ +parsing extensively. JWT authentication is unchanged from the legacy +JSON-RPC API; all existing requirements apply. + +--- + ## Motivation The remainder of this document is rationale and reference material: From b6a0dd2e7f8531dc2d67da2b3880b80312bf5834 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Fri, 8 May 2026 12:11:27 +0200 Subject: [PATCH 5/9] engine: cleanup --- src/engine/refactor-ssz.md | 340 ++++++++++++++++++++++++++++++------- src/engine/refactor.md | 178 +++++++++++++++++-- 2 files changed, 440 insertions(+), 78 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index 24fdbc451..76dcfa485 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -27,6 +27,13 @@ - [`PayloadAttributes` (Amsterdam)](#payloadattributes-amsterdam) - [`ForkchoiceState`](#forkchoicestate) - [`PayloadStatus`](#payloadstatus) +- [Per-fork container catalogue](#per-fork-container-catalogue) + - [`ExecutionPayload` per fork](#executionpayload-per-fork) + - [`PayloadAttributes` per fork](#payloadattributes-per-fork) + - [`ExecutionPayloadBody` per fork](#executionpayloadbody-per-fork) + - [`BlobsBundle` per revision](#blobsbundle-per-revision) + - [`BlobAndProof` per revision](#blobandproof-per-revision) + - [Identification & capabilities](#identification--capabilities) - [Endpoint containers](#endpoint-containers) - [`POST /amsterdam/payloads`](#post-amsterdampayloads) - [`POST /amsterdam/forkchoice`](#post-amsterdamforkchoice) @@ -58,26 +65,32 @@ ## `MAX_*` constants -These are sketch values — final values come from a follow-up that -matches the consensus-specs `Amsterdam` preset. They are listed here -for completeness so readers can size the on-wire bounds. - -| Constant | Sketch value | Where it's used | +| Constant | Value | Source | | - | - | - | -| `MAX_TXS_PER_PAYLOAD` | `1048576` | `ExecutionPayload.transactions` | -| `MAX_BYTES_PER_TX` | `1073741824` | element bound inside `transactions` | -| `MAX_WITHDRAWALS_PER_PAYLOAD` | `16` | `ExecutionPayload.withdrawals`, `PayloadAttributes.withdrawals` | -| `MAX_EXTRA_DATA_BYTES` | `32` | `ExecutionPayload.extra_data` | -| `MAX_BAL_BYTES` | TBD (EIP-7928) | `ExecutionPayload.block_access_list` | -| `MAX_EXECUTION_REQUESTS_PER_PAYLOAD` | TBD (EIP-7685) | `ExecutionPayloadEnvelope.execution_requests` | -| `MAX_BYTES_PER_EXECUTION_REQUEST` | TBD | element bound inside `execution_requests` | -| `MAX_VERSIONED_HASHES_PER_REQUEST` | `128` | `BlobsRequest.versioned_hashes` | -| `MAX_BODIES_REQUEST` | `128` | bodies request and response lists | -| `MAX_BLOBS_REQUEST` | `128` | blobs request and response lists | -| `MAX_BLOBS_PER_PAYLOAD` | `MAX_VERSIONED_HASHES_PER_REQUEST` | `BlobsBundle.commitments`, `.blobs` | -| `CELLS_PER_EXT_BLOB` | `128` (EIP-7594) | cell-proof and custody bitvectors | -| `BYTES_PER_BLOB` | `131072` | one blob (`4096 * 32`) | -| `MAX_ERROR_BYTES` | `1024` | `validation_error`, JSON error `detail` | +| `MAX_BYTES_PER_TX` | `2**30` (1,073,741,824) | [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844) | +| `MAX_TXS_PER_PAYLOAD` | `2**20` (1,048,576) | [Bellatrix](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md) | +| `MAX_WITHDRAWALS_PER_PAYLOAD` | `2**4` (16) | [Capella](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md) | +| `BYTES_PER_LOGS_BLOOM` | `256` | [Bellatrix](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md) | +| `MAX_EXTRA_DATA_BYTES` | `2**5` (32) | [Bellatrix](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md) | +| `MAX_BLOB_COMMITMENTS_PER_BLOCK` | `2**12` (4,096) | [Deneb](https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/beacon-chain.md) | +| `FIELD_ELEMENTS_PER_BLOB` | `4096` | [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844) | +| `BYTES_PER_FIELD_ELEMENT` | `32` | [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844) | +| `BYTES_PER_BLOB` | `FIELD_ELEMENTS_PER_BLOB * BYTES_PER_FIELD_ELEMENT` (131,072) | derived | +| `CELLS_PER_EXT_BLOB` | `128` | [EIP-7594](https://eips.ethereum.org/EIPS/eip-7594) | +| `BYTES_PER_CELL` | `BYTES_PER_BLOB / CELLS_PER_EXT_BLOB` (1,024) | derived | +| `MAX_BAL_BYTES` | `MAX_BYTES_PER_TX` | [EIP-7928](https://eips.ethereum.org/EIPS/eip-7928) (placeholder until EIP pins a tighter bound) | +| `MAX_EXECUTION_REQUESTS_PER_PAYLOAD` | `2**8` (256) | [EIP-7685](https://eips.ethereum.org/EIPS/eip-7685) | +| `MAX_BYTES_PER_EXECUTION_REQUEST` | `MAX_BYTES_PER_TX` | this spec (placeholder; reuse the tx bound) | +| `MAX_VERSIONED_HASHES_PER_REQUEST` | `128` | [Osaka](./osaka.md#engine_getblobsv2) | +| `MAX_BLOBS_REQUEST` | `MAX_VERSIONED_HASHES_PER_REQUEST` (128) | derived | +| `MAX_BODIES_REQUEST` | `2**5` (32) | [Shanghai](./shanghai.md#engine_getpayloadbodiesbyhashv1) | +| `MAX_ERROR_BYTES` | `1024` | this spec | +| `MAX_CLIENT_CODE_LENGTH` | `2` | this spec | +| `MAX_CLIENT_NAME_LENGTH` | `64` | this spec | +| `MAX_CLIENT_VERSION_LENGTH` | `64` | this spec | +| `MAX_CLIENT_VERSIONS` | `4` | this spec | +| `MAX_CAPABILITY_NAME_LENGTH` | `64` | this spec | +| `MAX_CAPABILITIES` | `64` | this spec | --- @@ -204,6 +217,214 @@ deserialises as `VALID` rather than as a reserved sentinel. --- +## Per-fork container catalogue + +Each fork URL (`/{fork}/payloads`, `/{fork}/forkchoice`, +`/{fork}/bodies`) uses its own SSZ container shape. ELs serving +`/cancun/...` MUST use the Cancun containers; ELs serving +`/amsterdam/...` MUST use the Amsterdam containers; etc. This section +catalogues every fork-scoped variant. + +### `ExecutionPayload` per fork + +Used by `POST /{fork}/payloads` (the inner `payload` field of +`ExecutionPayloadEnvelope`) and `GET /{fork}/payloads/{payloadId}` +(the inner `payload` field of `BuiltPayload`). + +``` +# Paris +ExecutionPayloadParis { + parent_hash: Hash32 + fee_recipient: Address + state_root: Hash32 + receipts_root: Hash32 + logs_bloom: Bloom + prev_randao: Bytes32 + block_number: Uint64 + gas_limit: Uint64 + gas_used: Uint64 + timestamp: Uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: Uint256 + block_hash: Hash32 + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] +} + +# Shanghai = Paris + withdrawals +ExecutionPayloadShanghai { + ...Paris fields... + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] +} + +# Cancun = Shanghai + blob_gas_used + excess_blob_gas +ExecutionPayloadCancun { + ...Shanghai fields... + blob_gas_used: Uint64 + excess_blob_gas: Uint64 +} + +# Prague = Cancun (no payload-shape change; execution_requests is at the envelope level) +ExecutionPayloadPrague = ExecutionPayloadCancun + +# Osaka = Prague (no payload-shape change; blobs bundle moved to BlobsBundleV2) +ExecutionPayloadOsaka = ExecutionPayloadPrague + +# Amsterdam = Cancun + block_access_list + slot_number +ExecutionPayloadAmsterdam { + ...Cancun fields... + block_access_list: ByteList[MAX_BAL_BYTES] + slot_number: Uint64 +} +``` + +The Amsterdam variant is identical to the +[`ExecutionPayload` (Amsterdam)](#executionpayload-amsterdam) shape +above; this section just makes the progression explicit. + +### `PayloadAttributes` per fork + +Used by the `payload_attributes` field of `ForkchoiceUpdate` (the +request body of `POST /{fork}/forkchoice`). + +``` +# Paris +PayloadAttributesParis { + timestamp: Uint64 + prev_randao: Bytes32 + suggested_fee_recipient: Address +} + +# Shanghai = Paris + withdrawals +PayloadAttributesShanghai { + ...Paris fields... + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] +} + +# Cancun = Shanghai + parent_beacon_block_root +PayloadAttributesCancun { + ...Shanghai fields... + parent_beacon_block_root: Root +} + +# Prague = Cancun (no shape change) +PayloadAttributesPrague = PayloadAttributesCancun + +# Osaka = Cancun (no shape change) +PayloadAttributesOsaka = PayloadAttributesCancun + +# Amsterdam = Cancun + slot_number +PayloadAttributesAmsterdam { + ...Cancun fields... + slot_number: Uint64 +} +``` + +### `ExecutionPayloadBody` per fork + +Used by the inner `body` field of `BodyEntry`. Each fork URL serves +only blocks from its own time range, so every field is +unconditionally present (no `Optional[T]`). + +``` +# Paris +ExecutionPayloadBodyParis { + transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] +} + +# Shanghai = Paris + withdrawals +ExecutionPayloadBodyShanghai { + ...Paris fields... + withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] +} + +# Cancun, Prague, Osaka = Shanghai (no shape change for the body) +ExecutionPayloadBodyCancun = ExecutionPayloadBodyShanghai +ExecutionPayloadBodyPrague = ExecutionPayloadBodyShanghai +ExecutionPayloadBodyOsaka = ExecutionPayloadBodyShanghai + +# Amsterdam = Shanghai + block_access_list +ExecutionPayloadBodyAmsterdam { + ...Shanghai fields... + block_access_list: ByteList[MAX_BAL_BYTES] +} +``` + +### `BlobsBundle` per revision + +Used by the `blobs_bundle` field of `BuiltPayload`. The bundle shape +follows the consensus-specs progression (V1 single proof, V2 cell +proofs). + +``` +# Cancun (V1) — one proof per blob +BlobsBundleV1 { + commitments: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK] + proofs: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK] + blobs: List[ByteVector[BYTES_PER_BLOB], MAX_BLOB_COMMITMENTS_PER_BLOCK] +} + +# Osaka+ (V2) — CELLS_PER_EXT_BLOB cell proofs per blob +BlobsBundleV2 { + commitments: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK] + proofs: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK * CELLS_PER_EXT_BLOB] + blobs: List[ByteVector[BYTES_PER_BLOB], MAX_BLOB_COMMITMENTS_PER_BLOCK] +} +``` + +`BuiltPayload` for Cancun / Prague carries `BlobsBundleV1`; +Osaka / Amsterdam carries `BlobsBundleV2`. + +### `BlobAndProof` per revision + +Used by `BlobEntry.contents` on the blob-pool endpoints (`/blobs/vN`). + +``` +# /blobs/v1 — Cancun whole-blob, single proof +BlobAndProofV1 { + blob: ByteVector[BYTES_PER_BLOB] + proof: Bytes48 +} + +# /blobs/v2 and /blobs/v3 — Osaka cell proofs +BlobAndProofV2 { + blob: ByteVector[BYTES_PER_BLOB] + proofs: List[Bytes48, CELLS_PER_EXT_BLOB] +} + +# /blobs/v4 — Amsterdam cell-range selection (per-cell nullable) +BlobCellsAndProofs { + blob_cells: List[Optional[ByteVector[BYTES_PER_CELL]], CELLS_PER_EXT_BLOB] + proofs: List[Optional[Bytes48], CELLS_PER_EXT_BLOB] +} +``` + +### Identification & capabilities + +Used by `GET /identity` and `GET /capabilities`. These are JSON on +the wire (see [refactor.md § Resource model](./refactor.md#resource-model-overview)), +but we list the SSZ shapes for completeness so future versions could +switch to SSZ if desired. + +``` +ClientVersion { + code: ByteList[MAX_CLIENT_CODE_LENGTH] + name: ByteList[MAX_CLIENT_NAME_LENGTH] + version: ByteList[MAX_CLIENT_VERSION_LENGTH] + commit: Bytes4 +} + +IdentityResponse { + versions: List[ClientVersion, MAX_CLIENT_VERSIONS] +} + +CapabilitiesResponse { + capabilities: List[ByteList[MAX_CAPABILITY_NAME_LENGTH], MAX_CAPABILITIES] + # ... plus the structured fields documented in refactor.md +} +``` + +--- + ## Endpoint containers ### `POST /amsterdam/payloads` @@ -225,7 +446,7 @@ from `payload.transactions`). #### Response -`PayloadStatus` (full enum, `1`/`2`/`3`/`4`). +`PayloadStatus` (full enum, `0`/`1`/`2`/`3`). ### `POST /amsterdam/forkchoice` @@ -359,8 +580,13 @@ BlobsV1Request { #### Response +`200 OK` carries the SSZ body below; `204 No Content` (with empty +body) signals "EL cannot serve this request" (e.g. syncing). + ``` -BlobsV1Response = Optional[List[BlobV1Entry, MAX_BLOBS_REQUEST]] +BlobsV1Response { + entries: List[BlobV1Entry, MAX_BLOBS_REQUEST] +} BlobV1Entry { available: Boolean @@ -374,8 +600,8 @@ BlobAndProofV1 { ``` When `available == false`, `contents` carries zero-valued bytes (a -`BYTES_PER_BLOB`-byte zero blob and a 48-byte zero proof). The outer -`Optional` returns `[]` when the EL cannot serve the request at all. +`BYTES_PER_BLOB`-byte zero blob and a 48-byte zero proof) and CLs +MUST ignore them. ### `POST /blobs/v2` @@ -391,8 +617,14 @@ BlobsV2Request { #### Response +`200 OK` carries the SSZ body below; `204 No Content` (with empty +body) signals either "EL cannot serve this request at all" or +"at least one requested blob is missing" (V2 is all-or-nothing). + ``` -BlobsV2Response = Optional[List[BlobV2Entry, MAX_BLOBS_REQUEST]] +BlobsV2Response { + entries: List[BlobV2Entry, MAX_BLOBS_REQUEST] +} BlobV2Entry { available: Boolean # always true for /v2 (all-or-nothing); included for shape symmetry @@ -405,9 +637,7 @@ BlobAndProofV2 { } ``` -All-or-nothing: if any requested blob is missing, the outer -`Optional` returns `[]` and no per-entry data is sent. CLs that need -partial responses use `/v3`. +CLs that need partial responses use `/v3`. ### `POST /blobs/v3` @@ -418,13 +648,14 @@ proofs). #### Response -Same shape as `/v2` (`BlobV2Entry` reused), but missing blobs -surface as `available=false` per entry rather than collapsing the -whole response to `[]`. Outer `Optional` returns `[]` only when the -EL cannot serve the request at all (e.g. syncing). +`200 OK` carries the SSZ body; missing blobs surface as +`available=false` per entry. `204 No Content` only when the EL +cannot serve the request at all (e.g. syncing). ``` -BlobsV3Response = Optional[List[BlobV2Entry, MAX_BLOBS_REQUEST]] +BlobsV3Response { + entries: List[BlobV2Entry, MAX_BLOBS_REQUEST] +} ``` ### `POST /blobs/v4` @@ -442,8 +673,13 @@ BlobsV4Request { #### Response +`200 OK` carries the SSZ body; `204 No Content` signals "EL cannot +serve this request at all." + ``` -BlobsV4Response = Optional[List[BlobV4Entry, MAX_BLOBS_REQUEST]] +BlobsV4Response { + entries: List[BlobV4Entry, MAX_BLOBS_REQUEST] +} BlobV4Entry { available: Boolean @@ -469,38 +705,22 @@ old spec). ## Open sketch questions -These are the items left to decide before promoting this sketch to -the canonical Amsterdam SSZ schema: - -1. **`MAX_*` placeholder values.** Several constants above are - `TBD` or sketch-only. They need to be pinned to the - consensus-specs `Amsterdam` preset values once those land. -2. **`MAX_BAL_BYTES`.** EIP-7928 defines the BAL but doesn't yet - pin a numeric upper bound that's friendly for SSZ. We need a - concrete number; otherwise the SSZ schema can't round-trip. +1. **`MAX_BAL_BYTES`.** EIP-7928 defines the BAL but hasn't pinned + a numeric upper bound yet. The catalogue currently uses + `MAX_BYTES_PER_TX` as a placeholder; this should be tightened + once the EIP lands. +2. **`MAX_BYTES_PER_EXECUTION_REQUEST`.** EIP-7685 hasn't pinned a + numeric per-element bound either. Same placeholder pattern as + `MAX_BAL_BYTES`; needs a concrete value. 3. **`Bitvector` SSZ encoding for `indices_bitarray` and `custody_columns`.** Both are `Bitvector[CELLS_PER_EXT_BLOB]` = `Bitvector[128]` = 16 bytes packed. Double-check that's the reading the Amsterdam spec wants (it currently describes it as "16 bytes interpreted as a bitarray"). -4. **`should_override_builder` typing.** SSZ has `bool` but it's - a 1-byte field. Keeping it inside `BuiltPayload` (rather than - moving to a header) was the [refactor.md](./refactor.md) - decision; this sketch follows that. -5. **`PayloadStatus` enum encoding.** A `uint8` with sentinel +4. **`PayloadStatus` enum encoding.** A `uint8` with sentinel values matches the JSON-RPC enum; SSZ has no native enum type so this is the cleanest mapping. Alternative: `Container { ... }` - wrapping a `uint8`. Open for discussion. -6. **`ExecutionPayloadBody` shared definition.** Today every fork - redefines `ExecutionPayloadBody` from scratch. The new spec - would benefit from a small set of fork-named containers - (`ExecutionPayloadBodyParis`, `ExecutionPayloadBodyShanghai`, - `ExecutionPayloadBodyAmsterdam`, …) with the URL `{fork}` - selecting which one. Not worked out here. -7. **Naming convention.** The legacy spec used `camelCase`; this + wrapping a `uint8`. +5. **Naming convention.** The legacy spec used `camelCase`; this sketch uses `snake_case` to match consensus-specs. Worth - confirming. -8. **`ByteVector[BYTES_PER_BLOB]` vs `ByteList[BYTES_PER_BLOB]`.** - A blob is fixed-size (131072 bytes), so `ByteVector` is the - correct typing. Verify against consensus-specs to keep - alignment. + confirming once before publication. diff --git a/src/engine/refactor.md b/src/engine/refactor.md index af62ab16b..b6d957b62 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -304,10 +304,13 @@ Amsterdam) and **MAY** serve any subset of older revisions alongside; All revisions use `POST` so that 128 versioned hashes (8 KiB hex) don't have to fit in the URL. All revisions return SSZ -`Optional[List[BlobEntry, MAX_BLOBS_REQUEST]]`, where each -`BlobEntry` carries an `available: boolean` per-entry flag for -per-blob misses on revisions that support partial responses. -Revision-specific contents live inside `BlobEntry.contents`. +`List[BlobEntry, MAX_BLOBS_REQUEST]` on `200 OK` and use HTTP +**`204 No Content`** to signal that the EL cannot serve the request +at all (syncing, blob pool unavailable, V2 all-or-nothing miss). +Within a `200` response, per-blob misses are reported via +`BlobEntry.available = false` on revisions that support partial +responses. Revision-specific contents live inside +`BlobEntry.contents`. #### `POST /engine/v2/blobs/v1` @@ -317,8 +320,8 @@ Replaces `engine_getBlobsV1` (Cancun, single-proof whole-blob). - **Response `BlobEntry.contents`:** `BlobAndProofV1 { blob, proof }` (one blob, one 48-byte KZG proof). - Partial responses supported: missing blobs surface as - `available=false` per entry. The outer `Optional` returns `None` - only if the EL cannot serve the request at all (e.g. syncing). + `available=false` per entry. `204 No Content` only when the EL + cannot serve the request at all (e.g. syncing). #### `POST /engine/v2/blobs/v2` @@ -327,8 +330,8 @@ Replaces `engine_getBlobsV2` (Osaka, all-or-nothing cell proofs). - **Request body:** SSZ `List[VersionedHash, MAX_BLOBS_REQUEST]`. - **Response `BlobEntry.contents`:** `BlobAndProofV2 { blob, proofs }` (one blob plus `CELLS_PER_EXT_BLOB` cell proofs). -- **All-or-nothing:** if any requested blob is missing, the outer - `Optional[List[...]]` returns `None`. Otherwise all entries have +- **All-or-nothing:** if any requested blob is missing, the EL + returns `204 No Content`. Otherwise `200 OK` and all entries have `available=true`. This matches today's V2 semantics. #### `POST /engine/v2/blobs/v3` @@ -339,8 +342,8 @@ proofs). - **Request body:** same as `/v2`. - **Response:** same `BlobEntry.contents` shape as `/v2`, but missing blobs surface as `available=false` per entry rather than collapsing - the whole response to `None`. The outer `Optional` returns `None` - only when the EL cannot serve the request at all. + the whole response. `204 No Content` only when the EL cannot serve + the request at all. #### `POST /engine/v2/blobs/v4` @@ -391,6 +394,66 @@ Returns JSON `ClientVersion[]` (same shape as today's `X-Engine-Client-Version` header on every request, removing the mutual-exchange handshake. +### Example: submit a payload + +```bash +curl -X POST http://localhost:8551/engine/v2/amsterdam/payloads \ + -H "Authorization: Bearer $JWT_TOKEN" \ + -H "Content-Type: application/octet-stream" \ + -H "Accept: application/octet-stream" \ + -H "X-Engine-Client-Version: LH/v6.2.1" \ + --data-binary @new_payload.ssz \ + -o payload_status.ssz +``` + +Request: + +``` +POST /engine/v2/amsterdam/payloads HTTP/2 +Host: localhost:8551 +Authorization: Bearer +Content-Type: application/octet-stream +Content-Length: 584 + +<584 bytes: SSZ(ExecutionPayloadEnvelope)> +``` + +Successful response (`status = VALID`): + +``` +HTTP/2 200 +Content-Type: application/octet-stream +Content-Length: 41 + +<41 bytes: SSZ(PayloadStatus)> +``` + +The 41 bytes break down as: `status` (1 byte = `0x00`, `VALID`) + +`latest_valid_hash` (4-byte offset + 32-byte hash = 36 bytes) ++ `validation_error` (4-byte offset + 0 bytes empty list). + +Error response (malformed body): + +``` +HTTP/2 400 +Content-Type: application/problem+json +Content-Length: 49 + +{ "type": "/engine-api/errors/ssz-decode-error" } +``` + +### Example: poll a built payload + +```bash +curl http://localhost:8551/engine/v2/amsterdam/payloads/0x1234567890abcdef \ + -H "Authorization: Bearer $JWT_TOKEN" \ + -H "Accept: application/octet-stream" \ + -o built_payload.ssz +``` + +Response carries `Cache-Control: no-store`; intermediaries MUST NOT +cache. See [Payload retrieval](#payload-retrieval). + --- ## Error model @@ -405,6 +468,15 @@ we use only **two** of the RFC 7807 fields: Omitted when the EL has nothing more to say than the `type` already conveys (e.g. canned SSZ-decode failures). +Success codes: + +| HTTP status | When | +| - | - | +| `200 OK` | SSZ-encoded response body | +| `204 No Content` | Null result (e.g. blob pool syncing on `/blobs/vN`); empty body | + +Error codes: + | HTTP status | `type` | Old JSON-RPC code | When | | - | - | - | - | | 400 Bad Request | `/engine-api/errors/parse-error` | -32700 | Body is not valid JSON / SSZ | @@ -416,6 +488,7 @@ we use only **two** of the RFC 7807 fields: | 409 Conflict | `/engine-api/errors/invalid-forkchoice` | -38002 | Forkchoice state is inconsistent (e.g. finalized not ancestor of head) | | 409 Conflict | `/engine-api/errors/reorg-too-deep` | -38006 | Reorg depth exceeds the EL's limit | | 413 Payload Too Large | `/engine-api/errors/request-too-large` | -38004 | Body exceeds an advertised `limits.*` value | +| 415 Unsupported Media Type | `/engine-api/errors/unsupported-media-type` | (new) | Request `Content-Type` does not match the endpoint's expected encoding (SSZ for hot-path, JSON for diagnostics) | | 422 Unprocessable Entity | `/engine-api/errors/invalid-body` | -32602 | Body decoded fine but has invalid values | | 422 Unprocessable Entity | `/engine-api/errors/invalid-attributes` | -38003 | `payload_attributes` validation failed | | 500 Internal Server Error | `/engine-api/errors/internal` | -32603 / -32000 | Unrecoverable server error; `detail` carries the message | @@ -467,6 +540,46 @@ understands via `GET /engine/v2/capabilities`. its supported fork schemas and endpoint set in a single JSON document at `/engine/v2/capabilities`. +### Capabilities format + +We considered advertising capabilities as a flat list of per-endpoint +strings (e.g. `"POST /amsterdam/payloads"`, the format used by the +existing `engine_exchangeCapabilities` method). The structured form +in `GET /capabilities` (separate `supported_forks`, +`fork_scoped_endpoints`, `independently_versioned`, +`unscoped_endpoints`, plus per-endpoint `limits`) is preferred +because: + +- Adding a fork doesn't multiply the capability list — one entry in + `supported_forks` covers every fork-scoped endpoint at once. +- The `limits.*` block can carry numeric per-endpoint bounds + (`bodies.max_count`, `blobs.max_versioned_hashes`, + `payload.max_bytes`) which a string-list form can't. +- It's easier to evolve: new fields land alongside, old CLs ignore + them. + +### Transition-window behavior + +During the rollout window, a CL upgraded to v2 may interact with an +EL still on the legacy JSON-RPC engine API. Two cases: + +- **EL doesn't expose `/engine/v2/...` at all.** The CL hits any v2 + URL and gets `404 Not Found` from the legacy server. The CL falls + back to JSON-RPC for the duration of that EL's lifetime — no + per-method retry dance. +- **EL exposes `/engine/v2/...` but doesn't know the URL fork.** The + CL hits `/{fork}/...` against an EL that only advertised + `supported_forks: [..., cancun]` while the CL is asking for + `amsterdam`. The EL returns + `400 /engine-api/errors/unsupported-fork`. The CL learns this once + from `GET /capabilities` and avoids issuing such requests; if it + doesn't, the per-request error is structured and explicit, not a + silent downgrade. + +There is **no per-method fallback ladder**. A CL either uses v2 or +JSON-RPC for the lifetime of an EL connection; mixing transports +within a connection is permitted but not required. + --- ## Authentication @@ -514,11 +627,19 @@ Unchanged in spirit: JWT (HS256, 256-bit shared secret). Differences: - **Trailing slashes are forbidden.** `/engine/v2/payloads` is the canonical form; `/engine/v2/payloads/` MUST return `404 method-not-found`. No automatic redirect. -- **Request body encoding:** `application/octet-stream` carrying SSZ - bytes for hot-path endpoints. JSON for diagnostic / metadata - endpoints (capabilities, identity, error bodies). -- **Response body encoding:** SSZ for hot-path data, JSON - (`application/json`) for diagnostics and error bodies. +- **Content-Type / Accept matrix:** + + | Channel | Header | Value | + | - | - | - | + | Hot-path request body (`/payloads`, `/forkchoice`, `/bodies`, `/blobs/vN`) | `Content-Type` | `application/octet-stream` (SSZ) | + | Hot-path request | `Accept` | `application/octet-stream` | + | Hot-path response success body | `Content-Type` | `application/octet-stream` (SSZ) | + | Diagnostic request / response (`/capabilities`, `/identity`) | `Content-Type` | `application/json` | + | Error response body (any endpoint) | `Content-Type` | `application/problem+json` | + + ELs MUST reject hot-path requests carrying any other `Content-Type` + with `415 Unsupported Media Type`. Diagnostic endpoints MUST be + served as JSON regardless of `Accept`. - **Compression:** Servers MAY support `Accept-Encoding: zstd, gzip`. Not required to implement; CLs MUST tolerate uncompressed responses. Blob bundles compress well, so operators are encouraged to enable @@ -771,6 +892,26 @@ We keep JSON available for **error bodies, capability discovery, and client identification** because those are ergonomic to debug with `curl` and not on the hot path. +#### Why not RLP? + +RLP is the EL's native encoding, so reusing it would cut one library +dependency on the EL side. We picked SSZ instead because: + +- **The CL natively serialises every payload field as SSZ today.** An + RLP transport would shift the conversion from "EL parses hex-JSON" + to "CL re-encodes SSZ as RLP" — same total work, just on a + different host. +- **SSZ pins fixed/variable lengths at the type level.** The + transport layer can enforce per-field size limits before + allocation, which RLP's recursive header structure makes harder. +- **`hash_tree_root` for free.** SSZ types come with a deterministic + Merkle root we can use for future content-addressed extensions + (e.g. payload identifiers, capability hashes). RLP would need a + separate hashing convention. +- **Alignment with the rest of the consensus stack.** Beacon API, + fork-choice store, gossip — all SSZ. Reusing the same encoding at + the EL/CL boundary keeps one mental model. + ### Simplifications & removed concepts 1. **`expectedBlobVersionedHashes`** — **removed**. The block-hash check @@ -873,9 +1014,10 @@ the summary exists for quick scanning. present; out-of-era blocks come back as `available=false`); independently versioned (`/blobs/vN`) gives each revision its own dedicated container. Both wrap their entries in - `BodyEntry { available, body }` / `BlobEntry { available, contents }` - with an outer `Optional[List[...]]` for the syncing / - all-or-nothing channel. Per-entry fork tags were rejected. + `BodyEntry { available, body }` / `BlobEntry { available, contents }`. + Whole-response "syncing / all-or-nothing miss" is signalled by + HTTP `204 No Content`, not an in-band SSZ sentinel. Per-entry fork + tags were rejected. #### Error model From 4e0fed12d3ebc9d1ca8829331a82b97b1d1bd154 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Thu, 28 May 2026 16:54:15 +0200 Subject: [PATCH 6/9] update to add TargetGasLimit --- src/engine/refactor-ssz.md | 6 ++++-- src/engine/refactor.md | 4 ++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index 76dcfa485..73f4664e7 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -169,6 +169,7 @@ PayloadAttributes { withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] parent_beacon_block_root: Root slot_number: Uint64 + target_gas_limit: Uint64 } ``` @@ -312,10 +313,11 @@ PayloadAttributesPrague = PayloadAttributesCancun # Osaka = Cancun (no shape change) PayloadAttributesOsaka = PayloadAttributesCancun -# Amsterdam = Cancun + slot_number +# Amsterdam = Cancun + slot_number + target_gas_limit PayloadAttributesAmsterdam { ...Cancun fields... - slot_number: Uint64 + slot_number: Uint64 + target_gas_limit: Uint64 } ``` diff --git a/src/engine/refactor.md b/src/engine/refactor.md index b6d957b62..1abafaf7b 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -150,6 +150,10 @@ Replaces `engine_forkchoiceUpdatedV{1..4}`. forkchoice update fails, no build is started and no custody change is applied. + When building a payload (Amsterdam+), the EL **MUST** use + `payload_attributes.target_gas_limit` as the target value for the + built block's `gas_limit`. + - **Response body:** SSZ-encoded `ForkchoiceUpdateResponse`: ``` From 83151eead3f87a354718f5765063f7817bde1628 Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Sun, 14 Jun 2026 10:43:31 +0200 Subject: [PATCH 7/9] engine-api: resolve feedback from implementers --- src/engine/refactor-ssz.md | 224 ++++++++++++++++++++++++++++++++++++- src/engine/refactor.md | 146 +++++++++++++++++------- 2 files changed, 321 insertions(+), 49 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index 73f4664e7..51716c222 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -32,6 +32,9 @@ - [`PayloadAttributes` per fork](#payloadattributes-per-fork) - [`ExecutionPayloadBody` per fork](#executionpayloadbody-per-fork) - [`BlobsBundle` per revision](#blobsbundle-per-revision) + - [`BuiltPayload` per fork](#builtpayload-per-fork) + - [`ExecutionPayloadEnvelope` per fork](#executionpayloadenvelope-per-fork) + - [`ForkchoiceUpdate` per fork](#forkchoiceupdate-per-fork) - [`BlobAndProof` per revision](#blobandproof-per-revision) - [Identification & capabilities](#identification--capabilities) - [Endpoint containers](#endpoint-containers) @@ -77,13 +80,15 @@ | `BYTES_PER_FIELD_ELEMENT` | `32` | [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844) | | `BYTES_PER_BLOB` | `FIELD_ELEMENTS_PER_BLOB * BYTES_PER_FIELD_ELEMENT` (131,072) | derived | | `CELLS_PER_EXT_BLOB` | `128` | [EIP-7594](https://eips.ethereum.org/EIPS/eip-7594) | -| `BYTES_PER_CELL` | `BYTES_PER_BLOB / CELLS_PER_EXT_BLOB` (1,024) | derived | +| `FIELD_ELEMENTS_PER_CELL` | `64` | [EIP-7594](https://eips.ethereum.org/EIPS/eip-7594) | +| `BYTES_PER_CELL` | `FIELD_ELEMENTS_PER_CELL * BYTES_PER_FIELD_ELEMENT` (2,048) | [EIP-7594](https://eips.ethereum.org/EIPS/eip-7594) | | `MAX_BAL_BYTES` | `MAX_BYTES_PER_TX` | [EIP-7928](https://eips.ethereum.org/EIPS/eip-7928) (placeholder until EIP pins a tighter bound) | | `MAX_EXECUTION_REQUESTS_PER_PAYLOAD` | `2**8` (256) | [EIP-7685](https://eips.ethereum.org/EIPS/eip-7685) | | `MAX_BYTES_PER_EXECUTION_REQUEST` | `MAX_BYTES_PER_TX` | this spec (placeholder; reuse the tx bound) | | `MAX_VERSIONED_HASHES_PER_REQUEST` | `128` | [Osaka](./osaka.md#engine_getblobsv2) | | `MAX_BLOBS_REQUEST` | `MAX_VERSIONED_HASHES_PER_REQUEST` (128) | derived | | `MAX_BODIES_REQUEST` | `2**5` (32) | [Shanghai](./shanghai.md#engine_getpayloadbodiesbyhashv1) | +| `MAX_REQUEST_BODY_SIZE` | `2**26` (67,108,864) | this spec (64 MiB; advertised as `limits.payload.max_bytes`) | | `MAX_ERROR_BYTES` | `1024` | this spec | | `MAX_CLIENT_CODE_LENGTH` | `2` | this spec | | `MAX_CLIENT_NAME_LENGTH` | `64` | this spec | @@ -215,6 +220,57 @@ deserialises as `VALID` rather than as a reserved sentinel. `3` from `/forkchoice` as a protocol error. `Optional[String]` resolves to `List[List[byte, MAX_ERROR_BYTES], 1]`. +This nesting is subtle, so two worked byte examples follow. (Some +implementations have mistakenly shipped a plain `List[byte, 1024]`, +which cannot distinguish an absent error from an empty-string error; +the wire shape below is the conformant one.) + +`PayloadStatus` is a variable-size SSZ container with one fixed field +(`status: uint8`, 1 byte) and two variable-size fields +(`latest_valid_hash`, `validation_error`), each contributing a 4-byte +offset in the fixed part. The fixed part is therefore `1 + 4 + 4` = `9` +bytes, and the variable parts follow in field order. + +**Example A — `VALID`, no error** (the 41-byte response from +[refactor.md § Example: submit a payload](./refactor.md#example-submit-a-payload)): + +``` +status : 0x00 # 1 byte, VALID +offset[lvh] : 0x09000000 # 4 bytes -> 9 +offset[verr] : 0x29000000 # 4 bytes -> 41 +latest_valid_hash : [<32-byte hash>] # Optional present: + # inner offset omitted because + # Hash32 is fixed-size; the List[T,1] + # body is just the 32 bytes +validation_error : # Optional absent: List length 0, 0 bytes +``` + +Total = `1 + 4 + 4 + 32 + 0` = `41` bytes. The `latest_valid_hash` +optional is *present* (length-1 list of a fixed-size element, so no +inner offset), and `validation_error` is *absent* (length-0 list). + +**Example B — `INVALID`, error present** (`"bad state root"`, 14 +bytes of UTF-8): + +``` +status : 0x01 # 1 byte, INVALID +offset[lvh] : 0x09000000 # 4 bytes -> 9 +offset[verr] : 0x09000000 # 4 bytes -> 9 (lvh is empty) +latest_valid_hash : # Optional absent: List length 0, 0 bytes +validation_error : 0x04000000 # outer List[..,1] present: one element, + # so one 4-byte offset -> 4 (relative to + # the start of this variable region) + : 0x6261642073746174... (14 bytes) # inner List[byte,1024] body: the UTF-8 text +``` + +The `validation_error` variable region is `4 + 14` = `18` bytes: a +single 4-byte offset (because the outer `List[String, 1]` has one +variable-size element, `String` = `List[byte, 1024]`, so its offset is +emitted) pointing at the start of the 14-byte text. Total = +`1 + 4 + 4 + 0 + 18` = `27` bytes. Note the inner 4-byte offset that +precedes the text whenever the error is present — this is exactly the +byte that a plain `List[byte, 1024]` implementation omits, and the +source of the divergence. --- @@ -376,6 +432,137 @@ BlobsBundleV2 { `BuiltPayload` for Cancun / Prague carries `BlobsBundleV1`; Osaka / Amsterdam carries `BlobsBundleV2`. +### `BuiltPayload` per fork + +Returned by `GET /{fork}/payloads/{payloadId}`. Like the +`ExecutionPayload` catalogue above, the shape evolves per fork; this +section pins every variant so there is no ambiguity for pre-Amsterdam +forks. **All variants use the same field order; SSZ fields are +positional, so the order below is normative and MUST be followed +exactly.** + +Two points where implementations have diverged, settled here: + +- **Field order.** `execution_requests` (when present) comes + **before** `should_override_builder`. This intentionally differs + from the legacy JSON-RPC `getPayload` envelope, which appended + `executionRequests` last. Implementations that kept the legacy order + are non-conformant. +- **Paris shape.** Paris `BuiltPayload` is **not** a bare + `ExecutionPayload`; it is the `{payload, block_value}` container + below. (`engine_getPayloadV1` returned a bare payload; `V2` + introduced the `{executionPayload, blockValue}` wrapper. The v2 API + uses the wrapper uniformly from Paris on, so every fork has the same + outer container and a CL never has to special-case Paris.) + +``` +# Paris — payload + block_value only +BuiltPayloadParis { + payload: ExecutionPayloadParis + block_value: Uint256 +} + +# Shanghai — Paris + should_override_builder +# (shouldOverrideBuilder was introduced in engine_getPayloadV3/Shanghai) +BuiltPayloadShanghai { + payload: ExecutionPayloadShanghai + block_value: Uint256 + should_override_builder: Boolean +} + +# Cancun — Shanghai + blobs_bundle (V1) +BuiltPayloadCancun { + payload: ExecutionPayloadCancun + block_value: Uint256 + blobs_bundle: BlobsBundleV1 + should_override_builder: Boolean +} + +# Prague — Cancun + execution_requests (note: before should_override_builder) +BuiltPayloadPrague { + payload: ExecutionPayloadPrague + block_value: Uint256 + blobs_bundle: BlobsBundleV1 + execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] + should_override_builder: Boolean +} + +# Osaka — Prague but blobs_bundle is V2 (cell proofs) +BuiltPayloadOsaka { + payload: ExecutionPayloadOsaka + block_value: Uint256 + blobs_bundle: BlobsBundleV2 + execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] + should_override_builder: Boolean +} + +# Amsterdam — Osaka with the Amsterdam ExecutionPayload (BAL + slot_number) +BuiltPayloadAmsterdam { + payload: ExecutionPayloadAmsterdam + block_value: Uint256 + blobs_bundle: BlobsBundleV2 + execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] + should_override_builder: Boolean +} +``` + +The Amsterdam variant is the one shown in the +[endpoint section below](#get-amsterdampayloadspayloadid). + +### `ExecutionPayloadEnvelope` per fork + +The request body of `POST /{fork}/payloads`. `parent_beacon_block_root` +exists from Cancun on (it was a separate `engine_newPayload` parameter +since Cancun); `execution_requests` from Prague on. Field order is +normative. + +``` +# Paris / Shanghai — bare payload, no envelope fields +ExecutionPayloadEnvelopeParis { + payload: ExecutionPayloadParis +} +ExecutionPayloadEnvelopeShanghai { + payload: ExecutionPayloadShanghai +} + +# Cancun / Osaka — + parent_beacon_block_root +ExecutionPayloadEnvelopeCancun { + payload: ExecutionPayloadCancun + parent_beacon_block_root: Root +} + +# Prague — Cancun + execution_requests +ExecutionPayloadEnvelopePrague { + payload: ExecutionPayloadPrague + parent_beacon_block_root: Root + execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] +} + +# Osaka = Prague shape (ExecutionPayloadOsaka inner) +# Amsterdam = Prague shape (ExecutionPayloadAmsterdam inner) +``` + +### `ForkchoiceUpdate` per fork + +The request body of `POST /{fork}/forkchoice`. `payload_attributes` +selects the matching per-fork `PayloadAttributes` shape; +`custody_columns` exists only from Amsterdam on. + +``` +# Paris .. Osaka +ForkchoiceUpdate { + forkchoice_state: ForkchoiceState + payload_attributes: Optional[PayloadAttributes] # the fork's PayloadAttributes shape +} + +# Amsterdam — + custody_columns +ForkchoiceUpdateAmsterdam { + forkchoice_state: ForkchoiceState + payload_attributes: Optional[PayloadAttributesAmsterdam] + custody_columns: Optional[Bitvector[CELLS_PER_EXT_BLOB]] +} +``` + ### `BlobAndProof` per revision Used by `BlobEntry.contents` on the blob-pool endpoints (`/blobs/vN`). @@ -532,9 +719,18 @@ BodyEntry { `available` is `false` when the requested block is unavailable / pruned, **or** when the block's timestamp falls outside the URL -fork's active range, **or** for range queries when the block number -is past the latest known block. When `available=false`, `body` is -zero-valued and CLs MUST ignore its contents. +fork's active range. When `available=false`, `body` is zero-valued +and CLs MUST ignore its contents. + +For **range queries**, blocks past the latest known block are **not** +represented by an `available=false` entry — they are **omitted**, and +the response is truncated at the latest known block (the legacy +`engine_getPayloadBodiesByRange` "no trailing nulls" rule). The +`entries` list therefore has length `min(count, head - from + 1)` for +`from <= head`, and is empty when `from > head`. Only `available=false` +appears for in-range-but-out-of-era or pruned blocks; never as +trailing padding. See +[refactor.md § Historical bodies](./refactor.md#historical-bodies). Each fork URL pairs with its own `ExecutionPayloadBody` schema. The Amsterdam variant carries every field unconditionally: @@ -660,6 +856,15 @@ BlobsV3Response { } ``` +`/v3` reuses `BlobV2Entry` (and therefore `BlobAndProofV2`) +**verbatim** — the wire encoding of a `/v3` entry is byte-identical to +a `/v2` entry; only the response-level semantics differ (`/v3` allows +per-entry `available=false`, `/v2` is all-or-nothing). There is **no** +separate `BlobV3Entry` type: implementations **MUST NOT** define one, +to avoid the two drifting apart. The only difference between the +revisions lives in the endpoint's response semantics, not the entry +container. + ### `POST /blobs/v4` Replaces `engine_getBlobsV4` (Amsterdam cell-range selection). @@ -700,8 +905,15 @@ the requested indices, individual missing cells are also `[]`, and the corresponding `proofs` entry MUST also be `[]` (`null` in the old spec). -`BYTES_PER_CELL` = `BYTES_PER_BLOB / CELLS_PER_EXT_BLOB` = `1024` -(EIP-7594). +`BYTES_PER_CELL` = `FIELD_ELEMENTS_PER_CELL * BYTES_PER_FIELD_ELEMENT` += `2048` ([EIP-7594](https://eips.ethereum.org/EIPS/eip-7594)). A cell +spans `FIELD_ELEMENTS_PER_CELL` (64) field elements of the *extended* +blob (`FIELD_ELEMENTS_PER_EXT_BLOB` = `2 * FIELD_ELEMENTS_PER_BLOB` = +`8192`), so there are `8192 / 64` = `128` = `CELLS_PER_EXT_BLOB` cells, +each `64 * 32` = `2048` bytes — `c-kzg-4844`'s `compute_cells` writes +exactly this. The earlier `BYTES_PER_BLOB / CELLS_PER_EXT_BLOB` +derivation was wrong: it divided the *original*-blob byte count over +the *extended*-blob cell count, halving the true cell size. --- diff --git a/src/engine/refactor.md b/src/engine/refactor.md index 1abafaf7b..75ff925e2 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -217,13 +217,23 @@ Replaces `engine_getPayloadV{1..6}`. ``` BuiltPayload { - payload: ExecutionPayload - block_value: Uint256 - blobs_bundle: BlobsBundle - execution_requests: List[Bytes, MAX_REQUESTS] + payload: ExecutionPayload + block_value: Uint256 + blobs_bundle: BlobsBundle + execution_requests: List[Bytes, MAX_REQUESTS] should_override_builder: bool } ``` + + The shape above is the Amsterdam variant. **Field order is + normative**: `execution_requests` precedes `should_override_builder`. + This deliberately diverges from the legacy JSON-RPC envelope, which + appended `executionRequests` last; SSZ fields are positional, so the + order is part of the wire format and **MUST** be followed exactly. + Pre-Amsterdam forks have their own `BuiltPayload` shapes (the fields + a fork doesn't have are absent, and the `blobs_bundle` revision + tracks the fork) — see the per-fork `BuiltPayload` catalogue in + [refactor-ssz.md](./refactor-ssz.md) for every variant from Paris up. - **404** if `payloadId` is unknown or expired. Polling semantics are unchanged from `engine_getPayload`: the CL calls @@ -257,6 +267,15 @@ blocks whose timestamp falls in `{fork}`'s active time range. A CL fetching bodies that span a fork boundary issues separate requests against each fork URL. +The EL **MUST** apply both meanings of `{fork}`: it **MUST** serialise +each entry against the `{fork}` schema **and MUST** filter the response +to blocks whose timestamp falls in `{fork}`'s active time range. +Using the segment only for schema selection — and returning blocks +from outside the fork's era — is **non-conformant**. A requested block +that exists but whose timestamp lies outside the URL fork's range +**MUST** come back as `available=false` (it is not omitted, unlike a +past-head block in a range query; see below). + Concretely: - `/cancun/bodies/hash` returns bodies *only* for blocks in the @@ -272,31 +291,48 @@ Concretely: Replaces `engine_getPayloadBodiesByHashV{1,2}`. Uses `POST` so that large hash lists travel in the request body rather than the URL. -- **Request body:** SSZ-encoded `List[Hash32, MAX_BODIES_REQUEST]`. +- **Request body:** SSZ-encoded `BodiesByHashRequest` (a single-field + container wrapping `block_hashes: List[Hash32, MAX_BODIES_REQUEST]`; + see [refactor-ssz.md](./refactor-ssz.md)). Top-level bodies are + wrapped in a container rather than sent as a bare SSZ list, matching + the beacon-API convention; this costs a 4-byte offset but lets future + revisions add fields without a breaking wire change. #### `GET /engine/v2/{fork}/bodies?from=N&count=M` Replaces `engine_getPayloadBodiesByRangeV{1,2}`. Range fits comfortably -in the URL. Block numbers outside the URL fork's active range come -back as `available=false`; if the requested range straddles a fork -boundary the CL re-issues against the next fork URL for the unfilled -suffix. - -- **Response body** (both endpoints): SSZ-encoded - `List[BodyEntry, MAX_BODIES_REQUEST]`. Each `BodyEntry` carries an - `available: boolean` flag and an `ExecutionPayloadBody` serialised - against the **`{fork}` schema from the URL**. `available` is false - in any of the following cases: - - the block is unavailable / pruned, - - the block's timestamp falls outside the URL fork's active range, - - or for range queries, the block number is past the latest known - block. +in the URL. Block numbers whose timestamp falls outside the URL fork's +active range come back as `available=false`; block numbers past the +latest known block are **omitted entirely** (the response is truncated +at head, not padded — see the response-length note below). If the +requested range straddles a fork boundary the CL re-issues against the +next fork URL for the unfilled suffix. + +- **Response body** (both endpoints): SSZ-encoded `BodiesResponse` + (a single-field container wrapping + `entries: List[BodyEntry, MAX_BODIES_REQUEST]`). Each `BodyEntry` + carries an `available: boolean` flag and an `ExecutionPayloadBody` + serialised against the **`{fork}` schema from the URL**. `available` + is false in either of the following cases: + - the block is unavailable / pruned, or + - the block's timestamp falls outside the URL fork's active range. When `available=false`, the `body` field is zero-valued and CLs MUST ignore its contents. See [SSZ encoding conventions](#ssz-encoding-conventions) for the `BodyEntry` wrapper definition. +- **Response length (range queries).** The response carries one + `BodyEntry` per known block in the requested range; it is + **truncated at the latest known block** and is **not** padded out to + `count` entries with `available=false` placeholders. A request whose + range extends past head therefore returns fewer than `count` + entries, and a request entirely past head returns an empty + `entries` list. This carries forward the legacy + `engine_getPayloadBodiesByRange` "no trailing nulls" rule. The CL + detects the unfilled suffix from the shortfall and re-issues against + the next fork URL if the range straddled a fork boundary. + ### Blob pool The blob endpoint is **independently versioned**: legacy @@ -307,20 +343,25 @@ Amsterdam) and **MAY** serve any subset of older revisions alongside; `GET /capabilities` advertises the actual list. All revisions use `POST` so that 128 versioned hashes (8 KiB hex) -don't have to fit in the URL. All revisions return SSZ -`List[BlobEntry, MAX_BLOBS_REQUEST]` on `200 OK` and use HTTP -**`204 No Content`** to signal that the EL cannot serve the request -at all (syncing, blob pool unavailable, V2 all-or-nothing miss). -Within a `200` response, per-blob misses are reported via -`BlobEntry.available = false` on revisions that support partial -responses. Revision-specific contents live inside +don't have to fit in the URL. All revisions take a single-field +request container (`BlobsVNRequest`) and return a single-field +response container (`BlobsVNResponse`) wrapping +`entries: List[BlobVNEntry, MAX_BLOBS_REQUEST]` on `200 OK`, and use +HTTP **`204 No Content`** to signal that the EL cannot serve the +request at all (syncing, blob pool unavailable, V2 all-or-nothing +miss). Wrapping top-level bodies in a container (rather than a bare +SSZ list) costs a 4-byte offset but matches the beacon-API convention +and keeps the revisions extensible. Within a `200` response, per-blob +misses are reported via `BlobEntry.available = false` on revisions +that support partial responses. Revision-specific contents live inside `BlobEntry.contents`. #### `POST /engine/v2/blobs/v1` Replaces `engine_getBlobsV1` (Cancun, single-proof whole-blob). -- **Request body:** SSZ `List[VersionedHash, MAX_BLOBS_REQUEST]`. +- **Request body:** SSZ `BlobsV1Request` (wraps + `versioned_hashes: List[VersionedHash, MAX_BLOBS_REQUEST]`). - **Response `BlobEntry.contents`:** `BlobAndProofV1 { blob, proof }` (one blob, one 48-byte KZG proof). - Partial responses supported: missing blobs surface as @@ -331,7 +372,8 @@ Replaces `engine_getBlobsV1` (Cancun, single-proof whole-blob). Replaces `engine_getBlobsV2` (Osaka, all-or-nothing cell proofs). -- **Request body:** SSZ `List[VersionedHash, MAX_BLOBS_REQUEST]`. +- **Request body:** SSZ `BlobsV2Request` (wraps + `versioned_hashes: List[VersionedHash, MAX_BLOBS_REQUEST]`). - **Response `BlobEntry.contents`:** `BlobAndProofV2 { blob, proofs }` (one blob plus `CELLS_PER_EXT_BLOB` cell proofs). - **All-or-nothing:** if any requested blob is missing, the EL @@ -379,13 +421,24 @@ the server is willing to serve in one request: "independently_versioned": { "blobs": ["v1", "v2", "v3", "v4"] }, "unscoped_endpoints": ["capabilities", "identity"], "limits": { - "bodies.max_count": 128, + "bodies.max_count": 32, "blobs.max_versioned_hashes": 128, "payload.max_bytes": 67108864 } } ``` +The `limits.*` values map onto the SSZ `MAX_*` constants where one +exists: `bodies.max_count` is bounded by `MAX_BODIES_REQUEST` (`32`, +inherited from Shanghai's `engine_getPayloadBodiesByHashV1`) and +`blobs.max_versioned_hashes` by `MAX_VERSIONED_HASHES_PER_REQUEST` +(`128`). `payload.max_bytes` is bounded by `MAX_REQUEST_BODY_SIZE` +(`2**26` = `67108864`, 64 MiB); see +[refactor-ssz.md § `MAX_*` constants](./refactor-ssz.md#max-constants). +The advertised numbers are an upper bound the server is willing to +serve; operators MAY advertise lower values, but MUST NOT advertise +higher than the corresponding `MAX_*` constant. + The `independently_versioned` map advertises endpoints whose URL carries an explicit `/vN` revision. ELs MAY support multiple revisions concurrently (e.g. `["v1", "v2"]`); CLs pick whichever they @@ -612,13 +665,20 @@ Unchanged in spirit: JWT (HS256, 256-bit shared secret). Differences: ## Transport & framing -- **Protocol:** HTTP/2 is **required**. Both TCP and IPC transports - use **h2c** (HTTP/2 cleartext); JWT-on-every-request provides - authentication, so TLS termination is left to a reverse proxy if - the operator wants it. HTTP/2 multiplexing means a single CL→EL - connection can carry the full request mix (forkchoice, payload - submission, blob fetches, body fetches) without head-of-line - blocking. HTTP/1.1 is not supported. +- **Protocol:** both **HTTP/2 and HTTP/1.1 MUST be supported**, with + HTTP/2 **preferred**. Both TCP and IPC transports use **cleartext** + (h2c for HTTP/2); JWT-on-every-request provides authentication, so + TLS termination is left to a reverse proxy if the operator wants + it. Servers and CLs **SHOULD** negotiate HTTP/2 where available + (ALPN over TLS, or the HTTP/2 prior-knowledge / `h2c` upgrade over + cleartext) and fall back to HTTP/1.1 only when the peer does not + speak h2. HTTP/2 multiplexing lets a single CL→EL connection carry + the full request mix (forkchoice, payload submission, blob fetches, + body fetches) without head-of-line blocking; on HTTP/1.1 that + benefit is lost (requests serialise per connection, or the CL opens + several connections), but the API is otherwise identical — same + paths, headers, bodies, and status codes. CLs that fall back to + HTTP/1.1 SHOULD use connection pooling to recover some concurrency. - **Default port:** `8551`, shared with the legacy JSON-RPC engine API. The two surfaces are distinguished by path: legacy JSON-RPC remains at `/` (and accepts JSON-RPC method calls), the new API lives under @@ -628,9 +688,6 @@ Unchanged in spirit: JWT (HS256, 256-bit shared secret). Differences: body schema (`paris`, `shanghai`, `cancun`, `prague`, `osaka`, `amsterdam`, …). Adding a fork = adding one path prefix and one set of SSZ schemas. See [Versioning](#versioning-model). -- **Trailing slashes are forbidden.** `/engine/v2/payloads` is the - canonical form; `/engine/v2/payloads/` MUST return - `404 method-not-found`. No automatic redirect. - **Content-Type / Accept matrix:** | Channel | Header | Value | @@ -968,14 +1025,17 @@ the summary exists for quick scanning. #### Transport -- **HTTP/2 required**, h2c (cleartext) for both TCP and IPC. No - HTTP/1.1 fallback. JWT-on-every-request authenticates; TLS - termination is left to a reverse proxy. +- **HTTP/2 and HTTP/1.1 both MUST be supported**, HTTP/2 preferred; + cleartext (h2c for HTTP/2) for both TCP and IPC. Peers negotiate + HTTP/2 where available and fall back to HTTP/1.1 otherwise. The API + surface (paths, headers, bodies, status codes) is identical on both; + only HTTP/2's stream multiplexing is lost on the 1.1 fallback. + JWT-on-every-request authenticates; TLS termination is left to a + reverse proxy. - **IPC** is h2c over UNIX socket — same paths and headers as TCP, single code path. - **Default port `8551`**, shared with the legacy JSON-RPC API (distinguished by path). -- **Trailing slashes are forbidden** — return `404 method-not-found`. - **Flow-control:** SHOULD set `INITIAL_WINDOW_SIZE` ≥ 1 MiB. `MAX_FRAME_SIZE` and `MAX_HEADER_LIST_SIZE` use HTTP/2 defaults. - **Connection lifecycle:** CLs MAY open fresh h2 connections per From e509f20f9e9107c262bc65a46ea71cb3c8d19a7d Mon Sep 17 00:00:00 2001 From: MariusVanDerWijden Date: Wed, 17 Jun 2026 10:56:19 +0200 Subject: [PATCH 8/9] advertise all forks --- src/engine/refactor-ssz.md | 129 +++++++++++++++++++++---------- src/engine/refactor.md | 151 ++++++++++++++++++++++++++++++++++++- 2 files changed, 240 insertions(+), 40 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index 51716c222..dfb41dacc 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -38,10 +38,10 @@ - [`BlobAndProof` per revision](#blobandproof-per-revision) - [Identification & capabilities](#identification--capabilities) - [Endpoint containers](#endpoint-containers) - - [`POST /amsterdam/payloads`](#post-amsterdampayloads) - - [`POST /amsterdam/forkchoice`](#post-amsterdamforkchoice) - - [`GET /amsterdam/payloads/{payloadId}`](#get-amsterdampayloadspayloadid) - - [`POST /amsterdam/bodies/hash` and `GET /amsterdam/bodies?...`](#post-amsterdambodieshash-and-get-amsterdambodies) + - [`POST /{fork}/payloads`](#post-forkpayloads) + - [`POST /{fork}/forkchoice`](#post-forkforkchoice) + - [`GET /{fork}/payloads/{payloadId}`](#get-forkpayloadspayloadid) + - [`POST /{fork}/bodies/hash` and `GET /{fork}/bodies?...`](#post-forkbodieshash-and-get-forkbodies) - [`POST /blobs/v1`](#post-blobsv1) - [`POST /blobs/v2`](#post-blobsv2) - [`POST /blobs/v3`](#post-blobsv3) @@ -455,6 +455,19 @@ Two points where implementations have diverged, settled here: uses the wrapper uniformly from Paris on, so every fork has the same outer container and a CL never has to special-case Paris.) +Field-introduction history (from the legacy `engine_getPayloadV{1..5}` +response evolution; see [shanghai.md](./shanghai.md), +[cancun.md](./cancun.md), [prague.md](./prague.md), +[osaka.md](./osaka.md)): + +| Field | Introduced | Notes | +| - | - | - | +| `payload`, `block_value` | Shanghai (`getPayloadV2`) | the wrapper itself; v2 API back-applies it to Paris | +| `blobs_bundle` | Cancun (`getPayloadV3`) | `BlobsBundleV1` (single proof) | +| `should_override_builder` | Cancun (`getPayloadV3`) | introduced alongside `blobs_bundle` — **not** Shanghai | +| `execution_requests` | Prague (`getPayloadV4`) | placed before `should_override_builder` | +| `blobs_bundle` → `BlobsBundleV2` | Osaka (`getPayloadV5`) | cell proofs replace the single proof | + ``` # Paris — payload + block_value only BuiltPayloadParis { @@ -462,15 +475,15 @@ BuiltPayloadParis { block_value: Uint256 } -# Shanghai — Paris + should_override_builder -# (shouldOverrideBuilder was introduced in engine_getPayloadV3/Shanghai) +# Shanghai — Paris + nothing new on the wrapper (getPayloadV2 added the +# {executionPayload, blockValue} wrapper; that's already the Paris shape here) BuiltPayloadShanghai { payload: ExecutionPayloadShanghai block_value: Uint256 - should_override_builder: Boolean } -# Cancun — Shanghai + blobs_bundle (V1) +# Cancun — Shanghai + blobs_bundle (V1) + should_override_builder +# (both introduced together in engine_getPayloadV3/Cancun) BuiltPayloadCancun { payload: ExecutionPayloadCancun block_value: Uint256 @@ -507,7 +520,7 @@ BuiltPayloadAmsterdam { ``` The Amsterdam variant is the one shown in the -[endpoint section below](#get-amsterdampayloadspayloadid). +[endpoint section below](#get-forkpayloadspayloadid). ### `ExecutionPayloadEnvelope` per fork @@ -525,7 +538,7 @@ ExecutionPayloadEnvelopeShanghai { payload: ExecutionPayloadShanghai } -# Cancun / Osaka — + parent_beacon_block_root +# Cancun — + parent_beacon_block_root ExecutionPayloadEnvelopeCancun { payload: ExecutionPayloadCancun parent_beacon_block_root: Root @@ -538,8 +551,8 @@ ExecutionPayloadEnvelopePrague { execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] } -# Osaka = Prague shape (ExecutionPayloadOsaka inner) -# Amsterdam = Prague shape (ExecutionPayloadAmsterdam inner) +# Osaka = Prague shape, with ExecutionPayloadOsaka inner +# Amsterdam = Prague shape, with ExecutionPayloadAmsterdam inner ``` ### `ForkchoiceUpdate` per fork @@ -616,15 +629,46 @@ CapabilitiesResponse { ## Endpoint containers -### `POST /amsterdam/payloads` - -Replaces `engine_newPayloadV5`. - -#### Request - -``` -ExecutionPayloadEnvelope { - payload: ExecutionPayload +The endpoint sketches below use the **Amsterdam** shapes as the worked +example. Every fork-scoped endpoint (`/{fork}/payloads`, +`/{fork}/forkchoice`, `/{fork}/bodies`) is defined for **every fork +from Paris onward**; substitute the matching entry from the +[per-fork container catalogue](#per-fork-container-catalogue) for the +URL's `{fork}`. For instance `POST /cancun/payloads` takes an +`ExecutionPayloadEnvelopeCancun` wrapping an `ExecutionPayloadCancun`, +and `GET /shanghai/payloads/{id}` returns a `BuiltPayloadShanghai`. + +> **Fork-invariant containers.** `PayloadStatus`, `ForkchoiceState`, +> `ForkchoiceUpdateResponse`, and `Withdrawal` have the **same shape +> across all forks** — only the fork-scoped payload/attributes/body +> containers and the `BuiltPayload` / `ExecutionPayloadEnvelope` / +> `ForkchoiceUpdate` wrappers that embed them vary by fork. + +> **Implementation note (monolithic vs. per-fork types).** This +> catalogue names a distinct container per fork +> (`ExecutionPayloadParis`, `…Shanghai`, …) so the wire shape of each +> fork is unambiguous. Implementations are free to model these as a +> single *monolithic* superset container per type whose fields are +> gated on the active fork (e.g. one `ExecutionPayload` struct where +> `withdrawals` participates only from Shanghai, `block_access_list` +> only from Amsterdam, etc.), driving the gate from the URL `{fork}`. +> This is a valid strategy **as long as the bytes on the wire are +> identical** to the per-fork shape for that fork — i.e. a gated-off +> field contributes neither an offset nor content. go-ethereum's +> implementation takes this monolithic approach. + +### `POST /{fork}/payloads` + +Replaces `engine_newPayloadV{1..5}` (Amsterdam shown; `engine_newPayloadV5`). +Each fork uses its `ExecutionPayloadEnvelope{Fork}` from the catalogue +above — Paris/Shanghai carry the bare payload, Cancun+ add +`parent_beacon_block_root`, Prague+ add `execution_requests`. + +#### Request (Amsterdam) + +``` +ExecutionPayloadEnvelopeAmsterdam { + payload: ExecutionPayloadAmsterdam parent_beacon_block_root: Root execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] } @@ -637,21 +681,25 @@ from `payload.transactions`). `PayloadStatus` (full enum, `0`/`1`/`2`/`3`). -### `POST /amsterdam/forkchoice` +### `POST /{fork}/forkchoice` -Replaces `engine_forkchoiceUpdatedV4`. +Replaces `engine_forkchoiceUpdatedV{1..4}` (Amsterdam shown; +`engine_forkchoiceUpdatedV4`). Each fork uses its `ForkchoiceUpdate{Fork}` +and `PayloadAttributes{Fork}` from the catalogue; `custody_columns` +exists only from Amsterdam on. `ForkchoiceState` and the response are +fork-invariant. -#### Request +#### Request (Amsterdam) ``` -ForkchoiceUpdate { +ForkchoiceUpdateAmsterdam { forkchoice_state: ForkchoiceState - payload_attributes: Optional[PayloadAttributes] + payload_attributes: Optional[PayloadAttributesAmsterdam] custody_columns: Optional[Bitvector[CELLS_PER_EXT_BLOB]] } ``` -#### Response +#### Response (all forks) ``` ForkchoiceUpdateResponse { @@ -660,15 +708,17 @@ ForkchoiceUpdateResponse { } ``` -### `GET /amsterdam/payloads/{payloadId}` +### `GET /{fork}/payloads/{payloadId}` -Replaces `engine_getPayloadV6`. +Replaces `engine_getPayloadV{1..6}` (Amsterdam shown; +`engine_getPayloadV6`). Each fork returns its `BuiltPayload{Fork}` from +the catalogue. -#### Response +#### Response (Amsterdam) ``` -BuiltPayload { - payload: ExecutionPayload +BuiltPayloadAmsterdam { + payload: ExecutionPayloadAmsterdam block_value: Uint256 blobs_bundle: BlobsBundleV2 # see consensus-specs Osaka execution_requests: List[ByteList[MAX_BYTES_PER_EXECUTION_REQUEST], MAX_EXECUTION_REQUESTS_PER_PAYLOAD] @@ -676,9 +726,9 @@ BuiltPayload { } BlobsBundleV2 { - commitments: List[Bytes48, MAX_BLOBS_PER_PAYLOAD] - proofs: List[Bytes48, MAX_BLOBS_PER_PAYLOAD * CELLS_PER_EXT_BLOB] - blobs: List[ByteVector[BYTES_PER_BLOB], MAX_BLOBS_PER_PAYLOAD] + commitments: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK] + proofs: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK * CELLS_PER_EXT_BLOB] + blobs: List[ByteVector[BYTES_PER_BLOB], MAX_BLOB_COMMITMENTS_PER_BLOCK] } ``` @@ -686,11 +736,12 @@ BlobsBundleV2 { have length `len(blobs) * CELLS_PER_EXT_BLOB` (mirrors the `engine_getPayloadV5` rule from osaka.md). -### `POST /amsterdam/bodies/hash` and `GET /amsterdam/bodies?...` +### `POST /{fork}/bodies/hash` and `GET /{fork}/bodies?...` -Replace `engine_getPayloadBodiesByHashV2` and -`engine_getPayloadBodiesByRangeV2`. Both return the same response -container. +Replace `engine_getPayloadBodiesByHashV{1,2}` and +`engine_getPayloadBodiesByRangeV{1,2}` (Amsterdam shown). Both return +the same response container; the inner `ExecutionPayloadBody` follows +the URL `{fork}` per the catalogue. #### Request — `/bodies/hash` diff --git a/src/engine/refactor.md b/src/engine/refactor.md index 75ff925e2..3e432249b 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -23,6 +23,7 @@ - [Historical bodies](#historical-bodies) - [Blob pool](#blob-pool) - [Capabilities & identification](#capabilities--identification) + - [Examples: every fork](#examples-every-fork) - [Error model](#error-model) - [Versioning model](#versioning-model) - [Authentication](#authentication) @@ -511,6 +512,140 @@ curl http://localhost:8551/engine/v2/amsterdam/payloads/0x1234567890abcdef \ Response carries `Cache-Control: no-store`; intermediaries MUST NOT cache. See [Payload retrieval](#payload-retrieval). +### Examples: every fork + +The examples above use Amsterdam. The shapes below show how the two +main hot-path bodies change fork-to-fork, so each fork's URL has a +worked example. Field names follow the +[per-fork container catalogue](./refactor-ssz.md#per-fork-container-catalogue); +`<…>` denotes nested SSZ. Only the fields that change per fork are +called out; every fork from Paris on is a valid `{fork}` segment. + +#### `POST /{fork}/payloads` request body (`ExecutionPayloadEnvelope{Fork}`) + +``` +# POST /engine/v2/paris/payloads +ExecutionPayloadEnvelopeParis { + payload: # base fields only (no withdrawals/blob-gas/BAL) +} + +# POST /engine/v2/shanghai/payloads +ExecutionPayloadEnvelopeShanghai { + payload: # + withdrawals +} + +# POST /engine/v2/cancun/payloads +ExecutionPayloadEnvelopeCancun { + payload: # + blob_gas_used, excess_blob_gas + parent_beacon_block_root: # NEW: was a side param since Cancun +} + +# POST /engine/v2/prague/payloads +ExecutionPayloadEnvelopePrague { + payload: # == Cancun payload shape + parent_beacon_block_root: + execution_requests: # NEW: was a side param since Prague +} + +# POST /engine/v2/osaka/payloads — identical envelope shape to Prague +ExecutionPayloadEnvelopeOsaka { + payload: # == Cancun payload shape + parent_beacon_block_root: + execution_requests: +} + +# POST /engine/v2/amsterdam/payloads +ExecutionPayloadEnvelopeAmsterdam { + payload: # + block_access_list, slot_number + parent_beacon_block_root: + execution_requests: +} +``` + +The response for **all** forks is `PayloadStatus` (fork-invariant). + +#### `GET /{fork}/payloads/{id}` response body (`BuiltPayload{Fork}`) + +``` +# GET /engine/v2/paris/payloads/{id} +BuiltPayloadParis { + payload: + block_value: +} + +# GET /engine/v2/shanghai/payloads/{id} +BuiltPayloadShanghai { + payload: + block_value: +} + +# GET /engine/v2/cancun/payloads/{id} +BuiltPayloadCancun { + payload: + block_value: + blobs_bundle: # NEW in Cancun (single proof) + should_override_builder: # NEW in Cancun +} + +# GET /engine/v2/prague/payloads/{id} +BuiltPayloadPrague { + payload: + block_value: + blobs_bundle: + execution_requests: # NEW in Prague, before should_override_builder + should_override_builder: +} + +# GET /engine/v2/osaka/payloads/{id} +BuiltPayloadOsaka { + payload: + block_value: + blobs_bundle: # CHANGED in Osaka (cell proofs) + execution_requests: + should_override_builder: +} + +# GET /engine/v2/amsterdam/payloads/{id} +BuiltPayloadAmsterdam { + payload: + block_value: + blobs_bundle: + execution_requests: + should_override_builder: +} +``` + +#### `POST /{fork}/forkchoice` request body (`ForkchoiceUpdate{Fork}`) + +The wrapper is the same Paris→Osaka (`custody_columns` is Amsterdam+); +only the inner `payload_attributes` shape changes per fork: + +``` +# Paris .. Osaka +ForkchoiceUpdate{Fork} { + forkchoice_state: # fork-invariant + payload_attributes: Optional[] +} + +# Amsterdam +ForkchoiceUpdateAmsterdam { + forkchoice_state: + payload_attributes: Optional[] # + slot_number, target_gas_limit + custody_columns: Optional[Bitvector[CELLS_PER_EXT_BLOB]] # NEW in Amsterdam +} +``` + +where the per-fork `payload_attributes` adds `withdrawals` at Shanghai, +`parent_beacon_block_root` at Cancun, and +`slot_number`/`target_gas_limit` at Amsterdam. The response +(`ForkchoiceUpdateResponse`) is fork-invariant. + +For the `/bodies` and `/blobs/vN` per-fork / per-revision bodies, see +the [container catalogue](./refactor-ssz.md#per-fork-container-catalogue); +`/bodies` varies the inner `ExecutionPayloadBody` (`+withdrawals` at +Shanghai, `+block_access_list` at Amsterdam) and `/blobs/vN` is +independently versioned rather than fork-scoped. + --- ## Error model @@ -584,12 +719,26 @@ Three layers: 2. **Per-fork body schema** — selected via the `{fork}` URL segment on hot-path endpoints (`/{fork}/payloads`, `/{fork}/forkchoice`, `/{fork}/bodies`). Tracks consensus-protocol changes that ride - along with fork activations. + along with fork activations. The named fork segments span + **Paris through Amsterdam** (`paris`, `shanghai`, `cancun`, + `prague`, `osaka`, `amsterdam`); Paris is the earliest fork with an + Engine API and therefore the lowest `{fork}` an EL accepts. A + request with a `{fork}` below the EL's earliest supported fork (or + one it doesn't recognise) returns + `400 /engine-api/errors/unsupported-fork`. 3. **Per-endpoint revisions** — selected via a `/vN` URL segment on endpoints whose protocol evolves independently of the fork schedule (currently just `/blobs/vN`). Tracks engine-API protocol changes that don't align with fork activations. +**Blob-parameter-only (BPO) forks** do **not** get their own `{fork}` +segment. A BPO fork only changes blob-count parameters, not any +Engine API body schema, so a chain in a BPO era keeps negotiating the +URL of the named fork it layers on — e.g. BPO1–BPO5 all use +`/osaka/...` and the Osaka wire shapes. Only named forks that change a +body schema introduce a new `{fork}` segment. CLs MUST map a BPO era +onto its base named fork when constructing the URL. + The server advertises which forks and which `/vN` revisions it understands via `GET /engine/v2/capabilities`. From d39e9a2710bf5f75eddee44fe8c2a6844bea23f0 Mon Sep 17 00:00:00 2001 From: Marius van der Wijden Date: Mon, 29 Jun 2026 14:25:37 +0200 Subject: [PATCH 9/9] engine: move fork into header out of path --- src/engine/refactor-ssz.md | 71 +++---- src/engine/refactor.md | 392 +++++++++++++++++++++---------------- 2 files changed, 264 insertions(+), 199 deletions(-) diff --git a/src/engine/refactor-ssz.md b/src/engine/refactor-ssz.md index dfb41dacc..d04a59622 100644 --- a/src/engine/refactor-ssz.md +++ b/src/engine/refactor-ssz.md @@ -38,10 +38,10 @@ - [`BlobAndProof` per revision](#blobandproof-per-revision) - [Identification & capabilities](#identification--capabilities) - [Endpoint containers](#endpoint-containers) - - [`POST /{fork}/payloads`](#post-forkpayloads) - - [`POST /{fork}/forkchoice`](#post-forkforkchoice) - - [`GET /{fork}/payloads/{payloadId}`](#get-forkpayloadspayloadid) - - [`POST /{fork}/bodies/hash` and `GET /{fork}/bodies?...`](#post-forkbodieshash-and-get-forkbodies) + - [`POST /payloads`](#post-payloads) + - [`POST /forkchoice`](#post-forkchoice) + - [`GET /payloads/{payloadId}`](#get-payloadspayloadid) + - [`POST /bodies/hash` and `GET /bodies?...`](#post-bodieshash-and-get-bodies) - [`POST /blobs/v1`](#post-blobsv1) - [`POST /blobs/v2`](#post-blobsv2) - [`POST /blobs/v3`](#post-blobsv3) @@ -276,16 +276,17 @@ source of the divergence. ## Per-fork container catalogue -Each fork URL (`/{fork}/payloads`, `/{fork}/forkchoice`, -`/{fork}/bodies`) uses its own SSZ container shape. ELs serving -`/cancun/...` MUST use the Cancun containers; ELs serving -`/amsterdam/...` MUST use the Amsterdam containers; etc. This section -catalogues every fork-scoped variant. +Each fork-scoped endpoint (`/payloads`, `/forkchoice`, `/bodies`) +uses its own SSZ container shape, selected by the +`Eth-Execution-Version` request header. ELs handling +`Eth-Execution-Version: cancun` MUST use the Cancun containers; ELs +handling `Eth-Execution-Version: amsterdam` MUST use the Amsterdam +containers; etc. This section catalogues every fork-scoped variant. ### `ExecutionPayload` per fork -Used by `POST /{fork}/payloads` (the inner `payload` field of -`ExecutionPayloadEnvelope`) and `GET /{fork}/payloads/{payloadId}` +Used by `POST /payloads` (the inner `payload` field of +`ExecutionPayloadEnvelope`) and `GET /payloads/{payloadId}` (the inner `payload` field of `BuiltPayload`). ``` @@ -341,7 +342,7 @@ above; this section just makes the progression explicit. ### `PayloadAttributes` per fork Used by the `payload_attributes` field of `ForkchoiceUpdate` (the -request body of `POST /{fork}/forkchoice`). +request body of `POST /forkchoice`). ``` # Paris @@ -434,7 +435,7 @@ Osaka / Amsterdam carries `BlobsBundleV2`. ### `BuiltPayload` per fork -Returned by `GET /{fork}/payloads/{payloadId}`. Like the +Returned by `GET /payloads/{payloadId}`. Like the `ExecutionPayload` catalogue above, the shape evolves per fork; this section pins every variant so there is no ambiguity for pre-Amsterdam forks. **All variants use the same field order; SSZ fields are @@ -524,7 +525,7 @@ The Amsterdam variant is the one shown in the ### `ExecutionPayloadEnvelope` per fork -The request body of `POST /{fork}/payloads`. `parent_beacon_block_root` +The request body of `POST /payloads`. `parent_beacon_block_root` exists from Cancun on (it was a separate `engine_newPayload` parameter since Cancun); `execution_requests` from Prague on. Field order is normative. @@ -557,7 +558,7 @@ ExecutionPayloadEnvelopePrague { ### `ForkchoiceUpdate` per fork -The request body of `POST /{fork}/forkchoice`. `payload_attributes` +The request body of `POST /forkchoice`. `payload_attributes` selects the matching per-fork `PayloadAttributes` shape; `custody_columns` exists only from Amsterdam on. @@ -630,13 +631,15 @@ CapabilitiesResponse { ## Endpoint containers The endpoint sketches below use the **Amsterdam** shapes as the worked -example. Every fork-scoped endpoint (`/{fork}/payloads`, -`/{fork}/forkchoice`, `/{fork}/bodies`) is defined for **every fork -from Paris onward**; substitute the matching entry from the +example. Every fork-scoped endpoint (`/payloads`, `/forkchoice`, +`/bodies`) is defined for **every fork from Paris +onward**; substitute the matching entry from the [per-fork container catalogue](#per-fork-container-catalogue) for the -URL's `{fork}`. For instance `POST /cancun/payloads` takes an +value of the `Eth-Execution-Version` request header. For instance +`POST /payloads` with `Eth-Execution-Version: cancun` takes an `ExecutionPayloadEnvelopeCancun` wrapping an `ExecutionPayloadCancun`, -and `GET /shanghai/payloads/{id}` returns a `BuiltPayloadShanghai`. +and `GET /payloads/{id}` with `Eth-Execution-Version: shanghai` +returns a `BuiltPayloadShanghai`. > **Fork-invariant containers.** `PayloadStatus`, `ForkchoiceState`, > `ForkchoiceUpdateResponse`, and `Withdrawal` have the **same shape @@ -651,13 +654,14 @@ and `GET /shanghai/payloads/{id}` returns a `BuiltPayloadShanghai`. > single *monolithic* superset container per type whose fields are > gated on the active fork (e.g. one `ExecutionPayload` struct where > `withdrawals` participates only from Shanghai, `block_access_list` -> only from Amsterdam, etc.), driving the gate from the URL `{fork}`. +> only from Amsterdam, etc.), driving the gate from the +> `Eth-Execution-Version` header. > This is a valid strategy **as long as the bytes on the wire are > identical** to the per-fork shape for that fork — i.e. a gated-off > field contributes neither an offset nor content. go-ethereum's > implementation takes this monolithic approach. -### `POST /{fork}/payloads` +### `POST /payloads` Replaces `engine_newPayloadV{1..5}` (Amsterdam shown; `engine_newPayloadV5`). Each fork uses its `ExecutionPayloadEnvelope{Fork}` from the catalogue @@ -681,7 +685,7 @@ from `payload.transactions`). `PayloadStatus` (full enum, `0`/`1`/`2`/`3`). -### `POST /{fork}/forkchoice` +### `POST /forkchoice` Replaces `engine_forkchoiceUpdatedV{1..4}` (Amsterdam shown; `engine_forkchoiceUpdatedV4`). Each fork uses its `ForkchoiceUpdate{Fork}` @@ -708,7 +712,7 @@ ForkchoiceUpdateResponse { } ``` -### `GET /{fork}/payloads/{payloadId}` +### `GET /payloads/{payloadId}` Replaces `engine_getPayloadV{1..6}` (Amsterdam shown; `engine_getPayloadV6`). Each fork returns its `BuiltPayload{Fork}` from @@ -736,12 +740,12 @@ BlobsBundleV2 { have length `len(blobs) * CELLS_PER_EXT_BLOB` (mirrors the `engine_getPayloadV5` rule from osaka.md). -### `POST /{fork}/bodies/hash` and `GET /{fork}/bodies?...` +### `POST /bodies/hash` and `GET /bodies?...` Replace `engine_getPayloadBodiesByHashV{1,2}` and `engine_getPayloadBodiesByRangeV{1,2}` (Amsterdam shown). Both return the same response container; the inner `ExecutionPayloadBody` follows -the URL `{fork}` per the catalogue. +the `Eth-Execution-Version` request header per the catalogue. #### Request — `/bodies/hash` @@ -769,7 +773,7 @@ BodyEntry { ``` `available` is `false` when the requested block is unavailable / -pruned, **or** when the block's timestamp falls outside the URL +pruned, **or** when the block's timestamp falls outside the header fork's active range. When `available=false`, `body` is zero-valued and CLs MUST ignore its contents. @@ -783,11 +787,12 @@ appears for in-range-but-out-of-era or pruned blocks; never as trailing padding. See [refactor.md § Historical bodies](./refactor.md#historical-bodies). -Each fork URL pairs with its own `ExecutionPayloadBody` schema. The -Amsterdam variant carries every field unconditionally: +Each `Eth-Execution-Version` value pairs with its own +`ExecutionPayloadBody` schema. The Amsterdam variant carries every +field unconditionally: ``` -# Amsterdam ExecutionPayloadBody (used by /amsterdam/bodies/...) +# Amsterdam ExecutionPayloadBody (Eth-Execution-Version: amsterdam) ExecutionPayloadBody { transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] @@ -799,19 +804,19 @@ Earlier-fork variants drop the fields their fork didn't have. For reference: ``` -# Cancun ExecutionPayloadBody (used by /cancun/bodies/...) +# Cancun ExecutionPayloadBody (Eth-Execution-Version: cancun) ExecutionPayloadBody { transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] withdrawals: List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD] } -# Paris ExecutionPayloadBody (used by /paris/bodies/...) +# Paris ExecutionPayloadBody (Eth-Execution-Version: paris) ExecutionPayloadBody { transactions: List[ByteList[MAX_BYTES_PER_TX], MAX_TXS_PER_PAYLOAD] } ``` -No `Optional[T]` cross-fork nullability anywhere — each fork URL +No `Optional[T]` cross-fork nullability anywhere — each fork returns only blocks from its own era, so every field is always present. diff --git a/src/engine/refactor.md b/src/engine/refactor.md index 3e432249b..ea7d243f3 100644 --- a/src/engine/refactor.md +++ b/src/engine/refactor.md @@ -1,10 +1,12 @@ # Engine API -- Refactor Proposal (REST + SSZ) -> **Status:** Draft / discussion document. This file proposes a v2 of the -> Engine API that moves from JSON-RPC over a single endpoint to a -> resource-oriented HTTP/REST API where request and response bodies are -> SSZ-encoded. It also takes the opportunity to simplify the surface that -> has accumulated since Paris. +> **Status:** Draft / discussion document. This file proposes a REST +> refactor of the Engine API that moves from JSON-RPC over a single +> endpoint to a resource-oriented HTTP/REST API where hot-path +> request and response bodies are SSZ-encoded. The new API ships at +> `/engine/v1/...` alongside the legacy `engine_*` JSON-RPC endpoint; +> the legacy endpoint is retired at a future fork, at which point +> the REST surface becomes the only way to drive an EL. > > **Target fork:** Amsterdam. The new API ships *as* the Amsterdam Engine > API; clients implement it instead of `engine_*` JSON-RPC at the @@ -37,6 +39,8 @@ - [Why SSZ?](#why-ssz) - [Simplifications & removed concepts](#simplifications--removed-concepts) - [Summary of design decisions](#summary-of-design-decisions) +- [Future evolution](#future-evolution) + - [Progressive merkleization](#progressive-merkleization) > **Reading order note.** The endpoint sketches reference SSZ types > like `Optional[T]`, `BodyEntry`, and `BlobEntry`. If a definition @@ -52,13 +56,16 @@ If you're migrating from the JSON-RPC engine API, this is the lookup table. Detail on each new endpoint follows in the sections below. +All hot-path endpoints select the fork via the +`Eth-Execution-Version: ` request header. + | Old method | New endpoint | Notes | | - | - | - | -| `engine_newPayloadV{1..5}` | `POST /{fork}/payloads` | `parentBeaconBlockRoot` and `executionRequests` folded into the SSZ envelope; `expectedBlobVersionedHashes` removed; `INVALID_BLOCK_HASH` removed from the status enum | -| `engine_forkchoiceUpdatedV{1..4}` | `POST /{fork}/forkchoice` | one atomic call; carries forkchoice state, optional `payload_attributes`, and (Amsterdam+) optional `custody_columns` | -| `engine_getPayloadV{1..6}` | `GET /{fork}/payloads/{id}` | poll-style, same semantics as today | -| `engine_getPayloadBodiesByHashV{1,2}` | `POST /{fork}/bodies/hash` | `{fork}` selects both the response schema and the era of returned blocks; `POST` because hash lists are too large for URLs | -| `engine_getPayloadBodiesByRangeV{1,2}` | `GET /{fork}/bodies?from=...&count=...` | `{fork}` selects both the response schema and the era of returned blocks | +| `engine_newPayloadV{1..5}` | `POST /payloads` | `parentBeaconBlockRoot` and `executionRequests` folded into the SSZ envelope; `expectedBlobVersionedHashes` removed; `INVALID_BLOCK_HASH` removed from the status enum | +| `engine_forkchoiceUpdatedV{1..4}` | `POST /forkchoice` | one atomic call; carries forkchoice state, optional `payload_attributes`, and (Amsterdam+) optional `custody_columns` | +| `engine_getPayloadV{1..6}` | `GET /payloads/{id}` | poll-style, same semantics as today | +| `engine_getPayloadBodiesByHashV{1,2}` | `POST /bodies/hash` | header selects both the response schema and the era of returned blocks; `POST` because hash lists are too large for URLs | +| `engine_getPayloadBodiesByRangeV{1,2}` | `GET /bodies?from=...&count=...` | header selects both the response schema and the era of returned blocks | | `engine_getBlobsV1` | `POST /blobs/v1` | independently versioned; legacy version numbers carry forward | | `engine_getBlobsV2` | `POST /blobs/v2` | all-or-nothing cell proofs | | `engine_getBlobsV3` | `POST /blobs/v3` | partial-response cell proofs | @@ -71,19 +78,20 @@ table. Detail on each new endpoint follows in the sections below. ## Resource model (overview) -Hot-path endpoints are scoped under `/engine/v2/{fork}/...`. Diagnostic -endpoints are unscoped. +All endpoints live under `/engine/v1/...`. Hot-path endpoints +require the `Eth-Execution-Version: ` request header; diagnostic +endpoints ignore it. | Resource | Endpoint | Purpose | | - | - | - | -| Payload | `POST /engine/v2/{fork}/payloads` | Submit a payload received from the CL gossip network for the EL to validate / import. Replaces `engine_newPayload`. | -| Payload | `GET /engine/v2/{fork}/payloads/{payloadId}` | Retrieve a built payload by id. Replaces `engine_getPayload`. CL polls when it wants a fresher snapshot. | -| Forkchoice | `POST /engine/v2/{fork}/forkchoice` | Atomic forkchoice update: update head/safe/finalized, optionally start a payload build, optionally update custody set. Replaces `engine_forkchoiceUpdated`. | -| Bodies | `POST /engine/v2/{fork}/bodies/hash` | Replaces `engine_getPayloadBodiesByHash`. `{fork}` selects both the response schema *and* the era of returned blocks; out-of-era blocks come back as `available=false`. | -| Bodies | `GET /engine/v2/{fork}/bodies?from=N&count=M` | Replaces `engine_getPayloadBodiesByRange`. Same fork scoping as `/bodies/hash`. | -| Blob pool | `POST /engine/v2/blobs/v{1..4}` | Replaces `engine_getBlobsV{1..4}`. The `vN` segment carries forward the legacy version numbers; `/v4` is the Amsterdam cell-range variant. | -| Capabilities | `GET /engine/v2/capabilities` | Replaces `engine_exchangeCapabilities`. Unscoped; advertises supported forks, `/blobs/vN` revisions, and per-endpoint request-size limits. | -| Identity | `GET /engine/v2/identity` | Replaces `engine_getClientVersion`. Unscoped. | +| Payload | `POST /engine/v1/payloads` | Submit a payload received from the CL gossip network for the EL to validate / import. Replaces `engine_newPayload`. Fork-scoped via `Eth-Execution-Version`. | +| Payload | `GET /engine/v1/payloads/{payloadId}` | Retrieve a built payload by id. Replaces `engine_getPayload`. Fork-scoped via `Eth-Execution-Version`. CL polls when it wants a fresher snapshot. | +| Forkchoice | `POST /engine/v1/forkchoice` | Atomic forkchoice update: update head/safe/finalized, optionally start a payload build, optionally update custody set. Replaces `engine_forkchoiceUpdated`. Fork-scoped via `Eth-Execution-Version`. | +| Bodies | `POST /engine/v1/bodies/hash` | Replaces `engine_getPayloadBodiesByHash`. `Eth-Execution-Version` selects both the response schema *and* the era of returned blocks; out-of-era blocks come back as `available=false`. | +| Bodies | `GET /engine/v1/bodies?from=N&count=M` | Replaces `engine_getPayloadBodiesByRange`. Same fork scoping as `/bodies/hash`. | +| Blob pool | `POST /engine/v1/blobs/v{1..4}` | Replaces `engine_getBlobsV{1..4}`. The `vN` segment carries forward the legacy version numbers; `/v4` is the Amsterdam cell-range variant. Independently versioned (not fork-scoped). | +| Capabilities | `GET /engine/v1/capabilities` | Replaces `engine_exchangeCapabilities`. Unscoped; advertises supported forks, `/blobs/vN` revisions, and per-endpoint request-size limits. | +| Identity | `GET /engine/v1/identity` | Replaces `engine_getClientVersion`. Unscoped. | Every hot-path body uses SSZ; every metadata endpoint uses JSON. @@ -93,7 +101,7 @@ Every hot-path body uses SSZ; every metadata endpoint uses JSON. ### Payload submission -#### `POST /engine/v2/{fork}/payloads` +#### `POST /engine/v1/payloads` Replaces `engine_newPayloadV{1..5}`. @@ -130,7 +138,7 @@ Replaces `engine_newPayloadV{1..5}`. ### Forkchoice update -#### `POST /engine/v2/{fork}/forkchoice` +#### `POST /engine/v1/forkchoice` Replaces `engine_forkchoiceUpdatedV{1..4}`. @@ -184,20 +192,20 @@ Replaces `engine_forkchoiceUpdatedV{1..4}`. finalization should not be able to roll the EL back. We keep the behaviour that has caught buggy CLs in the past. -- **Stale-fork URL:** an FCU at `/engine/v2/{fork}/forkchoice` +- **Stale-fork header:** an FCU with `Eth-Execution-Version: ` referencing a `head` from an earlier fork is **allowed**, *as long as `payload_attributes` is absent*. The CL needs to update head / safe / finalized across fork boundaries during sync and reorg - recovery, and the URL fork has no bearing on which historical + recovery, and the header fork has no bearing on which historical block can be referenced. TODO(MariusVanDerWijden) Is that really the case? - If `payload_attributes` is present, the URL `{fork}` MUST match - the fork that the new payload would belong to (i.e. the fork - determined by `payload_attributes.timestamp`). Mismatch returns - `400 unsupported-fork`. Building a payload is the only operation - where the URL fork is load-bearing on shape, so it's the only one - we strictly police. + If `payload_attributes` is present, the `Eth-Execution-Version` + header MUST match the fork that the new payload would belong to + (i.e. the fork determined by `payload_attributes.timestamp`). + Mismatch returns `400 unsupported-fork`. Building a payload is + the only operation where the header fork is load-bearing on shape, + so it's the only one we strictly police. - **Custody-set semantics** (Amsterdam+): the custody update runs independently of the forkchoice processing flow. An execution-time @@ -210,7 +218,7 @@ Replaces `engine_forkchoiceUpdatedV{1..4}`. ### Payload retrieval -#### `GET /engine/v2/{fork}/payloads/{payloadId}` +#### `GET /engine/v1/payloads/{payloadId}` Replaces `engine_getPayloadV{1..6}`. @@ -238,7 +246,7 @@ Replaces `engine_getPayloadV{1..6}`. - **404** if `payloadId` is unknown or expired. Polling semantics are unchanged from `engine_getPayload`: the CL calls -`GET /{fork}/payloads/{payloadId}` whenever it wants the latest +`GET /payloads/{payloadId}` whenever it wants the latest snapshot of the build. Each call returns the most recent version available at the time of receipt; the EL MAY stop the build process after serving a call. `payloadId` values are opaque server-assigned @@ -256,38 +264,38 @@ well-formed (8 bytes, hex) before dispatching to lookup logic; a malformed segment returns `400 invalid-request`. **Token TTL.** A `payloadId` is valid until either the payload was -retrieved by `GET /{fork}/payloads/{payloadId}` or another payload +retrieved by `GET /payloads/{payloadId}` or another payload was built via a forkchoice with payload attributes. ### Historical bodies These endpoints are **fork-scoped on both the response schema and the -era of the returned blocks.** The `{fork}` segment tells the EL which -`ExecutionPayloadBody` schema to use, *and* limits the response to -blocks whose timestamp falls in `{fork}`'s active time range. A CL -fetching bodies that span a fork boundary issues separate requests -against each fork URL. - -The EL **MUST** apply both meanings of `{fork}`: it **MUST** serialise -each entry against the `{fork}` schema **and MUST** filter the response -to blocks whose timestamp falls in `{fork}`'s active time range. -Using the segment only for schema selection — and returning blocks -from outside the fork's era — is **non-conformant**. A requested block -that exists but whose timestamp lies outside the URL fork's range -**MUST** come back as `available=false` (it is not omitted, unlike a -past-head block in a range query; see below). +era of the returned blocks.** The `Eth-Execution-Version` header +tells the EL which `ExecutionPayloadBody` schema to use, *and* limits +the response to blocks whose timestamp falls in that fork's active +time range. A CL fetching bodies that span a fork boundary issues +separate requests, one per header value. + +The EL **MUST** apply both meanings of the header value: it **MUST** +serialise each entry against the named schema **and MUST** filter the +response to blocks whose timestamp falls in that fork's active time +range. Using the header only for schema selection — and returning +blocks from outside the fork's era — is **non-conformant**. A +requested block that exists but whose timestamp lies outside the +header fork's range **MUST** come back as `available=false` (it is +not omitted, unlike a past-head block in a range query; see below). Concretely: -- `/cancun/bodies/hash` returns bodies *only* for blocks in the - Cancun time range. Requesting a Shanghai or Amsterdam hash yields - `available=false` for that entry. -- `/amsterdam/bodies/hash` returns bodies *only* for Amsterdam blocks. - All fields (including `block_access_list`) are unconditionally - present; older blocks the CL accidentally requested come back as - `available=false`. +- `POST /bodies/hash` with `Eth-Execution-Version: cancun` returns + bodies *only* for blocks in the Cancun time range. Requesting a + Shanghai or Amsterdam hash yields `available=false` for that entry. +- `POST /bodies/hash` with `Eth-Execution-Version: amsterdam` returns + bodies *only* for Amsterdam blocks. All fields (including + `block_access_list`) are unconditionally present; older blocks the + CL accidentally requested come back as `available=false`. -#### `POST /engine/v2/{fork}/bodies/hash` +#### `POST /engine/v1/bodies/hash` Replaces `engine_getPayloadBodiesByHashV{1,2}`. Uses `POST` so that large hash lists travel in the request body rather than the URL. @@ -299,24 +307,25 @@ large hash lists travel in the request body rather than the URL. the beacon-API convention; this costs a 4-byte offset but lets future revisions add fields without a breaking wire change. -#### `GET /engine/v2/{fork}/bodies?from=N&count=M` +#### `GET /engine/v1/bodies?from=N&count=M` Replaces `engine_getPayloadBodiesByRangeV{1,2}`. Range fits comfortably in the URL. Block numbers whose timestamp falls outside the URL fork's active range come back as `available=false`; block numbers past the latest known block are **omitted entirely** (the response is truncated at head, not padded — see the response-length note below). If the -requested range straddles a fork boundary the CL re-issues against the -next fork URL for the unfilled suffix. +requested range straddles a fork boundary the CL re-issues with a +different `Eth-Execution-Version` for the unfilled suffix. - **Response body** (both endpoints): SSZ-encoded `BodiesResponse` (a single-field container wrapping `entries: List[BodyEntry, MAX_BODIES_REQUEST]`). Each `BodyEntry` carries an `available: boolean` flag and an `ExecutionPayloadBody` - serialised against the **`{fork}` schema from the URL**. `available` - is false in either of the following cases: + serialised against the **schema named by `Eth-Execution-Version`**. + `available` is false in either of the following cases: - the block is unavailable / pruned, or - - the block's timestamp falls outside the URL fork's active range. + - the block's timestamp falls outside the header fork's active + range. When `available=false`, the `body` field is zero-valued and CLs MUST ignore its contents. See @@ -357,7 +366,7 @@ misses are reported via `BlobEntry.available = false` on revisions that support partial responses. Revision-specific contents live inside `BlobEntry.contents`. -#### `POST /engine/v2/blobs/v1` +#### `POST /engine/v1/blobs/v1` Replaces `engine_getBlobsV1` (Cancun, single-proof whole-blob). @@ -369,7 +378,7 @@ Replaces `engine_getBlobsV1` (Cancun, single-proof whole-blob). `available=false` per entry. `204 No Content` only when the EL cannot serve the request at all (e.g. syncing). -#### `POST /engine/v2/blobs/v2` +#### `POST /engine/v1/blobs/v2` Replaces `engine_getBlobsV2` (Osaka, all-or-nothing cell proofs). @@ -381,7 +390,7 @@ Replaces `engine_getBlobsV2` (Osaka, all-or-nothing cell proofs). returns `204 No Content`. Otherwise `200 OK` and all entries have `available=true`. This matches today's V2 semantics. -#### `POST /engine/v2/blobs/v3` +#### `POST /engine/v1/blobs/v3` Replaces `engine_getBlobsV3` (Osaka, partial responses with cell proofs). @@ -392,7 +401,7 @@ proofs). the whole response. `204 No Content` only when the EL cannot serve the request at all. -#### `POST /engine/v2/blobs/v4` +#### `POST /engine/v1/blobs/v4` Replaces `engine_getBlobsV4` (Amsterdam, cell-range selection). @@ -409,7 +418,7 @@ Replaces `engine_getBlobsV4` (Amsterdam, cell-range selection). ### Capabilities & identification -#### `GET /engine/v2/capabilities` +#### `GET /engine/v1/capabilities` Returns JSON. The advertisement includes per-endpoint maximum request sizes so the CL knows how many block-bodies / blob-cells / payloads @@ -445,7 +454,7 @@ carries an explicit `/vN` revision. ELs MAY support multiple revisions concurrently (e.g. `["v1", "v2"]`); CLs pick whichever they implement. -#### `GET /engine/v2/identity` +#### `GET /engine/v1/identity` Returns JSON `ClientVersion[]` (same shape as today's `engine_getClientVersionV1`). The CL identifies itself with a @@ -455,8 +464,9 @@ mutual-exchange handshake. ### Example: submit a payload ```bash -curl -X POST http://localhost:8551/engine/v2/amsterdam/payloads \ +curl -X POST http://localhost:8551/engine/v1/payloads \ -H "Authorization: Bearer $JWT_TOKEN" \ + -H "Eth-Execution-Version: amsterdam" \ -H "Content-Type: application/octet-stream" \ -H "Accept: application/octet-stream" \ -H "X-Engine-Client-Version: LH/v6.2.1" \ @@ -467,9 +477,10 @@ curl -X POST http://localhost:8551/engine/v2/amsterdam/payloads \ Request: ``` -POST /engine/v2/amsterdam/payloads HTTP/2 +POST /engine/v1/payloads HTTP/2 Host: localhost:8551 Authorization: Bearer +Eth-Execution-Version: amsterdam Content-Type: application/octet-stream Content-Length: 584 @@ -486,7 +497,7 @@ Content-Length: 41 <41 bytes: SSZ(PayloadStatus)> ``` -The 41 bytes break down as: `status` (1 byte = `0x00`, `VALID`) + +The 41 bytes break down as: `status` (1 byte = `0x01`, `VALID`) + `latest_valid_hash` (4-byte offset + 32-byte hash = 36 bytes) + `validation_error` (4-byte offset + 0 bytes empty list). @@ -503,8 +514,9 @@ Content-Length: 49 ### Example: poll a built payload ```bash -curl http://localhost:8551/engine/v2/amsterdam/payloads/0x1234567890abcdef \ +curl http://localhost:8551/engine/v1/payloads/0x1234567890abcdef \ -H "Authorization: Bearer $JWT_TOKEN" \ + -H "Eth-Execution-Version: amsterdam" \ -H "Accept: application/octet-stream" \ -o built_payload.ssz ``` @@ -515,46 +527,47 @@ cache. See [Payload retrieval](#payload-retrieval). ### Examples: every fork The examples above use Amsterdam. The shapes below show how the two -main hot-path bodies change fork-to-fork, so each fork's URL has a -worked example. Field names follow the +main hot-path bodies change fork-to-fork. The URL is the same for +every fork — only the `Eth-Execution-Version` header value changes +along with the body schema. Field names follow the [per-fork container catalogue](./refactor-ssz.md#per-fork-container-catalogue); `<…>` denotes nested SSZ. Only the fields that change per fork are -called out; every fork from Paris on is a valid `{fork}` segment. +called out; every fork from Paris on is a valid header value. -#### `POST /{fork}/payloads` request body (`ExecutionPayloadEnvelope{Fork}`) +#### `POST /payloads` request body (`ExecutionPayloadEnvelope{Fork}`) ``` -# POST /engine/v2/paris/payloads +# Eth-Execution-Version: paris ExecutionPayloadEnvelopeParis { payload: # base fields only (no withdrawals/blob-gas/BAL) } -# POST /engine/v2/shanghai/payloads +# Eth-Execution-Version: shanghai ExecutionPayloadEnvelopeShanghai { payload: # + withdrawals } -# POST /engine/v2/cancun/payloads +# Eth-Execution-Version: cancun ExecutionPayloadEnvelopeCancun { payload: # + blob_gas_used, excess_blob_gas parent_beacon_block_root: # NEW: was a side param since Cancun } -# POST /engine/v2/prague/payloads +# Eth-Execution-Version: prague ExecutionPayloadEnvelopePrague { payload: # == Cancun payload shape parent_beacon_block_root: execution_requests: # NEW: was a side param since Prague } -# POST /engine/v2/osaka/payloads — identical envelope shape to Prague +# Eth-Execution-Version: osaka — identical envelope shape to Prague ExecutionPayloadEnvelopeOsaka { payload: # == Cancun payload shape parent_beacon_block_root: execution_requests: } -# POST /engine/v2/amsterdam/payloads +# Eth-Execution-Version: amsterdam ExecutionPayloadEnvelopeAmsterdam { payload: # + block_access_list, slot_number parent_beacon_block_root: @@ -564,22 +577,22 @@ ExecutionPayloadEnvelopeAmsterdam { The response for **all** forks is `PayloadStatus` (fork-invariant). -#### `GET /{fork}/payloads/{id}` response body (`BuiltPayload{Fork}`) +#### `GET /payloads/{id}` response body (`BuiltPayload{Fork}`) ``` -# GET /engine/v2/paris/payloads/{id} +# Eth-Execution-Version: paris BuiltPayloadParis { payload: block_value: } -# GET /engine/v2/shanghai/payloads/{id} +# Eth-Execution-Version: shanghai BuiltPayloadShanghai { payload: block_value: } -# GET /engine/v2/cancun/payloads/{id} +# Eth-Execution-Version: cancun BuiltPayloadCancun { payload: block_value: @@ -587,7 +600,7 @@ BuiltPayloadCancun { should_override_builder: # NEW in Cancun } -# GET /engine/v2/prague/payloads/{id} +# Eth-Execution-Version: prague BuiltPayloadPrague { payload: block_value: @@ -596,7 +609,7 @@ BuiltPayloadPrague { should_override_builder: } -# GET /engine/v2/osaka/payloads/{id} +# Eth-Execution-Version: osaka BuiltPayloadOsaka { payload: block_value: @@ -605,7 +618,7 @@ BuiltPayloadOsaka { should_override_builder: } -# GET /engine/v2/amsterdam/payloads/{id} +# Eth-Execution-Version: amsterdam BuiltPayloadAmsterdam { payload: block_value: @@ -615,7 +628,7 @@ BuiltPayloadAmsterdam { } ``` -#### `POST /{fork}/forkchoice` request body (`ForkchoiceUpdate{Fork}`) +#### `POST /forkchoice` request body (`ForkchoiceUpdate{Fork}`) The wrapper is the same Paris→Osaka (`custody_columns` is Amsterdam+); only the inner `payload_attributes` shape changes per fork: @@ -674,7 +687,7 @@ Error codes: | 400 Bad Request | `/engine-api/errors/parse-error` | -32700 | Body is not valid JSON / SSZ | | 400 Bad Request | `/engine-api/errors/invalid-request` | -32600 | Request shape is wrong (missing required field, etc.) | | 400 Bad Request | `/engine-api/errors/ssz-decode-error` | (new) | SSZ decode failed; canned error, no `detail` | -| 400 Bad Request | `/engine-api/errors/unsupported-fork` | -38005 | URL `{fork}` is not supported by this EL | +| 400 Bad Request | `/engine-api/errors/unsupported-fork` | -38005 | `Eth-Execution-Version` value missing, unknown, or unsupported by this EL | | 404 Not Found | `/engine-api/errors/method-not-found` | -32601 | URL does not match any endpoint | | 404 Not Found | `/engine-api/errors/unknown-payload` | -38001 | `payloadId` does not exist | | 409 Conflict | `/engine-api/errors/invalid-forkchoice` | -38002 | Forkchoice state is inconsistent (e.g. finalized not ancestor of head) | @@ -714,42 +727,46 @@ format, and authentication problems. Three layers: -1. **Major (`/v2`)** — bumped only for breaking transport changes - (e.g. moving away from REST, swapping SSZ for something else). -2. **Per-fork body schema** — selected via the `{fork}` URL segment - on hot-path endpoints (`/{fork}/payloads`, `/{fork}/forkchoice`, - `/{fork}/bodies`). Tracks consensus-protocol changes that ride - along with fork activations. The named fork segments span - **Paris through Amsterdam** (`paris`, `shanghai`, `cancun`, - `prague`, `osaka`, `amsterdam`); Paris is the earliest fork with an - Engine API and therefore the lowest `{fork}` an EL accepts. A - request with a `{fork}` below the EL's earliest supported fork (or - one it doesn't recognise) returns +1. **Major (`/v1`, future `/v2`)** — `/v1` is the first REST version + of the engine API. A future `/v2` is reserved for breaking + transport changes (e.g. moving away from REST, swapping SSZ for + something else). +2. **Per-fork body schema** — selected via the + `Eth-Execution-Version: ` request header on hot-path + endpoints (`/payloads`, `/forkchoice`, `/bodies`). Tracks + consensus-protocol changes that ride along with fork activations. + Accepted values span **Paris through Amsterdam** (`paris`, + `shanghai`, `cancun`, `prague`, `osaka`, `amsterdam`); Paris is + the earliest fork with an Engine API and therefore the lowest + value an EL accepts. A request with a header value below the EL's + earliest supported fork, one it doesn't recognise, or a missing + header on a fork-scoped endpoint returns `400 /engine-api/errors/unsupported-fork`. 3. **Per-endpoint revisions** — selected via a `/vN` URL segment on endpoints whose protocol evolves independently of the fork schedule (currently just `/blobs/vN`). Tracks engine-API protocol changes that don't align with fork activations. -**Blob-parameter-only (BPO) forks** do **not** get their own `{fork}` -segment. A BPO fork only changes blob-count parameters, not any -Engine API body schema, so a chain in a BPO era keeps negotiating the -URL of the named fork it layers on — e.g. BPO1–BPO5 all use -`/osaka/...` and the Osaka wire shapes. Only named forks that change a -body schema introduce a new `{fork}` segment. CLs MUST map a BPO era -onto its base named fork when constructing the URL. +**Blob-parameter-only (BPO) forks** do **not** get their own +`Eth-Execution-Version` value. A BPO fork only changes blob-count +parameters, not any Engine API body schema, so a chain in a BPO era +keeps negotiating against the named fork it layers on — e.g. +BPO1–BPO5 all send `Eth-Execution-Version: osaka` and use the Osaka +wire shapes. Only named forks that change a body schema introduce a +new accepted header value. CLs MUST map a BPO era onto its base named +fork when constructing the header. The server advertises which forks and which `/vN` revisions it -understands via `GET /engine/v2/capabilities`. +understands via `GET /engine/v1/capabilities`. `engine_exchangeCapabilities` is **removed**. Instead the server lists its supported fork schemas and endpoint set in a single JSON document -at `/engine/v2/capabilities`. +at `/engine/v1/capabilities`. ### Capabilities format We considered advertising capabilities as a flat list of per-endpoint -strings (e.g. `"POST /amsterdam/payloads"`, the format used by the +strings (e.g. `"POST /payloads@amsterdam"`, the format used by the existing `engine_exchangeCapabilities` method). The structured form in `GET /capabilities` (separate `supported_forks`, `fork_scoped_endpoints`, `independently_versioned`, @@ -766,25 +783,26 @@ because: ### Transition-window behavior -During the rollout window, a CL upgraded to v2 may interact with an -EL still on the legacy JSON-RPC engine API. Two cases: +During the rollout window, a CL upgraded to the REST API may +interact with an EL still on the legacy JSON-RPC engine API. Two +cases: -- **EL doesn't expose `/engine/v2/...` at all.** The CL hits any v2 - URL and gets `404 Not Found` from the legacy server. The CL falls - back to JSON-RPC for the duration of that EL's lifetime — no +- **EL doesn't expose `/engine/v1/...` at all.** The CL hits any + REST URL and gets `404 Not Found` from the legacy server. The CL + falls back to JSON-RPC for the duration of that EL's lifetime — no per-method retry dance. -- **EL exposes `/engine/v2/...` but doesn't know the URL fork.** The - CL hits `/{fork}/...` against an EL that only advertised - `supported_forks: [..., cancun]` while the CL is asking for - `amsterdam`. The EL returns +- **EL exposes `/engine/v1/...` but doesn't know the requested fork.** + The CL hits a fork-scoped endpoint with + `Eth-Execution-Version: amsterdam` against an EL that only + advertised `supported_forks: [..., cancun]`. The EL returns `400 /engine-api/errors/unsupported-fork`. The CL learns this once from `GET /capabilities` and avoids issuing such requests; if it doesn't, the per-request error is structured and explicit, not a silent downgrade. -There is **no per-method fallback ladder**. A CL either uses v2 or -JSON-RPC for the lifetime of an EL connection; mixing transports -within a connection is permitted but not required. +There is **no per-method fallback ladder**. A CL either uses the +REST API or JSON-RPC for the lifetime of an EL connection; mixing +transports within a connection is permitted but not required. --- @@ -831,12 +849,20 @@ Unchanged in spirit: JWT (HS256, 256-bit shared secret). Differences: - **Default port:** `8551`, shared with the legacy JSON-RPC engine API. The two surfaces are distinguished by path: legacy JSON-RPC remains at `/` (and accepts JSON-RPC method calls), the new API lives under - `/engine/v2/...`. The same JWT secret authenticates both. -- **Base path:** `/engine/v2/{fork}/...`. The `/v2` segment is the - major-protocol version; the `{fork}` segment selects the fork-scoped - body schema (`paris`, `shanghai`, `cancun`, `prague`, `osaka`, - `amsterdam`, …). Adding a fork = adding one path prefix and one set - of SSZ schemas. See [Versioning](#versioning-model). + `/engine/v1/...`. The same JWT secret authenticates both. +- **Base path:** `/engine/v1/...`. `/v1` is the first major REST + version (a future `/v2` is reserved for breaking transport + changes). The fork-scoped body schema is selected by the + `Eth-Execution-Version: ` request header rather than a URL + segment (`paris`, `shanghai`, `cancun`, `prague`, `osaka`, + `amsterdam`, …). Adding a fork = adding one accepted header value + and one set of SSZ schemas. See [Versioning](#versioning-model). +- **Fork header:** every hot-path request MUST carry + `Eth-Execution-Version: `. Missing or unknown header on a + fork-scoped endpoint returns + `400 /engine-api/errors/unsupported-fork`. Unscoped endpoints + (`/capabilities`, `/identity`, `/blobs/vN`) MUST ignore the header + if present. - **Content-Type / Accept matrix:** | Channel | Header | Value | @@ -863,23 +889,29 @@ Unchanged in spirit: JWT (HS256, 256-bit shared secret). Differences: request or reuse a long-lived connection. JWT is per-request so token rotation works the same way in both patterns. -### Why fork-in-URL instead of method versioning? +### Why a fork header instead of method versioning? Today every change of a single field bumps the method version -(`engine_newPayloadV1..V5`). The new API puts the fork in the URL: +(`engine_newPayloadV1..V5`). The new API puts the fork in a request +header: ``` -POST /engine/v2/amsterdam/payloads +POST /engine/v1/payloads +Eth-Execution-Version: amsterdam Content-Type: application/octet-stream Authorization: Bearer ``` -The EL routes by fork segment, parses the body according to that fork's -SSZ schema, and returns a fork-shaped response. Adding a fork = adding -one path prefix and one set of SSZ schemas. URLs stay greppable and -discoverable in logs. +The EL routes by header value, parses the body according to that +fork's SSZ schema, and returns a fork-shaped response. Adding a fork += adding one accepted header value and one set of SSZ schemas. We +considered putting the fork in the URL (`/{fork}/payloads`) but +chose the header because it keeps URLs stable across forks — content +negotiation is the standard HTTP idiom for "same resource, different +schema version" and the Beacon API already uses +`Eth-Consensus-Version` for the same purpose on the CL side. --- @@ -931,16 +963,16 @@ base types map onto SSZ as follows: Endpoints that return data spanning multiple block-eras come in two flavours: -1. **Fork-scoped** (e.g. `/bodies`): the URL `{fork}` selects the - container schema *and* limits the response to blocks from that - fork's time range. Every field in the fork's body container is - unconditionally present (no `Optional[T]` for cross-fork +1. **Fork-scoped** (e.g. `/bodies`): `Eth-Execution-Version` selects + the container schema *and* limits the response to blocks from + that fork's time range. Every field in the fork's body container + is unconditionally present (no `Optional[T]` for cross-fork nullability); blocks outside the fork's range come back as `available=false` on the outer entry instead of as a zero-valued body: ``` - # /amsterdam/bodies/hash response + # POST /bodies/hash with Eth-Execution-Version: amsterdam BodyEntry { available: boolean body: ExecutionPayloadBody @@ -954,10 +986,11 @@ flavours: } ``` - A CL fetching a Cancun-era block calls `/cancun/bodies/hash` and - receives the Cancun container (no `block_access_list` field at - all, and no `Optional` wrapper on `withdrawals`). Cross-fork - ranges require multiple requests, one per fork URL. + A CL fetching a Cancun-era block sends + `Eth-Execution-Version: cancun` and receives the Cancun container + (no `block_access_list` field at all, and no `Optional` wrapper on + `withdrawals`). Cross-fork ranges require multiple requests, one + per header value. 2. **Independently versioned** (e.g. `/blobs/vN`): each revision is its own container, no nullable optionals across revisions. Old @@ -1037,7 +1070,7 @@ decision log for quick scanning. 2. **Stop the version sprawl.** Today every fork bumps every method that touches a changed structure (`engine_newPayload` is at V5, `engine_getPayload` at V6, etc.). The new API puts the fork in the - URL (`/engine/v2/{fork}/...`) so a single endpoint accepts whatever + URL (`/engine/v1/...`) so a single endpoint accepts whatever schema that fork mandates; adding a fork = adding one path prefix plus one set of SSZ schemas, not bumping every method name. 3. **Self-contained requests.** No more side-channel parameters @@ -1072,9 +1105,9 @@ that prompt this refactor: routinely multi-megabyte. - **No content negotiation.** A new fork structure forces a new method name (`engine_newPayloadV5`), even when the only change is one added - field. With a REST endpoint, the fork is part of the URL - (`/engine/v2/amsterdam/payloads`) and the body schema is selected by - routing, not by method-name suffix. + field. With a REST endpoint and a fork header + (`Eth-Execution-Version: amsterdam`), the body schema is selected + by routing + content negotiation, not by method-name suffix. - **Side-channel params.** JSON-RPC's positional params encourage bolting on extras like `parentBeaconBlockRoot` and `executionRequests` next to the payload, instead of inside it. @@ -1168,7 +1201,7 @@ the summary exists for quick scanning. - **`eth_*` JSON-RPC subset** (`eth_blockNumber`, `eth_call`, `eth_chainId`, `eth_getCode`, `eth_getBlockByHash`, `eth_getBlockByNumber`, `eth_getLogs`, `eth_sendRawTransaction`, - `eth_syncing`) is **not** mirrored under `/engine/v2/...`. CLs that + `eth_syncing`) is **not** mirrored under `/engine/v1/...`. CLs that need state / log access continue to call them via the legacy JSON-RPC root. @@ -1194,9 +1227,8 @@ the summary exists for quick scanning. #### Versioning -- **Fork-scoped endpoints:** `/{fork}/payloads`, `/{fork}/forkchoice`, - `/{fork}/bodies`. Fork in the URL, no `Eth-Consensus-Version` - header. +- **Fork-scoped endpoints:** `/payloads`, `/forkchoice`, `/bodies`. + Fork in the `Eth-Execution-Version` request header. - **Independently versioned endpoints:** `/blobs/vN`. Legacy `engine_getBlobsVN` numbers carry forward onto the URL. ELs MUST serve at least the revision matching their current fork @@ -1204,13 +1236,18 @@ the summary exists for quick scanning. alongside. Future blob-shape changes ship as `/blobs/v5`, `/v6`, etc. - **Unscoped endpoints:** `/capabilities`, `/identity`. -- **Major version `/v2`** is bumped only for breaking transport - changes (e.g. dropping REST or SSZ). +- **Major version `/v1`** is the first REST version; future `/v2` + reserved for breaking transport changes (e.g. dropping REST or + SSZ). #### Encoding - **Hot-path bodies use SSZ.** Diagnostic / metadata endpoints - (`/capabilities`, `/identity`, error bodies) use JSON. + (`/capabilities`, `/identity`, error bodies) use JSON. The legacy + `engine_*` JSON-RPC endpoint at `/` remains available alongside + this surface during the rollout window and is retired at a future + fork `F` (TODO: pin), at which point the REST + SSZ surface + becomes the only way to drive an EL. - **`Optional[T]` ≡ `List[T, 1]`** (length 0 = absent, length 1 = present). Universally supported by SSZ libraries. - **Strings ≡ `List[byte, MAX_ERROR_BYTES]`**, `MAX_ERROR_BYTES = 1024`. @@ -1222,9 +1259,9 @@ the summary exists for quick scanning. - **`MAX_*` constants** are defined in fork-scoped SSZ schema files; `MAX_ERROR_BYTES` is global. - **Cross-fork response containers** come in two flavours: - fork-scoped (`/bodies`) uses the URL `{fork}` to pick *both* the - schema and the era of returned blocks (every body field always - present; out-of-era blocks come back as `available=false`); + fork-scoped (`/bodies`) uses `Eth-Execution-Version` to pick + *both* the schema and the era of returned blocks (every body field + always present; out-of-era blocks come back as `available=false`); independently versioned (`/blobs/vN`) gives each revision its own dedicated container. Both wrap their entries in `BodyEntry { available, body }` / `BlobEntry { available, contents }`. @@ -1250,23 +1287,24 @@ the summary exists for quick scanning. `ACCEPTED → VALID/INVALID`) are allowed; ELs MUST NOT short-circuit retries. -#### Forkchoice update (`POST /{fork}/forkchoice`) +#### Forkchoice update (`POST /forkchoice`) - **Single atomic call** carrying forkchoice state, optional `payload_attributes`, and optional `custody_columns`. - **Skip-allowed semantics:** EL MAY skip applying state when the new `head` is a `VALID` ancestor of the latest finalized block, guarding against malformed CL FCUs. -- **Stale-fork URL** is allowed when `payload_attributes` is absent; - with `payload_attributes` present, URL `{fork}` MUST match the - timestamp's fork (otherwise `400 unsupported-fork`). +- **Stale-fork header** is allowed when `payload_attributes` is + absent; with `payload_attributes` present, + `Eth-Execution-Version` MUST match the timestamp's fork + (otherwise `400 unsupported-fork`). - **No HTTP-layer body cap** beyond SSZ `MAX_*` constants. - **Custody-set updates** run independently of the forkchoice flow; custody errors do not affect `payload_status`. - **Custody-set lifetime:** set until the next FCU that includes a `custody_columns` field. FCUs that omit it leave the set unchanged. -#### Payload submission (`POST /{fork}/payloads`) +#### Payload submission (`POST /payloads`) - **`expectedBlobVersionedHashes` removed.** EL recomputes from `payload.transactions`; block-hash check covers transactions. @@ -1275,7 +1313,7 @@ the summary exists for quick scanning. - **Transaction min-length** ("at least 1 byte") remains a receiver-side validation rule, not an SSZ schema invariant. -#### Payload retrieval (`GET /{fork}/payloads/{payloadId}`) +#### Payload retrieval (`GET /payloads/{payloadId}`) - **Poll-only**, same semantics as today's `engine_getPayload`. No SSE / long-poll. @@ -1283,7 +1321,7 @@ the summary exists for quick scanning. `POST /forkchoice`. CLs MUST NOT recompute or validate it. - **`payload_id` lifetime is build-bound, not time-bound.** A token remains valid until either the payload was retrieved by - `GET /{fork}/payloads/{payloadId}` or another payload was built + `GET /payloads/{payloadId}` or another payload was built via a forkchoice with payload attributes. - **`shouldOverrideBuilder`** lives inside the SSZ `BuiltPayload` body. @@ -1317,3 +1355,25 @@ the summary exists for quick scanning. - The mutual-exchange handshake of `engine_getClientVersionV1` — replaced by one-way `GET /identity` plus the `X-Engine-Client-Version` request header. + +--- + +## Future evolution + +### Progressive merkleization + +Progressive (chunked / streaming) SSZ merkleization is **not used** +in this draft. It would let the EL stream a partially-built +`BuiltPayload` without recomputing the `hash_tree_root` from +scratch on every `GET` poll, which is attractive for the polling +loop on `/payloads/{id}` — but the spec depends on container +shapes that are still in flux pending +[EIP-7688](https://eips.ethereum.org/EIPS/eip-7688). Adopting +progressive merkleization now would freeze container layout +choices we may want to revisit once 7688 is SFI-ed. + +Plan: revisit progressive merkleization in a follow-up revision of +this spec once EIP-7688 is SFI-ed. At that point we can redesign +the affected containers (`BuiltPayload`, `BlobsBundle`) to be +progressive-friendly without inheriting any constraints from the +current shapes.