From f59dce45fc1fa2d1a2d2eb285e665895a7a8b2d7 Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Tue, 3 Mar 2026 23:17:24 +0100 Subject: [PATCH 01/12] feat: add ssz to engine api --- src/engine/common.md | 9 ++ src/engine/ssz-encoding.md | 207 +++++++++++++++++++++++++++++++++++++ 2 files changed, 216 insertions(+) create mode 100644 src/engine/ssz-encoding.md diff --git a/src/engine/common.md b/src/engine/common.md index b76d03bf9..dd0b24a3d 100644 --- a/src/engine/common.md +++ b/src/engine/common.md @@ -15,6 +15,7 @@ This document specifies common definitions and requirements affecting Engine API - [Errors](#errors) - [Timeouts](#timeouts) - [Encoding](#encoding) + - [SSZ encoding](#ssz-encoding) - [Capabilities](#capabilities) - [engine_exchangeCapabilities](#engine_exchangecapabilities) - [Request](#request) @@ -136,6 +137,14 @@ Values of a field of `QUANTITY` type **MUST** be encoded as a hexadecimal string *Note:* Byte order of encoded value having `QUANTITY` type is big-endian. +### SSZ encoding + +Clients **MAY** optionally support SSZ encoding for Engine API payloads as an alternative to JSON encoding. SSZ encoding is not tied to any specific hard fork and can be implemented independently by any client team at any time. + +When both the consensus layer and execution layer clients support SSZ encoding, they **SHOULD** use it for reduced serialization overhead. When either client does not support SSZ, both **MUST** fall back to JSON encoding. + +SSZ encoding support is negotiated via standard HTTP content-type headers. The full specification is defined in the [SSZ Encoding](./ssz-encoding.md) document. + [json-rpc-spec]: https://playground.open-rpc.org/?schemaUrl=https://raw.githubusercontent.com/ethereum/execution-apis/assembled-spec/openrpc.json&uiSchema[appBar][ui:splitView]=false&uiSchema[appBar][ui:input]=false&uiSchema[appBar][ui:examplesDropdown]=false ## Capabilities diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md new file mode 100644 index 000000000..31cf8eb3b --- /dev/null +++ b/src/engine/ssz-encoding.md @@ -0,0 +1,207 @@ +# Engine API -- SSZ Encoding + +This document specifies an optional SSZ encoding for Engine API payloads as an alternative to the default JSON encoding. SSZ encoding reduces serialization overhead and aligns the Engine API with the native encoding format used by the consensus layer. + +## Table of contents + + + + +- [Motivation](#motivation) +- [Encoding negotiation](#encoding-negotiation) +- [SSZ type mappings](#ssz-type-mappings) +- [Request and response format](#request-and-response-format) +- [Example](#example) +- [Error handling](#error-handling) +- [Security considerations](#security-considerations) + + + +## Motivation + +The current JSON-RPC encoding introduces serialization overhead that grows with payload size. Binary data (hashes, addresses, bytecode) must be hex-encoded, doubling their size. As Ethereum scales through increased gas limits and blob transactions, this overhead becomes a bottleneck for block propagation and validation timing. + +The consensus layer already uses SSZ for all internal data structures and network communication. The current architecture requires converting between SSZ and JSON at the Engine API boundary in both directions. SSZ encoding for the Engine API eliminates this double conversion, reduces payload sizes by 40-60%, and provides deterministic encoding. + +## Encoding negotiation + +SSZ encoding support is negotiated via standard HTTP content negotiation headers. No additional capability exchange is required. + +| Header | Value | Meaning | +| - | - | - | +| `Content-Type` | `application/ssz` | The request body is SSZ-encoded | +| `Content-Type` | `application/json` | The request body is JSON-encoded (default) | +| `Accept` | `application/ssz` | The client prefers an SSZ-encoded response | +| `Accept` | `application/json` | The client prefers a JSON-encoded response (default) | + +The negotiation works as follows: + +1. The consensus layer client sends a request with `Accept: application/ssz` to indicate it can handle SSZ-encoded responses. + +2. If the execution layer client supports SSZ encoding, it **SHOULD** respond with `Content-Type: application/ssz` and an SSZ-encoded body. + +3. If the execution layer client does not support SSZ encoding, it **MUST** respond with `Content-Type: application/json` and a JSON-encoded body as usual. The `Accept` header is silently ignored. + +4. A client receiving a request with `Content-Type: application/ssz` that does not support SSZ encoding **MUST** respond with HTTP status `415 Unsupported Media Type`. The requesting client **MUST** then fall back to JSON encoding for subsequent requests. + +5. Clients **MUST** continue to support JSON encoding regardless of SSZ support. SSZ encoding is an optimization, not a replacement. + +6. If no `Content-Type` header is present, the request **MUST** be parsed as JSON. If no `Accept` header is present, the response **SHOULD** use the same encoding as the request. + +## SSZ type mappings + +Each JSON-encoded base type used in the Engine API maps to a specific SSZ type. The mappings below correspond to the types defined in the [base types schema](../schemas/base-types.yaml). + +### Fixed-size types + +| JSON Type | Size | SSZ Type | +| - | - | - | +| `address` | 20 bytes | `Bytes20` | +| `hash32` | 32 bytes | `Bytes32` | +| `bytes8` | 8 bytes | `Bytes8` | +| `bytes32` | 32 bytes | `Bytes32` | +| `bytes48` | 48 bytes | `Bytes48` | +| `bytes65` | 65 bytes | `Bytes65` | +| `bytes96` | 96 bytes | `Bytes96` | +| `bytes256` | 256 bytes | `ByteVector[256]` | +| `uint64` | 8 bytes | `uint64` | +| `uint256` | 32 bytes | `uint256` | +| `BOOLEAN` | 1 byte | `boolean` | + +### Variable-size types + +| JSON Type | SSZ Type | +| - | - | +| `bytes` (variable-length hex data) | `ByteList[MAX_LENGTH]` where `MAX_LENGTH` is context-dependent | +| `bytesMax32` (up to 32 bytes hex data) | `ByteList[32]` | + +### Composite types + +| JSON Type | SSZ Type | +| - | - | +| `Array of T` | `List[T, MAX_LENGTH]` where `MAX_LENGTH` is context-dependent | +| Object (e.g. `ExecutionPayloadV1`) | `Container` with fields mapped per this table | + +### Nullable fields + +Fields that may be `null` in the JSON encoding (e.g. `latestValidHash` in `PayloadStatusV1`, `withdrawals` in `ExecutionPayloadBodyV1`) are represented using `Optional[T]` in SSZ. + +## Request and response format + +SSZ-encoded Engine API requests and responses follow the existing JSON-RPC method semantics. The SSZ encoding applies to the method parameters and result values — the JSON-RPC envelope (`jsonrpc`, `id`, `method`) remains JSON-encoded. + +Specifically, when SSZ encoding is in use: + +1. The HTTP request body is a JSON-RPC request where each element of `params` is replaced with its SSZ-encoded hexadecimal representation (a `DATA` string). + +2. The HTTP response body is a JSON-RPC response where the `result` field is replaced with the SSZ-encoded hexadecimal representation of the result value. + +This approach preserves compatibility with JSON-RPC tooling while encoding the payload data in SSZ. + +*Note:* Future versions of this specification may define a fully binary request/response format that replaces the JSON-RPC envelope. + +## Example + +The following example shows an `engine_newPayloadV1` call with a minimal payload, first using JSON encoding and then using SSZ encoding. + +### JSON-encoded request (current behavior) + +```console +$ curl https://localhost:8551 \ + -X POST \ + -H "Content-Type: application/json" \ + -d '{ + "jsonrpc": "2.0", + "id": 1, + "method": "engine_newPayloadV1", + "params": [{ + "parentHash": "0x3b8fb240d288781d4aac94d3fd16809ee413bc99294a085798a589dae51ddd4a", + "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", + "stateRoot": "0xca3149fa9e37db08d1cd49c9061db1002ef1cd58db2210f2115c8c989b2bdf45", + "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421", + "logsBloom": "0x0000...0000", + "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", + "blockNumber": "0x1", + "gasLimit": "0x1c9c380", + "gasUsed": "0x0", + "timestamp": "0x5", + "extraData": "0x", + "baseFeePerGas": "0x7", + "blockHash": "0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858", + "transactions": [] + }] +}' +``` + +### JSON-encoded response + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "result": { + "status": "VALID", + "latestValidHash": "0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858", + "validationError": null + } +} +``` + +### SSZ-encoded request + +The consensus layer client sends the same `engine_newPayloadV1` call, but with `Content-Type: application/ssz` and `Accept: application/ssz`. The `params` array contains the SSZ-serialized `ExecutionPayloadV1` as a hex-encoded `DATA` string: + +```console +$ curl https://localhost:8551 \ + -X POST \ + -H "Content-Type: application/ssz" \ + -H "Accept: application/ssz" \ + -d '{ + "jsonrpc": "2.0", + "id": 1, + "method": "engine_newPayloadV1", + "params": ["0x3b8fb240d288781d4aac94d3fd16809ee413bc99294a085798a589dae51ddd4a..."] +}' +``` + +The single hex string in `params` is the SSZ serialization of the `ExecutionPayloadV1` container, encoding all fields (`parentHash`, `feeRecipient`, `stateRoot`, etc.) in their binary SSZ representations concatenated per the SSZ specification. + +### SSZ-encoded response + +The execution layer responds with `Content-Type: application/ssz`. The `result` field contains the SSZ-serialized `PayloadStatusV1` as a hex-encoded `DATA` string: + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "result": "0x00355...858" +} +``` + +Where the binary data encodes: +- `status`: `0x00` (VALID) +- `latestValidHash`: `0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858` +- `validationError`: absent (Optional not present) + +### Fallback behavior + +If the execution layer does not support SSZ, the same request with `Accept: application/ssz` returns a standard JSON response with `Content-Type: application/json`. The consensus layer detects this and continues using JSON for subsequent requests. + +If the consensus layer sends `Content-Type: application/ssz` to an execution layer that does not support it, the execution layer responds with HTTP `415 Unsupported Media Type`. The consensus layer **MUST** retry the request using JSON encoding. + +## Error handling + +SSZ encoding does not change the error semantics of the Engine API. All error codes defined in the [Errors](./common.md#errors) section apply equally to SSZ-encoded requests. + +Additionally: + +| Code | Message | Meaning | +| - | - | - | +| -32700 | Parse error | Invalid SSZ data was received by the server. | + +Clients **MUST** validate SSZ payloads against the expected schema before processing. Payloads that do not conform to the expected SSZ schema **MUST** be rejected with a `-32700` error. + +## Security considerations + +- SSZ deserialization **MUST** enforce the same size limits as JSON deserialization. Implementations **MUST** reject SSZ payloads exceeding defined maximum sizes before attempting full deserialization. +- Implementations **SHOULD** use well-tested SSZ libraries and fuzz test SSZ parsing extensively. From 905d42c58ea7872e00d06690682ded9124a4de6b Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Wed, 4 Mar 2026 09:54:43 +0100 Subject: [PATCH 02/12] add def for each method --- src/engine/ssz-encoding.md | 776 ++++++++++++++++++++++++++++++++++--- 1 file changed, 716 insertions(+), 60 deletions(-) diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md index 31cf8eb3b..22465066e 100644 --- a/src/engine/ssz-encoding.md +++ b/src/engine/ssz-encoding.md @@ -2,6 +2,8 @@ This document specifies an optional SSZ encoding for Engine API payloads as an alternative to the default JSON encoding. SSZ encoding reduces serialization overhead and aligns the Engine API with the native encoding format used by the consensus layer. +SSZ container definitions are provided for all Engine API structures and methods across all forks for backwards compatibility. + ## Table of contents @@ -9,7 +11,40 @@ This document specifies an optional SSZ encoding for Engine API payloads as an a - [Motivation](#motivation) - [Encoding negotiation](#encoding-negotiation) +- [Constants](#constants) - [SSZ type mappings](#ssz-type-mappings) +- [Container definitions](#container-definitions) + - [WithdrawalV1](#withdrawalv1) + - [ExecutionPayloadV1](#executionpayloadv1) + - [ExecutionPayloadV2](#executionpayloadv2) + - [ExecutionPayloadV3](#executionpayloadv3) + - [ExecutionPayloadV4](#executionpayloadv4) + - [PayloadStatusV1](#payloadstatusv1) + - [ForkchoiceStateV1](#forkchoicestatev1) + - [PayloadAttributesV1](#payloadattributesv1) + - [PayloadAttributesV2](#payloadattributesv2) + - [PayloadAttributesV3](#payloadattributesv3) + - [PayloadAttributesV4](#payloadattributesv4) + - [ForkchoiceUpdatedResponseV1](#forkchoiceupdatedresponsev1) + - [ExecutionPayloadBodyV1](#executionpayloadbodyv1) + - [ExecutionPayloadBodyV2](#executionpayloadbodyv2) + - [BlobsBundleV1](#blobsbundlev1) + - [BlobsBundleV2](#blobsbundlev2) + - [BlobAndProofV1](#blobandproofv1) + - [BlobAndProofV2](#blobandproofv2) + - [TransitionConfigurationV1](#transitionconfigurationv1) + - [GetPayloadResponseV2](#getpayloadresponsev2) + - [GetPayloadResponseV3](#getpayloadresponsev3) + - [GetPayloadResponseV4](#getpayloadresponsev4) + - [GetPayloadResponseV5](#getpayloadresponsev5) + - [GetPayloadResponseV6](#getpayloadresponsev6) +- [Method definitions](#method-definitions) + - [Paris methods](#paris-methods) + - [Shanghai methods](#shanghai-methods) + - [Cancun methods](#cancun-methods) + - [Prague methods](#prague-methods) + - [Osaka methods](#osaka-methods) + - [Amsterdam methods](#amsterdam-methods) - [Request and response format](#request-and-response-format) - [Example](#example) - [Error handling](#error-handling) @@ -48,43 +83,650 @@ The negotiation works as follows: 6. If no `Content-Type` header is present, the request **MUST** be parsed as JSON. If no `Accept` header is present, the response **SHOULD** use the same encoding as the request. +## Constants + +| Name | Value | Source | +| - | - | - | +| `MAX_BYTES_PER_TRANSACTION` | `2**30` (1,073,741,824) | [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844) | +| `MAX_TRANSACTIONS_PER_PAYLOAD` | `2**20` (1,048,576) | [Bellatrix](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md) | +| `MAX_WITHDRAWALS_PER_PAYLOAD` | `2**4` (16) | [Capella](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md) | +| `BYTES_PER_LOGS_BLOOM` | `256` | [Bellatrix](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md) | +| `MAX_EXTRA_DATA_BYTES` | `2**5` (32) | [Bellatrix](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md) | +| `MAX_BLOB_COMMITMENTS_PER_BLOCK` | `2**12` (4,096) | [Deneb](https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/beacon-chain.md) | +| `FIELD_ELEMENTS_PER_BLOB` | `4096` | [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844) | +| `BYTES_PER_FIELD_ELEMENT` | `32` | [EIP-4844](https://eips.ethereum.org/EIPS/eip-4844) | +| `CELLS_PER_EXT_BLOB` | `128` | [EIP-7594](https://eips.ethereum.org/EIPS/eip-7594) | +| `MAX_PAYLOAD_BODIES_REQUEST` | `2**5` (32) | [Shanghai](./shanghai.md#engine_getpayloadbodiesbyhashv1) | +| `MAX_BLOB_HASHES_REQUEST` | `128` | [Osaka](./osaka.md#engine_getblobsv2) | +| `MAX_EXECUTION_REQUESTS` | `2**8` (256) | [EIP-7685](https://eips.ethereum.org/EIPS/eip-7685) | +| `MAX_ERROR_MESSAGE_LENGTH` | `1024` | This specification | +| `BLOB_SIZE` | `FIELD_ELEMENTS_PER_BLOB * BYTES_PER_FIELD_ELEMENT` (131,072) | Derived | + ## SSZ type mappings Each JSON-encoded base type used in the Engine API maps to a specific SSZ type. The mappings below correspond to the types defined in the [base types schema](../schemas/base-types.yaml). -### Fixed-size types +| JSON Type | SSZ Type | +| - | - | +| `address` (20 bytes) | `Bytes20` | +| `hash32` (32 bytes) | `Bytes32` | +| `bytes8` (8 bytes) | `Bytes8` | +| `bytes32` (32 bytes) | `Bytes32` | +| `bytes48` (48 bytes) | `Bytes48` | +| `bytes256` (256 bytes) | `ByteVector[256]` | +| `uint64` | `uint64` | +| `uint256` | `uint256` | +| `BOOLEAN` | `boolean` | +| `bytes` (variable-length) | `ByteList[MAX_LENGTH]` (context-dependent) | +| `bytesMax32` (0 to 32 bytes) | `ByteList[32]` | +| `Array of T` | `List[T, MAX_LENGTH]` (context-dependent) | +| `T or null` | `Optional[T]` | + +## Container definitions + +### WithdrawalV1 + +Introduced in [Shanghai](./shanghai.md#withdrawalv1). + +```python +class WithdrawalV1(Container): + index: uint64 + validator_index: uint64 + address: Bytes20 + amount: uint64 +``` + +### ExecutionPayloadV1 + +Introduced in [Paris](./paris.md#executionpayloadv1). + +```python +class ExecutionPayloadV1(Container): + parent_hash: Bytes32 + fee_recipient: Bytes20 + state_root: Bytes32 + receipts_root: Bytes32 + logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM] + prev_randao: Bytes32 + block_number: uint64 + gas_limit: uint64 + gas_used: uint64 + timestamp: uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: uint256 + block_hash: Bytes32 + transactions: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_TRANSACTIONS_PER_PAYLOAD] +``` -| JSON Type | Size | SSZ Type | -| - | - | - | -| `address` | 20 bytes | `Bytes20` | -| `hash32` | 32 bytes | `Bytes32` | -| `bytes8` | 8 bytes | `Bytes8` | -| `bytes32` | 32 bytes | `Bytes32` | -| `bytes48` | 48 bytes | `Bytes48` | -| `bytes65` | 65 bytes | `Bytes65` | -| `bytes96` | 96 bytes | `Bytes96` | -| `bytes256` | 256 bytes | `ByteVector[256]` | -| `uint64` | 8 bytes | `uint64` | -| `uint256` | 32 bytes | `uint256` | -| `BOOLEAN` | 1 byte | `boolean` | - -### Variable-size types +### ExecutionPayloadV2 + +Introduced in [Shanghai](./shanghai.md#executionpayloadv2). Extends `ExecutionPayloadV1` with `withdrawals`. + +```python +class ExecutionPayloadV2(Container): + parent_hash: Bytes32 + fee_recipient: Bytes20 + state_root: Bytes32 + receipts_root: Bytes32 + logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM] + prev_randao: Bytes32 + block_number: uint64 + gas_limit: uint64 + gas_used: uint64 + timestamp: uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: uint256 + block_hash: Bytes32 + transactions: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_TRANSACTIONS_PER_PAYLOAD] + withdrawals: List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD] +``` -| JSON Type | SSZ Type | +### ExecutionPayloadV3 + +Introduced in [Cancun](./cancun.md#executionpayloadv3). Extends `ExecutionPayloadV2` with `blob_gas_used` and `excess_blob_gas`. + +```python +class ExecutionPayloadV3(Container): + parent_hash: Bytes32 + fee_recipient: Bytes20 + state_root: Bytes32 + receipts_root: Bytes32 + logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM] + prev_randao: Bytes32 + block_number: uint64 + gas_limit: uint64 + gas_used: uint64 + timestamp: uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: uint256 + block_hash: Bytes32 + transactions: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_TRANSACTIONS_PER_PAYLOAD] + withdrawals: List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD] + blob_gas_used: uint64 + excess_blob_gas: uint64 +``` + +### ExecutionPayloadV4 + +Introduced in [Amsterdam](./amsterdam.md#executionpayloadv4). Extends `ExecutionPayloadV3` with `block_access_list` and `slot_number`. + +```python +class ExecutionPayloadV4(Container): + parent_hash: Bytes32 + fee_recipient: Bytes20 + state_root: Bytes32 + receipts_root: Bytes32 + logs_bloom: ByteVector[BYTES_PER_LOGS_BLOOM] + prev_randao: Bytes32 + block_number: uint64 + gas_limit: uint64 + gas_used: uint64 + timestamp: uint64 + extra_data: ByteList[MAX_EXTRA_DATA_BYTES] + base_fee_per_gas: uint256 + block_hash: Bytes32 + transactions: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_TRANSACTIONS_PER_PAYLOAD] + withdrawals: List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD] + blob_gas_used: uint64 + excess_blob_gas: uint64 + block_access_list: ByteList[MAX_BYTES_PER_TRANSACTION] + slot_number: uint64 +``` + +### PayloadStatusV1 + +Introduced in [Paris](./paris.md#payloadstatusv1). The `status` field is encoded as a `uint8` enum. + +```python +class PayloadStatusV1(Container): + status: uint8 + latest_valid_hash: Optional[Bytes32] + validation_error: Optional[ByteList[MAX_ERROR_MESSAGE_LENGTH]] +``` + +| `status` value | Meaning | | - | - | -| `bytes` (variable-length hex data) | `ByteList[MAX_LENGTH]` where `MAX_LENGTH` is context-dependent | -| `bytesMax32` (up to 32 bytes hex data) | `ByteList[32]` | +| `0` | VALID | +| `1` | INVALID | +| `2` | SYNCING | +| `3` | ACCEPTED | +| `4` | INVALID_BLOCK_HASH | -### Composite types +### ForkchoiceStateV1 -| JSON Type | SSZ Type | +Introduced in [Paris](./paris.md#forkchoicestatev1). + +```python +class ForkchoiceStateV1(Container): + head_block_hash: Bytes32 + safe_block_hash: Bytes32 + finalized_block_hash: Bytes32 +``` + +### PayloadAttributesV1 + +Introduced in [Paris](./paris.md#payloadattributesv1). + +```python +class PayloadAttributesV1(Container): + timestamp: uint64 + prev_randao: Bytes32 + suggested_fee_recipient: Bytes20 +``` + +### PayloadAttributesV2 + +Introduced in [Shanghai](./shanghai.md#payloadattributesv2). Extends `PayloadAttributesV1` with `withdrawals`. + +```python +class PayloadAttributesV2(Container): + timestamp: uint64 + prev_randao: Bytes32 + suggested_fee_recipient: Bytes20 + withdrawals: List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD] +``` + +### PayloadAttributesV3 + +Introduced in [Cancun](./cancun.md#payloadattributesv3). Extends `PayloadAttributesV2` with `parent_beacon_block_root`. + +```python +class PayloadAttributesV3(Container): + timestamp: uint64 + prev_randao: Bytes32 + suggested_fee_recipient: Bytes20 + withdrawals: List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD] + parent_beacon_block_root: Bytes32 +``` + +### PayloadAttributesV4 + +Introduced in [Amsterdam](./amsterdam.md#payloadattributesv4). Extends `PayloadAttributesV3` with `slot_number`. + +```python +class PayloadAttributesV4(Container): + timestamp: uint64 + prev_randao: Bytes32 + suggested_fee_recipient: Bytes20 + withdrawals: List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD] + parent_beacon_block_root: Bytes32 + slot_number: uint64 +``` + +### ForkchoiceUpdatedResponseV1 + +Used by all versions of `engine_forkchoiceUpdated`. + +```python +class ForkchoiceUpdatedResponseV1(Container): + payload_status: PayloadStatusV1 + payload_id: Optional[Bytes8] +``` + +### ExecutionPayloadBodyV1 + +Introduced in [Shanghai](./shanghai.md#executionpayloadbodyv1). + +```python +class ExecutionPayloadBodyV1(Container): + transactions: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_TRANSACTIONS_PER_PAYLOAD] + withdrawals: Optional[List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD]] +``` + +### ExecutionPayloadBodyV2 + +Introduced in [Amsterdam](./amsterdam.md#executionpayloadbodyv2). Extends `ExecutionPayloadBodyV1` with `block_access_list`. + +```python +class ExecutionPayloadBodyV2(Container): + transactions: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_TRANSACTIONS_PER_PAYLOAD] + withdrawals: Optional[List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD]] + block_access_list: Optional[ByteList[MAX_BYTES_PER_TRANSACTION]] +``` + +### BlobsBundleV1 + +Introduced in [Cancun](./cancun.md#blobsbundlev1). + +```python +class BlobsBundleV1(Container): + commitments: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK] + proofs: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK] + blobs: List[ByteVector[BLOB_SIZE], MAX_BLOB_COMMITMENTS_PER_BLOCK] +``` + +### BlobsBundleV2 + +Introduced in [Osaka](./osaka.md#blobsbundlev2). Proofs are cell proofs with `CELLS_PER_EXT_BLOB` proofs per blob. + +```python +class BlobsBundleV2(Container): + commitments: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK] + proofs: List[Bytes48, MAX_BLOB_COMMITMENTS_PER_BLOCK * CELLS_PER_EXT_BLOB] + blobs: List[ByteVector[BLOB_SIZE], MAX_BLOB_COMMITMENTS_PER_BLOCK] +``` + +### BlobAndProofV1 + +Introduced in [Cancun](./cancun.md#blobandproofv1). + +```python +class BlobAndProofV1(Container): + blob: ByteVector[BLOB_SIZE] + proof: Bytes48 +``` + +### BlobAndProofV2 + +Introduced in [Osaka](./osaka.md#blobandproofv2). + +```python +class BlobAndProofV2(Container): + blob: ByteVector[BLOB_SIZE] + proofs: List[Bytes48, CELLS_PER_EXT_BLOB] +``` + +### TransitionConfigurationV1 + +Introduced in [Paris](./paris.md#transitionconfigurationv1). Deprecated in Cancun. + +```python +class TransitionConfigurationV1(Container): + terminal_total_difficulty: uint256 + terminal_block_hash: Bytes32 + terminal_block_number: uint64 +``` + +### GetPayloadResponseV2 + +Response container for [`engine_getPayloadV2`](./shanghai.md#engine_getpayloadv2). + +```python +class GetPayloadResponseV2(Container): + execution_payload: ExecutionPayloadV2 + block_value: uint256 +``` + +*Note:* `engine_getPayloadV2` may return `ExecutionPayloadV1` for pre-Shanghai timestamps. The SSZ encoding uses `ExecutionPayloadV2` in all cases; pre-Shanghai payloads have an empty `withdrawals` list. + +### GetPayloadResponseV3 + +Response container for [`engine_getPayloadV3`](./cancun.md#engine_getpayloadv3). + +```python +class GetPayloadResponseV3(Container): + execution_payload: ExecutionPayloadV3 + block_value: uint256 + blobs_bundle: BlobsBundleV1 + should_override_builder: boolean +``` + +### GetPayloadResponseV4 + +Response container for [`engine_getPayloadV4`](./prague.md#engine_getpayloadv4). + +```python +class GetPayloadResponseV4(Container): + execution_payload: ExecutionPayloadV3 + block_value: uint256 + blobs_bundle: BlobsBundleV1 + should_override_builder: boolean + execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] +``` + +### GetPayloadResponseV5 + +Response container for [`engine_getPayloadV5`](./osaka.md#engine_getpayloadv5). + +```python +class GetPayloadResponseV5(Container): + execution_payload: ExecutionPayloadV3 + block_value: uint256 + blobs_bundle: BlobsBundleV2 + should_override_builder: boolean + execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] +``` + +### GetPayloadResponseV6 + +Response container for [`engine_getPayloadV6`](./amsterdam.md#engine_getpayloadv6). + +```python +class GetPayloadResponseV6(Container): + execution_payload: ExecutionPayloadV4 + block_value: uint256 + blobs_bundle: BlobsBundleV2 + should_override_builder: boolean + execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] +``` + +## Method definitions + +This section defines the SSZ types for each method's parameters and result, organized by fork. Each parameter is individually SSZ-encoded in the JSON-RPC `params` array. Nullable parameters remain `null` when absent. + +### Paris methods + +#### engine_newPayloadV1 + +| Parameter | SSZ Type | +| - | - | +| `executionPayload` | [`ExecutionPayloadV1`](#executionpayloadv1) | + +| Result | SSZ Type | +| - | - | +| Payload status | [`PayloadStatusV1`](#payloadstatusv1) | + +#### engine_forkchoiceUpdatedV1 + +| Parameter | SSZ Type | +| - | - | +| `forkchoiceState` | [`ForkchoiceStateV1`](#forkchoicestatev1) | +| `payloadAttributes` | [`PayloadAttributesV1`](#payloadattributesv1) or `null` | + +| Result | SSZ Type | +| - | - | +| Forkchoice updated response | [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) | + +#### engine_getPayloadV1 + +| Parameter | SSZ Type | +| - | - | +| `payloadId` | `Bytes8` | + +| Result | SSZ Type | +| - | - | +| Execution payload | [`ExecutionPayloadV1`](#executionpayloadv1) | + +#### engine_exchangeTransitionConfigurationV1 + +Deprecated in Cancun. + +| Parameter | SSZ Type | +| - | - | +| `transitionConfiguration` | [`TransitionConfigurationV1`](#transitionconfigurationv1) | + +| Result | SSZ Type | +| - | - | +| Transition configuration | [`TransitionConfigurationV1`](#transitionconfigurationv1) | + +### Shanghai methods + +#### engine_newPayloadV2 + +| Parameter | SSZ Type | +| - | - | +| `executionPayload` | [`ExecutionPayloadV1`](#executionpayloadv1) or [`ExecutionPayloadV2`](#executionpayloadv2) (by timestamp) | + +| Result | SSZ Type | +| - | - | +| Payload status | [`PayloadStatusV1`](#payloadstatusv1) | + +#### engine_forkchoiceUpdatedV2 + +| Parameter | SSZ Type | +| - | - | +| `forkchoiceState` | [`ForkchoiceStateV1`](#forkchoicestatev1) | +| `payloadAttributes` | [`PayloadAttributesV1`](#payloadattributesv1), [`PayloadAttributesV2`](#payloadattributesv2), or `null` | + +| Result | SSZ Type | +| - | - | +| Forkchoice updated response | [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) | + +#### engine_getPayloadV2 + +| Parameter | SSZ Type | +| - | - | +| `payloadId` | `Bytes8` | + +| Result | SSZ Type | +| - | - | +| Get payload response | [`GetPayloadResponseV2`](#getpayloadresponsev2) | + +#### engine_getPayloadBodiesByHashV1 + +| Parameter | SSZ Type | +| - | - | +| `blockHashes` | `List[Bytes32, MAX_PAYLOAD_BODIES_REQUEST]` | + +| Result | SSZ Type | +| - | - | +| Payload bodies | `List[Optional[`[`ExecutionPayloadBodyV1`](#executionpayloadbodyv1)`], MAX_PAYLOAD_BODIES_REQUEST]` | + +#### engine_getPayloadBodiesByRangeV1 + +| Parameter | SSZ Type | +| - | - | +| `start` | `uint64` | +| `count` | `uint64` | + +| Result | SSZ Type | +| - | - | +| Payload bodies | `List[Optional[`[`ExecutionPayloadBodyV1`](#executionpayloadbodyv1)`], MAX_PAYLOAD_BODIES_REQUEST]` | + +### Cancun methods + +#### engine_newPayloadV3 + +| Parameter | SSZ Type | +| - | - | +| `executionPayload` | [`ExecutionPayloadV3`](#executionpayloadv3) | +| `expectedBlobVersionedHashes` | `List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK]` | +| `parentBeaconBlockRoot` | `Bytes32` | + +| Result | SSZ Type | +| - | - | +| Payload status | [`PayloadStatusV1`](#payloadstatusv1) | + +#### engine_forkchoiceUpdatedV3 + +| Parameter | SSZ Type | +| - | - | +| `forkchoiceState` | [`ForkchoiceStateV1`](#forkchoicestatev1) | +| `payloadAttributes` | [`PayloadAttributesV3`](#payloadattributesv3) or `null` | + +| Result | SSZ Type | +| - | - | +| Forkchoice updated response | [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) | + +#### engine_getPayloadV3 + +| Parameter | SSZ Type | +| - | - | +| `payloadId` | `Bytes8` | + +| Result | SSZ Type | +| - | - | +| Get payload response | [`GetPayloadResponseV3`](#getpayloadresponsev3) | + +#### engine_getBlobsV1 + +Deprecated in Osaka. + +| Parameter | SSZ Type | +| - | - | +| `blobVersionedHashes` | `List[Bytes32, MAX_BLOB_HASHES_REQUEST]` | + +| Result | SSZ Type | +| - | - | +| Blobs and proofs | `Optional[List[`[`BlobAndProofV1`](#blobandproofv1)`, MAX_BLOB_HASHES_REQUEST]]` | + +### Prague methods + +#### engine_newPayloadV4 + +| Parameter | SSZ Type | +| - | - | +| `executionPayload` | [`ExecutionPayloadV3`](#executionpayloadv3) | +| `expectedBlobVersionedHashes` | `List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK]` | +| `parentBeaconBlockRoot` | `Bytes32` | +| `executionRequests` | `List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS]` | + +| Result | SSZ Type | +| - | - | +| Payload status | [`PayloadStatusV1`](#payloadstatusv1) | + +#### engine_getPayloadV4 + +| Parameter | SSZ Type | +| - | - | +| `payloadId` | `Bytes8` | + +| Result | SSZ Type | +| - | - | +| Get payload response | [`GetPayloadResponseV4`](#getpayloadresponsev4) | + +### Osaka methods + +#### engine_getPayloadV5 + +| Parameter | SSZ Type | +| - | - | +| `payloadId` | `Bytes8` | + +| Result | SSZ Type | +| - | - | +| Get payload response | [`GetPayloadResponseV5`](#getpayloadresponsev5) | + +#### engine_getBlobsV2 + +Returns `null` for the entire result if any blob is missing or if syncing. + +| Parameter | SSZ Type | +| - | - | +| `blobVersionedHashes` | `List[Bytes32, MAX_BLOB_HASHES_REQUEST]` | + +| Result | SSZ Type | +| - | - | +| Blobs and proofs | `Optional[List[`[`BlobAndProofV2`](#blobandproofv2)`, MAX_BLOB_HASHES_REQUEST]]` | + +#### engine_getBlobsV3 + +Returns per-element `null` for missing blobs, or `null` for the entire result if syncing. + +| Parameter | SSZ Type | +| - | - | +| `blobVersionedHashes` | `List[Bytes32, MAX_BLOB_HASHES_REQUEST]` | + +| Result | SSZ Type | +| - | - | +| Blobs and proofs | `Optional[List[Optional[`[`BlobAndProofV2`](#blobandproofv2)`], MAX_BLOB_HASHES_REQUEST]]` | + +### Amsterdam methods + +#### engine_newPayloadV5 + +| Parameter | SSZ Type | | - | - | -| `Array of T` | `List[T, MAX_LENGTH]` where `MAX_LENGTH` is context-dependent | -| Object (e.g. `ExecutionPayloadV1`) | `Container` with fields mapped per this table | +| `executionPayload` | [`ExecutionPayloadV4`](#executionpayloadv4) | +| `expectedBlobVersionedHashes` | `List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK]` | +| `parentBeaconBlockRoot` | `Bytes32` | +| `executionRequests` | `List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS]` | -### Nullable fields +| Result | SSZ Type | +| - | - | +| Payload status | [`PayloadStatusV1`](#payloadstatusv1) | + +#### engine_getPayloadV6 + +| Parameter | SSZ Type | +| - | - | +| `payloadId` | `Bytes8` | + +| Result | SSZ Type | +| - | - | +| Get payload response | [`GetPayloadResponseV6`](#getpayloadresponsev6) | + +#### engine_forkchoiceUpdatedV4 + +| Parameter | SSZ Type | +| - | - | +| `forkchoiceState` | [`ForkchoiceStateV1`](#forkchoicestatev1) | +| `payloadAttributes` | [`PayloadAttributesV4`](#payloadattributesv4) or `null` | + +| Result | SSZ Type | +| - | - | +| Forkchoice updated response | [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) | + +#### engine_getPayloadBodiesByHashV2 + +| Parameter | SSZ Type | +| - | - | +| `blockHashes` | `List[Bytes32, MAX_PAYLOAD_BODIES_REQUEST]` | -Fields that may be `null` in the JSON encoding (e.g. `latestValidHash` in `PayloadStatusV1`, `withdrawals` in `ExecutionPayloadBodyV1`) are represented using `Optional[T]` in SSZ. +| Result | SSZ Type | +| - | - | +| Payload bodies | `List[Optional[`[`ExecutionPayloadBodyV2`](#executionpayloadbodyv2)`], MAX_PAYLOAD_BODIES_REQUEST]` | + +#### engine_getPayloadBodiesByRangeV2 + +| Parameter | SSZ Type | +| - | - | +| `start` | `uint64` | +| `count` | `uint64` | + +| Result | SSZ Type | +| - | - | +| Payload bodies | `List[Optional[`[`ExecutionPayloadBodyV2`](#executionpayloadbodyv2)`], MAX_PAYLOAD_BODIES_REQUEST]` | ## Request and response format @@ -92,9 +734,9 @@ SSZ-encoded Engine API requests and responses follow the existing JSON-RPC metho Specifically, when SSZ encoding is in use: -1. The HTTP request body is a JSON-RPC request where each element of `params` is replaced with its SSZ-encoded hexadecimal representation (a `DATA` string). +1. The HTTP request body is a JSON-RPC request where each element of `params` is replaced with its SSZ-encoded hexadecimal representation (a `DATA` string). Parameters that are `null` remain `null`. -2. The HTTP response body is a JSON-RPC response where the `result` field is replaced with the SSZ-encoded hexadecimal representation of the result value. +2. The HTTP response body is a JSON-RPC response where the `result` field is replaced with the SSZ-encoded hexadecimal representation of the result value. A `null` result remains `null`. This approach preserves compatibility with JSON-RPC tooling while encoding the payload data in SSZ. @@ -102,7 +744,7 @@ This approach preserves compatibility with JSON-RPC tooling while encoding the p ## Example -The following example shows an `engine_newPayloadV1` call with a minimal payload, first using JSON encoding and then using SSZ encoding. +The following example shows an `engine_newPayloadV5` call, first using JSON encoding and then using SSZ encoding. ### JSON-encoded request (current behavior) @@ -113,23 +755,33 @@ $ curl https://localhost:8551 \ -d '{ "jsonrpc": "2.0", "id": 1, - "method": "engine_newPayloadV1", - "params": [{ - "parentHash": "0x3b8fb240d288781d4aac94d3fd16809ee413bc99294a085798a589dae51ddd4a", - "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "stateRoot": "0xca3149fa9e37db08d1cd49c9061db1002ef1cd58db2210f2115c8c989b2bdf45", - "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421", - "logsBloom": "0x0000...0000", - "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", - "blockNumber": "0x1", - "gasLimit": "0x1c9c380", - "gasUsed": "0x0", - "timestamp": "0x5", - "extraData": "0x", - "baseFeePerGas": "0x7", - "blockHash": "0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858", - "transactions": [] - }] + "method": "engine_newPayloadV5", + "params": [ + { + "parentHash": "0x3b8fb240d288781d4aac94d3fd16809ee413bc99294a085798a589dae51ddd4a", + "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", + "stateRoot": "0xca3149fa9e37db08d1cd49c9061db1002ef1cd58db2210f2115c8c989b2bdf45", + "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421", + "logsBloom": "0x0000...0000", + "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", + "blockNumber": "0x1", + "gasLimit": "0x1c9c380", + "gasUsed": "0x0", + "timestamp": "0x5", + "extraData": "0x", + "baseFeePerGas": "0x7", + "blockHash": "0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858", + "transactions": [], + "withdrawals": [], + "blobGasUsed": "0x0", + "excessBlobGas": "0x0", + "blockAccessList": "0x", + "slotNumber": "0x1" + }, + [], + "0x0000000000000000000000000000000000000000000000000000000000000000", + [] + ] }' ``` @@ -149,39 +801,43 @@ $ curl https://localhost:8551 \ ### SSZ-encoded request -The consensus layer client sends the same `engine_newPayloadV1` call, but with `Content-Type: application/ssz` and `Accept: application/ssz`. The `params` array contains the SSZ-serialized `ExecutionPayloadV1` as a hex-encoded `DATA` string: +The consensus layer client sends the same call with `Content-Type: application/ssz` and `Accept: application/ssz`. Each element of the `params` array is individually SSZ-encoded as a hex `DATA` string: -```console -$ curl https://localhost:8551 \ - -X POST \ - -H "Content-Type: application/ssz" \ - -H "Accept: application/ssz" \ - -d '{ +```json +{ "jsonrpc": "2.0", "id": 1, - "method": "engine_newPayloadV1", - "params": ["0x3b8fb240d288781d4aac94d3fd16809ee413bc99294a085798a589dae51ddd4a..."] -}' + "method": "engine_newPayloadV5", + "params": [ + "0x3b8fb240d288781d4aac94d3fd16809ee413bc99...", + "0x", + "0x0000000000000000000000000000000000000000000000000000000000000000", + "0x" + ] +} ``` -The single hex string in `params` is the SSZ serialization of the `ExecutionPayloadV1` container, encoding all fields (`parentHash`, `feeRecipient`, `stateRoot`, etc.) in their binary SSZ representations concatenated per the SSZ specification. +- `params[0]`: SSZ-serialized `ExecutionPayloadV4` container +- `params[1]`: SSZ-serialized `List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK]` (empty list) +- `params[2]`: SSZ-serialized `Bytes32` (parent beacon block root) +- `params[3]`: SSZ-serialized `List[ByteList, MAX_EXECUTION_REQUESTS]` (empty list) ### SSZ-encoded response -The execution layer responds with `Content-Type: application/ssz`. The `result` field contains the SSZ-serialized `PayloadStatusV1` as a hex-encoded `DATA` string: +The execution layer responds with `Content-Type: application/ssz`. The `result` field contains the SSZ-serialized `PayloadStatusV1`: ```json { "jsonrpc": "2.0", "id": 1, - "result": "0x00355...858" + "result": "0x00013559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858" } ``` Where the binary data encodes: - `status`: `0x00` (VALID) -- `latestValidHash`: `0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858` -- `validationError`: absent (Optional not present) +- `latest_valid_hash`: present, `0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858` +- `validation_error`: absent ### Fallback behavior From 998b97e458611c8cdd3519448c19b6a6b5be9a07 Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Wed, 4 Mar 2026 20:25:09 +0100 Subject: [PATCH 03/12] docs(ssz-encoding): update SSZ type mappings and container notes to reflect optionality removal Removes `Optional[T]` mapping as it is replaced by specific zero/empty value encoding in container definitions. Updates `PayloadStatusV1`, `ForkchoiceUpdatedResponseV1`, `ExecutionPayloadBodyV1`, and `ExecutionPayloadBodyV2` to use non-optional types where JSON mapping implies a zero/empty value instead of true optionality. Updates SSZ mappings for `engine_getPayloadBodiesByHashV1/V2` and `engine_getBlobsV1/V2/V3` to use nested lists (`List[List[T, 1]]`) instead of `Optional[T]` to represent the presence or absence of data, consistent with non-null SSZ encoding for absent data. Adds notes explaining the zero/empty encoding for absent fields. --- src/engine/ssz-encoding.md | 49 +++++++++++++++++++++++++++----------- 1 file changed, 35 insertions(+), 14 deletions(-) diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md index 22465066e..b4d124b68 100644 --- a/src/engine/ssz-encoding.md +++ b/src/engine/ssz-encoding.md @@ -120,7 +120,6 @@ Each JSON-encoded base type used in the Engine API maps to a specific SSZ type. | `bytes` (variable-length) | `ByteList[MAX_LENGTH]` (context-dependent) | | `bytesMax32` (0 to 32 bytes) | `ByteList[32]` | | `Array of T` | `List[T, MAX_LENGTH]` (context-dependent) | -| `T or null` | `Optional[T]` | ## Container definitions @@ -240,10 +239,12 @@ Introduced in [Paris](./paris.md#payloadstatusv1). The `status` field is encoded ```python class PayloadStatusV1(Container): status: uint8 - latest_valid_hash: Optional[Bytes32] - validation_error: Optional[ByteList[MAX_ERROR_MESSAGE_LENGTH]] + latest_valid_hash: Bytes32 + validation_error: ByteList[MAX_ERROR_MESSAGE_LENGTH] ``` +*Note:* `latest_valid_hash` is all zeros when absent (e.g. when `status` is `SYNCING` or `ACCEPTED`). `validation_error` is empty when absent. + | `status` value | Meaning | | - | - | | `0` | VALID | @@ -320,9 +321,11 @@ Used by all versions of `engine_forkchoiceUpdated`. ```python class ForkchoiceUpdatedResponseV1(Container): payload_status: PayloadStatusV1 - payload_id: Optional[Bytes8] + payload_id: Bytes8 ``` +*Note:* `payload_id` is all zeros when no payload building was initiated. + ### ExecutionPayloadBodyV1 Introduced in [Shanghai](./shanghai.md#executionpayloadbodyv1). @@ -330,9 +333,11 @@ Introduced in [Shanghai](./shanghai.md#executionpayloadbodyv1). ```python class ExecutionPayloadBodyV1(Container): transactions: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_TRANSACTIONS_PER_PAYLOAD] - withdrawals: Optional[List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD]] + withdrawals: List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD] ``` +*Note:* `withdrawals` is empty for pre-Shanghai blocks. + ### ExecutionPayloadBodyV2 Introduced in [Amsterdam](./amsterdam.md#executionpayloadbodyv2). Extends `ExecutionPayloadBodyV1` with `block_access_list`. @@ -340,10 +345,12 @@ Introduced in [Amsterdam](./amsterdam.md#executionpayloadbodyv2). Extends `Execu ```python class ExecutionPayloadBodyV2(Container): transactions: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_TRANSACTIONS_PER_PAYLOAD] - withdrawals: Optional[List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD]] - block_access_list: Optional[ByteList[MAX_BYTES_PER_TRANSACTION]] + withdrawals: List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD] + block_access_list: ByteList[MAX_BYTES_PER_TRANSACTION] ``` +*Note:* `withdrawals` is empty for pre-Shanghai blocks. `block_access_list` is empty for pre-Amsterdam blocks. + ### BlobsBundleV1 Introduced in [Cancun](./cancun.md#blobsbundlev1). @@ -550,7 +557,9 @@ Deprecated in Cancun. | Result | SSZ Type | | - | - | -| Payload bodies | `List[Optional[`[`ExecutionPayloadBodyV1`](#executionpayloadbodyv1)`], MAX_PAYLOAD_BODIES_REQUEST]` | +| Payload bodies | `List[List[`[`ExecutionPayloadBodyV1`](#executionpayloadbodyv1)`, 1], MAX_PAYLOAD_BODIES_REQUEST]` | + +*Note:* Each inner list has 0 elements for unknown blocks and 1 element for known blocks. #### engine_getPayloadBodiesByRangeV1 @@ -561,7 +570,9 @@ Deprecated in Cancun. | Result | SSZ Type | | - | - | -| Payload bodies | `List[Optional[`[`ExecutionPayloadBodyV1`](#executionpayloadbodyv1)`], MAX_PAYLOAD_BODIES_REQUEST]` | +| Payload bodies | `List[List[`[`ExecutionPayloadBodyV1`](#executionpayloadbodyv1)`, 1], MAX_PAYLOAD_BODIES_REQUEST]` | + +*Note:* Each inner list has 0 elements for unknown blocks and 1 element for known blocks. ### Cancun methods @@ -608,7 +619,9 @@ Deprecated in Osaka. | Result | SSZ Type | | - | - | -| Blobs and proofs | `Optional[List[`[`BlobAndProofV1`](#blobandproofv1)`, MAX_BLOB_HASHES_REQUEST]]` | +| Blobs and proofs | `List[`[`BlobAndProofV1`](#blobandproofv1)`, MAX_BLOB_HASHES_REQUEST]` | + +*Note:* Returns `null` at the JSON-RPC level when syncing (SSZ encoding only applies to non-null results). ### Prague methods @@ -657,7 +670,9 @@ Returns `null` for the entire result if any blob is missing or if syncing. | Result | SSZ Type | | - | - | -| Blobs and proofs | `Optional[List[`[`BlobAndProofV2`](#blobandproofv2)`, MAX_BLOB_HASHES_REQUEST]]` | +| Blobs and proofs | `List[`[`BlobAndProofV2`](#blobandproofv2)`, MAX_BLOB_HASHES_REQUEST]` | + +*Note:* Returns `null` at the JSON-RPC level when syncing or any blob is missing (SSZ encoding only applies to non-null results). #### engine_getBlobsV3 @@ -669,7 +684,9 @@ Returns per-element `null` for missing blobs, or `null` for the entire result if | Result | SSZ Type | | - | - | -| Blobs and proofs | `Optional[List[Optional[`[`BlobAndProofV2`](#blobandproofv2)`], MAX_BLOB_HASHES_REQUEST]]` | +| Blobs and proofs | `List[List[`[`BlobAndProofV2`](#blobandproofv2)`, 1], MAX_BLOB_HASHES_REQUEST]` | + +*Note:* Returns `null` at the JSON-RPC level when syncing. Each inner list has 0 elements for a missing blob and 1 element for a present blob. ### Amsterdam methods @@ -715,7 +732,9 @@ Returns per-element `null` for missing blobs, or `null` for the entire result if | Result | SSZ Type | | - | - | -| Payload bodies | `List[Optional[`[`ExecutionPayloadBodyV2`](#executionpayloadbodyv2)`], MAX_PAYLOAD_BODIES_REQUEST]` | +| Payload bodies | `List[List[`[`ExecutionPayloadBodyV2`](#executionpayloadbodyv2)`, 1], MAX_PAYLOAD_BODIES_REQUEST]` | + +*Note:* Each inner list has 0 elements for unknown blocks and 1 element for known blocks. #### engine_getPayloadBodiesByRangeV2 @@ -726,7 +745,9 @@ Returns per-element `null` for missing blobs, or `null` for the entire result if | Result | SSZ Type | | - | - | -| Payload bodies | `List[Optional[`[`ExecutionPayloadBodyV2`](#executionpayloadbodyv2)`], MAX_PAYLOAD_BODIES_REQUEST]` | +| Payload bodies | `List[List[`[`ExecutionPayloadBodyV2`](#executionpayloadbodyv2)`, 1], MAX_PAYLOAD_BODIES_REQUEST]` | + +*Note:* Each inner list has 0 elements for unknown blocks and 1 element for known blocks. ## Request and response format From 47049bb7cbf73f0d8026d8763f3f1f4b7e2ebb70 Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Wed, 4 Mar 2026 20:27:14 +0100 Subject: [PATCH 04/12] fix spellchecker --- wordlist.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/wordlist.txt b/wordlist.txt index 848fbdacc..ac36cceae 100644 --- a/wordlist.txt +++ b/wordlist.txt @@ -40,6 +40,7 @@ forkchoiceupdatedresponsev getblobsbundlev getblobsv getclientversionv +getpayloadresponsev graphql gwei https From 3d3a00e6f29eb3cb9f922baf5df0f0cde614ce01 Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Wed, 4 Mar 2026 20:44:28 +0100 Subject: [PATCH 05/12] make payload ssz --- src/engine/common.md | 10 +- src/engine/ssz-encoding.md | 650 ++++++++++++++++++++----------------- 2 files changed, 359 insertions(+), 301 deletions(-) diff --git a/src/engine/common.md b/src/engine/common.md index dd0b24a3d..bddb7ce00 100644 --- a/src/engine/common.md +++ b/src/engine/common.md @@ -15,7 +15,7 @@ This document specifies common definitions and requirements affecting Engine API - [Errors](#errors) - [Timeouts](#timeouts) - [Encoding](#encoding) - - [SSZ encoding](#ssz-encoding) + - [Binary SSZ transport](#binary-ssz-transport) - [Capabilities](#capabilities) - [engine_exchangeCapabilities](#engine_exchangecapabilities) - [Request](#request) @@ -137,13 +137,13 @@ Values of a field of `QUANTITY` type **MUST** be encoded as a hexadecimal string *Note:* Byte order of encoded value having `QUANTITY` type is big-endian. -### SSZ encoding +### Binary SSZ transport -Clients **MAY** optionally support SSZ encoding for Engine API payloads as an alternative to JSON encoding. SSZ encoding is not tied to any specific hard fork and can be implemented independently by any client team at any time. +Clients **MAY** support a binary SSZ transport as an alternative to JSON-RPC. The binary transport uses raw SSZ bytes over HTTP with path-based method routing, eliminating JSON and hex-encoding overhead for fast CL-EL communication. -When both the consensus layer and execution layer clients support SSZ encoding, they **SHOULD** use it for reduced serialization overhead. When either client does not support SSZ, both **MUST** fall back to JSON encoding. +When both the consensus layer and execution layer clients support the binary SSZ transport, they **SHOULD** use it. When either client does not support it, both **MUST** fall back to JSON-RPC encoding. -SSZ encoding support is negotiated via standard HTTP content-type headers. The full specification is defined in the [SSZ Encoding](./ssz-encoding.md) document. +The full specification is defined in the [Binary SSZ Transport](./ssz-encoding.md) document. [json-rpc-spec]: https://playground.open-rpc.org/?schemaUrl=https://raw.githubusercontent.com/ethereum/execution-apis/assembled-spec/openrpc.json&uiSchema[appBar][ui:splitView]=false&uiSchema[appBar][ui:input]=false&uiSchema[appBar][ui:examplesDropdown]=false diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md index b4d124b68..426b7329b 100644 --- a/src/engine/ssz-encoding.md +++ b/src/engine/ssz-encoding.md @@ -1,6 +1,6 @@ -# Engine API -- SSZ Encoding +# Engine API -- Binary SSZ Transport -This document specifies an optional SSZ encoding for Engine API payloads as an alternative to the default JSON encoding. SSZ encoding reduces serialization overhead and aligns the Engine API with the native encoding format used by the consensus layer. +This document specifies a binary SSZ transport for Engine API communication between consensus layer (CL) and execution layer (EL) clients. The binary transport replaces JSON-RPC with raw SSZ over HTTP for fast, efficient CL-EL communication. SSZ container definitions are provided for all Engine API structures and methods across all forks for backwards compatibility. @@ -10,7 +10,10 @@ SSZ container definitions are provided for all Engine API structures and methods - [Motivation](#motivation) -- [Encoding negotiation](#encoding-negotiation) +- [Transport](#transport) + - [Request format](#request-format) + - [Response format](#response-format) + - [Negotiation and fallback](#negotiation-and-fallback) - [Constants](#constants) - [SSZ type mappings](#ssz-type-mappings) - [Container definitions](#container-definitions) @@ -38,6 +41,12 @@ SSZ container definitions are provided for all Engine API structures and methods - [GetPayloadResponseV4](#getpayloadresponsev4) - [GetPayloadResponseV5](#getpayloadresponsev5) - [GetPayloadResponseV6](#getpayloadresponsev6) + - [PayloadBodiesV1Response](#payloadbodiesv1response) + - [PayloadBodiesV2Response](#payloadbodiesv2response) + - [GetBlobsV1Response](#getblobsv1response) + - [GetBlobsV2Response](#getblobsv2response) + - [GetBlobsV3Response](#getblobsv3response) + - [ErrorResponse](#errorresponse) - [Method definitions](#method-definitions) - [Paris methods](#paris-methods) - [Shanghai methods](#shanghai-methods) @@ -45,7 +54,6 @@ SSZ container definitions are provided for all Engine API structures and methods - [Prague methods](#prague-methods) - [Osaka methods](#osaka-methods) - [Amsterdam methods](#amsterdam-methods) -- [Request and response format](#request-and-response-format) - [Example](#example) - [Error handling](#error-handling) - [Security considerations](#security-considerations) @@ -54,34 +62,68 @@ SSZ container definitions are provided for all Engine API structures and methods ## Motivation -The current JSON-RPC encoding introduces serialization overhead that grows with payload size. Binary data (hashes, addresses, bytecode) must be hex-encoded, doubling their size. As Ethereum scales through increased gas limits and blob transactions, this overhead becomes a bottleneck for block propagation and validation timing. +Fast communication between the consensus layer and execution layer is critical for block propagation and validation timing. The JSON-RPC transport introduces unnecessary overhead in this critical path: -The consensus layer already uses SSZ for all internal data structures and network communication. The current architecture requires converting between SSZ and JSON at the Engine API boundary in both directions. SSZ encoding for the Engine API eliminates this double conversion, reduces payload sizes by 40-60%, and provides deterministic encoding. +- Binary data (hashes, addresses, transactions, blobs) is hex-encoded, doubling wire size. +- JSON parsing and generation adds CPU overhead on both sides. +- The CL uses SSZ natively, forcing a round-trip conversion (SSZ to JSON, then JSON to internal types) at the Engine API boundary. -## Encoding negotiation +Binary SSZ eliminates all of this. The CL sends raw SSZ bytes over HTTP; the EL deserializes directly. No hex encoding, no JSON parsing, no intermediate representations. Payload sizes are reduced by 50% or more compared to JSON-RPC, and serialization is no longer a bottleneck in the critical path between CL and EL. -SSZ encoding support is negotiated via standard HTTP content negotiation headers. No additional capability exchange is required. +## Transport -| Header | Value | Meaning | -| - | - | - | -| `Content-Type` | `application/ssz` | The request body is SSZ-encoded | -| `Content-Type` | `application/json` | The request body is JSON-encoded (default) | -| `Accept` | `application/ssz` | The client prefers an SSZ-encoded response | -| `Accept` | `application/json` | The client prefers a JSON-encoded response (default) | +Binary SSZ uses HTTP with path-based method routing. Each Engine API method has a dedicated URL path. Request and response bodies are raw SSZ bytes. + +### Request format + +``` +POST /engine/ HTTP/1.1 +Content-Type: application/ssz + + +``` + +The URL path is `/engine/` where `` corresponds to the JSON-RPC method name with the `engine_` prefix removed. For example, `engine_newPayloadV5` maps to `POST /engine/newPayloadV5`. + +The request body is the SSZ serialization of the method's request container. Each method defines a request container that wraps all parameters into a single SSZ object. + +### Response format + +**Success with data:** + +``` +HTTP/1.1 200 OK +Content-Type: application/ssz + + +``` + +**Null result** (e.g., syncing): -The negotiation works as follows: +``` +HTTP/1.1 204 No Content +``` -1. The consensus layer client sends a request with `Accept: application/ssz` to indicate it can handle SSZ-encoded responses. +Methods that can return `null` at the JSON-RPC level use HTTP `204 No Content` with an empty body. -2. If the execution layer client supports SSZ encoding, it **SHOULD** respond with `Content-Type: application/ssz` and an SSZ-encoded body. +**Error:** -3. If the execution layer client does not support SSZ encoding, it **MUST** respond with `Content-Type: application/json` and a JSON-encoded body as usual. The `Accept` header is silently ignored. +``` +HTTP/1.1 +Content-Type: application/ssz + + +``` -4. A client receiving a request with `Content-Type: application/ssz` that does not support SSZ encoding **MUST** respond with HTTP status `415 Unsupported Media Type`. The requesting client **MUST** then fall back to JSON encoding for subsequent requests. +### Negotiation and fallback -5. Clients **MUST** continue to support JSON encoding regardless of SSZ support. SSZ encoding is an optimization, not a replacement. +1. The CL sends a request to the method's URL path with `Content-Type: application/ssz` and a raw SSZ request body. -6. If no `Content-Type` header is present, the request **MUST** be parsed as JSON. If no `Accept` header is present, the response **SHOULD** use the same encoding as the request. +2. If the EL supports the binary SSZ transport, it **MUST** respond with `Content-Type: application/ssz` and a raw SSZ response body. + +3. If the EL does not support the binary SSZ transport, it **MUST** respond with HTTP status `404 Not Found` or `415 Unsupported Media Type`. The CL **MUST** then fall back to JSON-RPC (`POST /`) for subsequent requests. + +4. Clients **MUST** continue to support JSON-RPC encoding as a fallback. Both the binary SSZ endpoint and the JSON-RPC endpoint coexist on the same port. ## Constants @@ -120,6 +162,9 @@ Each JSON-encoded base type used in the Engine API maps to a specific SSZ type. | `bytes` (variable-length) | `ByteList[MAX_LENGTH]` (context-dependent) | | `bytesMax32` (0 to 32 bytes) | `ByteList[32]` | | `Array of T` | `List[T, MAX_LENGTH]` (context-dependent) | +| `T or null` | `Optional[T]` (encoded as `List[T, 1]`) | + +`Optional[T]` is represented as `List[T, 1]` in SSZ encoding. An empty list (0 elements) denotes absence (`null`). A list with one element denotes presence. ## Container definitions @@ -467,418 +512,431 @@ class GetPayloadResponseV6(Container): execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] ``` +### PayloadBodiesV1Response + +Response container for `engine_getPayloadBodiesByHashV1` and `engine_getPayloadBodiesByRangeV1`. + +```python +class PayloadBodiesV1Response(Container): + payload_bodies: List[List[ExecutionPayloadBodyV1, 1], MAX_PAYLOAD_BODIES_REQUEST] +``` + +*Note:* Each inner list has 0 elements for unknown blocks and 1 element for known blocks. + +### PayloadBodiesV2Response + +Response container for `engine_getPayloadBodiesByHashV2` and `engine_getPayloadBodiesByRangeV2`. + +```python +class PayloadBodiesV2Response(Container): + payload_bodies: List[List[ExecutionPayloadBodyV2, 1], MAX_PAYLOAD_BODIES_REQUEST] +``` + +*Note:* Each inner list has 0 elements for unknown blocks and 1 element for known blocks. + +### GetBlobsV1Response + +Response container for `engine_getBlobsV1`. + +```python +class GetBlobsV1Response(Container): + blobs_and_proofs: List[BlobAndProofV1, MAX_BLOB_HASHES_REQUEST] +``` + +### GetBlobsV2Response + +Response container for `engine_getBlobsV2`. + +```python +class GetBlobsV2Response(Container): + blobs_and_proofs: List[BlobAndProofV2, MAX_BLOB_HASHES_REQUEST] +``` + +### GetBlobsV3Response + +Response container for `engine_getBlobsV3`. + +```python +class GetBlobsV3Response(Container): + blobs_and_proofs: List[List[BlobAndProofV2, 1], MAX_BLOB_HASHES_REQUEST] +``` + +*Note:* Each inner list has 0 elements for a missing blob and 1 element for a present blob. + +### ErrorResponse + +Used for error responses across all methods. + +```python +class ErrorResponse(Container): + code: uint64 + message: ByteList[MAX_ERROR_MESSAGE_LENGTH] +``` + +*Note:* Engine API error codes are negative integers in JSON-RPC. The `code` field stores the absolute value. For example, JSON-RPC error code `-38005` is encoded as `38005`. + ## Method definitions -This section defines the SSZ types for each method's parameters and result, organized by fork. Each parameter is individually SSZ-encoded in the JSON-RPC `params` array. Nullable parameters remain `null` when absent. +Each Engine API method has a dedicated URL path, a request container, and a response type. The request body is the SSZ serialization of the request container. The response body is the SSZ serialization of the response type. ### Paris methods #### engine_newPayloadV1 -| Parameter | SSZ Type | -| - | - | -| `executionPayload` | [`ExecutionPayloadV1`](#executionpayloadv1) | +`POST /engine/newPayloadV1` -| Result | SSZ Type | -| - | - | -| Payload status | [`PayloadStatusV1`](#payloadstatusv1) | +```python +class NewPayloadV1Request(Container): + execution_payload: ExecutionPayloadV1 +``` + +**Response:** [`PayloadStatusV1`](#payloadstatusv1) #### engine_forkchoiceUpdatedV1 -| Parameter | SSZ Type | -| - | - | -| `forkchoiceState` | [`ForkchoiceStateV1`](#forkchoicestatev1) | -| `payloadAttributes` | [`PayloadAttributesV1`](#payloadattributesv1) or `null` | +`POST /engine/forkchoiceUpdatedV1` -| Result | SSZ Type | -| - | - | -| Forkchoice updated response | [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) | +```python +class ForkchoiceUpdatedV1Request(Container): + forkchoice_state: ForkchoiceStateV1 + payload_attributes: Optional[PayloadAttributesV1] +``` + +**Response:** [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) #### engine_getPayloadV1 -| Parameter | SSZ Type | -| - | - | -| `payloadId` | `Bytes8` | +`POST /engine/getPayloadV1` -| Result | SSZ Type | -| - | - | -| Execution payload | [`ExecutionPayloadV1`](#executionpayloadv1) | +```python +class GetPayloadV1Request(Container): + payload_id: Bytes8 +``` + +**Response:** [`ExecutionPayloadV1`](#executionpayloadv1) #### engine_exchangeTransitionConfigurationV1 +`POST /engine/exchangeTransitionConfigurationV1` + Deprecated in Cancun. -| Parameter | SSZ Type | -| - | - | -| `transitionConfiguration` | [`TransitionConfigurationV1`](#transitionconfigurationv1) | +```python +class ExchangeTransitionConfigurationV1Request(Container): + transition_configuration: TransitionConfigurationV1 +``` -| Result | SSZ Type | -| - | - | -| Transition configuration | [`TransitionConfigurationV1`](#transitionconfigurationv1) | +**Response:** [`TransitionConfigurationV1`](#transitionconfigurationv1) ### Shanghai methods #### engine_newPayloadV2 -| Parameter | SSZ Type | -| - | - | -| `executionPayload` | [`ExecutionPayloadV1`](#executionpayloadv1) or [`ExecutionPayloadV2`](#executionpayloadv2) (by timestamp) | +`POST /engine/newPayloadV2` -| Result | SSZ Type | -| - | - | -| Payload status | [`PayloadStatusV1`](#payloadstatusv1) | +```python +class NewPayloadV2Request(Container): + execution_payload: ExecutionPayloadV2 +``` + +*Note:* Always uses `ExecutionPayloadV2`. Pre-Shanghai payloads have an empty `withdrawals` list. + +**Response:** [`PayloadStatusV1`](#payloadstatusv1) #### engine_forkchoiceUpdatedV2 -| Parameter | SSZ Type | -| - | - | -| `forkchoiceState` | [`ForkchoiceStateV1`](#forkchoicestatev1) | -| `payloadAttributes` | [`PayloadAttributesV1`](#payloadattributesv1), [`PayloadAttributesV2`](#payloadattributesv2), or `null` | +`POST /engine/forkchoiceUpdatedV2` -| Result | SSZ Type | -| - | - | -| Forkchoice updated response | [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) | +```python +class ForkchoiceUpdatedV2Request(Container): + forkchoice_state: ForkchoiceStateV1 + payload_attributes: Optional[PayloadAttributesV2] +``` + +*Note:* Always uses `PayloadAttributesV2`. Pre-Shanghai attributes have an empty `withdrawals` list. + +**Response:** [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) #### engine_getPayloadV2 -| Parameter | SSZ Type | -| - | - | -| `payloadId` | `Bytes8` | +`POST /engine/getPayloadV2` -| Result | SSZ Type | -| - | - | -| Get payload response | [`GetPayloadResponseV2`](#getpayloadresponsev2) | +```python +class GetPayloadV2Request(Container): + payload_id: Bytes8 +``` + +**Response:** [`GetPayloadResponseV2`](#getpayloadresponsev2) #### engine_getPayloadBodiesByHashV1 -| Parameter | SSZ Type | -| - | - | -| `blockHashes` | `List[Bytes32, MAX_PAYLOAD_BODIES_REQUEST]` | +`POST /engine/getPayloadBodiesByHashV1` -| Result | SSZ Type | -| - | - | -| Payload bodies | `List[List[`[`ExecutionPayloadBodyV1`](#executionpayloadbodyv1)`, 1], MAX_PAYLOAD_BODIES_REQUEST]` | +```python +class GetPayloadBodiesByHashV1Request(Container): + block_hashes: List[Bytes32, MAX_PAYLOAD_BODIES_REQUEST] +``` -*Note:* Each inner list has 0 elements for unknown blocks and 1 element for known blocks. +**Response:** [`PayloadBodiesV1Response`](#payloadbodiesv1response) #### engine_getPayloadBodiesByRangeV1 -| Parameter | SSZ Type | -| - | - | -| `start` | `uint64` | -| `count` | `uint64` | +`POST /engine/getPayloadBodiesByRangeV1` -| Result | SSZ Type | -| - | - | -| Payload bodies | `List[List[`[`ExecutionPayloadBodyV1`](#executionpayloadbodyv1)`, 1], MAX_PAYLOAD_BODIES_REQUEST]` | +```python +class GetPayloadBodiesByRangeV1Request(Container): + start: uint64 + count: uint64 +``` -*Note:* Each inner list has 0 elements for unknown blocks and 1 element for known blocks. +**Response:** [`PayloadBodiesV1Response`](#payloadbodiesv1response) ### Cancun methods #### engine_newPayloadV3 -| Parameter | SSZ Type | -| - | - | -| `executionPayload` | [`ExecutionPayloadV3`](#executionpayloadv3) | -| `expectedBlobVersionedHashes` | `List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK]` | -| `parentBeaconBlockRoot` | `Bytes32` | +`POST /engine/newPayloadV3` -| Result | SSZ Type | -| - | - | -| Payload status | [`PayloadStatusV1`](#payloadstatusv1) | +```python +class NewPayloadV3Request(Container): + execution_payload: ExecutionPayloadV3 + expected_blob_versioned_hashes: List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK] + parent_beacon_block_root: Bytes32 +``` + +**Response:** [`PayloadStatusV1`](#payloadstatusv1) #### engine_forkchoiceUpdatedV3 -| Parameter | SSZ Type | -| - | - | -| `forkchoiceState` | [`ForkchoiceStateV1`](#forkchoicestatev1) | -| `payloadAttributes` | [`PayloadAttributesV3`](#payloadattributesv3) or `null` | +`POST /engine/forkchoiceUpdatedV3` -| Result | SSZ Type | -| - | - | -| Forkchoice updated response | [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) | +```python +class ForkchoiceUpdatedV3Request(Container): + forkchoice_state: ForkchoiceStateV1 + payload_attributes: Optional[PayloadAttributesV3] +``` + +**Response:** [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) #### engine_getPayloadV3 -| Parameter | SSZ Type | -| - | - | -| `payloadId` | `Bytes8` | +`POST /engine/getPayloadV3` -| Result | SSZ Type | -| - | - | -| Get payload response | [`GetPayloadResponseV3`](#getpayloadresponsev3) | +```python +class GetPayloadV3Request(Container): + payload_id: Bytes8 +``` + +**Response:** [`GetPayloadResponseV3`](#getpayloadresponsev3) #### engine_getBlobsV1 -Deprecated in Osaka. +`POST /engine/getBlobsV1` -| Parameter | SSZ Type | -| - | - | -| `blobVersionedHashes` | `List[Bytes32, MAX_BLOB_HASHES_REQUEST]` | +Deprecated in Osaka. -| Result | SSZ Type | -| - | - | -| Blobs and proofs | `List[`[`BlobAndProofV1`](#blobandproofv1)`, MAX_BLOB_HASHES_REQUEST]` | +```python +class GetBlobsV1Request(Container): + blob_versioned_hashes: List[Bytes32, MAX_BLOB_HASHES_REQUEST] +``` -*Note:* Returns `null` at the JSON-RPC level when syncing (SSZ encoding only applies to non-null results). +**Response:** [`GetBlobsV1Response`](#getblobsv1response) or HTTP `204 No Content` when syncing. ### Prague methods #### engine_newPayloadV4 -| Parameter | SSZ Type | -| - | - | -| `executionPayload` | [`ExecutionPayloadV3`](#executionpayloadv3) | -| `expectedBlobVersionedHashes` | `List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK]` | -| `parentBeaconBlockRoot` | `Bytes32` | -| `executionRequests` | `List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS]` | +`POST /engine/newPayloadV4` -| Result | SSZ Type | -| - | - | -| Payload status | [`PayloadStatusV1`](#payloadstatusv1) | +```python +class NewPayloadV4Request(Container): + execution_payload: ExecutionPayloadV3 + expected_blob_versioned_hashes: List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK] + parent_beacon_block_root: Bytes32 + execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] +``` + +**Response:** [`PayloadStatusV1`](#payloadstatusv1) #### engine_getPayloadV4 -| Parameter | SSZ Type | -| - | - | -| `payloadId` | `Bytes8` | +`POST /engine/getPayloadV4` -| Result | SSZ Type | -| - | - | -| Get payload response | [`GetPayloadResponseV4`](#getpayloadresponsev4) | +```python +class GetPayloadV4Request(Container): + payload_id: Bytes8 +``` + +**Response:** [`GetPayloadResponseV4`](#getpayloadresponsev4) ### Osaka methods #### engine_getPayloadV5 -| Parameter | SSZ Type | -| - | - | -| `payloadId` | `Bytes8` | +`POST /engine/getPayloadV5` -| Result | SSZ Type | -| - | - | -| Get payload response | [`GetPayloadResponseV5`](#getpayloadresponsev5) | +```python +class GetPayloadV5Request(Container): + payload_id: Bytes8 +``` -#### engine_getBlobsV2 +**Response:** [`GetPayloadResponseV5`](#getpayloadresponsev5) -Returns `null` for the entire result if any blob is missing or if syncing. +#### engine_getBlobsV2 -| Parameter | SSZ Type | -| - | - | -| `blobVersionedHashes` | `List[Bytes32, MAX_BLOB_HASHES_REQUEST]` | +`POST /engine/getBlobsV2` -| Result | SSZ Type | -| - | - | -| Blobs and proofs | `List[`[`BlobAndProofV2`](#blobandproofv2)`, MAX_BLOB_HASHES_REQUEST]` | +```python +class GetBlobsV2Request(Container): + blob_versioned_hashes: List[Bytes32, MAX_BLOB_HASHES_REQUEST] +``` -*Note:* Returns `null` at the JSON-RPC level when syncing or any blob is missing (SSZ encoding only applies to non-null results). +**Response:** [`GetBlobsV2Response`](#getblobsv2response) or HTTP `204 No Content` when syncing or any blob is missing. #### engine_getBlobsV3 -Returns per-element `null` for missing blobs, or `null` for the entire result if syncing. +`POST /engine/getBlobsV3` -| Parameter | SSZ Type | -| - | - | -| `blobVersionedHashes` | `List[Bytes32, MAX_BLOB_HASHES_REQUEST]` | - -| Result | SSZ Type | -| - | - | -| Blobs and proofs | `List[List[`[`BlobAndProofV2`](#blobandproofv2)`, 1], MAX_BLOB_HASHES_REQUEST]` | +```python +class GetBlobsV3Request(Container): + blob_versioned_hashes: List[Bytes32, MAX_BLOB_HASHES_REQUEST] +``` -*Note:* Returns `null` at the JSON-RPC level when syncing. Each inner list has 0 elements for a missing blob and 1 element for a present blob. +**Response:** [`GetBlobsV3Response`](#getblobsv3response) or HTTP `204 No Content` when syncing. ### Amsterdam methods #### engine_newPayloadV5 -| Parameter | SSZ Type | -| - | - | -| `executionPayload` | [`ExecutionPayloadV4`](#executionpayloadv4) | -| `expectedBlobVersionedHashes` | `List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK]` | -| `parentBeaconBlockRoot` | `Bytes32` | -| `executionRequests` | `List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS]` | +`POST /engine/newPayloadV5` -| Result | SSZ Type | -| - | - | -| Payload status | [`PayloadStatusV1`](#payloadstatusv1) | +```python +class NewPayloadV5Request(Container): + execution_payload: ExecutionPayloadV4 + expected_blob_versioned_hashes: List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK] + parent_beacon_block_root: Bytes32 + execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] +``` + +**Response:** [`PayloadStatusV1`](#payloadstatusv1) #### engine_getPayloadV6 -| Parameter | SSZ Type | -| - | - | -| `payloadId` | `Bytes8` | +`POST /engine/getPayloadV6` -| Result | SSZ Type | -| - | - | -| Get payload response | [`GetPayloadResponseV6`](#getpayloadresponsev6) | +```python +class GetPayloadV6Request(Container): + payload_id: Bytes8 +``` + +**Response:** [`GetPayloadResponseV6`](#getpayloadresponsev6) #### engine_forkchoiceUpdatedV4 -| Parameter | SSZ Type | -| - | - | -| `forkchoiceState` | [`ForkchoiceStateV1`](#forkchoicestatev1) | -| `payloadAttributes` | [`PayloadAttributesV4`](#payloadattributesv4) or `null` | +`POST /engine/forkchoiceUpdatedV4` -| Result | SSZ Type | -| - | - | -| Forkchoice updated response | [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) | +```python +class ForkchoiceUpdatedV4Request(Container): + forkchoice_state: ForkchoiceStateV1 + payload_attributes: Optional[PayloadAttributesV4] +``` + +**Response:** [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) #### engine_getPayloadBodiesByHashV2 -| Parameter | SSZ Type | -| - | - | -| `blockHashes` | `List[Bytes32, MAX_PAYLOAD_BODIES_REQUEST]` | +`POST /engine/getPayloadBodiesByHashV2` -| Result | SSZ Type | -| - | - | -| Payload bodies | `List[List[`[`ExecutionPayloadBodyV2`](#executionpayloadbodyv2)`, 1], MAX_PAYLOAD_BODIES_REQUEST]` | +```python +class GetPayloadBodiesByHashV2Request(Container): + block_hashes: List[Bytes32, MAX_PAYLOAD_BODIES_REQUEST] +``` -*Note:* Each inner list has 0 elements for unknown blocks and 1 element for known blocks. +**Response:** [`PayloadBodiesV2Response`](#payloadbodiesv2response) #### engine_getPayloadBodiesByRangeV2 -| Parameter | SSZ Type | -| - | - | -| `start` | `uint64` | -| `count` | `uint64` | +`POST /engine/getPayloadBodiesByRangeV2` -| Result | SSZ Type | -| - | - | -| Payload bodies | `List[List[`[`ExecutionPayloadBodyV2`](#executionpayloadbodyv2)`, 1], MAX_PAYLOAD_BODIES_REQUEST]` | +```python +class GetPayloadBodiesByRangeV2Request(Container): + start: uint64 + count: uint64 +``` -*Note:* Each inner list has 0 elements for unknown blocks and 1 element for known blocks. +**Response:** [`PayloadBodiesV2Response`](#payloadbodiesv2response) -## Request and response format +## Example -SSZ-encoded Engine API requests and responses follow the existing JSON-RPC method semantics. The SSZ encoding applies to the method parameters and result values — the JSON-RPC envelope (`jsonrpc`, `id`, `method`) remains JSON-encoded. +The following example shows an `engine_newPayloadV5` call using the binary SSZ transport. -Specifically, when SSZ encoding is in use: +### Request -1. The HTTP request body is a JSON-RPC request where each element of `params` is replaced with its SSZ-encoded hexadecimal representation (a `DATA` string). Parameters that are `null` remain `null`. +``` +POST /engine/newPayloadV5 HTTP/1.1 +Host: localhost:8551 +Content-Type: application/ssz +Content-Length: 604 -2. The HTTP response body is a JSON-RPC response where the `result` field is replaced with the SSZ-encoded hexadecimal representation of the result value. A `null` result remains `null`. +<604 bytes: SSZ(NewPayloadV5Request)> +``` -This approach preserves compatibility with JSON-RPC tooling while encoding the payload data in SSZ. +The request body is the SSZ serialization of `NewPayloadV5Request` containing: +- `execution_payload`: an `ExecutionPayloadV4` with empty transactions, withdrawals, and block access list +- `expected_blob_versioned_hashes`: empty list +- `parent_beacon_block_root`: `0x0000000000000000000000000000000000000000000000000000000000000000` +- `execution_requests`: empty list -*Note:* Future versions of this specification may define a fully binary request/response format that replaces the JSON-RPC envelope. +### Response (success) -## Example +``` +HTTP/1.1 200 OK +Content-Type: application/ssz +Content-Length: 69 -The following example shows an `engine_newPayloadV5` call, first using JSON encoding and then using SSZ encoding. - -### JSON-encoded request (current behavior) - -```console -$ curl https://localhost:8551 \ - -X POST \ - -H "Content-Type: application/json" \ - -d '{ - "jsonrpc": "2.0", - "id": 1, - "method": "engine_newPayloadV5", - "params": [ - { - "parentHash": "0x3b8fb240d288781d4aac94d3fd16809ee413bc99294a085798a589dae51ddd4a", - "feeRecipient": "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b", - "stateRoot": "0xca3149fa9e37db08d1cd49c9061db1002ef1cd58db2210f2115c8c989b2bdf45", - "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421", - "logsBloom": "0x0000...0000", - "prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000", - "blockNumber": "0x1", - "gasLimit": "0x1c9c380", - "gasUsed": "0x0", - "timestamp": "0x5", - "extraData": "0x", - "baseFeePerGas": "0x7", - "blockHash": "0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858", - "transactions": [], - "withdrawals": [], - "blobGasUsed": "0x0", - "excessBlobGas": "0x0", - "blockAccessList": "0x", - "slotNumber": "0x1" - }, - [], - "0x0000000000000000000000000000000000000000000000000000000000000000", - [] - ] -}' -``` - -### JSON-encoded response - -```json -{ - "jsonrpc": "2.0", - "id": 1, - "result": { - "status": "VALID", - "latestValidHash": "0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858", - "validationError": null - } -} -``` - -### SSZ-encoded request - -The consensus layer client sends the same call with `Content-Type: application/ssz` and `Accept: application/ssz`. Each element of the `params` array is individually SSZ-encoded as a hex `DATA` string: - -```json -{ - "jsonrpc": "2.0", - "id": 1, - "method": "engine_newPayloadV5", - "params": [ - "0x3b8fb240d288781d4aac94d3fd16809ee413bc99...", - "0x", - "0x0000000000000000000000000000000000000000000000000000000000000000", - "0x" - ] -} -``` - -- `params[0]`: SSZ-serialized `ExecutionPayloadV4` container -- `params[1]`: SSZ-serialized `List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK]` (empty list) -- `params[2]`: SSZ-serialized `Bytes32` (parent beacon block root) -- `params[3]`: SSZ-serialized `List[ByteList, MAX_EXECUTION_REQUESTS]` (empty list) - -### SSZ-encoded response - -The execution layer responds with `Content-Type: application/ssz`. The `result` field contains the SSZ-serialized `PayloadStatusV1`: - -```json -{ - "jsonrpc": "2.0", - "id": 1, - "result": "0x00013559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858" -} -``` - -Where the binary data encodes: -- `status`: `0x00` (VALID) -- `latest_valid_hash`: present, `0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858` -- `validation_error`: absent +<69 bytes: SSZ(PayloadStatusV1)> +``` -### Fallback behavior +The response body is the SSZ serialization of `PayloadStatusV1` containing: +- `status`: `0` (VALID) +- `latest_valid_hash`: `0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858` +- `validation_error`: empty + +### Response (error) -If the execution layer does not support SSZ, the same request with `Accept: application/ssz` returns a standard JSON response with `Content-Type: application/json`. The consensus layer detects this and continues using JSON for subsequent requests. +``` +HTTP/1.1 400 Bad Request +Content-Type: application/ssz +Content-Length: 48 + +<48 bytes: SSZ(ErrorResponse)> +``` + +The response body is the SSZ serialization of `ErrorResponse` containing: +- `code`: `32602` (absolute value of `-32602`, invalid params) +- `message`: `"Invalid execution payload"` + +### Fallback behavior -If the consensus layer sends `Content-Type: application/ssz` to an execution layer that does not support it, the execution layer responds with HTTP `415 Unsupported Media Type`. The consensus layer **MUST** retry the request using JSON encoding. +If the EL does not support the binary SSZ transport, a request to `/engine/newPayloadV5` returns HTTP `404 Not Found` or `415 Unsupported Media Type`. The CL detects this and falls back to JSON-RPC at `POST /` for subsequent requests. ## Error handling -SSZ encoding does not change the error semantics of the Engine API. All error codes defined in the [Errors](./common.md#errors) section apply equally to SSZ-encoded requests. +Binary SSZ does not change the error semantics of the Engine API. All error codes defined in the [Errors](./common.md#errors) section apply equally. -Additionally: +Error responses use the [`ErrorResponse`](#errorresponse) container. The HTTP status code reflects the error category: -| Code | Message | Meaning | +| HTTP Status | Meaning | Engine API Errors | | - | - | - | -| -32700 | Parse error | Invalid SSZ data was received by the server. | +| `400` | Client error | `-32700` (parse error), `-32600` (invalid request), `-32602` (invalid params) | +| `404` | Method not found | `-32601` (method not found) | +| `415` | Unsupported media type | Binary SSZ not supported | +| `500` | Server error | `-32603` (internal error), `-38001` to `-38005` (engine-specific errors) | -Clients **MUST** validate SSZ payloads against the expected schema before processing. Payloads that do not conform to the expected SSZ schema **MUST** be rejected with a `-32700` error. +Clients **MUST** validate SSZ payloads against the expected schema before processing. Payloads that do not conform to the expected SSZ schema **MUST** be rejected with a `400` response containing an `ErrorResponse` with code `32700`. ## Security considerations - SSZ deserialization **MUST** enforce the same size limits as JSON deserialization. Implementations **MUST** reject SSZ payloads exceeding defined maximum sizes before attempting full deserialization. - Implementations **SHOULD** use well-tested SSZ libraries and fuzz test SSZ parsing extensively. +- The binary transport uses the same JWT authentication as the JSON-RPC endpoint. All existing authentication requirements apply. From 1228e8359e10c703a4db3720b9ceaf40d3b25402 Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Wed, 4 Mar 2026 20:55:30 +0100 Subject: [PATCH 06/12] add rest --- src/engine/common.md | 2 +- src/engine/ssz-encoding.md | 562 +++++++++++++++++++------------------ 2 files changed, 291 insertions(+), 273 deletions(-) diff --git a/src/engine/common.md b/src/engine/common.md index bddb7ce00..3acfe14bc 100644 --- a/src/engine/common.md +++ b/src/engine/common.md @@ -139,7 +139,7 @@ Values of a field of `QUANTITY` type **MUST** be encoded as a hexadecimal string ### Binary SSZ transport -Clients **MAY** support a binary SSZ transport as an alternative to JSON-RPC. The binary transport uses raw SSZ bytes over HTTP with path-based method routing, eliminating JSON and hex-encoding overhead for fast CL-EL communication. +Clients **MAY** support a binary SSZ transport as an alternative to JSON-RPC. The binary transport uses resource-oriented REST endpoints with raw SSZ request and response bodies (`application/octet-stream`), eliminating JSON and hex-encoding overhead for fast CL-EL communication. Endpoints follow Beacon API conventions with path-based versioning (e.g., `POST /engine/v5/payloads`). When both the consensus layer and execution layer clients support the binary SSZ transport, they **SHOULD** use it. When either client does not support it, both **MUST** fall back to JSON-RPC encoding. diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md index 426b7329b..a06b73633 100644 --- a/src/engine/ssz-encoding.md +++ b/src/engine/ssz-encoding.md @@ -1,6 +1,6 @@ # Engine API -- Binary SSZ Transport -This document specifies a binary SSZ transport for Engine API communication between consensus layer (CL) and execution layer (EL) clients. The binary transport replaces JSON-RPC with raw SSZ over HTTP for fast, efficient CL-EL communication. +This document specifies a binary SSZ transport for Engine API communication between consensus layer (CL) and execution layer (EL) clients. The binary transport replaces JSON-RPC with resource-oriented REST and raw SSZ encoding for fast, efficient CL-EL communication. SSZ container definitions are provided for all Engine API structures and methods across all forks for backwards compatibility. @@ -11,9 +11,12 @@ SSZ container definitions are provided for all Engine API structures and methods - [Motivation](#motivation) - [Transport](#transport) - - [Request format](#request-format) - - [Response format](#response-format) + - [Base URL](#base-url) + - [Content types](#content-types) + - [Authentication](#authentication) + - [Versioning](#versioning) - [Negotiation and fallback](#negotiation-and-fallback) +- [HTTP status codes](#http-status-codes) - [Constants](#constants) - [SSZ type mappings](#ssz-type-mappings) - [Container definitions](#container-definitions) @@ -46,16 +49,13 @@ SSZ container definitions are provided for all Engine API structures and methods - [GetBlobsV1Response](#getblobsv1response) - [GetBlobsV2Response](#getblobsv2response) - [GetBlobsV3Response](#getblobsv3response) - - [ErrorResponse](#errorresponse) -- [Method definitions](#method-definitions) - - [Paris methods](#paris-methods) - - [Shanghai methods](#shanghai-methods) - - [Cancun methods](#cancun-methods) - - [Prague methods](#prague-methods) - - [Osaka methods](#osaka-methods) - - [Amsterdam methods](#amsterdam-methods) +- [Endpoints](#endpoints) + - [Payloads](#payloads) + - [Forkchoice](#forkchoice) + - [Blobs](#blobs) + - [Transition configuration](#transition-configuration) + - [Endpoint summary](#endpoint-summary) - [Example](#example) -- [Error handling](#error-handling) - [Security considerations](#security-considerations) @@ -72,58 +72,88 @@ Binary SSZ eliminates all of this. The CL sends raw SSZ bytes over HTTP; the EL ## Transport -Binary SSZ uses HTTP with path-based method routing. Each Engine API method has a dedicated URL path. Request and response bodies are raw SSZ bytes. +The binary SSZ transport uses resource-oriented REST over HTTP. Endpoints are organized by resource type (payloads, forkchoice, blobs) with per-endpoint versioning, following the same conventions as the [Beacon API](https://github.com/ethereum/beacon-APIs). -### Request format +### Base URL ``` -POST /engine/ HTTP/1.1 -Content-Type: application/ssz - - +http://localhost:8551/engine ``` -The URL path is `/engine/` where `` corresponds to the JSON-RPC method name with the `engine_` prefix removed. For example, `engine_newPayloadV5` maps to `POST /engine/newPayloadV5`. - -The request body is the SSZ serialization of the method's request container. Each method defines a request container that wraps all parameters into a single SSZ object. +All endpoints are served under the `/engine` prefix on the existing Engine API port (8551). -### Response format +### Content types -**Success with data:** +| Direction | Content-Type | Description | +| - | - | - | +| Request body | `application/octet-stream` | SSZ-encoded request container | +| Response body (success) | `application/octet-stream` | SSZ-encoded response container | +| Response body (error) | `text/plain` | Human-readable error message | -``` -HTTP/1.1 200 OK -Content-Type: application/ssz +Request bodies are the SSZ serialization of the endpoint's request container. Response bodies are the SSZ serialization of the endpoint's response type. - -``` +### Authentication -**Null result** (e.g., syncing): +The binary transport uses the same JWT authentication as the JSON-RPC endpoint. All requests **MUST** include a valid JWT bearer token in the `Authorization` header: ``` -HTTP/1.1 204 No Content +Authorization: Bearer ``` -Methods that can return `null` at the JSON-RPC level use HTTP `204 No Content` with an empty body. +All existing authentication requirements from the [Engine API specification](./common.md#authentication) apply. -**Error:** +### Versioning -``` -HTTP/1.1 -Content-Type: application/ssz +Endpoints use path-based versioning following [Beacon API](https://github.com/ethereum/beacon-APIs) conventions. Each endpoint includes a version number in its path (e.g., `/engine/v5/payloads`). The version number corresponds to the JSON-RPC method version it replaces: - -``` +| REST Endpoint | JSON-RPC Equivalent | +| - | - | +| `POST /engine/v5/payloads` | `engine_newPayloadV5` | +| `GET /engine/v6/payloads/{payload_id}` | `engine_getPayloadV6` | +| `POST /engine/v4/forkchoice` | `engine_forkchoiceUpdatedV4` | +| `POST /engine/v3/blobs` | `engine_getBlobsV3` | + +When a new fork introduces a new method version, a new versioned endpoint is added. Older versioned endpoints **MAY** be deprecated but **SHOULD** remain available for backwards compatibility. ### Negotiation and fallback -1. The CL sends a request to the method's URL path with `Content-Type: application/ssz` and a raw SSZ request body. +1. The CL sends a request to a versioned REST endpoint with `Content-Type: application/octet-stream` and a raw SSZ request body. -2. If the EL supports the binary SSZ transport, it **MUST** respond with `Content-Type: application/ssz` and a raw SSZ response body. +2. If the EL supports the binary SSZ transport, it **MUST** respond with `Content-Type: application/octet-stream` and a raw SSZ response body. 3. If the EL does not support the binary SSZ transport, it **MUST** respond with HTTP status `404 Not Found` or `415 Unsupported Media Type`. The CL **MUST** then fall back to JSON-RPC (`POST /`) for subsequent requests. -4. Clients **MUST** continue to support JSON-RPC encoding as a fallback. Both the binary SSZ endpoint and the JSON-RPC endpoint coexist on the same port. +4. Clients **MUST** continue to support JSON-RPC encoding as a fallback. Both the REST endpoints and the JSON-RPC endpoint coexist on the same port. + +## HTTP status codes + +### Success + +| Status | Meaning | Usage | +| - | - | - | +| `200` | OK | Request succeeded, response body contains SSZ-encoded result | +| `204` | No Content | Null result (e.g., syncing), empty body | + +### Client errors + +| Status | Meaning | Usage | +| - | - | - | +| `400` | Bad Request | Malformed SSZ, invalid parameters | +| `401` | Unauthorized | Missing or invalid JWT token | +| `404` | Not Found | Unknown payload ID, unsupported endpoint | +| `409` | Conflict | Invalid forkchoice state (e.g., finalized block not ancestor of head) | +| `413` | Request Too Large | Request exceeds maximum size limits | +| `415` | Unsupported Media Type | Binary SSZ transport not supported | +| `422` | Unprocessable Entity | Invalid payload attributes (e.g., timestamp not greater than parent) | + +### Server errors + +| Status | Meaning | Usage | +| - | - | - | +| `500` | Internal Server Error | Unexpected server error | +| `501` | Not Implemented | Unsupported fork version | + +Error responses use `Content-Type: text/plain` with a human-readable error message body. ## Constants @@ -563,319 +593,307 @@ class GetBlobsV3Response(Container): *Note:* Each inner list has 0 elements for a missing blob and 1 element for a present blob. -### ErrorResponse - -Used for error responses across all methods. - -```python -class ErrorResponse(Container): - code: uint64 - message: ByteList[MAX_ERROR_MESSAGE_LENGTH] -``` +## Endpoints -*Note:* Engine API error codes are negative integers in JSON-RPC. The `code` field stores the absolute value. For example, JSON-RPC error code `-38005` is encoded as `38005`. +All endpoints use `Content-Type: application/octet-stream` for request and response bodies containing SSZ-encoded data. Error responses use `Content-Type: text/plain`. -## Method definitions +### Payloads -Each Engine API method has a dedicated URL path, a request container, and a response type. The request body is the SSZ serialization of the request container. The response body is the SSZ serialization of the response type. +#### `POST /engine/v{N}/payloads` — Submit execution payload -### Paris methods +Submit an execution payload for validation. The EL validates the payload and returns its status. -#### engine_newPayloadV1 +| Version | Fork | Request Container | JSON-RPC Equivalent | +| - | - | - | - | +| v1 | Paris | `NewPayloadV1Request` | `engine_newPayloadV1` | +| v2 | Shanghai | `NewPayloadV2Request` | `engine_newPayloadV2` | +| v3 | Cancun | `NewPayloadV3Request` | `engine_newPayloadV3` | +| v4 | Prague | `NewPayloadV4Request` | `engine_newPayloadV4` | +| v5 | Amsterdam | `NewPayloadV5Request` | `engine_newPayloadV5` | -`POST /engine/newPayloadV1` +**Request containers:** ```python class NewPayloadV1Request(Container): execution_payload: ExecutionPayloadV1 -``` -**Response:** [`PayloadStatusV1`](#payloadstatusv1) +class NewPayloadV2Request(Container): + execution_payload: ExecutionPayloadV2 -#### engine_forkchoiceUpdatedV1 +class NewPayloadV3Request(Container): + execution_payload: ExecutionPayloadV3 + expected_blob_versioned_hashes: List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK] + parent_beacon_block_root: Bytes32 -`POST /engine/forkchoiceUpdatedV1` +class NewPayloadV4Request(Container): + execution_payload: ExecutionPayloadV3 + expected_blob_versioned_hashes: List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK] + parent_beacon_block_root: Bytes32 + execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] -```python -class ForkchoiceUpdatedV1Request(Container): - forkchoice_state: ForkchoiceStateV1 - payload_attributes: Optional[PayloadAttributesV1] +class NewPayloadV5Request(Container): + execution_payload: ExecutionPayloadV4 + expected_blob_versioned_hashes: List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK] + parent_beacon_block_root: Bytes32 + execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] ``` -**Response:** [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) +*Note:* `NewPayloadV2Request` always uses `ExecutionPayloadV2`. Pre-Shanghai payloads have an empty `withdrawals` list. -#### engine_getPayloadV1 +**Response:** `200 OK` — [`PayloadStatusV1`](#payloadstatusv1) -`POST /engine/getPayloadV1` +**Errors:** -```python -class GetPayloadV1Request(Container): - payload_id: Bytes8 -``` +| Status | Condition | +| - | - | +| `400` | Malformed SSZ | +| `500` | Internal server error | -**Response:** [`ExecutionPayloadV1`](#executionpayloadv1) +--- -#### engine_exchangeTransitionConfigurationV1 +#### `GET /engine/v{N}/payloads/{payload_id}` — Retrieve built payload -`POST /engine/exchangeTransitionConfigurationV1` +Retrieve an execution payload previously requested via forkchoice update with payload attributes. The `{payload_id}` path parameter is the hex-encoded `Bytes8` payload identifier (e.g., `0x1234567890abcdef`). -Deprecated in Cancun. +This is a safe, idempotent GET operation. The EL may continue optimizing the payload until the slot deadline. -```python -class ExchangeTransitionConfigurationV1Request(Container): - transition_configuration: TransitionConfigurationV1 -``` +| Version | Fork | Response Type | JSON-RPC Equivalent | +| - | - | - | - | +| v1 | Paris | `ExecutionPayloadV1` | `engine_getPayloadV1` | +| v2 | Shanghai | `GetPayloadResponseV2` | `engine_getPayloadV2` | +| v3 | Cancun | `GetPayloadResponseV3` | `engine_getPayloadV3` | +| v4 | Prague | `GetPayloadResponseV4` | `engine_getPayloadV4` | +| v5 | Osaka | `GetPayloadResponseV5` | `engine_getPayloadV5` | +| v6 | Amsterdam | `GetPayloadResponseV6` | `engine_getPayloadV6` | -**Response:** [`TransitionConfigurationV1`](#transitionconfigurationv1) +**Request:** No body. The payload ID is in the URL path. -### Shanghai methods +**Response:** `200 OK` — SSZ-encoded response type from the table above. -#### engine_newPayloadV2 +**Errors:** -`POST /engine/newPayloadV2` +| Status | Condition | +| - | - | +| `400` | Invalid payload ID format | +| `404` | Unknown payload ID | +| `500` | Internal server error | -```python -class NewPayloadV2Request(Container): - execution_payload: ExecutionPayloadV2 -``` +--- -*Note:* Always uses `ExecutionPayloadV2`. Pre-Shanghai payloads have an empty `withdrawals` list. +#### `POST /engine/v{N}/payloads/bodies/by-hash` — Get payload bodies by hash -**Response:** [`PayloadStatusV1`](#payloadstatusv1) +Retrieve execution payload bodies for a list of block hashes. Used for historical sync and block reconstruction. -#### engine_forkchoiceUpdatedV2 +| Version | Fork | Request Container | Response Type | JSON-RPC Equivalent | +| - | - | - | - | - | +| v1 | Shanghai | `GetPayloadBodiesByHashV1Request` | `PayloadBodiesV1Response` | `engine_getPayloadBodiesByHashV1` | +| v2 | Amsterdam | `GetPayloadBodiesByHashV2Request` | `PayloadBodiesV2Response` | `engine_getPayloadBodiesByHashV2` | -`POST /engine/forkchoiceUpdatedV2` +**Request containers:** ```python -class ForkchoiceUpdatedV2Request(Container): - forkchoice_state: ForkchoiceStateV1 - payload_attributes: Optional[PayloadAttributesV2] -``` - -*Note:* Always uses `PayloadAttributesV2`. Pre-Shanghai attributes have an empty `withdrawals` list. - -**Response:** [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) - -#### engine_getPayloadV2 - -`POST /engine/getPayloadV2` +class GetPayloadBodiesByHashV1Request(Container): + block_hashes: List[Bytes32, MAX_PAYLOAD_BODIES_REQUEST] -```python -class GetPayloadV2Request(Container): - payload_id: Bytes8 +class GetPayloadBodiesByHashV2Request(Container): + block_hashes: List[Bytes32, MAX_PAYLOAD_BODIES_REQUEST] ``` -**Response:** [`GetPayloadResponseV2`](#getpayloadresponsev2) +**Response:** `200 OK` — [`PayloadBodiesV1Response`](#payloadbodiesv1response) or [`PayloadBodiesV2Response`](#payloadbodiesv2response) -#### engine_getPayloadBodiesByHashV1 +**Errors:** -`POST /engine/getPayloadBodiesByHashV1` +| Status | Condition | +| - | - | +| `400` | Malformed SSZ | +| `413` | Request exceeds `MAX_PAYLOAD_BODIES_REQUEST` hashes | +| `500` | Internal server error | -```python -class GetPayloadBodiesByHashV1Request(Container): - block_hashes: List[Bytes32, MAX_PAYLOAD_BODIES_REQUEST] -``` +--- -**Response:** [`PayloadBodiesV1Response`](#payloadbodiesv1response) +#### `POST /engine/v{N}/payloads/bodies/by-range` — Get payload bodies by range -#### engine_getPayloadBodiesByRangeV1 +Retrieve execution payload bodies for a contiguous range of block numbers. -`POST /engine/getPayloadBodiesByRangeV1` +| Version | Fork | Request Container | Response Type | JSON-RPC Equivalent | +| - | - | - | - | - | +| v1 | Shanghai | `GetPayloadBodiesByRangeV1Request` | `PayloadBodiesV1Response` | `engine_getPayloadBodiesByRangeV1` | +| v2 | Amsterdam | `GetPayloadBodiesByRangeV2Request` | `PayloadBodiesV2Response` | `engine_getPayloadBodiesByRangeV2` | + +**Request containers:** ```python class GetPayloadBodiesByRangeV1Request(Container): start: uint64 count: uint64 -``` - -**Response:** [`PayloadBodiesV1Response`](#payloadbodiesv1response) - -### Cancun methods - -#### engine_newPayloadV3 - -`POST /engine/newPayloadV3` -```python -class NewPayloadV3Request(Container): - execution_payload: ExecutionPayloadV3 - expected_blob_versioned_hashes: List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK] - parent_beacon_block_root: Bytes32 +class GetPayloadBodiesByRangeV2Request(Container): + start: uint64 + count: uint64 ``` -**Response:** [`PayloadStatusV1`](#payloadstatusv1) - -#### engine_forkchoiceUpdatedV3 - -`POST /engine/forkchoiceUpdatedV3` - -```python -class ForkchoiceUpdatedV3Request(Container): - forkchoice_state: ForkchoiceStateV1 - payload_attributes: Optional[PayloadAttributesV3] -``` +**Response:** `200 OK` — [`PayloadBodiesV1Response`](#payloadbodiesv1response) or [`PayloadBodiesV2Response`](#payloadbodiesv2response) -**Response:** [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) +**Errors:** -#### engine_getPayloadV3 +| Status | Condition | +| - | - | +| `400` | Malformed SSZ, `start` < 1, `count` < 1 | +| `413` | `count` exceeds `MAX_PAYLOAD_BODIES_REQUEST` | +| `500` | Internal server error | -`POST /engine/getPayloadV3` +### Forkchoice -```python -class GetPayloadV3Request(Container): - payload_id: Bytes8 -``` +#### `POST /engine/v{N}/forkchoice` — Update fork choice -**Response:** [`GetPayloadResponseV3`](#getpayloadresponsev3) +Update the EL's fork choice state and optionally start building a new payload. The EL updates its canonical chain view and prunes blocks no longer reachable from the head. -#### engine_getBlobsV1 +When `payload_attributes` is present (non-empty `Optional`), the EL begins building a new block. The returned `payload_id` can be used with `GET /engine/v{N}/payloads/{payload_id}` to retrieve the built payload. -`POST /engine/getBlobsV1` +| Version | Fork | Request Container | JSON-RPC Equivalent | +| - | - | - | - | +| v1 | Paris | `ForkchoiceUpdatedV1Request` | `engine_forkchoiceUpdatedV1` | +| v2 | Shanghai | `ForkchoiceUpdatedV2Request` | `engine_forkchoiceUpdatedV2` | +| v3 | Cancun | `ForkchoiceUpdatedV3Request` | `engine_forkchoiceUpdatedV3` | +| v4 | Amsterdam | `ForkchoiceUpdatedV4Request` | `engine_forkchoiceUpdatedV4` | -Deprecated in Osaka. +**Request containers:** ```python -class GetBlobsV1Request(Container): - blob_versioned_hashes: List[Bytes32, MAX_BLOB_HASHES_REQUEST] -``` - -**Response:** [`GetBlobsV1Response`](#getblobsv1response) or HTTP `204 No Content` when syncing. - -### Prague methods +class ForkchoiceUpdatedV1Request(Container): + forkchoice_state: ForkchoiceStateV1 + payload_attributes: Optional[PayloadAttributesV1] -#### engine_newPayloadV4 +class ForkchoiceUpdatedV2Request(Container): + forkchoice_state: ForkchoiceStateV1 + payload_attributes: Optional[PayloadAttributesV2] -`POST /engine/newPayloadV4` +class ForkchoiceUpdatedV3Request(Container): + forkchoice_state: ForkchoiceStateV1 + payload_attributes: Optional[PayloadAttributesV3] -```python -class NewPayloadV4Request(Container): - execution_payload: ExecutionPayloadV3 - expected_blob_versioned_hashes: List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK] - parent_beacon_block_root: Bytes32 - execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] +class ForkchoiceUpdatedV4Request(Container): + forkchoice_state: ForkchoiceStateV1 + payload_attributes: Optional[PayloadAttributesV4] ``` -**Response:** [`PayloadStatusV1`](#payloadstatusv1) - -#### engine_getPayloadV4 - -`POST /engine/getPayloadV4` - -```python -class GetPayloadV4Request(Container): - payload_id: Bytes8 -``` +*Note:* `ForkchoiceUpdatedV2Request` always uses `PayloadAttributesV2`. Pre-Shanghai attributes have an empty `withdrawals` list. -**Response:** [`GetPayloadResponseV4`](#getpayloadresponsev4) +**Response:** `200 OK` — [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) -### Osaka methods +**Errors:** -#### engine_getPayloadV5 +| Status | Condition | +| - | - | +| `400` | Malformed SSZ | +| `409` | Invalid forkchoice state (e.g., finalized block not ancestor of head) | +| `422` | Invalid payload attributes (e.g., timestamp not greater than parent) | +| `500` | Internal server error | -`POST /engine/getPayloadV5` +### Blobs -```python -class GetPayloadV5Request(Container): - payload_id: Bytes8 -``` +#### `POST /engine/v{N}/blobs` — Get blobs by versioned hash -**Response:** [`GetPayloadResponseV5`](#getpayloadresponsev5) +Retrieve blobs from the EL's blob pool by their versioned hashes. -#### engine_getBlobsV2 +| Version | Fork | Request Container | Response Type | JSON-RPC Equivalent | +| - | - | - | - | - | +| v1 | Cancun | `GetBlobsV1Request` | `GetBlobsV1Response` | `engine_getBlobsV1` | +| v2 | Osaka | `GetBlobsV2Request` | `GetBlobsV2Response` | `engine_getBlobsV2` | +| v3 | Osaka | `GetBlobsV3Request` | `GetBlobsV3Response` | `engine_getBlobsV3` | -`POST /engine/getBlobsV2` +**Request containers:** ```python -class GetBlobsV2Request(Container): +class GetBlobsV1Request(Container): blob_versioned_hashes: List[Bytes32, MAX_BLOB_HASHES_REQUEST] -``` -**Response:** [`GetBlobsV2Response`](#getblobsv2response) or HTTP `204 No Content` when syncing or any blob is missing. - -#### engine_getBlobsV3 - -`POST /engine/getBlobsV3` +class GetBlobsV2Request(Container): + blob_versioned_hashes: List[Bytes32, MAX_BLOB_HASHES_REQUEST] -```python class GetBlobsV3Request(Container): blob_versioned_hashes: List[Bytes32, MAX_BLOB_HASHES_REQUEST] ``` -**Response:** [`GetBlobsV3Response`](#getblobsv3response) or HTTP `204 No Content` when syncing. - -### Amsterdam methods - -#### engine_newPayloadV5 - -`POST /engine/newPayloadV5` - -```python -class NewPayloadV5Request(Container): - execution_payload: ExecutionPayloadV4 - expected_blob_versioned_hashes: List[Bytes32, MAX_BLOB_COMMITMENTS_PER_BLOCK] - parent_beacon_block_root: Bytes32 - execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] -``` +**Response:** `200 OK` — SSZ-encoded response type from the table above, or `204 No Content` when the EL is syncing (or for v2, when any blob is missing). -**Response:** [`PayloadStatusV1`](#payloadstatusv1) +*Note:* `GetBlobsV3Response` uses `List[BlobAndProofV2, 1]` inner lists for per-element nullability (0 elements = missing, 1 element = present). The whole-result null (syncing) uses HTTP `204`. -#### engine_getPayloadV6 +**Errors:** -`POST /engine/getPayloadV6` +| Status | Condition | +| - | - | +| `400` | Malformed SSZ | +| `413` | Request exceeds `MAX_BLOB_HASHES_REQUEST` hashes | +| `500` | Internal server error | -```python -class GetPayloadV6Request(Container): - payload_id: Bytes8 -``` +### Transition configuration -**Response:** [`GetPayloadResponseV6`](#getpayloadresponsev6) +#### `POST /engine/v1/transition-configuration` — Exchange transition configuration -#### engine_forkchoiceUpdatedV4 +Deprecated in Cancun. Exchange PoW-to-PoS transition configuration between CL and EL. -`POST /engine/forkchoiceUpdatedV4` +**Request container:** ```python -class ForkchoiceUpdatedV4Request(Container): - forkchoice_state: ForkchoiceStateV1 - payload_attributes: Optional[PayloadAttributesV4] +class ExchangeTransitionConfigurationV1Request(Container): + transition_configuration: TransitionConfigurationV1 ``` -**Response:** [`ForkchoiceUpdatedResponseV1`](#forkchoiceupdatedresponsev1) - -#### engine_getPayloadBodiesByHashV2 - -`POST /engine/getPayloadBodiesByHashV2` +**Response:** `200 OK` — [`TransitionConfigurationV1`](#transitionconfigurationv1) + +### Endpoint summary + +All endpoints organized by resource and fork: + +| HTTP Method | Path | Fork | JSON-RPC Equivalent | +| - | - | - | - | +| `POST` | `/engine/v1/payloads` | Paris | `engine_newPayloadV1` | +| `POST` | `/engine/v2/payloads` | Shanghai | `engine_newPayloadV2` | +| `POST` | `/engine/v3/payloads` | Cancun | `engine_newPayloadV3` | +| `POST` | `/engine/v4/payloads` | Prague | `engine_newPayloadV4` | +| `POST` | `/engine/v5/payloads` | Amsterdam | `engine_newPayloadV5` | +| `GET` | `/engine/v1/payloads/{payload_id}` | Paris | `engine_getPayloadV1` | +| `GET` | `/engine/v2/payloads/{payload_id}` | Shanghai | `engine_getPayloadV2` | +| `GET` | `/engine/v3/payloads/{payload_id}` | Cancun | `engine_getPayloadV3` | +| `GET` | `/engine/v4/payloads/{payload_id}` | Prague | `engine_getPayloadV4` | +| `GET` | `/engine/v5/payloads/{payload_id}` | Osaka | `engine_getPayloadV5` | +| `GET` | `/engine/v6/payloads/{payload_id}` | Amsterdam | `engine_getPayloadV6` | +| `POST` | `/engine/v1/payloads/bodies/by-hash` | Shanghai | `engine_getPayloadBodiesByHashV1` | +| `POST` | `/engine/v2/payloads/bodies/by-hash` | Amsterdam | `engine_getPayloadBodiesByHashV2` | +| `POST` | `/engine/v1/payloads/bodies/by-range` | Shanghai | `engine_getPayloadBodiesByRangeV1` | +| `POST` | `/engine/v2/payloads/bodies/by-range` | Amsterdam | `engine_getPayloadBodiesByRangeV2` | +| `POST` | `/engine/v1/forkchoice` | Paris | `engine_forkchoiceUpdatedV1` | +| `POST` | `/engine/v2/forkchoice` | Shanghai | `engine_forkchoiceUpdatedV2` | +| `POST` | `/engine/v3/forkchoice` | Cancun | `engine_forkchoiceUpdatedV3` | +| `POST` | `/engine/v4/forkchoice` | Amsterdam | `engine_forkchoiceUpdatedV4` | +| `POST` | `/engine/v1/blobs` | Cancun | `engine_getBlobsV1` | +| `POST` | `/engine/v2/blobs` | Osaka | `engine_getBlobsV2` | +| `POST` | `/engine/v3/blobs` | Osaka | `engine_getBlobsV3` | +| `POST` | `/engine/v1/transition-configuration` | Paris | `engine_exchangeTransitionConfigurationV1` | -```python -class GetPayloadBodiesByHashV2Request(Container): - block_hashes: List[Bytes32, MAX_PAYLOAD_BODIES_REQUEST] -``` - -**Response:** [`PayloadBodiesV2Response`](#payloadbodiesv2response) +## Example -#### engine_getPayloadBodiesByRangeV2 +The following example shows an `engine_newPayloadV5` call using the binary SSZ transport. -`POST /engine/getPayloadBodiesByRangeV2` +### Submit payload -```python -class GetPayloadBodiesByRangeV2Request(Container): - start: uint64 - count: uint64 +```bash +curl -X POST http://localhost:8551/engine/v5/payloads \ + -H "Authorization: Bearer $JWT_TOKEN" \ + -H "Content-Type: application/octet-stream" \ + -H "Accept: application/octet-stream" \ + --data-binary @new_payload_request.ssz \ + -o payload_status.ssz ``` -**Response:** [`PayloadBodiesV2Response`](#payloadbodiesv2response) - -## Example - -The following example shows an `engine_newPayloadV5` call using the binary SSZ transport. - -### Request +**Request:** ``` -POST /engine/newPayloadV5 HTTP/1.1 +POST /engine/v5/payloads HTTP/1.1 Host: localhost:8551 -Content-Type: application/ssz +Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... +Content-Type: application/octet-stream Content-Length: 604 <604 bytes: SSZ(NewPayloadV5Request)> @@ -887,11 +905,11 @@ The request body is the SSZ serialization of `NewPayloadV5Request` containing: - `parent_beacon_block_root`: `0x0000000000000000000000000000000000000000000000000000000000000000` - `execution_requests`: empty list -### Response (success) +**Response (success):** ``` HTTP/1.1 200 OK -Content-Type: application/ssz +Content-Type: application/octet-stream Content-Length: 69 <69 bytes: SSZ(PayloadStatusV1)> @@ -902,41 +920,41 @@ The response body is the SSZ serialization of `PayloadStatusV1` containing: - `latest_valid_hash`: `0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858` - `validation_error`: empty -### Response (error) +**Response (error):** ``` HTTP/1.1 400 Bad Request -Content-Type: application/ssz -Content-Length: 48 +Content-Type: text/plain -<48 bytes: SSZ(ErrorResponse)> +Invalid SSZ: unexpected end of input at offset 128 ``` -The response body is the SSZ serialization of `ErrorResponse` containing: -- `code`: `32602` (absolute value of `-32602`, invalid params) -- `message`: `"Invalid execution payload"` - -### Fallback behavior +### Retrieve built payload -If the EL does not support the binary SSZ transport, a request to `/engine/newPayloadV5` returns HTTP `404 Not Found` or `415 Unsupported Media Type`. The CL detects this and falls back to JSON-RPC at `POST /` for subsequent requests. - -## Error handling +```bash +curl -X GET http://localhost:8551/engine/v6/payloads/0x1234567890abcdef \ + -H "Authorization: Bearer $JWT_TOKEN" \ + -H "Accept: application/octet-stream" \ + -o get_payload_response.ssz +``` -Binary SSZ does not change the error semantics of the Engine API. All error codes defined in the [Errors](./common.md#errors) section apply equally. +### Update fork choice -Error responses use the [`ErrorResponse`](#errorresponse) container. The HTTP status code reflects the error category: +```bash +curl -X POST http://localhost:8551/engine/v4/forkchoice \ + -H "Authorization: Bearer $JWT_TOKEN" \ + -H "Content-Type: application/octet-stream" \ + --data-binary @forkchoice_request.ssz \ + -o forkchoice_response.ssz +``` -| HTTP Status | Meaning | Engine API Errors | -| - | - | - | -| `400` | Client error | `-32700` (parse error), `-32600` (invalid request), `-32602` (invalid params) | -| `404` | Method not found | `-32601` (method not found) | -| `415` | Unsupported media type | Binary SSZ not supported | -| `500` | Server error | `-32603` (internal error), `-38001` to `-38005` (engine-specific errors) | +### Fallback behavior -Clients **MUST** validate SSZ payloads against the expected schema before processing. Payloads that do not conform to the expected SSZ schema **MUST** be rejected with a `400` response containing an `ErrorResponse` with code `32700`. +If the EL does not support the binary SSZ transport, a request to `/engine/v5/payloads` returns HTTP `404 Not Found` or `415 Unsupported Media Type`. The CL detects this and falls back to JSON-RPC at `POST /` for subsequent requests. ## Security considerations - SSZ deserialization **MUST** enforce the same size limits as JSON deserialization. Implementations **MUST** reject SSZ payloads exceeding defined maximum sizes before attempting full deserialization. - Implementations **SHOULD** use well-tested SSZ libraries and fuzz test SSZ parsing extensively. - The binary transport uses the same JWT authentication as the JSON-RPC endpoint. All existing authentication requirements apply. +- The `{payload_id}` path parameter **MUST** be validated as a well-formed hex-encoded `Bytes8` before processing. From 234a3e6a80a5f167f68ece084b03cf4e44637b05 Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Wed, 4 Mar 2026 21:02:05 +0100 Subject: [PATCH 07/12] add capabilities and version --- src/engine/ssz-encoding.md | 105 +++++++++++++++++++++++++++++++++---- 1 file changed, 96 insertions(+), 9 deletions(-) diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md index a06b73633..6e21ae22b 100644 --- a/src/engine/ssz-encoding.md +++ b/src/engine/ssz-encoding.md @@ -49,10 +49,14 @@ SSZ container definitions are provided for all Engine API structures and methods - [GetBlobsV1Response](#getblobsv1response) - [GetBlobsV2Response](#getblobsv2response) - [GetBlobsV3Response](#getblobsv3response) + - [ClientVersionV1](#clientversionv1) + - [GetClientVersionV1Response](#getclientversionv1response) + - [ExchangeCapabilitiesResponse](#exchangecapabilitiesresponse) - [Endpoints](#endpoints) - [Payloads](#payloads) - [Forkchoice](#forkchoice) - [Blobs](#blobs) + - [Client](#client) - [Transition configuration](#transition-configuration) - [Endpoint summary](#endpoint-summary) - [Example](#example) @@ -84,13 +88,14 @@ All endpoints are served under the `/engine` prefix on the existing Engine API p ### Content types -| Direction | Content-Type | Description | +| Header | Value | Description | | - | - | - | -| Request body | `application/octet-stream` | SSZ-encoded request container | -| Response body (success) | `application/octet-stream` | SSZ-encoded response container | -| Response body (error) | `text/plain` | Human-readable error message | +| `Content-Type` (request) | `application/octet-stream` | SSZ-encoded request container | +| `Content-Type` (response) | `application/octet-stream` | SSZ-encoded response (success) | +| `Content-Type` (response) | `text/plain` | Human-readable error message | +| `Accept` (request) | `application/octet-stream` | Client accepts SSZ-encoded responses | -Request bodies are the SSZ serialization of the endpoint's request container. Response bodies are the SSZ serialization of the endpoint's response type. +Request bodies are the SSZ serialization of the endpoint's request container. Response bodies are the SSZ serialization of the endpoint's response type. GET requests with no body **SHOULD** include the `Accept` header to indicate SSZ preference. ### Authentication @@ -172,6 +177,12 @@ Error responses use `Content-Type: text/plain` with a human-readable error messa | `MAX_BLOB_HASHES_REQUEST` | `128` | [Osaka](./osaka.md#engine_getblobsv2) | | `MAX_EXECUTION_REQUESTS` | `2**8` (256) | [EIP-7685](https://eips.ethereum.org/EIPS/eip-7685) | | `MAX_ERROR_MESSAGE_LENGTH` | `1024` | This specification | +| `MAX_CLIENT_CODE_LENGTH` | `2` | This specification | +| `MAX_CLIENT_NAME_LENGTH` | `64` | This specification | +| `MAX_CLIENT_VERSION_LENGTH` | `64` | This specification | +| `MAX_CLIENT_VERSIONS` | `4` | This specification | +| `MAX_CAPABILITY_NAME_LENGTH` | `64` | This specification | +| `MAX_CAPABILITIES` | `64` | This specification | | `BLOB_SIZE` | `FIELD_ELEMENTS_PER_BLOB * BYTES_PER_FIELD_ELEMENT` (131,072) | Derived | ## SSZ type mappings @@ -542,6 +553,36 @@ class GetPayloadResponseV6(Container): execution_requests: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_EXECUTION_REQUESTS] ``` +### ClientVersionV1 + +Introduced in [Client Version Specification](./identification.md#clientversionv1). + +```python +class ClientVersionV1(Container): + code: ByteList[MAX_CLIENT_CODE_LENGTH] + name: ByteList[MAX_CLIENT_NAME_LENGTH] + version: ByteList[MAX_CLIENT_VERSION_LENGTH] + commit: Bytes4 +``` + +### GetClientVersionV1Response + +Response container for `engine_getClientVersionV1`. + +```python +class GetClientVersionV1Response(Container): + versions: List[ClientVersionV1, MAX_CLIENT_VERSIONS] +``` + +### ExchangeCapabilitiesResponse + +Response container for `engine_exchangeCapabilities`. + +```python +class ExchangeCapabilitiesResponse(Container): + capabilities: List[ByteList[MAX_CAPABILITY_NAME_LENGTH], MAX_CAPABILITIES] +``` + ### PayloadBodiesV1Response Response container for `engine_getPayloadBodiesByHashV1` and `engine_getPayloadBodiesByRangeV1`. @@ -827,6 +868,50 @@ class GetBlobsV3Request(Container): | `413` | Request exceeds `MAX_BLOB_HASHES_REQUEST` hashes | | `500` | Internal server error | +### Client + +#### `POST /engine/v1/client/version` — Exchange client version + +Exchange client version information between CL and EL. The CL identifies itself in the request; the EL returns its own version(s) in the response. See the [Client Version Specification](./identification.md) for details. + +**Request container:** + +```python +class GetClientVersionV1Request(Container): + client_version: ClientVersionV1 +``` + +**Response:** `200 OK` — [`GetClientVersionV1Response`](#getclientversionv1response) + +**Errors:** + +| Status | Condition | +| - | - | +| `400` | Malformed SSZ | +| `500` | Internal server error | + +--- + +#### `POST /engine/v1/capabilities` — Exchange capabilities + +Exchange the list of supported Engine API endpoints between CL and EL. Capability names use the format `"METHOD /path"` (e.g., `"POST /engine/v5/payloads"`). See the [Capabilities specification](./common.md#capabilities) for details. + +**Request container:** + +```python +class ExchangeCapabilitiesRequest(Container): + capabilities: List[ByteList[MAX_CAPABILITY_NAME_LENGTH], MAX_CAPABILITIES] +``` + +**Response:** `200 OK` — [`ExchangeCapabilitiesResponse`](#exchangecapabilitiesresponse) + +**Errors:** + +| Status | Condition | +| - | - | +| `400` | Malformed SSZ | +| `500` | Internal server error | + ### Transition configuration #### `POST /engine/v1/transition-configuration` — Exchange transition configuration @@ -870,6 +955,8 @@ All endpoints organized by resource and fork: | `POST` | `/engine/v1/blobs` | Cancun | `engine_getBlobsV1` | | `POST` | `/engine/v2/blobs` | Osaka | `engine_getBlobsV2` | | `POST` | `/engine/v3/blobs` | Osaka | `engine_getBlobsV3` | +| `POST` | `/engine/v1/client/version` | All | `engine_getClientVersionV1` | +| `POST` | `/engine/v1/capabilities` | All | `engine_exchangeCapabilities` | | `POST` | `/engine/v1/transition-configuration` | Paris | `engine_exchangeTransitionConfigurationV1` | ## Example @@ -894,9 +981,9 @@ POST /engine/v5/payloads HTTP/1.1 Host: localhost:8551 Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... Content-Type: application/octet-stream -Content-Length: 604 +Content-Length: 584 -<604 bytes: SSZ(NewPayloadV5Request)> +<584 bytes: SSZ(NewPayloadV5Request)> ``` The request body is the SSZ serialization of `NewPayloadV5Request` containing: @@ -910,9 +997,9 @@ The request body is the SSZ serialization of `NewPayloadV5Request` containing: ``` HTTP/1.1 200 OK Content-Type: application/octet-stream -Content-Length: 69 +Content-Length: 37 -<69 bytes: SSZ(PayloadStatusV1)> +<37 bytes: SSZ(PayloadStatusV1)> ``` The response body is the SSZ serialization of `PayloadStatusV1` containing: From bfa6d56f41a700ff845af0ce2597989fce27850e Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Wed, 4 Mar 2026 21:21:56 +0100 Subject: [PATCH 08/12] docs(ssz-encoding): clarify SSZ representation of optional types and update examples The documentation for how `T or null` maps to SSZ encoding is being clarified. Replaced the generic statement about `Optional[T]` being encoded as `List[T, 1]` with a more direct explanation that it is represented as `List[T, 1]`. Also updated the description for `payload_attributes` in `ForkchoiceUpdated` requests to explicitly state that presence is indicated by a list with 1 element, matching the SSZ type definition. Additionally, added missing vocabulary words to `wordlist.txt` to improve future documentation generation tools. --- src/engine/ssz-encoding.md | 16 ++++++++-------- wordlist.txt | 3 +++ 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md index 6e21ae22b..eb1e4d90a 100644 --- a/src/engine/ssz-encoding.md +++ b/src/engine/ssz-encoding.md @@ -203,9 +203,9 @@ Each JSON-encoded base type used in the Engine API maps to a specific SSZ type. | `bytes` (variable-length) | `ByteList[MAX_LENGTH]` (context-dependent) | | `bytesMax32` (0 to 32 bytes) | `ByteList[32]` | | `Array of T` | `List[T, MAX_LENGTH]` (context-dependent) | -| `T or null` | `Optional[T]` (encoded as `List[T, 1]`) | +| `T or null` | `List[T, 1]` | -`Optional[T]` is represented as `List[T, 1]` in SSZ encoding. An empty list (0 elements) denotes absence (`null`). A list with one element denotes presence. +Nullable types are represented as `List[T, 1]` in SSZ encoding. An empty list (0 elements) denotes absence (`null`). A list with one element denotes presence. ## Container definitions @@ -789,7 +789,7 @@ class GetPayloadBodiesByRangeV2Request(Container): Update the EL's fork choice state and optionally start building a new payload. The EL updates its canonical chain view and prunes blocks no longer reachable from the head. -When `payload_attributes` is present (non-empty `Optional`), the EL begins building a new block. The returned `payload_id` can be used with `GET /engine/v{N}/payloads/{payload_id}` to retrieve the built payload. +When `payload_attributes` is present (list with 1 element), the EL begins building a new block. The returned `payload_id` can be used with `GET /engine/v{N}/payloads/{payload_id}` to retrieve the built payload. | Version | Fork | Request Container | JSON-RPC Equivalent | | - | - | - | - | @@ -803,19 +803,19 @@ When `payload_attributes` is present (non-empty `Optional`), the EL begins build ```python class ForkchoiceUpdatedV1Request(Container): forkchoice_state: ForkchoiceStateV1 - payload_attributes: Optional[PayloadAttributesV1] + payload_attributes: List[PayloadAttributesV1, 1] class ForkchoiceUpdatedV2Request(Container): forkchoice_state: ForkchoiceStateV1 - payload_attributes: Optional[PayloadAttributesV2] + payload_attributes: List[PayloadAttributesV2, 1] class ForkchoiceUpdatedV3Request(Container): forkchoice_state: ForkchoiceStateV1 - payload_attributes: Optional[PayloadAttributesV3] + payload_attributes: List[PayloadAttributesV3, 1] class ForkchoiceUpdatedV4Request(Container): forkchoice_state: ForkchoiceStateV1 - payload_attributes: Optional[PayloadAttributesV4] + payload_attributes: List[PayloadAttributesV4, 1] ``` *Note:* `ForkchoiceUpdatedV2Request` always uses `PayloadAttributesV2`. Pre-Shanghai attributes have an empty `withdrawals` list. @@ -979,7 +979,7 @@ curl -X POST http://localhost:8551/engine/v5/payloads \ ``` POST /engine/v5/payloads HTTP/1.1 Host: localhost:8551 -Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... +Authorization: Bearer $JWT_TOKEN Content-Type: application/octet-stream Content-Length: 584 diff --git a/wordlist.txt b/wordlist.txt index ac36cceae..077c0bc48 100644 --- a/wordlist.txt +++ b/wordlist.txt @@ -24,6 +24,7 @@ eip eips EIPS EL +EL's endian enum EOA @@ -41,6 +42,7 @@ getblobsbundlev getblobsv getclientversionv getpayloadresponsev +payloadbodiesv graphql gwei https @@ -53,6 +55,7 @@ kzg mempool merkle multicallV +natively npm ommers openrpc From 2e1fad18ee0fe34a38f29a7ef9a91bc2e364e6c1 Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Wed, 4 Mar 2026 22:06:50 +0100 Subject: [PATCH 09/12] fix all ambiguity --- src/engine/ssz-encoding.md | 42 +++++++++++++++----------------------- 1 file changed, 16 insertions(+), 26 deletions(-) diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md index eb1e4d90a..d5c1cdf90 100644 --- a/src/engine/ssz-encoding.md +++ b/src/engine/ssz-encoding.md @@ -126,7 +126,7 @@ When a new fork introduces a new method version, a new versioned endpoint is add 2. If the EL supports the binary SSZ transport, it **MUST** respond with `Content-Type: application/octet-stream` and a raw SSZ response body. -3. If the EL does not support the binary SSZ transport, it **MUST** respond with HTTP status `404 Not Found` or `415 Unsupported Media Type`. The CL **MUST** then fall back to JSON-RPC (`POST /`) for subsequent requests. +3. If the EL does not support the binary SSZ transport, it **MUST** respond with HTTP status `404 Not Found`. The CL **MUST** then fall back to JSON-RPC (`POST /`) for subsequent requests. 4. Clients **MUST** continue to support JSON-RPC encoding as a fallback. Both the REST endpoints and the JSON-RPC endpoint coexist on the same port. @@ -143,20 +143,18 @@ When a new fork introduces a new method version, a new versioned endpoint is add | Status | Meaning | Usage | | - | - | - | -| `400` | Bad Request | Malformed SSZ, invalid parameters | +| `400` | Bad Request | Malformed SSZ encoding | | `401` | Unauthorized | Missing or invalid JWT token | -| `404` | Not Found | Unknown payload ID, unsupported endpoint | -| `409` | Conflict | Invalid forkchoice state (e.g., finalized block not ancestor of head) | -| `413` | Request Too Large | Request exceeds maximum size limits | -| `415` | Unsupported Media Type | Binary SSZ transport not supported | -| `422` | Unprocessable Entity | Invalid payload attributes (e.g., timestamp not greater than parent) | +| `404` | Not Found | Unknown payload ID | +| `409` | Conflict | Invalid forkchoice state | +| `413` | Request Too Large | Request exceeds maximum element count | +| `422` | Unprocessable Entity | Invalid payload attributes | ### Server errors | Status | Meaning | Usage | | - | - | - | | `500` | Internal Server Error | Unexpected server error | -| `501` | Not Implemented | Unsupported fork version | Error responses use `Content-Type: text/plain` with a human-readable error message body. @@ -687,8 +685,7 @@ class NewPayloadV5Request(Container): | Status | Condition | | - | - | -| `400` | Malformed SSZ | -| `500` | Internal server error | +| `400` | Malformed SSZ encoding | --- @@ -717,7 +714,6 @@ This is a safe, idempotent GET operation. The EL may continue optimizing the pay | - | - | | `400` | Invalid payload ID format | | `404` | Unknown payload ID | -| `500` | Internal server error | --- @@ -746,9 +742,8 @@ class GetPayloadBodiesByHashV2Request(Container): | Status | Condition | | - | - | -| `400` | Malformed SSZ | +| `400` | Malformed SSZ encoding | | `413` | Request exceeds `MAX_PAYLOAD_BODIES_REQUEST` hashes | -| `500` | Internal server error | --- @@ -779,9 +774,8 @@ class GetPayloadBodiesByRangeV2Request(Container): | Status | Condition | | - | - | -| `400` | Malformed SSZ, `start` < 1, `count` < 1 | +| `400` | Malformed SSZ encoding | | `413` | `count` exceeds `MAX_PAYLOAD_BODIES_REQUEST` | -| `500` | Internal server error | ### Forkchoice @@ -826,10 +820,9 @@ class ForkchoiceUpdatedV4Request(Container): | Status | Condition | | - | - | -| `400` | Malformed SSZ | -| `409` | Invalid forkchoice state (e.g., finalized block not ancestor of head) | -| `422` | Invalid payload attributes (e.g., timestamp not greater than parent) | -| `500` | Internal server error | +| `400` | Malformed SSZ encoding | +| `409` | Invalid forkchoice state | +| `422` | Invalid payload attributes | ### Blobs @@ -864,9 +857,8 @@ class GetBlobsV3Request(Container): | Status | Condition | | - | - | -| `400` | Malformed SSZ | +| `400` | Malformed SSZ encoding | | `413` | Request exceeds `MAX_BLOB_HASHES_REQUEST` hashes | -| `500` | Internal server error | ### Client @@ -887,8 +879,7 @@ class GetClientVersionV1Request(Container): | Status | Condition | | - | - | -| `400` | Malformed SSZ | -| `500` | Internal server error | +| `400` | Malformed SSZ encoding | --- @@ -909,8 +900,7 @@ class ExchangeCapabilitiesRequest(Container): | Status | Condition | | - | - | -| `400` | Malformed SSZ | -| `500` | Internal server error | +| `400` | Malformed SSZ encoding | ### Transition configuration @@ -1037,7 +1027,7 @@ curl -X POST http://localhost:8551/engine/v4/forkchoice \ ### Fallback behavior -If the EL does not support the binary SSZ transport, a request to `/engine/v5/payloads` returns HTTP `404 Not Found` or `415 Unsupported Media Type`. The CL detects this and falls back to JSON-RPC at `POST /` for subsequent requests. +If the EL does not support the binary SSZ transport, a request to `/engine/v5/payloads` returns HTTP `404 Not Found`. The CL detects this and falls back to JSON-RPC at `POST /` for subsequent requests. ## Security considerations From 71476f2d01b6952ce94dc7af54e5fe766c8b3205 Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Wed, 4 Mar 2026 22:41:57 +0100 Subject: [PATCH 10/12] docs(ssz-encoding.md): update SSZ REST transport negotiation documentation to reflect capability exchange The transport negotiation mechanism has been updated to exclusively use `engine_exchangeCapabilities` over JSON-RPC for determining support of SSZ REST endpoints. This change clarifies the required steps for clients to discover and utilize the binary SSZ transport. --- src/engine/ssz-encoding.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md index d5c1cdf90..6d19245b0 100644 --- a/src/engine/ssz-encoding.md +++ b/src/engine/ssz-encoding.md @@ -122,13 +122,15 @@ When a new fork introduces a new method version, a new versioned endpoint is add ### Negotiation and fallback -1. The CL sends a request to a versioned REST endpoint with `Content-Type: application/octet-stream` and a raw SSZ request body. +Transport negotiation uses `engine_exchangeCapabilities` over JSON-RPC. -2. If the EL supports the binary SSZ transport, it **MUST** respond with `Content-Type: application/octet-stream` and a raw SSZ response body. +1. At startup, the CL calls `engine_exchangeCapabilities` over JSON-RPC. The CL includes the SSZ REST endpoints it supports in the capabilities list (e.g., `"POST /engine/v5/payloads"`). -3. If the EL does not support the binary SSZ transport, it **MUST** respond with HTTP status `404 Not Found`. The CL **MUST** then fall back to JSON-RPC (`POST /`) for subsequent requests. +2. If the EL's response includes SSZ REST endpoints, the CL **MUST** use the binary SSZ transport for those endpoints. The CL sends requests with `Content-Type: application/octet-stream` and `Accept: application/octet-stream`. The EL **MUST** respond with `Content-Type: application/octet-stream`. -4. Clients **MUST** continue to support JSON-RPC encoding as a fallback. Both the REST endpoints and the JSON-RPC endpoint coexist on the same port. +3. If the EL's response does not include SSZ REST endpoints, the CL **MUST** use JSON-RPC for all Engine API calls. + +4. Clients **MUST** support JSON-RPC encoding. Both the REST endpoints and the JSON-RPC endpoint coexist on the same port. ## HTTP status codes @@ -1027,7 +1029,7 @@ curl -X POST http://localhost:8551/engine/v4/forkchoice \ ### Fallback behavior -If the EL does not support the binary SSZ transport, a request to `/engine/v5/payloads` returns HTTP `404 Not Found`. The CL detects this and falls back to JSON-RPC at `POST /` for subsequent requests. +If the EL does not advertise SSZ REST endpoints in its `engine_exchangeCapabilities` response, the CL uses JSON-RPC for all Engine API calls. ## Security considerations From c5b490f318a63b75675868553f07893f346899cd Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Wed, 4 Mar 2026 22:46:22 +0100 Subject: [PATCH 11/12] docs(ssz-encoding): clarify SSZ transport negotiation details Update documentation to better explain how JSON-RPC remains the default for negotiation and fallback, explicitly stating when binary SSZ is used. This clarifies the steps involved for CL and EL during initialization. --- src/engine/ssz-encoding.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md index 6d19245b0..ae31eccd7 100644 --- a/src/engine/ssz-encoding.md +++ b/src/engine/ssz-encoding.md @@ -122,15 +122,17 @@ When a new fork introduces a new method version, a new versioned endpoint is add ### Negotiation and fallback -Transport negotiation uses `engine_exchangeCapabilities` over JSON-RPC. +Transport negotiation uses the existing JSON-RPC `engine_exchangeCapabilities` method. JSON-RPC is the default transport. The binary SSZ transport is only used when both sides explicitly advertise support. -1. At startup, the CL calls `engine_exchangeCapabilities` over JSON-RPC. The CL includes the SSZ REST endpoints it supports in the capabilities list (e.g., `"POST /engine/v5/payloads"`). +1. At startup, the CL calls `engine_exchangeCapabilities` over JSON-RPC (`POST /`). This call always uses JSON-RPC regardless of SSZ support. The CL includes the SSZ REST endpoints it supports in the capabilities list (e.g., `"POST /engine/v5/payloads"`) alongside its supported JSON-RPC methods. -2. If the EL's response includes SSZ REST endpoints, the CL **MUST** use the binary SSZ transport for those endpoints. The CL sends requests with `Content-Type: application/octet-stream` and `Accept: application/octet-stream`. The EL **MUST** respond with `Content-Type: application/octet-stream`. +2. The EL responds over JSON-RPC with its own capabilities list. An EL that supports binary SSZ **MUST** include the SSZ REST endpoints it supports. An EL that does not support binary SSZ returns only JSON-RPC method names — no changes are required to existing EL implementations. -3. If the EL's response does not include SSZ REST endpoints, the CL **MUST** use JSON-RPC for all Engine API calls. +3. The CL inspects the EL's response. For each endpoint that both sides advertise as an SSZ REST endpoint, the CL **SHOULD** use binary SSZ. For all other methods, the CL **MUST** use JSON-RPC. -4. Clients **MUST** support JSON-RPC encoding. Both the REST endpoints and the JSON-RPC endpoint coexist on the same port. +4. When using binary SSZ, the CL sends requests with `Content-Type: application/octet-stream` and `Accept: application/octet-stream`. The EL **MUST** respond with `Content-Type: application/octet-stream`. + +5. Both CL and EL **MUST** support JSON-RPC encoding at all times. JSON-RPC remains available as a fallback even when binary SSZ is in use. Both the REST endpoints and the JSON-RPC endpoint (`POST /`) coexist on the same port. ## HTTP status codes From c616edd181278b7d92641d0251570f779a17c308 Mon Sep 17 00:00:00 2001 From: Barnabas Busa Date: Thu, 16 Apr 2026 15:43:54 +0200 Subject: [PATCH 12/12] =?UTF-8?q?docs(ssz-encoding):=20address=20PR=20#764?= =?UTF-8?q?=20review=20=E2=80=94=20caching,=20DoS,=20nullable=20types?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Drop misleading "idempotent" claim on GET /payloads/{payload_id} and require Cache-Control: no-store; the payload mutates until the slot deadline so caches/intermediaries must not store or revalidate it. - Expand security considerations with explicit DoS guidance: pre-read Content-Length rejection, length/offset validation before allocation, and operationally enforced per-endpoint body caps. The protocol-level maxima bound on-chain validity, not per-request resource use. - Encode truly-nullable fields per the documented List[T, 1] rule: PayloadStatusV1.latest_valid_hash, ForkchoiceUpdatedResponseV1.payload_id, and ExecutionPayloadBodyV2.block_access_list. Restores parity with the JSON spec (each is non-required / oneOf null) and removes the zero-sentinel ambiguity. Example response length updated 37 -> 41. --- src/engine/ssz-encoding.md | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/src/engine/ssz-encoding.md b/src/engine/ssz-encoding.md index ae31eccd7..3d377a1fd 100644 --- a/src/engine/ssz-encoding.md +++ b/src/engine/ssz-encoding.md @@ -327,11 +327,11 @@ Introduced in [Paris](./paris.md#payloadstatusv1). The `status` field is encoded ```python class PayloadStatusV1(Container): status: uint8 - latest_valid_hash: Bytes32 + latest_valid_hash: List[Bytes32, 1] validation_error: ByteList[MAX_ERROR_MESSAGE_LENGTH] ``` -*Note:* `latest_valid_hash` is all zeros when absent (e.g. when `status` is `SYNCING` or `ACCEPTED`). `validation_error` is empty when absent. +*Note:* `latest_valid_hash` follows the nullable encoding (`List[T, 1]`): 0 elements denote absence (e.g. when `status` is `SYNCING` or `ACCEPTED`), 1 element carries the hash. `validation_error` is a `ByteList`; an empty list denotes absence of an error message. | `status` value | Meaning | | - | - | @@ -409,10 +409,10 @@ Used by all versions of `engine_forkchoiceUpdated`. ```python class ForkchoiceUpdatedResponseV1(Container): payload_status: PayloadStatusV1 - payload_id: Bytes8 + payload_id: List[Bytes8, 1] ``` -*Note:* `payload_id` is all zeros when no payload building was initiated. +*Note:* `payload_id` follows the nullable encoding (`List[T, 1]`): 0 elements when no payload building was initiated, 1 element carrying the identifier when it was. ### ExecutionPayloadBodyV1 @@ -434,10 +434,10 @@ Introduced in [Amsterdam](./amsterdam.md#executionpayloadbodyv2). Extends `Execu class ExecutionPayloadBodyV2(Container): transactions: List[ByteList[MAX_BYTES_PER_TRANSACTION], MAX_TRANSACTIONS_PER_PAYLOAD] withdrawals: List[WithdrawalV1, MAX_WITHDRAWALS_PER_PAYLOAD] - block_access_list: ByteList[MAX_BYTES_PER_TRANSACTION] + block_access_list: List[ByteList[MAX_BYTES_PER_TRANSACTION], 1] ``` -*Note:* `withdrawals` is empty for pre-Shanghai blocks. `block_access_list` is empty for pre-Amsterdam blocks. +*Note:* `withdrawals` is empty for pre-Shanghai blocks. `block_access_list` is nullable in JSON (`null` when unavailable) and follows the nullable encoding (`List[T, 1]`): 0 elements denote absence (e.g. pre-Amsterdam blocks), 1 element carries the RLP-encoded list. ### BlobsBundleV1 @@ -697,7 +697,7 @@ class NewPayloadV5Request(Container): Retrieve an execution payload previously requested via forkchoice update with payload attributes. The `{payload_id}` path parameter is the hex-encoded `Bytes8` payload identifier (e.g., `0x1234567890abcdef`). -This is a safe, idempotent GET operation. The EL may continue optimizing the payload until the slot deadline. +The EL may continue optimizing the payload until the slot deadline, so successive GETs against the same `{payload_id}` may return different bytes. The EL **MUST** include `Cache-Control: no-store` on the response, and intermediaries **MUST NOT** cache or revalidate this resource. Clients **MUST NOT** treat the response as cacheable. | Version | Fork | Response Type | JSON-RPC Equivalent | | - | - | - | - | @@ -991,14 +991,14 @@ The request body is the SSZ serialization of `NewPayloadV5Request` containing: ``` HTTP/1.1 200 OK Content-Type: application/octet-stream -Content-Length: 37 +Content-Length: 41 -<37 bytes: SSZ(PayloadStatusV1)> +<41 bytes: SSZ(PayloadStatusV1)> ``` The response body is the SSZ serialization of `PayloadStatusV1` containing: - `status`: `0` (VALID) -- `latest_valid_hash`: `0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858` +- `latest_valid_hash`: list with one element, `0x3559e851470f6e7bbed1db474980683e8c315bfce99b2a6ef47c057c04de7858` - `validation_error`: empty **Response (error):** @@ -1036,6 +1036,10 @@ If the EL does not advertise SSZ REST endpoints in its `engine_exchangeCapabilit ## Security considerations - SSZ deserialization **MUST** enforce the same size limits as JSON deserialization. Implementations **MUST** reject SSZ payloads exceeding defined maximum sizes before attempting full deserialization. +- The constant maxima above (e.g. `MAX_BYTES_PER_TRANSACTION = 2**30`, `MAX_TRANSACTIONS_PER_PAYLOAD = 2**20`) bound on-chain validity, not per-request resource use. A naive decoder facing crafted lengths or offsets can be coerced into large allocations or scans before semantic rejection. Implementations **MUST**: + - Reject requests by `Content-Length` against an endpoint-specific maximum **before** reading the body when the header is present, and cap the bytes read from the body in all cases. + - Validate SSZ length prefixes and offsets against remaining buffer size **before** allocating backing storage for variable-length fields. + - Enforce per-endpoint body size limits operationally (reverse proxy, server config) in addition to library-level checks; the protocol-level constants are an upper bound, not a target. - Implementations **SHOULD** use well-tested SSZ libraries and fuzz test SSZ parsing extensively. - The binary transport uses the same JWT authentication as the JSON-RPC endpoint. All existing authentication requirements apply. - The `{payload_id}` path parameter **MUST** be validated as a well-formed hex-encoded `Bytes8` before processing.