Produce block v4 with payload#580
Conversation
| - Validator | ||
| - ValidatorRequiredApi | ||
| parameters: | ||
| - name: slot |
There was a problem hiding this comment.
while implementing this, I noticed the slot alone is not definitive enough (eg. in case of forks), we should consider also passing the beacon_block_root here
There was a problem hiding this comment.
hm, what do you think about making it optional to pass beacon_block_root like in 0c45859? pretty sure the VC already computes the block's tree hash root during signing, so passing it here should be straightforward
There was a problem hiding this comment.
Yes, we do already compute the block root on the vc side. I am not a big fan of optional parameters, so I'd rather have it as required or remove it if it's not needed, so far it makes our implementation simpler, maybe get more feedback from others who already implemented the apis
There was a problem hiding this comment.
I feel you, perhaps @eserilev @tersec @StefanBratanov or others from your teams have some thoughts here
There was a problem hiding this comment.
Yes, prefer required to optional fields.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
| description: "Slot for which the execution payload envelope is requested." | ||
| schema: | ||
| $ref: "../../beacon-node-oapi.yaml#/components/schemas/Uint64" | ||
| - name: builder_index |
There was a problem hiding this comment.
Will this endpoint be used for all builders outside of self building? I think the pipeline might endup being very different which makes me question if we would ever be caching more than just the current slot's payload envelope. terence asked this internally and I didn't have a good answer.
There was a problem hiding this comment.
this endpoint is only meant for self-building if you are the proposer or the "act as a builder" mode where you build the payload for another proposer
builders that produce payloads outside of the beacon node would get their own endpoint which would be under the /builder namespace (at least that's how I am thinking about it right now)
it should be sufficient to only cache the current slot as you consume the cache within the same slot, maybe if that's the case my comment here #580 (comment) is also not needed but I feel like the block root is useful to specify, not just the slot
There was a problem hiding this comment.
After thinking about it more, maybe we keep the current apis under /validator to be only used by proposers, which means we can remove builder_index here as it's only used for self-building
There was a problem hiding this comment.
makes sense, removed builder_index in 0c45859
There was a problem hiding this comment.
so looks like we might wanna use this for builders too if the validator "acts as a builder"
worth revisiting if we wanna re-add the builder_index here
Edit:
we might be fine, I think it's worth to wait until someone actually implement the "act as a builder" flow e2e, I rather tend towards not mixing up apis required by validators and builders too much
see discord for conversation about this
Notable changes
- implement epbs stateful block production (self-build only)
- `GET /eth/v4/validator/blocks/{slot}` added
- `GET
/eth/v1/validator/execution_payload_envelope/{slot}/{beacon_block_root}`
added
- `POST /eth/v1/beacon/execution_payload_envelope` added
- `POST /eth/v2/beacon/blocks` updated
- implement envelope state root computation
- update block production cache for gloas (includes state root)
- add validator block production flow for gloas
- add envelope signing to validator store / remote signer
- data column sidecar changes required to wire up new gloas type
- update beacon-api spec to `v5.0.0-alpha.0`
see ethereum/beacon-APIs#580 for reference
| * if `exec_node_payload_value >= builder_boost_factor * (builder_payload_value // 100)`, | ||
| then return a full (unblinded) block containing the execution node payload. | ||
| * otherwise, return a blinded block containing the builder payload header. |
There was a problem hiding this comment.
I don't think this applies with gloas anymore. Regardless of whether we self-build or fetch a bid from a builder. We will get back a SignedExecutionPayloadBid in the beacon block. In the case of the builder, the envelope won't be available with the validator.
There was a problem hiding this comment.
this needs to be rephrased for gloas, I think I brought that up in the other PR as well
There was a problem hiding this comment.
made minimal changes to the description to remove the blinded/unblinded references in 8bf9080
agreed builder_boost_factor needs to be rethought for gloas. we discussed briefly in the old PR here and briefly in the discord
I also found this builder-specs PR defining how BN can set preferences between external bids/p2p bis/VC bids
The question is whether we want to be able to override these BN preferences via the VC. I'm sure the answer is yes, and the question becomes "how"?
Would we want to modify the validator registrations endpoint to signal a VC's per builder bid boost preferences similar to as proposed in the builder spec PR? Pretty sure this POST fires every epoch or when one of the values change
This could replace builder_boost_factor for example
Open to other ideas as well
There was a problem hiding this comment.
Would we want to modify the validator registrations endpoint to signal a VC's per builder bid boost preferences similar to as proposed in the builder spec PR? Pretty sure this POST fires every epoch or when one of the values change
We want to get rid of this endpoint, by default, there won't be any registrations or builder preferences submitted to the beacon node, we only submit proposer preferences.
I left a comment related to this here #580 (comment). In my opinion, the beacon-api spec should have no notion of per builder configuration. The builder_boost_factor is still useful for the validator client to signal to the beacon node how it should decide on whether to self-build or pick a bid from the p2p pool, and the bid from the p2p pool is simply the one with the highest value.
|
|
||
| Servers should use saturating arithmetic or another technique to ensure that large values of | ||
| the `builder_boost_factor` do not trigger overflows or errors. If this parameter is | ||
| provided and the beacon node is not configured with a builder then the beacon node MUST |
There was a problem hiding this comment.
I guess with ePBS, the beacon node is always configured with a builder as it listens to the builder bids on the gossip topic anyways.
There was a problem hiding this comment.
not necessarily, we currently have an option to only produce local blocks which I intend to keep, but the default behavior will likely change with gloas to either use bid from p2p or build local, there are still valid reasons to force the node to build a local block, eg. builders censoring, which especially until FOCIL is shipped requires local blocks or a really sophisticated logic in clients to filter out bids from censoring builders
| description: | | ||
| Instructs the beacon node to broadcast a signed execution payload envelope to the network, | ||
| to be gossiped for payload validation. A success response (20x) indicates | ||
| that the envelope passed gossip validation and was successfully broadcast onto the network. |
There was a problem hiding this comment.
In publishBlockV2 we return 202 if block passed gossip validation but failed integration.
Are we doing something similar for publishExecutionPayloadEnvelope?
There was a problem hiding this comment.
I guess from another angle to think about this is we broadcast the payload regardless the result of the payload integration, or we integrate first, if it goes well, we then gossip?
There was a problem hiding this comment.
We probably want to propagate the envelope as quickly as possible, so ideally:
- Apply gossip validation
- Broadcast
- Run
process_execution_payloadand import to DB
If (1) fails, we 400. If (3) fails we should probably 202 just like for blocks.
I was hoping we could avoid the nastiness of broadcast_validation for envelopes, but I think there might actually be an unbundling attack (background) for envelopes now:
I think there is an unbundling attack something like:
- Malicious proposer publishes block
Aat slotNcommitting tobidAandpayloadA, wherebidAcomes from an external builder (could be P2P or otherwise). - Malicious proposer publishes block
B(slashable) at slotNcommitting tobidBand payloadpayloadB. They will be slashed at the next slot. - Builder for
bidAstarts to publishpayloadA. Even on beacon nodes that have seen the equivocating block, the payload will PASS gossip validation as long as block A was imported. - Proposer publishes
payloadBfor blockB. Envelope gossip validation passes forpayloadBbecause it is signed by a different builder frompayloadA. - Result: either A or B could become head, depending on timing. If B wins, the builder of A has revealed potentially valuable information, which could be worth the proposer slashing themselves for (they could use it for a self-build in the next slot).
Proposed fixes:
- We could put the burden on the builder to check for block equivocations prior to revealing their payload (no API/spec changes).
- We could add a gossip condition to reject envelopes for slashable blocks.
- We could add a
broadcast_validation=equivocationflag (like we have for pre-Gloas blocks) to check that the envelope's block is not an equivocation. This flag would likely be used by all builders when interacting with their BNs. - We could make the block equivocation check a default part of the API (same as
broadcast_validation=equivocationfrom fix 3, but no flag required and no ability to opt-out).
There was a problem hiding this comment.
There was a problem hiding this comment.
I'm just catching up on the discussions here, but I think the broadcast_validation flag and its values make sense to me.
If im understanding correctly, the broadcast_validation =consensus_and_equivocation value on the publish_block endpoint will no longer be useful for unbundling protection? Probably no harm to keep it though.
There was a problem hiding this comment.
If im understanding correctly, the broadcast_validation =consensus_and_equivocation value on the publish_block endpoint will no longer be useful for unbundling protection?
no, it's not needed for this anymore, we could discuss if we wanna deprecate broadcast_validation on publishBlockV2, but might still be useful to have basic checks before publishing the block
**What type of PR is this?**
Feature
**What does this PR do? Why is it needed?**
implements the new GET /eth/v4/validator/blocks/{slot} endpoint, we
don't hook up the validator client to use it yet for post gloas in this
pr.
**Which issues(s) does this PR fix?**
Fixes ethereum/beacon-APIs#580
**Other notes for review**
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description with sufficient context for reviewers
to understand this PR.
- [x] I have tested that my changes work as expected and I added a
testing plan to the PR description (if applicable).
**What type of PR is this?**
Feature
**What does this PR do? Why is it needed?**
implements the new GET /eth/v4/validator/blocks/{slot} endpoint, we
don't hook up the validator client to use it yet for post gloas in this
pr.
**Which issues(s) does this PR fix?**
Fixes ethereum/beacon-APIs#580
**Other notes for review**
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description with sufficient context for reviewers
to understand this PR.
- [x] I have tested that my changes work as expected and I added a
testing plan to the PR description (if applicable).
Add a docstring on publish_execution_payload_envelope linking to the in-flight beacon-APIs PR that defines this endpoint (ethereum/beacon-APIs#580) and recording: - stateful-only mode (SignedExecutionPayloadEnvelope; the stateless SignedExecutionPayloadEnvelopeContents form is unimplemented) - spec endorsement of broadcast-before-integration ordering - spec response codes (200/202/400) and that current handler returns 200 even on post-broadcast failures because envelope integration into state is not yet implemented; integration-time failures should switch to 202 once it is.
**What type of PR is this?**
Feature
**What does this PR do? Why is it needed?**
Introduces a stateless mode for validator client using getBlockv4 with
payload included in response
```
participants_matrix:
el:
- el_type: geth
el_image: ethpandaops/geth:glamsterdam-devnet-0
cl:
- cl_type: prysm
cl_image: gcr.io/offchainlabs/prysm/beacon-chain:latest
vc_image: gcr.io/offchainlabs/prysm/validator:latest
vc_extra_params:
- --enable-beacon-rest-api
- --enable-stateless
- --verbosity=debug
cl_extra_params:
- --verbosity=debug
- cl_type: lodestar
cl_image: ethpandaops/lodestar:glamsterdam-devnet-0
- cl_type: lighthouse
cl_image: ethpandaops/lighthouse:glamsterdam-devnet-0
count: 2
network_params:
gloas_fork_epoch: 1
seconds_per_slot: 4
withdrawal_type: "0x01"
validator_balance: 40000
num_validator_keys_per_node: 1000
additional_services:
- dora
- assertoor
- spamoor
- checkpointz
spamoor_params:
image: ethpandaops/spamoor:master
assertoor_params:
run_stability_check: false
run_block_proposal_check: false
tests:
- { file: "https://raw.githubusercontent.com/ethpandaops/assertoor/refs/heads/master/playbooks/gloas-dev/builder-lifecycle.yaml" }
snooper_enabled: false
global_log_level: debug
port_publisher:
additional_services:
enabled: true
checkpointz_params:
image: ethpandaops/checkpointz:gloas-latest
docker_cache_params:
enabled: false
```
needs OffchainLabs#16705
**Which issues(s) does this PR fix?**
related to ethereum/beacon-APIs#580
**Other notes for review**
**Acknowledgements**
- [x] I have read
[CONTRIBUTING.md](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md).
- [x] I have included a uniquely named [changelog fragment
file](https://github.com/prysmaticlabs/prysm/blob/develop/CONTRIBUTING.md#maintaining-changelogmd).
- [x] I have added a description with sufficient context for reviewers
to understand this PR.
- [x] I have tested that my changes work as expected and I added a
testing plan to the PR description (if applicable).
---------
Co-authored-by: james-prysm <jhe@offchainlabs.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Part of #8828 for the stateful path and helps align gloas `produceBlockV4` with beacon-APIs [PR](ethereum/beacon-APIs#580) - Plumb `include_payload` query through the handler. Ignored for now since stateless mode isn't wired up yet - Add `execution_payload_included` metadata field + `Eth-Execution-Payload-Included` header per spec. Both `false` until stateless lands - Drop the `{builder_index}` segment from the envelope GET URL since no longer included in spec Co-Authored-By: shane-moore <skm1790@gmail.com> Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
|
is this sorted now? |
this pr could be merged as is imo. not sure if others think otherwise. my lingering question in #580 (comment) is about what to do with |
This comment was marked as spam.
This comment was marked as spam.
|
sounds like the outstanding issues from discord are
|
@rolfyone could you elaborate the stateful endpoint issue? AFAICT the functionality is covered in this PR, or am I missing something? EDIT: never mind, i see the R&D discussion now. |
I think builder-specific flags like builder_boost, min_bid etc can be specified in the builder-specs. The community did want some standardization of these flags. I can update ethereum/builder-specs#144 to include all the necessary flags and I think the builder-specs is probably the right place to keep these builder based configurations. |
| "200": | ||
| description: "Successful response" | ||
| headers: | ||
| Eth-Consensus-Version: |
There was a problem hiding this comment.
we don't need this header on a get request right?
There was a problem hiding this comment.
the header is in the response, and from what I can see, it's needed
| - $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Gloas.SignedExecutionPayloadEnvelope" | ||
| application/octet-stream: | ||
| schema: | ||
| description: "SSZ serialized `SignedExecutionPayloadEnvelopeContents` or `SignedExecutionPayloadEnvelope` bytes. Use Content-Type header to specify this format" |
There was a problem hiding this comment.
how are you checking which one was passed the contents of the payload envelope , i'm checking ssz offsets right now, but wondering if it makes sense to just add something in the header to separate the two
There was a problem hiding this comment.
how are you checking which one was passed the contents of the payload envelope , i'm checking ssz offsets right now, but wondering if it makes sense to just add something in the header to separate the two
Yea Lodestar ran into the same problem. I think we should have Eth-Blob-Data-Included or something to distinguish the two.
There was a problem hiding this comment.
We can either have a separate endpoint or add a header to differentiate both, ideally for the local stateful flow we just wanna pass a blinded execution payload envelope around to get the most out of it as passing the transactions around is not useful. If we go with BlindedExecutionPayloadEnvelope we could also nicely reuse the existing Eth-Execution-Payload-Blinded header here
| description: | | ||
| Instructs the beacon node to broadcast a signed execution payload envelope to the network, | ||
| to be gossiped for payload validation. A success response (20x) indicates | ||
| that the envelope passed gossip validation and was successfully broadcast onto the network. |
There was a problem hiding this comment.
If im understanding correctly, the broadcast_validation =consensus_and_equivocation value on the publish_block endpoint will no longer be useful for unbundling protection?
no, it's not needed for this anymore, we could discuss if we wanna deprecate broadcast_validation on publishBlockV2, but might still be useful to have basic checks before publishing the block
| - $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Gloas.SignedExecutionPayloadEnvelope" | ||
| application/octet-stream: | ||
| schema: | ||
| description: "SSZ serialized `SignedExecutionPayloadEnvelopeContents` or `SignedExecutionPayloadEnvelope` bytes. Use Content-Type header to specify this format" |
There was a problem hiding this comment.
We can either have a separate endpoint or add a header to differentiate both, ideally for the local stateful flow we just wanna pass a blinded execution payload envelope around to get the most out of it as passing the transactions around is not useful. If we go with BlindedExecutionPayloadEnvelope we could also nicely reuse the existing Eth-Execution-Payload-Blinded header here
| enum: [gloas] | ||
| example: "gloas" | ||
| data: | ||
| $ref: "../../beacon-node-oapi.yaml#/components/schemas/Gloas.ExecutionPayloadEnvelope" |
There was a problem hiding this comment.
I would be in favor of just returning a BlindedExecutionPayloadEnvelope here, that's the minimal object we need to pass to the validator client to sign over
| schema: | ||
| type: boolean | ||
| default: true | ||
| - name: builder_boost_factor |
There was a problem hiding this comment.
@bharath-123 if a client doesn't have builders configured via api, we don't wanna submit builder preferences at all, so moving boost factor and other parameters there seem wrong to me, we need a way for the validator to signal this to the beacon node. And I don't think the beacon api should be concerned about off protocol builders or per builder configs
| - name: beacon_block_root | ||
| in: query | ||
| required: true |
There was a problem hiding this comment.
I think we should move this to path instead
Introduces the block production and execution payload envelope endpoints for post-Gloas (ePBS) forks. These were separated out from #552 to be discussed and reviewed independently. Prior discussion in PR 552 thread and shane-moore/beacon-APIs#2 as well.
New endpoints:
GET /eth/v4/validator/blocks/{slot}— Post-Gloas block production endpoint (produceBlockV4).Supports an
include_payloadquery parameter that controls whether the execution payload envelope and blobs are returned inline (stateless multi-BN operation) or cached by the beacon node for separate retrieval (stateful single-BN operation).GET /eth/v1/validator/execution_payload_envelope/{slot}— Fetch the cached execution payload envelope for the currentproposer's self-built payload. Only available for the current slot; returns 404 for past slots or when no envelope has been cached. Used in stateful mode.
POST /eth/v1/beacon/execution_payload_envelope— Publish a signed execution payload envelope to the network.Open Questions
BlindedExecutionPayloadEnvelope(omitting transactions) should replace the full envelope in the stateful POST flow for both proposer self-building and builder envelope submission, since the beacon node already has the data cached (see discussion)