Supernova meta processing by AdoAdoAdo · Pull Request #7354 · multiversx/mx-chain-go

AdoAdoAdo · 2025-10-20T07:01:55Z

Reasoning behind the pull request

This pull request implements supernova meta block processing functionality, introducing support for V3 meta headers and execution results-based block creation.
The changes enable meta blocks to reference shard headers through execution results rather than direct inclusion, and refactor the epoch start trigger interface to accept round parameters.

Proposed changes

Added new CreateNewHeaderProposal and CreateBlockProposal methods for V3 meta block creation
Implemented ShardInfoCreator component to handle shard info creation from both V3 and legacy headers
Refactored EpochStartTriggerHandler interface, replacing UpdateRound() with parameter on SetEpochChange(round)
Renamed meta block reference tracking methods from MetaBlock to generic Header for broader applicability

Testing procedure

Pre-requisites

Based on the Contributing Guidelines the PR author and the reviewers must check the following requirements are met:

was the PR targeted to the correct branch?
if this is a larger feature that probably needs more than one PR, is there a feat branch created?
if this is a feat branch merging, do all satellite projects have a proper tag inside go.mod?

codecov · 2025-10-20T07:10:43Z

Codecov Report

❌ Patch coverage is 91.50327% with 52 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.35%. Comparing base (0562801) to head (7faab3c).
⚠️ Report is 68 commits behind head on feat/supernova-async-exec.

Files with missing lines	Patch %	Lines
process/block/metablockProposal.go	93.19%	11 Missing and 5 partials ⚠️
process/block/shardblockProposal.go	62.50%	15 Missing ⚠️
process/block/metablock.go	74.35%	5 Missing and 5 partials ⚠️
process/block/baseProcess.go	94.02%	2 Missing and 2 partials ⚠️
process/common.go	0.00%	2 Missing and 2 partials ⚠️
factory/processing/blockProcessorCreator.go	80.00%	1 Missing and 1 partial ⚠️
epochStart/shardchain/trigger.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@                      Coverage Diff                      @@
##           feat/supernova-async-exec    #7354      +/-   ##
=============================================================
+ Coverage                      77.30%   77.35%   +0.04%     
=============================================================
  Files                            867      868       +1     
  Lines                         116906   117321     +415     
=============================================================
+ Hits                           90379    90758     +379     
- Misses                         20565    20591      +26     
- Partials                        5962     5972      +10

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sstanculeanu · 2025-10-24T10:28:19Z

+			"hash", currentBlockHash)
+		bp.revertGasForCrossShardDstMeMiniBlocks(currMiniBlocksAdded, pendingMiniBlocks)
+
+		return false, pendingMiniBlocks, nil


just a note, this might be misleading, as the returned pendingMiniBlocks are not pending anymore, as they were reverted.

although the comment from metablockProposal.go clarifies this a bit

refactored this part

sstanculeanu · 2025-10-24T10:34:19Z


+// RevertIncomingMiniBlocks reverts the gas consumption for the given incoming mini blocks
+func (gc *gasConsumption) RevertIncomingMiniBlocks(miniBlockHashes [][]byte) {
+	// TODO: implement this


implemented on #7357

will remove todo once it is merged.

sstanculeanu · 2025-10-24T10:43:40Z

+		if !ok {
+			continue
+		}
+		mp.addHeader(shardHdrInfo.Hash, shardHdrInfo.Header, shardHdrInfo.UsedInBlock)


pass the entire structure?

addHeader was not refactored in this PR

sstanculeanu · 2025-10-24T12:14:52Z

+func (mp *metaProcessor) createProposalMiniBlocks(haveTime func() bool) error {
+	if !haveTime() {
+		log.Debug("metaProcessor.createProposalMiniBlocks", "error", process.ErrTimeIsOut)
+		return nil


return nil here = create a proposal with empty body?

sstanculeanu · 2025-10-24T12:51:20Z

+	shardHeaders []data.HeaderHandler,
+	shardHeaderHashes [][]byte,
+) ([]data.ShardDataHandler, error) {
+	if !metaHeader.IsHeaderV3() {


check header for nil

sstanculeanu · 2025-10-24T12:51:24Z

+	shardHeaders []data.ShardHeaderHandler,
+	shardHeaderHashes [][]byte,
+) ([]data.ShardDataHandler, error) {
+	if metaHeader.IsHeaderV3() {


check header for nil

sstanculeanu · 2025-10-24T12:56:25Z

+		return nil, process.ErrWrongTypeAssertion
+	}
+
+	execResultHandler.GetMiniBlockHeadersHandlers()


to be removed

yes, removed

sstanculeanu · 2025-10-24T13:01:02Z

+	execResultHandler data.ExecutionResultHandler,
+) []block.MiniBlockHeader {
+	mbHeaderHandlers := execResultHandler.GetMiniBlockHeadersHandlers()
+	shardMiniBlockHeaders := make([]block.MiniBlockHeader, 0)


Suggested change

shardMiniBlockHeaders := make([]block.MiniBlockHeader, 0)

shardMiniBlockHeaders := make([]block.MiniBlockHeader, 0, len(mbHeaderHandlers))

# Conflicts: # process/block/pendingMb/pendingMiniBlocks.go

…ersx/mx-chain-go into shard-data-proposal-integration # Conflicts: # go.mod # go.sum

Shard data proposal integration

…create-shardinfo-testing

# Conflicts: # go.sum

ssd04 · 2025-11-05T12:20:27Z

+		return &CrossShardIncomingMbsCreationResult{
+			HeaderFinished:    false,
+			PendingMiniBlocks: pendingMiniBlocks,
+			AddedMiniBlocks:   currMiniBlocksAdded,
+		}, nil
+	}
+
+	return &CrossShardIncomingMbsCreationResult{
+		HeaderFinished:    true,
+		PendingMiniBlocks: pendingMiniBlocks,
+		AddedMiniBlocks:   currMiniBlocksAdded,
+	}, nil


Suggested change

return &CrossShardIncomingMbsCreationResult{

HeaderFinished: false,

PendingMiniBlocks: pendingMiniBlocks,

AddedMiniBlocks: currMiniBlocksAdded,

}, nil

}

return &CrossShardIncomingMbsCreationResult{

HeaderFinished: true,

PendingMiniBlocks: pendingMiniBlocks,

AddedMiniBlocks: currMiniBlocksAdded,

}, nil

}

return &CrossShardIncomingMbsCreationResult{

HeaderFinished: hdrFinished,

PendingMiniBlocks: pendingMiniBlocks,

AddedMiniBlocks: currMiniBlocksAdded,

}, nil

return hdrFinished directly?

ssd04 · 2025-11-05T12:24:55Z

-func (mp *metaProcessor) getLastCrossNotarizedShardHdrs() (map[uint32]data.HeaderHandler, error) {
-	lastCrossNotarizedHeader := make(map[uint32]data.HeaderHandler, mp.shardCoordinator.NumberOfShards())
+func (mp *metaProcessor) getLastCrossNotarizedShardHdrsAndAddToCurrentBlock() (map[uint32]data.HeaderHandler, error) {
+	lastCrossNotarizedHeader, err := mp.getLastCrossNotarizedShardHdrs()


rename to ...Headers? same below

ssd04 · 2025-11-05T12:27:45Z

+func (mp *metaProcessor) createShardInfo(metaHdr data.MetaHeaderHandler) ([]data.ShardDataHandler, error) {
 	var shardInfo []data.ShardDataHandler
-	if mp.epochStartTrigger.IsEpochStart() {
+	if metaHdr.IsStartOfEpochBlock() {


we don't need to check by epochStartTrigger.IsEpochStart() on legacy?

adding of epoch start data should have already happened on the condition of epochStartTrigger.IsEpochStart() earlier (on block creation), so at this point the conditions should be equivalent for legacy.

ssd04 · 2025-11-05T12:40:27Z

+		return nil, process.ErrWrongTypeAssertion
+	}
+
+	header, err := sic.headersPool.GetHeaderByHash(execResultHandler.GetHeaderHash())


i'm thinking if it might help to use GetHeader func (from process common which search in pool and then in storage) it should be in the pool but if there is an edge case

that would mean giving the component access also to the storage service. I would leave it like this for now, the header should be in the pool. We can see first how the testing goes and decide later if if is really needed.

ssd04 · 2025-11-05T12:55:49Z

+	shardHeader, ok := header.(data.ShardHeaderHandler)
+	if !ok {
+		return nil, process.ErrWrongTypeAssertion
+	}
+


add IsHeaderV3 check? as for meta header proposal

right, added.

ssd04 · 2025-11-05T12:58:36Z

+		return nil, err
+	}
+
+	err = mp.addExecutionResultsOnHeader(metaHeader)


do we add execution results on header on CreateNewHeaderProposal? shoudn't be set on CreateBlockProposal?

this was required on the metachain with the proposal flow, as otherwise we would not know that the epoch change should be done (we do the epoch change now when we notarize the execution results from an epoch change proposal, rather than relying on the epoch change trigger component).

This was done in order to have minimal changes to the trigger, considering that the detection of the epoch change conditions are now not in sync with the execution of an epoch change, and the commit of the epoch change.

ssd04 · 2025-11-05T13:08:28Z

+	return nil, nil
+}
+
+func (mp *metaProcessor) hasStartOfEpochExecutionResults(metaHeader data.MetaHeaderHandler) (bool, error) {


Suggested change

func (mp *metaProcessor) hasStartOfEpochExecutionResults(metaHeader data.MetaHeaderHandler) (bool, error) {

func hasStartOfEpochExecutionResults(metaHeader data.MetaHeaderHandler) (bool, error) {

*optional: i might refactor a bit and use this also in epoch start bootstrap in another PR

yes, this could be moved and refactored, I would do this on another PR, there is an ongoing work of adding unit tests for this in a separate PR.

ssd04 · 2025-11-05T13:17:42Z

+	}
+


on legacy there was also go mp.requestShardHeadersIfNeeded(hdrsAddedForShard, lastShardHdr) we don't need it anymore?

yes, I think we need something similar, but requesting both headers and proofs, and not based on the finality as the headers are already final once they have a proof.

This should be done on another PR.

sstanculeanu · 2025-11-05T15:45:43Z

 	defer s.mut.Unlock()

-	if check.IfNil(metaBlock) || len(metaBlockHash) == 0 {
+	if check.IfNil(header) || len(headerHash) == 0 {


not related to this pr: can be done before mutex lock

sstanculeanu · 2025-11-05T15:47:38Z

+	// TODO: pendingMiniBlocksHandler should be updated
+	shardData.NumPendingMiniBlocks = uint32(len(sic.pendingMiniBlocksHandler.GetPendingMiniBlocks(header.GetShardID())))


done, coming in another PR

MX-17245 Adding unit tests for shardinfo

…chain-go into supernova-meta-processing

ssd04 · 2025-11-06T11:33:34Z

-func (mp *metaProcessor) getLastCrossNotarizedShardHdrs() (map[uint32]data.HeaderHandler, error) {
-	lastCrossNotarizedHeader := make(map[uint32]data.HeaderHandler, mp.shardCoordinator.NumberOfShards())
+func (mp *metaProcessor) getLastCrossNotarizedShardHdrsAndAddToCurrentBlock() (map[uint32]data.HeaderHandler, error) {
+	lastCrossNotarizedHeader, err := mp.getLastCrossNotarizedShardHeaders()


here i was referring to the variable name, it confused me at first, but not directly related to this PR, it can remain as it is

AdoAdoAdo added 3 commits October 17, 2025 12:07

update trigger removing the current round.

a40d8e3

update create new block header and sketch meta block proposal

9feb6b1

deactivate supernova in relay txs tests

52e4b1e

AdoAdoAdo and others added 15 commits October 21, 2025 18:20

renaming interface methods, move common func to base processor

17eda28

refactor miniblock selection session (renaming)

a534f5a

add component to create shard info for meta blocks

dc614f0

update metablock proposal

1b1eaf5

fixes integration tests

44a64d9

fixes

6d1e264

fixes and renaming

d9ab0f9

finalize create meta block proposal

bf4f9f4

add todo

dac8325

add unit tests for metachain create new header proposal

02f2345

more unit tests and fixes

f85d11c

unit tests and fixes

a3c97ac

unit tests

d98b17a

unit tests and fixes

aa6194d

MX-17245 Adding unit tests for shardinfo

37cee1f

sstanculeanu reviewed Oct 24, 2025

View reviewed changes

mradian1 and others added 10 commits October 25, 2025 04:37

MX-17245 coverage for createShardMiniblockHeader

24f64a5

added maps to avoid duplicates

8d06617

removed gasProvided

640c2b5

fix

e8a514f

MX-17245 Added unit tests

c4efd84

refactor & fixes

cd0277c

add new unit tests & fix after refactor

89e0745

fixes

bb8ba06

fix ineffectual assignment

914f628

remove redundant types

f8cd262

AdoAdoAdo and others added 9 commits November 3, 2025 16:22

fixes after merge

d272d30

fixes tests

eb50014

Merge branch 'feat/supernova-async-exec' into supernova-meta-processing

703982c

# Conflicts: # process/block/pendingMb/pendingMiniBlocks.go

fix after merge

f5ac835

Merge branch 'supernova-meta-processing' of https://github.com/multiv…

1053679

…ersx/mx-chain-go into shard-data-proposal-integration # Conflicts: # go.mod # go.sum

updated core-go after merge

3d33c6c

fix after review

2496cca

fixed tests

3e659c7

Merge pull request #7388 from multiversx/shard-data-proposal-integration

9ab3b65

Shard data proposal integration

ssd04 self-requested a review November 4, 2025 14:41

mradian1 and others added 6 commits November 4, 2025 17:48

MX-17245 Fixes after review

666212f

Merge remote-tracking branch 'origin/supernova-meta-processing' into …

642ab8c

…create-shardinfo-testing

MX-17245 Fixes after merge

6a5fccf

MX-17245 Fixes after review

a6ff0e5

Merge branch 'feat/supernova-async-exec' into supernova-meta-processing

3069ad9

# Conflicts: # go.sum

fix after merge

1d3fc91

ssd04 reviewed Nov 5, 2025

View reviewed changes

sstanculeanu reviewed Nov 5, 2025

View reviewed changes

mradian1 and others added 7 commits November 6, 2025 11:03

MX-17245 Fixes after review

0f1b025

MX-17245 Fixes after re-review

d6cd4cb

Merge pull request #7369 from multiversx/create-shardinfo-testing

14c4ca0

MX-17245 Adding unit tests for shardinfo

fixes after review

b71641d

Merge branch 'supernova-meta-processing' of github.com:multiversx/mx-…

58b9d43

…chain-go into supernova-meta-processing

sort imports

5e15a60

Merge branch 'feat/supernova-async-exec' into supernova-meta-processing

7faab3c

sstanculeanu approved these changes Nov 6, 2025

View reviewed changes

ssd04 reviewed Nov 6, 2025

View reviewed changes

ssd04 approved these changes Nov 6, 2025

View reviewed changes

AdoAdoAdo merged commit d97945b into feat/supernova-async-exec Nov 6, 2025
11 checks passed

AdoAdoAdo deleted the supernova-meta-processing branch November 6, 2025 11:38

	shardMiniBlockHeaders := make([]block.MiniBlockHeader, 0)
	shardMiniBlockHeaders := make([]block.MiniBlockHeader, 0, len(mbHeaderHandlers))

	func (mp *metaProcessor) hasStartOfEpochExecutionResults(metaHeader data.MetaHeaderHandler) (bool, error) {
	func hasStartOfEpochExecutionResults(metaHeader data.MetaHeaderHandler) (bool, error) {

		// TODO: pendingMiniBlocksHandler should be updated
		shardData.NumPendingMiniBlocks = uint32(len(sic.pendingMiniBlocksHandler.GetPendingMiniBlocks(header.GetShardID())))

Uh oh!

Conversation

AdoAdoAdo commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reasoning behind the pull request

Proposed changes

Testing procedure

Pre-requisites

Uh oh!

codecov Bot commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AdoAdoAdo Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AdoAdoAdo Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AdoAdoAdo Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

AdoAdoAdo commented Oct 20, 2025 •

edited

Loading

codecov Bot commented Oct 20, 2025 •

edited

Loading

AdoAdoAdo Nov 6, 2025 •

edited

Loading

AdoAdoAdo Nov 6, 2025 •

edited

Loading

AdoAdoAdo Nov 6, 2025 •

edited

Loading