Skip to content

Report filtered delayed transactions to filtering-report service - NIT-4644#4632

Open
mahdy-nasr wants to merge 14 commits into
masterfrom
report-filtered-delayed-transactions
Open

Report filtered delayed transactions to filtering-report service - NIT-4644#4632
mahdy-nasr wants to merge 14 commits into
masterfrom
report-filtered-delayed-transactions

Conversation

@mahdy-nasr
Copy link
Copy Markdown
Contributor

  • Replace FilteredTxHashes []common.Hash with pendingFilteredTxReports []FilteredTxReport in DelayedFilteringSequencingHooks

closes #NIT-4644

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends delayed-transaction filtering to emit structured FilteredTxReport payloads to the filtering-report service (in addition to reporting hashes to the transaction-filterer), replacing the prior “just collect tx hashes” approach in delayed sequencing.

Changes:

  • Replace FilteredTxHashes []common.Hash with pendingFilteredTxReports []FilteredTxReport in delayed sequencing hooks and enrich reports with tx RLP + delayed inbox request id.
  • Add non-blocking reporting of structured filtered-tx reports to the filtering-report RPC service while still reporting hashes to the transaction-filterer RPC.
  • Adjust a system test call-value constant and update FilteredTxReport.PositionInBlock type.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
system_tests/retryable_tickets_filtering_test.go Updates a call-value constant used in a delayed filtering cascade system test.
execution/gethexec/executionengine.go Builds and reports FilteredTxReport entries for filtered delayed txs; continues reporting hashes to the filterer; halts delayed sequencing on filtered activity.
execution/gethexec/addressfilter/filter_report.go Changes PositionInBlock type on the report struct.
changelog/mnasr-nit-4644.md Adds changelog entry for structured filtered delayed-tx reporting.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread system_tests/retryable_tickets_filtering_test.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 10, 2026

Codecov Report

❌ Patch coverage is 87.50000% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.80%. Comparing base (9d92834) to head (3ec0fd9).

❗ There is a different number of reports uploaded between BASE (9d92834) and HEAD (3ec0fd9). Click for more details.

HEAD has 2 uploads less than BASE
Flag BASE (9d92834) HEAD (3ec0fd9)
9 7
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4632      +/-   ##
==========================================
- Coverage   55.94%   48.80%   -7.15%     
==========================================
  Files         506      506              
  Lines       73739    73800      +61     
==========================================
- Hits        41252    36015    -5237     
- Misses      27271    32788    +5517     
+ Partials     5216     4997     -219     

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 10, 2026

❌ 9 Tests Failed:

Tests completed Failed Passed Skipped
4970 9 4961 0
View the top 3 failed tests by shortest run time
TestRedisProduceComplex/one_producer,_all_consumers_are_active
Stack Traces | 1.340s run time
... [CONTENT TRUNCATED: Keeping last 20 lines]
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: setting result                 �[36mcid�[0m=c78e04fb-ecc1-4146-8bde-f0542f50f387 �[36mmsgIdInStream�[0m=1776937849939-0  �[36mresultKeyInRedis�[0m=result-key:stream:644819c8-4dc7-4676-908b-06c163082173.1776937849939-0
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: xack                           �[36mcid�[0m=d75a3f3f-bda4-469f-adcb-964569947b63 �[36mmessageId�[0m=1776937849912-4
�[36mDEBUG�[0m[04-23|09:50:51.052] Redis stream consuming                   �[36mconsumer_id�[0m=9843fef4-1a9a-4f94-9b54-dbe3a1f50b6d �[36mmessage_id�[0m=1776937849921-0
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: setting result                 �[36mcid�[0m=9843fef4-1a9a-4f94-9b54-dbe3a1f50b6d �[36mmsgIdInStream�[0m=1776937849921-0  �[36mresultKeyInRedis�[0m=result-key:stream:644819c8-4dc7-4676-908b-06c163082173.1776937849921-0
�[36mDEBUG�[0m[04-23|09:50:51.052] Redis stream consuming                   �[36mconsumer_id�[0m=81eea842-b716-4ff8-b618-053c6fe23a22 �[36mmessage_id�[0m=1776937849938-0
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: setting result                 �[36mcid�[0m=81eea842-b716-4ff8-b618-053c6fe23a22 �[36mmsgIdInStream�[0m=1776937849938-0  �[36mresultKeyInRedis�[0m=result-key:stream:644819c8-4dc7-4676-908b-06c163082173.1776937849938-0
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: xack                           �[36mcid�[0m=392a8bab-2354-4c7a-9b6a-fd968cefe47b �[36mmessageId�[0m=1776937849912-7
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: xdel                           �[36mcid�[0m=d75a3f3f-bda4-469f-adcb-964569947b63 �[36mmessageId�[0m=1776937849912-4
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: xack                           �[36mcid�[0m=c78e04fb-ecc1-4146-8bde-f0542f50f387 �[36mmessageId�[0m=1776937849939-0
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: xack                           �[36mcid�[0m=9843fef4-1a9a-4f94-9b54-dbe3a1f50b6d �[36mmessageId�[0m=1776937849921-0
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: xack                           �[36mcid�[0m=81eea842-b716-4ff8-b618-053c6fe23a22 �[36mmessageId�[0m=1776937849938-0
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: xdel                           �[36mcid�[0m=c78e04fb-ecc1-4146-8bde-f0542f50f387 �[36mmessageId�[0m=1776937849939-0
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: xdel                           �[36mcid�[0m=392a8bab-2354-4c7a-9b6a-fd968cefe47b �[36mmessageId�[0m=1776937849912-7
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: xdel                           �[36mcid�[0m=9843fef4-1a9a-4f94-9b54-dbe3a1f50b6d �[36mmessageId�[0m=1776937849921-0
�[36mDEBUG�[0m[04-23|09:50:51.053] consumer: xdel                           �[36mcid�[0m=81eea842-b716-4ff8-b618-053c6fe23a22 �[36mmessageId�[0m=1776937849938-0
�[33mWARN �[0m[04-23|09:50:51.053] XClaimJustID returned empty response when indicating heartbeat �[33mmsgID�[0m=1776937849921-1
�[36mDEBUG�[0m[04-23|09:50:51.124] checkResponses                           �[36mresponded�[0m=91 �[36merrored�[0m=0 �[36mchecked�[0m=91
�[31mERROR�[0m[04-23|09:50:51.131] Error from XpendingExt in getting PEL for auto claim �[31merr�[0m="context canceled" �[31mpendingLen�[0m=0
�[36mDEBUG�[0m[04-23|09:50:51.231] Error destroying a stream group          �[36merror�[0m="dial tcp 127.0.0.1:43727: connect: connection refused"
--- FAIL: TestRedisProduceComplex/one_producer,_all_consumers_are_active (1.34s)
TestSequencerPriceAdjustsFrom25Gwei
Stack Traces | 5.270s run time
... [CONTENT TRUNCATED: Keeping last 20 lines]
INFO [04-23|10:04:09.072] Imported new potential chain segment     number=38  hash=5c69c7..42d022 blocks=1  txs=1   mgas=0.021 elapsed=1.835ms     mgasps=11.438   triediffs=193.21KiB triedirty=0.00B
INFO [04-23|10:04:09.072] Chain head was updated                   number=38  hash=5c69c7..42d022 root=bba108..3a3aba elapsed="68.938µs"
INFO [04-23|10:04:09.074] Starting work on payload                 id=0x039e1b53b4c5ef66
INFO [04-23|10:04:09.074] Updated payload                          id=0x039e1b53b4c5ef66 number=39  hash=36b162..22dda5 txs=0   withdrawals=0 gas=0         fees=0              root=713ad0..fddbf2 elapsed="844.934µs"
INFO [04-23|10:04:09.075] Stopping work on payload                 id=0x039e1b53b4c5ef66 reason=delivery
INFO [04-23|10:04:09.075] Imported new potential chain segment     number=39  hash=36b162..22dda5 blocks=1  txs=0   mgas=0.000 elapsed="911.519µs" mgasps=0.000    triediffs=196.40KiB triedirty=0.00B
INFO [04-23|10:04:09.076] Chain head was updated                   number=39  hash=36b162..22dda5 root=713ad0..fddbf2 elapsed="44.814µs"
INFO [04-23|10:04:09.079] Submitted transaction                    hash=0x3f766dbc5b327d7d3354a73bc3bcf797e672c6bd4b6cecd01bfe3a6d1eea4c22 from=0xb386a74Dcab67b66F8AC07B4f08365d37495Dd23 nonce=4   recipient=0x1e08DC5Fc1D1C226096AF4F9e6ED268397C812da value=0
INFO [04-23|10:04:09.079] DataPoster sent transaction              nonce=4   hash=3f766d..ea4c22 feeCap=10,305,217,740 tipCap=50,000,000    blobFeeCap=<nil> gas=157,566
INFO [04-23|10:04:09.080] BatchPoster: batch sent                  sequenceNumber=5   from=9   to=11  prevDelayed=3   currentDelayed=4   totalSegments=4  numBlobs=0
INFO [04-23|10:04:09.080] Submitted transaction                    hash=0x508b0bb90c6df779a1ae0fd1b1dd6725efedff6338de38108653cb9c3b6f01dc from=0x26E554a8acF9003b83495c7f45F06edCB803d4e3 nonce=7   recipient=0x26E554a8acF9003b83495c7f45F06edCB803d4e3 value=1
INFO [04-23|10:04:09.081] Starting work on payload                 id=0x0300c1083edbe1af
INFO [04-23|10:04:09.083] Submitted transaction                    hash=0x19a3135f535ac5d4d2ffaeac8cfcd637d72f5fded0eb80becbe12f945a21ce0b from=0xaF24Ca6c2831f4d4F629418b50C227DF0885613A nonce=3   recipient=0xaF24Ca6c2831f4d4F629418b50C227DF0885613A value=1
INFO [04-23|10:04:09.084] Updated payload                          id=0x0300c1083edbe1af number=40  hash=28faa5..c69a8c txs=1   withdrawals=0 gas=144,858   fees=7.2429e-06     root=28b789..53f24b elapsed=2.695ms
WARN [04-23|10:04:09.084] Error getting finalizedMessageCount from syncMonitor err="block number not supported: finalized block not found"
INFO [04-23|10:04:09.084] Stopping work on payload                 id=0x0300c1083edbe1af reason=delivery
INFO [04-23|10:04:09.087] Starting peer-to-peer node               instance=test-stack-name/linux-amd64/go1.25.9
WARN [04-23|10:04:09.087] P2P server will be useless, neither dialing nor listening
WARN [04-23|10:04:09.089] Getting file info                        dir= error="stat : no such file or directory"
--- FAIL: TestSequencerPriceAdjustsFrom25Gwei (5.27s)
TestTwoNodesSimpleLocalAnyTrust
Stack Traces | 12.700s run time
... [CONTENT TRUNCATED: Keeping last 20 lines]
        	/opt/hostedtoolcache/go/1.25.9/x64/src/testing/testing.go:1934 +0xea
        created by testing.(*T).Run in goroutine 1
        	/opt/hostedtoolcache/go/1.25.9/x64/src/testing/testing.go:1997 +0x465
        
    twonodes_test.go:56: �[31;1m [] context deadline exceeded �[0;0m
WARN [04-23|10:03:16.537] Failed to Close AnyTrust component       err="context deadline exceeded"
INFO [04-23|10:03:16.537] Persisting dirty state                   root=cc8398..b64aa1 layers=0
INFO [04-23|10:03:16.537] Persisted dirty state to disk            size=74.00B    elapsed="23.524µs"
INFO [04-23|10:03:16.538] Blockchain stopped
WARN [04-23|10:03:16.538] Failed to Close AnyTrust component       err="context deadline exceeded"
WARN [04-23|10:03:16.538] Failed to Close AnyTrust component       err="context deadline exceeded"
INFO [04-23|10:03:16.538] Persisting dirty state                   head=2   root=830b91..32c127 layers=2
INFO [04-23|10:03:16.538] Persisted dirty state to disk            size=14.88KiB  elapsed="162.402µs"
INFO [04-23|10:03:16.538] Blockchain stopped
INFO [04-23|10:03:16.539] Ethereum protocol stopped
INFO [04-23|10:03:16.539] Transaction pool stopped
INFO [04-23|10:03:16.539] Persisting dirty state                   head=62  root=19d48c..31100b layers=62
INFO [04-23|10:03:16.541] Persisted dirty state to disk            size=274.88KiB elapsed=2.063ms
INFO [04-23|10:03:16.541] Blockchain stopped
--- FAIL: TestTwoNodesSimpleLocalAnyTrust (12.70s)

📣 Thoughts on this report? Let Codecov know! | Powered by Codecov

Comment thread execution/gethexec/executionengine.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
} else {
report.TxRLP = hexutil.Bytes(txRLP)
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Add a log when the hash is not found

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in this else branch, tx.MarshalBinary() succeeded. So we won't need logging. the if part that handle failed case already do logging.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant the case where txByHash does not contain report.TxHash (the outer if tx, ok := txByHash[report.TxHash]; ok check). Even if that path is unreachable with the current code, the logic is spread across components, so adding a log in the else branch would make it less fragile and easier to debug

Comment thread execution/gethexec/executionengine.go Outdated
Copy link
Copy Markdown
Contributor

@diegoximenes diegoximenes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's extend some filtering tests to check that cmd/filtering-report is properly logging filtered transactions.

In a future PR those tests will be extended to check if the external endpoint, that cmd/filtering-report calls, received the filtered transaction report.

Comment thread execution/gethexec/addressfilter/filter_report.go Outdated
Comment thread execution/gethexec/executionengine.go
Comment thread execution/gethexec/executionengine.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
Comment thread execution/gethexec/executionengine.go
Comment thread execution/gethexec/executionengine.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
Base automatically changed from add-report-filtered-txn-api-2 to master April 17, 2026 13:46
Comment thread cmd/filtering-report/api/report_filtered_transactions.go Outdated
Comment thread arbos/block_processor.go Outdated
Comment thread arbos/block_processor.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
Comment thread execution/gethexec/executionengine.go Outdated
Comment thread system_tests/delayed_message_filter_test.go
Copy link
Copy Markdown
Contributor

@diegoximenes diegoximenes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't do a full review round this time, but as discussed here and in a meeting, since design review can take a while, let's implement the discussed test scenarios in this PR

@mahdy-nasr
Copy link
Copy Markdown
Contributor Author

I updated the PR split the txFilteredHashes from reporting lists.

But I have one concern came to my mind (in case this error scenario of MarshalBinary happened for TxRLP). The concern is that we continued the filtering process without reporting that we filtered such txn. This is also violate the expected behaviour.

But anyway, lets stick these changes so we can move forward.

Copy link
Copy Markdown
Contributor

@diegoximenes diegoximenes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • There is an issue reported by the CI that should be addressed.
  • Master was updated, chain id is now included in the report struct, which affects this PR

Comment thread arbos/block_processor.go
PreTxFilter(*params.ChainConfig, *types.Header, *state.StateDB, *arbosState.ArbosState, *types.Transaction, *arbitrum_types.ConditionalOptions, common.Address, *L1Info) error
// PostTxFilter rejects a tx after execution.
PostTxFilter(*types.Header, *state.StateDB, *arbosState.ArbosState, *types.Transaction, common.Address, uint64, *core.ExecutionResult) error
// PreTxFilter rejects a tx before execution. positionInBlock is len(receipts) at call time.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to remove this positionInBlock comment.
Arguments are not named here.

Comment thread arbos/block_processor.go
PostTxFilter(*types.Header, *state.StateDB, *arbosState.ArbosState, *types.Transaction, common.Address, uint64, *core.ExecutionResult) error
// PreTxFilter rejects a tx before execution. positionInBlock is len(receipts) at call time.
PreTxFilter(*params.ChainConfig, *types.Header, *state.StateDB, *arbosState.ArbosState, *types.Transaction, *arbitrum_types.ConditionalOptions, common.Address, *L1Info, int) error
// PostTxFilter rejects a tx after execution. positionInBlock is len(receipts) at call time.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to remove this positionInBlock comment.
Arguments are not named here.


// mockFilteringReportAPI is a test mock that records reports received via the
// filteringreport_reportFilteredTransactions RPC endpoint.
type mockFilteringReportAPI struct {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use the mocked external endpoint instead of mocked filtering report api.


// Core identity
require.Equal(t, txHash, report.TxHash)
require.NotEmpty(t, report.ID)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check it is uuid v7

require.NotNil(t, report.DelayedReportData, "delayed report data should be set")

// TxRLP should be populated
require.NotEmpty(t, report.TxRLP, "TxRLP should be populated")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check it can be unmarshalled.

require.Equal(t, txHash, report.TxHash)
require.NotEmpty(t, report.ID)
require.True(t, report.IsDelayed)
require.NotNil(t, report.DelayedReportData, "delayed report data should be set")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check content of the field.

delayedBridge, err := arbnode.NewDelayedBridge(builder.L1.Client, builder.L1Info.GetAddress("Bridge"), 0)
require.NoError(t, err)

lookupL2Tx := func(l1Receipt *types.Receipt) *types.Transaction {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a getLookupL2Tx func already, can't be used here?

// TxRLP should be populated
require.NotEmpty(t, report.TxRLP, "TxRLP should be populated")

// Block metadata: block number non-zero, parent block hash matches block N-1
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Abstract that into a function

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants