Add pending tx and auto relay mechanism in RPC and CLI

[superboyiii]

Motivation: Transactions can fail relay with NotYetValid when ValidUntilBlock is too far ahead of the current chain head. This PR adds the DeferredRelay plugin so nodes can queue those transactions locally (when enabled) and retry relay later, instead of dropping them immediately.

DeferredRelay.json: Under PluginConfiguration, set MaxTransactions and CheckFrequency both > 0 to enable the feature; CheckFrequency sets how often (in persisted blocks) the node rescans the local pending store. Requires the RpcServer plugin.

CLI: New command list pending — prints the JSON snapshot of locally queued NotYetValid transactions and related height / plugin settings (same data shape the RPC exposes).

RPC: New method getpendingvaliduntilrelay (no params) — returns that snapshot as JSON; when disabled or the store is not active, enabled is false. sendrawtransaction still returns -510 with data: "NotYetValid"; when DeferredRelay is enabled, qualifying txs are queued automatically via relay events.

[codecov]

Codecov Report

❌ Patch coverage is 65.33865% with 87 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.94%. Comparing base (273d729) to head (47d0ea0).

Files with missing lines	Patch %	Lines
plugins/DeferredRelay/DeferredRelayPlugin.cs	0.00%	45 Missing ⚠️
plugins/DeferredRelay/DeferredRelayEngine.cs	78.04%	20 Missing and 7 partials ⚠️
plugins/DeferredRelay/DeferredRelayActor.cs	83.78%	4 Missing and 2 partials ⚠️
src/Neo.CLI/CLI/MainService.Wallet.cs	0.00%	5 Missing ⚠️
src/Neo.CLI/CLI/MainService.Network.cs	60.00%	1 Missing and 1 partial ⚠️
plugins/DeferredRelay/DeferredRelaySettings.cs	96.00%	1 Missing ⚠️
src/Neo.CLI/CLI/MainService.cs	75.00%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##           master-n3    #1047      +/-   ##
=============================================
+ Coverage      48.49%   48.94%   +0.44%     
=============================================
  Files            274      278       +4     
  Lines          15850    16089     +239     
  Branches        2023     2057      +34     
=============================================
+ Hits            7687     7875     +188     
- Misses          7627     7665      +38     
- Partials         536      549      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Wi1l-B0t]

More unit tests?

[vncoelho]

I think this should be an extra plugin as an optional node service.

[shargon]

Maybe it's easier allow the user to config a higer value than MaxValidUntilBlockIncrement

if (ValidUntilBlock <= height || ValidUntilBlock > height + snapshot.GetMaxValidUntilBlockIncrement(settings))
    return VerifyResult.Expired;

We already have ProtocolSettings in tx.VerifyStateDependent, so if the node wants to retain higher values, it can simply set a higher one

[superboyiii]

Maybe it's easier allow the user to config a higer value than MaxValidUntilBlockIncrement

if (ValidUntilBlock <= height || ValidUntilBlock > height + snapshot.GetMaxValidUntilBlockIncrement(settings))
    return VerifyResult.Expired;

We already have ProtocolSettings in tx.VerifyStateDependent, so if the node wants to retain higher values, it can simply set a higher one

According to what I discussed with @roman-khimov previously, for safety reasons, we even tend to narrow the MaxValidUntilBlockIncrement period. Especially for the multi-sig scene, currently we only have 1/5 day as MaxValidUntilBlockIncrement. We urgently need this storing pending tx and auto relay mechanism. In furure, we recommend constructing transactions and collecting signatures in early time and set validuntilblock at enough time after Currnet height + MaxValidUntilBlockIncrement .

[superboyiii]

I think this should be an extra plugin as an optional node service.

It's not complex, the trigger is relay command in CLI and RPC, so it's not worth creating a new plugin just for this.

[superboyiii]

More unit tests?

Done

[vncoelho]

Maybe it's easier allow the user to config a higer value than MaxValidUntilBlockIncrement

if (ValidUntilBlock <= height || ValidUntilBlock > height + snapshot.GetMaxValidUntilBlockIncrement(settings))
    return VerifyResult.Expired;

We already have ProtocolSettings in tx.VerifyStateDependent, so if the node wants to retain higher values, it can simply set a higher one

According to what I discussed with @roman-khimov previously, for safety reasons, we even tend to narrow the MaxValidUntilBlockIncrement period. Especially for the multi-sig scene, currently we only have 1/5 day as MaxValidUntilBlockIncrement. We urgently need this storing pending tx and auto relay mechanism. In furure, we recommend constructing transactions and collecting signatures in early time and set validuntilblock at enough time after Currnet height + MaxValidUntilBlockIncrement .

I do not think this is the correct way to go, adding an "extra mempool".

[superboyiii]

I do not think this is the correct way to go, adding an "extra mempool".

It's a very little local IStore which will not be synced to peers. It's only for temporary use and it will be removed when the height is passed. In terms of resource consumption, it is imperceptible and almost negligible.

[vncoelho]

I do not think this is the correct way to go, adding an "extra mempool".

It's a very little local mempool which will not be synced to peers. It's only for temporary use and it will be removed when the height is passed. In terms of resource consumption, it is imperceptible and almost negligible.

Since it is not synced, it is indeed a local mempool.

However, in my opinion, it should be a plugin.

[superboyiii]

However, in my opinion, it should be a plugin.

It's not active on default. Not a problem. A new plugin means new command and new RPC, it's not necessary.

[shargon]

Maybe it's easier allow the user to config a higer value than MaxValidUntilBlockIncrement

if (ValidUntilBlock <= height || ValidUntilBlock > height + snapshot.GetMaxValidUntilBlockIncrement(settings))
    return VerifyResult.Expired;

We already have ProtocolSettings in tx.VerifyStateDependent, so if the node wants to retain higher values, it can simply set a higher one

According to what I discussed with @roman-khimov previously, for safety reasons, we even tend to narrow the MaxValidUntilBlockIncrement period. Especially for the multi-sig scene, currently we only have 1/5 day as MaxValidUntilBlockIncrement. We urgently need this storing pending tx and auto relay mechanism. In furure, we recommend constructing transactions and collecting signatures in early time and set validuntilblock at enough time after Currnet height + MaxValidUntilBlockIncrement .

It's a bypass of MaxValidUntilBlockIncrement, ao whats the difference with this new mempool or having the tx in the original mempool?

[superboyiii]

According to what I discussed with @roman-khimov previously, for safety reasons, we even tend to narrow the MaxValidUntilBlockIncrement period. Especially for the multi-sig scene, currently we only have 1/5 day as MaxValidUntilBlockIncrement. We urgently need this storing pending tx and auto relay mechanism. In furure, we recommend constructing transactions and collecting signatures in early time and set validuntilblock at enough time after Currnet height + MaxValidUntilBlockIncrement .

It's a bypass of MaxValidUntilBlockIncrement, ao whats the difference with this new mempool or having the tx in the original mempool?

This new one is an isolated IStore. It's not in cache, so it's not mempool in fact. Such kind of pending tx will not go into original mempool before because it's verified as Expired. But this time, I make a chance for these txs go into this isolated IStore. They're pending until correct time, then they will be relayed automatically and be removed from this isolated IStore.

[roman-khimov]

I tend to agree with @vncoelho on this, an additional plugin makes sense. Overriding MVUBI locally to put into the main mempool is not a solution, there are subscriptions to mempool, there are interfaces to obtain pooled transactions, there are rebroadcasts tied to mempool, CNs get their transactions from the same main mempool as well. We can't have invalid (as per protocol) transactions there, something will break for sure this way.

So given that it's a separate store the only question is where to put it. Currently it's tied to CLI, but it'd not dependent on CLI (transactions can be pushed via RPC and CLI code shouldn't care much about where transactions are stored), so a separate plugin would be structurally more sound.

[superboyiii]

I will wait for conclusion on neo-project/neo#4545, so I can decide how to make changes. Since @vncoelho and @roman-khimov prefer a Plugin way, it's OK for me.

[Jim8y]

Since neo-project/neo#4545 has been merged, this PR needs to be updated before the design can be re-reviewed. The far-future ValidUntilBlock case no longer maps to VerifyResult.Expired; after rebasing onto current master-n3, it should use the new VerifyResult.ValidUntilBlockTooFarInFuture result explicitly. Otherwise those transactions will not enter the pending store through the CLI/RPC offer path.

[Wi1l-B0t]

Conflicts

[superboyiii]

Since neo-project/neo#4545 has been merged, this PR needs to be updated before the design can be re-reviewed. The far-future ValidUntilBlock case no longer maps to VerifyResult.Expired; after rebasing onto current master-n3, it should use the new VerifyResult.ValidUntilBlockTooFarInFuture result explicitly. Otherwise those transactions will not enter the pending store through the CLI/RPC offer path.

I will make it as a plugin. Need a little time to modify.

[superboyiii]

@shargon @ajara87 Please review again.

[shargon]

It can relay to a different network, so this value should be taken from the ProtocolSettings

[superboyiii]

It can relay to a different network, so this value should be taken from the ProtocolSettings

No. It's no way to relay to another chain. If DeferredRelay.json Network ≠ the node’s ProtocolSettings.Network, OnSystemLoaded returns immediately: no store, no actor, no queue, no retry. The plugin does nothing on that node. Cross-chain relay is not possible here; The worst case is a silent misconfiguration (you think DeferredRelay is on, but it isn’t).

[shargon]

Why the exclude?

[superboyiii]

Why the exclude?

RpcClient is not Plugin, it's SDK. I don't release it in zip package. It will be only update to Nuget.
MPTTire is packed inside StateService.zip, so it's not necessary to be packed alone again.