Implement E2E tests for registrar pallet operations#572
Merged
rockbmb merged 112 commits intoMay 28, 2026
Merged
Conversation
…ck#604) Bumps the npm_and_yarn group with 1 update in the / directory: [uuid](https://github.com/uuidjs/uuid). Updates `uuid` from 11.1.0 to 11.1.1 - [Release notes](https://github.com/uuidjs/uuid/releases) - [Changelog](https://github.com/uuidjs/uuid/blob/v11.1.1/CHANGELOG.md) - [Commits](uuidjs/uuid@v11.1.0...v11.1.1) --- updated-dependencies: - dependency-name: uuid dependency-version: 11.1.1 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Skip coretimeKusama -> assetHubKusama XCM test Fails due to missing ConcatenatedOpaqueVersionedXcm variant in @polkadot/types XcmpMessageFormat enum. Fix pending upstream in polkadot-js/api. * Skip coretimePolkadot -> assetHubPolkadot XCM test Same root cause as coretimeKusama -> assetHubKusama: missing ConcatenatedOpaqueVersionedXcm in @polkadot/types. polkadot-js/api#6255. * Update block numbers * Apply linter
…b3-stack#608) Bumps the npm_and_yarn group with 1 update in the / directory: [ip-address](https://github.com/beaugunderson/ip-address). Updates `ip-address` from 10.1.0 to 10.2.0 - [Commits](https://github.com/beaugunderson/ip-address/commits) --- updated-dependencies: - dependency-name: ip-address dependency-version: 10.2.0 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…-web3-stack#609) * Add beforeEach to DescribeNode * Snapshot restore truncates storage layers * Fix network teardown * Share network across accounts suite * Share network across scheduler suite * Share network across bounties suite * Share network across childBounties suite * Share network across vesting suite * Share network across governance suite * Share network across multisig suite * Share network across multisig.proxy suite * Share network across nomination-pools suite * Share network across staking suite * Share network across proxy suite * Share network across remoteProxy suite * Share network across bounties suite * Share network across childBounties suite * Share network across governance suite * Share network across multisig suite * Share network across multisig.proxy suite * Share network across preimage suite * Share network across proxy suite * Share network across remoteProxy suite * Share network across staking suite * Patch chopsticks-core to expose resetStorageLayers on Block * Update yarn.lock for chopsticks-core patch * Switch CI to forks pool with 5 workers The shared-client refactor is incompatible with vitest's threads pool: multiple workers contend for the same chopsticks WS server, causing RPC timeouts and retry loops that see leaked state from earlier attempts. Forks pool gives each worker its own process and avoids the contention entirely. * Fix accounts suite teardown and await dev.setHead Two issues flagged by automated review on PR open-web3-stack#609: 1. The accounts suite's beforeEach was missing an await on baseClient.dev.setHead(blockNumber). The setHead RPC is async and the unawaited promise leaves a pending request that can reject after the test completes. 2. The relay-fallback paths in the force_transfer/force_unreserve/ force_set_balance/force_adjust_total_issuance tests called setupNetworks from inside the test body. setupNetworks registers beforeEach and afterAll hooks at the describe level, conflicting with the suite-level lifecycle and producing resource leaks. The chains that take this path (e.g. bridgeHubPolkadot, which lacks the scheduler pallet) hit assertion failures and timeouts. Replaced with createNetworks plus a per-function teardownExtras closure that disconnects and tears down both the relay and re-created base clients at the end of the test. * Await setupBalances calls in preimage tests Nine call sites kicked off the storage seed without awaiting it. The underlying setStorage RPC can reject asynchronously on chains with strict transaction-pool validation (basilisk, karura), and the unawaited rejection escaped as an unhandled rejection after the test had moved on. * Bump block numbers * Fix lint errors after shared-client refactor - postAhmFiltering: PostAhmTest.testFn was typed (chain: Chain<>) but the leaf functions it points to all take Client<>; updated the type. - preimage, proxy, scheduler: drop unused setupNetworks imports left over from the refactor. - governance: drop the dead 'chain' parameter from injectDecisionPeriodEnd (unused since the function was refactored to take a Client). * Share network across people suite * Share networks across treasury suite * Share networks across collectives suite * Share network across configuration suite * Use createNetworks with explicit teardown in upgrade suite The conditional 1- to 3-chain network creation in upgrade tests cannot be lifted to a suite-level beforeAll because the same-vs-distinct chain decision is per-test. Replaced setupNetworks (which registers describe-level beforeEach/afterAll hooks at runtime, leaking state across tests) with createNetworks plus a try/finally that disconnects every distinct client at the end of each test. * Share networks across system suite * Share network across recovery suite * Patch chopsticks-utils.sendTransaction to unsubscribe and swallow late rejections After a tx hits isInBlock/isFinalized, sendTransaction's deferred is resolved but the polkadot.js subscription stays active and keeps firing. A subsequent isError callback (e.g. a tx-pool rejection that arrives after the block was already produced) calls deferred.reject on the already-settled promise — a no-op for that promise, but the underlying signed.send subscription's own error path can still produce unhandled rejections. Two changes: explicitly unsubscribe after either terminal status, and attach a no-op catch to the deferred promise so any late settle is silenced rather than escaping as an unhandled rejection. * Update Bifrost Kusama <-> KAH XCM snap * Skip failing CoretimeK <-> KAH XCM test * Bump block numbers * Skip failing KAH <-> PeopleKusama XCM tests * Regenerate chopsticks patches from pet-perf-stack branch tip Replaces the hand-crafted patches with full rebuilds from AcalaNetwork/chopsticks#pet-perf-stack (commit e267237). Adds chopsticks-db patch covering the new PagedKeys and RpcCall entities (§1 of open-web3-stack#606). Updates chopsticks-core and chopsticks-utils patches to include all remaining commits: getKeysPaged guard refinement, getKeysPaged disk cache, cache-hit extension fix, in-flight dedup, setHead subscriber fix, and dryRunExtrinsicsAmortized. Drop when chopsticks#1028 is published. * Use dryRunExtrinsicsAmortized for proxy call-filtering loop Replaces the sendTransaction + newBlock per-proxy-action loop with a single dryRunExtrinsicsAmortized batch. Pre-signs every action with the proxy account's current nonce (storage layer pops between extrinsics revert nonce, so sequential nonces would be rejected). Adds eventsFromAmortizedDryRunResult helper that decodes the system events Vec from a per-extrinsic storageDiff and returns a { events: Promise<Codec[]> } shape compatible with checkEvents. Verified locally on Polkadot proxy filtering tests (16 of 16 pass). * Hoist accounts relay client to suite scope Each XCM-based test in the accounts suite was spinning up its own ephemeral relay+parachain pair via createNetworks inside the test body, then tearing them down with a per-test teardownExtras closure. Move that to the suite-level beforeAll: accountsE2ETests now takes an optional relayChain parameter, creates the connected pair once in beforeAll, restores both via captureSnapshot in beforeEach, and tears both down in afterAll. The 13 XCM test functions now take an already-connected relayClient instead of a relayChain plus the boilerplate to bootstrap one. Drops the relayChain field from AccountsTestConfig (it is no longer test-local data) and the TInitStoragesRelay generic from the test fns. Verified locally: peoplePolkadot.accounts (55/55 pass, uses relayClient path) and assetHubPolkadot.accounts (46/46 pass, no-relay path). * Hoist people relay client to suite scope addRegistrarViaRelayAsRoot was creating its own ephemeral relay+people pair via createNetworks on every invocation, ignoring the peopleClient already provisioned by basePeopleChainE2ETests's beforeAll. Move relay creation to suite-level beforeAll, restore both via captureSnapshot(peopleClient, relayClient) in beforeEach, and tear both down in afterAll. The function now takes the suite-shared (relayClient, peopleClient) directly. Verified: peoplePolkadot.e2e (5/5 pass). * Use more vitest forks in CI workflow
…ersionedXcm` (open-web3-stack#613) * Update dependencies * Patch `@polkadot/types` to add `XcmpMessageFormat::ConcatenatedOpaqueVersionedXcm` * Re-enable `assetHubKusama` <-> `coretimeKusama` XCM tests * Re-enable `assetHubPolkadot` <-> `coretimePolkadot` XCM tests * Re-enable `assetHubPolkadot` <-> `bridgeHubPolkadot` XCM tests * Re-enable `assetHubKusama` <-> `bridgeHubKusama` XCM tests * Update block numbers
…-web3-stack#614) * Bump dependencies (Chopsticks -> 1.4.0) This bump brings in the performance improvements from chopsticks#1027/#1028, thus removing the need for the existing patch. * Drop `@acala-network/chopsticks` patches now covered by 1.4.0 * Bump `@acala-network/chopsticks` to `^1.4.0` * Regen lockfile to drop obsolote ref to tar@npm:^6.1.11
…ack#616) * Add endpoint-health probe script scripts/probe-endpoints.mjs measures every RPC endpoint in `packages/networks/src/pet-chain-endpoints.json` and emits a per-endpoint health report sorted by score. Each endpoint is probed over a single persistent WebSocket (matching production traffic shape used by Subway and Chopsticks/polkadot.js) with SAMPLES pings paced across WINDOW_MS, mixing `chain_getHeader`, `chain_getBlockHash`, and `state_getStorage` at the pinned block from KNOWN_GOOD_BLOCK_NUMBERS_*.env. The score combines failure rate, p95 ping latency, and an archive-capable flag. Chains are probed serially by default to avoid host-IO contention inflating latency numbers; CHAIN_CONCURRENCY=N can override for local fast iterations. All other knobs (SAMPLES, WINDOW_MS, INFLIGHT, CONNECT_TIMEOUT_MS, RPC_TIMEOUT_MS, CHAINS, KEEP_PROBE_LOG) have safe defaults documented in the script header. * Add script to apply probe ranking to `pet-chain-endpoints.json` scripts/apply-probe-ranking.mjs reads `endpoint-probe.json` and rewrites `packages/networks/src/pet-chain-endpoints.json` with each chain's endpoint array sorted by ascending health score from the probe. Policy is sort, never filter: a poorly-scoring endpoint is moved to the end of the list rather than removed. This preserves the failover ladder that both Subway (via `shuffle_endpoints: false` + round-robin rotation) and Chopsticks (via polkadot.js `WsProvider`'s `#endpointIndex + 1` on reconnect) rely on to escape from a runtime stall. Bails with non-zero exit if the probe and the endpoints JSON disagree on chain set (added/removed chains require a deliberate human decision, not a silent overwrite). Idempotent: exit 0 with no rewrite if the file already matches the probe order. * Add daily endpoint-probe workflow Runs scripts/probe-endpoints.mjs over every chain in `pet-chain-endpoints.json`, then runs scripts/apply-probe-ranking.mjs to rewrite the JSON sorted by score. If the rewrite changed the file, opens a PR against master. Cadence is daily (04:17 UTC); endpoint health drifts on the order of weeks but daily caps the lag between an endpoint going bad and the JSON reflecting it. `workflow_dispatch` accepts overrides for the probe knobs (samples, window_ms, inflight, chains) so a human can re-run with tighter or looser parameters as needed. The PR is opened by the github-actions app (same token pattern as `update-known-good.yml`) and gated by human review. * Sort `pet-chain-endpoints.json` by initial probe ranking Bootstrap result from running scripts/apply-probe-ranking.mjs against the first probe of all 23 chain pools. 14 chains had a different order; 9 were already sorted by score. Notable demotions to the bottom of their respective lists (non-archive or sustained loss under repeated `state_getStorage` at the pinned block): - polkadot: `polkadot.public.curie.radiumblock.co/ws`, `polkadot-rpc.publicnode.com` - assetHubPolkadot: `statemint.public.curie.radiumblock.co/ws` - assetHubKusama: `statemine.public.curie.radiumblock.co/ws` - bridgeHubKusama: `bridgehub-kusama.public.curie.radiumblock.co/ws` The daily probe-endpoints workflow keeps this ordering current. * Disable Subway endpoint shuffling `shuffle_endpoints: true` randomly permuted the endpoint list at Subway startup, which made the ranking maintained in `pet-chain-endpoints.json` meaningless: Subway would pick whatever endpoint landed at index 0 of the shuffle, regardless of health. With shuffling off, Subway uses array[0] as its initial endpoint and treats array[1..N-1] as the round-robin failover sequence on stall. This matches the behavior of Chopsticks/polkadot.js `WsProvider` for chains that don't go through Subway, so both paths now respect the health-sorted order set by the daily probe workflow. * Add local-CI runner script scripts/run-ci-locally.sh reproduces the CI Subway setup on the local machine: it boots one Subway per chain in the chosen network using the same chain/port mapping that `.github/workflows/ci.yml` uses, waits for each `/health` endpoint, sources the matching KNOWN_GOOD_BLOCK_NUMBERS env file, exports the per-chain `<CHAIN>_ENDPOINT=ws://localhost:<port>` vars, and runs `yarn test:<network>` inside a backgrounded tmux session with two panes (Subway logs above, test output below). Cleanup on script exit (or Ctrl-C) kills the Subway PIDs but leaves `/tmp/subway-*.log` in place for after-the-fact inspection. Useful for sanity-checking endpoint-ranking changes or reproducing CI-only flakes without waiting for a GitHub Actions run. * Document scoring, pacing, and tmux/env idioms in endpoint scripts The bare scripts compiled fine but the rationale behind the non-obvious decisions wasn't preserved anywhere: - scripts/probe-endpoints.mjs: scoring weights (50 per loss-percent, 5000 archive penalty, p95 over median), the three-way method mix and why each is included, the pacing-and-inflight model, why pinned-hash failures fall back to header-only sampling, the universal System.Number key. - scripts/apply-probe-ranking.mjs: the chain-set invariant and why the per-chain disagreement branches return without mutating order. - scripts/run-ci-locally.sh: the $$-suffixed session name for parallel-run safety, the `set -a`/`set +a` env-sourcing idiom, the `${arr[*]@q}` re-quoting, the two-pane detached-tmux layout. Matches the existing inline-comment style in scripts/start-subway.sh. * Fix and polish `run-ci-locally.sh` - Replace `(( elapsed++ ))` and `(( elapsed < 60 ))` with `[[ ]]` / arith substitution. The post-increment form returns exit status 1 when the pre-increment value is 0, which trips `set -e` and kills the script immediately after Subways finish booting (before any 'ready' line prints). - Split the pane vertically (`split-window -h`) so Subway logs and test output sit side by side instead of stacked. - Auto-attach to the new session after a 3-second settle, using `switch-client` if already inside tmux and `attach-session` otherwise. - Pass `--pool=forks --maxWorkers=8` to the test command to match the CI invocation in `.github/workflows/ci.yml`. * Add `both` mode to `run-ci-locally.sh` Adds a third network argument that boots all 17 Polkadot+Kusama Subways in one tmux session and runs `yarn test` (every package). Westend is not booted because CI does not run a Westend job; Westend tests fall back to their public endpoints from `pet-chain-endpoints.json`. The two-array model also keeps the polkadot and kusama single-network modes intact: `both` is the concatenation of `POLKADOT_CHAINS` and `KUSAMA_CHAINS`, and the KNOWN_GOOD env files for the chosen networks are concatenated into the per-run env file. * Commit re-ranked endpoints directly to master Drop the `peter-evans/create-pull-request` step in favour of a `git config` / `git add` / `git commit -m '[ci skip] ...' ` / `git push` pattern matching `update-known-good.yml`. The probe is bot-driven, deterministic, and analogous to the known-good block bump (also committed directly to master), so the PR layer added no review value. * Lint fixes and Westend exclusion in `both` mode - scripts/probe-endpoints.mjs, scripts/apply-probe-ranking.mjs: Biome formatting (tabs) and template-literal lint fixes. - scripts/run-ci-locally.sh: `both` mode now invokes `vitest packages/polkadot packages/kusama` rather than `yarn test`, so Westend tests do not run (their Subway pool is not booted). - biome.json: bump schema version from 2.4.12 to 2.4.15 to match the installed CLI. * Bump block numbers
* Refactor known good block number update workflow * Add CI matrix generator script Reads chain definitions from source configs (packages/networks/src/chains/*.ts) and endpoint lists from pet-chain-endpoints.json to generate the GitHub Actions matrix JSON. Replaces hardcoded chain/port/endpoint mappings in workflows. * Replace hardcoded matrix with generator script in known-good workflow Removes the inline JSON blob and uses scripts/generate-ci-matrix.mjs. Also removes the workflow-level retry mechanism (--retry=3 in Vitest is sufficient) and adds per-chain failure notification via Vitest JSON output. * Use matrix generator script in ci.yml and update-snapshot.yml Replaces the hardcoded chain/port/endpoint matrix in both workflows with a define-matrix job that runs scripts/generate-ci-matrix.mjs.
* Add comments to recovery E2E tests * Fix tests/comments based on further review
Contributor
|
This is a massive and impressive pull request that introduces several new major features and significantly refactors the existing test suite structure for better performance and maintainability. The introduction of E2E tests for the I've found a couple of minor issues in the new Review Suggestionspackages/shared/src/registrar.ts:276 In
The packages/shared/src/registrar.ts:432 In
This storage item is likely a scripts/run-ci-locally.sh:155-157 The comment here describes a workaround for The comment is now misleading and should be removed or updated to reflect the current implementation. The code itself is correct. |
Contributor
Author
|
Updated, please take another look. @rockbmb |
rockbmb
requested changes
May 27, 2026
rockbmb
left a comment
rockbmb
left a comment
Collaborator
There was a problem hiding this comment.
LGTM; could you redact para IDs from snapshots? Then this can be merged.
I have some remarks on missing tests, but these can be handled in a future PR.
Untested error paths
register / force_register
InvalidCode,CodeTooLarge,HeadDataTooLarge(input validation invalidate_onboarding_data)- Insufficient balance for deposit delta
deregister
NotRegisteredon non-existent para ID
remove_lock
- No test for non-existent para ID (silent no-op, but unverified)
- No test for unauthorized caller (e.g. the manager) →
BadOrigin
add_lock
- Manager calling
add_lockon already-locked para →ParaLocked(not tested)
add_lock / remove_lock cycle
- No test exercises the full unlock→op succeeds→lock→op fails→unlock→op succeeds round-trip for any single extrinsic
schedule_code_upgrade
- Para origin not tested
swap, add_lock, schedule_code_upgrade, set_current_head
NotRegisteredon non-existent para ID (idem)
Contributor
Author
|
Updated. @rockbmb Will you create a new issue for the missing cases? I can pick that up later. |
rockbmb
approved these changes
May 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #303