Skip to content

Commit 52a18c8

Browse files
rockbmbrockbmb
authored
Improve performance by sharing client object within test suites (#609)
* Add beforeEach to DescribeNode * Snapshot restore truncates storage layers * Fix network teardown * Share network across accounts suite * Share network across scheduler suite * Share network across bounties suite * Share network across childBounties suite * Share network across vesting suite * Share network across governance suite * Share network across multisig suite * Share network across multisig.proxy suite * Share network across nomination-pools suite * Share network across staking suite * Share network across proxy suite * Share network across remoteProxy suite * Share network across bounties suite * Share network across childBounties suite * Share network across governance suite * Share network across multisig suite * Share network across multisig.proxy suite * Share network across preimage suite * Share network across proxy suite * Share network across remoteProxy suite * Share network across staking suite * Patch chopsticks-core to expose resetStorageLayers on Block * Update yarn.lock for chopsticks-core patch * Switch CI to forks pool with 5 workers The shared-client refactor is incompatible with vitest's threads pool: multiple workers contend for the same chopsticks WS server, causing RPC timeouts and retry loops that see leaked state from earlier attempts. Forks pool gives each worker its own process and avoids the contention entirely. * Fix accounts suite teardown and await dev.setHead Two issues flagged by automated review on PR #609: 1. The accounts suite's beforeEach was missing an await on baseClient.dev.setHead(blockNumber). The setHead RPC is async and the unawaited promise leaves a pending request that can reject after the test completes. 2. The relay-fallback paths in the force_transfer/force_unreserve/ force_set_balance/force_adjust_total_issuance tests called setupNetworks from inside the test body. setupNetworks registers beforeEach and afterAll hooks at the describe level, conflicting with the suite-level lifecycle and producing resource leaks. The chains that take this path (e.g. bridgeHubPolkadot, which lacks the scheduler pallet) hit assertion failures and timeouts. Replaced with createNetworks plus a per-function teardownExtras closure that disconnects and tears down both the relay and re-created base clients at the end of the test. * Await setupBalances calls in preimage tests Nine call sites kicked off the storage seed without awaiting it. The underlying setStorage RPC can reject asynchronously on chains with strict transaction-pool validation (basilisk, karura), and the unawaited rejection escaped as an unhandled rejection after the test had moved on. * Bump block numbers * Fix lint errors after shared-client refactor - postAhmFiltering: PostAhmTest.testFn was typed (chain: Chain<>) but the leaf functions it points to all take Client<>; updated the type. - preimage, proxy, scheduler: drop unused setupNetworks imports left over from the refactor. - governance: drop the dead 'chain' parameter from injectDecisionPeriodEnd (unused since the function was refactored to take a Client). * Share network across people suite * Share networks across treasury suite * Share networks across collectives suite * Share network across configuration suite * Use createNetworks with explicit teardown in upgrade suite The conditional 1- to 3-chain network creation in upgrade tests cannot be lifted to a suite-level beforeAll because the same-vs-distinct chain decision is per-test. Replaced setupNetworks (which registers describe-level beforeEach/afterAll hooks at runtime, leaking state across tests) with createNetworks plus a try/finally that disconnects every distinct client at the end of each test. * Share networks across system suite * Share network across recovery suite * Patch chopsticks-utils.sendTransaction to unsubscribe and swallow late rejections After a tx hits isInBlock/isFinalized, sendTransaction's deferred is resolved but the polkadot.js subscription stays active and keeps firing. A subsequent isError callback (e.g. a tx-pool rejection that arrives after the block was already produced) calls deferred.reject on the already-settled promise — a no-op for that promise, but the underlying signed.send subscription's own error path can still produce unhandled rejections. Two changes: explicitly unsubscribe after either terminal status, and attach a no-op catch to the deferred promise so any late settle is silenced rather than escaping as an unhandled rejection. * Update Bifrost Kusama <-> KAH XCM snap * Skip failing CoretimeK <-> KAH XCM test * Bump block numbers * Skip failing KAH <-> PeopleKusama XCM tests * Regenerate chopsticks patches from pet-perf-stack branch tip Replaces the hand-crafted patches with full rebuilds from AcalaNetwork/chopsticks#pet-perf-stack (commit e267237). Adds chopsticks-db patch covering the new PagedKeys and RpcCall entities (§1 of #606). Updates chopsticks-core and chopsticks-utils patches to include all remaining commits: getKeysPaged guard refinement, getKeysPaged disk cache, cache-hit extension fix, in-flight dedup, setHead subscriber fix, and dryRunExtrinsicsAmortized. Drop when chopsticks#1028 is published. * Use dryRunExtrinsicsAmortized for proxy call-filtering loop Replaces the sendTransaction + newBlock per-proxy-action loop with a single dryRunExtrinsicsAmortized batch. Pre-signs every action with the proxy account's current nonce (storage layer pops between extrinsics revert nonce, so sequential nonces would be rejected). Adds eventsFromAmortizedDryRunResult helper that decodes the system events Vec from a per-extrinsic storageDiff and returns a { events: Promise<Codec[]> } shape compatible with checkEvents. Verified locally on Polkadot proxy filtering tests (16 of 16 pass). * Hoist accounts relay client to suite scope Each XCM-based test in the accounts suite was spinning up its own ephemeral relay+parachain pair via createNetworks inside the test body, then tearing them down with a per-test teardownExtras closure. Move that to the suite-level beforeAll: accountsE2ETests now takes an optional relayChain parameter, creates the connected pair once in beforeAll, restores both via captureSnapshot in beforeEach, and tears both down in afterAll. The 13 XCM test functions now take an already-connected relayClient instead of a relayChain plus the boilerplate to bootstrap one. Drops the relayChain field from AccountsTestConfig (it is no longer test-local data) and the TInitStoragesRelay generic from the test fns. Verified locally: peoplePolkadot.accounts (55/55 pass, uses relayClient path) and assetHubPolkadot.accounts (46/46 pass, no-relay path). * Hoist people relay client to suite scope addRegistrarViaRelayAsRoot was creating its own ephemeral relay+people pair via createNetworks on every invocation, ignoring the peopleClient already provisioned by basePeopleChainE2ETests's beforeAll. Move relay creation to suite-level beforeAll, restore both via captureSnapshot(peopleClient, relayClient) in beforeEach, and tear both down in afterAll. The function now takes the suite-shared (relayClient, peopleClient) directly. Verified: peoplePolkadot.e2e (5/5 pass). * Use more vitest forks in CI workflow
1 parent 918f631 commit 52a18c8

46 files changed

Lines changed: 3809 additions & 2558 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -140,7 +140,7 @@ jobs:
140140
done
141141
142142
- name: Run ${{ matrix.network.name }} tests
143-
run: yarn test:${{ matrix.network.name }} --pool=threads --maxWorkers=3 --retry=3
143+
run: yarn test:${{ matrix.network.name }} --pool=forks --maxWorkers=8 --retry=3
144144

145145
- name: Cleanup Subway instances
146146
if: always()

‎.yarn/patches/@acala-network-chopsticks-core-npm-1.3.1-9f30da61ad.patch‎

Lines changed: 945 additions & 0 deletions
Large diffs are not rendered by default.

‎.yarn/patches/@acala-network-chopsticks-db-npm-1.3.1-3e5ee4ba00.patch‎

Lines changed: 301 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,301 @@
1+
diff --git a/dist/cjs/base-sql.d.ts b/dist/cjs/base-sql.d.ts
2+
index 04c6083e33ca834e3e46599cc1b9f02ec6d2e08a..697b5010b7d311b89bfd87c2b732ca3bae16a095 100644
3+
--- a/dist/cjs/base-sql.d.ts
4+
+++ b/dist/cjs/base-sql.d.ts
5+
@@ -13,4 +13,8 @@ export declare abstract class BaseSqlDatabase implements Database {
6+
saveStorage(blockHash: HexString, key: HexString, value: HexString | null): Promise<void>;
7+
saveStorageBatch(entries: KeyValueEntry[]): Promise<void>;
8+
queryStorage(blockHash: HexString, key: HexString): Promise<KeyValueEntry | null>;
9+
+ queryPagedKeys(blockHash: HexString, prefix: HexString): Promise<HexString[] | null>;
10+
+ savePagedKeys(blockHash: HexString, prefix: HexString, keys: HexString[]): Promise<void>;
11+
+ queryRpcCall(scope: string, method: string, params: string): Promise<string | null>;
12+
+ saveRpcCall(scope: string, method: string, params: string, result: string): Promise<void>;
13+
}
14+
diff --git a/dist/cjs/base-sql.js b/dist/cjs/base-sql.js
15+
index 5734ae19b072c0a878ae8aca70e1c7392075c579..f10d667006466c265564def304271733d0de6d3f 100644
16+
--- a/dist/cjs/base-sql.js
17+
+++ b/dist/cjs/base-sql.js
18+
@@ -107,6 +107,51 @@ class BaseSqlDatabase {
19+
}
20+
});
21+
}
22+
+ async queryPagedKeys(blockHash, prefix) {
23+
+ const db = await this.datasource;
24+
+ const row = await db.getRepository(_entities.PagedKeys).findOne({
25+
+ where: {
26+
+ blockHash,
27+
+ prefix
28+
+ }
29+
+ });
30+
+ return row ? JSON.parse(row.keys) : null;
31+
+ }
32+
+ async savePagedKeys(blockHash, prefix, keys) {
33+
+ const db = await this.datasource;
34+
+ await db.getRepository(_entities.PagedKeys).upsert({
35+
+ blockHash,
36+
+ prefix,
37+
+ keys: JSON.stringify(keys)
38+
+ }, [
39+
+ 'blockHash',
40+
+ 'prefix'
41+
+ ]);
42+
+ }
43+
+ async queryRpcCall(scope, method, params) {
44+
+ const db = await this.datasource;
45+
+ const row = await db.getRepository(_entities.RpcCall).findOne({
46+
+ where: {
47+
+ scope,
48+
+ method,
49+
+ params
50+
+ }
51+
+ });
52+
+ return row?.result ?? null;
53+
+ }
54+
+ async saveRpcCall(scope, method, params, result) {
55+
+ const db = await this.datasource;
56+
+ await db.getRepository(_entities.RpcCall).upsert({
57+
+ scope,
58+
+ method,
59+
+ params,
60+
+ result
61+
+ }, [
62+
+ 'scope',
63+
+ 'method',
64+
+ 'params'
65+
+ ]);
66+
+ }
67+
constructor(){
68+
_define_property(this, "close", async ()=>{
69+
const db = await this.datasource;
70+
diff --git a/dist/cjs/db/entities.d.ts b/dist/cjs/db/entities.d.ts
71+
index 879c444011b699b4a7bf4068c8874c2895209405..24cb0016d0535caf10a644591f6b3becd989dfdb 100644
72+
--- a/dist/cjs/db/entities.d.ts
73+
+++ b/dist/cjs/db/entities.d.ts
74+
@@ -1,4 +1,17 @@
75+
import type { BlockEntry, KeyValueEntry } from '@acala-network/chopsticks-core';
76+
import { EntitySchema } from 'typeorm';
77+
+export type PagedKeysEntry = {
78+
+ blockHash: string;
79+
+ prefix: string;
80+
+ keys: string;
81+
+};
82+
+export type RpcCallEntry = {
83+
+ scope: string;
84+
+ method: string;
85+
+ params: string;
86+
+ result: string;
87+
+};
88+
export declare const KeyValuePair: EntitySchema<KeyValueEntry>;
89+
export declare const BlockEntity: EntitySchema<BlockEntry>;
90+
+export declare const PagedKeys: EntitySchema<PagedKeysEntry>;
91+
+export declare const RpcCall: EntitySchema<RpcCallEntry>;
92+
diff --git a/dist/cjs/db/entities.js b/dist/cjs/db/entities.js
93+
index 5e19c84a85685cc01772a85c104d642ed50e4f8a..e03ba46a3abdb0e4b23196604537d0aebed5669a 100644
94+
--- a/dist/cjs/db/entities.js
95+
+++ b/dist/cjs/db/entities.js
96+
@@ -14,6 +14,12 @@ _export(exports, {
97+
},
98+
get KeyValuePair () {
99+
return KeyValuePair;
100+
+ },
101+
+ get PagedKeys () {
102+
+ return PagedKeys;
103+
+ },
104+
+ get RpcCall () {
105+
+ return RpcCall;
106+
}
107+
});
108+
const _typeorm = require("typeorm");
109+
@@ -66,3 +72,46 @@ const BlockEntity = new _typeorm.EntitySchema({
110+
}
111+
}
112+
});
113+
+const PagedKeys = new _typeorm.EntitySchema({
114+
+ name: 'PagedKeys',
115+
+ columns: {
116+
+ blockHash: {
117+
+ primary: true,
118+
+ type: 'varchar',
119+
+ nullable: false
120+
+ },
121+
+ prefix: {
122+
+ primary: true,
123+
+ type: 'varchar',
124+
+ nullable: false
125+
+ },
126+
+ keys: {
127+
+ type: 'text',
128+
+ nullable: false
129+
+ }
130+
+ }
131+
+});
132+
+const RpcCall = new _typeorm.EntitySchema({
133+
+ name: 'RpcCall',
134+
+ columns: {
135+
+ scope: {
136+
+ primary: true,
137+
+ type: 'varchar',
138+
+ nullable: false
139+
+ },
140+
+ method: {
141+
+ primary: true,
142+
+ type: 'varchar',
143+
+ nullable: false
144+
+ },
145+
+ params: {
146+
+ primary: true,
147+
+ type: 'varchar',
148+
+ nullable: false
149+
+ },
150+
+ result: {
151+
+ type: 'text',
152+
+ nullable: false
153+
+ }
154+
+ }
155+
+});
156+
diff --git a/dist/esm/base-sql.d.ts b/dist/esm/base-sql.d.ts
157+
index 04c6083e33ca834e3e46599cc1b9f02ec6d2e08a..697b5010b7d311b89bfd87c2b732ca3bae16a095 100644
158+
--- a/dist/esm/base-sql.d.ts
159+
+++ b/dist/esm/base-sql.d.ts
160+
@@ -13,4 +13,8 @@ export declare abstract class BaseSqlDatabase implements Database {
161+
saveStorage(blockHash: HexString, key: HexString, value: HexString | null): Promise<void>;
162+
saveStorageBatch(entries: KeyValueEntry[]): Promise<void>;
163+
queryStorage(blockHash: HexString, key: HexString): Promise<KeyValueEntry | null>;
164+
+ queryPagedKeys(blockHash: HexString, prefix: HexString): Promise<HexString[] | null>;
165+
+ savePagedKeys(blockHash: HexString, prefix: HexString, keys: HexString[]): Promise<void>;
166+
+ queryRpcCall(scope: string, method: string, params: string): Promise<string | null>;
167+
+ saveRpcCall(scope: string, method: string, params: string, result: string): Promise<void>;
168+
}
169+
diff --git a/dist/esm/base-sql.js b/dist/esm/base-sql.js
170+
index 539cb415345dc9ac4eb0d32eb9e4bbda6e54df75..08739df5d4f3676e0c4d99c3e5ddd415caee854a 100644
171+
--- a/dist/esm/base-sql.js
172+
+++ b/dist/esm/base-sql.js
173+
@@ -1,4 +1,4 @@
174+
-import { BlockEntity, KeyValuePair } from './db/entities.js';
175+
+import { BlockEntity, KeyValuePair, PagedKeys, RpcCall } from './db/entities.js';
176+
export class BaseSqlDatabase {
177+
close = async ()=>{
178+
const db = await this.datasource;
179+
@@ -88,4 +88,49 @@ export class BaseSqlDatabase {
180+
}
181+
});
182+
}
183+
+ async queryPagedKeys(blockHash, prefix) {
184+
+ const db = await this.datasource;
185+
+ const row = await db.getRepository(PagedKeys).findOne({
186+
+ where: {
187+
+ blockHash,
188+
+ prefix
189+
+ }
190+
+ });
191+
+ return row ? JSON.parse(row.keys) : null;
192+
+ }
193+
+ async savePagedKeys(blockHash, prefix, keys) {
194+
+ const db = await this.datasource;
195+
+ await db.getRepository(PagedKeys).upsert({
196+
+ blockHash,
197+
+ prefix,
198+
+ keys: JSON.stringify(keys)
199+
+ }, [
200+
+ 'blockHash',
201+
+ 'prefix'
202+
+ ]);
203+
+ }
204+
+ async queryRpcCall(scope, method, params) {
205+
+ const db = await this.datasource;
206+
+ const row = await db.getRepository(RpcCall).findOne({
207+
+ where: {
208+
+ scope,
209+
+ method,
210+
+ params
211+
+ }
212+
+ });
213+
+ return row?.result ?? null;
214+
+ }
215+
+ async saveRpcCall(scope, method, params, result) {
216+
+ const db = await this.datasource;
217+
+ await db.getRepository(RpcCall).upsert({
218+
+ scope,
219+
+ method,
220+
+ params,
221+
+ result
222+
+ }, [
223+
+ 'scope',
224+
+ 'method',
225+
+ 'params'
226+
+ ]);
227+
+ }
228+
}
229+
diff --git a/dist/esm/db/entities.d.ts b/dist/esm/db/entities.d.ts
230+
index 879c444011b699b4a7bf4068c8874c2895209405..24cb0016d0535caf10a644591f6b3becd989dfdb 100644
231+
--- a/dist/esm/db/entities.d.ts
232+
+++ b/dist/esm/db/entities.d.ts
233+
@@ -1,4 +1,17 @@
234+
import type { BlockEntry, KeyValueEntry } from '@acala-network/chopsticks-core';
235+
import { EntitySchema } from 'typeorm';
236+
+export type PagedKeysEntry = {
237+
+ blockHash: string;
238+
+ prefix: string;
239+
+ keys: string;
240+
+};
241+
+export type RpcCallEntry = {
242+
+ scope: string;
243+
+ method: string;
244+
+ params: string;
245+
+ result: string;
246+
+};
247+
export declare const KeyValuePair: EntitySchema<KeyValueEntry>;
248+
export declare const BlockEntity: EntitySchema<BlockEntry>;
249+
+export declare const PagedKeys: EntitySchema<PagedKeysEntry>;
250+
+export declare const RpcCall: EntitySchema<RpcCallEntry>;
251+
diff --git a/dist/esm/db/entities.js b/dist/esm/db/entities.js
252+
index b9acbe3aba398e7ddd034efeca4e1a081b478a83..bc92dd457944aed10638dd3d66faa9cb8bd4b031 100644
253+
--- a/dist/esm/db/entities.js
254+
+++ b/dist/esm/db/entities.js
255+
@@ -48,3 +48,46 @@ export const BlockEntity = new EntitySchema({
256+
}
257+
}
258+
});
259+
+export const PagedKeys = new EntitySchema({
260+
+ name: 'PagedKeys',
261+
+ columns: {
262+
+ blockHash: {
263+
+ primary: true,
264+
+ type: 'varchar',
265+
+ nullable: false
266+
+ },
267+
+ prefix: {
268+
+ primary: true,
269+
+ type: 'varchar',
270+
+ nullable: false
271+
+ },
272+
+ keys: {
273+
+ type: 'text',
274+
+ nullable: false
275+
+ }
276+
+ }
277+
+});
278+
+export const RpcCall = new EntitySchema({
279+
+ name: 'RpcCall',
280+
+ columns: {
281+
+ scope: {
282+
+ primary: true,
283+
+ type: 'varchar',
284+
+ nullable: false
285+
+ },
286+
+ method: {
287+
+ primary: true,
288+
+ type: 'varchar',
289+
+ nullable: false
290+
+ },
291+
+ params: {
292+
+ primary: true,
293+
+ type: 'varchar',
294+
+ nullable: false
295+
+ },
296+
+ result: {
297+
+ type: 'text',
298+
+ nullable: false
299+
+ }
300+
+ }
301+
+});

‎.yarn/patches/@acala-network-chopsticks-utils-npm-1.3.1-1c404ca1be.patch‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
diff --git a/dist/cjs/index.js b/dist/cjs/index.js
2+
index 9cf516b64e3c90359229c86745cc27cfc823756f..68a4dfb2ef3443eaf662482bd64d66fbf0f904c2 100644
3+
--- a/dist/cjs/index.js
4+
+++ b/dist/cjs/index.js
5+
@@ -178,15 +178,21 @@ function defer() {
6+
const sendTransaction = async (tx)=>{
7+
const signed = await tx;
8+
const deferred = defer();
9+
- await signed.send((status)=>{
10+
+ // Swallow rejections that arrive after the deferred has already settled (the
11+
+ // subscription continues to fire callbacks after isInBlock/isFinalized).
12+
+ deferred.promise.catch(()=>{});
13+
+ let unsub;
14+
+ unsub = await signed.send((status)=>{
15+
logger.debug({
16+
status: status.status.toHuman()
17+
}, 'Transaction status');
18+
if (status.isInBlock || status.isFinalized) {
19+
deferred.resolve(status.events);
20+
+ if (unsub) unsub();
21+
}
22+
if (status.isError) {
23+
deferred.reject(status.status);
24+
+ if (unsub) unsub();
25+
}
26+
});
27+
return {
28+
diff --git a/dist/esm/index.js b/dist/esm/index.js
29+
index 041cd24640792496bcc04c876d4471589e5b91c5..6509ac6f609c0abcef9996b563181930ae933413 100644
30+
--- a/dist/esm/index.js
31+
+++ b/dist/esm/index.js
32+
@@ -152,15 +152,21 @@ export * from './signFake.js';
33+
*/ export const sendTransaction = async (tx)=>{
34+
const signed = await tx;
35+
const deferred = defer();
36+
- await signed.send((status)=>{
37+
+ // Swallow rejections that arrive after the deferred has already settled (the
38+
+ // subscription continues to fire callbacks after isInBlock/isFinalized).
39+
+ deferred.promise.catch(()=>{});
40+
+ let unsub;
41+
+ unsub = await signed.send((status)=>{
42+
logger.debug({
43+
status: status.status.toHuman()
44+
}, 'Transaction status');
45+
if (status.isInBlock || status.isFinalized) {
46+
deferred.resolve(status.events);
47+
+ if (unsub) unsub();
48+
}
49+
if (status.isError) {
50+
deferred.reject(status.status);
51+
+ if (unsub) unsub();
52+
}
53+
});
54+
return {

‎.yarn/patches/@polkadot-types-npm-16.5.6-6fe2703ed8.patch‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
diff --git a/cjs/interfaces/xcm/definitions.js b/cjs/interfaces/xcm/definitions.js
2+
index b3b00ebb14cc89292378f1bb2ec329bf14cb90bf..b43d3e0e2a72a5bae4db35be517466de92f81e02 100644
3+
--- a/cjs/interfaces/xcm/definitions.js
4+
+++ b/cjs/interfaces/xcm/definitions.js
5+
@@ -15,7 +15,7 @@ const xcm = {
6+
}
7+
},
8+
XcmpMessageFormat: {
9+
- _enum: ['ConcatenatedVersionedXcm', 'ConcatenatedEncodedBlob', 'Signals']
10+
+ _enum: ['ConcatenatedVersionedXcm', 'ConcatenatedEncodedBlob', 'Signals', 'ConcatenatedOpaqueVersionedXcm']
11+
},
12+
XcmAssetId: {
13+
_enum: {
14+
diff --git a/interfaces/xcm/definitions.js b/interfaces/xcm/definitions.js
15+
index 2753b56a2b27577b63cca658d37fd6b4d1482f53..32a481113255036019b41f2f76445155eb9970bd 100644
16+
--- a/interfaces/xcm/definitions.js
17+
+++ b/interfaces/xcm/definitions.js
18+
@@ -13,7 +13,7 @@ const xcm = {
19+
}
20+
},
21+
XcmpMessageFormat: {
22+
- _enum: ['ConcatenatedVersionedXcm', 'ConcatenatedEncodedBlob', 'Signals']
23+
+ _enum: ['ConcatenatedVersionedXcm', 'ConcatenatedEncodedBlob', 'Signals', 'ConcatenatedOpaqueVersionedXcm']
24+
},
25+
XcmAssetId: {
26+
_enum: {

0 commit comments

Comments
 (0)