Skip to content

Commit fa5f8bc

Browse files
javiergarciaverametamaskbotsethkfman
authored
test: Fix for performance tests (MetaMask#22606)
<!-- Please submit this PR as a draft initially. Do not mark it as "Ready for review" until the template has been completely filled out, and PR status checks have passed at least once. --> ## **Description** this is a PR that fixes: performance build issues test changes according to the new UI test flakiness fixes appwright patch to improve some DOM issues we had finding locators due to nested components <!-- Write a short description of the changes included in this pull request, also include relevant motivation and context. Have in mind the following questions: 1. What is the reason for the change? 2. What is the improvement/solution? --> ## **Changelog** <!-- If this PR is not End-User-Facing and should not show up in the CHANGELOG, you can choose to either: 1. Write `CHANGELOG entry: null` 2. Label with `no-changelog` If this PR is End-User-Facing, please write a short User-Facing description in the past tense like: `CHANGELOG entry: Added a new tab for users to see their NFTs` `CHANGELOG entry: Fixed a bug that was causing some NFTs to flicker` (This helps the Release Engineer do their job more quickly and accurately) --> CHANGELOG entry: ## **Related issues** Fixes: ## **Manual testing steps** ```gherkin Feature: my feature name Scenario: user [verb for user action] Given [describe expected initial app state] When user [verb for user action] Then [describe expected outcome] ``` ## **Screenshots/Recordings** <!-- If applicable, add screenshots and/or recordings to visualize the before and after of your change. --> ### **Before** <!-- [screenshots/recordings] --> ### **After** <!-- [screenshots/recordings] --> ## **Pre-merge author checklist** - [ ] I’ve followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Mobile Coding Standards](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/CODING_GUIDELINES.md). - [ ] I've completed the PR template to the best of my ability - [ ] I’ve included tests if applicable - [ ] I’ve documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [ ] I’ve applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Patches Appwright (locators, BrowserStack settings) and refactors performance tests/screens with new selectors, utilities, and modal handling to reduce flakiness and match the updated UI. > > - **Appwright (patched via Yarn)**: > - Lower retries/workers in default config; adjust element lookup to skip text matching; minor logging. > - BrowserStack: add/update capabilities (networkProfile, selfHeal, appProfiling), env‑driven buildName, Appium settings, optional video download via `DISABLE_VIDEO_DOWNLOAD`; apply `updateSettings` on session start. > - Wire patch in `package.json`/`yarn.lock`. > - **Performance tests (Appwright)**: > - Update login/onboarding/send/swap/asset view flows to new UI; tweak timers; change tokens (e.g., `USDC`, `LINK`). > - Add `dissmissPredictionsModal` and use across scenarios; skip perps onboarding test. > - Simplify fixture cleanup; always attach metrics; remove orphan timer recovery/fallback. > - **Utilities**: > - New `appwright/utils/Utils.splitAmountIntoDigits`; refactor amount entry in `AmountScreen`, `SwapScreen`, `BridgeScreen`, `PerpsDepositScreen`. > - **Screen objects**: > - Numerous selector/tap fixes (IDs/text/XPath), iOS exact ID matching, improved visibility timeouts, scroll-into-view, and modal buttons (e.g., Rewards, Login, Networks, Wallet actions). > - **Config and tooling**: > - Appwright project configs unchanged functionally (comment cleanup); extend `.depcheckrc.yml` ignores for Appwright/Appium; minor test config/timeouts. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit e34ae38. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: metamaskbot <metamaskbot@users.noreply.github.com> Co-authored-by: sethkfman <10342624+sethkfman@users.noreply.github.com>
1 parent eeb676b commit fa5f8bc

42 files changed

Lines changed: 370 additions & 507 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.depcheckrc.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# List things here that *are - 'used, that depcheck is wrong about'
2+
23
ignores:
34
- '@metamask/oss-attribution-generator'
45
- '@metamask/test-dapp-multichain'

.yarn/patches/appwright-npm-0.1.45-f282bc1c1b.patch

Lines changed: 109 additions & 136 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,107 @@ index a51913b3f9380fd5740b1d52cd1d3472e29233c3..6479c9b92ed17319f654e9323f689792
1313
reporter: [["list"], ["html", { open: "always" }]],
1414
use: {
1515
// TODO: Use this for actions
16+
diff --git a/dist/device/index.js b/dist/device/index.js
17+
index fda54b682837d06eb579596087826c862c35f33e..b61aceca45b8b3293dee14552c26f002b38a4e36 100644
18+
--- a/dist/device/index.js
19+
+++ b/dist/device/index.js
20+
@@ -237,7 +237,7 @@ let Device = (() => {
21+
findStrategy: isAndroid
22+
? "-android uiautomator"
23+
: "-ios predicate string",
24+
- textToMatch: text,
25+
+ textToMatch: null,
26+
});
27+
}
28+
/**
29+
diff --git a/dist/locator/index.js b/dist/locator/index.js
30+
index 03f00574615d647d1887440ee7ae10798934c250..40089ce61ad598625afeb42d1149da8ce007dc26 100644
31+
--- a/dist/locator/index.js
32+
+++ b/dist/locator/index.js
33+
@@ -244,51 +244,33 @@ let Locator = (() => {
34+
]);
35+
}
36+
}
37+
- /**
38+
- * Retrieves the element reference based on the `selector`.
39+
- *
40+
- * @returns
41+
- */
42+
+
43+
async getElement() {
44+
- /**
45+
- * Determine whether `path` is a regex or string, and find elements accordingly.
46+
- *
47+
- * If `path` is a regex:
48+
- * - Iterate through all the elements on the page
49+
- * - Extract text content of each element
50+
- * - Return the first matching element
51+
- *
52+
- * If `path` is a string:
53+
- * - Use `findStrategy` (either XPath, Android UIAutomator, or iOS predicate string) to find elements
54+
- * - Apply regex to clean extra characters from the matched element’s text
55+
- * - Return the first element that matches
56+
- */
57+
- let elements = await this.webDriverClient.findElements(this.findStrategy, this.selector);
58+
- // If there is only one element, return it
59+
- if (elements.length === 1) {
60+
- return elements[0];
61+
- }
62+
- // If there are multiple elements, we reverse the order since the probability
63+
- // of finding the element is higher at higher depth
64+
+ const elements = await this.webDriverClient.findElements(this.findStrategy, this.selector);
65+
const reversedElements = elements.reverse();
66+
+ console.log('Reversed elements length', reversedElements.length);
67+
+ // If no text matching is needed, just return the first (deepest) element
68+
+ if (!this.textToMatch) {
69+
+ console.log('No text to match, returning first element');
70+
+ return reversedElements[0] || null;
71+
+ }
72+
+
73+
+ // Only iterate and fetch text if we need to match text
74+
for (const element of reversedElements) {
75+
let elementText = await this.webDriverClient.getElementText(element["element-6066-11e4-a52e-4f735466cecf"]);
76+
- if (this.textToMatch) {
77+
- if (this.textToMatch instanceof RegExp &&
78+
- this.textToMatch.test(elementText)) {
79+
- return element;
80+
- }
81+
- if (typeof this.textToMatch === "string" &&
82+
- elementText.includes(this.textToMatch)) {
83+
- return element;
84+
- }
85+
+ console.log('Element text', elementText);
86+
+ console.log('Text to match', this.textToMatch);
87+
+
88+
+ if (this.textToMatch instanceof RegExp &&
89+
+ this.textToMatch.test(elementText)) {
90+
+ return element;
91+
}
92+
- else {
93+
- // This is returned for cases where xpath is findStrategy and we want
94+
- // to return the last element found in the list
95+
+ if (typeof this.textToMatch === "string" &&
96+
+ elementText.includes(this.textToMatch)) {
97+
return element;
98+
}
99+
}
100+
+
101+
return null;
102+
}
103+
};
16104
diff --git a/dist/providers/browserstack/index.js b/dist/providers/browserstack/index.js
17-
index 638c5df686413ffe050ef7e7329b9e9446e98002..61ecd1508400f59302d6139a805e7a912e9a1036 100644
105+
index 638c5df686413ffe050ef7e7329b9e9446e98002..34e211940e22cc195002e6d88a98d7b8488d88d0 100644
18106
--- a/dist/providers/browserstack/index.js
19107
+++ b/dist/providers/browserstack/index.js
20-
@@ -120,6 +120,12 @@ class BrowserStackDeviceProvider {
108+
@@ -104,6 +104,7 @@ class BrowserStackDeviceProvider {
109+
async createDriver(config) {
110+
const WebDriver = (await import("webdriver")).default;
111+
const webDriverClient = await WebDriver.newSession(config);
112+
+
113+
this.sessionId = webDriverClient.sessionId;
114+
const bundleId = await this.getAppBundleIdFromSession();
115+
const testOptions = {
116+
@@ -120,6 +121,12 @@ class BrowserStackDeviceProvider {
21117
return this.sessionDetails?.app_details.app_name ?? "";
22118
}
23119
static async downloadVideo(sessionId, outputDir, fileName) {
@@ -30,7 +126,7 @@ index 638c5df686413ffe050ef7e7329b9e9446e98002..61ecd1508400f59302d6139a805e7a91
30126
const sessionData = await getSessionDetails(sessionId);
31127
const sessionDetails = sessionData?.automation_session;
32128
const videoURL = sessionDetails?.video_url;
33-
@@ -241,7 +247,10 @@ class BrowserStackDeviceProvider {
129+
@@ -241,7 +248,10 @@ class BrowserStackDeviceProvider {
34130
capabilities: {
35131
"bstack:options": {
36132
debug: true,
@@ -41,7 +137,7 @@ index 638c5df686413ffe050ef7e7329b9e9446e98002..61ecd1508400f59302d6139a805e7a91
41137
networkLogs: true,
42138
appiumVersion: "2.6.0",
43139
enableCameraImageInjection: this.project.use.device?.enableCameraImageInjection,
44-
@@ -250,10 +259,10 @@ class BrowserStackDeviceProvider {
140+
@@ -250,10 +260,10 @@ class BrowserStackDeviceProvider {
45141
osVersion: this.project.use.device.osVersion,
46142
platformName: platformName,
47143
deviceOrientation: this.project.use.device?.orientation,
@@ -54,144 +150,21 @@ index 638c5df686413ffe050ef7e7329b9e9446e98002..61ecd1508400f59302d6139a805e7a91
54150
: process.env.USER,
55151
},
56152
"appium:autoGrantPermissions": true,
57-
@@ -261,6 +270,11 @@ class BrowserStackDeviceProvider {
153+
@@ -261,6 +271,17 @@ class BrowserStackDeviceProvider {
58154
"appium:autoAcceptAlerts": true,
59155
"appium:fullReset": true,
60156
"appium:settings[snapshotMaxDepth]": 62,
61157
+ "appium:settings[autoGrantPermissions]": true,
62-
+ "appium:settings[waitForIdleTimeout]": 10000,
158+
+ "appium:settings[waitForIdleTimeout]": 2000,
63159
+ "appium:settings[actionAcknowledgmentTimeout]": 3000,
64160
+ "appium:settings[ignoreUnimportantViews]": true,
65-
+ "appium:settings[waitForSelectorTimeout]": 10000,
161+
+ "appium:settings[waitForSelectorTimeout]": 1000,
162+
+ "appium:bstackPageSource": {
163+
+ "enable": true,
164+
+ "samplesX": 15,
165+
+ "samplesY": 15,
166+
+ "maxDepth": 75
167+
+ }
66168
},
67169
};
68170
}
69-
diff --git a/dist/reporter.js b/dist/reporter.js
70-
index 516da403677d49af5fbb18edaf9f308dad401e55..ae1895be0513e2aa3ada45ad93a3f3c489e42911 100644
71-
--- a/dist/reporter.js
72-
+++ b/dist/reporter.js
73-
@@ -6,8 +6,8 @@ Object.defineProperty(exports, "__esModule", { value: true });
74-
const providers_1 = require("./providers");
75-
const fs_1 = __importDefault(require("fs"));
76-
const path_1 = __importDefault(require("path"));
77-
-const fluent_ffmpeg_1 = __importDefault(require("fluent-ffmpeg"));
78-
-const ffmpeg_1 = __importDefault(require("@ffmpeg-installer/ffmpeg"));
79-
+// const fluent_ffmpeg_1 = __importDefault(require("fluent-ffmpeg"));
80-
+// const ffmpeg_1 = __importDefault(require("@ffmpeg-installer/ffmpeg"));
81-
const logger_1 = require("./logger");
82-
const utils_1 = require("./utils");
83-
const workerInfo_1 = require("./fixture/workerInfo");
84-
@@ -183,25 +183,32 @@ function trimVideo({ originalVideoPath, startSecs, durationSecs, outputPath, })
85-
fs_1.default.copyFileSync(originalVideoPath, copyFullPath);
86-
return new Promise((resolve, reject) => {
87-
let stdErrs = "";
88-
- (0, fluent_ffmpeg_1.default)(copyFullPath)
89-
- .setFfmpegPath(ffmpeg_1.default.path)
90-
- .setStartTime(startSecs)
91-
- .setDuration(durationSecs)
92-
- .output(fullOutputPath)
93-
- .on("end", () => {
94-
- logger_1.logger.log(`Trimmed video saved at: ${fullOutputPath}`);
95-
- fs_1.default.unlinkSync(copyFullPath);
96-
- resolve(fullOutputPath);
97-
- })
98-
- .on("stderr", (stderrLine) => {
99-
- stdErrs += stderrLine + "\n";
100-
- })
101-
- .on("error", (err) => {
102-
- logger_1.logger.error("ffmpeg error:", err);
103-
- logger_1.logger.error("ffmpeg stderr:", stdErrs);
104-
- reject(err);
105-
- })
106-
- .run();
107-
+ // Video trimming disabled - ffmpeg removed
108-
+ // (0, fluent_ffmpeg_1.default)(copyFullPath)
109-
+ // .setFfmpegPath(ffmpeg_1.default.path)
110-
+ // .setStartTime(startSecs)
111-
+ // .setDuration(durationSecs)
112-
+ // .output(fullOutputPath)
113-
+ // .on("end", () => {
114-
+ // logger_1.logger.log(`Trimmed video saved at: ${fullOutputPath}`);
115-
+ // fs_1.default.unlinkSync(copyFullPath);
116-
+ // resolve(fullOutputPath);
117-
+ // })
118-
+ // .on("stderr", (stderrLine) => {
119-
+ // stdErrs += stderrLine + "\n";
120-
+ // })
121-
+ // .on("error", (err) => {
122-
+ // logger_1.logger.error("ffmpeg error:", err);
123-
+ // logger_1.logger.error("ffmpeg stderr:", stdErrs);
124-
+ // reject(err);
125-
+ // })
126-
+ // .run();
127-
+
128-
+ // Just copy the original video without trimming
129-
+ fs_1.default.copyFileSync(copyFullPath, fullOutputPath);
130-
+ fs_1.default.unlinkSync(copyFullPath);
131-
+ logger_1.logger.log(`Video copied without trimming: ${fullOutputPath}`);
132-
+ resolve(fullOutputPath);
133-
});
134-
}
135-
async function getWorkerStartTime(idx) {
136-
diff --git a/dist/vision/index.js b/dist/vision/index.js
137-
index bf485308fe5ddde81b396bf87dbb365cc37efa0e..8e5f191e6b210c5839257bd6a1879b1f9ce45228 100644
138-
--- a/dist/vision/index.js
139-
+++ b/dist/vision/index.js
140-
@@ -38,8 +38,8 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
141-
};
142-
Object.defineProperty(exports, "__esModule", { value: true });
143-
exports.VisionProvider = void 0;
144-
-const vision_1 = require("@empiricalrun/llm/vision");
145-
-const point_1 = require("@empiricalrun/llm/vision/point");
146-
+// const vision_1 = require("@empiricalrun/llm/vision");
147-
+// const point_1 = require("@empiricalrun/llm/vision/point");
148-
const fs_1 = __importDefault(require("fs"));
149-
const test_1 = __importDefault(require("@playwright/test"));
150-
const utils_1 = require("../utils");
151-
@@ -64,42 +64,12 @@ let VisionProvider = (() => {
152-
this.webDriverClient = webDriverClient;
153-
}
154-
async query(prompt, options) {
155-
- test_1.default.skip(!process.env.OPENAI_API_KEY, "LLM vision based extract text is not enabled. Set the OPENAI_API_KEY environment variable to enable it");
156-
- let base64Screenshot = options?.screenshot;
157-
- if (!base64Screenshot) {
158-
- base64Screenshot = await this.webDriverClient.takeScreenshot();
159-
- }
160-
- return await (0, vision_1.query)(base64Screenshot, prompt, options);
161-
+ test_1.default.skip(true, "LLM vision based extract text is disabled - @empiricalrun/llm removed");
162-
+ throw new Error("LLM vision functionality has been disabled");
163-
}
164-
async tap(prompt, options) {
165-
- test_1.default.skip(!process.env.EMPIRICAL_API_KEY, "LLM vision based tap is not enabled. Set the EMPIRICAL_API_KEY environment variable to enable it");
166-
- const base64Image = await this.webDriverClient.takeScreenshot();
167-
- const coordinates = await (0, point_1.getCoordinatesFor)(prompt, base64Image, options);
168-
- if (coordinates.annotatedImage) {
169-
- const random = Math.floor(1000 + Math.random() * 9000);
170-
- const file = test_1.default.info().outputPath(`${random}.png`);
171-
- await fs_1.default.promises.writeFile(file, Buffer.from(coordinates.annotatedImage, "base64"));
172-
- await test_1.default.info().attach(`${random}`, { path: file });
173-
- }
174-
- const driverSize = await this.webDriverClient.getWindowRect();
175-
- const { container: imageSize, x, y } = coordinates;
176-
- const scaleFactorWidth = imageSize.width / driverSize.width;
177-
- const scaleFactorHeight = imageSize.height / driverSize.height;
178-
- if (scaleFactorWidth !== scaleFactorHeight) {
179-
- logger_1.logger.warn(`Scale factors are different: ${scaleFactorWidth} vs ${scaleFactorHeight}`);
180-
- }
181-
- const tapTargetX = x / scaleFactorWidth;
182-
- // This uses the width scale factor because getWindowRect on LambdaTest returns a smaller
183-
- // height value than the screenshot height, which causes disproportionate scaling
184-
- // for width and height.
185-
- // For example, Pixel 8 screenshot is 1080 (w) x 2400 (h), but LambdaTest returns
186-
- // 1080 (w) x 2142 (h) for getWindowRect.
187-
- const tapTargetY = y / scaleFactorWidth;
188-
- await this.device.tap({
189-
- x: tapTargetX,
190-
- y: tapTargetY,
191-
- });
192-
- return { x: tapTargetX, y: tapTargetY };
193-
+ test_1.default.skip(true, "LLM vision based tap is disabled - @empiricalrun/llm removed");
194-
+ throw new Error("LLM vision functionality has been disabled");
195-
}
196-
};
197-
})();
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
diff --git a/dist/providers/browserstack/index.js b/dist/providers/browserstack/index.js
2+
index 61ecd1508400f59302d6139a805e7a912e9a1036..c7772bec15413478dd0d446f7c9f805e16b00851 100644
3+
--- a/dist/providers/browserstack/index.js
4+
+++ b/dist/providers/browserstack/index.js
5+
@@ -104,6 +104,21 @@ class BrowserStackDeviceProvider {
6+
async createDriver(config) {
7+
const WebDriver = (await import("webdriver")).default;
8+
const webDriverClient = await WebDriver.newSession(config);
9+
+ await webDriverClient.updateSettings({
10+
+ waitForIdleTimeout: 100,
11+
+ waitForSelectorTimeout: 0,
12+
+ shouldWaitForQuiescence: false,
13+
+ snapshotMaxDepth: 62,
14+
+ snapshotTimeout: 50000,
15+
+ "appium:settings[snapshotMaxDepth]": 62,
16+
+ "appium:settings[customSnapshotTimeout]": 50000,
17+
+ "appium:bstackPageSource": {
18+
+ "enable": true,
19+
+ "samplesX": 15,
20+
+ "samplesY": 15,
21+
+ "maxDepth": 100
22+
+ }
23+
+ });
24+
this.sessionId = webDriverClient.sessionId;
25+
const bundleId = await this.getAppBundleIdFromSession();
26+
const testOptions = {

appwright/appwright.config.ts

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ export default defineConfig({
2929
osVersion: '14', // this can be changed to your emulator version
3030
},
3131
buildPath: 'PATH-TO-BUILD', // Path to your .apk file
32-
expectTimeout: 30 * 1000, //90 seconds increased since login the app takes longer
32+
expectTimeout: 30 * 1000,
3333
},
3434
},
3535
{
@@ -42,7 +42,7 @@ export default defineConfig({
4242
osVersion: '16.0', // this can be changed to your simulator version
4343
},
4444
buildPath: 'PATH-TO-BUILD', // Path to your .app file
45-
expectTimeout: 30 * 1000, //90 seconds increased since login the app takes longer
45+
expectTimeout: 30 * 1000,
4646
},
4747
},
4848
{
@@ -56,7 +56,7 @@ export default defineConfig({
5656
osVersion: process.env.BROWSERSTACK_OS_VERSION || '13.0', // this can changed
5757
},
5858
buildPath: process.env.BROWSERSTACK_ANDROID_APP_URL, // Path to Browserstack url
59-
expectTimeout: 30 * 1000, //90 seconds increased since login the app takes longer
59+
expectTimeout: 30 * 1000,
6060
},
6161
},
6262
{
@@ -70,7 +70,7 @@ export default defineConfig({
7070
osVersion: process.env.BROWSERSTACK_OS_VERSION || '16.0',
7171
},
7272
buildPath: process.env.BROWSERSTACK_IOS_APP_URL,
73-
expectTimeout: 30 * 1000, //90 seconds increased since login the app takes longer
73+
expectTimeout: 30 * 1000,
7474
},
7575
},
7676
{
@@ -84,7 +84,7 @@ export default defineConfig({
8484
osVersion: process.env.BROWSERSTACK_OS_VERSION || '13.0',
8585
},
8686
buildPath: process.env.BROWSERSTACK_ANDROID_CLEAN_APP_URL,
87-
expectTimeout: 30 * 1000, //90 seconds increased since login the app takes longer
87+
expectTimeout: 30 * 1000,
8888
},
8989
},
9090
{
@@ -98,7 +98,7 @@ export default defineConfig({
9898
osVersion: process.env.BROWSERSTACK_OS_VERSION || '16.0',
9999
},
100100
buildPath: process.env.BROWSERSTACK_IOS_CLEAN_APP_URL,
101-
expectTimeout: 30 * 1000, //90 seconds increased since login the app takes longer
101+
expectTimeout: 30 * 1000,
102102
},
103103
},
104104
],

0 commit comments

Comments
 (0)