test(e2e): migrate runner to rstest#7776
Conversation
There was a problem hiding this comment.
Code Review
This pull request migrates the E2E testing framework from Playwright to Rstest, replacing @playwright/test with @rstest/core and implementing custom E2E test fixtures, assertions, and browser management. The review feedback highlights a correctness bug in getDeepTextContent when traversing shadow hosts and slot elements, a potential unhandled error in waitForExpectation if the timeout is too small, and redundant runtime Windows platform checks inside several test bodies since the skip logic has been moved to definition-time.
| const getLocatorTextContents = (locator: Locator) => | ||
| locator.evaluateAll((elements) => { | ||
| const getDeepTextContent = (node: Node): string => { | ||
| let text = ''; | ||
|
|
||
| if (node.nodeType === Node.TEXT_NODE) { | ||
| return node.textContent ?? ''; | ||
| } | ||
|
|
||
| if (node instanceof Element && node.shadowRoot) { | ||
| text += getDeepTextContent(node.shadowRoot); | ||
| } | ||
|
|
||
| for (const child of node.childNodes) { | ||
| text += getDeepTextContent(child); | ||
| } | ||
|
|
||
| return text; | ||
| }; | ||
|
|
||
| return elements.map((element) => getDeepTextContent(element)); | ||
| }); |
There was a problem hiding this comment.
The custom getDeepTextContent implementation has a correctness bug when traversing shadow hosts. If a shadow host contains a <slot> with fallback content, and light DOM children are assigned to it, the current implementation will traverse both the shadow root (including the fallback content inside the <slot>) and the light DOM children (node.childNodes). This results in both the fallback content (which is hidden/not rendered) and the slotted content being concatenated together.\n\nAdditionally, any light DOM children that are not assigned to any slot will still be traversed and included in the text.\n\nTo fix this, we should only traverse the shadow root if it exists, and when encountering an HTMLSlotElement, we should traverse its assigned nodes (slot.assignedNodes()) instead of its fallback children (unless assignedNodes() is empty).
const getLocatorTextContents = (locator: Locator) =>
locator.evaluateAll((elements) => {
const getDeepTextContent = (node: Node): string => {
if (node.nodeType === Node.TEXT_NODE) {
return node.textContent ?? '';
}
if (node instanceof HTMLSlotElement) {
const assigned = node.assignedNodes();
if (assigned.length > 0) {
return assigned.map(getDeepTextContent).join('');
}
}
if (node instanceof Element && node.shadowRoot) {
return getDeepTextContent(node.shadowRoot);
}
let text = '';
for (const child of node.childNodes) {
text += getDeepTextContent(child);
}
return text;
};
return elements.map((element) => getDeepTextContent(element));
});| const waitForExpectation = async ( | ||
| check: () => Promise<void>, | ||
| options?: MatcherOptions, | ||
| ) => { | ||
| const timeout = options?.timeout ?? DEFAULT_EXPECT_TIMEOUT; | ||
| const start = Date.now(); | ||
| let lastError: unknown; | ||
|
|
||
| while (Date.now() - start <= timeout) { | ||
| try { | ||
| await check(); | ||
| return; | ||
| } catch (error) { | ||
| lastError = error; | ||
| await sleep(EXPECT_POLL_INTERVAL); | ||
| } | ||
| } | ||
|
|
||
| if (lastError instanceof Error) { | ||
| throw lastError; | ||
| } | ||
| throw new Error(String(lastError)); | ||
| }; |
There was a problem hiding this comment.
Using a while loop for waitForExpectation can lead to issues if the timeout is 0 or very small, or if there is clock drift, potentially throwing an unhandled undefined error because lastError is never initialized. \n\nUsing a do-while loop guarantees that the assertion check runs at least once, which is the standard behavior for assertion polling. We should also initialize lastError to a default fallback error to prevent throwing undefined.
const waitForExpectation = async (
check: () => Promise<void>,
options?: MatcherOptions,
) => {
const timeout = options?.timeout ?? DEFAULT_EXPECT_TIMEOUT;
const start = Date.now();
let lastError: unknown = new Error('Timeout');
do {
try {
await check();
return;
} catch (error) {
lastError = error;
await sleep(EXPECT_POLL_INTERVAL);
}
} while (Date.now() - start <= timeout);
if (lastError instanceof Error) {
throw lastError;
}
throw new Error(String(lastError));
};|
|
||
| // https://github.com/web-infra-dev/rsbuild/issues/5176 | ||
| // Failed to run this case on Windows | ||
| const test = process.platform === 'win32' ? baseTest.skip : baseTest; |
There was a problem hiding this comment.
Since you have moved the Windows skip logic to definition-time via const test = process.platform === 'win32' ? baseTest.skip : baseTest;, the runtime check if (process.platform === 'win32') { test.skip(); } inside the test body is now redundant and unreachable. Consider removing the redundant runtime check from the test body.
| import { expect, test as baseTest } from '@e2e/helper'; | ||
|
|
||
| // Prefresh does not work as expected on Windows | ||
| const test = process.platform === 'win32' ? baseTest.skip : baseTest; |
There was a problem hiding this comment.
Since you have moved the Windows skip logic to definition-time via const test = process.platform === 'win32' ? baseTest.skip : baseTest;, the runtime check if (process.platform === 'win32') { test.skip(); } inside the test body is now redundant and unreachable. Consider removing the redundant runtime check from the test body.
| import { expect, test as baseTest } from '@e2e/helper'; | ||
|
|
||
| // Prefresh does not work as expected on Windows | ||
| const test = process.platform === 'win32' ? baseTest.skip : baseTest; |
There was a problem hiding this comment.
Since you have moved the Windows skip logic to definition-time via const test = process.platform === 'win32' ? baseTest.skip : baseTest;, the runtime check if (process.platform === 'win32') { test.skip(); } inside the test body is now redundant and unreachable. Consider removing the redundant runtime check from the test body.
| import { expect, test as baseTest } from '@e2e/helper'; | ||
|
|
||
| // Failed to run this case on Windows | ||
| const test = process.platform === 'win32' ? baseTest.skip : baseTest; |
There was a problem hiding this comment.
Since you have moved the Windows skip logic to definition-time via const test = process.platform === 'win32' ? baseTest.skip : baseTest;, the runtime check if (process.platform === 'win32') { test.skip(); } inside the test body is now redundant and unreachable. Consider removing the redundant runtime check from the test body.
|
|
||
| // https://github.com/web-infra-dev/rspack/issues/6633 | ||
| // TODO: fixme on Windows | ||
| const test = process.platform === 'win32' ? baseTest.skip : baseTest; |
There was a problem hiding this comment.
Since you have moved the Windows skip logic to definition-time via const test = process.platform === 'win32' ? baseTest.skip : baseTest;, the runtime check if (process.platform === 'win32') { test.skip(); } inside the test body is now redundant and unreachable. Consider removing the redundant runtime check from the test body.
…alse) With `isolate: false`, all test files share one worker but each file owns its own birpc channel. The host disposes a file's channel as soon as the file finishes, while the worker only closes it when the next file starts. A console reference captured by a file (e.g. a logger that flushes from a late `setTimeout`/microtask) could fire in or after that window and forward the log over a disposed/closed channel. Because `onConsoleLog` was a normal birpc call, the late log registered a pending request that `$close()` then rejected with `[birpc] rpc is closed`. Since the console interceptor does not await it, the rejection surfaced as an `unhandledRejection` that failed the run even though every assertion passed, and was misattributed to whichever file happened to be running. `onConsoleLog` is fire-and-forget telemetry whose result the worker never reads, so model it as a one-way birpc event (`eventNames`): it no longer registers a pending request, so there is nothing for `$close()` to reject. Forwarding is also made best-effort — a send that races channel teardown is swallowed rather than crashing the run, and a dropped late log stays subject to the host's `onConsoleLog` policy, matching `isolate: true` where such logs are lost as the worker is torn down. Covered by an e2e regression test (with console suppression) across both the forks and threads pools and both isolate modes, plus a unit test asserting `onConsoleLog` is registered as a one-way event. Originally surfaced while migrating Rsbuild's e2e runner to rstest (web-infra-dev/rsbuild#7776), then reduced to the minimal repro in #1367. Closes #1367
…alse) With `isolate: false`, all test files share one worker but each file owns its own birpc channel. The host disposes a file's channel as soon as the file finishes, while the worker only closes it when the next file starts. A console reference captured by a file (e.g. a logger that flushes from a late `setTimeout`/microtask) could fire in or after that window and forward the log over a disposed/closed channel. Because `onConsoleLog` was a normal birpc call, the late log registered a pending request that `$close()` then rejected with `[birpc] rpc is closed`. Since the console interceptor does not await it, the rejection surfaced as an `unhandledRejection` that failed the run even though every assertion passed, and was misattributed to whichever file happened to be running. Console forwarding is best-effort, one-way telemetry, with two failure modes handled at their own layer: - Delivery failures (a disposed/closed channel, a teardown race) are an expected lifecycle artifact. `onConsoleLog` is now a one-way birpc event (`eventNames`), so it registers no pending request for `$close()` to reject, and the worker swallows any send error — the late log is dropped, matching `isolate: true` where such logs are lost as the worker is torn down. - Handler failures (a user `onConsoleLog` filter or a reporter's `onUserConsoleLog` throwing) are real defects that cannot travel back over the one-way channel. The host now surfaces them where they occur instead of letting birpc silently drop the event handler's rejection, without failing the otherwise-passing run. Covered by e2e regression tests (with console suppression) across the forks and threads pools and both isolate modes, plus a unit test asserting `onConsoleLog` is registered as a one-way event. Originally surfaced while migrating Rsbuild's e2e runner to rstest (web-infra-dev/rsbuild#7776), then reduced to the minimal repro in #1367. Closes #1367
…alse) With `isolate: false`, all test files share one worker but each file owns its own birpc channel. The host disposes a file's channel as soon as the file finishes, while the worker only closes it when the next file starts. A console reference captured by a file (e.g. a logger that flushes from a late `setTimeout`/microtask) could fire in or after that window and forward the log over a disposed/closed channel. The console interceptor does not await the forward, so the rejected send surfaced as an `unhandledRejection` (`[birpc] rpc is closed`) that failed the run even though every assertion passed, and was misattributed to whichever file happened to be running. Console forwarding is best-effort, with two failure modes handled at their own layer: - Delivery failures (a disposed/closed channel, a teardown race) are an expected lifecycle artifact. The worker forwards the log fire-and-forget and swallows the rejected send, so the late log is dropped instead of crashing the run — matching `isolate: true` where such logs are lost as the worker is torn down. - Handler failures (a user `onConsoleLog` filter or a reporter's `onUserConsoleLog` throwing) would otherwise be hidden by that swallow. The host now surfaces them where they occur, without failing the otherwise-passing run. Covered by e2e regression tests (with console suppression) across the forks and threads pools and both isolate modes, plus one for a throwing `onConsoleLog` hook. Originally surfaced while migrating Rsbuild's e2e runner to rstest (web-infra-dev/rsbuild#7776), then reduced to the minimal repro in #1367. Closes #1367
Summary
@playwright/testto@rstest/corewith a standalonee2e/rstest.config.ts, while keeping Playwright for browser automation.browser,context,page,request, and locator assertion compatibility so existing test cases can stay mostly unchanged.Validated with
pnpm exec rslint --type-check e2eand full Rstest E2E coverage locally.