Skip to content

test(e2e): migrate runner to rstest#7776

Draft
chenjiahan wants to merge 1 commit into
mainfrom
chenjiahan/feat-e2e-rstest-runner
Draft

test(e2e): migrate runner to rstest#7776
chenjiahan wants to merge 1 commit into
mainfrom
chenjiahan/feat-e2e-rstest-runner

Conversation

@chenjiahan
Copy link
Copy Markdown
Member

Summary

  • Migrates the E2E test runner from @playwright/test to @rstest/core with a standalone e2e/rstest.config.ts, while keeping Playwright for browser automation.
  • Updates E2E helpers to provide Playwright-like browser, context, page, request, and locator assertion compatibility so existing test cases can stay mostly unchanged.
  • Restores Windows skip behavior for the affected E2E cases and applies targeted fixes needed for Rstest execution.

Validated with pnpm exec rslint --type-check e2e and full Rstest E2E coverage locally.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request migrates the E2E testing framework from Playwright to Rstest, replacing @playwright/test with @rstest/core and implementing custom E2E test fixtures, assertions, and browser management. The review feedback highlights a correctness bug in getDeepTextContent when traversing shadow hosts and slot elements, a potential unhandled error in waitForExpectation if the timeout is too small, and redundant runtime Windows platform checks inside several test bodies since the skip logic has been moved to definition-time.

Comment thread e2e/helper/fixture.ts
Comment on lines +305 to +326
const getLocatorTextContents = (locator: Locator) =>
locator.evaluateAll((elements) => {
const getDeepTextContent = (node: Node): string => {
let text = '';

if (node.nodeType === Node.TEXT_NODE) {
return node.textContent ?? '';
}

if (node instanceof Element && node.shadowRoot) {
text += getDeepTextContent(node.shadowRoot);
}

for (const child of node.childNodes) {
text += getDeepTextContent(child);
}

return text;
};

return elements.map((element) => getDeepTextContent(element));
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The custom getDeepTextContent implementation has a correctness bug when traversing shadow hosts. If a shadow host contains a <slot> with fallback content, and light DOM children are assigned to it, the current implementation will traverse both the shadow root (including the fallback content inside the <slot>) and the light DOM children (node.childNodes). This results in both the fallback content (which is hidden/not rendered) and the slotted content being concatenated together.\n\nAdditionally, any light DOM children that are not assigned to any slot will still be traversed and included in the text.\n\nTo fix this, we should only traverse the shadow root if it exists, and when encountering an HTMLSlotElement, we should traverse its assigned nodes (slot.assignedNodes()) instead of its fallback children (unless assignedNodes() is empty).

const getLocatorTextContents = (locator: Locator) =>
  locator.evaluateAll((elements) => {
    const getDeepTextContent = (node: Node): string => {
      if (node.nodeType === Node.TEXT_NODE) {
        return node.textContent ?? '';
      }

      if (node instanceof HTMLSlotElement) {
        const assigned = node.assignedNodes();
        if (assigned.length > 0) {
          return assigned.map(getDeepTextContent).join('');
        }
      }

      if (node instanceof Element && node.shadowRoot) {
        return getDeepTextContent(node.shadowRoot);
      }

      let text = '';
      for (const child of node.childNodes) {
        text += getDeepTextContent(child);
      }
      return text;
    };

    return elements.map((element) => getDeepTextContent(element));
  });

Comment thread e2e/helper/fixture.ts
Comment on lines +343 to +365
const waitForExpectation = async (
check: () => Promise<void>,
options?: MatcherOptions,
) => {
const timeout = options?.timeout ?? DEFAULT_EXPECT_TIMEOUT;
const start = Date.now();
let lastError: unknown;

while (Date.now() - start <= timeout) {
try {
await check();
return;
} catch (error) {
lastError = error;
await sleep(EXPECT_POLL_INTERVAL);
}
}

if (lastError instanceof Error) {
throw lastError;
}
throw new Error(String(lastError));
};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using a while loop for waitForExpectation can lead to issues if the timeout is 0 or very small, or if there is clock drift, potentially throwing an unhandled undefined error because lastError is never initialized. \n\nUsing a do-while loop guarantees that the assertion check runs at least once, which is the standard behavior for assertion polling. We should also initialize lastError to a default fallback error to prevent throwing undefined.

const waitForExpectation = async (
  check: () => Promise<void>,
  options?: MatcherOptions,
) => {
  const timeout = options?.timeout ?? DEFAULT_EXPECT_TIMEOUT;
  const start = Date.now();
  let lastError: unknown = new Error('Timeout');

  do {
    try {
      await check();
      return;
    } catch (error) {
      lastError = error;
      await sleep(EXPECT_POLL_INTERVAL);
    }
  } while (Date.now() - start <= timeout);

  if (lastError instanceof Error) {
    throw lastError;
  }
  throw new Error(String(lastError));
};


// https://github.com/web-infra-dev/rsbuild/issues/5176
// Failed to run this case on Windows
const test = process.platform === 'win32' ? baseTest.skip : baseTest;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since you have moved the Windows skip logic to definition-time via const test = process.platform === 'win32' ? baseTest.skip : baseTest;, the runtime check if (process.platform === 'win32') { test.skip(); } inside the test body is now redundant and unreachable. Consider removing the redundant runtime check from the test body.

import { expect, test as baseTest } from '@e2e/helper';

// Prefresh does not work as expected on Windows
const test = process.platform === 'win32' ? baseTest.skip : baseTest;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since you have moved the Windows skip logic to definition-time via const test = process.platform === 'win32' ? baseTest.skip : baseTest;, the runtime check if (process.platform === 'win32') { test.skip(); } inside the test body is now redundant and unreachable. Consider removing the redundant runtime check from the test body.

import { expect, test as baseTest } from '@e2e/helper';

// Prefresh does not work as expected on Windows
const test = process.platform === 'win32' ? baseTest.skip : baseTest;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since you have moved the Windows skip logic to definition-time via const test = process.platform === 'win32' ? baseTest.skip : baseTest;, the runtime check if (process.platform === 'win32') { test.skip(); } inside the test body is now redundant and unreachable. Consider removing the redundant runtime check from the test body.

import { expect, test as baseTest } from '@e2e/helper';

// Failed to run this case on Windows
const test = process.platform === 'win32' ? baseTest.skip : baseTest;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since you have moved the Windows skip logic to definition-time via const test = process.platform === 'win32' ? baseTest.skip : baseTest;, the runtime check if (process.platform === 'win32') { test.skip(); } inside the test body is now redundant and unreachable. Consider removing the redundant runtime check from the test body.


// https://github.com/web-infra-dev/rspack/issues/6633
// TODO: fixme on Windows
const test = process.platform === 'win32' ? baseTest.skip : baseTest;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since you have moved the Windows skip logic to definition-time via const test = process.platform === 'win32' ? baseTest.skip : baseTest;, the runtime check if (process.platform === 'win32') { test.skip(); } inside the test body is now redundant and unreachable. Consider removing the redundant runtime check from the test body.

@fi3ework fi3ework self-assigned this Jun 3, 2026
fi3ework added a commit to web-infra-dev/rstest that referenced this pull request Jun 3, 2026
…alse)

With `isolate: false`, all test files share one worker but each file owns its
own birpc channel. The host disposes a file's channel as soon as the file
finishes, while the worker only closes it when the next file starts. A console
reference captured by a file (e.g. a logger that flushes from a late
`setTimeout`/microtask) could fire in or after that window and forward the log
over a disposed/closed channel.

Because `onConsoleLog` was a normal birpc call, the late log registered a
pending request that `$close()` then rejected with `[birpc] rpc is closed`.
Since the console interceptor does not await it, the rejection surfaced as an
`unhandledRejection` that failed the run even though every assertion passed,
and was misattributed to whichever file happened to be running.

`onConsoleLog` is fire-and-forget telemetry whose result the worker never reads,
so model it as a one-way birpc event (`eventNames`): it no longer registers a
pending request, so there is nothing for `$close()` to reject. Forwarding is
also made best-effort — a send that races channel teardown is swallowed rather
than crashing the run, and a dropped late log stays subject to the host's
`onConsoleLog` policy, matching `isolate: true` where such logs are lost as the
worker is torn down.

Covered by an e2e regression test (with console suppression) across both the
forks and threads pools and both isolate modes, plus a unit test asserting
`onConsoleLog` is registered as a one-way event.

Originally surfaced while migrating Rsbuild's e2e runner to rstest
(web-infra-dev/rsbuild#7776), then reduced to the minimal repro in #1367.

Closes #1367
fi3ework added a commit to web-infra-dev/rstest that referenced this pull request Jun 3, 2026
…alse)

With `isolate: false`, all test files share one worker but each file owns its
own birpc channel. The host disposes a file's channel as soon as the file
finishes, while the worker only closes it when the next file starts. A console
reference captured by a file (e.g. a logger that flushes from a late
`setTimeout`/microtask) could fire in or after that window and forward the log
over a disposed/closed channel.

Because `onConsoleLog` was a normal birpc call, the late log registered a
pending request that `$close()` then rejected with `[birpc] rpc is closed`.
Since the console interceptor does not await it, the rejection surfaced as an
`unhandledRejection` that failed the run even though every assertion passed,
and was misattributed to whichever file happened to be running.

Console forwarding is best-effort, one-way telemetry, with two failure modes
handled at their own layer:

- Delivery failures (a disposed/closed channel, a teardown race) are an expected
  lifecycle artifact. `onConsoleLog` is now a one-way birpc event (`eventNames`),
  so it registers no pending request for `$close()` to reject, and the worker
  swallows any send error — the late log is dropped, matching `isolate: true`
  where such logs are lost as the worker is torn down.
- Handler failures (a user `onConsoleLog` filter or a reporter's
  `onUserConsoleLog` throwing) are real defects that cannot travel back over the
  one-way channel. The host now surfaces them where they occur instead of
  letting birpc silently drop the event handler's rejection, without failing the
  otherwise-passing run.

Covered by e2e regression tests (with console suppression) across the forks and
threads pools and both isolate modes, plus a unit test asserting `onConsoleLog`
is registered as a one-way event.

Originally surfaced while migrating Rsbuild's e2e runner to rstest
(web-infra-dev/rsbuild#7776), then reduced to the minimal repro in #1367.

Closes #1367
fi3ework added a commit to web-infra-dev/rstest that referenced this pull request Jun 4, 2026
…alse)

With `isolate: false`, all test files share one worker but each file owns its
own birpc channel. The host disposes a file's channel as soon as the file
finishes, while the worker only closes it when the next file starts. A console
reference captured by a file (e.g. a logger that flushes from a late
`setTimeout`/microtask) could fire in or after that window and forward the log
over a disposed/closed channel.

The console interceptor does not await the forward, so the rejected send
surfaced as an `unhandledRejection` (`[birpc] rpc is closed`) that failed the
run even though every assertion passed, and was misattributed to whichever file
happened to be running.

Console forwarding is best-effort, with two failure modes handled at their own
layer:

- Delivery failures (a disposed/closed channel, a teardown race) are an expected
  lifecycle artifact. The worker forwards the log fire-and-forget and swallows
  the rejected send, so the late log is dropped instead of crashing the run —
  matching `isolate: true` where such logs are lost as the worker is torn down.
- Handler failures (a user `onConsoleLog` filter or a reporter's
  `onUserConsoleLog` throwing) would otherwise be hidden by that swallow. The
  host now surfaces them where they occur, without failing the otherwise-passing
  run.

Covered by e2e regression tests (with console suppression) across the forks and
threads pools and both isolate modes, plus one for a throwing `onConsoleLog`
hook.

Originally surfaced while migrating Rsbuild's e2e runner to rstest
(web-infra-dev/rsbuild#7776), then reduced to the minimal repro in #1367.

Closes #1367
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants