Skip to content

👷 run latest firefox and webkit in CI, default local e2e to chromium#4618

Open
thomas-lebeau wants to merge 1 commit into
mainfrom
thomas.lebeau/e2e-latest-browsers-in-ci
Open

👷 run latest firefox and webkit in CI, default local e2e to chromium#4618
thomas-lebeau wants to merge 1 commit into
mainfrom
thomas.lebeau/e2e-latest-browsers-in-ci

Conversation

@thomas-lebeau
Copy link
Copy Markdown
Collaborator

Motivation

We only ran the latest Chromium in CI e2e, alongside pinned variants of Chromium 120, Firefox 119 and WebKit 17.4 (added in #4612). This left a gap: regressions on the latest Firefox/WebKit versions weren't caught until they shipped. Locally, running all browsers also made the e2e feedback loop slow.

Changes

  • .gitlab-ci.yml: add firefox and webkit to the e2e job's BROWSER matrix (in addition to the existing chromium/*-pinned projects). Bump CURRENT_CI_IMAGE to 108.
  • Dockerfile: install Playwright's firefox and webkit browsers (with deps) in the CI image so the new matrix entries have what they need.
  • package.json: default yarn test:e2e to --project=chromium to keep the local feedback loop fast. Other browsers remain available via yarn test:e2e --project=firefox (etc.).

Test instructions

  • CI: confirm the e2e job runs the new firefox and webkit matrix entries successfully on the freshly built image 108.
  • Local: yarn test:e2e should now run chromium only. yarn test:e2e --project=firefox still works for opting in to other browsers.

Checklist

  • Tested locally
  • Tested on staging
  • Added unit tests for this change.
  • Added e2e/integration tests for this change.
  • Updated documentation and/or relevant AGENTS.md file

@thomas-lebeau thomas-lebeau requested a review from a team as a code owner May 13, 2026 13:42
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 544f22ee07

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread package.json
Adds firefox and webkit to the CI e2e matrix alongside chromium so we
catch cross-browser regressions on the latest browser versions (in
addition to the existing pinned variants). Installs the matching
Playwright browsers in the CI image and bumps CURRENT_CI_IMAGE to 108.

Defaults local `yarn test:e2e` to chromium only to keep the local
feedback loop fast; other browsers can still be run via `--project=`.
@thomas-lebeau thomas-lebeau force-pushed the thomas.lebeau/e2e-latest-browsers-in-ci branch from 544f22e to 4ea9e76 Compare May 13, 2026 13:48
@cit-pr-commenter-54b7da
Copy link
Copy Markdown

Bundles Sizes Evolution

📦 Bundle Name Base Size Local Size 𝚫 𝚫% Status
Rum 169.51 KiB 169.51 KiB 0 B 0.00%
Rum Profiler 5.97 KiB 5.97 KiB 0 B 0.00%
Rum Recorder 21.23 KiB 21.23 KiB 0 B 0.00%
Logs 54.70 KiB 54.70 KiB 0 B 0.00%
Rum Slim 127.85 KiB 127.85 KiB 0 B 0.00%
Worker 22.99 KiB 22.99 KiB 0 B 0.00%
🚀 CPU Performance

Pending...

🧠 Memory Performance

Pending...

🔗 RealWorld

@datadog-prod-us1-6
Copy link
Copy Markdown

datadog-prod-us1-6 Bot commented May 13, 2026

Tests

Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 8 Tests failed

Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › async from sessionStore.scenario.ts   View in Datadog   (Fix with Cursor)
sessionStore.scenario.ts:50:17 async

[webkit] › sessionStore.scenario.ts:50:17 › Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › async 

    Error: expect(received).toEqual(expected) // deep equality

    Expected: ObjectContaining {"domain": "foo.bar.localhost"}
    Received: undefined

      54 |           const cookies = await browserContext.cookies()
...
Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › bundle from sessionStore.scenario.ts   View in Datadog   (Fix with Cursor)
sessionStore.scenario.ts:50:17 bundle

[webkit] › sessionStore.scenario.ts:50:17 › Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › bundle 

    Error: expect(received).toEqual(expected) // deep equality

    Expected: ObjectContaining {"domain": "foo.bar.localhost"}
    Received: undefined

      54 |           const cookies = await browserContext.cookies()
...
Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › npm from sessionStore.scenario.ts   View in Datadog   (Fix with Cursor)
sessionStore.scenario.ts:50:17 npm

[webkit] › sessionStore.scenario.ts:50:17 › Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › npm 

    Error: expect(received).toEqual(expected) // deep equality

    Expected: ObjectContaining {"domain": "foo.bar.localhost"}
    Received: undefined

      54 |           const cookies = await browserContext.cookies()
...
View all

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 76.96% (+0.00%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4ea9e76 | Docs | Datadog PR Page | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants