[PM-35196] add rerendering benchmark

[audreyality]

🎟️ Tracking

https://bitwarden.atlassian.net/browse/PM-35196

📔 Objective

Introduce BIT benchmarks. The first benchmark targets the rerendering input grid scenario.

Benchmarks are implemented as a separate vertical from the existing BIT tests. There's some in-depth documentation explaining detailed design decisions and usage.

[github-actions]

🤖 Bitwarden Claude Code Review

Overall Assessment: APPROVE

This PR introduces a Playwright-based benchmark harness as a separate vertical from the existing test suite, adding a new fixture (benchmarks/fixtures.benchmark.ts), a custom CSV reporter (perf-summary-reporter.ts), per-run JSON output, and a dedicated playwright.benchmark.config.ts. It refactors shared extension setup logic out of tests/fixtures.browser.ts into a new fixtures/extension-setup.ts module that is consumed by both the existing test fixture and the new benchmark fixture. The harness is well-isolated (testIgnore, separate tsconfig, separate npm script, BW_INCLUDE_CONTENT_SCRIPT_MEASUREMENTS build-time gate) and the design tradeoffs around capture timing, poisoning detection, and route-handler overhead are explicitly documented in docs/performance.design.md.

Code Review Details

No findings to report.

The shared extension-setup.ts refactor preserves behavior of the existing tests: the screenshot in tests/fixtures.browser.ts is retained between prepareEnvironment and submitEnvironment, login flow is functionally identical, and await use(...) is now used consistently in manifestVersion/background (a small correctness improvement over the prior unawaited use(...)). The benchmark fixture's perfCapture route handler correctly filters non-main-frame and non-navigation requests, awaits extractMeasures before route.continue() so the prior page's Performance Timeline is read before unload, and the seenUrls set plus the chrome-extension:///about:blank filter prevent spurious captures during the login flow. The pathToExtension calculation matches the established tests/fixtures.browser.ts pattern (compiled output in benchmarks-out/ is sibling to tests-out/, so ../../ resolves identically). The assertInstrumentationEnabled early-failure path gives operators an actionable error pointing at build:extension:bench instead of silent zero-count data. The previously raised NUL-byte issue in perf-summary-reporter.ts was resolved in commit a2ffbf8. Open conversation threads (shared base config, README link to benchmarks, command naming) are tracked for follow-up rather than being blockers; the contract drift between :autofill:bw/perf:enabled:autofill:bw and the clients-side performance.ts constants is already called out in benchmarks/utils.ts:10-16.

[bensbits91]

Oops, wrong PR! Sorry. Disregard.

One non-blocking comment/question/nit: Should we update the JSDoc comment at performance.ts#L191 to add the new namespace (${name}:poison > ${name}:poison:autofill:bw?

I added my review and comment to the wrong PR. So silly, so sorry.

[jprusik]

A few nits and observations, but nothing blocking

[jprusik]

The static in the other node commands is to distinguish between the "live" site tests (which we do not run in CI, and do not want to run accidentally locally). We could probably drop it (unless you feel there's value in running benchmarks against the live website hosting a copy of the target page).

It probably would be helpful to encode what we're benchmarking in the command however, since we will likely introduce other benchmark vectors in the future.

[audreyality]

I'm following the convention that each suite of tests gets its own folder. test:static runs the tests in the tests/static folder. benchmark:static runs the benchmarks in the benchmarks/static folder. I could rename it test-site or something similar, so it matches the folder that contains the test-the-web code.

When it runs benchmarks, it's running the whole suite sequentially, at least for now. You get all of the output from all benchmarks that ran in test-summary/benchmark-summary.csv, labeled by suite and the page the measure was captured on.

[jprusik]

We should probably capture (not now, but in future iterations) the shared config concerns in a base file and import them into the (now divergent) benchmark and test configs.

[audreyality]

I think that largely comes down to sharing use, projects and webServer.

The benchmark config eliminates a lot of the nice things in the test config because they could interfere with timing.

[jprusik]

Once we've had some time with it, we should update the project README with some details about what this is (similar to the a11y tests) and link to the README in the benchmarks folder.

[audreyality]

Yeah, I agree. I didn't add it into the root README because it's already quite long, and I didn't want to make it longer. 😅

Over time, I'm hoping that we can split it out a bit, providing instructions for a routine setup in the root README, and then using the docs/ folder for more complex setups, troubleshooting, and so on. This PR adds a new README in benchmarks and stuff into docs/, which I need to clean up. I think the benchmark readme is probably better as a Claude skill; that takes much more time to get right, though.

I'll circle back around once we've got a fix in place for our immediate performance bottleneck.