Skip to content

Commit 5039b8e

Browse files
GuillaumeLagrangeclaude
GuillaumeLagrange
and
claude
committed
feat: add @codspeed/playwright package to bench electron apps
Introduce a @codspeed/playwright package exposing a Playwright-style `bench(name, fn, options)` API. The benchmark function receives a `{ page }` fixture mirroring Playwright's `test`, and a minimal options argument carrying the number of rounds and Electron launch config. Refs COD-2721 Co-Authored-By: Claude <noreply@anthropic.com>
1 parent ed6274a commit 5039b8e

6 files changed

Lines changed: 562 additions & 6 deletions

File tree

‎packages/playwright-plugin/README.md‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
<div align="center">
2+
<h1><code>@codspeed/playwright-plugin</code></h1>
3+
4+
[Playwright](https://playwright.dev) integration for [CodSpeed](https://codspeed.io), to drive an app through Playwright and report measured user flows.
5+
6+
[![CI](https://github.com/CodSpeedHQ/codspeed-node/actions/workflows/ci.yml/badge.svg)](https://github.com/CodSpeedHQ/codspeed-node/actions/workflows/ci.yml)
7+
[![npm (scoped)](https://img.shields.io/npm/v/@codspeed/playwright-plugin)](https://www.npmjs.com/package/@codspeed/playwright-plugin)
8+
[![Discord](https://img.shields.io/badge/chat%20on-discord-7289da.svg)](https://discord.com/invite/MxpaCfKSqF)
9+
[![CodSpeed Badge](https://img.shields.io/endpoint?url=https://codspeed.io/badge.json)](https://codspeed.io/CodSpeedHQ/codspeed-node)
10+
11+
</div>
12+
13+
> [!NOTE]
14+
> The `@codspeed/playwright-plugin` integration currently supports only the
15+
> [walltime instrument](https://docs.codspeed.io/instruments/walltime). CPU
16+
> Simulation is not available.
17+
18+
`@codspeed/playwright-plugin` is the CodSpeed integration for
19+
[Playwright](https://playwright.dev). It runs a user-defined flow against a
20+
target application, measures the time spent inside that flow, and reports it to
21+
CodSpeed. The flow itself is plain Playwright code, so anything Playwright can
22+
drive can be benchmarked.
23+
24+
> [!IMPORTANT]
25+
> Today the plugin supports [Electron](https://www.electronjs.org) apps as a
26+
> target. Browser-based targets (existing dev servers, static builds, hosted
27+
> URLs) are on the roadmap and will be added under the same `bench` API.
28+
29+
## Documentation
30+
31+
Check out the [documentation](https://docs.codspeed.io/benchmarks/nodejs/playwright) for complete integration instructions.
32+
33+
## Installation
34+
35+
Install the plugin alongside `playwright`:
36+
37+
```sh
38+
npm install --save-dev @codspeed/playwright-plugin playwright
39+
```
40+
41+
or with `yarn`:
42+
43+
```sh
44+
yarn add --dev @codspeed/playwright-plugin playwright
45+
```
46+
47+
or with `pnpm`:
48+
49+
```sh
50+
pnpm add --save-dev @codspeed/playwright-plugin playwright
51+
```
52+
53+
## Example usage with Electron
54+
55+
Build your Electron app first so the main entrypoint exists (e.g.,
56+
`out/main/index.js`), then declare a benchmark with `target.kind` set to
57+
`"electron"`:
58+
59+
```ts title="bench/inbox.bench.ts"
60+
import { bench } from "@codspeed/playwright-plugin";
61+
import path from "node:path";
62+
63+
bench(
64+
"inbox-search",
65+
async ({ page }) => {
66+
await page.fill("#search", "quarterly report");
67+
await page.waitForSelector("#results");
68+
},
69+
{
70+
target: {
71+
kind: "electron",
72+
appPath: path.resolve("out/main/index.js"),
73+
},
74+
beforeRound: async ({ page }) => {
75+
await page.waitForSelector("#main:not(.loading)");
76+
},
77+
rounds: 5,
78+
},
79+
);
80+
```
81+
82+
For each round, the plugin launches Electron with the provided main entrypoint,
83+
waits for the first window, runs `beforeRound`, measures `fn`, runs
84+
`afterRound`, then closes the app.
85+
86+
## API
87+
88+
The plugin exposes a single `bench` function. Its shape is target-agnostic:
89+
90+
```ts
91+
import { bench } from "@codspeed/playwright-plugin";
92+
93+
bench(name, fn, options);
94+
```
95+
96+
| Argument | Type | Required | Description |
97+
| --------- | ------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
98+
| `name` | `string` | yes | Identifier of the benchmark, used by CodSpeed to track it across runs. |
99+
| `fn` | `({ page }) => void \| Promise<void>` | yes | The function whose execution time is measured. Receives a Playwright [`Page`](https://playwright.dev/docs/api/class-page) bound to the target. Everything inside `fn` counts toward the reported timing. |
100+
| `options` | `BenchOptions` | yes | Target configuration and benchmark settings, detailed below. |
101+
102+
### Options
103+
104+
| Option | Type | Default | Description |
105+
| ------------- | ------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
106+
| `target` | `Target` | _(required)_ | Discriminated union describing what to drive. The `kind` field selects the target; the remaining fields are specific to that kind. Current variants: [`{ kind: "electron", ... }`](#electron-target-options). |
107+
| `rounds` | `number` | `1` | Number of measurement rounds. Can be overridden at runtime via the `CODSPEED_PLAYWRIGHT_ROUNDS` environment variable. |
108+
| `beforeRound` | `({ page }) => void \| Promise<void>` | — | Runs before each round, after the target is ready. Use it to bring the app to a ready state. Not measured. |
109+
| `afterRound` | `({ page }) => void \| Promise<void>` | — | Runs after each round, before the target is torn down. Not measured. |
110+
111+
### Electron target options
112+
113+
| Option | Type | Default | Description |
114+
| ------------------------------- | ------------ | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
115+
| `target.kind` | `"electron"` | _(required)_ | Selects the Electron target. |
116+
| `target.appPath` | `string` | _(required)_ | Absolute path to the Electron main entrypoint, e.g., `out/main/index.js`. |
117+
| `target.electronArgs` | `string[]` | `[]` | Extra CLI flags forwarded to the Electron process. |
118+
| `target.cwd` | `string` | `process.cwd()` | Working directory for the Electron process. Also the directory `electron` is resolved from when `target.electronExecutablePath` is not set. |
119+
| `target.electronExecutablePath` | `string` | resolved `electron` | Absolute path to the Electron binary. Only set this to override the default resolution. |
120+
121+
## Running the benchmarks locally
122+
123+
With node 24+, you can run typescript files directly:
124+
125+
```bash
126+
$ node bench/inbox.bench.ts
127+
[CodSpeed] [round 1/5] 42.13 ms
128+
[CodSpeed] [round 2/5] 41.78 ms
129+
[CodSpeed] [round 3/5] 42.05 ms
130+
[CodSpeed] [round 4/5] 41.92 ms
131+
[CodSpeed] [round 5/5] 42.21 ms
132+
```
133+
134+
Locally, `bench` runs the app and prints per-round timings to the terminal.
135+
Results are uploaded to CodSpeed only when running in the
136+
[CI environment](https://docs.codspeed.io/benchmarks/nodejs/playwright#running-the-benchmarks-in-your-ci)
137+
or when using the
138+
[CodSpeed CLI](https://docs.codspeed.io/cli#running-benchmarks).

‎packages/playwright-plugin/package.json‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
{
2+
"name": "@codspeed/playwright-plugin",
3+
"version": "5.4.0",
4+
"description": "Playwright benchmarking integration for CodSpeed",
5+
"keywords": [
6+
"codspeed",
7+
"benchmark",
8+
"playwright",
9+
"electron",
10+
"performance"
11+
],
12+
"main": "dist/index.cjs",
13+
"module": "dist/index.es5.js",
14+
"types": "dist/index.d.ts",
15+
"type": "module",
16+
"exports": {
17+
"types": "./dist/index.d.ts",
18+
"import": "./dist/index.es5.js",
19+
"require": "./dist/index.cjs"
20+
},
21+
"files": [
22+
"dist"
23+
],
24+
"scripts": {
25+
"build": "NODE_NO_WARNINGS=1 rollup -c rollup.config.ts --configPlugin typescript",
26+
"test": "echo 'no tests'",
27+
"test/integ": "echo 'no integ tests'",
28+
"lint": "eslint .",
29+
"typecheck": "tsc --noEmit --pretty",
30+
"format": "prettier --config ../../.prettierrc.json --ignore-path ../../.prettierignore --check .",
31+
"fix-format": "prettier --config ../../.prettierrc.json --ignore-path ../../.prettierignore --write .",
32+
"clean": "rm -rf dist"
33+
},
34+
"author": "Guillaume Lagrange <guillaume@codspeed.io>",
35+
"repository": "https://github.com/CodSpeedHQ/codspeed-node",
36+
"homepage": "https://codspeed.io",
37+
"license": "Apache-2.0",
38+
"devDependencies": {
39+
"playwright": "^1.48.0",
40+
"playwright-core": "^1.48.0",
41+
"vitest": "^3.2.4"
42+
},
43+
"dependencies": {
44+
"@codspeed/core": "workspace:^5.4.0"
45+
},
46+
"peerDependencies": {
47+
"playwright": ">=1.40.0"
48+
}
49+
}

‎packages/playwright-plugin/rollup.config.ts‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
import { defineConfig } from "rollup";
2+
import { declarationsPlugin, jsPlugins } from "../../rollup.options";
3+
import pkg from "./package.json" assert { type: "json" };
4+
5+
const entrypoint = "src/index.ts";
6+
7+
export default defineConfig([
8+
{
9+
input: entrypoint,
10+
output: [
11+
{
12+
file: pkg.types,
13+
format: "es",
14+
sourcemap: true,
15+
},
16+
],
17+
plugins: declarationsPlugin({ compilerOptions: { composite: false } }),
18+
},
19+
{
20+
input: entrypoint,
21+
output: [
22+
{
23+
file: pkg.main,
24+
format: "cjs",
25+
sourcemap: true,
26+
},
27+
{ file: pkg.module, format: "es", sourcemap: true },
28+
],
29+
plugins: jsPlugins(pkg.version),
30+
external: ["@codspeed/core", "playwright", "playwright-core"],
31+
},
32+
]);

0 commit comments

Comments
 (0)