Skip to content

Commit 1dd2034

Browse files
test.runOnce implementation withing deploy
1 parent c32cacc commit 1dd2034

11 files changed

Lines changed: 229 additions & 145 deletions

File tree

docs/CLAUDE.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,8 @@ docs/
4747

4848
test.describe("Feature", () => {
4949
test.beforeAll(async ({ rhdh }) => {
50-
await test.runOnce("my-plugin-deploy", async () => { /* deploy */ });
50+
await rhdh.configure({ auth: "keycloak" });
51+
await rhdh.deploy();
5152
});
5253
test.beforeEach(async ({ loginHelper }) => { /* login */ });
5354
test("should...", async ({ uiHelper }) => { /* test */ });

docs/api/playwright/test-fixtures.md

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -16,14 +16,12 @@ import { test, expect } from "@red-hat-developer-hub/e2e-test-utils/test";
1616

1717
**Type:** `RHDHDeployment`
1818

19-
Shared RHDH deployment across all tests in a worker. Wrap expensive setup in `test.runOnce` to avoid re-deploying when workers restart after test failures.
19+
Shared RHDH deployment across all tests in a worker. `deploy()` automatically skips if the deployment already succeeded, even after worker restarts.
2020

2121
```typescript
2222
test.beforeAll(async ({ rhdh }) => {
23-
await test.runOnce("my-plugin-deploy", async () => {
24-
await rhdh.configure({ auth: "keycloak" });
25-
await rhdh.deploy();
26-
});
23+
await rhdh.configure({ auth: "keycloak" });
24+
await rhdh.deploy();
2725
});
2826

2927
test("access rhdh", async ({ rhdh }) => {
@@ -88,23 +86,29 @@ test("using baseURL", async ({ page, baseURL }) => {
8886
test.runOnce(key: string, fn: () => Promise<void> | void): Promise<boolean>
8987
```
9088

91-
Executes `fn` exactly once per test run, even across worker restarts. Returns `true` if executed, `false` if skipped. Useful for expensive or persistent operations (deployments, database seeding, service provisioning) that should not repeat after a worker restart.
89+
Executes `fn` exactly once per test run, even across worker restarts. Returns `true` if executed, `false` if skipped.
90+
91+
::: tip
92+
`rhdh.deploy()` already uses `runOnce` internally, so you don't need to wrap simple deployments. Use `test.runOnce` when you have **additional expensive operations** (external services, scripts, data seeding) alongside `deploy()`.
93+
:::
9294

9395
| Parameter | Type | Description |
9496
|-----------|------|-------------|
9597
| `key` | `string` | Unique identifier for this operation |
9698
| `fn` | `() => Promise<void> \| void` | Function to execute once |
9799

98100
```typescript
101+
// Wrap pre-deploy setup that shouldn't repeat
99102
test.beforeAll(async ({ rhdh }) => {
100-
await test.runOnce("my-deploy", async () => {
103+
await test.runOnce("full-setup", async () => {
104+
await $`bash deploy-external-service.sh`;
101105
await rhdh.configure({ auth: "keycloak" });
102-
await rhdh.deploy();
106+
await rhdh.deploy(); // safe to nest, has its own internal protection
103107
});
104108
});
105109
```
106110

107-
See [Playwright Fixtures — `test.runOnce`](/guide/core-concepts/playwright-fixtures#test-runonce-—-execute-a-function-once-per-test-run) for detailed usage and examples.
111+
See [Deployment Protection](/guide/core-concepts/playwright-fixtures#deployment-protection-built-in) and [`test.runOnce`](/guide/core-concepts/playwright-fixtures#test-runonce-—-run-any-expensive-operation-once) for details.
108112

109113
## Exported Types
110114

@@ -121,10 +125,8 @@ import { test, expect } from "@red-hat-developer-hub/e2e-test-utils/test";
121125

122126
test.describe("My Tests", () => {
123127
test.beforeAll(async ({ rhdh }) => {
124-
await test.runOnce("my-plugin-deploy", async () => {
125-
await rhdh.configure({ auth: "keycloak" });
126-
await rhdh.deploy();
127-
});
128+
await rhdh.configure({ auth: "keycloak" });
129+
await rhdh.deploy();
128130
});
129131

130132
test.beforeEach(async ({ page, loginHelper }) => {

docs/changelog.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@ All notable changes to this project will be documented in this file.
55
## [1.1.14] - Current
66

77
### Added
8-
- **`test.runOnce(key, fn)`**: Execute a function exactly once per test run, even across worker restarts. Prevents re-running expensive operations (deployments, service provisioning, data seeding) when Playwright creates new workers after test failures.
8+
- **`deploy()` built-in protection**: `rhdh.deploy()` now automatically skips if the deployment already succeeded in the current test run. No code changes needed — existing `beforeAll` patterns work as before, but deployments are no longer repeated when Playwright restarts workers after test failures.
9+
- **`test.runOnce(key, fn)`**: Execute any function exactly once per test run, even across worker restarts. Use for expensive pre-deploy operations (external services, setup scripts, data seeding) that `deploy()` alone doesn't cover. Safe to nest with `deploy()`'s built-in protection.
910
- **Teardown reporter**: Built-in Playwright reporter that automatically deletes Kubernetes namespaces after all tests complete. Active only in CI (`process.env.CI`).
1011
- **`registerTeardownNamespace(projectName, namespace)`**: Register custom namespaces for automatic cleanup. Import from `@red-hat-developer-hub/e2e-test-utils/teardown`.
1112

docs/guide/core-concepts/playwright-fixtures.md

Lines changed: 95 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -154,17 +154,36 @@ export default defineConfig({
154154
});
155155
```
156156

157-
## `test.runOnce` — Execute a Function Once Per Test Run
157+
## Deployment Protection (Built-in)
158158

159-
Playwright's `beforeAll` runs once **per worker**, not once per test run. When a test fails, Playwright kills the worker and creates a new one for remaining tests — causing `beforeAll` to run again. For operations that are expensive or produce persistent side effects, this leads to unnecessary re-execution.
159+
`rhdh.deploy()` is automatically protected against redundant re-execution. When a test fails and Playwright restarts the worker, `deploy()` detects that the deployment already succeeded and skips — no re-deployment, no wasted time.
160160

161-
`test.runOnce` ensures a function executes **exactly once per test run**, even across worker restarts:
161+
This works out of the box. A simple `beforeAll` is all you need:
162162

163163
```typescript
164164
test.beforeAll(async ({ rhdh }) => {
165-
await test.runOnce("my-plugin-deploy", async () => {
165+
await rhdh.configure({ auth: "keycloak" });
166+
await rhdh.deploy(); // runs once, skips on worker restart
167+
});
168+
```
169+
170+
::: tip Why is this needed?
171+
Playwright's `beforeAll` runs once **per worker**, not once per test run. When a test fails, Playwright kills the worker and creates a new one for remaining tests — causing `beforeAll` to run again. Without protection, this would re-deploy RHDH from scratch every time a test fails.
172+
:::
173+
174+
## `test.runOnce` — Run Any Expensive Operation Once
175+
176+
While `rhdh.deploy()` has built-in protection, you may have **other expensive operations** in your `beforeAll` that also shouldn't repeat on worker restart — deploying external services, seeding databases, running setup scripts, etc.
177+
178+
`test.runOnce` ensures any function executes **exactly once per test run**, even across worker restarts:
179+
180+
```typescript
181+
test.beforeAll(async ({ rhdh }) => {
182+
await test.runOnce("tech-radar-setup", async () => {
166183
await rhdh.configure({ auth: "keycloak" });
167-
await rhdh.deploy();
184+
await $`bash ${setupScript} ${namespace}`; // expensive external service
185+
process.env.DATA_URL = await rhdh.k8sClient.getRouteLocation(namespace, "my-service");
186+
await rhdh.deploy(); // also protected internally, nesting is safe
168187
});
169188
});
170189
```
@@ -173,40 +192,81 @@ test.beforeAll(async ({ rhdh }) => {
173192

174193
- Uses file-based flags scoped to the Playwright runner process
175194
- When a worker restarts after a test failure, `runOnce` detects the flag and skips
176-
- Any state created by the function (deployments, databases, services) stays alive
195+
- Any state created by the function (deployments, services, data) stays alive
177196
- Flags reset automatically between test runs
178197

179198
### When to Use
180199

181-
Use `test.runOnce` when your `beforeAll` performs an operation that:
182-
- Is **expensive** (deployments, database seeding, service provisioning)
183-
- Creates **persistent state** that survives beyond the worker process (Kubernetes resources, external services, test data)
184-
- Should **not repeat** once successfully completed
185-
186-
Common examples:
187-
- RHDH deployment (`rhdh.deploy()`)
188-
- External service deployment (customization providers, mock APIs)
189-
- Database seeding or migration
190-
- Any setup script that takes significant time
200+
| Scenario | What to use |
201+
|----------|------------|
202+
| Just `configure()` + `deploy()` | Nothing extra — `deploy()` is already protected |
203+
| Pre-deploy setup (external services, scripts, env vars) + `deploy()` | Wrap the entire block in `test.runOnce` |
204+
| Multiple independent expensive operations | Use separate `test.runOnce` calls with different keys |
191205

192-
### Key: Unique Identifier
206+
### Examples
193207

194-
The `key` parameter must be unique across all `runOnce` calls in your test run. Use a descriptive name:
208+
**Simple deployment — no `test.runOnce` needed:**
195209

196210
```typescript
197-
// Deploy RHDH
198-
await test.runOnce("tech-radar-deploy", async () => {
211+
test.beforeAll(async ({ rhdh }) => {
212+
await rhdh.configure({ auth: "keycloak" });
199213
await rhdh.deploy();
200214
});
215+
```
216+
217+
**Pre-deploy setup — wrap in `test.runOnce`:**
218+
219+
```typescript
220+
test.beforeAll(async ({ rhdh }) => {
221+
await test.runOnce("tech-radar-full-setup", async () => {
222+
await rhdh.configure({ auth: "keycloak" });
223+
await $`bash deploy-external-service.sh ${rhdh.deploymentConfig.namespace}`;
224+
process.env.DATA_URL = await rhdh.k8sClient.getRouteLocation(
225+
rhdh.deploymentConfig.namespace, "data-provider"
226+
);
227+
await rhdh.deploy();
228+
});
229+
});
230+
```
231+
232+
**Multiple independent operations with separate keys:**
233+
234+
```typescript
235+
test.describe("Feature A", () => {
236+
test.beforeAll(async ({ rhdh }) => {
237+
await test.runOnce("seed-catalog-data", async () => {
238+
await apiHelper.importEntity("https://example.com/catalog-info.yaml");
239+
});
240+
});
241+
});
201242

202-
// Deploy an external service
203-
await test.runOnce("tech-radar-data-provider", async () => {
204-
await $`bash ${setupScript} ${namespace}`;
243+
test.describe("Feature B", () => {
244+
test.beforeAll(async () => {
245+
await test.runOnce("deploy-mock-api", async () => {
246+
await $`bash deploy-mock.sh`;
247+
});
248+
});
205249
});
250+
```
251+
252+
### Key: Unique Identifier
253+
254+
The `key` parameter must be unique across all `runOnce` calls in your test run. Use a descriptive name that reflects the operation:
255+
256+
```typescript
257+
await test.runOnce("tech-radar-deploy", async () => { ... });
258+
await test.runOnce("tech-radar-data-provider", async () => { ... });
259+
await test.runOnce("catalog-seed-data", async () => { ... });
260+
```
261+
262+
### Nesting
206263

207-
// Seed test data
208-
await test.runOnce("catalog-seed-data", async () => {
209-
await apiHelper.importEntity("https://example.com/catalog-info.yaml");
264+
`test.runOnce` can be safely nested. Since `rhdh.deploy()` uses `runOnce` internally, wrapping it in an outer `test.runOnce` is harmless — the outer call skips everything on worker restart, and the inner one never runs:
265+
266+
```typescript
267+
await test.runOnce("full-setup", async () => {
268+
await $`bash setup.sh`; // protected by outer runOnce
269+
await rhdh.deploy(); // has its own internal runOnce (harmless)
210270
});
211271
```
212272

@@ -215,7 +275,7 @@ await test.runOnce("catalog-seed-data", async () => {
215275
In CI environments (`CI` environment variable is set), namespaces are automatically deleted after all tests complete. This is handled by a built-in **teardown reporter** that:
216276

217277
1. Runs in the main Playwright process (survives worker restarts)
218-
2. Waits for **all tests** in a project to finish
278+
2. Waits for **all tests** to finish
219279
3. Deletes the namespace matching the project name
220280

221281
### Default Behavior
@@ -238,11 +298,9 @@ If you deploy to a namespace that differs from the project name, register it for
238298
import { registerTeardownNamespace } from "@red-hat-developer-hub/e2e-test-utils/teardown";
239299

240300
test.beforeAll(async ({ rhdh }) => {
241-
await test.runOnce("custom-deploy", async () => {
242-
await rhdh.configure({ namespace: "my-custom-ns", auth: "keycloak" });
243-
await rhdh.deploy();
244-
registerTeardownNamespace("my-project", "my-custom-ns");
245-
});
301+
await rhdh.configure({ namespace: "my-custom-ns", auth: "keycloak" });
302+
await rhdh.deploy();
303+
registerTeardownNamespace("my-project", "my-custom-ns");
246304
});
247305
```
248306

@@ -300,14 +358,12 @@ import { test, expect } from "@red-hat-developer-hub/e2e-test-utils/test";
300358

301359
test.describe("My Plugin Tests", () => {
302360
test.beforeAll(async ({ rhdh }) => {
303-
await test.runOnce("my-plugin-deploy", async () => {
304-
await rhdh.configure({
305-
auth: "keycloak",
306-
appConfig: "tests/config/app-config.yaml",
307-
dynamicPlugins: "tests/config/plugins.yaml",
308-
});
309-
await rhdh.deploy();
361+
await rhdh.configure({
362+
auth: "keycloak",
363+
appConfig: "tests/config/app-config.yaml",
364+
dynamicPlugins: "tests/config/plugins.yaml",
310365
});
366+
await rhdh.deploy();
311367
});
312368

313369
test.beforeEach(async ({ page, loginHelper }) => {

docs/guide/deployment/rhdh-deployment.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -33,10 +33,8 @@ import { test } from "@red-hat-developer-hub/e2e-test-utils/test";
3333

3434
test.beforeAll(async ({ rhdh }) => {
3535
// rhdh is already instantiated with namespace from project name
36-
await test.runOnce("my-plugin-deploy", async () => {
37-
await rhdh.configure({ auth: "keycloak" });
38-
await rhdh.deploy();
39-
});
36+
await rhdh.configure({ auth: "keycloak" });
37+
await rhdh.deploy(); // automatically skips if already deployed
4038
});
4139

4240
test("example", async ({ rhdh }) => {
@@ -112,6 +110,8 @@ test.setTimeout(900_000);
112110
await rhdh.deploy({ timeout: null });
113111
```
114112

113+
`deploy()` automatically skips if the deployment already succeeded in the current test run (e.g., after a worker restart due to test failure). This prevents expensive re-deployments.
114+
115115
This method:
116116
1. Merges configuration files (common → auth → project)
117117
2. [Injects plugin metadata](/guide/configuration/config-files#plugin-metadata-injection) into dynamic plugins config
@@ -261,7 +261,8 @@ import { test } from "@red-hat-developer-hub/e2e-test-utils/test";
261261
import { $ } from "@red-hat-developer-hub/e2e-test-utils/utils";
262262

263263
test.beforeAll(async ({ rhdh }) => {
264-
await test.runOnce("my-plugin-deploy", async () => {
264+
// Wrap in test.runOnce because the setup script is also expensive
265+
await test.runOnce("my-plugin-setup", async () => {
265266
const namespace = rhdh.deploymentConfig.namespace;
266267

267268
// Configure RHDH
@@ -276,7 +277,7 @@ test.beforeAll(async ({ rhdh }) => {
276277
"my-service"
277278
);
278279

279-
// Deploy RHDH (uses env vars set above)
280+
// Deploy RHDH (has built-in protection, safe to nest inside runOnce)
280281
await rhdh.deploy();
281282
});
282283
});

0 commit comments

Comments
 (0)