Skip to content

Commit 090747c

Browse files
test.runOnce implementation and spec level cleanup. (#47)
* test.runOnce implementation * test.runOnce implementation withing deploy * suppress the rhdh configure logs * suppress the fixture logs
1 parent 06eab81 commit 090747c

File tree

15 files changed

+490
-93
lines changed

15 files changed

+490
-93
lines changed

docs/CLAUDE.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,10 @@ docs/
4646
import { test, expect } from "@red-hat-developer-hub/e2e-test-utils/test";
4747

4848
test.describe("Feature", () => {
49-
test.beforeAll(async ({ rhdh }) => { /* deploy */ });
49+
test.beforeAll(async ({ rhdh }) => {
50+
await rhdh.configure({ auth: "keycloak" });
51+
await rhdh.deploy();
52+
});
5053
test.beforeEach(async ({ loginHelper }) => { /* login */ });
5154
test("should...", async ({ uiHelper }) => { /* test */ });
5255
});
@@ -121,6 +124,8 @@ When documenting, reference these source files:
121124
| Fixtures | `src/playwright/fixtures/test.ts` |
122125
| Base Config | `src/playwright/base-config.ts` |
123126
| Global Setup | `src/playwright/global-setup.ts` |
127+
| Teardown Reporter | `src/playwright/teardown-reporter.ts` |
128+
| Teardown Namespaces | `src/playwright/teardown-namespaces.ts` |
124129

125130
## Common Tasks
126131

@@ -209,5 +214,6 @@ Base URL is configured as `/rhdh-e2e-test-utils/` in `config.ts`.
209214
| Helpers | `@red-hat-developer-hub/e2e-test-utils/helpers` | UIhelper, LoginHelper, etc. |
210215
| Page objects | `@red-hat-developer-hub/e2e-test-utils/pages` | CatalogPage, HomePage, etc. |
211216
| Utilities | `@red-hat-developer-hub/e2e-test-utils/utils` | KubernetesClientHelper, etc. |
217+
| Teardown | `@red-hat-developer-hub/e2e-test-utils/teardown` | Custom namespace teardown registration |
212218
| ESLint | `@red-hat-developer-hub/e2e-test-utils/eslint` | ESLint config |
213219
| TypeScript | `@red-hat-developer-hub/e2e-test-utils/tsconfig` | TSConfig base |

docs/api/playwright/base-config.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ Raw base configuration object. Use for advanced customization.
5959
retries: Number(process.env.PLAYWRIGHT_RETRIES ?? 0),
6060
workers: process.env.PLAYWRIGHT_WORKERS || "50%",
6161
outputDir: "node_modules/.cache/e2e-test-results",
62-
reporter: [["list"], ["html"], ["json"]],
62+
reporter: [["list"], ["html"], ["json"], ["teardown-reporter"]],
6363
use: {
6464
viewport: { width: 1920, height: 1080 },
6565
video: { mode: "retain-on-failure", size: { width: 1280, height: 720 } },

docs/api/playwright/test-fixtures.md

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ import { test, expect } from "@red-hat-developer-hub/e2e-test-utils/test";
1616

1717
**Type:** `RHDHDeployment`
1818

19-
Shared RHDH deployment across all tests in a worker.
19+
Shared RHDH deployment across all tests in a worker. `deploy()` automatically skips if the deployment already succeeded, even after worker restarts.
2020

2121
```typescript
2222
test.beforeAll(async ({ rhdh }) => {
@@ -80,6 +80,36 @@ test("using baseURL", async ({ page, baseURL }) => {
8080
});
8181
```
8282

83+
## `test.runOnce`
84+
85+
```typescript
86+
test.runOnce(key: string, fn: () => Promise<void> | void): Promise<boolean>
87+
```
88+
89+
Executes `fn` exactly once per test run, even across worker restarts. Returns `true` if executed, `false` if skipped.
90+
91+
::: tip
92+
`rhdh.deploy()` already uses `runOnce` internally, so you don't need to wrap simple deployments. Use `test.runOnce` when you have **additional expensive operations** (external services, scripts, data seeding) alongside `deploy()`.
93+
:::
94+
95+
| Parameter | Type | Description |
96+
|-----------|------|-------------|
97+
| `key` | `string` | Unique identifier for this operation |
98+
| `fn` | `() => Promise<void> \| void` | Function to execute once |
99+
100+
```typescript
101+
// Wrap pre-deploy setup that shouldn't repeat
102+
test.beforeAll(async ({ rhdh }) => {
103+
await test.runOnce("full-setup", async () => {
104+
await $`bash deploy-external-service.sh`;
105+
await rhdh.configure({ auth: "keycloak" });
106+
await rhdh.deploy(); // safe to nest, has its own internal protection
107+
});
108+
});
109+
```
110+
111+
See [Deployment Protection](/guide/core-concepts/playwright-fixtures#deployment-protection-built-in) and [`test.runOnce`](/guide/core-concepts/playwright-fixtures#test-runonce-—-run-any-expensive-operation-once) for details.
112+
83113
## Exported Types
84114

85115
```typescript

docs/changelog.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,18 @@
22

33
All notable changes to this project will be documented in this file.
44

5-
## [1.1.13] - Current
5+
## [1.1.14] - Current
6+
7+
### Added
8+
- **`deploy()` built-in protection**: `rhdh.deploy()` now automatically skips if the deployment already succeeded in the current test run. No code changes needed — existing `beforeAll` patterns work as before, but deployments are no longer repeated when Playwright restarts workers after test failures.
9+
- **`test.runOnce(key, fn)`**: Execute any function exactly once per test run, even across worker restarts. Use for expensive pre-deploy operations (external services, setup scripts, data seeding) that `deploy()` alone doesn't cover. Safe to nest with `deploy()`'s built-in protection.
10+
- **Teardown reporter**: Built-in Playwright reporter that automatically deletes Kubernetes namespaces after all tests complete. Active only in CI (`process.env.CI`).
11+
- **`registerTeardownNamespace(projectName, namespace)`**: Register custom namespaces for automatic cleanup. Import from `@red-hat-developer-hub/e2e-test-utils/teardown`.
12+
13+
### Changed
14+
- Namespace cleanup moved from worker fixture to teardown reporter to prevent premature deletion on test failures.
15+
16+
## [1.1.13]
617

718
### Added
819
- Support for GitHub authentication provider

docs/guide/core-concepts/playwright-fixtures.md

Lines changed: 152 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -154,16 +154,161 @@ export default defineConfig({
154154
});
155155
```
156156

157-
## Auto-Cleanup
157+
## Deployment Protection (Built-in)
158158

159-
In CI environments (when `CI` environment variable is set):
159+
`rhdh.deploy()` is automatically protected against redundant re-execution. When a test fails and Playwright restarts the worker, `deploy()` detects that the deployment already succeeded and skips — no re-deployment, no wasted time.
160160

161-
- Namespaces are automatically deleted after tests complete
162-
- Prevents resource accumulation on shared clusters
161+
This works out of the box. A simple `beforeAll` is all you need:
163162

164-
For local development:
165-
- Namespaces are preserved for debugging
166-
- Manual cleanup may be required
163+
```typescript
164+
test.beforeAll(async ({ rhdh }) => {
165+
await rhdh.configure({ auth: "keycloak" });
166+
await rhdh.deploy(); // runs once, skips on worker restart
167+
});
168+
```
169+
170+
::: tip Why is this needed?
171+
Playwright's `beforeAll` runs once **per worker**, not once per test run. When a test fails, Playwright kills the worker and creates a new one for remaining tests — causing `beforeAll` to run again. Without protection, this would re-deploy RHDH from scratch every time a test fails.
172+
:::
173+
174+
## `test.runOnce` — Run Any Expensive Operation Once
175+
176+
While `rhdh.deploy()` has built-in protection, you may have **other expensive operations** in your `beforeAll` that also shouldn't repeat on worker restart — deploying external services, seeding databases, running setup scripts, etc.
177+
178+
`test.runOnce` ensures any function executes **exactly once per test run**, even across worker restarts:
179+
180+
```typescript
181+
test.beforeAll(async ({ rhdh }) => {
182+
await test.runOnce("tech-radar-setup", async () => {
183+
await rhdh.configure({ auth: "keycloak" });
184+
await $`bash ${setupScript} ${namespace}`; // expensive external service
185+
process.env.DATA_URL = await rhdh.k8sClient.getRouteLocation(namespace, "my-service");
186+
await rhdh.deploy(); // also protected internally, nesting is safe
187+
});
188+
});
189+
```
190+
191+
### How It Works
192+
193+
- Uses file-based flags scoped to the Playwright runner process
194+
- When a worker restarts after a test failure, `runOnce` detects the flag and skips
195+
- Any state created by the function (deployments, services, data) stays alive
196+
- Flags reset automatically between test runs
197+
198+
### When to Use
199+
200+
| Scenario | What to use |
201+
|----------|------------|
202+
| Just `configure()` + `deploy()` | Nothing extra — `deploy()` is already protected |
203+
| Pre-deploy setup (external services, scripts, env vars) + `deploy()` | Wrap the entire block in `test.runOnce` |
204+
| Multiple independent expensive operations | Use separate `test.runOnce` calls with different keys |
205+
206+
### Examples
207+
208+
**Simple deployment — no `test.runOnce` needed:**
209+
210+
```typescript
211+
test.beforeAll(async ({ rhdh }) => {
212+
await rhdh.configure({ auth: "keycloak" });
213+
await rhdh.deploy();
214+
});
215+
```
216+
217+
**Pre-deploy setup — wrap in `test.runOnce`:**
218+
219+
```typescript
220+
test.beforeAll(async ({ rhdh }) => {
221+
await test.runOnce("tech-radar-full-setup", async () => {
222+
await rhdh.configure({ auth: "keycloak" });
223+
await $`bash deploy-external-service.sh ${rhdh.deploymentConfig.namespace}`;
224+
process.env.DATA_URL = await rhdh.k8sClient.getRouteLocation(
225+
rhdh.deploymentConfig.namespace, "data-provider"
226+
);
227+
await rhdh.deploy();
228+
});
229+
});
230+
```
231+
232+
**Multiple independent operations with separate keys:**
233+
234+
```typescript
235+
test.describe("Feature A", () => {
236+
test.beforeAll(async ({ rhdh }) => {
237+
await test.runOnce("seed-catalog-data", async () => {
238+
await apiHelper.importEntity("https://example.com/catalog-info.yaml");
239+
});
240+
});
241+
});
242+
243+
test.describe("Feature B", () => {
244+
test.beforeAll(async () => {
245+
await test.runOnce("deploy-mock-api", async () => {
246+
await $`bash deploy-mock.sh`;
247+
});
248+
});
249+
});
250+
```
251+
252+
### Key: Unique Identifier
253+
254+
The `key` parameter must be unique across all `runOnce` calls in your test run. Use a descriptive name that reflects the operation:
255+
256+
```typescript
257+
await test.runOnce("tech-radar-deploy", async () => { ... });
258+
await test.runOnce("tech-radar-data-provider", async () => { ... });
259+
await test.runOnce("catalog-seed-data", async () => { ... });
260+
```
261+
262+
### Nesting
263+
264+
`test.runOnce` can be safely nested. Since `rhdh.deploy()` uses `runOnce` internally, wrapping it in an outer `test.runOnce` is harmless — the outer call skips everything on worker restart, and the inner one never runs:
265+
266+
```typescript
267+
await test.runOnce("full-setup", async () => {
268+
await $`bash setup.sh`; // protected by outer runOnce
269+
await rhdh.deploy(); // has its own internal runOnce (harmless)
270+
});
271+
```
272+
273+
## Namespace Cleanup (Teardown)
274+
275+
In CI environments (`CI` environment variable is set), namespaces are automatically deleted after all tests complete. This is handled by a built-in **teardown reporter** that:
276+
277+
1. Runs in the main Playwright process (survives worker restarts)
278+
2. Waits for **all tests** to finish
279+
3. Deletes the namespace matching the project name
280+
281+
### Default Behavior
282+
283+
No configuration needed. The namespace is derived from your project name:
284+
285+
```typescript
286+
// playwright.config.ts
287+
projects: [
288+
{ name: "tech-radar" }, // Namespace "tech-radar" deleted after all tests
289+
{ name: "catalog" }, // Namespace "catalog" deleted after all tests
290+
]
291+
```
292+
293+
### Custom Namespaces
294+
295+
If you deploy to a namespace that differs from the project name, register it for cleanup:
296+
297+
```typescript
298+
import { registerTeardownNamespace } from "@red-hat-developer-hub/e2e-test-utils/teardown";
299+
300+
test.beforeAll(async ({ rhdh }) => {
301+
await rhdh.configure({ namespace: "my-custom-ns", auth: "keycloak" });
302+
await rhdh.deploy();
303+
registerTeardownNamespace("my-project", "my-custom-ns");
304+
});
305+
```
306+
307+
Multiple namespaces per project are supported — all registered namespaces are deleted after that project's tests complete.
308+
309+
### Local Development
310+
311+
Namespaces are **not** deleted locally (only in CI). This preserves deployments for debugging.
167312

168313
## Best Practices for Projects and Spec Files
169314

docs/guide/deployment/rhdh-deployment.md

Lines changed: 19 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ import { test } from "@red-hat-developer-hub/e2e-test-utils/test";
3434
test.beforeAll(async ({ rhdh }) => {
3535
// rhdh is already instantiated with namespace from project name
3636
await rhdh.configure({ auth: "keycloak" });
37-
await rhdh.deploy();
37+
await rhdh.deploy(); // automatically skips if already deployed
3838
});
3939

4040
test("example", async ({ rhdh }) => {
@@ -110,6 +110,8 @@ test.setTimeout(900_000);
110110
await rhdh.deploy({ timeout: null });
111111
```
112112

113+
`deploy()` automatically skips if the deployment already succeeded in the current test run (e.g., after a worker restart due to test failure). This prevents expensive re-deployments.
114+
113115
This method:
114116
1. Merges configuration files (common → auth → project)
115117
2. [Injects plugin metadata](/guide/configuration/config-files#plugin-metadata-injection) into dynamic plugins config
@@ -176,7 +178,7 @@ await deployment.teardown();
176178
```
177179

178180
::: warning
179-
This permanently deletes all resources in the namespace. In CI, this happens automatically.
181+
You typically don't need to call this manually. In CI, the built-in teardown reporter automatically deletes namespaces after all tests complete. See [Namespace Cleanup](/guide/core-concepts/playwright-fixtures#namespace-cleanup-teardown).
180182
:::
181183

182184
## Properties
@@ -259,21 +261,24 @@ import { test } from "@red-hat-developer-hub/e2e-test-utils/test";
259261
import { $ } from "@red-hat-developer-hub/e2e-test-utils/utils";
260262

261263
test.beforeAll(async ({ rhdh }) => {
262-
const namespace = rhdh.deploymentConfig.namespace;
264+
// Wrap in test.runOnce because the setup script is also expensive
265+
await test.runOnce("my-plugin-setup", async () => {
266+
const namespace = rhdh.deploymentConfig.namespace;
263267

264-
// Configure RHDH
265-
await rhdh.configure({ auth: "keycloak" });
268+
// Configure RHDH
269+
await rhdh.configure({ auth: "keycloak" });
266270

267-
// Run custom setup before deployment
268-
await $`bash scripts/setup.sh ${namespace}`;
271+
// Run custom setup before deployment
272+
await $`bash scripts/setup.sh ${namespace}`;
269273

270-
// Set runtime environment variables
271-
process.env.MY_CUSTOM_URL = await rhdh.k8sClient.getRouteLocation(
272-
namespace,
273-
"my-service"
274-
);
274+
// Set runtime environment variables
275+
process.env.MY_CUSTOM_URL = await rhdh.k8sClient.getRouteLocation(
276+
namespace,
277+
"my-service"
278+
);
275279

276-
// Deploy RHDH (uses env vars set above)
277-
await rhdh.deploy();
280+
// Deploy RHDH (has built-in protection, safe to nest inside runOnce)
281+
await rhdh.deploy();
282+
});
278283
});
279284
```

0 commit comments

Comments
 (0)