You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: document skip tag support for overlay E2E tests
Add documentation for the new tag-based test skipping feature that
auto-derives --grep-invert from JOB_NAME in run-e2e.sh. Updated
run-e2e reference, CI pipeline tutorial, and spec files guide.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/overlay/reference/run-e2e.md
+48-1Lines changed: 48 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -82,7 +82,54 @@ A workspace is discovered when it has a `workspaces/<name>/e2e-tests/` directory
82
82
|----------|-------------|---------|
83
83
|`E2E_NIGHTLY_MODE`| When `true`, uses released OCI images from metadata; defaults `E2E_TEST_UTILS_VERSION` to `latest`|`false`|
84
84
|`GIT_PR_NUMBER`| PR number for OCI URL generation (uses PR-built images) | - |
85
-
|`JOB_NAME`| CI job name; if contains `periodic-`, disables metadata injection | - |
85
+
|`JOB_NAME`| CI job name; if contains `periodic-`, disables metadata injection. Also used to [auto-derive skip tags](#skip-tags). | - |
86
+
87
+
## Skip Tags
88
+
89
+
When `JOB_NAME` is set (by OpenShift CI), the script auto-derives a Playwright tag and injects `--grep-invert` to exclude tests tagged with it. This lets test authors skip specific tests in specific CI jobs using standard Playwright tags.
90
+
91
+
### How It Works
92
+
93
+
The job suffix is extracted from `JOB_NAME` by stripping everything up to and including `-e2e-`:
94
+
95
+
| JOB_NAME (suffix shown) | Derived tag |`--grep-invert`|
Skip tags work the same way when running tests from a workspace — pass `--grep-invert` manually:
124
+
125
+
```bash
126
+
cd workspaces/tech-radar/e2e-tests
127
+
yarn test -- --grep-invert "@skip-ocp-helm"
128
+
```
129
+
130
+
::: info Precedence
131
+
The auto-derived `--grep-invert` is prepended to the Playwright arguments. If you also pass `--grep-invert` on the command line, Playwright uses the last value (last wins), so your manual flag takes precedence.
In CI, namespaces are automatically deleted after all tests complete via the built-in teardown reporter. No manual cleanup code is needed. See [Namespace Cleanup](/guide/core-concepts/playwright-fixtures#namespace-cleanup-teardown) for details.
53
53
:::
54
54
55
+
## Tagging Tests
56
+
57
+
Use Playwright tags to control which tests run in which CI jobs. Tags are added via the second argument to `test.describe` or `test`:
When running in CI, `run-e2e.sh` auto-derives the skip tag from `JOB_NAME` and passes `--grep-invert` to Playwright. From a workspace directory, pass it manually:
75
+
76
+
```bash
77
+
yarn test -- --grep-invert "@skip-ocp-helm"
78
+
```
79
+
80
+
See [Skip Tags](/overlay/reference/run-e2e#skip-tags) for the full list of tags and derivation logic.
81
+
55
82
## Imports
56
83
57
84
Import test utilities from `@red-hat-developer-hub/e2e-test-utils`:
Copy file name to clipboardExpand all lines: docs/overlay/tutorials/ci-pipeline.md
+31Lines changed: 31 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -93,6 +93,37 @@ To test nightly job changes via the release repo:
93
93
2. Comment `/test e2e-ocp-helm-nightly` on the PR
94
94
3. The rehearse job runs with `JOB_MODE=nightly`, testing all workspaces
95
95
96
+
## Skipping Tests by CI Job
97
+
98
+
Tests can opt out of specific CI jobs using Playwright tags. The `run-e2e.sh` script auto-derives a skip tag from `JOB_NAME` and passes `--grep-invert` to Playwright.
99
+
100
+
For example, to skip a test in the nightly helm job:
0 commit comments