You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: cmd/ui/tests/README.md
+20-38Lines changed: 20 additions & 38 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,6 @@
1
1
# BloodHound UI Playwright Tests
2
2
3
-
Playwright test suites and shared scaffolding for the `bloodhound-ui` workspace (`cmd/ui`).
4
-
5
-
This directory is the entry point for browser-driven tests that run against the BloodHound UI. It is organized into per-suite subdirectories (today only `a11y/`) so additional suites with different purposes — end-to-end, visual regression, and so on — can be added as siblings without disturbing existing scaffolding. Each suite gets its own Playwright config, `testMatch` pattern, and artifact subfolder under `cmd/ui/playwright/`. Unit tests live next to their source under `cmd/ui/src/**/*.test.ts(x)` and are run with Vitest — they are not part of this directory. Vitest is configured (in `cmd/ui/vite.config.ts`) to exclude `tests/**` from collection, so `yarn test` will not pick up the `*.a11y.spec.ts` files; the Playwright suite only runs via `yarn test:a11y`.
3
+
This directory is the entry point for browser-driven Bloodhound UI Playwright test suites. It is organized into per-suite subdirectories. Each suite has its own Playwright config, `testMatch` pattern, and artifact subfolder under `cmd/ui/playwright/`. Vitest, Bloodhound UI's unit test framework, is configured to exclude `tests/**` from collection (`cmd/ui/vite.config.ts`), so `yarn test` will not pick up the `*.a11y.spec.ts` files; the Playwright suite only runs via `yarn test:a11y`.
6
4
7
5
## Layout
8
6
@@ -14,25 +12,27 @@ tests/
14
12
└── *.a11y.spec.ts
15
13
```
16
14
17
-
Shared Playwright building blocks (the `axe` fixture, the auth-snapshot helper, the cypher stub, theme types, and the per-theme storage-state path convention) live in the `bh-playwright-testing` workspace package (`packages/javascript/bh-playwright-testing`) so they can be reused by other consumers (e.g. BHE's `cmd/ui`). See that package's README for the full module map.
15
+
Shared Playwright building blocks (fixtures, stubs, and auth helper) live in the `bh-playwright-testing` workspace package (`packages/javascript/bh-playwright-testing`) for reusablility. See that package's README for the full module map.
18
16
19
17
## Shared Scaffolding
20
18
21
19
### `global.setup.ts`
22
20
23
21
A Playwright **setup project** that runs once per Playwright invocation before any browser-theme project executes. It:
24
22
25
-
1. Installs the cypher stub from `bh-playwright-testing/stubs/graph-has-data` to control "No Data Available" dialog state.
26
-
2. Delegates to `loginAndSnapshotThemes` from `bh-playwright-testing/auth`, passing `A11Y_TEST_USERNAME` / `A11Y_TEST_PASSWORD`, the per-theme storage-state path resolver (`authStorageStateFor` from `bh-playwright-testing/themes`), and a `dismissPostLogin` hook that dismisses the "No Data Available" dialog if it raced ahead of the stub.
23
+
1. Installs any stubs
24
+
2. Delegates to `bh-playwright-testing` helpers to manage per-theme auth sessions
27
25
28
-
`loginAndSnapshotThemes` logs in via `/ui/login`, snapshots `storageState` to `playwright/.auth/user-light.json`, toggles dark mode (waiting for the throttled `persistedState` write to land), then snapshots `playwright/.auth/user-dark.json`. Capturing both theme snapshots from a single session avoids the parallel-login race where two setups as the same user would invalidate each other's session. Downstream projects load the snapshot that matches their theme via `authStorageStateFor(theme)`.
26
+
> About auth session management:\
27
+
> `loginAndSnapshotThemes` logs in via `/ui/login`, snapshots `storageState` to `playwright/.auth/user-light.json`, toggles dark mode (waiting for the throttled `persistedState` write to land), then snapshots `playwright/.auth/user-dark.json`. Capturing both theme snapshots from a single session avoids the parallel-login race where two setups as the same user would invalidate each other's session. Downstream projects load the snapshot that matches their theme via `authStorageStateFor(theme)`.
29
28
30
29
## Conventions
31
30
32
-
-**Filenames.** Suite test files use a `<name>.<suite>.spec.ts` pattern (e.g. `login.a11y.spec.ts`). Each suite's Playwright config picks them up with a `testMatch` regex.
33
-
-**Env loading.** Suite configs load `cmd/ui/.env` via `dotenv` at config-evaluation time. See `cmd/ui/.env.example` for required keys.
34
-
-**Artifacts directory.** All Playwright output is written under `cmd/ui/playwright/`, which is gitignored. Each suite scopes its artifacts to its own subfolder (e.g. `playwright/a11y/...`).
35
-
-**Auth bootstrapping.** Suites that need an authenticated session depend on the `setup` project and load the appropriate `storageState` rather than logging in per-test.
31
+
-**Folders** - Each suite had its own folder: `cmd/ui/tests/<suite>` (e.g. `cmd/ui/tests/a11y`)
32
+
-**Filenames** - Within each suite folder, test files use a `<name>.<suite>.spec.ts` pattern (e.g. `login.a11y.spec.ts`). Each suite's Playwright config picks them up with a `testMatch` regex.
33
+
-**Env loading** - Suite configs may load `cmd/ui/.env` via `dotenv` at config-evaluation time. See `cmd/ui/.env.example` for example keys used by the `a11y` suite.
34
+
-**Artifacts directory** - All Playwright output is written under `cmd/ui/playwright/`, which is gitignored. Each suite scopes its artifacts to its own subfolder (e.g. `playwright/a11y/...`).
35
+
-**Auth bootstrapping** - Suites that need an authenticated session depend on the `setup` project and load the appropriate `storageState` rather than logging in per-test.
36
36
37
37
## Accessibility Tests (`a11y/`)
38
38
@@ -42,13 +42,11 @@ The accessibility suite runs `axe-core` scans against the live BloodHound UI thr
42
42
43
43
The a11y suite's goal is automated WCAG 2.x accessibility regression coverage for user-facing pages in the BloodHound UI. See [Adding A New A11y Spec](#adding-a-new-a11y-spec) for the per-spec recipe.
44
44
45
-
Each spec is a self-contained scan of one route or one in-page state. Specs are kept narrow rather than chained because axe violations are easier to triage when the failure points at a single, well-scoped DOM subtree.
45
+
Each spec is a self-contained scan of one route, covering any of its in-page states. Specs are kept narrow rather than chained because axe violations are easier to triage when the failure points at a single, well-scoped DOM subtree.
46
46
47
47
### `fixtures.ts`
48
48
49
-
A wrapper around `bh-playwright-testing`'s `test` that adds a `page` fixture which installs the shared `installGraphHasDataStub` from `bh-playwright-testing/stubs/graph-has-data`. It lives at `tests/fixtures.ts` (one level above `a11y/`) because it is suite-agnostic — any future suite that wants the same "graph has data" precondition can import the same wrapper. Specs that need the precondition import `test` from `../fixtures`; specs that don't (currently only `login.a11y.spec.ts`) import directly from `bh-playwright-testing`.
50
-
51
-
`fixtures.ts` also re-exports `expect` and `expectNoAccessibilityViolations` so a spec can import everything it needs from one module.
49
+
A wrapper around `bh-playwright-testing`'s `test` that adds a `page` fixture which installs a shared stub from `bh-playwright-testing` for managing graph data state (ensuring the "No Data" dialog is not present). `fixtures.ts` also re-exports `expect` and `expectNoAccessibilityViolations` so a spec can import everything it needs from one module.
52
50
53
51
### Running The Suite
54
52
@@ -64,7 +62,7 @@ Running in interactive UI mode
64
62
yarn test:a11y --ui
65
63
```
66
64
67
-
The `bhce/` script delegates to the `bloodhound-ui` workspace, whose `test:a11y` script clears the playwright artifact directory (`cmd/ui/playwright`) and runs the a11y test suite as configured in `playwright.a11y.config.ts` — the clean step is baked in so every run starts with a fresh `playwright/` directory. CI-mode behavior (single worker, 1 retry, `forbidOnly`) is auto-enabled when `process.env.CI` is set, so there is no separate `:ci` script. Note that a full CI mode process is not yet integrated other than config values. The Playwright config (`cmd/ui/playwright.a11y.config.ts`) generates a project matrix of `chromium-light`, `chromium-dark`, `firefox-light`, `firefox-dark`, each depending on the `setup` project.
65
+
The `bhce/` script delegates to the `bloodhound-ui` workspace, whose `test:a11y` script clears the playwright artifact directory (`cmd/ui/playwright`) and runs the a11y test suite as configured in `playwright.a11y.config.ts` — the clean step is baked in so every run starts with a fresh `playwright/` directory. CI-mode behavior (single worker, 1 retry, `forbidOnly`) is auto-enabled when `process.env.CI` is set, so there is no separate `:ci` script. The Playwright config (`cmd/ui/playwright.a11y.config.ts`) generates a project matrix of `chromium-light`, `chromium-dark`, `firefox-light`, `firefox-dark`, each depending on the `setup` project.
68
66
69
67
By default, the full 2x2 matrix (browsers x themes) is run, but projects may be individually specified:
Populate `cmd/ui/.env` (see `cmd/ui/.env.example`):
79
77
80
78
-`A11Y_TEST_URL` — base URL for the UI (e.g. `http://127.0.0.1:3000`, `http://bloodhound.localhost`).
81
-
-`A11Y_TEST_USERNAME` / `A11Y_TEST_PASSWORD` — credentials used by `global.setup.ts`.
79
+
-`A11Y_TEST_USERNAME` / `A11Y_TEST_PASSWORD` — app login credentials used by `global.setup.ts`.
82
80
-`A11Y_TEST_SERVE` — when `true`, Playwright starts the Vite dev server itself via `yarn dev --host <host> --port <port>` derived from `A11Y_TEST_URL`. When unset or `false`, Playwright expects an already-running target at `A11Y_TEST_URL`. Used to target other environments such as `test`.
83
81
84
82
### Artifacts And Reports
@@ -89,31 +87,17 @@ Each run writes to `cmd/ui/playwright/a11y/`:
89
87
-`html-report/` — Playwright HTML report (browsable as-is).
90
88
-`allure-results/` — Allure raw results (`*-result.json`). See [Viewing The Reports](#viewing-the-reports) below for how to render it.
91
89
92
-
Every assertion via `expectNoAccessibilityViolations` attaches `axe-results.json` (always) and `a11y-violations.md` (only when violations exist) to the Playwright test result, which surfaces in both the HTML and Allure reports. The specs in this suite pass `{ page }` as the third argument, which adds per-node element screenshots (`a11y-<rule>-<n>.png`, up to 5 nodes per violation, excess violations are ignored to reduce redundancy) so each violation has a visual indicator next to its textual description.
90
+
Every assertion via `expectNoAccessibilityViolations` attaches `axe-results.json` (always) and `a11y-violations.md` (only when violations exist) to the Playwright test result, which surfaces in both the HTML and Allure reports. The specs in this suite pass `{ page }` as the third argument, which adds per-node element screenshots (`a11y-<rule>-<n>.png`, up to 5 nodes per violation; additional nodes are skipped to reduce redundancy) so each violation has a visual indicator next to its textual description.
93
91
94
92
The artifacts described above are produced and consumed locally; CI integration is a separate follow-up.
95
93
96
94
### Viewing The Reports
97
95
98
-
End-to-end local workflow: run the suite (which wipes stale output as part of `test:a11y`), then point a report viewer at the output. Yarn scripts are mirrored at both the BHCE root (`bhce/`) and the workspace (`cmd/ui/`); pick whichever cwd is convenient. Examples below default to `bhce/`.
96
+
Local workflow: run the suite, then point a report viewer at the output. Yarn scripts are mirrored at both the BHCE root (`bhce/`) and the workspace (`cmd/ui/`); pick whichever cwd is convenient. Examples below default to `bhce/`.
99
97
100
98
When BHCE is consumed as a submodule of BHE, the BHE workspace can define its own `report:a11y` (e.g. one that aggregates BHCE + BHE allure results into a combined report). The scripts here intentionally cover only BHCE's own suite.
Shows the full test tree, attached files (`axe-results.json`, `a11y-violations.md`, the per-node `a11y-<rule>-<n>.png` screenshots), the failure trace, and the screenshot Playwright captured at failure time. This is the fastest path for "what failed and what did it look like".
115
-
116
-
#### Allure report
100
+
#### Allure Report
117
101
118
102
The Allure reporter only writes raw `*-result.json` files — viewing them requires the `allure` CLI to generate HTML.
119
103
@@ -128,10 +112,8 @@ npm i -g allure-commandline # cross-platform, requires Java on PATH
128
112
**Per-run workflow (from `bhce/`):**
129
113
130
114
```sh
131
-
# 1. Run the suite. The `test:a11y` script first clears the `cmd/ui/playwright/`
132
-
# folder, otherwise the Allure reporter would append across runs, bleeding
133
-
# in stale results. If this step is not run first, user will see the error
134
-
# "does not exist" as the Allure results directory will be empty.
115
+
# 1. If this step is not run first, user will see the error "does not exist"
116
+
# as the Allure results directory will be empty.
135
117
yarn test:a11y
136
118
137
119
# 2a. Ad-hoc: build HTML to a temp dir, serve it, open browser. Ctrl+C cleans up.
Copy file name to clipboardExpand all lines: packages/javascript/bh-playwright-testing/README.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,13 +19,13 @@ Consumers compose those concerns on top of the modules below.
19
19
20
20
The package is consumed via subpath imports so each consumer pulls only what it uses. Each subpath is a thin, focused module — no cross-module coupling beyond shared `Theme` types.
|`bh-playwright-testing`|`test`, `expect`, `WCAG_TAGS`, `attachAxeReport`, `expectNoAccessibilityViolations`, `AttachAxeReportOptions`| Package root entry — the axe-core fixture and reporting helpers, exposed as named exports (no default export). Convenience re-export of `./axe`. |
25
-
|`bh-playwright-testing/axe`| same as above | Same named exports as the package root entry. Use this path when you want to be explicit. |
26
-
|`bh-playwright-testing/themes`|`Theme`, `THEMES`, `TestOptions`, `authStorageStateFor`| Type-only and constant helpers for the per-theme storage-state convention used by `loginAndSnapshotThemes` and Playwright configs. |
27
-
|`bh-playwright-testing/auth`|`loginAndSnapshotThemes`, `LoginAndSnapshotThemesOptions`| One-time login helper that snapshots `storageState` for both light and dark themes from a single session. |
28
-
|`bh-playwright-testing/stubs/graph-has-data`|`installGraphHasDataStub`| Stubs `POST /api/v2/graphs/cypher` so the `useGraphHasData` probe resolves to "true" and the "No Data Available" upload dialog stays closed. |
25
+
|`bh-playwright-testing/axe`| same as above | Same named exports as the package root entry. Use this path when you want to be explicit. |
26
+
|`bh-playwright-testing/themes`|`Theme`, `THEMES`, `TestOptions`, `authStorageStateFor`| Type-only and constant helpers for the per-theme storage-state convention used by `loginAndSnapshotThemes` and Playwright configs. |
27
+
|`bh-playwright-testing/auth`|`loginAndSnapshotThemes`, `LoginAndSnapshotThemesOptions`| One-time login helper that snapshots `storageState` for both light and dark themes from a single session. |
28
+
|`bh-playwright-testing/stubs/graph-has-data`|`installGraphHasDataStub`| Stubs `POST /api/v2/graphs/cypher` so the `useGraphHasData` probe resolves to "true" and the "No Data Available" upload dialog stays closed. |
The package ships TypeScript source via the `exports` map — there is no compiled `dist`. Consumers run it directly through their own Vite/Playwright TS pipelines. This avoids a build step that would only ever be consumed inside the monorepo and keeps the modules editable in place. `tsc --noEmit` (`yarn check-types`, also wired to `yarn lint`) is the only type-check.
158
+
The package ships TypeScript source via the `exports` map — there is no compiled `dist`. Consumers run it directly through their own Vite/Playwright TS pipelines. This avoids a build step that would only ever be consumed inside the monorepo and keeps the modules editable in place. `tsc --noEmit` (`yarn check-types`) is the only type-check.
0 commit comments