Skip to content

test: UI testing framework — Jest regression specs + Playwright smoke harness#487

Merged
nicdavidson merged 4 commits intodevelopfrom
ui-testing-framework
Apr 22, 2026
Merged

test: UI testing framework — Jest regression specs + Playwright smoke harness#487
nicdavidson merged 4 commits intodevelopfrom
ui-testing-framework

Conversation

@nicdavidson
Copy link
Copy Markdown
Contributor

@nicdavidson nicdavidson commented Apr 22, 2026

Why

Every bug fixed during the 2026-04-22 customer fire-drill (Triskele: ace-builds 404, case-interceptor mangling event names, dropdown empty for underscored services, stuck loading spinner, 400 "No record(s) detected" on save) could have been caught by automated tests that don't exist today. This branch puts the plumbing in place so the next one doesn't reach a customer.

What

Tier 1 — Jest in CI

  • .github/workflows/node.js.yml now runs npm run lint, npm run test:ci, npm run build (it only ran build before).
  • jest.config.ci.js scopes CI to the subset of specs that currently pass. The rest of the 70-suite inventory fails because @ngneat/transloco ships ESM-only and jest-preset-angular@13 in this repo can't resolve it without an ESM migration — that migration is a separate tracked effort.
  • Regression specs for today's fixes:
    • df-loading-spinner.service.spec.ts — rapid toggle race settles to inactive.
    • case.interceptor.spec.ts — /system/event responses passthrough; /api_docs still exempt; body transforms still convert camel→snake.
    • df-script-details.submit.spec.ts — submit uses || (not ??); all three mat-select (selectionChange) bindings present; raw response[serviceName] lookup, no api_docs → apiDocs rename reintroduced.
  • Added "node" to tsconfig.spec.json types so specs can readFileSync the committed templates for contract assertions.

Tier 2 — Playwright scaffolding

  • @playwright/test@1.59 installed (chromium only, --with-deps in CI).
  • playwright.config.ts — PLAYWRIGHT_BASE_URL env var drives the target; trace/screenshot/video retained on failure.
  • e2e/fixtures/admin-login.ts — UI-driven login (localStorage seed doesn't work; app hydrates from a cookie).
  • e2e/smoke.spec.ts — PASSING. Login + app shell paint, no raw transloco key leakage (nav.ai.nav), ace-builds asset 200s, top-nav entries navigate without 5xx.
  • e2e/event-scripts.spec.ts, api-connections.spec.ts, roles.spec.ts, mcp-service.spec.ts — test.fixme scaffolds with full intent comments. Follow-up work needs a stable sidenav selector pattern; full logic for the event-script journey is inlined so finishing it is edit-and-unfixme.
  • .github/workflows/e2e.yml — runs on push/PR to develop against a spun-up df-docker-dev stack, plus nightly cron (override URL via E2E_NIGHTLY_URL secret), plus manual dispatch.
  • Scripts: npm run e2e, e2e:ui, e2e:smoke.

Test plan

  • npm run test:ci passes locally (37/37).
  • npm run e2e:smoke passes locally against http://localhost:8080 (df-docker-dev).
  • CI run on this PR is green.
  • Follow-up: finish the four test.fixme journey specs.
  • Follow-up: Jest ESM migration to un-quarantine the remaining 55 suites.

Not in scope (tracked separately)

  • Tier 3 visual regression (Playwright toHaveScreenshot), backend contract tests (PHPUnit), pre-merge hook wiring.

@nicdavidson
test(ci): wire jest into CI with regression specs for 2026-04-22 fixes
3d6bb50 
Previously the only CI check was `npm run build`, which type-checks but
never runs tests. This commit activates jest in CI via a scoped
`jest.config.ci.js` so the suite passes today while the broader ESM /
@ngneat/transloco migration is tackled separately.

Adds regression specs for each bug hit on the customer fire-drill:

* df-loading-spinner.service.spec.ts — rapid toggles inside one tick now
  settle to false (was stuck-on due to stale BehaviorSubject read).
* case.interceptor.spec.ts — /system/event responses pass through
  unchanged; /api_docs exemption still works; body request transforms
  still convert camelCase → snake_case.
* df-script-details.submit.spec.ts — submit fallback uses `||` (not
  `??`) so empty completeScriptName falls back to selectedRouteItem;
  template wiring verified for all three mat-selects; raw serviceName
  lookup (no re-introduction of the api_docs → apiDocs rename).

Also adds "node" to tsconfig.spec.json types so specs can `readFileSync`
the committed HTML/TS for contract assertions.
@nicdavidson
test(e2e): scaffold Playwright with smoke specs + CI workflow
52f2c84 
Adds a Playwright harness so user-journey regressions can be caught in
CI instead of on customer calls. This is the second tier of the
ui-testing-framework work.

New:
* playwright.config.ts — chromium-only, baseURL via PLAYWRIGHT_BASE_URL,
  trace/screenshot/video retained on failure, 30s timeout. Default target
  is http://localhost:8080 (df-docker-dev); CI can override for a nightly
  canary against crucible.
* e2e/fixtures/admin-login.ts — UI-based login helper. (localStorage seed
  does not work — the app hydrates userData from a cookie + the /session
  roundtrip.)
* e2e/smoke.spec.ts — PASSING. Covers: login + app shell paint, no
  uncaught JS exceptions, no raw transloco keys leaking (nav.ai.nav),
  ace-builds asset 200s, top-nav entries navigate without 5xx.
* e2e/event-scripts.spec.ts — `test.fixme`. Full intent is laid out;
  selector work around lazy-loaded routes via hash navigation needs
  follow-up.
* e2e/api-connections.spec.ts, e2e/roles.spec.ts, e2e/mcp-service.spec.ts
  — `test.fixme` stubs with intent comments.
* .github/workflows/e2e.yml — runs on push/PR to develop against a
  spun-up df-docker-dev stack, plus nightly cron against a configurable
  URL, plus manual dispatch.
* npm scripts: e2e, e2e:ui, e2e:smoke.
* jest.config.js: testPathIgnorePatterns exclude e2e/ so Jest doesn't
  try to run Playwright specs.
@nicdavidson
ci: drop pre-existing lint gate + scope E2E to manual/nightly only
c6ea71c 
Two CI issues surfaced on PR #487 opening the ui-testing-framework
branch, both self-inflicted:

1. `npm run lint` exposed 47 errors + 360 warnings of pre-existing tech
   debt. The PR's scope is adding tests, not landing a lint cleanup;
   gating CI on it blocks this and any other PR until that cleanup
   happens. Dropping the step for now. A proper lint re-enablement can
   land on its own PR.

2. The E2E job tried to check out `dreamfactorysoftware/df-docker-dev`
   which does not exist on the public org — that's a local-only dev
   stack. Without a shared containerized DreamFactory image the E2E
   can't provision its own backend. Switching the workflow to
   manual-dispatch + nightly-cron-only until that infra lands; the
   smoke specs still work fine when pointed at a live instance via
   PLAYWRIGHT_BASE_URL.
@nicdavidson
test(e2e): add discovery spec — reports nav automation limits
e8e77f3 
Not a regression gate. Runs five journeys (event-scripts, api-db,
roles, admins, mcp) and reports what breaks. First run surfaces one
cross-cutting finding more than any per-journey bug: the admin sidebar
is resistant to standard Playwright selectors.

Concrete observations from the first run (see console output):
- clicking any sidebar nav-item button (even with force:true) lands
  at /home instead of the target route. Suggests (a) the click event
  isn't reaching the Angular router handler, (b) navigation happens
  but a guard / welcome flow redirects, or (c) click falls through to
  a nearby element (the "Admins" click landed at /ai, which sits close
  by in the DOM).
- no clickable [routerLink] or href on the nav-item elements; Angular
  binds via (click) on <mat-list-item> wrappers.
- actionability without force: true always times out.

Fixing this right likely needs data-testid attributes on nav items
or a dedicated NavPage object that understands the accordion state.
Parking for now — ships the spec as a runnable record of the
automation gap.
@nicdavidson nicdavidson mentioned this pull request Apr 22, 2026
Merged
3 tasks
@nicdavidson nicdavidson merged commit 3f8c62a into develop Apr 22, 2026
1 check passed
nicdavidson added a commit that referenced this pull request Apr 22, 2026
@nicdavidson
ci: drop pre-existing lint gate + scope E2E to manual/nightly only
be00ec4 
Two CI issues surfaced on PR #487 opening the ui-testing-framework
branch, both self-inflicted:

1. `npm run lint` exposed 47 errors + 360 warnings of pre-existing tech
   debt. The PR's scope is adding tests, not landing a lint cleanup;
   gating CI on it blocks this and any other PR until that cleanup
   happens. Dropping the step for now. A proper lint re-enablement can
   land on its own PR.

2. The E2E job tried to check out `dreamfactorysoftware/df-docker-dev`
   which does not exist on the public org — that's a local-only dev
   stack. Without a shared containerized DreamFactory image the E2E
   can't provision its own backend. Switching the workflow to
   manual-dispatch + nightly-cron-only until that infra lands; the
   smoke specs still work fine when pointed at a live instance via
   PLAYWRIGHT_BASE_URL.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@nicdavidson