Skip to content

Commit 4e01662

Browse files
refactor: Add end-to-end testing and presentation synchronization scripts
- Create `run-e2e-playwright.sh` to automate E2E testing with Playwright. - Add `sync-presentations.sh` to synchronize presentation files and generate a YAML data file. - Introduce initial test results file for tracking E2E test outcomes. - Implement E2E tests for critical site pages in `site-smoke.spec.mjs`. - Add regression tests in `site-regression.test.mjs` to validate homepage content and presentations data integrity.
1 parent 4b5fc03 commit 4e01662

42 files changed

Lines changed: 5090 additions & 393 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 255 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,255 @@
1+
---
2+
name: e2e-testing
3+
description: 'Playwright E2E regression workflow for websites. Use when you want to prevent regressions, secure critical user journeys, and validate that tests themselves still detect failures (anti-drift).'
4+
argument-hint: 'Base URL, critical pages, and risk areas to protect from regression'
5+
---
6+
7+
# E2E Website Regression
8+
9+
Use this skill to build and maintain a regression safety net for your website.
10+
11+
## Outcomes
12+
- Protect critical user journeys from regressions.
13+
- Catch visual, content, navigation, and accessibility breakages early.
14+
- Ensure tests are trustworthy by verifying they fail when expected.
15+
16+
## When to Use
17+
- You changed UI, layout, routing, content, scripts, or dependencies.
18+
- You want a repeatable E2E gate before merge/deploy.
19+
- You need to reduce flaky tests and avoid silent test drift.
20+
21+
## Suggested Structure
22+
23+
```
24+
tests/
25+
├── e2e/
26+
│ ├── smoke/
27+
│ │ └── home-loads.spec.ts
28+
│ ├── critical/
29+
│ │ ├── navigation.spec.ts
30+
│ │ ├── key-content.spec.ts
31+
│ │ └── contact-flow.spec.ts
32+
│ ├── visual/
33+
│ │ └── snapshots.spec.ts
34+
│ └── meta/
35+
│ ├── canary-fail.spec.ts
36+
│ └── anti-drift.spec.ts
37+
├── fixtures/
38+
│ └── routes.ts
39+
└── playwright.config.ts
40+
```
41+
42+
## Workflow
43+
44+
1. Define critical journeys
45+
- List pages and flows that must never break.
46+
- Tag tests with `@critical`, `@smoke`, `@visual`.
47+
48+
2. Build deterministic tests
49+
- Prefer resilient selectors (`getByRole`, `getByTestId`).
50+
- Avoid arbitrary sleeps; wait for explicit conditions.
51+
52+
3. Add regression assertions
53+
- URL and title checks.
54+
- Visibility and content checks on key sections.
55+
- Broken-link check on major pages.
56+
- Accessibility smoke checks for critical routes.
57+
58+
4. Validate test integrity (test the tests)
59+
- Keep one intentional failing canary in `tests/e2e/meta`.
60+
- Run anti-drift checks to ensure selectors/assertions are meaningful.
61+
- Repeat critical tests to detect flakiness.
62+
63+
5. Gate CI with clear pass/fail rules
64+
- Block merge on `@smoke` + `@critical` failures.
65+
- Keep visual and extended suites as required or scheduled.
66+
67+
## Example Critical Test
68+
69+
```typescript
70+
import { Page, Locator } from '@playwright/test'
71+
72+
export class HomePage {
73+
readonly page: Page
74+
readonly heroTitle: Locator
75+
readonly nav: Locator
76+
77+
constructor(page: Page) {
78+
this.page = page
79+
this.heroTitle = page.getByRole('heading', { level: 1 })
80+
this.nav = page.getByRole('navigation')
81+
}
82+
83+
async goto() {
84+
await this.page.goto('/')
85+
await this.page.waitForLoadState('networkidle')
86+
}
87+
}
88+
```
89+
90+
```typescript
91+
import { test, expect } from '@playwright/test'
92+
93+
test.describe('@critical home regression', () => {
94+
test('loads homepage and renders key content', async ({ page }) => {
95+
await page.goto('/')
96+
await expect(page).toHaveURL(/\/$/)
97+
await expect(page.getByRole('navigation')).toBeVisible()
98+
await expect(page.getByRole('heading', { level: 1 })).toBeVisible()
99+
})
100+
})
101+
```
102+
103+
## Test the Tests (Anti-Drift)
104+
105+
### 1) Canary Failure
106+
107+
Create one test that must fail. CI should verify it actually fails.
108+
109+
```typescript
110+
import { test, expect } from '@playwright/test'
111+
112+
test('@meta canary should fail', async () => {
113+
expect(1).toBe(2)
114+
})
115+
```
116+
117+
CI check pattern:
118+
119+
```bash
120+
npx playwright test tests/e2e/meta/canary-fail.spec.ts && exit 1 || true
121+
```
122+
123+
### 2) Selector Health Check
124+
125+
Add a meta test that checks key selectors exist on critical pages. If a selector disappears silently, this test fails immediately.
126+
127+
```typescript
128+
import { test, expect } from '@playwright/test'
129+
130+
test('@meta critical selectors exist', async ({ page }) => {
131+
await page.goto('/')
132+
await expect(page.getByRole('navigation')).toBeVisible()
133+
await expect(page.getByRole('heading', { level: 1 })).toBeVisible()
134+
})
135+
```
136+
137+
### 3) Flakiness Probe
138+
139+
```bash
140+
npx playwright test --grep @critical --repeat-each=5 --retries=0
141+
```
142+
143+
Treat inconsistent runs as a quality issue to fix before merge.
144+
145+
## Playwright Configuration
146+
147+
```typescript
148+
import { defineConfig, devices } from '@playwright/test'
149+
150+
export default defineConfig({
151+
testDir: './tests/e2e',
152+
fullyParallel: true,
153+
forbidOnly: !!process.env.CI,
154+
retries: process.env.CI ? 2 : 0,
155+
workers: process.env.CI ? 1 : undefined,
156+
reporter: [
157+
['html', { outputFolder: 'playwright-report' }],
158+
['junit', { outputFile: 'playwright-results.xml' }],
159+
['json', { outputFile: 'playwright-results.json' }]
160+
],
161+
use: {
162+
baseURL: process.env.BASE_URL || 'http://localhost:3000',
163+
trace: 'on-first-retry',
164+
screenshot: 'only-on-failure',
165+
video: 'retain-on-failure',
166+
actionTimeout: 10000,
167+
navigationTimeout: 30000,
168+
},
169+
projects: [
170+
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
171+
{ name: 'firefox', use: { ...devices['Desktop Firefox'] } },
172+
{ name: 'webkit', use: { ...devices['Desktop Safari'] } },
173+
{ name: 'mobile-chrome', use: { ...devices['Pixel 5'] } },
174+
],
175+
webServer: {
176+
command: 'npm run dev',
177+
url: 'http://localhost:3000',
178+
reuseExistingServer: !process.env.CI,
179+
timeout: 120000,
180+
},
181+
})
182+
```
183+
184+
## CI Gate Example
185+
186+
```yaml
187+
name: E2E Regression
188+
on: [push, pull_request]
189+
190+
jobs:
191+
e2e:
192+
runs-on: ubuntu-latest
193+
steps:
194+
- uses: actions/checkout@v4
195+
- uses: actions/setup-node@v4
196+
with:
197+
node-version: 20
198+
- run: npm ci
199+
- run: npx playwright install --with-deps
200+
- run: npx playwright test --grep @smoke
201+
- run: npx playwright test --grep @critical
202+
- run: npx playwright test tests/e2e/meta/anti-drift.spec.ts
203+
- run: npx playwright test tests/e2e/meta/canary-fail.spec.ts && exit 1 || true
204+
- uses: actions/upload-artifact@v4
205+
if: always()
206+
with:
207+
name: playwright-report
208+
path: playwright-report/
209+
```
210+
211+
## Quality Checklist
212+
213+
- Every critical flow has at least one `@critical` test.
214+
- No `waitForTimeout` in critical tests.
215+
- All selectors in critical tests are resilient.
216+
- Anti-drift meta tests are enabled in CI.
217+
- Canary failure step proves CI catches failing tests.
218+
- Flakiness probe is run periodically.
219+
220+
## Common Failure Patterns and Fixes
221+
222+
- Race condition: switch from raw `click` to locator-based interactions.
223+
- Network timing: wait on explicit response or stable UI state.
224+
- Animation instability: wait for visible and stable elements before actions.
225+
- Over-broad assertions: assert concrete business-visible outcomes.
226+
227+
## Report Template
228+
229+
```markdown
230+
# E2E Regression Report
231+
232+
**Date:** YYYY-MM-DD HH:MM
233+
**Duration:** Xm Ys
234+
**Status:** PASSING / FAILING
235+
236+
## Summary
237+
- Smoke: X passed / Y failed
238+
- Critical: X passed / Y failed
239+
- Meta anti-drift: PASS / FAIL
240+
- Canary fail check: PASS / FAIL
241+
242+
## Failed Tests
243+
244+
### test-name
245+
**File:** `tests/e2e/feature.spec.ts:45`
246+
**Error:** Expected element to be visible
247+
**Screenshot:** artifacts/failed.png
248+
**Recommended Fix:** [description]
249+
250+
## Artifacts
251+
- HTML Report: playwright-report/index.html
252+
- Screenshots: artifacts/*.png
253+
- Videos: artifacts/videos/*.webm
254+
- Traces: artifacts/*.zip
255+
```

0 commit comments

Comments
 (0)