Skip to content

Commit d7317c7

Browse files
github-actions[bot]GitHub CopilotCopilotdsyme
authored
Add Multi-Device Docs Tester workflow (#232)
Adds a generalised version of the multi-device documentation testing workflow from Peli's Agent Factory. Builds and serves the project's documentation site locally, then runs Playwright tests across mobile, tablet, and desktop viewports to catch responsive layout issues, broken navigation, and accessibility problems. Co-authored-by: GitHub Copilot <copilot@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Don Syme <dsyme@users.noreply.github.com>
1 parent 10f0876 commit d7317c7

3 files changed

Lines changed: 342 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ These workflows analyze the repository, code, and activity to produce reports, i
4949
You can use the "/plan" agent to turn the reports into actionable issues which can then be assigned to the appropriate team members or agents.
5050

5151
- [🔍 Daily Accessibility Review](docs/daily-accessibility-review.md) - Review application accessibility by automatically running and using the application
52+
- [📱 Multi-Device Docs Tester](docs/daily-multi-device-docs-tester.md) - Test documentation sites across mobile, tablet, and desktop viewports for responsive layout and interaction issues
5253
- [🔎 Daily Adhoc QA](docs/daily-qa.md) - Perform adhoc explorative quality assurance tasks
5354

5455
### Code Improvement Workflows (by making changes, producing pull requests)

‎docs/daily-multi-device-docs-tester.md‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# 📱 Multi-Device Docs Tester
2+
3+
> For an overview of all available workflows, see the [main README](../README.md).
4+
5+
**Build and test your documentation site across mobile, tablet, and desktop devices to catch responsive design issues before they reach users**
6+
7+
The [Multi-Device Docs Tester workflow](../workflows/daily-multi-device-docs-tester.md?plain=1) builds your documentation site locally, serves it, and runs Playwright-powered tests across a range of device viewports. It checks for layout problems, inaccessible navigation, overflowing content, and broken interactive elements — then creates a GitHub issue with a detailed report when problems are found.
8+
9+
## Installation
10+
11+
```bash
12+
# Install the 'gh aw' extension
13+
gh extension install github/gh-aw
14+
15+
# Add the workflow to your repository
16+
gh aw add-wizard githubnext/agentics/daily-multi-device-docs-tester
17+
```
18+
19+
This walks you through adding the workflow to your repository.
20+
21+
## How It Works
22+
23+
```mermaid
24+
graph LR
25+
A[Build Docs Site] --> B[Start Preview Server]
26+
B --> C[Test Each Device]
27+
C --> D{Issues Found?}
28+
D -->|Yes| E[Create Issue Report]
29+
D -->|No| F[Noop: All Passed]
30+
```
31+
32+
The workflow builds your docs site using npm, starts a local preview server, and runs Playwright browser automation across mobile (390–393 px), tablet (768–834 px), and desktop (1366–1920 px) viewports. For each device it checks page load, navigation usability, content readability, image sizing, interactive element reachability, and basic accessibility.
33+
34+
## Requirements
35+
36+
Your repository must have a documentation site that:
37+
38+
- Lives in a subdirectory (default: `docs/`)
39+
- Has a `package.json` with a `build` script and a `preview` (or equivalent serve) script
40+
- Serves on a local port when running the preview command
41+
42+
Common frameworks that work out of the box include [Astro Starlight](https://starlight.astro.build/), [Docusaurus](https://docusaurus.io/), [VitePress](https://vitepress.dev/), and similar npm-based documentation tools.
43+
44+
## Usage
45+
46+
### Configuration
47+
48+
The workflow can be customised via `workflow_dispatch` inputs:
49+
50+
| Input | Default | Description |
51+
|-------|---------|-------------|
52+
| `devices` | `mobile,tablet,desktop` | Comma-separated list of device types to test |
53+
| `docs_dir` | `docs` | Directory containing the documentation site |
54+
| `build_command` | `npm run build` | Command to build the site |
55+
| `serve_command` | `npm run preview` | Command to serve the built site |
56+
| `server_port` | `4321` | Port the local server listens on |
57+
58+
After editing run `gh aw compile` to update the workflow and commit all changes to the default branch.
59+
60+
### Commands
61+
62+
You can start a run of this workflow immediately by running:
63+
64+
```bash
65+
gh aw run daily-multi-device-docs-tester
66+
```
67+
68+
Or trigger it from the GitHub Actions tab using workflow dispatch to customise which devices to test.
69+
70+
### When Issues Are Found
71+
72+
The workflow creates a GitHub issue titled **📱 Multi-Device Docs Testing Report** containing:
73+
74+
- A summary table (passed / warnings / critical per device)
75+
- A visible list of critical issues that block functionality
76+
- Collapsible sections with per-device details and warning listings
77+
- Accessibility findings
78+
- Actionable recommendations
79+
80+
Issues expire after 2 days so the tracker stays clean as problems are fixed.

‎workflows/daily-multi-device-docs-tester.md‎

Lines changed: 261 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,261 @@
1+
---
2+
name: Multi-Device Docs Tester
3+
description: Tests a documentation site for responsive layout issues, accessibility problems, and broken interactions across mobile, tablet, and desktop device form factors
4+
on:
5+
schedule: daily
6+
workflow_dispatch:
7+
inputs:
8+
devices:
9+
description: 'Device types to test (comma-separated: mobile,tablet,desktop)'
10+
required: false
11+
default: 'mobile,tablet,desktop'
12+
docs_dir:
13+
description: 'Directory containing the documentation site (relative to repository root)'
14+
required: false
15+
default: 'docs'
16+
build_command:
17+
description: 'Command to build the documentation site'
18+
required: false
19+
default: 'npm run build'
20+
serve_command:
21+
description: 'Command to serve the built documentation site'
22+
required: false
23+
default: 'npm run preview'
24+
server_port:
25+
description: 'Port the documentation server listens on'
26+
required: false
27+
default: '4321'
28+
permissions:
29+
contents: read
30+
issues: read
31+
pull-requests: read
32+
tracker-id: daily-multi-device-docs-tester
33+
engine:
34+
id: claude
35+
max-turns: 30
36+
strict: true
37+
timeout-minutes: 30
38+
network:
39+
allowed:
40+
- defaults
41+
- node
42+
tools:
43+
playwright:
44+
version: "v1.56.1"
45+
bash:
46+
- "npm install*"
47+
- "npm run build*"
48+
- "npm run preview*"
49+
- "npm run start*"
50+
- "npm run serve*"
51+
- "npx playwright*"
52+
- "curl*"
53+
- "kill*"
54+
- "lsof*"
55+
- "ls*"
56+
- "pwd*"
57+
- "cat*"
58+
- "echo*"
59+
- "sleep*"
60+
safe-outputs:
61+
upload-asset:
62+
create-issue:
63+
expires: 2d
64+
labels: [documentation, testing]
65+
imports:
66+
- shared/reporting.md
67+
---
68+
69+
# Multi-Device Documentation Testing
70+
71+
You are a documentation testing specialist. Your task is to build the project's documentation site and test it across multiple device form factors to catch responsive design issues, accessibility problems, and broken interactions before they reach users.
72+
73+
## Context
74+
75+
- **Repository**: ${{ github.repository }}
76+
- **Run ID**: ${{ github.run_id }}
77+
- **Triggered by**: @${{ github.actor }}
78+
- **Devices to test**: ${{ inputs.devices || 'mobile,tablet,desktop' }}
79+
- **Docs directory**: ${{ inputs.docs_dir || 'docs' }}
80+
- **Build command**: ${{ inputs.build_command || 'npm run build' }}
81+
- **Serve command**: ${{ inputs.serve_command || 'npm run preview' }}
82+
- **Server port**: ${{ inputs.server_port || '4321' }}
83+
- **Working directory**: ${{ github.workspace }}
84+
85+
## Step 1: Verify the Documentation Site Exists
86+
87+
Check that the documentation directory exists and has a package.json:
88+
89+
```bash
90+
ls -la ${{ github.workspace }}/${{ inputs.docs_dir || 'docs' }}/
91+
cat ${{ github.workspace }}/${{ inputs.docs_dir || 'docs' }}/package.json 2>/dev/null | head -20 || echo "No package.json found"
92+
```
93+
94+
If the docs directory doesn't exist or has no package.json, call the `noop` safe output explaining that this repository doesn't have a buildable documentation site and stop.
95+
96+
## Step 2: Build the Documentation Site
97+
98+
Navigate to the docs directory and build the site:
99+
100+
```bash
101+
cd ${{ github.workspace }}/${{ inputs.docs_dir || 'docs' }}
102+
npm install
103+
${{ inputs.build_command || 'npm run build' }}
104+
```
105+
106+
If the build fails, create a GitHub issue titled "📱 Multi-Device Docs Test Failed - Build Error" with the error details and stop.
107+
108+
## Step 3: Start the Preview Server
109+
110+
Start the preview server in the background and wait for it to be ready:
111+
112+
```bash
113+
cd ${{ github.workspace }}/${{ inputs.docs_dir || 'docs' }}
114+
${{ inputs.serve_command || 'npm run preview' }} > /tmp/docs-preview.log 2>&1 &
115+
echo $! > /tmp/docs-server.pid
116+
echo "Server started with PID: $(cat /tmp/docs-server.pid)"
117+
```
118+
119+
Wait for the server to be ready:
120+
121+
```bash
122+
PORT=${{ inputs.server_port || '4321' }}
123+
for i in {1..30}; do
124+
curl -s http://localhost:$PORT > /dev/null && echo "Server ready on port $PORT!" && break
125+
echo "Waiting for server... ($i/30)" && sleep 2
126+
done
127+
curl -s http://localhost:$PORT > /dev/null || echo "WARNING: Server may not have started properly"
128+
```
129+
130+
## Step 4: Device Configuration
131+
132+
Use these viewport sizes based on the `${{ inputs.devices || 'mobile,tablet,desktop' }}` input:
133+
134+
**Mobile devices** (test if "mobile" in input):
135+
- iPhone 12: 390×844
136+
- Pixel 5: 393×851
137+
- Galaxy S21: 360×800
138+
139+
**Tablet devices** (test if "tablet" in input):
140+
- iPad: 768×1024
141+
- iPad Pro 11": 834×1194
142+
143+
**Desktop devices** (test if "desktop" in input):
144+
- HD: 1366×768
145+
- FHD: 1920×1080
146+
147+
## Step 5: Run Playwright Tests
148+
149+
**IMPORTANT: Use Playwright via MCP tools only — do NOT install or require Playwright as an npm package.**
150+
151+
Use Playwright MCP tools (e.g., `mcp__playwright__browser_navigate`, `mcp__playwright__browser_run_code`, `mcp__playwright__browser_snapshot`) to test the documentation site.
152+
153+
For **each device viewport** in the requested device types, perform the following checks:
154+
155+
```javascript
156+
// Example: set viewport, navigate, snapshot
157+
mcp__playwright__browser_run_code({
158+
code: `async (page) => {
159+
await page.setViewportSize({ width: 390, height: 844 });
160+
await page.goto('http://localhost:${{ inputs.server_port || '4321' }}/');
161+
return { url: page.url(), title: await page.title() };
162+
}`
163+
})
164+
```
165+
166+
For each device, check:
167+
1. **Page loads** successfully (no 404, 500 errors)
168+
2. **Navigation** is usable (menu accessible, links work)
169+
3. **Content** is readable without horizontal scrolling
170+
4. **Images** are properly sized and not overflowing
171+
5. **Interactive elements** (search, buttons, tabs) are reachable and tappable
172+
6. **Text** is not truncated or overlapping
173+
7. **Accessibility** basics: headings present, alt text on images, sufficient contrast
174+
175+
Take screenshots on failure for evidence. Use `upload-asset` safe output to store screenshots.
176+
177+
## Step 6: Analyze Results
178+
179+
Categorize findings by severity:
180+
- 🔴 **Critical**: Blocks navigation or makes content unreadable
181+
- 🟡 **Warning**: Layout issues that degrade experience but don't block content
182+
- 🟢 **Passed**: Device renders correctly
183+
184+
## Step 7: Stop the Preview Server
185+
186+
Always clean up when done:
187+
188+
```bash
189+
kill $(cat /tmp/docs-server.pid) 2>/dev/null || true
190+
rm -f /tmp/docs-server.pid /tmp/docs-preview.log
191+
echo "Server stopped"
192+
```
193+
194+
## Step 8: Report Results
195+
196+
### If NO Issues Found
197+
198+
Call the `noop` safe output to log completion:
199+
200+
```json
201+
{
202+
"noop": {
203+
"message": "Multi-device documentation testing complete. All devices tested successfully with no issues found."
204+
}
205+
}
206+
```
207+
208+
**You MUST invoke the noop tool — do not just write this message as text.**
209+
210+
### If Issues ARE Found
211+
212+
Create a GitHub issue titled "📱 Multi-Device Docs Testing Report - [Date]" with:
213+
214+
```markdown
215+
### Test Summary
216+
- Triggered by: @${{ github.actor }}
217+
- Workflow run: [§${{ github.run_id }}](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
218+
- Devices tested: {count}
219+
- Test date: {date}
220+
221+
### Results Overview
222+
- 🟢 Passed: {count}
223+
- 🟡 Warnings: {count}
224+
- 🔴 Critical: {count}
225+
226+
### Critical Issues
227+
[List issues that block functionality or readability — keep visible]
228+
229+
<details>
230+
<summary><b>View All Warnings</b></summary>
231+
232+
[Minor layout and UX issues with device names and details]
233+
234+
</details>
235+
236+
<details>
237+
<summary><b>View Detailed Test Results by Device</b></summary>
238+
239+
#### Mobile Devices
240+
[Test results per device]
241+
242+
#### Tablet Devices
243+
[Test results per device]
244+
245+
#### Desktop Devices
246+
[Test results per device]
247+
248+
</details>
249+
250+
### Accessibility Findings
251+
[Key accessibility issues — keep visible as they are important]
252+
253+
### Recommendations
254+
[Actionable steps to fix the issues found]
255+
```
256+
257+
**Important**: If no action is needed after completing your analysis, you **MUST** call the `noop` safe-output tool with a brief explanation. Failing to call any safe-output tool is the most common cause of workflow failures.
258+
259+
```json
260+
{"noop": {"message": "No action needed: [brief explanation of what was analyzed and why no action was required]"}}
261+
```

0 commit comments

Comments
 (0)