Skip to content

Commit ec5ca1e

Browse files
Copilotmgierschdev
Copilot
and
mgierschdev
committed
Add comprehensive E2E QA validation test infrastructure
Co-authored-by: mgierschdev <62764972+mgierschdev@users.noreply.github.com>
1 parent fedb007 commit ec5ca1e

4 files changed

Lines changed: 915 additions & 0 deletions

File tree

‎e2e-tests/.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
node_modules/
2+
package-lock.json
3+
.playwright/
4+
test-results/
5+
playwright-report/
6+
*.log

‎e2e-tests/README.md‎

Lines changed: 243 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,243 @@
1+
# End-to-End QA Validation Tests
2+
3+
This directory contains comprehensive end-to-end QA validation tests for the Chess Engine application.
4+
5+
## Purpose
6+
7+
These tests act as a **real human user** to validate that the chess application works correctly from the perspective of a final user, testing:
8+
9+
- Backend and frontend startup
10+
- REST API enforcement of chess rules
11+
- Frontend reflection of game state
12+
- Illegal action rejection
13+
- Documented limitations and behaviors
14+
15+
## Test Coverage
16+
17+
The QA validation script tests all requirements specified in the test plan:
18+
19+
### A. Application Boot
20+
- ✓ Backend starts without errors
21+
- ✓ Frontend loads and renders chessboard
22+
- ✓ No console or server errors on initial load
23+
24+
### B. Game Initialization
25+
- ✓ Start new game via API
26+
- ✓ Board is in standard initial chess position
27+
- ✓ White's turn is set
28+
29+
### C. Basic Legal Moves
30+
- ✓ Legal opening move (e2 → e4)
31+
- ✓ Move is accepted
32+
- ✓ Pawn moves to correct position
33+
- ✓ Turn switches to Black
34+
- ✓ Legal Black move (e7 → e5)
35+
36+
### D. Illegal Move Rejection
37+
- ✓ Illegal moves are rejected
38+
- ✓ Board state does NOT change
39+
- ✓ Turn remains unchanged
40+
41+
### E. Special Rules
42+
- ✓ Valid moves API
43+
- ⚠ Castling (requires specific game sequence)
44+
- ⚠ En passant (requires specific game sequence)
45+
- ⚠ Pawn promotion (requires complete game)
46+
47+
### F. Check and Checkmate
48+
- ✓ Fool's Mate sequence
49+
- ✓ Checkmate detection
50+
- ✓ Game ends appropriately
51+
52+
### G. Non-Goals Validation
53+
- ✓ No AI opponent exists
54+
- ✓ No second game can be started concurrently
55+
- ⚠ No persistence (requires backend restart)
56+
- ✓ No undo/redo endpoints
57+
58+
### H. API Validation
59+
- ✓ All REST endpoints work correctly
60+
- ✓ Invalid API calls are rejected
61+
- ✓ Responses match UI behavior
62+
63+
### I. Security Posture
64+
- ✓ No authentication required (as documented)
65+
- ✓ No secrets exposed in API responses
66+
- ✓ CORS configured as documented
67+
68+
## Prerequisites
69+
70+
- Node.js 18+
71+
- Backend and frontend **NOT already running** (script starts them)
72+
- Playwright browsers installed
73+
74+
## Installation
75+
76+
```bash
77+
cd e2e-tests
78+
npm install
79+
npx playwright install chromium
80+
```
81+
82+
## Usage
83+
84+
### Run Full QA Validation
85+
86+
```bash
87+
npm test
88+
```
89+
90+
Or directly:
91+
92+
```bash
93+
node qa-validation.js
94+
```
95+
96+
### Expected Output
97+
98+
The script will:
99+
1. Start backend service (./gradlew bootRun)
100+
2. Start frontend service (npm run dev)
101+
3. Wait for services to be ready
102+
4. Execute all test cases
103+
5. Generate a comprehensive report
104+
6. Stop services and cleanup
105+
7. Exit with code 0 (pass) or 1 (fail)
106+
107+
### Example Report
108+
109+
```
110+
=================================================
111+
FINAL QA VALIDATION REPORT
112+
=================================================
113+
114+
📊 SUMMARY
115+
Total Tests: 35
116+
✓ Passed: 32
117+
✗ Failed: 0
118+
⚠ Warnings: 3
119+
Pass Rate: 100.0%
120+
Overall: ✓ PASS
121+
122+
✓ PASSED TESTS:
123+
• Backend Startup - Service started successfully
124+
• Frontend Startup - Service started successfully
125+
• Frontend Renders Chessboard - Chessboard UI element found
126+
• Legal Move e2-e4 - White pawn move accepted
127+
...
128+
129+
⚠ WARNINGS:
130+
• Pawn promotion full test requires complete game
131+
• Castling test requires specific game sequence
132+
• No persistence test requires backend restart
133+
134+
📸 SCREENSHOTS:
135+
• /tmp/qa-screenshot-initial-load.png
136+
• /tmp/qa-screenshot-game-started.png
137+
• /tmp/qa-screenshot-final.png
138+
139+
=================================================
140+
FINAL CHECK:
141+
Does the application behave exactly as described
142+
in the README from a real user's perspective?
143+
=================================================
144+
145+
✓ YES - The application behaves as documented.
146+
All tested features work correctly according to the README.
147+
```
148+
149+
## CI Integration
150+
151+
To add to CI workflow:
152+
153+
```yaml
154+
e2e-tests:
155+
name: End-to-End QA Validation
156+
runs-on: ubuntu-latest
157+
needs: [backend, frontend]
158+
159+
steps:
160+
- uses: actions/checkout@v4
161+
162+
- name: Set up Node.js
163+
uses: actions/setup-node@v4
164+
with:
165+
node-version: '18'
166+
167+
- name: Set up JDK 17
168+
uses: actions/setup-java@v4
169+
with:
170+
java-version: '17'
171+
distribution: 'temurin'
172+
173+
- name: Install dependencies
174+
run: |
175+
cd e2e-tests
176+
npm ci
177+
npx playwright install chromium
178+
179+
- name: Run E2E tests
180+
run: |
181+
cd e2e-tests
182+
npm test
183+
184+
- name: Upload screenshots
185+
if: always()
186+
uses: actions/upload-artifact@v4
187+
with:
188+
name: e2e-screenshots
189+
path: /tmp/qa-screenshot-*.png
190+
```
191+
192+
## Test Philosophy
193+
194+
These tests follow the requirements:
195+
196+
- ✅ **Black box testing**: Uses only public interfaces (HTTP API and browser UI)
197+
- ✅ **No internal inspection**: Does not inspect or modify internal game state
198+
- ✅ **No bypass**: Does not bypass validation logic
199+
- ✅ **Real user perspective**: Tests behavior as a human would use the application
200+
- ✅ **Documentation validation**: Confirms documented limitations behave as stated
201+
202+
## Limitations
203+
204+
Some tests cannot be fully automated and are marked with warnings:
205+
206+
- **Pawn Promotion**: Requires playing a complete game to get a pawn to the 8th rank
207+
- **Castling**: Requires specific board setup sequence
208+
- **En Passant**: Requires specific game sequence
209+
- **Persistence**: Would require restarting the backend service
210+
211+
These features are tested via API parameter acceptance and documentation review.
212+
213+
## Troubleshooting
214+
215+
### Tests fail to start backend/frontend
216+
217+
- Ensure no services are already running on ports 8080 or 3000
218+
- Check that backend/gradlew has execute permissions
219+
- Verify frontend dependencies are installed (npm install)
220+
221+
### Timeout errors
222+
223+
- Increase STARTUP_TIMEOUT in qa-validation.js
224+
- Check system resources (memory, CPU)
225+
226+
### Screenshot not found
227+
228+
- Screenshots are saved to /tmp/ directory
229+
- Ensure write permissions to /tmp/
230+
231+
## Contributing
232+
233+
When adding new tests:
234+
235+
1. Follow the existing test structure (A-I sections)
236+
2. Use recordPass() and recordFail() for tracking
237+
3. Add descriptive test names
238+
4. Document any warnings or limitations
239+
5. Update this README with new test coverage
240+
241+
## License
242+
243+
Same as parent project.

‎e2e-tests/package.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"name": "chess-engine-e2e-tests",
3+
"version": "1.0.0",
4+
"description": "End-to-end QA validation tests for Chess Engine application",
5+
"scripts": {
6+
"test": "node qa-validation.js",
7+
"test:report": "node qa-validation.js --report"
8+
},
9+
"dependencies": {
10+
"@playwright/test": "^1.40.0",
11+
"playwright": "^1.40.0"
12+
},
13+
"devDependencies": {}
14+
}

0 commit comments

Comments
 (0)