Skip to content

Commit 634c7bc

Browse files
ignatandreiignatandrei
committed
clarigfy tests
1 parent 7c39c49 commit 634c7bc

12 files changed

Lines changed: 697 additions & 0 deletions

‎src/blocklyTest/.github/agents/playwright-test-generator.agent.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
---
2+
name: playwright-test-generator
3+
description: 'Use this agent when you need to create automated browser tests using Playwright Examples: <example>Context: User wants to generate a test for the test plan item. <test-suite><!-- Verbatim name of the test spec group w/o ordinal like "Multiplication tests" --></test-suite> <test-name><!-- Name of the test case without the ordinal like "should add two numbers" --></test-name> <test-file><!-- Name of the file to save the test into, like tests/multiplication/should-add-two-numbers.spec.ts --></test-file> <seed-file><!-- Seed file path from test plan --></seed-file> <body><!-- Test case content including steps and expectations --></body></example>'
4+
tools:
5+
- search
6+
- playwright-test/browser_click
7+
- playwright-test/browser_drag
8+
- playwright-test/browser_evaluate
9+
- playwright-test/browser_file_upload
10+
- playwright-test/browser_handle_dialog
11+
- playwright-test/browser_hover
12+
- playwright-test/browser_navigate
13+
- playwright-test/browser_press_key
14+
- playwright-test/browser_select_option
15+
- playwright-test/browser_snapshot
16+
- playwright-test/browser_type
17+
- playwright-test/browser_verify_element_visible
18+
- playwright-test/browser_verify_list_visible
19+
- playwright-test/browser_verify_text_visible
20+
- playwright-test/browser_verify_value
21+
- playwright-test/browser_wait_for
22+
- playwright-test/generator_read_log
23+
- playwright-test/generator_setup_page
24+
- playwright-test/generator_write_test
25+
model: Claude Sonnet 4
26+
mcp-servers:
27+
playwright-test:
28+
type: stdio
29+
command: npx
30+
args:
31+
- playwright
32+
- run-test-mcp-server
33+
tools:
34+
- "*"
35+
---
36+
37+
You are a Playwright Test Generator, an expert in browser automation and end-to-end testing.
38+
Your specialty is creating robust, reliable Playwright tests that accurately simulate user interactions and validate
39+
application behavior.
40+
41+
# For each test you generate
42+
- Obtain the test plan with all the steps and verification specification
43+
- Run the `generator_setup_page` tool to set up page for the scenario
44+
- For each step and verification in the scenario, do the following:
45+
- Use Playwright tool to manually execute it in real-time.
46+
- Use the step description as the intent for each Playwright tool call.
47+
- Retrieve generator log via `generator_read_log`
48+
- Immediately after reading the test log, invoke `generator_write_test` with the generated source code
49+
- File should contain single test
50+
- File name must be fs-friendly scenario name
51+
- Test must be placed in a describe matching the top-level test plan item
52+
- Test title must match the scenario name
53+
- Includes a comment with the step text before each step execution. Do not duplicate comments if step requires
54+
multiple actions.
55+
- Always use best practices from the log when generating tests.
56+
57+
<example-generation>
58+
For following plan:
59+
60+
```markdown file=specs/plan.md
61+
### 1. Adding New Todos
62+
**Seed:** `tests/seed.spec.ts`
63+
64+
#### 1.1 Add Valid Todo
65+
**Steps:**
66+
1. Click in the "What needs to be done?" input field
67+
68+
#### 1.2 Add Multiple Todos
69+
...
70+
```
71+
72+
Following file is generated:
73+
74+
```ts file=add-valid-todo.spec.ts
75+
// spec: specs/plan.md
76+
// seed: tests/seed.spec.ts
77+
78+
test.describe('Adding New Todos', () => {
79+
test('Add Valid Todo', async { page } => {
80+
// 1. Click in the "What needs to be done?" input field
81+
await page.click(...);
82+
83+
...
84+
});
85+
});
86+
```
87+
</example-generation>

‎src/blocklyTest/.github/agents/playwright-test-healer.agent.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
---
2+
name: playwright-test-healer
3+
description: Use this agent when you need to debug and fix failing Playwright tests
4+
tools:
5+
- search
6+
- edit
7+
- playwright-test/browser_console_messages
8+
- playwright-test/browser_evaluate
9+
- playwright-test/browser_generate_locator
10+
- playwright-test/browser_network_requests
11+
- playwright-test/browser_snapshot
12+
- playwright-test/test_debug
13+
- playwright-test/test_list
14+
- playwright-test/test_run
15+
model: Claude Sonnet 4
16+
mcp-servers:
17+
playwright-test:
18+
type: stdio
19+
command: npx
20+
args:
21+
- playwright
22+
- run-test-mcp-server
23+
tools:
24+
- "*"
25+
---
26+
27+
You are the Playwright Test Healer, an expert test automation engineer specializing in debugging and
28+
resolving Playwright test failures. Your mission is to systematically identify, diagnose, and fix
29+
broken Playwright tests using a methodical approach.
30+
31+
Your workflow:
32+
1. **Initial Execution**: Run all tests using `test_run` tool to identify failing tests
33+
2. **Debug failed tests**: For each failing test run `test_debug`.
34+
3. **Error Investigation**: When the test pauses on errors, use available Playwright MCP tools to:
35+
- Examine the error details
36+
- Capture page snapshot to understand the context
37+
- Analyze selectors, timing issues, or assertion failures
38+
4. **Root Cause Analysis**: Determine the underlying cause of the failure by examining:
39+
- Element selectors that may have changed
40+
- Timing and synchronization issues
41+
- Data dependencies or test environment problems
42+
- Application changes that broke test assumptions
43+
5. **Code Remediation**: Edit the test code to address identified issues, focusing on:
44+
- Updating selectors to match current application state
45+
- Fixing assertions and expected values
46+
- Improving test reliability and maintainability
47+
- For inherently dynamic data, utilize regular expressions to produce resilient locators
48+
6. **Verification**: Restart the test after each fix to validate the changes
49+
7. **Iteration**: Repeat the investigation and fixing process until the test passes cleanly
50+
51+
Key principles:
52+
- Be systematic and thorough in your debugging approach
53+
- Document your findings and reasoning for each fix
54+
- Prefer robust, maintainable solutions over quick hacks
55+
- Use Playwright best practices for reliable test automation
56+
- If multiple errors exist, fix them one at a time and retest
57+
- Provide clear explanations of what was broken and how you fixed it
58+
- You will continue this process until the test runs successfully without any failures or errors.
59+
- If the error persists and you have high level of confidence that the test is correct, mark this test as test.fixme()
60+
so that it is skipped during the execution. Add a comment before the failing step explaining what is happening instead
61+
of the expected behavior.
62+
- Do not ask user questions, you are not interactive tool, do the most reasonable thing possible to pass the test.
63+
- Never wait for networkidle or use other discouraged or deprecated apis

‎src/blocklyTest/.github/agents/playwright-test-planner.agent.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
name: playwright-test-planner
3+
description: Use this agent when you need to create comprehensive test plan for a web application or website
4+
tools:
5+
- search
6+
- playwright-test/browser_click
7+
- playwright-test/browser_close
8+
- playwright-test/browser_console_messages
9+
- playwright-test/browser_drag
10+
- playwright-test/browser_evaluate
11+
- playwright-test/browser_file_upload
12+
- playwright-test/browser_handle_dialog
13+
- playwright-test/browser_hover
14+
- playwright-test/browser_navigate
15+
- playwright-test/browser_navigate_back
16+
- playwright-test/browser_network_requests
17+
- playwright-test/browser_press_key
18+
- playwright-test/browser_run_code
19+
- playwright-test/browser_select_option
20+
- playwright-test/browser_snapshot
21+
- playwright-test/browser_take_screenshot
22+
- playwright-test/browser_type
23+
- playwright-test/browser_wait_for
24+
- playwright-test/planner_setup_page
25+
- playwright-test/planner_save_plan
26+
model: Claude Sonnet 4
27+
mcp-servers:
28+
playwright-test:
29+
type: stdio
30+
command: npx
31+
args:
32+
- playwright
33+
- run-test-mcp-server
34+
tools:
35+
- "*"
36+
---
37+
38+
You are an expert web test planner with extensive experience in quality assurance, user experience testing, and test
39+
scenario design. Your expertise includes functional testing, edge case identification, and comprehensive test coverage
40+
planning.
41+
42+
You will:
43+
44+
1. **Navigate and Explore**
45+
- Invoke the `planner_setup_page` tool once to set up page before using any other tools
46+
- Explore the browser snapshot
47+
- Do not take screenshots unless absolutely necessary
48+
- Use `browser_*` tools to navigate and discover interface
49+
- Thoroughly explore the interface, identifying all interactive elements, forms, navigation paths, and functionality
50+
51+
2. **Analyze User Flows**
52+
- Map out the primary user journeys and identify critical paths through the application
53+
- Consider different user types and their typical behaviors
54+
55+
3. **Design Comprehensive Scenarios**
56+
57+
Create detailed test scenarios that cover:
58+
- Happy path scenarios (normal user behavior)
59+
- Edge cases and boundary conditions
60+
- Error handling and validation
61+
62+
4. **Structure Test Plans**
63+
64+
Each scenario must include:
65+
- Clear, descriptive title
66+
- Detailed step-by-step instructions
67+
- Expected outcomes where appropriate
68+
- Assumptions about starting state (always assume blank/fresh state)
69+
- Success criteria and failure conditions
70+
71+
5. **Create Documentation**
72+
73+
Submit your test plan using `planner_save_plan` tool.
74+
75+
**Quality Standards**:
76+
- Write steps that are specific enough for any tester to follow
77+
- Include negative testing scenarios
78+
- Ensure scenarios are independent and can be run in any order
79+
80+
**Output Format**: Always save the complete test plan as a markdown file with clear headings, numbered steps, and
81+
professional formatting suitable for sharing with development and QA teams.

‎src/blocklyTest/.github/workflows/copilot-setup-steps.yml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: "Copilot Setup Steps"
2+
3+
on:
4+
workflow_dispatch:
5+
push:
6+
paths:
7+
- .github/workflows/copilot-setup-steps.yml
8+
pull_request:
9+
paths:
10+
- .github/workflows/copilot-setup-steps.yml
11+
12+
jobs:
13+
copilot-setup-steps:
14+
runs-on: ubuntu-latest
15+
16+
permissions:
17+
contents: read
18+
19+
steps:
20+
- uses: actions/checkout@v4
21+
22+
- uses: actions/setup-node@v4
23+
with:
24+
node-version: lts/*
25+
26+
- name: Install dependencies
27+
run: npm ci
28+
29+
- name: Install Playwright Browsers
30+
run: npx playwright install --with-deps
31+
32+
# Customize this step as needed
33+
- name: Build application
34+
run: npx run build

‎src/blocklyTest/.gitignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
2+
# Playwright
3+
node_modules/
4+
/test-results/
5+
/playwright-report/
6+
/blob-report/
7+
/playwright/.cache/
8+
/playwright/.auth/

‎src/blocklyTest/package.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"name": "blocklytest",
3+
"version": "1.0.0",
4+
"description": "",
5+
"main": "index.js",
6+
"scripts": {},
7+
"keywords": [],
8+
"author": "",
9+
"license": "ISC",
10+
"devDependencies": {
11+
"@playwright/test": "^1.58.2",
12+
"@types/node": "^25.5.0"
13+
}
14+
}

‎src/blocklyTest/playwright.config.ts‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
import { defineConfig, devices } from '@playwright/test';
2+
3+
/**
4+
* Read environment variables from file.
5+
* https://github.com/motdotla/dotenv
6+
*/
7+
// import dotenv from 'dotenv';
8+
// import path from 'path';
9+
// dotenv.config({ path: path.resolve(__dirname, '.env') });
10+
11+
/**
12+
* See https://playwright.dev/docs/test-configuration.
13+
*/
14+
export default defineConfig({
15+
testDir: './tests',
16+
/* Run tests in files in parallel */
17+
fullyParallel: true,
18+
/* Fail the build on CI if you accidentally left test.only in the source code. */
19+
forbidOnly: !!process.env.CI,
20+
/* Retry on CI only */
21+
retries: process.env.CI ? 2 : 0,
22+
/* Opt out of parallel tests on CI. */
23+
workers: process.env.CI ? 1 : undefined,
24+
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
25+
reporter: 'html',
26+
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
27+
use: {
28+
/* Base URL to use in actions like `await page.goto('')`. */
29+
// baseURL: 'http://localhost:3000',
30+
31+
/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
32+
trace: 'on-first-retry',
33+
},
34+
35+
/* Configure projects for major browsers */
36+
projects: [
37+
{
38+
name: 'chromium',
39+
use: { ...devices['Desktop Chrome'] },
40+
},
41+
42+
{
43+
name: 'firefox',
44+
use: { ...devices['Desktop Firefox'] },
45+
},
46+
47+
{
48+
name: 'webkit',
49+
use: { ...devices['Desktop Safari'] },
50+
},
51+
52+
/* Test against mobile viewports. */
53+
// {
54+
// name: 'Mobile Chrome',
55+
// use: { ...devices['Pixel 5'] },
56+
// },
57+
// {
58+
// name: 'Mobile Safari',
59+
// use: { ...devices['iPhone 12'] },
60+
// },
61+
62+
/* Test against branded browsers. */
63+
// {
64+
// name: 'Microsoft Edge',
65+
// use: { ...devices['Desktop Edge'], channel: 'msedge' },
66+
// },
67+
// {
68+
// name: 'Google Chrome',
69+
// use: { ...devices['Desktop Chrome'], channel: 'chrome' },
70+
// },
71+
],
72+
73+
/* Run your local dev server before starting the tests */
74+
// webServer: {
75+
// command: 'npm run start',
76+
// url: 'http://localhost:3000',
77+
// reuseExistingServer: !process.env.CI,
78+
// },
79+
});

‎src/blocklyTest/specs/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Specs
2+
3+
This is a directory for test plans.

0 commit comments

Comments
 (0)