Skip to content

Commit 316c64e

Browse files
committed
Add AI demo script and resolve open design decisions
- Add docs/ai-demo-script.md with 15-min demo flow and backup test - Resolve 10+ DECIDE items in PLAN.md (CI scope, AI config, screenshots, GitLab sync, robocop, browser default, etc.) - Mark verified items: selectors, setup-uv version, MCP package
1 parent 80d8848 commit 316c64e

2 files changed

Lines changed: 120 additions & 17 deletions

File tree

PLAN.md

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -301,7 +301,7 @@ Note: `uv.lock` is NOT in .gitignore — it must be committed for reproducible i
301301
### CURRENT ISSUES — Phase 1
302302

303303
- [x] **DECIDED:** Repo created via `gh repo create` CLI (public, with description).
304-
- [ ] **DECIDE:** Should GitLab project be manually created or left for auto-sync?
304+
- [x] **DECIDED:** Left for auto-sync. `github-sync` CronJob auto-discovers all HackXIt repos and mirrors to `github-mirrors` group.
305305
- [x] **RESOLVED:** GitHub auth required `workflow` scope for pushing workflow files. Added via `gh auth refresh -s workflow`.
306306

307307
---
@@ -392,10 +392,10 @@ Must verify:
392392

393393
### CURRENT ISSUES — Phase 2
394394

395-
- [ ] **VERIFY:** Does the `ghcr.io/va-h/devcontainers-features/uv:1` feature work reliably in Codespaces? If not, fall back to curl-based install in post-create.sh.
395+
- [ ] **VERIFY:** Does the `ghcr.io/va-h/devcontainers-features/uv:1` feature work reliably in Codespaces? Fallback already in post-create.sh.
396396
- [ ] **VERIFY:** Does `rfbrowser init chromium` complete within the Codespace timeout?
397-
- [ ] **VERIFY:** Are there Playwright system dependencies missing in the bookworm base image? May need `npx playwright install-deps` or `apt-get install` for missing libs.
398-
- [ ] **KNOWN ISSUE:** First Codespace build downloads ~250MB of Chromium binaries. On slow connections this can take several minutes. Consider: Codespace prebuild configuration to cache this.
397+
- [x] **MITIGATED:** Playwright system deps — added `npx playwright install-deps chromium` to post-create.sh.
398+
- [ ] **KNOWN ISSUE:** First Codespace build downloads ~250MB of Chromium binaries. Consider Codespace prebuild for caching.
399399

400400
---
401401

@@ -519,7 +519,7 @@ Then look for a `chromium-*` directory inside the found path.
519519

520520
### CURRENT ISSUES — Phase 3
521521

522-
- [ ] **DECIDE:** Should the script also check for `robocop` (linter) or only core runtime?
522+
- [x] **DECIDED:** Core runtime only. Robocop is a dev tool, not required for workshop tests to run.
523523
- [ ] **VERIFY:** Playwright browsers path detection works correctly in Codespaces (may use a non-standard path).
524524
- [ ] **VERIFY:** ANSI color codes render correctly in Codespaces terminal and Windows Terminal.
525525

@@ -633,9 +633,9 @@ Example Test Case
633633

634634
### CURRENT ISSUES — Phase 4
635635

636-
- [ ] **VERIFY:** SauceDemo CSS selectors are current (they occasionally change their markup).
636+
- [x] **VERIFIED:** SauceDemo CSS selectors are current — all 25 tests pass locally and in CI (March 2026).
637637
- [ ] **VERIFY:** Browser Library's `New Browser` keyword works headless in Codespaces (no display server).
638-
- [ ] **DECIDE:** Should example tests use `browser=chromium` explicitly or rely on default?
638+
- [x] **DECIDED:** Rely on default. `common.resource` uses `chromium` in `New Browser`, which is sufficient.
639639
- [ ] **KNOWN RISK:** SauceDemo could go down during the workshop. Prepare a fallback SUT (automationexercise.com, the-internet.herokuapp.com).
640640

641641
---
@@ -721,9 +721,9 @@ Deploy `results/` to GitHub Pages on main branch pushes:
721721

722722
### CURRENT ISSUES — Phase 5
723723

724-
- [ ] **VERIFY:** `astral-sh/setup-uv@v7` is the latest stable version of the action.
725-
- [ ] **DECIDE:** Should CI run ALL tests or only student exercise tests? Running all on every PR may be noisy.
726-
- [ ] **DECIDE:** Should failed tests block PR merge or just report? For a workshop, reporting only is more friendly.
724+
- [x] **VERIFIED:** `astral-sh/setup-uv@v7` is latest major (v7.3.1 latest release, Feb 2026). No changes needed.
725+
- [x] **DECIDED:** CI runs ALL tests on every PR. This validates students don't break existing tests. 27 tests run in ~1 min.
726+
- [x] **DECIDED:** Failed tests do not block merge (no branch protection rules). CI reports status but workshop is learning-focused.
727727
- [ ] **VERIFY:** GitHub Actions free tier limits (2000 min/month for free accounts, 3000 min/month for Pro). Each run with Playwright install may take ~3-5 min.
728728

729729
---
@@ -790,9 +790,9 @@ This demonstrates the "AI-assisted test automation" angle.
790790

791791
### CURRENT ISSUES — Phase 6
792792

793-
- [ ] **VERIFY:** `robotframework-mcp` package exists on PyPI and is compatible with current RF/Browser versions.
794-
- [ ] **DECIDE:** Should Claude Code / AI be pre-configured in the DevContainer, or shown as a demo only?
795-
- [ ] **DECIDE:** Should the repo have GitHub Copilot or Claude Code review PRs automatically via GitHub Actions?
793+
- [x] **VERIFIED:** `robotframework-mcp` exists but is SeleniumLibrary-based (not suitable). Using `rf-mcp` (0.30+, 71+ releases, mature). Added to dev deps.
794+
- [x] **DECIDED:** AI demo only (instructor shows Claude Code). Not pre-configured for students — adds complexity. `.claude/settings.json` exists for instructor use.
795+
- [x] **DECIDED:** No automatic AI PR review. Workshop is about learning, not automated gatekeeping. Instructor reviews manually.
796796

797797
---
798798

@@ -909,8 +909,8 @@ Guide for the AI demo portion:
909909

910910
### CURRENT ISSUES — Phase 7
911911

912-
- [ ] **DECIDE:** Should docs include screenshots? If yes, they need to be captured during Phase 11 pre-check.
913-
- [ ] **DECIDE:** How much RF theory vs hands-on in the cheatsheet? Students learn C-style, so comparison tables (C if/else → RF IF/ELSE) might be helpful.
912+
- [x] **DECIDED:** Screenshots are nice-to-have. Text instructions are sufficient; screenshots can be added during Phase 11 pre-check if time permits.
913+
- [x] **DECIDED:** C comparison tables already included in `docs/02-rf-syntax-cheatsheet.md`. Balance is good: minimal theory, heavy on examples.
914914
- [ ] **VERIFY:** Windows setup doc should be tested on an actual Windows machine or VM.
915915

916916
---
@@ -1250,8 +1250,8 @@ Phase 1 (Repo + pyproject.toml)
12501250
- [x] Create `.claude/settings.json` with MCP server config
12511251
- [x] Verify MCP package on PyPI — `robotframework-mcp` (SeleniumLibrary-based, not suitable); using `rf-mcp` (0.30+, mature)
12521252
- [x] Add `rf-mcp` to `pyproject.toml` dev dependencies
1253-
- [ ] Prepare AI demo script (natural language → RF test → run → refine)
1254-
- [ ] Test the demo end-to-end
1253+
- [x] Prepare AI demo script `docs/ai-demo-script.md` with flow, prompts, backup test
1254+
- [ ] Test the demo end-to-end (requires live Claude Code session)
12551255

12561256
### Phase 7: Documentation (Day 4-5)
12571257
- [x] Create `README.md` (quick start, prerequisites, agenda, badges)

docs/ai-demo-script.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# AI-Assisted Testing Demo Script
2+
3+
**Duration:** ~15 minutes
4+
**When:** 19:15 in the agenda
5+
6+
## Setup
7+
8+
- Have Claude Code (or Copilot) open in a Codespace terminal
9+
- Have the project README open for context
10+
11+
## Demo Flow
12+
13+
### 1. Natural Language to Test (3 min)
14+
15+
**Say:** "Let me show you how AI can help write tests. I'll describe what I want in plain English."
16+
17+
**Prompt to give Claude Code:**
18+
19+
> Write a Robot Framework test that verifies products on SauceDemo can be sorted by name Z-to-A. The test should log in as standard_user, select the Z-A sort option, and verify the first product name starts with a letter later in the alphabet than the last product.
20+
21+
**Expected output:** A complete `.robot` file with Settings, Variables, Test Cases, and Keywords sections.
22+
23+
### 2. Review the Generated Test (3 min)
24+
25+
**Say:** "Now let's review what the AI generated. Does it look correct?"
26+
27+
Walk through:
28+
- Does it import the right resources?
29+
- Are the selectors correct for SauceDemo?
30+
- Does the sorting assertion make sense?
31+
- Are there any issues?
32+
33+
**Key teaching moment:** AI gets the structure right but may use wrong selectors or flawed assertions. The tester must validate.
34+
35+
### 3. Run the Test (2 min)
36+
37+
```bash
38+
uv run robot tests/student_exercises/ai_generated_sort_test.robot
39+
```
40+
41+
If it passes: "Great, but we still need to verify it's actually testing what we intended."
42+
43+
If it fails: "This is normal! Let's debug and fix it together."
44+
45+
### 4. Refine (3 min)
46+
47+
If the test had issues, refine the prompt:
48+
49+
> The sort dropdown on SauceDemo uses the CSS selector `.product_sort_container` and the Z-A option has value `za`. After sorting, verify the first `.inventory_item_name` text is "Test.allTheThings() T-Shirt (Red)" which is alphabetically last.
50+
51+
### 5. Compare with Hand-Written (2 min)
52+
53+
**Say:** "Now compare this AI-generated test with the hand-written product sort test in our test suite."
54+
55+
Open `tests/03_product_tests/product_catalog.robot` and show the `Sort Products By Price Low To High` test.
56+
57+
**Key points:**
58+
- The hand-written test uses resource file keywords (better abstraction)
59+
- The AI test might use raw selectors (less maintainable)
60+
- AI is great for boilerplate, humans are needed for test strategy
61+
62+
### 6. Wrap Up (2 min)
63+
64+
**Key takeaways:**
65+
- AI saves time on repetitive code
66+
- AI doesn't replace test design thinking
67+
- Always run the test — don't trust unexecuted code
68+
- Use AI for the "how", you decide the "what"
69+
70+
## Backup Demo (if AI tools unavailable)
71+
72+
If Claude Code / Copilot aren't available:
73+
74+
1. Show a pre-generated test file
75+
2. Walk through how it was created
76+
3. Run it live
77+
4. Still do the comparison with hand-written tests
78+
79+
## Pre-Generated Test (backup)
80+
81+
Save this as `tests/student_exercises/ai_demo_sort_test.robot` before the workshop:
82+
83+
```robotframework
84+
*** Settings ***
85+
Documentation AI-generated test: Verify Z-A product sorting on SauceDemo.
86+
Library Browser
87+
Resource ../../resources/common.resource
88+
Resource ../../resources/login_page.resource
89+
Resource ../../resources/products_page.resource
90+
91+
Suite Setup Open SauceDemo
92+
Suite Teardown Close SauceDemo
93+
94+
*** Test Cases ***
95+
Products Should Sort By Name Z To A
96+
[Documentation] Verify that selecting Z-A sort orders products correctly.
97+
[Tags] student exercise ai-generated
98+
Login With Credentials ${VALID_USER} ${VALID_PASSWORD}
99+
Login Should Succeed
100+
Sort Products By Name (Z to A)
101+
${first_product}= Get Text css=.inventory_item_name >> nth=0
102+
Should Be Equal ${first_product} Test.allTheThings() T-Shirt (Red)
103+
```

0 commit comments

Comments
 (0)