[workshop-sim] Workshop Simulation Report — 2026-07-23 (Run #7, 1000×Monte Carlo)

[github-actions]

Overview

• Date: 2026-07-23
• Students simulated: 46 × 1,000 Monte Carlo runs (46,000 total)
• Workshop steps available: 36/36
• Overall success rate: 19.0% (95% Monte Carlo interval: 18.7%–19.4%)
• Highest-dropout step: 14b-pr-reviewer-workflow (30.8% conditional dropout among 16,020 at-risk runs; 95% CI: 30.1%–31.5%)
• Lowest curriculum quality step: 15-conditional-logic.md (overall score 7.53/10)
• Learning KPI index: 7.44/10 (active_learning 4.72 · checkpoint_quality 8.75 · scaffolding 9.31)
• Model: none / 2026-07-assumption-model-v2 (parameter hash none)
• Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty

Part Summary

Part	Files	Mean Score	Std Dev
Part 1 — core path (lessons 00–14)	24	9.10 / 10.0	±0.76
Part 2 — advanced (lessons 15+)	12	8.73 / 10.0	±0.48
Overall corpus	36	8.98 / 10.0	±0.66

No pages are classified as other in this cohort run.

Critical Findings

1. CCA learners are systematically blocked at 14b-pr-reviewer-workflow (Part 1 advanced): The step has no /agentic-workflows invocation guidance and no browser-only authoring path, causing 3,737/4,936 failures from copilot-skill-guidance-missing. Six students using tool: CCA recorded 0% success rate, including an actions-user who had enough technical skill to succeed otherwise.
2. copilot-access-missing is the leading blocker at 07-first-workflow (Part 1 core): 4,372/4,951 failures trace to learners who have not confirmed Copilot access before reaching Step 7. The 07d model-access verification step is present but learners who skip it are disproportionately affected.
3. 05-agentic-intro (Part 1 core) shows the highest raw failure count (7,335): Its active_learning score is 2.7/10 — the lowest among learning pages. The optional match-and-reveal exercise does not enforce validation before advancing, so conceptually underprepared learners carry the gap into 05c-agentic-practice.
4. Learning quality health is acceptable for learners who stay: The cohort learning KPI index is 7.44/10. Checkpoint quality (8.75) and scaffolding (9.31) are strong; only active_learning (4.72) drags the index. Learners who complete the workshop are building skills effectively.
5. The highest-impact single repair belongs to Part 1 (14b-pr-reviewer-workflow): Adding CCA/Agents-tab authoring guidance and a browser-only compile path would directly address the top per-step dropout rate (30.8%) without reducing learning quality.

Top Repairs to Prioritize

Note: some student dropout is expected and acceptable. Repairs must maintain or improve the learning KPI index — do not lower the cognitive bar or remove practice to chase headline completion numbers.

1. Add /agentic-workflows CCA guidance and browser-only path to 14b-pr-reviewer-workflow (completion impact: ↑ · learning KPI impact: ↑)
2. Add a validated pre-check callout for Copilot access at the top of 07-your-first-workflow.md (completion impact: ↑ · learning KPI impact: ↔)
3. Replace the optional match exercise in 05-agentic-workflows-intro.md with a required decision gate before the checkpoint (completion impact: ↔ · learning KPI impact: ↑)

Dropout by step

Step	At-risk runs	Dropout rate	95% CI	Failure mode	Top reason
14b-pr-reviewer-workflow	16,020	30.8%	30.1–31.5%	Access barrier	No /agentic-workflows skill invocation guidance for CCA/Agents-tab learners
07-first-workflow	23,333	21.2%	20.7–21.7%	Access barrier	Copilot model access not confirmed before authoring step
05-agentic-intro	38,606	19.0%	18.6–19.4%	Learning barrier	Conceptual gap: agentic workflow model vs. traditional CI/CD
05c-agentic-practice	31,271	13.5%	13.1–13.9%	Learning barrier	Classification exercise exposes unfilled concept gap from step 05
05b-agentic-security	27,056	9.8%	9.4–10.1%	Learning barrier	Security model unfamiliar to beginners
04-actions-intro	42,706	9.6%	9.3–9.9%	Learning barrier	Concept overload for no-coding/beginner learners
02-setup	46,000	7.2%	6.9–7.4%	Access barrier	Environment setup friction; enterprise proxy and local path complexity
12-test-and-iterate	17,002	5.8%	5.4–6.1%	Learning barrier	Test-and-iterate friction after complex workflow authoring
06-install-gh-aw	24,415	4.4%	4.2–4.7%	Access barrier	CLI extension install fails or is skipped on browser-only path
09-agentic-editing	17,716	4.0%	3.8–4.3%	Learning barrier	Workflow editing friction for non-CLI learners

Learning quality KPIs

Step file	Overall	active_learning	checkpoint_quality	scaffolding	KPI index	Repair priority
05-agentic-workflows-intro.md	8.16	2.7	10.0	10.0	7.35	High — active_learning
07a-your-first-workflow-terminal.md	7.73	5.9	5.0	10.0	6.69	High — checkpoint_quality
14b-pr-reviewer-workflow.md	8.84	4.2	10.0	10.0	7.89	High — access + active_learning
02a-setup-codespace.md	7.88	5.2	10.0	5.0	6.89	Medium — scaffolding
02c-setup-browser.md	8.04	5.9	10.0	5.0	7.15	Medium — scaffolding
17-add-mcp-tools.md	8.32	2.5	10.0	10.0	7.27	Medium — active_learning
05b-agentic-workflows-security.md	8.64	3.2	10.0	10.0	7.53	Medium — active_learning
08-run-your-workflow.md	8.58	3.4	10.0	10.0	7.60	Medium
15-conditional-logic.md	7.53	4.0	10.0	10.0	7.82	Medium — overall score
Cohort mean	8.98	4.72	8.75	9.31	7.44	—

Curriculum quality metrics

Step file	Overall score	Lowest rubric dimension	Recommended repair focus
15-conditional-logic.md	7.53	active_learning (4.0)	Add a step-through conditional example with a learner decision point
07a-your-first-workflow-terminal.md	7.73	checkpoint_quality (5.0)	Add a mid-step sub-checkpoint after first compile succeeds
02a-setup-codespace.md	7.88	scaffolding (5.0)	Add numbered step progression with explicit completion cues
02c-setup-browser.md	8.04	scaffolding (5.0)	Mirror the Codespace scaffolding improvements
05-agentic-workflows-intro.md	8.16	active_learning (2.7)	Replace optional match exercise with a required gate before checkpoint
17-add-mcp-tools.md	8.32	active_learning (2.5)	Add a guided try-it exercise before the full MCP configuration task
08-run-your-workflow.md	8.58	active_learning (3.4)	Add a predict-then-verify step before the learner triggers the run
07d-confirm-model-access.md	8.66	active_learning (5.8)	Strengthen the model-access test prompt into a gate checkpoint
14b-pr-reviewer-workflow.md	8.84	active_learning (4.2)	Add CCA authoring guidance and a browser-only compile/commit alternative

Segment breakdowns

By technical level

Level	Students	Mean success rate
advanced	5	39.3%
actions-user	11	43.4%
github-basic	19	10.5%
beginner	11	0.2%

By personality

Personality	Students	Mean success rate
confused	6	22.5%
impatient	6	21.8%
skeptical	7	21.5%
methodical	12	18.8%
curious	15	15.5%

By UI preference

UI preferred	Students	Mean success rate
CLI / terminal preferred	24	31.7%
UI preferred	22	5.2%

UI-preferred learners are significantly disadvantaged: most high-dropout steps (especially 14b) have terminal-centric instructions with no browser-only alternative.

Notable student journeys (3)

Surprising success — Learner 026 (advanced, confused, devops, CLI, 61.6%)
Despite the confused personality, deep DevOps experience and CLI preference made terminal steps low-friction. High familiarity with Actions YAML reduced the agentic-concept gap. Most remaining failures clustered at 14b-pr-reviewer-workflow — a new event-driven authoring pattern even for experienced DevOps engineers.

Unexpected dropout — Learner 006 (actions-user, impatient, backend-dev, CCA, 0%)
Technically capable of completing the workshop, but tool: CCA and impatient personality interact fatally at 14b: the step provides no /agentic-workflows invocation guidance for the Agents tab. An impatient learner will not hunt for the missing pattern. 529/1,000 runs failed at Step 14b alone despite passing all earlier steps.

Content-gap case — Learner 003 (github-basic, skeptical, program-manager, CCA, 0%)
Most failures hit 05c-agentic-practice — a learning barrier. The prior 05-agentic-intro optional match exercise does not enforce comprehension before advancing. A skeptical program manager evaluating agentic workflows for their team needs confident concept mastery; the absence of a required decision gate is a curriculum gap rather than a learner limitation.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔬 Workshop Student Simulator · 96.6 AIC · ⌖ 5.28 AIC · ⊞ 10.7K · ◷

• expires on Jul 24, 2026, 3:21 PM UTC