Skip to content

Convert remaining automated: false TCs to assisted mode #504

@couimet

Description

@couimet

132 TCs remain automated: false after the assisted mode infrastructure landed in #503. The infrastructure is in place — waitForHuman, log-based QuickPick assertions (assertQuickPickItemsLogged), tristate YAML, test:release:grep filtering. Now we need to convert the remaining manual TCs.

Strategy: Hybrid (Option 3)

3-4 high-value feature-area PRs establish patterns, then a sweep PR handles the long tail (~5 PRs total).

Phase 1 — Pattern-establishing PRs

  • Terminal Picker + File Picker (22 TCs) — Category A: QuickPick content validation via assertQuickPickItemsLogged
  • R-D Bind to Destination (12 TCs) — Category A: multi-step assisted pattern (bind + verify toast)
  • Context Menus (all 4 types: 23 TCs) — Category B: command-trigger + toast/status bar verification
  • R-G Go to Link (9 TCs) — Category B: keybinding-trigger + navigation verification

Phase 2 — Sweep PR

  • Everything else (~66 TCs) — uses patterns from Phase 1 PRs: settings-dependent TCs (Clipboard Preservation, Dirty Buffer Warning, Two-Verb Vocabulary), multi-step scenarios (R-V, Text Editor Destination), remaining small feature areas

Phase 3 — Triage truly manual TCs

  • Per-TC triage — identify Category E TCs that genuinely cannot convert (visual-only checks, platform-specific behavior, external tool interaction). These stay automated: false permanently.

TC breakdown by feature area

 13  Terminal Picker
 12  R-D Bind to Destination
 11  Context Menus — Editor Content
 10  Send File Path
  9  R-G Go to Link
  9  File Picker
  7  Changed — Two-Verb Vocabulary
  6  R-M Status Bar Menu (remaining)
  6  Clipboard Preservation
  5  R-V Send Terminal Selection
  5  Dirty Buffer Warning
  5  Context Menus — Explorer
  4  Context Menus — Editor Tab
  4  Changed — Picker When Unbound
  4  Changed — Editor Binding Validation
  3  Core Send Commands — R-L
  3  Context Menus — Terminal
  3  Changed — Text Editor No Split Required
  2  Filename-Only Navigation Fallback
  2  Core Send Commands — R-C
  5  Text Editor Destination (various)
  1  R-U Unbind
  1  Link Navigation — URL Exclusion
  1  Clickable Links — Document Link Tooltip

Infrastructure enhancements that may be needed

  • Context menu trigger helper — simulates right-click context menu (or command equivalent) for Category B tests
  • Terminal text assertion — log-based assertion for sendText calls to verify text was sent to terminal
  • Multi-step waitForHuman — some TCs need 2-3 sequential human actions with intermediate assertions (already supported by current API)

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions