Skip to content

Commit 6e80419

Browse files
CopilotCopilot
andcommitted
test(autotest): fix 2 remaining CI flakes (dep-viewer JDK label, gradle-java25 completion)
- java-dependency-viewer: drop verify: on verify-jdk step. The wiki uses 'JDK Libraries' as a category label, but the actual tree node label is 'JRE System Library' (with child modules like java.base). The deterministic 'expandTreeItem JRE System Library' action is the ground truth (it fails fast if the node doesn't exist); the verify: text was causing LLM downgrades because BEFORE/AFTER screenshots correctly showed JRE System Library expansion but the LLM expected a separate 'JDK Libraries' grouping that doesn't exist in current vscode-java. - java-gradle-java25: drop verify: on verify-completion (same flake as the other 4 completion plans fixed in the previous commit — Gradle java25 plan was missed). Add waitBefore: 5 so the popup has time to render before screenshot capture. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent aae2dd6 commit 6e80419

2 files changed

Lines changed: 7 additions & 2 deletions

File tree

test-plans/java-dependency-viewer.yaml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,9 @@ steps:
5858
timeout: 15
5959

6060
# ── Verify JDK Libraries node ───────────────────────────
61+
# Drop `verify:` — the deterministic `expandTreeItem JRE System Library`
62+
# is the ground truth (succeeds only if the node exists). The wiki uses
63+
# "JDK Libraries" as a category name but the actual tree label is
64+
# "JRE System Library", which can confuse the LLM verifier.
6165
- id: "verify-jdk"
6266
action: "expandTreeItem JRE System Library"
63-
verify: "JDK Libraries node visible and expandable"

test-plans/java-gradle-java25.yaml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,11 +45,13 @@ steps:
4545
timeout: 15
4646

4747
# ── Step 3: Verify completion ───────────────────────────
48+
# Drop `verify:` — completion popup timing flakes on CI (transient
49+
# "Loading…" indicator). verifyCompletion.notEmpty is the ground truth.
4850
- id: "verify-completion"
4951
action: "triggerCompletionAt endOfMethod"
50-
verify: "Code completion works at end of method body"
5152
verifyCompletion:
5253
notEmpty: true
54+
waitBefore: 5
5355

5456
# ── Step 4: Verify editing ────────────────────────────────
5557
- id: "goto-line"

0 commit comments

Comments
 (0)