Skip to content

Commit 7750caa

Browse files
roomoteroomote
committed
ci: scope E2E retries to binary download, probe CDN endpoint, prune stale binaries
Address review feedback on #1045: - Retry only the VS Code binary download (its own step), so genuine test failures fail fast instead of re-running the whole suite. - Probe the archive CDN endpoint (via the 302-redirect HEAD) in addition to the update API, so the stale-binary fallback also engages when only the CDN host is down. - Restore/save the binary cache with restore+save actions and prune older vscode-linux-x64-* dirs so each saved entry keeps one binary instead of growing with every version bump; skip the save when the stale fallback was used. - Emit a ::warning:: when endpoints are unreachable with no cached binary, and echo why the pass marker is skipped on fallback runs.
1 parent 314f490 commit 7750caa

1 file changed

Lines changed: 65 additions & 15 deletions

File tree

‎.github/workflows/e2e.yml‎

Lines changed: 65 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -45,10 +45,10 @@ jobs:
4545
VERSION=$(node -p 'require("./apps/vscode-e2e/package.json").devDependencies["@types/vscode"]')
4646
echo "version=$VERSION" >> $GITHUB_OUTPUT
4747
48-
- name: Cache VS Code test binary
48+
- name: Restore VS Code test binary cache
4949
if: github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true'
5050
id: vscode-cache
51-
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
51+
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
5252
with:
5353
path: |
5454
apps/vscode-e2e/.vscode-test/
@@ -58,21 +58,34 @@ jobs:
5858
restore-keys: |
5959
vscode-test-${{ runner.os }}-
6060
61-
- name: Probe VS Code update API
62-
if: github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true'
61+
- name: Probe VS Code download endpoints
62+
if: (github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true') && steps.vscode-cache.outputs.cache-hit != 'true'
6363
id: vscode-probe
64+
# The version-resolution API and the archive CDN are different hosts. The
65+
# archive URL 302-redirects to the CDN, so a HEAD probe that follows the
66+
# redirect covers both. The fallback engages when either is unreachable.
6467
run: |
68+
API_OK=false
69+
CDN_OK=false
6570
if curl -sf --max-time 10 https://update.code.visualstudio.com/api/releases/stable > /dev/null; then
71+
API_OK=true
72+
fi
73+
if curl -sfIL --max-time 20 -o /dev/null "https://update.code.visualstudio.com/${{ steps.vscode-ver.outputs.version }}/linux-x64/stable"; then
74+
CDN_OK=true
75+
fi
76+
if [ "$API_OK" = "true" ] && [ "$CDN_OK" = "true" ]; then
6677
echo "reachable=true" >> "$GITHUB_OUTPUT"
6778
else
6879
echo "reachable=false" >> "$GITHUB_OUTPUT"
80+
echo "::warning::VS Code download endpoints unreachable (api=$API_OK, cdn=$CDN_OK); will try the stale cached binary"
6981
fi
7082
7183
# @vscode/test-electron only skips its live version-resolution request when the
7284
# requested version already exists in .vscode-test/. On an exact-key miss it calls
7385
# the update API before its download retry loop, so an unreachable CDN fails the
74-
# job before any test runs. When that API is down, point the runner at the stale
75-
# binary restored above (runTest.ts honors VSCODE_VERSION) so the suite still runs.
86+
# job before any test runs. When the endpoints are down, point the runner at the
87+
# stale binary restored above (runTest.ts honors VSCODE_VERSION) so the suite
88+
# still runs.
7689
- name: Fall back to stale VS Code binary when CDN is unreachable
7790
if: (github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true') && steps.vscode-cache.outputs.cache-hit != 'true' && steps.vscode-probe.outputs.reachable == 'false'
7891
id: vscode-fallback
@@ -81,29 +94,66 @@ jobs:
8194
if [ -n "$STALE" ]; then
8295
echo "VSCODE_VERSION=${STALE##*-}" >> "$GITHUB_ENV"
8396
echo "used=true" >> "$GITHUB_OUTPUT"
84-
echo "VS Code update API is unreachable; falling back to cached VS Code ${STALE##*-}"
97+
echo "VS Code download endpoints are unreachable; falling back to cached VS Code ${STALE##*-}"
8598
else
86-
echo "VS Code update API is unreachable and no cached binary is available; the test step may fail"
99+
echo "::warning::VS Code download endpoints are unreachable and no cached binary is available; the download step will retry but may fail"
87100
fi
88101
89-
- name: Run mocked E2E tests
90-
id: run-e2e
91-
# merge_group and workflow_dispatch always run; cache skip is pull_request only
102+
# Retry only the binary download: version resolution and the archive fetch are
103+
# the network-fragile parts. Genuine test failures should fail fast, so the
104+
# test step itself is deliberately not retried.
105+
- name: Download VS Code test binary
92106
if: github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true'
93-
# Retry the whole step so transient CDN/network blips during version resolution
94-
# or the binary download don't fail the entire merge queue run.
107+
id: vscode-download
95108
run: |
109+
cd apps/vscode-e2e
96110
for attempt in 1 2 3; do
97-
if xvfb-run -a pnpm --filter @roo-code/vscode-e2e test:ci:mock; then
111+
if node -e "const { downloadAndUnzipVSCode } = require('@vscode/test-electron'); const version = process.env.VSCODE_VERSION || require('./package.json').devDependencies['@types/vscode']; downloadAndUnzipVSCode(version).catch((err) => { console.error(err); process.exit(1); });"; then
98112
exit 0
99113
fi
100-
echo "E2E attempt ${attempt} failed"
114+
echo "VS Code download attempt ${attempt} failed"
101115
if [ "$attempt" -lt 3 ]; then
102116
sleep 15
103117
fi
104118
done
105119
exit 1
106120
121+
# restore-keys can bring back older binaries alongside the one just
122+
# downloaded; keep only the effective version so each saved cache entry
123+
# stays at one binary instead of growing with every version bump.
124+
- name: Prune stale VS Code binaries
125+
if: github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true'
126+
run: |
127+
EFFECTIVE="${VSCODE_VERSION:-${{ steps.vscode-ver.outputs.version }}}"
128+
for dir in apps/vscode-e2e/.vscode-test/vscode-linux-x64-*; do
129+
[ -e "$dir" ] || continue
130+
if [ "$(basename "$dir")" != "vscode-linux-x64-$EFFECTIVE" ]; then
131+
echo "Pruning stale VS Code binary $(basename "$dir")"
132+
rm -rf "$dir"
133+
fi
134+
done
135+
136+
# Skip the save when the stale-binary fallback ran: the pinned-version key
137+
# must not be populated with an older binary.
138+
- name: Save VS Code test binary cache
139+
if: (github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true') && steps.vscode-cache.outputs.cache-hit != 'true' && steps.vscode-fallback.outputs.used != 'true'
140+
continue-on-error: true
141+
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
142+
with:
143+
path: |
144+
apps/vscode-e2e/.vscode-test/
145+
key: vscode-test-${{ runner.os }}-${{ steps.vscode-ver.outputs.version }}-v1
146+
147+
- name: Run mocked E2E tests
148+
id: run-e2e
149+
# merge_group and workflow_dispatch always run; cache skip is pull_request only
150+
if: github.event_name != 'pull_request' || steps.e2e-marker.outputs.cache-hit != 'true'
151+
run: xvfb-run -a pnpm --filter @roo-code/vscode-e2e test:ci:mock
152+
153+
- name: Explain skipped mocked E2E pass marker
154+
if: steps.e2e-marker.outputs.cache-hit != 'true' && steps.run-e2e.outcome == 'success' && steps.vscode-fallback.outputs.used == 'true'
155+
run: echo "Skipping mocked E2E pass marker because tests ran against a stale cached VS Code binary (VS Code download endpoints unreachable)."
156+
107157
- name: Write mocked E2E pass marker
108158
# Skip when the stale-binary fallback ran: a pass against an older VS Code
109159
# must not mint a marker for the intended-version source hash.

0 commit comments

Comments
 (0)