Skip to content

Commit e4be811

Browse files
authored
Fix flaky UI test (#684)
* Make more screenshots to find issue. * Fix screenshot upload path. * Error on missing screenshots, indicating wrong path. * Do not fail-fast OS matrix. * Update VS Code and extester. * Wait for synatx highlighting. * Downgrade VS Code. * Revert unrelated changes. * Fix shellcheck warnings. * Always upload screenshots. * Await opening resources. Since 8.17.0, the artificial delay in `openResources` has been removed: redhat-developer/vscode-extension-tester@aba24cc#diff-1af7accc0763e50d55026f156144e4248fd262eb7a6bfddd91851aac41e51ecc * Revert most changes in preparation of squash.
1 parent 9bee119 commit e4be811

4 files changed

Lines changed: 55 additions & 22 deletions

File tree

.github/workflows/build.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,8 +120,9 @@ jobs:
120120
if: failure()
121121
with:
122122
name: screenshots-${{ matrix.os }}
123-
path: ./rascal-vscode-extension/uitests/**/screenshots/*.png
123+
path: ./rascal-vscode-extension/uitests/screenshots/**/*.png
124124
retention-days: 5
125+
if-no-files-found: error
125126

126127
- name: "cleanup before cache"
127128
shell: bash

rascal-vscode-extension/package-lock.json

Lines changed: 49 additions & 17 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

rascal-vscode-extension/src/test/vscode-suite/utils.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -297,7 +297,7 @@ export class IDEOperations {
297297
}
298298

299299
async openModule(file: string): Promise<TextEditor> {
300-
this.browser.openResources(file); // intentionally not waiting, since it sleeps for 3s without anything happening
300+
await this.browser.openResources(file);
301301
return this.driver.wait(async () => {
302302
const result = await ignoreFails(new Workbench().getEditorView().openEditor(path.basename(file))) as TextEditor;
303303
if (result && await ignoreFails(result.getTitle()) === path.basename(file)) {

runUItests.sh

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# !/bin/sh
1+
#!/bin/sh
22

33
# This documents (and performs) necessary commands for local UI testing.
44
# Experts might run these manually on demand. For example, repeatedly
@@ -20,9 +20,9 @@ rm -rf $UITESTS || true
2020
npm run compile-tests
2121

2222
# test what was compiled
23-
VSCODE_VERSION=`grep '"vscode":' package.json | awk -F\^ '{ print $2 }' | awk -F\" '{ print $1 }'`
23+
VSCODE_VERSION=$(grep '"vscode":' package.json | awk -F^ '{ print $2 }' | awk -F\" '{ print $1 }')
2424
echo "Running tests with VSCode version $VSCODE_VERSION"
2525
exec npx extest setup-and-run out/test/vscode-suite/*.test.js \
26-
--code_version ${VSCODE_VERSION} \
26+
--code_version "${VSCODE_VERSION}" \
2727
--storage $UITESTS \
2828
--extensions_dir $UITESTS/extensions_dir

0 commit comments

Comments
 (0)