Problem
The integration-test job in ci.yml currently only runs on macos-latest. Since VS Code is a cross-platform editor and our extension interacts with OS-level features (binary discovery, file system, PATH resolution), integration tests should cover all three supported platforms.
Current test matrix:
| Job |
Ubuntu |
macOS |
Windows |
| unit-test |
yes |
yes |
yes |
| integration-test |
no |
yes |
no |
Proposed changes
- Convert
integration-test to a matrix job with os: [ubuntu-latest, macos-latest, windows-latest]
- Ubuntu: wrap VS Code test commands with
xvfb-run to provide a virtual display (standard pattern for headless GUI testing)
- Windows: verify ExTester and
@vscode/test-electron work on Windows runners; adjust paths if needed
- macOS: keep existing setup, including
hide-test-vscode.sh (which already no-ops on non-Darwin)
Additional cleanup
Remove step-security/harden-runner from the integration-test job. It only supports Ubuntu VMs and prints a confusing warning on macOS:
This job is not running in a GitHub Actions Hosted Runner Ubuntu VM. Harden Runner is only supported on Ubuntu VM. This job will not be monitored.
Since it does nothing on macOS (or Windows), it should be removed from this job entirely. If the job is expanded to a matrix, harden-runner can be conditionally included only for the Ubuntu leg:
- name: Harden runner
if: runner.os == 'Linux'
uses: step-security/harden-runner@...
with:
egress-policy: audit
Cost
All runners are free for public repos. No billing impact.
References
Problem
The
integration-testjob inci.ymlcurrently only runs onmacos-latest. Since VS Code is a cross-platform editor and our extension interacts with OS-level features (binary discovery, file system, PATH resolution), integration tests should cover all three supported platforms.Current test matrix:
Proposed changes
integration-testto a matrix job withos: [ubuntu-latest, macos-latest, windows-latest]xvfb-runto provide a virtual display (standard pattern for headless GUI testing)@vscode/test-electronwork on Windows runners; adjust paths if neededhide-test-vscode.sh(which already no-ops on non-Darwin)Additional cleanup
Remove
step-security/harden-runnerfrom theintegration-testjob. It only supports Ubuntu VMs and prints a confusing warning on macOS:Since it does nothing on macOS (or Windows), it should be removed from this job entirely. If the job is expanded to a matrix, harden-runner can be conditionally included only for the Ubuntu leg:
Cost
All runners are free for public repos. No billing impact.
References