Skip to content

Commit 2c48986

Browse files
data-douserCopilot
data-douser
and
Copilot
authored
Ensure cross-platform support via client integration tests run on ubuntu-latest and windows-latest (#22)
* Initial plan * fix: cross-platform support for Windows compatibility - Use pathToFileURL() instead of file:// string concatenation in: - server/src/ql-mcp-server.ts (entrypoint check) - server/src/tools/codeql/language-server-eval.ts (workspace URI) - client/src/ql-mcp-client.js (entrypoint check) - Fix path.includes() to normalize separators in cli-tool-registry.ts - Replace .split('/').pop() with path.basename() for cross-platform path handling - Update client-integration-tests.yml to run on matrix of ubuntu-latest and windows-latest - Add platform-specific CI steps for process cleanup and OS dependencies - Update setup-codeql-environment action with Windows cache paths Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> * refactor: improve path normalization performance using replace() instead of split/join Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> * Bash on Windows as an actions workflow fix... * Initial plan * Fix Windows integration test: use bash shell for CodeQL CLI check Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> * fix: create Windows-compatible codeql.cmd wrapper On Windows, gh codeql install-stub creates a bash script (codeql) which is not discoverable by Node.js child_process.spawn(). This causes 'spawn codeql ENOENT' errors in integration tests. Add a step that creates a codeql.cmd wrapper delegating to 'gh codeql' so that spawn('codeql', ...) resolves correctly on Windows. Aligns with github/gh-codeql#21 which adds native Windows support to install-stub. This workaround can be removed once that PR is merged. * Another attempted fix for client-integration-tests on Windows * Yet another attempted fix on Windows * Cleanup OS tmp dir client integration tests * Fixes for PR review comments * More fixes for latest PR review comments --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
1 parent b52db7b commit 2c48986

File tree

18 files changed

+224
-71
lines changed

18 files changed

+224
-71
lines changed

‎.github/actions/setup-codeql-environment/action.yml‎

Lines changed: 58 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,8 +75,9 @@ runs:
7575
echo " CodeQL Version: $CODEQL_VERSION"
7676
echo " Cache Key: $CODEQL_CACHE_KEY"
7777
78-
- name: Cache `gh-codeql` extension and CodeQL packages
79-
id: cache-codeql
78+
- name: Cache `gh-codeql` extension and CodeQL packages (Unix)
79+
id: cache-codeql-unix
80+
if: runner.os != 'Windows'
8081
uses: actions/cache@v4
8182
with:
8283
path: |
@@ -86,6 +87,18 @@ runs:
8687
restore-keys: |
8788
gh-codeql-${{ runner.os }}-${{ steps.codeql-version.outputs.codeql-version }}-
8889
90+
- name: Cache `gh-codeql` extension and CodeQL packages (Windows)
91+
id: cache-codeql-windows
92+
if: runner.os == 'Windows'
93+
uses: actions/cache@v4
94+
with:
95+
path: |
96+
~\AppData\Local\GitHub\gh-codeql
97+
~\.codeql\packages
98+
key: ${{ steps.codeql-version.outputs.codeql-cache-key }}
99+
restore-keys: |
100+
gh-codeql-${{ runner.os }}-${{ steps.codeql-version.outputs.codeql-version }}-
101+
89102
# Install GitHub CLI CodeQL extension and set `codeql` CLI version
90103
- name: Install GitHub CLI CodeQL extension and set version
91104
id: install-gh-codeql
@@ -136,6 +149,49 @@ runs:
136149
137150
echo "✅ GitHub CLI CodeQL extension installed successfully"
138151
152+
# On Windows, gh codeql install-stub creates a bash script which is not
153+
# discoverable by Node.js child_process.spawn() or execFile(), since
154+
# these functions only resolve real executables (.exe), not scripts.
155+
# Find the actual codeql.exe binary from the gh-codeql distribution
156+
# and add its directory to PATH so that spawn('codeql', ...) works.
157+
# This workaround can be removed once github/gh-codeql#21 is merged,
158+
# which adds native Windows support to install-stub.
159+
- name: Add CodeQL binary directory to PATH (Windows)
160+
if: runner.os == 'Windows'
161+
shell: bash
162+
run: |
163+
echo "🔧 Locating actual codeql.exe binary for Windows compatibility..."
164+
165+
# The gh-codeql extension stores the CodeQL CLI binary under the
166+
# GitHub CLI extensions directory. On Windows runners this is:
167+
# $LOCALAPPDATA/GitHub CLI/extensions/gh-codeql/dist/release/<version>/
168+
GH_EXTENSIONS_DIR="${LOCALAPPDATA:-$HOME/AppData/Local}/GitHub CLI/extensions/gh-codeql"
169+
170+
if [ ! -d "$GH_EXTENSIONS_DIR" ]; then
171+
echo "⚠️ gh-codeql extensions directory not found at: $GH_EXTENSIONS_DIR"
172+
echo "Searching more broadly under LOCALAPPDATA..."
173+
GH_EXTENSIONS_DIR="${LOCALAPPDATA:-$HOME/AppData/Local}"
174+
fi
175+
176+
# Find the codeql.exe binary in the gh-codeql distribution
177+
CODEQL_EXE=$(find "$GH_EXTENSIONS_DIR" -name "codeql.exe" -type f 2>/dev/null | head -1)
178+
179+
if [ -z "$CODEQL_EXE" ]; then
180+
echo "❌ Error: codeql.exe not found under $GH_EXTENSIONS_DIR"
181+
echo "Directory listing (top 30 files):"
182+
find "$GH_EXTENSIONS_DIR" -maxdepth 5 -type f 2>/dev/null | head -30
183+
exit 1
184+
fi
185+
186+
CODEQL_BIN_DIR=$(dirname "$CODEQL_EXE")
187+
echo "Found codeql.exe at: $CODEQL_EXE"
188+
echo "Adding $CODEQL_BIN_DIR to PATH"
189+
190+
# Prepend the directory containing codeql.exe to PATH
191+
echo "$CODEQL_BIN_DIR" >> "$GITHUB_PATH"
192+
193+
echo "✅ Added CodeQL binary directory to PATH for Windows"
194+
139195
- name: Setup CodeQL environment variables
140196
id: setup-codeql-env
141197
shell: bash

‎.github/agents/ql-mcp-tool-tester.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ My `ql-mcp-tool-tester` agent:
2525
- NEVER makes anything up about CodeQL CLI behavior or MCP protocol.
2626
- NEVER modifies the MCP server or client code; focuses solely on testing and validating the tools/primitives.
2727
- NEVER "pipes" or redirects `npm test` or `npm run test*` command outputs in any way. Just observe the raw output and use exit codes to determine success/failure.
28+
- **NEVER uses `os.tmpdir()`, `/tmp`, or any OS-level temporary directory** in test code, fixtures, or tool invocations. The OS temp directory is world-readable and triggers CWE-377/CWE-378 vulnerabilities. All temporary files MUST use the project-local `<repoRoot>/.tmp/` directory. In integration test fixtures the `{{tmpdir}}` placeholder resolves to this project-local directory at runtime — it does NOT resolve to the OS temp directory.
2829

2930
## Related Skills
3031

‎.github/instructions/server_test_ts.instructions.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,3 +35,4 @@ This file contains instructions for working with TypeScript test files in the `s
3535
- NEVER write tests that depend on external resources or network calls without proper mocking.
3636
- NEVER write overly complex tests that test multiple concerns in a single test case.
3737
- NEVER skip writing tests for new functionality or bug fixes.
38+
- **NEVER use `os.tmpdir()`, `/tmp`, or any OS-level temporary directory** in test code or test fixtures. The OS temp directory is world-readable and triggers CWE-377/CWE-378 vulnerabilities. Instead, ALWAYS use the project-local `.tmp/` directory via `getProjectTmpDir()`, `createProjectTempDir()`, or `getProjectTmpBase()` from `server/src/utils/temp-dir.ts`. For integration test fixtures, use the `{{tmpdir}}` placeholder which resolves at runtime to `<repoRoot>/.tmp/`.

‎.github/workflows/client-integration-tests.yml‎

Lines changed: 57 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,17 +6,17 @@ on:
66
paths:
77
- '.github/actions/setup-codeql-environment/action.yml'
88
- '.github/workflows/client-integration-tests.yml'
9-
- '.node-version'
109
- '.codeql-version'
10+
- '.node-version'
1111
- 'client/**'
1212
- 'server/**'
1313
pull_request:
1414
branches: [main]
1515
paths:
1616
- '.github/actions/setup-codeql-environment/action.yml'
1717
- '.github/workflows/client-integration-tests.yml'
18-
- '.node-version'
1918
- '.codeql-version'
19+
- '.node-version'
2020
- 'client/**'
2121
- 'server/**'
2222
workflow_dispatch:
@@ -26,7 +26,13 @@ permissions:
2626

2727
jobs:
2828
integration-tests:
29-
runs-on: ubuntu-latest
29+
name: Integration Tests (${{ matrix.os }})
30+
runs-on: ${{ matrix.os }}
31+
32+
strategy:
33+
fail-fast: false
34+
matrix:
35+
os: [ubuntu-latest, windows-latest]
3036

3137
env:
3238
HTTP_HOST: 'localhost'
@@ -44,9 +50,14 @@ jobs:
4450
cache: 'npm'
4551
node-version-file: '.node-version'
4652

47-
- name: MCP Integration Tests - Install OS dependencies
53+
- name: MCP Integration Tests - Install OS dependencies (Ubuntu)
54+
if: runner.os == 'Linux'
4855
run: sudo apt-get install -y jq
4956

57+
- name: MCP Integration Tests - Install OS dependencies (Windows)
58+
if: runner.os == 'Windows'
59+
run: choco install jq -y
60+
5061
- name: MCP Integration Tests - Install node dependencies for client and server workspaces
5162
run: npm ci --workspace=client && npm ci --workspace=server
5263

@@ -55,44 +66,80 @@ jobs:
5566
with:
5667
install-language-runtimes: false
5768

69+
## Verify that the CodeQL CLI is spawnable from Node.js, not just from
70+
## bash. On Windows, Node.js spawn()/execFile() require a real .exe
71+
## binary on PATH, not a bash stub or .cmd wrapper. Fail fast here
72+
## instead of waiting for integration tests to time out.
73+
- name: MCP Integration Tests - Verify CodeQL CLI is spawnable from Node.js
74+
shell: bash
75+
run: |
76+
node -e "
77+
const { execFile } = require('child_process');
78+
execFile('codeql', ['version', '--format=terse'], (err, stdout) => {
79+
if (err) {
80+
console.error('❌ CodeQL CLI is not spawnable from Node.js:', err.message);
81+
console.error('This typically means codeql.exe is not on PATH (Windows).');
82+
process.exit(1);
83+
}
84+
console.log('✅ CodeQL CLI is spawnable from Node.js, version:', stdout.trim());
85+
});
86+
"
87+
5888
## Install packs used in the integration tests.
5989
- name: MCP Integration Tests - Install CodeQL packs
90+
shell: bash
6091
run: ./server/scripts/install-packs.sh
6192

6293
## Extract test databases used in the integration tests.
6394
- name: MCP Integration Tests - Extract test databases
95+
shell: bash
6496
run: ./server/scripts/extract-test-databases.sh
6597

98+
## Configure npm to use bash for running scripts on Windows, since the
99+
## integration test scripts are bash scripts that cmd.exe cannot execute.
100+
- name: MCP Integration Tests - Configure npm script shell (Windows)
101+
if: runner.os == 'Windows'
102+
shell: bash
103+
run: npm config set script-shell "$(which bash)"
104+
66105
## Run integration tests. This script builds the server bundle and runs tests.
67106
## We do NOT use 'npm run build-and-test' as it runs query unit tests which
68107
## have a dedicated workflow (query-unit-tests.yml).
69108
- name: MCP Integration Tests - Run integration tests
109+
shell: bash
70110
run: npm run test:integration --workspace=client
71111

72112
- name: MCP Integration Tests - Stop the background MCP server process
73113
if: always()
114+
shell: bash
74115
run: |
75116
if [ -f server.pid ]; then
76117
PID=$(cat server.pid)
77118
echo "Stopping server with PID $PID"
78-
if kill -0 $PID 2>/dev/null; then
79-
kill $PID || true
119+
if kill -0 "$PID" 2>/dev/null; then
120+
kill "$PID" || true
80121
sleep 2
81122
# Force kill if still running
82-
if kill -0 $PID 2>/dev/null; then
123+
if kill -0 "$PID" 2>/dev/null; then
83124
echo "Force killing server process"
84-
kill -9 $PID || true
125+
kill -9 "$PID" || true
85126
fi
86127
else
87128
echo "Server process was not running"
88129
fi
89-
rm server.pid
130+
rm -f server.pid
90131
else
91132
echo "No server.pid file found"
92133
fi
93134
94135
# Clean up log files
95136
if [ -f server.log ]; then
96137
echo "Removing server.log"
97-
rm server.log
138+
rm -f server.log
98139
fi
140+
141+
- name: MCP Integration Tests - Summary
142+
shell: bash
143+
run: |
144+
echo "## Integration Tests Summary (${{ matrix.os }})" >> $GITHUB_STEP_SUMMARY
145+
echo "✅ MCP server integration tests passed on ${{ matrix.os }}" >> $GITHUB_STEP_SUMMARY

‎client/integration-tests/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,15 +38,15 @@ Common mistakes to avoid:
3838
- ❌ **DO NOT** commit files like `evaluator-log.json`, `query-results.bqrs`, `*.bqrs` files to the repository root
3939
- ❌ **DO NOT** commit temporary files created during integration test development to the repository root
4040
- ✅ **DO** place all test output files in the appropriate `client/integration-tests/primitives/tools/<tool_name>/<test_name>/after/` directory
41-
- ✅ **DO** use `/tmp/` paths for temporary files during development and testing
41+
- ✅ **DO** use `{{tmpdir}}` as a placeholder for the project-local temporary directory in test fixture paths (resolved at runtime to `<repoRoot>/.tmp/`)
4242

4343
**File generation best practices:**
4444

4545
1. **Generate test files correctly**: When creating integration tests that involve file generation (e.g., BQRS files, evaluator logs):
4646
- Run the actual tool command to generate authentic files
4747
- Copy the generated files from their temporary location to the correct `after/` directory
4848
- Never fabricate or "make up" binary file contents
49-
2. **Use proper paths**: Always use absolute paths or `/tmp/` paths when running commands that generate files during development
49+
2. **Use proper paths**: Always use `{{tmpdir}}/` as a placeholder for the project-local temp directory in test fixture JSON files (e.g., `"output": "{{tmpdir}}/results.sarif"`). This resolves at runtime to `<repoRoot>/.tmp/`, **not** the OS temp directory, to avoid CWE-377/CWE-378 (world-readable temp files).
5050
3. **Verify placement**: Before committing, verify that generated files are in the correct `after/` directory, not in the repository root
5151

5252
The `.gitignore` file has been updated to help prevent accidental commits of common integration test output files in the root directory.

‎client/integration-tests/primitives/tools/codeql_database_analyze/analyze_javascript_database/before/monitoring-state.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,6 @@
44
"database": "server/ql/javascript/examples/test/ExampleQuery1/ExampleQuery1.testproj",
55
"queries": "server/ql/javascript/examples/src/ExampleQuery1/ExampleQuery1.ql",
66
"format": "sarif-latest",
7-
"output": "/tmp/integration-test-analyze-results.sarif"
7+
"output": "{{tmpdir}}/integration-test-analyze-results.sarif"
88
}
99
}

‎client/integration-tests/primitives/tools/codeql_database_create/create_javascript_database/before/monitoring-state.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"sessions": [],
33
"parameters": {
4-
"database": "/tmp/codeql-integration-test-db",
4+
"database": "{{tmpdir}}/codeql-integration-test-db",
55
"language": "javascript",
66
"source-root": "server/ql/javascript/examples/test/ExampleQuery1",
77
"overwrite": true

‎client/integration-tests/primitives/tools/codeql_query_run/evaluator_logging_with_tuple_counting/after/monitoring-state.json‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,17 +10,17 @@
1010
"arguments": {
1111
"query": "client/integration-tests/static/javascript/src/ExampleQuery1/ExampleQuery1.ql",
1212
"database": "client/integration-tests/static/javascript/test/ExampleQuery1/ExampleQuery1.testproj",
13-
"output": "/tmp/test-query-results.bqrs",
14-
"evaluator-log": "/tmp/test-evaluator-log.json",
13+
"output": "{{tmpdir}}/test-query-results.bqrs",
14+
"evaluator-log": "{{tmpdir}}/test-evaluator-log.json",
1515
"evaluator-log-level": 5,
1616
"tuple-counting": true
1717
},
1818
"response": {
1919
"stdout": "Evaluation completed successfully",
2020
"stderr": "WARNING: Ignoring local version because local version support is off.",
2121
"filesGenerated": [
22-
"/tmp/test-query-results.bqrs",
23-
"/tmp/test-evaluator-log.json"
22+
"{{tmpdir}}/test-query-results.bqrs",
23+
"{{tmpdir}}/test-evaluator-log.json"
2424
]
2525
}
2626
}

‎client/integration-tests/primitives/tools/codeql_query_run/evaluator_logging_with_tuple_counting/test-config.json‎

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎client/integration-tests/primitives/tools/create_codeql_query/create_javascript_query/before/monitoring-state.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"sessions": [],
33
"parameters": {
4-
"basePath": "/tmp/codeql-test-query",
4+
"basePath": "{{tmpdir}}/codeql-test-query",
55
"queryName": "TestQuery",
66
"language": "javascript",
77
"description": "Integration test query"

0 commit comments

Comments
 (0)