Commit 3bd0471
* Resolve database lock contention w/ vscode-codeql
Resolves #117
Fixes a known compatibility issue for databases added, and therefore
locked, via the GitHub.vscode-codeql extension.
The vscode-codeql query server creates .lock files in the cache
directory of every registered CodeQL database, preventing the ql-mcp
server from running CLI commands (codeql_query_run,
codeql_database_analyze) against those same databases.
Add a DatabaseCopier that syncs databases from vscode-codeql storage
into a managed directory under the `vscode-codeql-development-mcp-server`
extension's globalStorage, stripping .lock files from the copy. The
EnvironmentBuilder now sets CODEQL_DATABASES_BASE_DIRS to this managed
directory by default (configurable via codeql-mcp.copyDatabases).
- New DatabaseCopier class with incremental sync (skips unchanged databases)
- StoragePaths.getManagedDatabaseStoragePath() for the managed databases/ dir
- EnvironmentBuilder accepts injectable DatabaseCopierFactory for testability
- codeql-mcp.copyDatabases setting (default: true)
- 11 unit tests for DatabaseCopier (real filesystem operations)
- 15 unit tests for EnvironmentBuilder (updated for copy mode + fallback)
- 3 bridge integration tests (managed dir structure, no .lock files)
- 4 E2E integration tests: inject .lock → copy → codeql_query_run +
codeql_database_analyze succeed against the lock-free copy
* Address PR review comments
* Address more PR review comments
* Add search_ql_code and codeql_resolve_files tools
Add search_ql_code and codeql_resolve_files tools in order to
eliminate grep/CLI dependencies.
- New tools: search_ql_code (QL text/regex search) and codeql_resolve_files
(file discovery by extension/glob) so LLMs never need shell access
- Rewrite profile_codeql_query_from_logs with two-tier design: compact
inline JSON + line-indexed detail file for targeted read_file access;
parser now captures RA operations and pipeline-stage tuple progressions
- Fix codeql_resolve_database to probe child directories for databases
- Remove all grep/CLI references from prompts and resources
- Cross-platform: normalize \r\n line endings in parser and search tool
* Add "after" files for query evaluation integration tests
* address Code Scanning TOCTOU race and PR review feedback
- Eliminate filesystem race condition in search-ql-code.ts (read-then-check
instead of stat-then-read)
- Add symlink cycle detection using lstatSync and visited-path tracking
- Fix tool description field names in profile-codeql-query-from-logs.ts
({startLine,endLine} → detailLines: {start,end})
- Fix monitoring-state.json fixtures to use standard sessions format
- Rename find_qll_files → find_ql_files to match actual .ql extension
* Stream large files instead of loading into memory
- addresses latest review feedback for PR #119
- search-ql-code: check file size via lstatSync before reading; stream
large files (>5 MB) line-by-line instead of skipping them
- evaluator-log-parser: replace readFileSync with streaming async
generator (createReadStream + readline) for brace-depth JSON parsing;
parseEvaluatorLog now reads the file once instead of twice
- profile-codeql-query: convert local parser to streaming with Map-based
lookups instead of O(n) events.find()
- database-copier: use lstat in removeLockFiles to skip symlinks; throw
on fatal mkdir failures for proper fallback in EnvironmentBuilder
- Validate contextLines/maxResults with schema bounds and clamping
- Add environment-builder test for syncAll-throws fallback
* Fix tool issues found during explain-codeql-query workflow testing
- search_ql_code: add missing await in tool handler; skip .codeql,
node_modules, and .git directories to avoid duplicate results from
compiled pack caches
- cli-tool-registry: extract resolveDatabasePath helper for multi-language
DB root auto-resolution; apply to codeql_query_run, codeql_database_analyze,
and codeql_resolve_database
- environment-builder: route CODEQL_MCP_TMP_DIR to workspace-local
.codeql/ql-mcp scratch directory (configurable via scratchDir setting);
add CODEQL_MCP_WORKSPACE_FOLDERS env var
- query-file-finder: add contextual hints array for missing tests,
documentation, and expected results
* [UPDATE PRIMITIVE] Fix transient HTTP 503 failures in install-packs.sh via exponential backoff retry (#121)
* Initial plan
* fix: add retry logic with exponential backoff to install-packs.sh
The GitHub Actions integration test was failing on windows-latest with
HTTP 503 "Egress is over the account limit" when downloading CodeQL
packs from GHCR.io.
Add a run_with_retry() helper function that retries a command up to 3
times with exponential backoff (10s, 20s, 40s). Both codeql pack
install calls in install_packs() now use run_with_retry to handle
transient network errors gracefully.
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
* deterministic profiler output and search efficiency
- addresses latest feedback for PR #119 ;
- profile-codeql-query-from-logs: remove non-deterministic `Generated:`
timestamp from detail file header to ensure reproducible output for
integration test fixtures ;
- search-ql-code: early-exit file processing once maxResults matches are
collected; subsequent files are scanned cheaply for totalMatches count
only, avoiding large array allocations and context extraction ;
* Fix TOCTOU bug for search_ql_code tool
* Stream-count large files & detect ambiguous DB paths
- search-ql-code: use streaming (readline) for totalMatches counting on
large files in the early-exit path; eliminates TOCTOU race from prior
lstatSync check
- cli-tool-registry: resolveDatabasePath now collects all candidate
children and throws on ambiguity instead of silently picking the first
- Add tests for cross-file totalMatches accuracy under truncation, single-
child DB auto-resolve, and multi-child DB ambiguity error
* Address latest PR review comments
* Use fstatSync(fd) to avoid OOM w/ searchFile
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
1 parent bb2cc42 commit 3bd0471
File tree
51 files changed
+4530
-954
lines changed- .github/instructions
- client/integration-tests/primitives/tools
- codeql_resolve_files/find_ql_files
- after
- before
- profile_codeql_query_from_logs
- multi_query_raw_log/after
- single_query_raw_log/after
- search_ql_code/search_predicate_name
- after
- before
- extensions/vscode
- src/bridge
- test
- bridge
- suite
- server
- dist
- scripts
- src
- lib
- prompts
- resources
- tools
- codeql
- test/src
- lib
- tools
- codeql
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
51 files changed
+4530
-954
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | 27 | | |
32 | 28 | | |
33 | 29 | | |
34 | 30 | | |
35 | | - | |
| 31 | + | |
| 32 | + | |
Lines changed: 26 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
Lines changed: 14 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
Lines changed: 7 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
Lines changed: 62 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
Lines changed: 62 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
18 | 30 | | |
19 | 31 | | |
20 | 32 | | |
| |||
25 | 37 | | |
26 | 38 | | |
27 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
28 | 52 | | |
29 | 53 | | |
30 | 54 | | |
| |||
42 | 66 | | |
43 | 67 | | |
44 | 68 | | |
45 | | - | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
46 | 82 | | |
47 | 83 | | |
48 | 84 | | |
| |||
53 | 89 | | |
54 | 90 | | |
55 | 91 | | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
56 | 104 | | |
57 | 105 | | |
58 | 106 | | |
| |||
65 | 113 | | |
66 | 114 | | |
67 | 115 | | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
68 | 128 | | |
69 | 129 | | |
70 | 130 | | |
| |||
Lines changed: 154 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
0 commit comments