Skip to content

Commit de790bb

Browse files
MichaelRFairhurstMichaelRFairhurst
authored
Add a new prompt & tool for diagnosing FPs/FNs from query runs. (#70)
1 parent bdfa53d commit de790bb

File tree

21 files changed

+4253
-127
lines changed

21 files changed

+4253
-127
lines changed

‎client/integration-tests/primitives/tools/read_database_source/list_source_files/README.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# `read_database_source` - list_source_files
2+
3+
## Purpose
4+
5+
Tests the `read_database_source` tool in listing mode by omitting `filePath` to list all source files in a CodeQL database source archive.
6+
7+
## Inputs
8+
9+
- **database**: Path to the JavaScript example test database
10+
11+
## Expected Behavior
12+
13+
The tool should return a listing of all source files in the database's `src/` directory, including `totalEntries`, `returnedEntries`, and `truncated` metadata.
14+
15+
## Static Files Referenced
16+
17+
- `server/ql/javascript/examples/test/ExampleQuery1/ExampleQuery1.testproj`

‎client/integration-tests/primitives/tools/read_database_source/list_source_files/after/monitoring-state.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"sessions": [
3+
{
4+
"id": "integration_test_session",
5+
"calls": [
6+
{
7+
"tool": "read_database_source",
8+
"timestamp": "2025-09-25T16:06:00.000Z",
9+
"status": "success"
10+
}
11+
]
12+
}
13+
]
14+
}

‎client/integration-tests/primitives/tools/read_database_source/list_source_files/before/monitoring-state.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"sessions": [],
3+
"parameters": {
4+
"databasePath": "server/ql/javascript/examples/test/ExampleQuery1/ExampleQuery1.testproj"
5+
}
6+
}

‎client/integration-tests/primitives/tools/read_database_source/read_source_file/README.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# `read_database_source` - read_source_file
2+
3+
## Purpose
4+
5+
Tests the `read_database_source` tool in file-read mode by requesting a specific source file from a CodeQL database source archive.
6+
7+
## Inputs
8+
9+
- **database**: Path to the JavaScript example test database
10+
- **filePath**: Path to a source file within the database archive
11+
12+
## Expected Behavior
13+
14+
The tool should return the contents of the requested source file along with metadata including `entryPath`, `sourceType`, `totalLines`, `startLine`, and `endLine`.
15+
16+
## Static Files Referenced
17+
18+
- `server/ql/javascript/examples/test/ExampleQuery1/ExampleQuery1.testproj`

‎client/integration-tests/primitives/tools/read_database_source/read_source_file/after/monitoring-state.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"sessions": [
3+
{
4+
"id": "integration_test_session",
5+
"calls": [
6+
{
7+
"tool": "read_database_source",
8+
"timestamp": "2025-09-25T16:06:00.000Z",
9+
"status": "success"
10+
}
11+
]
12+
}
13+
]
14+
}

‎client/integration-tests/primitives/tools/read_database_source/read_source_file/before/monitoring-state.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"sessions": [],
3+
"parameters": {
4+
"databasePath": "server/ql/javascript/examples/test/ExampleQuery1/ExampleQuery1.testproj",
5+
"filePath": "ExampleQuery1.js"
6+
}
7+
}

‎docs/ql-mcp/prompts.md‎

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,23 @@
44
55
## Overview
66

7-
The server provides **10 prompts** that guide AI assistants through common CodeQL development workflows. Each prompt is backed by a `*.prompt.md` file containing structured instructions.
7+
The server provides **11 prompts** that guide AI assistants through common CodeQL development workflows. Each prompt is backed by a `*.prompt.md` file containing structured instructions.
88

99
## Prompt Reference
1010

11-
| Prompt | Description |
12-
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------- |
13-
| `document_codeql_query` | Create or update documentation for a CodeQL query |
14-
| `explain_codeql_query` | Generate a detailed explanation of a CodeQL query for workshop learning content |
15-
| `ql_lsp_iterative_development` | Iterative CodeQL query development using LSP tools for completion, navigation, and validation |
16-
| `ql_tdd_advanced` | Advanced test-driven CodeQL development with AST visualization, control flow, and call graph analysis |
17-
| `ql_tdd_basic` | Test-driven CodeQL query development checklist — write tests first, implement query, iterate until tests pass |
18-
| `sarif_rank_false_positives` | Analyze SARIF results to identify likely false positives in CodeQL query results |
19-
| `sarif_rank_true_positives` | Analyze SARIF results to identify likely true positives in CodeQL query results |
20-
| `test_driven_development` | Test-driven development workflow for CodeQL queries using MCP tools |
21-
| `tools_query_workflow` | Guide for using built-in tools queries (PrintAST, PrintCFG, CallGraphFrom, CallGraphTo) to understand code structure |
22-
| `workshop_creation_workflow` | Guide for creating CodeQL query development workshops from production-grade queries |
11+
| Prompt | Description |
12+
| ----------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
13+
| `document_codeql_query` | Create or update documentation for a CodeQL query |
14+
| `explain_codeql_query` | Generate a detailed explanation of a CodeQL query for workshop learning content |
15+
| `ql_lsp_iterative_development` | Iterative CodeQL query development using LSP tools for completion, navigation, and validation |
16+
| `ql_tdd_advanced` | Advanced test-driven CodeQL development with AST visualization, control flow, and call graph analysis |
17+
| `ql_tdd_basic` | Test-driven CodeQL query development checklist — write tests first, implement query, iterate until tests pass |
18+
| `run_query_and_summarize_false_positives` | Run a CodeQL query and summarize its false positives |
19+
| `sarif_rank_false_positives` | Analyze SARIF results to identify likely false positives in CodeQL query results |
20+
| `sarif_rank_true_positives` | Analyze SARIF results to identify likely true positives in CodeQL query results |
21+
| `test_driven_development` | Test-driven development workflow for CodeQL queries using MCP tools |
22+
| `tools_query_workflow` | Guide for using built-in tools queries (PrintAST, PrintCFG, CallGraphFrom, CallGraphTo) to understand code structure |
23+
| `workshop_creation_workflow` | Guide for creating CodeQL query development workshops from production-grade queries |
2324

2425
## Prompt Categories
2526

@@ -41,6 +42,7 @@ The server provides **10 prompts** that guide AI assistants through common CodeQ
4142
### Documentation and Quality
4243

4344
- **`document_codeql_query`** — Generates standardized markdown documentation as a sibling file to a query.
45+
- **`run_query_and_summarize_false_positives`** — Runs a CodeQL query on a database and groups results into false-positive categories by root cause.
4446
- **`sarif_rank_false_positives`** / **`sarif_rank_true_positives`** — Help assess query precision by ranking SARIF results.
4547

4648
### Workshop Creation

‎extensions/vscode/test/fixtures/multi-root-workspace/folder-a/codeql-storage/queries/SqlInjection.ql-def456/query.log‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[2026-01-20 15:00:00] [SPAMMY] execute query-server2> runQuery called with /workspace/src/SqlInjection.ql
2+
[2026-01-20 15:00:01] Calling plumbing command: codeql resolve upgrades --dbscheme=/databases/test-python-db/db-python/semmlecode.python.dbscheme --format=json

‎package-lock.json‎

Lines changed: 21 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)