Skip to content

Commit f2dc456

Browse files
authored
docs: ROADMAP from open issues, gh CLI workflow; parsing: diff-prefix repair (#28) (#43)
- docs/ROADMAP.md: Backlog from open issues (high/medium/low), gh issue list/edit workflow, labels (priority: *, area: *). Shipped section notes #20 secrets done, #28 parsing fallbacks in progress, #23 verification in place. - parsing: In repair_json_candidates(), add candidate that strips leading '+' from each line when LLM echoes JSON with diff-style prefixes (issue #28). Test: parse_json_with_diff_prefix_artifact. - CLAUDE.md: Link to docs/ and ROADMAP.md. - README Contributing: Link to ROADMAP and gh issue list for triage. - GitHub: Added labels priority: high|medium|low, area: * to open issues via gh. Made-with: Cursor
1 parent f4e9526 commit f2dc456

File tree

4 files changed

+117
-6
lines changed

4 files changed

+117
-6
lines changed

CLAUDE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ cargo run -- --help # CLI usage
2020
- `migrations/` — PostgreSQL migrations (sqlx)
2121
- `eval/` — Evaluation and benchmarking
2222
- `examples/` — Usage examples
23+
- `docs/` — Mutation testing, **ROADMAP.md** (enhancement backlog + gh CLI workflow)
2324

2425
## Development
2526
- Run `bash scripts/install-hooks.sh` once to install pre-commit and pre-push hooks (merge conflict check, trailing newline, conditional Rust/web checks, shellcheck; pre-push adds version sync, cargo audit, full web build+test). Mutation testing runs in CI only; see `docs/mutation-testing.md`.

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1000,7 +1000,7 @@ The summary includes:
10001000

10011001
## Contributing
10021002

1003-
Contributions are welcome! Please open an issue first to discuss what you would like to change.
1003+
Contributions are welcome! Please open an issue first to discuss what you would like to change. Enhancement backlog and triage: see [docs/ROADMAP.md](docs/ROADMAP.md) and `gh issue list --label "priority: high"`.
10041004

10051005
**PR workflow:** Open a PR → ensure CI is green (version, lint, security, test, mutation, review) → merge when ready. Use a short test plan in the PR description. Small, focused PRs are preferred.
10061006

docs/ROADMAP.md

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# DiffScope roadmap
2+
3+
Enhancement backlog derived from open GitHub issues. Use **gh CLI** for triage and filtering.
4+
5+
## gh CLI workflow
6+
7+
```bash
8+
# List open issues (default: 30)
9+
gh issue list
10+
11+
# Filter by label
12+
gh issue list --label "priority: high"
13+
gh issue list --label "area: review-pipeline"
14+
15+
# Search (advanced filters)
16+
gh issue list --search "no:assignee sort:created-asc"
17+
gh issue list --search "verification OR RAG"
18+
19+
# View one issue
20+
gh issue view 32
21+
22+
# Add/remove labels (labels must exist: gh label create "name" --color "hex" --description "desc")
23+
gh issue edit 32 --add-label "priority: medium,area: plugins"
24+
gh issue edit 32 --remove-label "help wanted"
25+
26+
# Add to project (requires project scope)
27+
gh issue edit 32 --add-project "Roadmap"
28+
```
29+
30+
Create labels once: `priority: high`, `priority: medium`, `priority: low`, `area: review-pipeline`, `area: plugins`, `area: platform`.
31+
32+
---
33+
34+
## Priority: High / Critical
35+
36+
| # | Title | Area | Notes |
37+
|---|--------|------|--------|
38+
| [27](https://github.com/evalops/diffscope/issues/27) | Embedding-based false positive filtering from developer feedback | review | Greptile-style: block if similar to 3+ downvoted; pass if 3+ upvoted. Per-team. |
39+
| [23](https://github.com/evalops/diffscope/issues/23) | Verification pass to catch hallucinations | review | Second LLM pass validates findings vs actual code; drop below score. **Partially done** (verification in config/pipeline). |
40+
| [22](https://github.com/evalops/diffscope/issues/22) | Embedding-based RAG pipeline with function-level chunking | review | NL summaries + pgvector; highest leverage for catch rate. |
41+
| [24](https://github.com/evalops/diffscope/issues/24) | Agentic review loop with tool use | review | Tools: search_code, read_file, search_symbols, git_log, git_blame. |
42+
| [21](https://github.com/evalops/diffscope/issues/21) | Multi-agent architecture: review + fix + test agents | platform | Fix Agent, Test Agent, Triage Agent; orchestration. |
43+
| [10](https://github.com/evalops/diffscope/issues/10) | Deep codebase graph context in review prompts | review | Pre-index repo; inject callers/callees/contracts into prompt. |
44+
45+
## Priority: Medium
46+
47+
| # | Title | Area | Notes |
48+
|---|--------|------|--------|
49+
| [32](https://github.com/evalops/diffscope/issues/32) | In-sandbox linter/analyzer execution | plugins | ToolSandbox + AnalysisTool trait; Clippy, Ruff, Gitleaks, ShellCheck, actionlint. |
50+
| [31](https://github.com/evalops/diffscope/issues/31) | AST-based structural pattern matching (ast-grep) | plugins | Pre-analyzer plugin; coderabbitai/ast-grep-essentials rules. |
51+
| [30](https://github.com/evalops/diffscope/issues/30) | Adaptive patch compression for large PRs | review | Full → Compressed → Clipped → MultiCall; token budget. |
52+
| [29](https://github.com/evalops/diffscope/issues/29) | File triage: classify before expensive review | review | NeedsReview vs Cosmetic/ConfigChange/TestOnly; heuristic + cheap model. |
53+
| [28](https://github.com/evalops/diffscope/issues/28) | Robust LLM output parsing with fallback strategies | review | **In progress:** code-block extraction, trailing commas, diff-prefix strip. More fallbacks in parsing/llm_response.rs. |
54+
| [25](https://github.com/evalops/diffscope/issues/25) | Dynamic context: enclosing function/class boundary | review | Search upward for boundary; reuse symbol_index patterns. |
55+
56+
## Priority: Low / Tier 2–3
57+
58+
| # | Title | Area | Notes |
59+
|---|--------|------|--------|
60+
| [20](https://github.com/evalops/diffscope/issues/20) | Built-in secrets detection scanner | plugins | **Done:** `secret_scanner.rs` with AWS, GitHub, Slack, JWT, PEM, etc. |
61+
| [19](https://github.com/evalops/diffscope/issues/19) | Compliance review command | platform | `diffscope compliance` — security, secrets, rules, ticket, licenses, duplication. |
62+
| [18](https://github.com/evalops/diffscope/issues/18) | Authentication layer for web UI (SSO/SAML) | platform | Basic → OAuth/OIDC → SAML, RBAC. |
63+
| [17](https://github.com/evalops/diffscope/issues/17) | GitLab, Azure DevOps, Bitbucket support | platform | GitPlatform trait; GitLab first. |
64+
| [15](https://github.com/evalops/diffscope/issues/15) | Auto-generate Mermaid sequence diagrams in PRs | review | Symbol graph → sequence diagram in PR comment. |
65+
| [14](https://github.com/evalops/diffscope/issues/14) | VS Code / IDE extension | platform | Staged/unstaged review, inline diagnostics, Quick Fix. |
66+
| [13](https://github.com/evalops/diffscope/issues/13) | Ticket validation (Jira/Linear/GitHub Issues) | platform | Fetch ticket, validate acceptance criteria. |
67+
| [12](https://github.com/evalops/diffscope/issues/12) | Natural language custom review rules | review | Prose rules in YAML or `.diffscope-rules`. |
68+
| [11](https://github.com/evalops/diffscope/issues/11) | PR analytics and review metrics dashboard | platform | Persist to PG, aggregation, dashboard. |
69+
| [9](https://github.com/evalops/diffscope/issues/9) | Structured PR description auto-generation | platform | `pr describe` — walkthrough, labels, breaking, testing notes. |
70+
71+
---
72+
73+
## Shipped (recent)
74+
75+
- **LLM parsing (#28):** Repair candidate for diff-style line prefixes (`+` on each line) in `repair_json_candidates`; test `parse_json_with_diff_prefix_artifact`.
76+
- **Secrets (#20):** Built-in secret scanner in `plugins/builtin/secret_scanner.rs`.
77+
- **Verification (#23):** Verification pass and config (verification.*) in pipeline.
78+
79+
---
80+
81+
## References
82+
83+
- [GitHub CLI manual](https://cli.github.com/manual/)
84+
- [Advanced issue search](https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests)
85+
- [gh issue edit](https://cli.github.com/manual/gh_issue_edit) — labels, assignees, projects, milestones

src/parsing/llm_response.rs

Lines changed: 30 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -347,13 +347,27 @@ fn find_balanced_json(content: &str, open: char, close: char) -> Option<String>
347347
fn repair_json_candidates(candidate: &str) -> Vec<String> {
348348
static TRAILING_COMMAS: Lazy<Regex> = Lazy::new(|| Regex::new(r",\s*([}\]])").unwrap());
349349

350-
let mut candidates = vec![candidate.trim().to_string()];
351-
let without_trailing_commas = TRAILING_COMMAS
352-
.replace_all(candidate.trim(), "$1")
353-
.to_string();
354-
if without_trailing_commas != candidate.trim() {
350+
let trimmed = candidate.trim();
351+
let mut candidates = vec![trimmed.to_string()];
352+
353+
let without_trailing_commas = TRAILING_COMMAS.replace_all(trimmed, "$1").to_string();
354+
if without_trailing_commas != trimmed {
355355
candidates.push(without_trailing_commas);
356356
}
357+
358+
// When LLM echoes JSON with diff-style line prefixes (leading "+"), strip them (issue #28).
359+
let without_diff_prefix: String = trimmed
360+
.lines()
361+
.map(|line| line.strip_prefix('+').map(str::trim).unwrap_or(line))
362+
.collect::<Vec<_>>()
363+
.join("\n");
364+
let without_diff_prefix = without_diff_prefix.trim();
365+
if without_diff_prefix != trimmed
366+
&& (without_diff_prefix.starts_with('[') || without_diff_prefix.starts_with('{'))
367+
{
368+
candidates.push(without_diff_prefix.to_string());
369+
}
370+
357371
candidates
358372
}
359373

@@ -1218,6 +1232,17 @@ let data = &input;
12181232
assert_eq!(comments[0].line_number, 5);
12191233
}
12201234

1235+
#[test]
1236+
fn parse_json_with_diff_prefix_artifact() {
1237+
// LLM sometimes echoes JSON in a code block with leading "+" on each line (diff artifact); repair strips (issue #28).
1238+
let input = "```json\n+[{\"line\": 7, \"issue\": \"Missing check\"}]\n+\n```";
1239+
let file_path = PathBuf::from("src/lib.rs");
1240+
let comments = parse_llm_response(input, &file_path).unwrap();
1241+
assert_eq!(comments.len(), 1);
1242+
assert_eq!(comments[0].line_number, 7);
1243+
assert!(comments[0].content.contains("Missing check"));
1244+
}
1245+
12211246
// ── Bug: find_json_array uses mismatched brackets ──────────────────
12221247
//
12231248
// `find_json_array` uses `find('[')` (first) + `rfind(']')` (last).

0 commit comments

Comments
 (0)