Skip to content

Commit 0369edf

Browse files
moonrunnerkcclaude
andcommitted
fix: resolve all P0/P1/P2 issues from tech debt review
P0: Generated ESLint configs are invalid - Flat config: rule entries use object property syntax with quoted severity strings; plugin keys with special chars are quoted; plugins registered in plugins object - Legacy config: output is valid JSON via JSON.stringify(); unmappable rules appear as comment block after JSON - Added tests that parse generated configs and validate syntax P0: Unsupported parser rules become false passes - All CATEGORY_MAP entries set to null; unmatched categories go to unparseable instead of creating generic rules - Verifiers return {passed: false, skipped: true} for unsupported pattern types instead of silently passing - Added skipped field to RuleResult type P0: Drift and extract async config parsing - parseEslintConfigAsync() supports .js, .mjs, .cjs, .ts files - drift.ts and extract.ts now use async parser - Synchronous parser throws for JS/TS files with actionable message - Added resolveSafePath() to lint-config, drift, and extract commands P0: GitHub Action drift mode broken - Added RULEPROBE_BIN env var for resolved CLI path - Made findEslintFileInWorkspace() async with proper await - Both drift and verify steps in action.yml set RULEPROBE_BIN P1: ESLint rule mapping accuracy - no-wildcard-exports: import/no-namespace -> import/no-anonymous-default-export - no-namespace-imports: @typescript-eslint/consistent-type-imports -> import/no-namespace - no-unused-exports: no-unused-vars -> import/no-unused-modules - no-implicit-any: @typescript-eslint/no-implicit-any -> @typescript-eslint/no-explicit-any - no-deep-relative-imports: now includes maxDepth in options P1: Package output contains stale build artifacts - Added prebuild script (rm -rf dist) to clean before every build - Added files allowlist to package.json P1: README and reference docs stale - Updated README format descriptions and verify format list - Removed removed commands (compare, tasks, task, run) from cli-reference.md - Added lint-config, drift, extract command docs - Updated matchers.md to reflect 34 mappable matchers across 7 categories P2: Path boundary and config validation - resolveSafePath() applied to lint-config, drift, and extract - Config validation now rejects unknown categories and verifiers - verify command accepts all valid report formats (summary, detailed, ci added) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent ad98a6c commit 0369edf

32 files changed

Lines changed: 623 additions & 643 deletions

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,3 +31,6 @@ test-treesitter*.mjs
3131
.github/ruleprobe-e2e-verification-guide.md
3232
docs/ruleprobe-build-guide.md
3333
.ruleprobe-semantic/
34+
.codex
35+
.codex-tmp/
36+
.ruleprobe-semantic/

README.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -65,15 +65,15 @@ ruleprobe parse CLAUDE.md
6565
ruleprobe parse AGENTS.md --show-unparseable
6666
```
6767

68-
**Verify agent output** against extracted rules (legacy mode):
68+
**Verify agent output** against extracted rules (legacy mode). Valid formats: `text`, `json`, `markdown`, `rdjson`, `summary`, `detailed`, `ci`.
6969

7070
```bash
7171
ruleprobe verify CLAUDE.md ./agent-output --format text
7272
```
7373

7474
## What It Does
7575

76-
**Translate.** Reads 7 instruction file formats (CLAUDE.md, AGENTS.md, .cursorrules, copilot-instructions.md, GEMINI.md, .windsurfrules, .rules) and emits an ESLint config. Each extractable rule maps to an ESLint rule with appropriate severity and options. Rules that have no ESLint equivalent appear as comments in the output so you know what wasn't covered.
76+
**Translate.** Reads 7 instruction file formats (CLAUDE.md, AGENTS.md, .cursorrules, copilot-instructions.md, GEMINI.md, .windsurfrules, .rules) and emits an ESLint config. Flat config is the default; pass `--format legacy` for `.eslintrc.json` output. Each extractable rule maps to an ESLint rule with appropriate severity and options. Rules that have no ESLint equivalent appear as comments in the output so you know what wasn't covered.
7777

7878
**Detect drift.** Compares parsed rules against an existing ESLint config. Reports rules in the instruction file but missing from ESLint (you're not enforcing what you said), rules in ESLint but not in the instructions (you're enforcing what you never stated), severity mismatches, and argument differences. Use `--format markdown` for PR-ready output.
7979

@@ -133,7 +133,7 @@ Six commands: `parse`, `verify`, `analyze`, `lint-config`, `drift`, `extract`.
133133

134134
```bash
135135
ruleprobe parse CLAUDE.md --show-unparseable
136-
ruleprobe verify AGENTS.md ./src --format summary --threshold 0.9
136+
ruleprobe verify AGENTS.md ./src --format detailed --threshold 0.9
137137
ruleprobe lint-config CLAUDE.md --format flat --output eslint.config.js
138138
ruleprobe drift CLAUDE.md .eslintrc.json --format markdown
139139
ruleprobe extract .eslintrc.json --output rules-section.md
@@ -247,12 +247,12 @@ The mapper translates extractable rules into ESLint rule entries. The drift dete
247247
| Category | Count | Verifier | Examples |
248248
|----------|------:|----------|----------|
249249
| naming | 5 | AST, Filesystem | camelCase variables, PascalCase types, kebab-case files, UPPER_CASE constants |
250-
| forbidden-pattern | 5 | AST, Regex | no `any`, no `console.log`, no `var`, no `TODO` comments |
251-
| structure | 5 | AST, Filesystem | named exports, JSDoc required, max file length, max line length, no unused exports |
250+
| forbidden-pattern | 5 | AST, Regex | no `any`, no `console.log`, no `var`, max line length, no `TODO` comments |
251+
| structure | 4 | AST, Filesystem | named exports, JSDoc required, max file length, no unused exports |
252252
| import-pattern | 4 | AST | no path aliases, no deep relative imports, no namespace imports, no wildcard exports |
253253
| error-handling | 2 | AST | no empty catch, throw Error only |
254254
| type-safety | 5 | AST, Regex | no enums, no type assertions, no non-null assertions, no implicit any, no ts directives |
255-
| code-style | 8 | AST, Regex | no var, prefer const, no else after return, no nested ternary, no magic numbers, semicolons, quotes, max function length, max params |
255+
| code-style | 9 | AST, Regex | prefer const, no else after return, no nested ternary, no magic numbers, semicolons, quotes, max function length, max params, no TODO comments |
256256
257257
Rules that can't map to ESLint (test file requirements, project config, git conventions) are reported as unmappable so you can enforce them through other tooling.
258258

action.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ runs:
120120
GITHUB_EVENT_NAME: ${{ github.event_name }}
121121
GITHUB_WORKSPACE: ${{ github.workspace }}
122122
GITHUB_OUTPUT: ${{ github.output_file }}
123+
RULEPROBE_BIN: ${{ github.action_path }}/dist/cli.js
123124
run: node "${{ github.action_path }}/dist/action/main.js"
124125

125126
- name: Run ruleprobe verify

docs/cli-reference.md

Lines changed: 21 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -85,65 +85,49 @@ Checks for `CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `.github/copilot-instructio
8585

8686
---
8787

88-
## `ruleprobe compare <instruction-file> <dirs...>`
88+
## `ruleprobe lint-config <instruction-file>`
8989

90-
Run verification against multiple agent outputs and produce a comparison.
90+
Parse an instruction file and emit an ESLint config. Flat config is the default; use `--format legacy` for `.eslintrc.json` output.
9191

9292
```bash
93-
ruleprobe compare AGENTS.md ./claude-output ./copilot-output --agents claude,copilot --format markdown
93+
ruleprobe lint-config CLAUDE.md
94+
ruleprobe lint-config CLAUDE.md --format legacy --output .eslintrc.json
95+
ruleprobe lint-config AGENTS.md --format flat --output eslint.config.js
9496
```
9597

9698
| Option | Default | Description |
9799
|--------|---------|-------------|
98-
| `--agents <names>` | none | Comma-separated labels for each directory |
99-
| `--format <format>` | `markdown` | Report format: `text`, `json`, or `markdown` |
100-
| `--output <path>` | stdout | Write report to file |
101-
| `--allow-symlinks` | `false` | Follow symlinks outside the working directory |
102-
| `--config <path>` | auto-discovered | Path to config file |
100+
| `--format <format>` | `flat` | Output format: `flat` (ESLint flat config) or `legacy` (`.eslintrc.json`) |
101+
| `--output <path>` | stdout | Write config to file |
103102

104103
---
105104

106-
## `ruleprobe tasks`
105+
## `ruleprobe drift <md-file> <eslint-file>`
107106

108-
List available task templates and their descriptions.
107+
Detect drift between an instruction file and an ESLint config. Reports rules present in only one side, severity mismatches, and argument differences.
109108

110109
```bash
111-
ruleprobe tasks
110+
ruleprobe drift CLAUDE.md .eslintrc.json
111+
ruleprobe drift CLAUDE.md eslint.config.js --format markdown
112+
ruleprobe drift AGENTS.md .eslintrc.json --format json --output drift-report.json
112113
```
113114

114-
---
115-
116-
## `ruleprobe task <template-id>`
117-
118-
Output the full task prompt for a given template. Three templates ship: `rest-endpoint`, `utility-module`, `react-component`.
119-
120-
```bash
121-
ruleprobe task rest-endpoint
122-
```
115+
| Option | Default | Description |
116+
|--------|---------|-------------|
117+
| `--format <format>` | `text` | Output format: `text`, `json`, or `markdown` |
118+
| `--output <path>` | stdout | Write report to file |
123119

124120
---
125121

126-
## `ruleprobe run <instruction-file>`
122+
## `ruleprobe extract <eslint-file>`
127123

128-
Invoke an AI agent on a task template, then verify its output. Requires `@anthropic-ai/claude-agent-sdk` and `ANTHROPIC_API_KEY` for SDK mode. Alternatively, use `--watch` to point at a directory where an agent will write output.
124+
Parse an ESLint config and emit a markdown rules section suitable for pasting into an instruction file. Stylistic rules are reported but excluded from the output by default.
129125

130126
```bash
131-
# SDK mode: invoke Claude, verify, report
132-
ruleprobe run CLAUDE.md --task rest-endpoint --agent claude-code --model sonnet --format text
133-
134-
# Watch mode: wait for output in a directory, then verify
135-
ruleprobe run CLAUDE.md --watch ./agent-output --timeout 300 --format json
127+
ruleprobe extract .eslintrc.json
128+
ruleprobe extract eslint.config.js --output rules-section.md
136129
```
137130

138131
| Option | Default | Description |
139132
|--------|---------|-------------|
140-
| `--task <template-id>` | `rest-endpoint` | Task template to give the agent |
141-
| `--agent <name>` | `claude-code` | Agent identifier |
142-
| `--model <name>` | `sonnet` | Model to use for the agent |
143-
| `--format <format>` | `text` | Report format: `text`, `json`, `markdown`, or `rdjson` |
144-
| `--output-dir <path>` | none | Directory to persist agent output |
145-
| `--watch <dir>` | none | Watch a directory for agent output instead of invoking |
146-
| `--timeout <seconds>` | `300` | Watch mode timeout in seconds |
147-
| `--allow-symlinks` | `false` | Follow symlinks outside the working directory |
148-
| `--config <path>` | auto-discovered | Path to config file |
149-
| `--project <tsconfig>` | none | tsconfig.json path for type-aware checks |
133+
| `--output <path>` | stdout | Write output to file |

0 commit comments

Comments
 (0)