Skip to content

Commit ad98a6c

Browse files
moonrunnerkcclaude
andcommitted
feat: v4.5.0 release - pivot to ESLint config translation
Pivot RuleProbe from "verify adherence" to "translate instruction files into ESLint configs." Three new commands: lint-config, drift, extract. Remove compare, tasks, task, and run commands. Audit parser matchers down to 34 ESLint-mappable matchers (from ~103). Remove 67 unmappable matchers and 7 empty categories. BREAKING: compare, tasks, task, and run commands removed. BREAKING: runner module removed from public API. BREAKING: formatComparisonMarkdown removed from reporter. BREAKING: RuleCategory narrowed to 7 categories + agent-behavior. BREAKING: VerifierType narrowed to ast, regex, filesystem. BREAKING: verify command deprecated (still works, primary focus shifted). - Remove src/commands/compare.ts, tasks.ts, run.ts - Remove src/runner/agent-configs.ts, agent-invoker.ts, watch-mode.ts, task-templates.ts, task-templates/*.md - Remove 6 matcher files: preference, file-structure, tooling, testing, config-file, git-history - Remove individual unmappable matchers from remaining files - Add src/commands/drift.ts, extract.ts (from prior commits) - Add src/drift/, src/extractor/, src/mappings/ (from prior commits) - Rewrite README around translate/drift/extract pitch - Add CHANGELOG v4.5.0 entry - Bump package.json to 4.5.0 - Update .npmignore and .gitignore for dev artifacts - Update all test files for removed matchers and commands - Clean stale dist files Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 5add65c commit ad98a6c

90 files changed

Lines changed: 4220 additions & 4359 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ coverage/
77
# Environment variables
88
.env
99

10+
# Phase 0 data collection cache and output
11+
.cache/
12+
data/
13+
1014
# Scripts (local utilities, not shipped)
1115
scripts/
1216

.npmignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,10 @@ scraped-instructions/
66

77
# Development files
88
.vscode/
9+
.claude/
10+
.codex/
11+
.codex
12+
.codex-tmp/
913
.github/
1014
docs/
1115
SECURITY.md
@@ -15,6 +19,9 @@ tsconfig.json
1519
.env
1620
.env.*
1721

22+
# RuleProbe semantic cache
23+
.ruleprobe-semantic/
24+
1825
# Git
1926
.git/
2027
.gitignore

AGENTS.md

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# AGENTS.md
2+
3+
Instructions for AI coding agents working on the RuleProbe codebase.
4+
5+
RuleProbe verifies whether agents follow instruction files. This file is the instruction file for agents working on RuleProbe itself. It is parsed by RuleProbe in the self-check workflow, so every rule below is written to be machine-verifiable.
6+
7+
## Project
8+
9+
- Repository: https://github.com/moonrunnerkc/ruleprobe
10+
- Package: https://www.npmjs.com/package/ruleprobe
11+
- Language: TypeScript (strict)
12+
- Runtime: Node.js >= 18
13+
- License: MIT
14+
15+
## Build and Test
16+
17+
- Use `npm` as the package manager. Do not switch to pnpm, yarn, or bun.
18+
- Use `vitest` as the test runner. Do not introduce jest or mocha.
19+
- Run `npm test` before declaring any change complete.
20+
- Run `npm run build` to verify the TypeScript compile is clean.
21+
- A `package-lock.json` must exist at the repo root.
22+
- Pinned dependency versions are required in `package.json` (no `^` or `~` ranges).
23+
24+
## Code Style
25+
26+
- Use TypeScript strict mode. Never disable strict checks.
27+
- Never use `any`. Use `unknown` and narrow, or define a precise type.
28+
- Always use named exports. Never use default exports.
29+
- Use camelCase for variables and functions.
30+
- Use PascalCase for types, interfaces, and classes.
31+
- Use kebab-case for filenames.
32+
- Prefer `const` over `let`.
33+
- Prefer `interface` over `type` for object shapes.
34+
- Prefer `async/await` over `.then()` chains.
35+
- Never use `console.log` in production code. Use the structured logger.
36+
- Never use `eval`.
37+
- No magic numbers without a named constant or inline comment justifying the value.
38+
- No em dashes anywhere in source, comments, docs, or commit messages. Use commas, colons, semicolons, parentheses, or separate sentences.
39+
- Files must stay under 300 lines. If a file approaches the limit, decompose it.
40+
- Add full JSDoc to every exported function, class, and type.
41+
- Avoid nested ternaries. Use early returns to flatten control flow.
42+
43+
## Architecture Boundaries
44+
45+
- Parser code lives under `src/parser/`. Do not call verifier code from the parser.
46+
- Verifier engines live under `src/verifiers/`. Each engine exports a single entrypoint that returns `VerificationResult[]`.
47+
- The semantic tier lives under `src/semantic/`. Source code must never leave the user's machine. Only numeric AST vectors, opaque sub-tree hashes, boolean flags, and rule text may be sent to an LLM.
48+
- The CLI lives under `src/cli/`. Commands compose pipeline functions; they do not contain pipeline logic.
49+
- Shared types live under `src/types/`. Do not duplicate type definitions across modules.
50+
51+
## Verifier Engines
52+
53+
There are eight verifier engines: `ast`, `filesystem`, `regex`, `treesitter`, `preference`, `tooling`, `config-file`, `git-history`. When adding a check, use an existing engine. Adding a new engine requires a written justification in the PR description.
54+
55+
- AST checks must use `ts-morph`. Do not parse TypeScript with regex.
56+
- Tree-sitter WASM loading must fail gracefully. If a grammar fails to load, log a warning and skip the check. Never block other verifiers.
57+
- Type-aware AST checks (implicit any, unused exports, unresolved imports) require a `tsconfig.json` and the `--project` flag. Skip cleanly when absent.
58+
- Semantic tier failures must never prevent deterministic results from returning.
59+
60+
## Parser Rules
61+
62+
- The parser supports 7 instruction file formats: `CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `copilot-instructions.md`, `GEMINI.md`, `.windsurfrules`, `.rules`. Parser changes must not break extraction for any of them.
63+
- Lines that cannot be mapped to a deterministic check go into the `unparseable` array. Do not invent rules to inflate the parse rate.
64+
- LLM-extracted rules must be tagged `extractionMethod: 'llm'` with `confidence: 'medium'`.
65+
- Rubric-decomposed rules must be tagged `confidence: 'low'`.
66+
67+
## Testing
68+
69+
- Every new function requires at least one test.
70+
- Test files live under `tests/` and mirror the `src/` directory structure.
71+
- Test names describe behavior, not implementation.
72+
- Tests must validate real behavior, not wiring. Reading the implementation should not be required to understand what a test verifies.
73+
- No mocks except at external API boundaries (Anthropic API, OpenAI API, GitHub API, filesystem boundaries when testing error paths).
74+
- Use `describe` and `it` blocks. Do not use `test()` directly.
75+
- Never use `console.log` in tests.
76+
- New matchers require: the matcher implementation, a test file with real-world instruction examples, and an entry in `docs/matchers.md`.
77+
78+
## Security
79+
80+
- Never execute scanned code.
81+
- Never modify files in the scanned directory.
82+
- All user-supplied paths must be resolved and bounded to the working directory.
83+
- Symlinks resolving outside the project root must be skipped unless `--allow-symlinks` is passed.
84+
- Never write API keys to disk or include them in reports.
85+
- Network calls are allowed only when the user opts in: `--llm-extract`, `--rubric-decompose`, `--semantic`, or `ruleprobe run`.
86+
87+
## Imports
88+
89+
- No path aliases. Use relative imports.
90+
- No barrel imports from deep internal modules. Import directly from the file that defines the symbol.
91+
- No wildcard imports.
92+
- Do not import lodash. Use native JavaScript methods.
93+
94+
## Error Handling
95+
96+
- Never use empty catch blocks.
97+
- Never swallow errors silently. Log or rethrow.
98+
- Catch clauses must declare the caught type as `unknown` and narrow.
99+
- Error messages must include what failed and what to do about it.
100+
101+
## Git Workflow
102+
103+
- Use conventional commit messages: `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`.
104+
- Branch names use kebab-case: `feat/new-matcher`, `fix/parser-bug`.
105+
- Pull requests must pass the self-check workflow before merge.
106+
- Do not commit `.env` files or any file containing secrets.
107+
108+
## Configuration Files
109+
110+
- ESLint config lives at `.eslintrc.json` or `eslint.config.js`.
111+
- Prettier config lives at `.prettierrc` or `.prettierrc.json`.
112+
- TypeScript config lives at `tsconfig.json`.
113+
- Vitest config lives at `vitest.config.ts`.
114+
- Do not add competing tools (Biome alongside ESLint, Rome, etc.).
115+
116+
## Documentation
117+
118+
- Update `docs/matchers.md` when adding or modifying a matcher.
119+
- Update `docs/cli-reference.md` when adding or changing a CLI command or flag.
120+
- Update `docs/api-reference.md` when changing the public API surface.
121+
- Update the relevant release notes file under `docs/` for any user-facing change.
122+
123+
## What Not To Do
124+
125+
- Do not introduce a new agent SDK adapter without a corresponding integration test.
126+
- Do not weaken the security boundary to make a check easier to implement.
127+
- Do not push deterministic logic into the semantic tier because it is easier to write.
128+
- Do not add features that require API keys for the default deterministic path.
129+
- Do not add dependencies without a clear justification. Each dependency is a maintenance cost.

CHANGELOG.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,49 @@
22

33
All notable changes to this project will be documented in this file.
44

5+
## [4.5.0] - 2026-05-08
6+
7+
### Breaking Changes
8+
9+
- **Default action mode changed.** The primary workflow is now `lint-config`, `drift`, and `extract`, not `verify`. The GitHub Action defaults to drift detection mode.
10+
- **`compare` command removed.** Agent comparison is no longer a primary use case. Use drift detection instead.
11+
- **`tasks` and `task` commands removed.** Task template listing and printing removed.
12+
- **`run` command removed.** Agent invocation via the Claude Agent SDK removed. The `@anthropic-ai/claude-agent-sdk` is no longer a dependency.
13+
- **Runner module removed from public API.** `buildAgentConfig`, `invokeAgent`, `isAgentSdkAvailable`, `hasAgentOutput`, `watchForCompletion`, `countCodeFiles`, `AgentInvocationConfig`, `RunOptions`, `InvocationResult`, `WatchOptions`, `WatchResult` are no longer exported.
14+
- **`formatComparisonMarkdown` removed.** The comparison report formatter is no longer exported from `reporter/index`.
15+
- **`verify` command deprecated.** Still works, but the primary workflow is now translate, detect drift, and extract.
16+
- **67 unmappable matchers removed.** Categories removed: `test-requirement`, `dependency`, `preference`, `file-structure`, `tooling`, `testing`, `workflow`. Verifier types removed: `treesitter`, `preference`, `tooling`, `config-file`, `git-history`. The remaining 34 matchers all map to ESLint rules.
17+
- **`RuleCategory` union narrowed.** Removed: `test-requirement`, `dependency`, `preference`, `file-structure`, `tooling`, `testing`, `workflow`. Remaining: `naming`, `forbidden-pattern`, `structure`, `import-pattern`, `error-handling`, `type-safety`, `code-style`, `agent-behavior`.
18+
- **`VerifierType` union narrowed.** Removed: `treesitter`, `preference`, `tooling`, `config-file`, `git-history`. Remaining: `ast`, `regex`, `filesystem`.
19+
20+
### New Features
21+
22+
- **`lint-config` command.** Translates an instruction file into a flat or legacy ESLint config. Unmappable rules appear as comments in the output.
23+
- **`drift` command.** Compares an instruction file against an existing ESLint config. Reports rules present in one but missing from the other, severity mismatches, and config argument differences.
24+
- **`extract` command.** Parses an ESLint config and emits a markdown rules section suitable for pasting into an instruction file.
25+
26+
### Removed
27+
28+
- `compare` command and `formatComparisonMarkdown` export.
29+
- `tasks` and `task` commands and `src/runner/task-templates/` directory.
30+
- `run` command and `src/runner/agent-configs.ts`, `src/runner/agent-invoker.ts`, `src/runner/watch-mode.ts`.
31+
- Matcher files: `rule-patterns-preference.ts`, `rule-patterns-file-structure.ts`, `rule-patterns-tooling.ts`, `rule-patterns-testing.ts`, `rule-patterns-config-file.ts`, `rule-patterns-git-history.ts`.
32+
- Individual unmappable matchers from remaining files: `test-files-exist`, `test-named-pattern`, `structure-strict-mode`, `error-async-try-catch`, `structure-typescript-required`, `error-log-contextual`, `import-no-unresolved`, `naming-python-snake-case`, `naming-python-class`, `naming-go-conventions`, `style-python-function-length`, `style-go-function-length`, `style-concise-conditionals`, `naming-kebab-case-directories`, `structure-no-barrel-files`, `test-no-settimeout`, `test-no-only`, `test-no-skip`, `import-banned-package`, `structure-readme-exists`, `structure-changelog-exists`, `structure-formatter-config`, `dependency-pinned-versions`.
33+
34+
### Stats
35+
36+
| Metric | v4.0.0 | v4.5.0 |
37+
|--------|--------|--------|
38+
| Rule matchers | ~103 | 34 |
39+
| Rule categories | 14 | 7 (+ `agent-behavior`) |
40+
| Verifier engines | 8 | 3 |
41+
| CLI commands | 9 | 6 |
42+
| Public API exports | ~40 | ~25 |
43+
44+
## [4.0.0] - 2026-04-28
45+
46+
Major release consolidating the three-repo architecture. See [docs/release-v4.0.0.md](docs/release-v4.0.0.md) for full details.
47+
548
## [1.0.0] - 2026-04-07
649

750
14 commits, 100 files changed, +9,017 lines since v0.1.0.

CLAUDE.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# CLAUDE.md
2+
3+
Instructions for Claude (Claude Code, Claude SDK, claude.ai) when working on the RuleProbe codebase.
4+
5+
The full engineering rules live in [AGENTS.md](./AGENTS.md). Read that file first. Everything below is Claude-specific workflow guidance that supplements it. When the two conflict, AGENTS.md wins.
6+
7+
## Project
8+
9+
- Repository: https://github.com/moonrunnerkc/ruleprobe
10+
- Package: https://www.npmjs.com/package/ruleprobe
11+
- Language: TypeScript (strict)
12+
- Runtime: Node.js >= 18
13+
14+
## Working Style
15+
16+
- Make targeted edits over full rewrites when a specific issue is flagged.
17+
- Never pad documentation, README, or release notes with marginal results. Only genuinely interesting findings belong.
18+
- Run verification before treating any feature as done: `npm test` and `npm run build`.
19+
- Push back when a claim is not backed by real data. Do not invent metrics, repo counts, or compliance scores.
20+
- Use `git status` and `git diff` to confirm what you actually changed before reporting completion.
21+
22+
## Engineering Standards
23+
24+
These are enforced by the self-check workflow (RuleProbe verifies its own codebase). Following them keeps the build green.
25+
26+
- Always use TypeScript strict mode.
27+
- Never use `any`. Use `unknown` and narrow.
28+
- Always use named exports. Never use default exports.
29+
- Use kebab-case for filenames.
30+
- Use camelCase for variables and functions.
31+
- Use PascalCase for types, interfaces, and classes.
32+
- Prefer `const` over `let`.
33+
- Prefer `interface` over `type` for object shapes.
34+
- Files must stay under 300 lines.
35+
- Add full JSDoc to every exported symbol.
36+
- Never use em dashes anywhere.
37+
- No magic numbers without a named constant.
38+
39+
## Tooling
40+
41+
- Use `npm` as the package manager.
42+
- Use `vitest` as the test runner.
43+
- Use `eslint` for linting and `prettier` for formatting.
44+
- Do not introduce competing tools (no biome, no jest, no pnpm).
45+
46+
## Pipeline Boundaries
47+
48+
When editing pipeline code, respect the three-stage boundary: parse, verify, report. Each stage is independently testable and lives under its own directory in `src/`.
49+
50+
- Parser code lives under `src/parser/`. The parser must not call verifier code.
51+
- Verifier engines live under `src/verifiers/`. Each engine returns `VerificationResult[]`.
52+
- Report generation lives under `src/report/`. Reports must not run verifications.
53+
- The semantic tier under `src/semantic/` must never send source code, file paths, variable names, or comments to any LLM.
54+
55+
## Tree-sitter and Type-aware Checks
56+
57+
- Tree-sitter WASM grammars must fail gracefully. If a grammar fails to load, log a warning and continue. Never block other verifiers.
58+
- Type-aware AST checks require `--project` and a valid `tsconfig.json`. Skip cleanly when absent.
59+
- Semantic tier failures must never prevent deterministic results from returning.
60+
61+
## Testing
62+
63+
- Every new function requires at least one test.
64+
- Test files live under `tests/` and mirror the `src/` structure.
65+
- Tests must validate real behavior, not wiring.
66+
- No mocks except at external API boundaries.
67+
- Use `describe` and `it` blocks.
68+
- Never use `console.log` in tests.
69+
- New matchers require a test file with real-world instruction examples.
70+
71+
## Security
72+
73+
- Never execute scanned code.
74+
- Never modify files in the scanned directory.
75+
- Bound all paths to the working directory.
76+
- Skip symlinks resolving outside the project root unless `--allow-symlinks` is passed.
77+
- Never write API keys to disk.
78+
79+
## Claude Code Workflow
80+
81+
- When invoked through Claude Code, prefer running tests and builds in the integrated terminal so output is captured in the session.
82+
- For multi-file changes, list the files you intend to touch before editing.
83+
- For changes that affect the parser, run `ruleprobe parse` against `tests/fixtures/` instruction files to confirm no regression in extraction.
84+
- For changes that affect a verifier, run `npm test -- <verifier-name>` before running the full suite.
85+
- For changes that touch the semantic tier, manually verify that no source code, file paths, variable names, or comments appear in any outbound payload.
86+
87+
## Commit Messages
88+
89+
Use conventional commit format: `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`.
90+
91+
Examples:
92+
- `feat: add tree-sitter naming check for Rust`
93+
- `fix: parser drops rules with backtick-wrapped patterns`
94+
- `docs: update matchers.md with new file-structure entries`
95+
96+
## What Not To Do
97+
98+
- Do not refactor code outside the scope of the requested change.
99+
- Do not weaken the security boundary to simplify an implementation.
100+
- Do not push deterministic logic into the semantic tier.
101+
- Do not add dependencies without justification.
102+
- Do not write release notes that make claims unsupported by the actual diff.
103+
- Do not present completion as fact without running tests and build.

0 commit comments

Comments
 (0)