You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+30-1Lines changed: 30 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
8
8
## [Unreleased]
9
9
10
+
## [1.2.0] - 2026-04-29
11
+
12
+
Backward compatibility: previously-passing skills still pass. Some previously-failing skills now warn instead of error and produce exit code 0 instead of 1.
13
+
14
+
### Added
15
+
16
+
-`template.detected` info-level rule and `src/skillcheck/template_detection.py` module.
17
+
-`ECOSYSTEM_FIELDS` classification for `license`, `repository`, `homepage`, and `template`.
18
+
- Config support for `[frontmatter] extension_fields` in `skillcheck.toml`.
19
+
20
+
### Changed
21
+
22
+
-`frontmatter.name.reserved-word` demoted from ERROR to WARNING; source tag changed from `spec` to `advisory`; message rewritten.
23
+
-`frontmatter.description.person-voice` demoted from ERROR to WARNING; messages rewritten to acknowledge the heuristic.
24
+
- Budget-message phrasing aligned with the spec's "recommended" language across `sizing.*` and `disclosure.*` rules.
25
+
26
+
### Fixed
27
+
28
+
-`frontmatter.field.unknown` no longer fires on `license`, `repository`, `homepage`, or `template`; these now produce info-level `frontmatter.field.ecosystem` diagnostics or are silent for user extensions.
29
+
- Templates (placeholder content, `template: true` flag, or files under `template/` or `templates/` directories) no longer trigger deployment-blocking checks (`frontmatter.name.directory-mismatch`, `compat.vscode-dirname`, `description.quality-score`).
30
+
31
+
### Internal
32
+
33
+
- Renamed `config.KNOWN_FRONTMATTER_FIELDS` to `config.SPEC_FIELDS`.
34
+
- New `template.detected` rule wired into `rules/__init__.py`.
35
+
- Frontmatter rule implementation split into smaller modules while preserving `skillcheck.rules.frontmatter` imports.
36
+
- Root `SKILL.md` restored so `skillcheck SKILL.md` self-validation works from the repository root.
37
+
- New fixture set under `tests/fixtures/` covering ecosystem fields, user extensions, template detection, and demoted severities.
38
+
10
39
## [1.1.0] - 2026-04-28
11
40
12
41
External audit against v1.0.1 surfaced eight repo defects ranging from documentation drift to a CI-confusing exit-code conflation. v1.1.0 ships fixes for all eight, reverses one v1.0.1 behavior change that turned out wrong, and tightens the description scorer's vague-word rubric. The minor bump is driven by the exit-code semantics change (now distinguishes warning-only from input error) and the new `--warnings-as-errors` flag.
@@ -24,7 +53,7 @@ External audit against v1.0.1 surfaced eight repo defects ranging from documenta
24
53
### Changed
25
54
26
55
-`action.yml` install step pins `skillcheck>=1.0.1` so consumers fail loudly on unpublished v1 features instead of silently running v0.2.0.
27
-
- Description scorer rubric documented and tightened: dropped `comprehensive`, `robust`, and `flexible`from `_VAGUE_WORDS` because each can describe a concrete attribute when qualified ("comprehensive coverage of N file formats", "robust against malformed input"). The inclusion rubric is now documented inline. Verified against `anthropics/skills` (17 SKILL.md files): zero score changes, because none of those skills use the dropped words. The rubric edit is a no-op against the current corpus; the new regression tests are forward-looking guards against scoring drift if the list is ever re-expanded.
56
+
- Description scorer rubric documented and tightened: dropped `comprehensive`, `flexible`, and the malformed-input term from `_VAGUE_WORDS` because each can describe a concrete attribute when qualified ("comprehensive coverage of N file formats", "handles malformed input"). The inclusion rubric is now documented inline. Verified against `anthropics/skills` (17 SKILL.md files): zero score changes, because none of those skills use the dropped words. The rubric edit is a no-op against the current corpus; the new regression tests are forward-looking guards against scoring drift if the list is ever re-expanded.
28
57
- Description scorer verb matching: collapsed `_ACTION_VERBS` from 86 entries (base + 3rd-person duplicates) to 42 base forms. Added `_is_action_verb()` to handle stem normalization across `-s`, `-es`, and `-ies` endings. Adding a new verb now only requires the base form.
29
58
- README test count bumped from 663 to 667 to include the drift-guard test, two description-scorer regression tests, and the `--warnings-as-errors` test.
30
59
- README field-test citations: replaced seven gitignored `runs/...` path references with the exact `skillcheck` commands needed to reproduce each finding. Readers can now verify the claims without access to private artifacts.
User documentation lives in [README.md](README.md). This file covers testing, maintainer workflows, and adding new rules.
4
+
5
+
## Testing
6
+
7
+
```bash
8
+
pip install -e ".[dev]"
9
+
python3 -m pytest tests/ -q
10
+
```
11
+
12
+
683 tests cover all rule modules, CLI exit codes, graph analyzers, divergence detection, critique parsing, history round-trips, and the full self-host pipeline against `skills/skillcheck/SKILL.md`. Fixtures are in `tests/fixtures/`; every rule has at least one positive and one negative test case. `tests/test_readme_test_count_claim.py` asserts this count matches `pytest --collect-only`, so any future suite change has to update the number in the same commit or CI fails.
13
+
14
+
## Maintainer Notes
15
+
16
+
After editing `skills/skillcheck/SKILL.md`, regenerate the self-host test fixtures so the integration suite stays pinned to the current graph:
17
+
18
+
```bash
19
+
make regen-self-host-fixtures
20
+
```
21
+
22
+
This runs `scripts/regen_self_host_fixtures.py`, which extracts a fresh heuristic graph and writes it to `tests/fixtures/self_host/graph_clean.json`.
23
+
24
+
To summarize a batch of skillcheck JSON outputs across many repos (the layout the field-test runs use, with one directory per repo, one subdirectory per skill, and `01-symbolic.json` / `02-strict-vscode.json` / `03-graph-analyze.json` / `04-graph-extracted.json` / `08-critique-report.json` / `09-graph-agent-report.json` / `10-full-pipeline.json` per skill), run:
It writes `summary.csv` and `findings.md` next to the batch directory. The script is intended for benchmark and field-test workflows; it is not part of the CLI surface and is not exposed as a console script.
31
+
32
+
To add a new rule: implement `def check_something(skill: ParsedSkill) -> list[Diagnostic]` in the appropriate module under `src/skillcheck/rules/`, register it in `src/skillcheck/rules/__init__.py`, add at least one positive and one negative fixture, and add a row to the Rules table above. Full conventions are in [`.github/CLAUDE.md`](.github/CLAUDE.md).
0 commit comments