Add --self-test flag to bin/validate-patterns (closes #71)#95
Merged
etagwerker merged 1 commit intomainfrom May 5, 2026
Merged
Add --self-test flag to bin/validate-patterns (closes #71)#95etagwerker merged 1 commit intomainfrom
etagwerker merged 1 commit intomainfrom
Conversation
Closes the test-coverage gap surfaced during PR #70 review. Until now the validator's rejection paths (missing required keys, broken regex, unknown kind: value) had no automated coverage — CI ran the validator on the 12 in-repo files which all pass, so a future refactor could silently drop a guard without breaking CI. `bin/validate-patterns --self-test` builds 8 fixture cases in a tmpdir and calls validate(path) in-process for each: - 4 valid cases (one per kind: value) - 4 invalid cases: - missing top-level upgrade_findings: key - missing required pattern key (kind:) - pattern: regex that does not compile - unknown kind: value (bogus) Asserts valid → errors.empty?; invalid → errors contains the expected substring. Prints "OK self-test (N/N passed)" and exits 0/1. Wired into .github/workflows/validate-patterns.yml as an additional step after the existing file-validation step. Single file, stdlib-only — no Bundler, no test framework. Matches the validator's existing philosophy. Refs #53, closes #71
etagwerker
approved these changes
May 5, 2026
Member
etagwerker
left a comment
There was a problem hiding this comment.
@JuanVqz Looks good, thanks! 👍🏻
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #71. Adds a
--self-testflag tobin/validate-patternsthat writes valid + invalid fixture YAMLs to a tmpdir and asserts the validator's behavior end-to-end. Wired into CI alongside the existing file-validation step. Single-file, stdlib-only, no Bundler / no test framework — matches the validator's existing philosophy.Closes the test-coverage gap that was surfaced during PR #70 review: the validator's rejection paths (missing required keys, broken regex, unknown
kind:value) are exercised manually but never asserted in CI. Without this, someone could refactor the validator and silently drop a guard while the 12 in-repo files keep passing.What
--self-testdoesDir.mktmpdir.validate(path)in-process (not via subprocess).errors.empty?errorscontains a substring matching the expected rejectionOK self-test (N/N passed)and exits 0; or prints which assertions failed and exits 1.The 8 cases
kind: breakingkind: deprecationkind: migrationkind: optionalupgrade_findings:top-level keymissing top-level key: upgrade_findingskind:per-pattern keymissing key: kindpattern:regex[unclosedpattern regex does not compilekind: bogusinvalid kind: "bogus"Files changed
bin/validate-patterns— addedself_testfunction (~60 lines) and a--self-testdispatch branch. Updated usage docstring..github/workflows/validate-patterns.yml— addedruby bin/validate-patterns --self-teststep after the existing validation step.CLAUDE.md— documented the--self-testinvocation under Repository tooling.rails-upgrade/CHANGELOG.md— Unreleased entry.Test plan
bin/validate-patterns --self-testexits 0 withOK self-test (8/8 passed)bin/validate-patterns(no flag) still validates all 12 in-repo files clean — no regressionWhy this approach
The original issue body proposed a separate
bin/validate-patterns-testscript. Switched to a--self-testflag on the same binary because:validate(path)in-process — cleaner than shelling out and re-parsing exit codes.Linked issues
kind:field to detection patterns; renamebreaking_changes:→upgrade_findings:#53 (parent rollout — this is the test-coverage follow-up surfaced during the rollout)Out of scope
bin/validate-patternsinto a class