Skip to content

Add --self-test flag to bin/validate-patterns (cover rejection paths) #71

@JuanVqz

Description

@JuanVqz

Parent: #53

Problem

bin/validate-patterns has no automated test coverage. CI runs it on the 12 in-repo pattern files, but those files exercise only the happy path — none of the rejection paths (missing required keys, broken regex, invalid kind: value) are ever asserted in CI. If someone refactors the validator and accidentally drops a guard, CI still passes.

This gap was surfaced during code review of #70 (the first sub-issue of #53), where the new kind: enum-rejection path is correctly implemented but not covered by any automated test.

Approach: --self-test flag on the existing validator

Rather than adding a separate bin/validate-patterns-test script, fold the assertions into the existing bin/validate-patterns binary as a --self-test flag. Keeps everything in one file, matches the validator's own "single-file, stdlib-only, no Bundler" philosophy.

bin/validate-patterns                    # validate every patterns file (today)
bin/validate-patterns path/to/file.yml   # validate specific file (today)
bin/validate-patterns --self-test        # NEW: run built-in fixture assertions

What --self-test does

  1. Writes tiny pattern YAML files to Dir.mktmpdir:
    • Valid fixtures: one per kind value (breaking, deprecation, migration, optional)
    • Invalid fixtures (one per rejection path):
      • Missing top-level key (no upgrade_findings:)
      • Missing required per-pattern key (e.g. no kind:)
      • pattern: regex that does not compile
      • Unknown kind: value (e.g. bogus)
  2. Calls validate(path) on each fixture in-process (not a shell subprocess).
  3. Asserts:
    • Valid → errors.empty?
    • Invalid → errors non-empty AND contains a substring matching the rejection ("missing key: kind", "regex does not compile", "invalid kind:", etc.)
  4. Prints OK self-test (N/N passed) and exits 0; or prints which assertions failed and exits 1.

Acceptance

  • bin/validate-patterns --self-test exits 0 when all assertions pass
  • Asserts the four positive paths (one per valid kind: value)
  • Asserts the rejection paths: missing top-level key, missing required pattern key, broken regex, unknown kind: value
  • Wired into .github/workflows/validate-patterns.yml as an additional step
  • Validator stays a single file; no new entry points; no Bundler / Gemfile / test framework introduced

Out of scope

  • Adopting a test framework (RSpec/Minitest)
  • Reorganizing the validator into a class
  • Adding tests for things outside the rejection paths (e.g., file I/O behavior)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions