fix(ci): honest validators + Hypatia baseline from full 195-finding triage (#399 #434 #435 #477 #381 #507)#521
Merged
Conversation
…files validate-k9.sh (#434): - Two dialects, validated appropriately: plain .k9 keeps the literal K9! magic requirement; .k9.ncl (Nickel — where a bare K9! line is a syntax error) accepts a magic_number = "K9!" field, a K9! preamble line, or a pedigree-schema import/merge (library/contractile modules, option (a)). - Pedigree/field checks are file-scope by design: the old brace-depth block tracker skipped its own opening brace (any nested block's close ended the scan) and could not see let-bound (`let component_pedigree`) or merged (`base_k9.pedigree_schema &`) factorings. Header documents why block scoping must not return. - Plain-dialect colon forms (metadata:, name:, leash:) now recognised. - normalise_level: full whitespace trim (single-space trim let `'Kennel # comment` survive as invalid "kennel "); {{placeholder}} levels in scaffolds are a notice, not an invalid-level error. - Repo measurement: 96 errors -> 45, every survivor verified real (PROTOCOL.k9 stubs, avow/axel configs with no pedigree, k9-svc site samples). must.k9.ncl, hello.k9.ncl, coordination.k9 all pass. validate-a2ml.sh (#435, option a): - The SPEC's only identity rule is per-record (@record needs author/tool/kind, §7); the file-level identity ERROR was invented and flagged 465/1550 canonical files (scorecards identify via spec_id, contractiles via @abstract, the six-file set via [metadata]). Identity is now a lint warning checked against the real estate shapes. - Exempt-by-design classes: AI manifests (incl. AI.a2ml), templates (basename or {{PLACEHOLDER}} body), and design-rationale trees via INPUT_DESIGN_TREES (default: machine-readable-design/, self-validating/examples/, docs/templates/). - Repo measurement: 465 errors -> 0; 398 honest warnings remain. Both: GITHUB_OUTPUT defaults to /dev/null — under `set -u` the unset expansion inside a redirection aborted the script outside Actions and `|| true` cannot catch an expansion error. Refs #434 #435 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… GS007, CSA001) structural_drift, git_state and code_scanning_alerts emit uppercase mnemonic codes as the finding `type`; the schema's lowercase-only pattern made every such finding unbaselineable (38 SD022 + 54 CSA on this repo alone). Pattern widened to exactly the two observed shapes — lowercase snake_case or 2-3 uppercase letters + 3 digits — verified against all 24 distinct type values in the current Hypatia run plus GS007. Refs #477 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#381) The pre-commit registry guard (#381 option 1) already exists in hooks/pre-commit and was live-verified both ways on this branch: it caught a planted canary edit AND this branch's own schema/validator edits (exit 1 with fix commands), and passes on a clean tree. This commit resyncs REGISTRY.a2ml for those edits and closes the remaining gap — option 3, documenting `just hooks-install` + `just registry` in CONTRIBUTING.md so the workflow is discoverable before the first red PR. Refs #381 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
governance-reusable.yml carried a comment claiming the validate-baseline job "schema-checks" .hypatia-baseline.json; no step anywhere did, and apply-baseline.sh listed ajv as an "optional" dependency it never invoked. A malformed baseline (typo'd key, wrong severity string) would silently match nothing — the exact silent-suppression class the gate exists to prevent. apply-baseline.sh now structurally validates every baseline in jq, mirroring hypatia-baseline.schema.json: required keys, file XOR file_pattern, closed key set, severity enums, rule_module/type/ expires_at/tracking_issue shapes. Malformed baselines exit 2 (invalid input) with each violation named. Canary-tested both directions: a valid entry with the new uppercase mnemonic type (SD022) passes; a six-violation adversarial fixture is rejected naming all six. The workflow comment now states what actually happens. Refs #477 #399 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…et-scanner templates (#507) The canon retired inline TruffleHog as redundant to the pinned gitleaks gate (secret-scanner-reusable.yml documents the rationale), but six nested product copies still carried the dead job. They are inert as workflows today (Actions only reads the root .github/workflows) — but they are exactly the files that go live the day each product evicts to its own repo (#490-#496), so the rot ships unless the template is fixed at source: 0-ai-gatekeeper-protocol, a2ml, consent-aware-http, lol, meta-a2ml, rhodium-standard-repositories. Estate scope for #507, measured 2026-07-21 (gh code search, default branches): only 3 LIVE repos carry the dead job — ideas-to-alphas (startup_failure), polystack (failure), QuantumCircuit.jl (failure); 17 more matched on the canonical rationale *comment* only (no action); julia-ecosystem has 10 inert nested per-package copies (+ .gitlab-ci.yml twins, live if GitLab-mirrored); rsr-template-how-to's copies are deliberate teaching specimens under specimen/automation-quarantine/. Refs #507 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
FIX_NOW class) - scorecard-enforcer.yml: timeout-minutes on all 3 jobs (15/10/10); signed-push-smoke.yml: timeout-minutes 10 — clears 4 of the 5 workflow_audit missing_timeout_minutes findings from the first completed Hypatia run. - push-email-notify.yml deleted (the 5th): estate directive is to remove the broken push-email workflow everywhere except 007, robodog-defensive-systems-lab and defensive-multiplicity; standards is not an exception. Deletion also clears its missing-timeout finding. Refs #399 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…195 findings (#399) Every finding from Hypatia's first completed run on standards main was triaged and adversarially verified (24-agent workflow, per-finding read + refute pass). 118 baseline entries: - 96 permanent FALSE_POSITIVE: mirror double-reports (#378), guarded unwraps, bench expect(), XML-namespace http:// URIs, public credential-less CORS manifests, .envrc {{placeholder}} exports, idiomatic CString FFI as_ptr, doc-only historical dir references. - 22 TRACKED DEBT with expires_at 2026-10-21 + tracking issue: real issues owned elsewhere (TS->AffineScript #254, product evictions #491/#492/#494/#496, third-party #493, proof debt #124, the scorecard.yml naming reconciliation). Deliberately NOT baselined (the gate stays honest): - The 40 secret_detected findings in rhodium-standard-repositories/rhodium-pipeline/docs/QUICKSTART.adoc are 40 occurrences of ONE live GitLab PAT (see PR body — needs owner revocation, NOT suppression). The baseline leaves these blocking on purpose; the governance gate is RED until the token is revoked + purged. Suppressing a live credential is the one thing a baseline must never do. - 5 findings this branch already FIXES (job timeouts + push-email removal) — they clear on rescan; baselining a fixed finding would be dishonest. Path fidelity: entries mirror Hypatia's own `.file` shape per module (basenames for workflow_audit, repo-relative elsewhere) — matching the agent-resolved paths instead silently suppressed nothing. The baseline now validates against hypatia-baseline.schema.json (enforced by apply-baseline.sh) and was filter-tested against the real findings: 150 suppressed, 45 kept (40 live-token + 5 fixed). Adding this file activates the previously-skipped validate-hypatia- baseline job on standards itself (has_baseline gate). Refs #399 #178 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 21, 2026
Open
hyperpolymath
marked this pull request as ready for review
July 21, 2026 12:55
Signed-off-by: Jonathan D.A. Jewell <6759885+hyperpolymath@users.noreply.github.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Addresses the standards-local backlog foundationally: makes the canon's
own validators/gates honest, then triages Hypatia's first completed run
(all 195 findings) into an honest baseline. Seven commits.
The triage found a live, write-scoped credential committed into this
public repo (an accidentally-pasted session transcript, one token, 40
occurrences, in
rhodium-standard-repositories/rhodium-pipeline/docs/ QUICKSTART.adoc). The token value is not reproduced here or in anycommit — only the file is named (it is already in
main).These 40 findings are deliberately left unbaselined, so the
validate-hypatia-baselinegate stays red until the credential is dealtwith. Prerequisite for a green gate = revoke the token at its provider,
then purge it from the tree + history. Suppressing it in the baseline is
the one thing that must never happen — that would silence a live-credential
alarm. A future maintainer must not "fix" this red by adding a baseline
entry for it.
What's in the PR
Validators stop flagging the canon's own files
validate-k9.sh(k9-validate over-fires on contractile Nickel library modules (missing K9! / pedigree.name) #434):.k9vs.k9.nclare now distinct dialects.Nickel library/contractile modules (
base_k9.pedigree_schema & {…},magic_number = "K9!") are first-class, not violations. Removed abrace-depth block tracker that skipped its own opening brace and could
not see
let/&factoring. 96 → 45 errors, every survivor verified real.validate-a2ml.sh(a2ml-validate flags design-rationale/example .a2ml for "missing identity field" #435, option a): the file-level identity errorwas never in the SPEC (identity is per-
@record, §7); it flagged 465/1550canonical files. Now a lint warning against the real estate shapes
(
spec_id,[metadata]/[scorecard],@abstract), with design-tree /template / AI-manifest exemptions (
INPUT_DESIGN_TREES). 465 → 0 errors.GITHUB_OUTPUTdefaults to/dev/null(an unset var underset -uaborted the script outside Actions).
Baseline machinery made honest
typepattern widened for the uppercase mnemonic codesstructural_drift/git_state/code_scanning_alertsactually emit(
SD022,GS007,CSA001) (hypatia baseline:typecasing mismatch — structural_drift/git_state emit uppercase codes (SD022/GS007) that violate the baseline schema'stypepattern #477).apply-baseline.shnow enforces that schema in jq. A workflow commentclaimed a job schema-checked the baseline; none did, and a malformed
baseline would silently match nothing. Now it exits 2, naming each
violation. Canary-tested both directions.
Registry drift (#381) — the pre-commit guard already exists; verified it
catches drift (it caught this branch's own edits). Documented
just hooks-install+just registryin CONTRIBUTING.md.Dead
trufflehog:job removed from 6 nested product secret-scannertemplates (#507). Estate scope measured: only 3 live repos still carry it
(ideas-to-alphas, polystack, QuantumCircuit.jl); 17 matched the rationale
comment only.
Workflow hygiene — job timeouts on scorecard-enforcer + signed-push-smoke;
removed
push-email-notify.ymlper the estate dual-use directive..hypatia-baseline.json— 118 entries from the full triage (24-agenttriage → adversarial verify): 96 permanent false-positives (mirror
double-reports #378, guarded unwraps, bench
expect(), XML-namespace httpURIs, public credential-less CORS manifests,
.envrc{{placeholder}}exports, idiomatic CString FFI) + 22 tracked debt (expiry 2026-10-21 +
tracking issue: #254, #491–#496, #124, scorecard.yml reconciliation).
Adding the file activates the previously-skipped baseline job.
Verification caveat
The baseline was filter-tested against the last completed Hypatia scan
(195 findings), not re-run against this branch's HEAD — the Elixir scanner
isn't buildable in this environment. The path-fidelity work (entries mirror
Hypatia's own per-module
.fileshape) makes drift low-risk, but CI's freshscan is the real check. Result of the local filter: 150 suppressed / 45
kept (40 live-token + 5 findings this branch already fixes).
Not fixed here (baselined + flagged, by design)
source template (boj-server
ffi/zig/src) and re-propagate, not inthis lone copy.
Refs #399 #178 #434 #435 #477 #381 #507