-
-
Notifications
You must be signed in to change notification settings - Fork 937
docs(governance): propose a quality harness (layered gates + full-suite-before-seal) #794
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,37 @@ | ||||||||||||||
| audit_finding: | ||||||||||||||
| version: "1.0" | ||||||||||||||
| id: "AF-20260614-quality-harness" | ||||||||||||||
| date: "2026-06-14" | ||||||||||||||
| auditor: "external-contributor (@aiox-master in a consumer project)" | ||||||||||||||
| source_session: "Comparative audit of a hardened consumer fork against aiox-core v5.2.9" | ||||||||||||||
|
|
||||||||||||||
| context: | ||||||||||||||
| project: "a production consumer project (anonymized)" | ||||||||||||||
| epic: "framework learning-loop audit" | ||||||||||||||
| triggered_by: "'Quality First' is a constitution article but nothing mechanizes it per moment/owner" | ||||||||||||||
|
|
||||||||||||||
| finding: | ||||||||||||||
| summary: "'Quality First' (Article V) is declarative — there is no harness tying gates to moments and owning agents, no rule forcing the full suite before a seal, and no mechanical 'no push without a QA gate'." | ||||||||||||||
| evidence: | ||||||||||||||
| - "Article V states the principle but no mechanism ties gates to moments/owners/fail-modes" | ||||||||||||||
| - "No 'run the full suite (no path filter) + build + typecheck before seal' rule → subset runs hide regressions in untouched modules" | ||||||||||||||
| - "No mechanical enforcement that a push carries a QA gate verdict for modified work" | ||||||||||||||
| impact_observed: | ||||||||||||||
| blast_radius: "medium" | ||||||||||||||
| affected_artifacts: ["every change that ships", "the constitution's Quality First article"] | ||||||||||||||
| cost: "Cross-module regressions slip through subset runs; quality depends on humans remembering" | ||||||||||||||
|
|
||||||||||||||
| framework_candidate: true | ||||||||||||||
| framework_candidate_rationale: | | ||||||||||||||
| Project-agnostic. Any project using the framework benefits from a quality harness with | ||||||||||||||
| named owners + automatic gates, and from running the full suite before a seal. All gates | ||||||||||||||
| start advisory; only the PR/CI gate blocks (already the correct place) — preserves freedom. | ||||||||||||||
|
|
||||||||||||||
| references: | ||||||||||||||
| - kind: pattern | ||||||||||||||
| path: "Quality harness: layered gates with an owning agent and a fail-mode" | ||||||||||||||
|
Comment on lines
+30
to
+32
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Use a documented reference kind. The Consider using 🔧 Proposed fix references:
- - kind: pattern
+ - kind: existing-pattern-source
path: "Quality harness: layered gates with an owning agent and a fail-mode"📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||
|
|
||||||||||||||
| proposed_disposition: | ||||||||||||||
| - "Add a quality-suite rule: a moment x owning-agent x fail-mode map; gates run automatically" | ||||||||||||||
| - "Add full-suite-before-seal: whole suite + build + typecheck before work is shippable" | ||||||||||||||
| - "Add a pre-push QA gate (no push without a gate verdict for modified work)" | ||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| framework_proposal: | ||
| version: "1.0" | ||
| id: "PROP-20260614-quality-harness" | ||
| date: "2026-06-14" | ||
| proposer: "external-contributor (consumer-project audit)" | ||
| source_finding: "AF-20260614-quality-harness" | ||
|
|
||
| target: | ||
| layer: "L2" | ||
| artifact_type: "rule" | ||
| artifact_path: ".claude/rules/quality-suite.md (NEW) + .claude/rules/full-suite-before-seal.md (NEW) + optional pre-push QA gate" | ||
| operation: "create" | ||
|
|
||
| generalization: | ||
| pattern_name: "Quality harness: layered gates, each with an owning agent and a fail-mode" | ||
| when_to_apply: | | ||
| Any project where "quality first" is a stated principle but has no mechanism. | ||
| Three invariants make it real: | ||
| 1. every gate has a NAMED owning agent (not "whoever remembers"); | ||
| 2. every gate runs AUTOMATICALLY at its moment (git hook or CI), never by human initiative; | ||
| 3. the PR/CI gate is the one nobody can bypass (merge is blocked there). | ||
| Plus "full-suite-before-seal": run the WHOLE test suite (no path filter) + build + | ||
| typecheck before marking work shippable. Subset runs hide cross-module regressions — | ||
| a change shifts lines/exports and breaks a test in an *untouched* module. | ||
| examples_from_other_domains: | ||
| - "A 'Quality First' constitution article with no enforcement is a slogan, not a gate" | ||
| - "Subset test runs (--testPathPattern) pass while an untouched module's test breaks" | ||
| - "CI runs after push; the seal happens before — so the seal needs its own full check" | ||
|
|
||
| migration_path: | ||
| breaking_change: false | ||
| affected_consumers: [] | ||
| rollout_plan: | | ||
| Additive. Adds process rules + a moment x owning-agent x fail-mode map. Gates start | ||
| advisory; a project opts each gate into blocking via config. Only the PR/CI gate | ||
| blocks by default (already the correct place). | ||
|
|
||
| deprecation_plan: | ||
| deprecates: [] | ||
| sunset_window: "n/a" | ||
|
|
||
| cost_benefit: | ||
| cost: "~2 documentation rules + a moment x owner x fail-mode map; optional pre-push QA gate" | ||
| benefit: | | ||
| Turns the constitution's own "Quality First" (Article V) from a declarative principle | ||
| into a mechanism with named owners and automatic gates. full-suite-before-seal catches | ||
| the cross-module regressions that subset runs miss. | ||
| risk: "low-medium — additive; advisory by default; only the PR/CI gate blocks" | ||
|
|
||
| approval: | ||
| eliel_decision: "PENDING" | ||
| eliel_decision_at: null | ||
| eliel_decision_rationale: "" | ||
| revision_request: [] | ||
|
|
||
| implementation: | ||
| pr_url: "" | ||
| merged_at: "" | ||
| distributed_to: [] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ISO 8601 date format violation in both governance documents. Both the audit finding and the proposal use
2026-06-14without time and timezone components. The governance templates require full ISO 8601 format (e.g.,2026-05-07T18:30:00Z) to ensure reliable date parsing across tooling.audits/promoted/AF-20260614-quality-harness.yaml#L4: Change to2026-06-14T00:00:00Zgovernance/proposals/PROP-20260614-quality-harness.yaml#L4: Change to2026-06-14T00:00:00Z📍 Affects 2 files
audits/promoted/AF-20260614-quality-harness.yaml#L4-L4(this comment)governance/proposals/PROP-20260614-quality-harness.yaml#L4-L4🤖 Prompt for AI Agents