Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion AGENT_FRAMEWORK.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Agent Operating Framework v1.6
# Agent Operating Framework v1.7

> A behavioral operating system for AI coding agents — born from production failures, not theory.
>
Expand Down Expand Up @@ -327,6 +327,12 @@ Five of six rules ship with hook backing as of v1.5. The single remaining adviso
**Meta-hooks** (not bound to a single rule):
- [`deprecated-field-gate.sh`](examples/hooks/deprecated-field-gate.sh) — template for blocking writes that reference deprecated DB columns or API fields *(fail-mode: closed, blast-radius: destructive)*
- [`empty-rule-body-gate.sh`](examples/hooks/empty-rule-body-gate.sh) — pre-merge CI check that rejects rule files with empty bodies (< 200 bytes) or missing `## Why` sections *(fail-mode: closed, blast-radius: security — protects framework integrity against false-positive "applied" claims)*
- [`startup-gate.sh`](examples/hooks/startup-gate.sh) — SessionStart hook; checks all four governance surfaces (repo, AGENTS.md, active plan, hook registration) and writes a drift report to `~/.claude/startup-gate-report.md` *(fail-mode: open, blast-radius: advisory)*
- [`hook-telemetry-stop.sh`](examples/hooks/hook-telemetry-stop.sh) — Stop hook; reads per-hook fire/block breadcrumbs and bulk-INSERTs session telemetry *(fail-mode: open, blast-radius: advisory)*

**Libraries** (source into hooks, do not execute directly):
- [`breadcrumb-lib.sh`](examples/hooks/breadcrumb-lib.sh) — session-scoped breadcrumb API
- [`examples/hooks/lib/normalize-hook-input.sh`](examples/hooks/lib/normalize-hook-input.sh) — cross-runtime payload normalization (Claude Code + Grok field names and tool-name literals)

### 5.4 Rule Consolidation
Rules accumulate naturally as lessons are captured. Left unchecked, they become unreadable — too many rules means none get followed. When the rule count grows past 15-20:
Expand Down Expand Up @@ -379,9 +385,17 @@ guides/
auto-optimization.md ← How the self-improvement loop works
enforcement-architecture.md ← Memory → Rules → Hooks design patterns
rule-consolidation.md ← How to cluster and compress rules at scale
hook-operations.md ← The three operational questions (v1.6)
advanced/
done-criteria-schema.md ← Done Criteria spec and validator
hook-audit-methodology.md ← 4-track audit pattern (v1.6)
silent-failure-discipline.md ← Every fail-open path must log (v1.7)
agents-md-standard.md ← Three-level repo governance contract (v1.7)
examples/
claude-code-rules/ ← Sample rule files for Claude Code
hooks/ ← Reference hook implementations (Claude Code-specific)
lib/
normalize-hook-input.sh ← Cross-runtime payload normalization (v1.7)
```

**Claude Code note:** Hooks use Claude Code's `PreToolUse`/`PostToolUse` lifecycle. Rule prose is portable to any agent platform.
Expand All @@ -392,6 +406,7 @@ Full per-release notes live in [CHANGELOG.md](CHANGELOG.md). The framework file

Headline changes from recent versions:

- **v1.7** — Provable hooks release: `startup-gate.sh` (SessionStart governance check), `normalize-hook-input.sh` (cross-runtime payload normalization), `hook-telemetry-stop.sh` (session-end telemetry). New guides: `silent-failure-discipline.md` (ADR 0012 — every fail-open path must log) and `agents-md-standard.md` (three-level repo governance contract). Incidents #36–#38.
- **v1.6** — Hook operations layer: `breadcrumb-lib.sh` shared session library, `CLAUDE_HOOKS_SAFE_MODE` emergency bypass pattern, `# fail-mode: silent-skip` taxonomy tier for watcher-class hooks. New guides: `hook-operations.md` (the three operational questions) and `hook-audit-methodology.md` (4-track audit pattern). §5.2 updated with bypass + breadcrumb protocol as standard requirements. AOF self-eval harness (`b3d8451`) on main.
- **v1.5** — `secure-config-gate.sh`, `focus-breadcrumb.sh` + `focus-confirmation-gate.sh`, `dormant-code-gate.sh`. §5.3 coverage moves from 3-of-6 enforced to **5-of-6 enforced**. `no-local-infrastructure` rewritten as a hosting decision framework (advisory by design).
- **v1.4** — §5.3 Rule-to-Hook Coverage matrix added (accurate enforced-vs-advisory accounting). §5.2 fail-mode taxonomy. §1.3 precedence rule. §0.5 italic scope-anchor (Phase 1 Step 4), Done Criteria pre-condition (Phase 3 Step 1), doctor-clean log entry (Phase 3 Step 5). CI shipped: `doc-link-check.yml`, `rules-lint.yml`, `validate-done-criteria.py`. `empty-rule-body-gate.sh` added.
Expand Down
64 changes: 42 additions & 22 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,56 @@ All notable changes to this framework follow [Keep a Changelog](https://keepacha

---

## [1.7] — TBD (Notes)
## [1.7] — 2026-06

### Candidate content
### Background

The hooks × memory × skills design document (`smokin-os/spec/hooks-memory-skills-design.md`, written 2026-06-07) is the architectural blueprint for v1.7. Five concrete items emerge from it:
The hooks × memory × skills design document (`smokin-os/spec/hooks-memory-skills-design.md`, written 2026-06-07) is the architectural blueprint for v1.7. Five concrete items emerged from it; three ship with this release as public reference implementations. The remaining two (FORGET mechanism, `distill-memory.py` fix) are private-infrastructure items deferred to v1.8.

**1. FORGET mechanism** — the confirmed missing leg of memory integrity (READ + WRITE are built; FORGET is not). Required components: `valid-as-of` + `falsification-pointer` fields on every significant memory entry; an invalidation signal hook (PostToolUse, fires on file rename/delete, flags matching memory entries as stale — writes to a sidecar path, never to MEMORY.md directly per g-immutable); a reconciliation detector script that produces stale/ok/unverified report across the full Hindsight store; Hindsight store expiry path (TTL + last-accessed). This is the hard prerequisite for Phase D (reorg/global deploy) — Phase D must not start until steps 0.1–0.5 of the FORGET track are complete.
### What v1.7 covers (the "provable hooks" release)

**2. Startup gate hook** — a SessionStart hook that checks all four surfaces (memory, hooks, AGENTS.md, skills) at cold start and reports drift before any task begins. 7 checks: repo detection, AGENTS.md verification, active plan detection, memory conflict check, skills manifest currency, hook registration gap, conflict resolution. Key design decision: output goes to `~/.claude/startup-gate-report.md` (NOT `additionalContext` — confirmed ineffective by DC3 live test 2026-06-07 where two fresh sessions never surfaced the alert). Session-lifecycle.md Phase 1 reads the file explicitly. Blocked until the hook normalization helper (item 3) exists.
v1.6 shipped hooks. v1.7 answers: *can you prove they're working, can they survive across runtimes, and do you know what to do at cold start?* Three new components:

**3. Hook normalization helper** — `normalize-hook-input.sh` — a shared library that normalizes hook input JSON across runtimes. Hooks written for Claude Code's `tool_name`/`tool_input` snake_case + `"Bash"`/`"Edit"` literals silently bypass on Grok's `toolName`/`toolInput` camelCase + `"run_terminal_cmd"`/`"search_replace"` shape. Without this helper, every new hook has the same silent-bypass failure mode on non-Claude-Code runtimes.
### Added

- **`examples/hooks/lib/normalize-hook-input.sh`** — shared payload normalization library. Hooks written for Claude Code's `tool_name`/`tool_input` snake_case + `"Bash"`/`"Edit"` literals silently bypass on Grok's `toolName`/`toolInput` camelCase + `"run_terminal_cmd"`/`"search_replace"` shape. This library normalizes both field names and tool-name literals so every hook that sources it works regardless of runtime. Provides two functions: `nh_normalize` (full normalization, stdout), `nh_tool_name` (quick dispatch helper). Fail-open: if python3 is missing or JSON parse fails, returns input unchanged — pre-normalization behavior, no regression.

- **`examples/hooks/startup-gate.sh`** — SessionStart hook. Checks all four governance surfaces at cold start and writes a drift report to `~/.claude/startup-gate-report.md`. 7 checks: repo detection, AGENTS.md verification, active plan detection, memory conflict detection (stub — FORGET mechanism pending), skills manifest currency, hook registration gap (disk vs settings.json diff), conflict resolution. Key design decision: output goes to a file (NOT `additionalContext` — confirmed ineffective by live test where fresh sessions never surfaced the alert). Session-lifecycle.md Phase 1 reads the file explicitly. Annotations: `# fail-mode: open`, `# blast-radius: advisory`.

- **`examples/hooks/hook-telemetry-stop.sh`** — Stop hook. Runs at session end; reads fire and block breadcrumbs from `breadcrumb-lib.sh`, bulk-INSERTs one row per hook into a telemetry store. Answers v1.6's "are hooks theater?" question with operational data. Sanitized to use a configurable `AOF_TELEMETRY_URL` + `AOF_TELEMETRY_KEY` environment variable pattern (private implementation uses smokin-ops; adapters plug in at the env-var layer). Annotations: `# fail-mode: open`, `# blast-radius: advisory`.

- **`guides/advanced/silent-failure-discipline.md`** — documents ADR 0012: every hook fail-open path must write a tab-separated line to `.errors.log` via a `bc_write_error` call or equivalent direct write. Pattern, self-test command, and the 1441-entry error log as proof of the contract working. Sourced from a full governance audit of 63 hooks (2026-06-07) that found ~50 silent fail-opens.

- **`guides/advanced/agents-md-standard.md`** — codifies the AGENTS.md governance contract. Three levels: Level 1 (identity sentence only), Level 2 (identity + subagent routing table), Level 3 (identity + routing + procedures + ADR register). Reference implementations from `smokin-os` and `smokin-memory` repos (2026-06-07). Explains why a repo-level governance declaration matters for multi-agent environments.

- **3 new sanitized incidents** in `INCIDENTS.md` (#36, #37, #38).

### Changed

**4. Per-invocation telemetry + opportunity counter** — extends `telemetry.hook_events` with `repo_cwd` + `tool_outcome` columns; adds `eval.opportunities` table for the AOF self-eval DPMO denominator. This feeds the self-eval harness (`b3d8451`, already on main) with real operational data — rules scoreboard, DPMO chart, hook compliance scoring. Has a mandatory day-7 goal DC requiring an active fire mechanism (a reminder hook that persists until the DC is confirmed by live query). Neither this item nor the self-eval Phase 1 dashboard close without a unified migration covering both the telemetry columns and the self-eval columns (`latency_ms_avg`, `latency_ms_p95`, `first_block_at`, `exit_codes_seen`).
- **`AGENT_FRAMEWORK.md`** — version bump to v1.7. §5.3 matrix updated: `startup-gate.sh` added as a meta-hook covering the session-lifecycle open step; `normalize-hook-input.sh` added as a cross-runtime library. Framework structure section updated with new `examples/hooks/lib/` directory and two new advanced guides.
- **`README.md`** — v1.7 references, 38 incidents, new guides in library table, `startup-gate.sh` and `normalize-hook-input.sh` in hook table.
- **`examples/hooks/README.md`** — 3 new inventory rows.

**5. `distill-memory.py` regex fix (PREREQ-B1)** — the Phase B memory distiller has been silently no-op'ing since ship (2026-05-28) because the commit SHA regex matches YYYYMMDD dates as hex strings, firing the info-loss invariant on every cron run. Fix: tighten the regex to require ≥1 non-decimal hex digit. This unblocks both the FORGET track and the telemetry track — neither can be properly validated while the distiller is dead.
### What is NOT in v1.7 (deferred to v1.8)

**Plus (from v1.6 deferred list):**
- `telemetry.hook_events` schema — sanitized port to `examples/` (coordinates with item 4)
- `hook-telemetry-stop.sh` — sanitized port to `examples/hooks/`
- Silent failure discipline (ADR 0012) — 63-hook governance audit + ~50 fail-opens fixed 2026-06-07; the "we can prove it works" release. Pattern: every fail-open path writes to `.errors.log` via `bc_write_error`; watchdog surfaces on session close.
- **FORGET mechanism** — the missing leg of memory integrity (READ + WRITE are built; FORGET is not). Requires `valid-as-of` + `falsification-pointer` fields on memory entries, an invalidation signal hook, and a reconciliation detector. This is the hard prerequisite for Phase D (global deploy).
- **`distill-memory.py` regex fix** — Phase B memory distiller has been silently no-op'ing since ship (2026-05-28): commit SHA regex matches YYYYMMDD dates as hex strings, firing the info-loss invariant on every cron run. Fix: tighten regex to require ≥1 non-decimal hex digit.
- **AOF self-eval Phase 1 dashboard** — harness (`b3d8451`) is on main; full operational data integration requires the telemetry columns + opportunity counter migration that `hook-telemetry-stop.sh` feeds.

### Release checklist

- [x] Fix CHANGELOG duplicate [1.7] section
- [x] Port `examples/hooks/lib/normalize-hook-input.sh`
- [x] Port `examples/hooks/startup-gate.sh`
- [x] Port `examples/hooks/hook-telemetry-stop.sh`
- [x] Write `guides/advanced/silent-failure-discipline.md`
- [x] Write `guides/advanced/agents-md-standard.md`
- [x] Add incidents #36, #37, #38
- [x] Update `AGENT_FRAMEWORK.md` v1.6 → v1.7
- [x] Update `README.md`
- [x] Update `examples/hooks/README.md`
- [x] `git tag v1.7 && git push --tags`
- [x] Create GitHub release

---

Expand Down Expand Up @@ -106,16 +136,6 @@ This v1.6 release notes draft consolidates two consecutive overnight sessions (2

---

## [1.7] — TBD (Notes)

Candidate themes and material surfaced during v1.6 work. Not committed — placeholder only.

- **AOF self-eval harness** — `b3d8451` on main, untagged by v1.6. The framework measuring itself: rules scoreboard, DPMO chart, hook compliance scoring (`eval.rules`, `eval.aof_eval_checkpoints`, `eval.opportunities`). Large enough to anchor v1.7.
- **Silent failure discipline** — full governance audit of 63 hooks done 2026-06-07; ~50 silent fail-opens fixed and wired to the error log via ADR 0012 (silent-hook watchdog). Directly answers v1.6's "are hooks theater?" question with a systematic, measurable fix.
- **AGENTS.md governance standard** — repos (`smokin-os`, `smokin-memory`) shipped Level 3 `AGENTS.md` files 2026-06-07. New pattern AOF doesn't document yet: how a repo declares its agent governance contract. v1.7 could codify the spec.

---

## [1.5] — 2026-05

### Added
Expand Down
4 changes: 4 additions & 0 deletions INCIDENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,10 @@ Each row shows the incident, the lesson, and the rule it hardened. Dates are mon
| 34 | 2026-05 | A breadcrumb library used an undocumented env var (`CLAUDE_SESSION_ID`) as its primary session key. The var was set on Mac via shell init but empty on Win11, causing source-truth-write gates to silently reject all writes on Win11 for ~14 days. Registration and annotation checks both passed; the failure was invisible until a separate audit. | Hook libraries must document their runtime dependencies as inline `SCHEMA ASSUMPTION` blocks naming the canonical env var, any legacy backstops, and the conditions under which each may be removed. Cross-subprocess PID stability must be verified by a regression smoke that tests two separate subprocess invocations, not just one. | [`breadcrumb-lib.sh`](examples/hooks/breadcrumb-lib.sh) + [`guides/advanced/hook-audit-methodology.md`](guides/advanced/hook-audit-methodology.md) Track 3 |
| 35 | 2026-05 | A watcher hook auto-pushed to a branch-protected repo and logged `PUSH-FAIL` approximately every 30 seconds for ~14 days. The watcher continued running — it committed locally on each cycle and then failed push — so there was no crash to notice. Discovered only during an unrelated hook audit. | Watcher-class hooks that auto-push to a protected branch must use the `PUSH-SKIPPED` pattern: keep the commit step (local backup), skip the push step, log `PUSH-SKIPPED branch-protection-active`, and document the manual PR step. Without telemetry on hook fires, this class of silent degradation is undetectable. | [`guides/hook-operations.md`](guides/hook-operations.md) + `# fail-mode: silent-skip` taxonomy tier |

| 36 | 2026-06 | A SessionStart hook emitted a detailed governance report via `additionalContext`. Live test with two fresh sessions confirmed the content was never visible to the model — the harness assembles the system prompt before the hook's output is available. The hook author had assumed `additionalContext` was equivalent to a system-prompt injection; it is not. | SessionStart hook output that needs to reach the agent must be written to a file (e.g., `~/.claude/startup-gate-report.md`) and read explicitly in Phase 1 of session-lifecycle. Never rely on `additionalContext` as the delivery mechanism for governance content. | [`examples/hooks/startup-gate.sh`](examples/hooks/startup-gate.sh) |
| 37 | 2026-06 | Hooks written for Claude Code silently bypassed on a Grok runtime because Grok uses camelCase field names (`toolName`, `toolInput`) and different tool-name literals (`run_terminal_cmd` instead of `Bash`). Every hook that checked `tool_name == "Bash"` never matched on Grok — gates that should have blocked allowed unconditionally. Discovery was delayed because both runtimes produce the same exit-0 behavior on a successful gate pass. | Any hook intended to be multi-runtime must normalize the input payload before checking field names or tool-name literals. The normalization library (`normalize-hook-input.sh`) handles Claude Code → Grok field mapping and tool-name remapping; source it near the top of every hook that will run on more than one runtime. | [`examples/hooks/lib/normalize-hook-input.sh`](examples/hooks/lib/normalize-hook-input.sh) |
| 38 | 2026-06 | A governance audit of 63 hooks found ~50 fail-open paths that exited 0 without writing to the error log. The watchdog hook (ADR 0012) could only report on hooks that already wrote error-log entries — the ~50 silent hooks were invisible to it. The watchdog's "no errors found" output was therefore meaningless: it was reporting on a biased sample (the hooks that already knew how to fail noisily), not on the full population. | The error-log contract is only valuable if it covers every fail-open path, not just the paths that already knew about it. Audit each hook at ship time: force the fail-open path and confirm a row appears in `.errors.log`. See `guides/advanced/silent-failure-discipline.md` for the full pattern and self-test command. | [`guides/advanced/silent-failure-discipline.md`](guides/advanced/silent-failure-discipline.md) |

---

*This log is maintained alongside the rules. When a new rule is added, the incident that produced it belongs here.*
Loading
Loading