Skip to content

Commit 4571088

Browse files
hyperpolymathclaude
hyperpolymath
and
claude
authored
docs(data): ADR-0002 + per-directory READMEs — populate, not remove (#5) (#12)
The dispatch/patterns/recipes/outcomes/policy/health/.well-known dirs flagged as "empty placeholders" are now live data sinks for the scan→triage→autofix pipeline. Records the populate-not-remove decision in ADR-0002 and adds the missing per-directory README explaining contents, format, writer/reader, and append-vs-rewrite semantics for each. No data files moved or removed; outcomes/.gitkeep documented as intentional. Closes #5. Co-authored-by: hyperpolymath <hyperpolymath@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 152629d commit 4571088

8 files changed

Lines changed: 217 additions & 0 deletions

File tree

‎.well-known/README.adoc‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
= .well-known/ — RFC 8615 web metadata
3+
:status: populated web metadata (see docs/decisions/ADR-0002-data-subdir-population.adoc)
4+
5+
RFC 8615 well-known URIs, published at the site root when this repo is served
6+
via GitHub Pages. This is web-standard metadata, *not* a pipeline data sink —
7+
it is listed here only because issue #5 grouped it with the data dirs.
8+
9+
Files::
10+
* `ai.txt` — AI interaction policy (training/summarisation/generation
11+
permissions; PMPL Emotional Lineage requirement). Points agents at
12+
`0-AI-MANIFEST.a2ml`.
13+
* `humans.txt` — humanstxt.org credits/colophon.
14+
* `security.txt` — RFC 9116 security disclosure contact.
15+
16+
Writer: maintainers (hand-edited). Reader: web clients, crawlers, AI
17+
agents, security researchers. Edited in place; not deleted.

‎dispatch/README.adoc‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
= dispatch/ — scan→triage work queue
3+
:status: populated data store (see docs/decisions/ADR-0002-data-subdir-population.adoc)
4+
5+
Append-only JSONL work queue emitted by the panic-attacker scan → triage
6+
stage. One JSON object per line; one finding per object.
7+
8+
Files::
9+
* `dispatch-YYYY-MM-DD.jsonl` — the day's dispatched findings (immutable once
10+
written).
11+
* `pending.jsonl` — findings accepted but not yet actioned by a bot.
12+
* `held.jsonl` — findings deferred or quarantined (e.g. low confidence,
13+
manual-review required).
14+
15+
Record shape::
16+
`action`, `category`, `confidence`, `auto_fixable`, `description`,
17+
`pattern_id`, `recipe_id`, `replacement`, `repo`, `program_path`, `severity`,
18+
`strategy`, `tier`, `timestamp`.
19+
20+
Writer: the triage stage of the ingest pipeline. Reader: the autofix /
21+
review bots. Append-only — never rewritten in place.

‎docs/decisions/ADR-0002-data-subdir-population.adoc‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
3+
= ADR-0002: data subdirectories are populated and self-documented, not removed
4+
:revdate: 2026-05-17
5+
:status: Accepted
6+
7+
== Status
8+
9+
Accepted — 2026-05-17.
10+
11+
Resolves: https://github.com/hyperpolymath/verisimdb-data/issues/5[V-L3-S1].
12+
13+
Builds on: xref:ADR-0001-repo-purpose.adoc[ADR-0001] (the two-purpose framing;
14+
these directories are purpose 1, the flat-file data store).
15+
16+
== Context
17+
18+
Issue #5 observed that `dispatch/`, `patterns/`, `recipes/`, `outcomes/`,
19+
`policy/`, `health/` and `.well-known/` looked like "mostly empty placeholder
20+
directories" and asked for a binary decision: *populate* them (one issue per
21+
subdir to make the contents concrete) or *delete* them and rebuild as content
22+
lands.
23+
24+
That premise is now stale. ADR-0001 already declared all of these subtrees as
25+
the repository's flat-file data store, and the panic-attacker → triage →
26+
autofix pipeline has since filled every one of them with production data:
27+
28+
* `dispatch/` — dated work-queue JSONL plus `pending.jsonl` / `held.jsonl`
29+
* `patterns/` — `registry.json` (the cross-repo pattern registry)
30+
* `recipes/` — ~46 `recipe-*.json` remediation recipes
31+
* `outcomes/` — monthly applied-fix ledgers plus a fleet import
32+
* `policy/` — `policy.ncl` (the Nickel baseline policy contract)
33+
* `health/` — `sitrep.txt` and `hypatia.json` operational telemetry
34+
* `.well-known/` — RFC 8615 web metadata (`ai.txt`, `humans.txt`,
35+
`security.txt`)
36+
37+
There are no empty placeholder directories left in the tree. The only real
38+
gap against the issue's acceptance criteria was the absence of a per-directory
39+
README explaining what each one holds.
40+
41+
== Decision
42+
43+
**Populate, not remove.** The data subdirectories are a permanent, declared
44+
part of the repository (per ADR-0001) and are now backed by real, actively
45+
written content. They are *not* placeholders and must not be deleted.
46+
47+
To make the contents concrete and self-documenting — the spirit of the issue's
48+
"one issue per subdir" alternative, achieved in one PR — each data directory
49+
carries a `README.adoc` that states:
50+
51+
. what artefact(s) it holds and their on-disk format,
52+
. who writes it (which pipeline stage / bot) and who reads it,
53+
. whether it is append-only or rewritten in place,
54+
. its retention / `.gitkeep` behaviour.
55+
56+
Splitting this into seven tracking issues was rejected: the directories are
57+
already populated, so seven issues would each open and immediately close as
58+
"add a README" with no design content. One ADR plus seven READMEs records the
59+
decision and discharges the acceptance criteria together.
60+
61+
== Consequences
62+
63+
. The `dispatch/ patterns/ recipes/ outcomes/ policy/ health/ .well-known/`
64+
directories each gain a `README.adoc`; no data files are moved or removed.
65+
. `outcomes/.gitkeep` is retained: the ledger is monthly, so the directory
66+
can legitimately contain only `.gitkeep` at the start of a month, and that
67+
is now documented rather than mistaken for an empty placeholder.
68+
. Future data directories under purpose 1 must ship with a `README.adoc`
69+
from the first commit (close the "is this a placeholder?" question at
70+
creation time).
71+
. ADR-0001's directory-layout list remains authoritative for *which* subtree
72+
serves *which* purpose; this ADR documents the *contents* of the data
73+
subtree.
74+
75+
== Alternatives considered
76+
77+
Delete the directories and rebuild as content lands::
78+
Rejected. The content has already landed and is being written continuously by
79+
the ingest/triage/autofix pipeline; deleting live data sinks would break that
80+
pipeline and lose history. This alternative only made sense under the (now
81+
false) assumption that the directories were empty.
82+
83+
One tracking issue per subdirectory::
84+
Rejected as busywork. The directories are populated; each issue would reduce
85+
to "add a README." Folded into this single ADR + the seven READMEs instead.
86+
87+
Move telemetry / web metadata out of the data store::
88+
Out of scope. `health/` and `.well-known/` are small and co-located by
89+
design; relocating them is a separate decision if either grows.

‎health/README.adoc‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
= health/ — operational telemetry
3+
:status: populated data store (see docs/decisions/ADR-0002-data-subdir-population.adoc)
4+
5+
Latest-state operational telemetry for the data store and its pipeline.
6+
These are *snapshots* (rewritten in place each run), not append-only logs —
7+
history lives in `outcomes/` and `dispatch/`.
8+
9+
Files::
10+
* `sitrep.txt` — human-readable situation report: last-run timestamp,
11+
counts (scans / patterns / actions / outcomes / recipes), the
12+
auto/review/report split, process up/down, contingency status, run time.
13+
* `hypatia.json` — machine-readable health snapshot consumed by the
14+
Hypatia scanner / Scorecard surface.
15+
16+
Writer: the pipeline's reporting stage. Reader: humans (sitrep) and the
17+
Hypatia / health-monitoring tooling (json).

‎outcomes/README.adoc‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
= outcomes/ — applied-fix outcome ledger
3+
:status: populated data store (see docs/decisions/ADR-0002-data-subdir-population.adoc)
4+
5+
Append-only JSONL ledger of what happened when a recipe was applied to a
6+
repo. Feeds the recipe success/fail counters and health reporting.
7+
8+
Files::
9+
* `YYYY-MM.jsonl` — one month of outcomes (immutable once the month closes).
10+
* `*-fleet-import.jsonl` — bulk historical imports.
11+
* `.gitkeep` — retained on purpose: the ledger is monthly, so at the start
12+
of a month this directory can legitimately hold only `.gitkeep`. That is
13+
the expected steady state, *not* an empty placeholder (see ADR-0002).
14+
15+
Record shape::
16+
`pattern`, `repo`, `bot`, `outcome` (`success` / failure), `fixed_at`.
17+
18+
Writer: the autofix/review bots after each attempt. Reader: recipe
19+
counter rollup and `health/`. Append-only.

‎patterns/README.adoc‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
= patterns/ — canonical cross-repo pattern registry
3+
:status: populated data store (see docs/decisions/ADR-0002-data-subdir-population.adoc)
4+
5+
`registry.json` — the single canonical pattern registry. It deduplicates
6+
raw findings across every scanned repo into trackable, long-lived patterns.
7+
8+
Shape::
9+
Top-level `description`, `last_updated`, and a `patterns` map keyed by
10+
pattern id. Each entry: `id`, `category`, `description`, `pa_rule`,
11+
`occurrences`, `first_seen`, `last_seen`, `recipe_id` (the remediation
12+
recipe, or `null` if none yet), `repo_paths`.
13+
14+
Writer: the pattern-aggregation stage of the ingest pipeline (rewritten in
15+
place on each run — it is a registry, not a log). Reader: triage (to attach
16+
`recipe_id`) and reporting/health.

‎policy/README.adoc‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
= policy/ — baseline policy contract
3+
:status: populated data store (see docs/decisions/ADR-0002-data-subdir-population.adoc)
4+
5+
`policy.ncl` — the Nickel baseline policy contract for this data store.
6+
7+
Contents::
8+
* `enforcement` — repo hygiene flags (`require_spdx_headers`,
9+
`require_ci_security_checks`, `block_committed_secrets`,
10+
`require_pinned_actions`).
11+
* `triage` — confidence thresholds: `auto_fix_min_confidence`,
12+
`review_min_confidence`.
13+
* `version` — policy schema version.
14+
15+
Writer: maintainers (hand-edited; version-bumped on change). Reader:
16+
`contractiles/must/Mustfile` consumes it to gate enforcement, and triage
17+
reads the confidence thresholds to choose auto-fix vs. review vs. report.
18+
Single file, edited in place.

‎recipes/README.adoc‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
= recipes/ — remediation recipe knowledge base
3+
:status: populated data store (see docs/decisions/ADR-0002-data-subdir-population.adoc)
4+
5+
One `recipe-*.json` file per remediation recipe — the knowledge base the
6+
autofix bot consults to turn a matched pattern into a fix.
7+
8+
Shape (per file)::
9+
`id`, `action`, `description`, `confidence`, `auto_fixable`, `fix_script`
10+
(or `null`), `languages`, `match`, `replacement`, `pattern_ids` (the
11+
patterns this recipe remediates), `proven_module`, `triangle_tier`
12+
(eliminate / substitute / …), and running counters
13+
`total_attempts` / `successful_fixes` / `failed_fixes`.
14+
15+
Naming::
16+
`recipe-<slug>.json`; the `recipe-scorecard-*` family maps to OpenSSF
17+
Scorecard checks.
18+
19+
Writer: curated, plus counter updates from outcome ingest. Reader: the
20+
autofix bot and triage (recipe lookup by `pattern_id`).

0 commit comments

Comments
 (0)