Skip to content

Commit 35b5f7e

Browse files
hyperpolymathclaude
hyperpolymath
and
claude
authored
Step 0+1: dual-purpose ADR + doc alignment + index.json regen + freshness CI (#4)
* docs(adr): record dual-purpose framing — scan store + ABI dogfood Resolves the conflicting identities advertised across README/EXPLAINME/ TOPOLOGY by accepting that the repo serves two purposes and documenting the split. Closes #2 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: align README/EXPLAINME/TOPOLOGY around two-purpose framing Per ADR-0001 (repo purpose) the repository serves two explicit purposes: flat-file scan/drift data store, and ABI dogfood for the hyperpolymath Idris2 + Zig standard. The three top-level docs each described a different subset of this; bringing them into agreement. - README.adoc — rewritten with explicit two-purpose intro, a layout-by-purpose table, a Related-repos cross-link to verisimiser, and an index-freshness pointer (the CI lane added in a sibling commit). Preserves the existing scan ingest workflow + JSON format documentation under Purpose 1. - EXPLAINME.adoc — quote at top reflects both purposes; file map expanded to call out which subtrees serve which purpose; pointer to verisimiser added. - TOPOLOGY.md — Purpose section restated; module map updated to reflect the actual directory layout (no `hardware/` or `drift/` dirs — those were placeholders in the old doc); ffi/zig/ called out; integration points include verisimiser. Closes #2 (V-L1-J1 implementation; the ADR was the decision). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * data: regen index.json from scans/ (301 entries) + add freshness CI The committed index.json was stale: total_scans=301 (correct) but repos=["nafa-app","outreach","playbooks","project-cb","v-ecosystem"] (5 entries when there are 301 on disk), last_updated=2026-03-07 (two months stale at time of regen). Changes: - scripts/regen-index.sh — deterministic regen from scans/ on disk. last_updated derived from `git log -1 --format=%cI -- scans` so running it twice produces byte-identical output. Schema preserved (last_updated/total_scans/total_repos/repos); subdirectories under scans/ (e.g. scans/octads/) are intentionally not counted in total_scans — they are a separate space; if they need their own index, that is a follow-up. - index.json — regenerated. 301 scans, 301 repos (one repo per top-level file), last_updated=2026-03-28T08:28:32+00:00 (commit time of the most recent scans/ change). - .github/workflows/index-freshness.yml — runs regen-index.sh on push/PR and fails if index.json drifts from scans/. Triggers only on changes to scans/, index.json, the regen script, or the workflow itself. Pair with branch protection so the check is required for merge. Verified locally: - bash scripts/regen-index.sh produces a clean tree on second run (`git diff --quiet index.json` passes). Closes #1 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 8a7b644 commit 35b5f7e

7 files changed

Lines changed: 622 additions & 48 deletions

File tree

‎.github/workflows/index-freshness.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# SPDX-License-Identifier: PMPL-1.0-or-later
2+
# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
3+
#
4+
# Fails if the committed index.json drifts from what scripts/regen-index.sh
5+
# produces from scans/ on disk. Pair with branch protection so this check
6+
# is required on PRs touching scans/ or index.json.
7+
#
8+
# Closes V-L3-M1.
9+
10+
name: index.json freshness
11+
12+
on:
13+
push:
14+
branches: [main]
15+
pull_request:
16+
branches: [main]
17+
paths:
18+
- 'scans/**'
19+
- 'index.json'
20+
- 'scripts/regen-index.sh'
21+
- '.github/workflows/index-freshness.yml'
22+
23+
concurrency:
24+
group: index-freshness-${{ github.ref }}
25+
cancel-in-progress: true
26+
27+
permissions:
28+
contents: read
29+
30+
jobs:
31+
check:
32+
name: index.json matches scans/
33+
runs-on: ubuntu-latest
34+
timeout-minutes: 5
35+
steps:
36+
- name: Checkout (with history; regen-index needs git log on scans/)
37+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
38+
with:
39+
fetch-depth: 0
40+
41+
- name: Regenerate index.json
42+
run: bash scripts/regen-index.sh
43+
44+
- name: Fail if index.json drifted
45+
run: |
46+
if ! git diff --quiet --exit-code index.json; then
47+
echo "::error::index.json is out of sync with scans/."
48+
echo "Run 'bash scripts/regen-index.sh' locally and commit the result."
49+
echo ""
50+
echo "----- diff -----"
51+
git --no-pager diff index.json
52+
exit 1
53+
fi
54+
echo "index.json is in sync with scans/."

‎EXPLAINME.adoc‎

Lines changed: 30 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -7,36 +7,54 @@ The README makes claims. This file backs them up.
77

88
[quote, README]
99
____
10-
Git-backed flat-file storage for scan results and drift detection data.
10+
This repository serves two explicit purposes: (1) a flat-file data
11+
store for panic-attacker scan results, hardware-crash-team findings,
12+
and drift snapshots; (2) an ABI dogfood site for the hyperpolymath
13+
Idris2 + Zig ABI shared with `proven`, `burble`, `gossamer`.
1114
____
1215

13-
== Technology Choices
16+
See `docs/decisions/ADR-0001-repo-purpose.adoc` for why both purposes
17+
live in one repo.
18+
19+
== Technology choices (Purpose 2 — ABI)
1420

1521
[cols="1,2"]
1622
|===
1723
| Technology | Learn More
1824

19-
| **Zig** | https://ziglang.org
25+
| **Zig** | https://ziglang.org
2026
| **Idris2 ABI** | https://www.idris-lang.org
2127
|===
2228

23-
== Dogfooded Across The Account
24-
25-
Uses the hyperpolymath ABI/FFI standard (Idris2 + Zig). Same pattern used across
29+
The Zig FFI implementation lives in `ffi/zig/` and follows the same
30+
pattern as
2631
https://github.com/hyperpolymath/proven[proven],
2732
https://github.com/hyperpolymath/burble[burble], and
2833
https://github.com/hyperpolymath/gossamer[gossamer].
2934

30-
== File Map
35+
== File map
3136

32-
[cols="1,2"]
37+
[cols="1,2,1"]
3338
|===
34-
| Path | What's There
35-
36-
| `src/` | Source code
37-
| `ffi/` | Foreign function interface
39+
| Path | What's there | Purpose
40+
41+
| `scans/` | Per-repo scan results (JSON; ~301 files) | data
42+
| `dispatch/` | Dispatch records | data
43+
| `patterns/` | Drift / scan pattern definitions | data
44+
| `recipes/` | Ingest and aggregation recipes | data
45+
| `outcomes/` | Outcome records | data
46+
| `policy/` | Storage and retention policy notes | data
47+
| `health/` | Health-state snapshots | data
48+
| `index.json` | Master index of stored data | data
49+
| `ffi/zig/` | Zig FFI implementation | ABI dogfood
3850
|===
3951

52+
== Related
53+
54+
`verisimiser` (https://github.com/hyperpolymath/verisimiser) is the
55+
augmentation CLI; this repo holds its scan/drift output but is not a
56+
runtime dependency.
57+
4058
== Questions?
4159

4260
Open an issue or reach out directly — happy to explain anything in more detail.

‎README.adoc‎

Lines changed: 54 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,49 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
13
= VeriSimDB Data Repository
24

3-
Git-backed flat-file storage for scan results and drift detection data.
5+
This repository serves *two explicit purposes*, per
6+
`docs/decisions/ADR-0001-repo-purpose.adoc`:
47

5-
== Structure
8+
. **Flat-file data store** for panic-attacker scan results,
9+
hardware-crash-team findings, and drift snapshots.
10+
. **ABI dogfood** for the hyperpolymath Idris2 + Zig ABI standard
11+
shared with https://github.com/hyperpolymath/proven[`proven`],
12+
https://github.com/hyperpolymath/burble[`burble`], and
13+
https://github.com/hyperpolymath/gossamer[`gossamer`].
614

7-
- `scans/` - panic-attack scan results per repo
8-
- `hardware/` - hardware-crash-team findings
9-
- `drift/` - drift detection snapshots
10-
- `index.json` - Master index of all stored data
15+
== Related repos
1116

12-
== Usage
17+
* https://github.com/hyperpolymath/verisimiser[`verisimiser`] — the
18+
augmentation CLI. This repo holds its scan and drift output;
19+
standalone consumers (`panic-attacker`, `hardware-crash-team`) also
20+
write here. Verisimiser does not require this repo at runtime.
1321

14-
This repo receives scan results via GitHub Actions workflow_dispatch events
15-
and stores them as JSON files. The ingest workflow updates the index
16-
automatically.
22+
== Layout (by purpose)
1723

18-
== Integration
24+
[cols="1,2,1"]
25+
|===
26+
| Path | What's there | Purpose
1927

20-
=== Sending Scan Results
28+
| `scans/` | Per-repo scan results (JSON; ~301 files) | data
29+
| `dispatch/` | Dispatch records | data
30+
| `patterns/` | Drift / scan pattern definitions | data
31+
| `recipes/` | Ingest and aggregation recipes | data
32+
| `outcomes/` | Outcome records | data
33+
| `policy/` | Storage and retention policy notes | data
34+
| `health/` | Health-state snapshots | data
35+
| `index.json` | Master index of stored data | data
36+
| `ffi/zig/` | Zig FFI implementation | ABI dogfood
37+
| `.machine_readable/` | Contractile machinery (org-wide) | both
38+
| `.github/workflows/` | CI (ingest, mirrors, governance) | both
39+
| `docs/decisions/` | ADRs | both
40+
|===
2141

22-
Other repos can send scan results using the reusable workflow:
42+
== Purpose 1 — flat-file data store
43+
44+
=== Receiving scan results
45+
46+
Other repos send scan results using the reusable workflow:
2347

2448
[source,yaml]
2549
----
@@ -35,19 +59,19 @@ jobs:
3559
uses: hyperpolymath/panic-attacker/.github/workflows/scan-and-report.yml@main
3660
----
3761

38-
=== Querying Results
62+
=== Querying results
3963

4064
Clone this repo and read the JSON files:
4165

4266
[source,bash]
4367
----
4468
git clone https://github.com/hyperpolymath/verisimdb-data
4569
cd verisimdb-data
46-
cat scans/echidna.json # View specific repo results
47-
jq '.repos' index.json # View all scan summaries
70+
cat scans/echidna.json # specific repo
71+
jq '.repos' index.json # all scan summaries
4872
----
4973

50-
== File Format
74+
=== File format
5175

5276
Each scan result in `scans/` is a JSON file with the structure:
5377

@@ -75,6 +99,19 @@ Each scan result in `scans/` is a JSON file with the structure:
7599
}
76100
----
77101

102+
=== Index freshness
103+
104+
`index.json` is the master summary. A CI lane regenerates it from
105+
`scans/` on every push and fails if the working tree changes (i.e. the
106+
committed index must match what the regen script produces from the
107+
files on disk). See `scripts/regen-index.sh`.
108+
109+
== Purpose 2 — ABI dogfood
110+
111+
`ffi/zig/` is a working Zig FFI implementation of the hyperpolymath
112+
Idris2 + Zig ABI standard. The same pattern is used in `proven`,
113+
`burble`, and `gossamer`; this repo serves as the dogfood instance.
114+
78115
== License
79116

80117
SPDX-License-Identifier: PMPL-1.0-or-later

‎TOPOLOGY.md‎

Lines changed: 35 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -5,30 +5,50 @@
55

66
## Purpose
77

8-
VeriSimDB Data Repository is a git-backed flat-file storage system for scan results and drift detection data. Receives panic-attack scan results, hardware-crash-team findings, and drift detection snapshots via GitHub Actions workflow_dispatch events. Maintains master index and enables historical analysis of repository health and compliance drift over time.
8+
This repository serves two explicit purposes per
9+
[`docs/decisions/ADR-0001-repo-purpose.adoc`](docs/decisions/ADR-0001-repo-purpose.adoc):
910

10-
## Module Map
11+
1. **Flat-file data store** for panic-attacker scan results,
12+
hardware-crash-team findings, and drift snapshots received via
13+
GitHub Actions `workflow_dispatch` events. Maintains a master
14+
index for historical analysis of repository health and compliance
15+
drift over time.
16+
2. **ABI dogfood** for the hyperpolymath Idris2 + Zig ABI standard
17+
(shared with `proven`, `burble`, `gossamer`). Lives in `ffi/zig/`.
18+
19+
## Module map
1120

1221
```
1322
verisimdb-data/
14-
├── scans/ # panic-attack scan results per repo
15-
├── hardware/ # hardware-crash-team findings
16-
├── drift/ # drift detection snapshots
17-
├── index.json # Master index of all stored data
18-
└── .github/workflows/ # Ingest workflows (receive results)
23+
├── scans/ # panic-attacker scan results per repo (data)
24+
├── dispatch/ # dispatch records (data)
25+
├── patterns/ # drift / scan pattern definitions (data)
26+
├── recipes/ # ingest and aggregation recipes (data)
27+
├── outcomes/ # outcome records (data)
28+
├── policy/ # storage and retention policy notes (data)
29+
├── health/ # health-state snapshots (data)
30+
├── index.json # master index of stored data (data)
31+
├── ffi/
32+
│ └── zig/ # Zig FFI implementation (ABI dogfood)
33+
├── scripts/ # ingest + index regen scripts (data)
34+
├── docs/
35+
│ └── decisions/ # ADRs
36+
└── .github/workflows/ # ingest + governance workflows
1937
```
2038

21-
## Data Flow
39+
## Data flow (Purpose 1)
2240

2341
```
2442
[Workflow Dispatch] ──► [Ingest Handler] ──► [JSON Validation] ──► [File Storage]
25-
↓
26-
[Index Update] ──► [Query Ready]
43+
│
44+
[Index Regen on push] ──► [Query Ready]
2745
```
2846

29-
## Integration Points
47+
## Integration points
3048

31-
- **panic-attack**: Upstream scanner sending results
32-
- **hardware-crash-team**: Hardware failure analysis
33-
- **Drift detection**: Compliance change tracking
34-
- **Hyperpolymath CI/CD**: Automated data aggregation
49+
- **panic-attacker**: Upstream scanner sending results into `scans/`.
50+
- **hardware-crash-team**: Hardware failure analysis (Purpose 1 consumer).
51+
- **Drift detection**: Compliance change tracking against `scans/` history.
52+
- **verisimiser**: Consumes scan/drift data when wired to this repo as
53+
its sidecar storage. Not a runtime dependency.
54+
- **hyperpolymath CI/CD**: Automated data aggregation.

‎docs/decisions/ADR-0001-repo-purpose.adoc‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
3+
= ADR-0001: verisimdb-data carries two explicit purposes
4+
:revdate: 2026-05-13
5+
:status: Accepted
6+
7+
== Status
8+
9+
Accepted — 2026-05-13.
10+
11+
Resolves: https://github.com/hyperpolymath/verisimdb-data/issues/2[V-L1-J1].
12+
13+
== Context
14+
15+
This repository advertised three different identities across its top-level
16+
docs:
17+
18+
. `README.adoc` — "Git-backed flat-file storage for scan results and drift
19+
detection data" with panic-attacker scan ingest workflow.
20+
. `EXPLAINME.adoc` — "Uses the hyperpolymath ABI/FFI standard
21+
(Idris2 + Zig). Same pattern used across `proven`, `burble`, and
22+
`gossamer`."
23+
. `TOPOLOGY.md` — "Receives panic-attack scan results,
24+
hardware-crash-team findings, and drift detection snapshots… enables
25+
historical analysis of repository health and compliance drift over time."
26+
27+
The directory layout backs two of these stories with real content:
28+
29+
* `scans/` holds ~301 JSON files of scan results (production data).
30+
* `ffi/zig/` holds working Zig FFI code (`build.zig` + three `.zig` files).
31+
32+
The contractile and `.machine_readable/` scaffolding is org-wide
33+
infrastructure that applies to either purpose.
34+
35+
== Decision
36+
37+
The repository serves **two purposes**, explicitly declared in `README.adoc`:
38+
39+
. **Flat-file data store** for panic-attacker scan output and drift
40+
snapshots (`scans/`, `dispatch/`, `patterns/`, `recipes/`, `outcomes/`,
41+
`policy/`, `health/`).
42+
. **ABI dogfood site** for the hyperpolymath Idris2 + Zig ABI standard
43+
shared with `proven`, `burble`, and `gossamer` (`ffi/zig/`).
44+
45+
Both purposes coexist; neither is moved out. The acceptable layout is
46+
documented at the top of `README.adoc` and reflected in `TOPOLOGY.md`.
47+
48+
Verisimiser does *not* depend on `ffi/zig/` here; if a downstream consumer
49+
needs the Zig FFI extracted, that is a future ADR.
50+
51+
== Consequences
52+
53+
. `README.adoc`, `EXPLAINME.adoc`, and `TOPOLOGY.md` are brought into
54+
agreement on the two-purpose framing (one PR with this ADR).
55+
. The directory layout document in `README.adoc` lists which subtrees
56+
serve which purpose.
57+
. Future expansions to either purpose are fine — the repo is explicitly
58+
multi-purpose, not "drifting" between purposes.
59+
60+
== Alternatives considered
61+
62+
Scan-store only (move `ffi/zig/` out)::
63+
Rejected. The Zig FFI lives here for dogfooding reasons that predate
64+
verisimiser; relocating it costs more than documenting the dual purpose.
65+
66+
ABI dogfood only (move `scans/` out)::
67+
Rejected. The scan ingest workflow is wired up and producing data;
68+
moving 301 JSON files plus the ingest pipeline elsewhere has no payoff.
69+
70+
Split into two repos::
71+
Rejected for the same combined reason. Maybe revisit in a year if either
72+
purpose grows large enough to warrant its own repo and CI surface.

0 commit comments

Comments
 (0)