feat(aggregator): estate organization — manifest, thread runner, generated superproject (#63)

## What

Establishes the **one-aggregator** organization for the ~300-repo estate
so the dual-layout confusion (flat clones vs. stale submodule
superproject) is removed.

**The rule:** flat clones in `$REPOS_DIR` = single source of truth ·
`reposystem` = coordination layer · `repos-monorepo` = generated output
(Pages/mirror/governance keep working).

## Changes

- `repos.toml` — generated manifest of all 297 estate repos (`just
repos-manifest`)
- `repos.groups.toml` — hand-maintained epic/group index, survives
regeneration
- `scripts/thread.sh` — cross-repo thread runner; each repo merges via
its own PR
- `scripts/sync-aggregator.sh` — regenerates `repos-monorepo` as
generated artifact; per-submodule walk survives the URL-less
`.git-private-farm` entry
- `scripts/gen-repos-manifest.sh` — `.gitmodules` → `repos.toml`
generator
- `ESTATE-ORGANIZATION.adoc` — the rule + migration checklist
- `Justfile` — new recipes; **fixes a pre-existing parse-breaking
dedent** in `doctor` that was breaking the entire Justfile

## Notes

- No `STATE`/`contractiles`/`.bot_directives` changes.
- Verified: `just --list` parses; `just repos-manifest` → 297 entries;
`sync-aggregator` dry-run no longer aborts on `.git-private-farm`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: hyperpolymath

.machine_readable/6a2/STATE.a2ml

@@ -37,6 +37,13 @@ repo-batcher = "2026-05-18 — scaffoldia/repo-batcher: real `just e2e` ran agai
 [recent-changes]
 - 2026-05-18: repo-batcher #56 from-scratch build-out — L1–L4 ATS2 core (just tc-ats2=0), L5 self-contained C archive (ATS_DYNLOADFLAG 0 link fix, just ats2-lib=0), L6 Idris2 typed ABI w/ machine-checked theorems (just abi-check=0), L7 patscc-linked Zig CLI binding the genuine ATS2 C ABI, L8 fixture-backed just e2e=0. Independently fiction-reviewed from a clean clone. PR #59 merged (4c2d29f); issue #56 CLOSED-COMPLETED.
 - 2026-05-18: docs-truth — README/IMPLEMENTATION-STATUS/ARCHITECTURE corrected from the inverse anti-fiction defect ("still a stub") to the verified L1–L8 reality (PR #61). "No V ever" retained as true history.
+- 2026-05-18: Estate-aggregator organization landed (PR #63, branch feat/estate-aggregator-organization)
+- repos.toml (generated, 297 repos) + repos.groups.toml (hand-maintained epic index)
+- scripts/{gen-repos-manifest,thread,sync-aggregator,fix-stale-submodule-urls}.sh
+- Justfile: repos-manifest/thread/sync-aggregator/aggregator-drift recipes + fixed
+  pre-existing parse-breaking dedent in the doctor recipe
+- ESTATE-ORGANIZATION.adoc (linked from README) defines the source-of-truth rule:
+  flat clones authoritative, reposystem coordinates, repos-monorepo is generated
 - 2026-04-04: Added comprehensive test suite (E2E, property-based, aspect, benchmarks)
 - E2E tests: 20 tests covering CLI smoke, scan operations, config, security
 - Property tests: 12 tests for determinism, consistency, graph properties
@@ -50,6 +57,11 @@ repo-batcher = "2026-05-18 — scaffoldia/repo-batcher: real `just e2e` ran agai
 #             #56 CLOSED-COMPLETED). Docs-truth follow-up PR #61 merged.
 #             STATE.a2ml full refresh — factual fields updated; CRG grade and
 #             test count NOT fabricated (formal audit / re-run still required).
+# 2026-05-18: Estate-aggregator organization. One-source-of-truth rule (flat
+#             clones), repos.toml manifest + thread runner + generated
+#             repos-monorepo via sync-aggregator. PR #63. Pointer reconcile
+#             (sync-aggregator --push) executed; snif->snifs rename class found,
+#             fix-stale-submodule-urls.sh added. ml-dsa Dependabot bump queued.
 # 2026-04-25: D→C blocker sweep. EXPLAINME.adoc rewritten (15 fabricated paths
 #             removed, 31 verified-real paths added). 24 per-directory README.adoc
 #             files added across all subprojects.

ESTATE-ORGANIZATION.adoc

@@ -0,0 +1,106 @@
+// SPDX-License-Identifier: PMPL-1.0-or-later
+// SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
+= Estate Organization: one aggregator, threads off it
+:toc: preamble
+
+== The rule
+
+There are three layers and exactly one is the source of truth.
+
+[cols="1,3,3",options="header"]
+|===
+| Layer | Path | Role
+| Flat clone root | `$REPOS_DIR` (default `~/dev/repos/<repo>`) | *Single source of truth.* All work and branches happen here, in ordinary independent clones. Per-repo PRs/issues unchanged.
+| Aggregator | `reposystem` (this repo) | Coordination layer over the clones: the `repos.toml` index + the `thread` runner. Owns no code history; pins nothing.
+| Published artifact | `repos-monorepo` | *Generated output.* Submodule pointers are an auto-bumped index that keeps GitHub Pages, the GitLab/Bitbucket mirror, and the governance workflows working.
+|===
+
+Never `cd` into `repos-monorepo` to edit code. Never hand-resolve a submodule
+there. It is regenerated, not maintained.
+
+== Files
+
+* `repos.toml` — *generated* from `repos-monorepo/.gitmodules` by
+  `scripts/gen-repos-manifest.sh`. Regenerate with `just repos-manifest`.
+  NOTE: `0-AI-MANIFEST.a2ml` prefers generated files out of source control;
+  `repos.toml` is a *deliberate exception* — it is the coordination index and
+  must be present on a fresh clone / in CI without a regeneration step. It
+  carries an SPDX header and is fully reproducible from the documented command.
+* `repos.groups.toml` — *hand-maintained*. Names the repo sets a cross-repo
+  thread spans (epics). Survives manifest regeneration.
+
+== Cross-repo threads (the merge model)
+
+A thread is one logical change across N repos. Each repo still merges through
+its *own normal PR* — the runner only orchestrates the fan-out.
+
+[source,sh]
+----
+just thread resolve --kind julia                              # who is in scope
+just thread start  standards-130 --repos echo-types,affinescript   # branch per clone
+just thread pr     standards-130 --refs standards#130              # one PR per repo
+just thread status standards-130 --repos echo-types,affinescript   # CI/merge state
+just thread land   standards-130 --repos echo-types,affinescript --yes
+----
+
+Selectors: `--group <name>` (from `repos.groups.toml`), `--kind <kind>` (from
+`repos.toml`), or `--repos a,b,c`. `start` skips dirty worktrees; `land`
+refuses without `--yes` and only merges PRs GitHub reports `MERGEABLE`.
+
+== Keeping the published artifact current
+
+[source,sh]
+----
+just sync-aggregator            # DRY-RUN: show pointers that would bump
+just sync-aggregator --commit   # record the bump
+just sync-aggregator --push     # record + trigger Pages deploy + mirror
+----
+
+Submodules are updated one at a time (a single `git submodule update
+--recursive` aborts on the URL-less `.git-private-farm` entry — see
+`scripts/gitlink-drift.jl`). Run after `thread land`, or on a schedule, so the
+artifact never drifts.
+
+== Drift alarm
+
+[source,sh]
+----
+just aggregator-drift           # classify gitlink drift across the estate (A2ML)
+----
+
+== Warned submodules after a sync
+
+`sync-aggregator` leaves un-fetchable submodules as `warn ... (left as-is)`
+rather than aborting. `scripts/fix-stale-submodule-urls.sh` classifies them
+into four classes and remediates only on explicit, separately-gated flags:
+
+[cols="1,3,2",options="header"]
+|===
+| Class | Cause | Remediation
+| `RENAMED`     | Repo renamed; `.gitmodules` URL stale (API redirect resolves elsewhere). | `--apply-renames` rewrites the URL.
+| `NONEXISTENT` | `.gitmodules` over-declares a repo that was never created (API 404). URL is already basename-correct, so a rewrite cannot help. | `--prune-nonexistent` removes the declaration (*destructive*).
+| `WIKI`        | `.wikis/<repo>.wiki` entry with no enabled wiki — not tracked content. | `--prune-nonexistent` (labelled `WIKI`).
+| `TRANSIENT`   | Repo exists and URL is canonical — a fetch blip. | Re-run `just sync-aggregator`.
+|===
+
+[source,sh]
+----
+sh scripts/fix-stale-submodule-urls.sh                       # classify (dry-run)
+sh scripts/fix-stale-submodule-urls.sh --apply-renames       # safe: URL rewrites
+sh scripts/fix-stale-submodule-urls.sh --prune-nonexistent   # destructive: prune
+----
+
+It refuses to run while `sync-aggregator` holds the aggregator git lock, and
+never commits/pushes — re-run `just sync-aggregator --push` afterwards.
+
+== Migration status
+
+. [x] Rule declared (this document).
+. [ ] One-shot pointer reconcile: `just sync-aggregator --push` (fixes every
+  stale/empty gitlink — incl. `echo-types` — in one commit). *Run deliberately;
+  it fetches ~300 repos.*
+. [x] `repos.toml` generated from `.gitmodules` (297 repos).
+. [x] `repos.groups.toml` stub + `thread` / `sync-aggregator` /
+  `aggregator-drift` recipes.
+. [ ] Schedule `sync-aggregator --push` (cron/CI) so the artifact self-heals.
+. [ ] Optional: have `reposystem scan` assert manifest⇔clone parity.

Justfile

@@ -414,14 +414,14 @@ doctor:
     check "git"               git       "2.40" 
     check "Rust (cargo)"      cargo     "1.80" 
     check "Zig"               zig       "0.13" 
-# Optional tools
-if command -v panic-attack >/dev/null 2>&1; then
-    echo "  [OK]   panic-attack — available"
-    PASS=$((PASS + 1))
-else
-    echo "  [WARN] panic-attack — not found (pre-commit scanner)"
-    WARN=$((WARN + 1))
-fi
+    # Optional tools
+    if command -v panic-attack >/dev/null 2>&1; then
+        echo "  [OK]   panic-attack — available"
+        PASS=$((PASS + 1))
+    else
+        echo "  [WARN] panic-attack — not found (pre-commit scanner)"
+        WARN=$((WARN + 1))
+    fi
     echo ""
     echo "  Result: $PASS passed, $FAIL failed, $WARN warnings"
     if [ "$FAIL" -gt 0 ]; then
@@ -447,3 +447,34 @@ crg-badge:
       D) color="orange" ;; E) color="red" ;; F) color="critical" ;; \
       *) color="lightgrey" ;; esac; \
     echo "[![CRG $$grade](https://img.shields.io/badge/CRG-$$grade-$$color?style=flat-square)](https://github.com/hyperpolymath/standards/tree/main/component-readiness-grades)"
+
+
+# ============================================================================
+# ESTATE AGGREGATOR (flat clones = source of truth; this is the coordination
+# layer; repos-monorepo is generated output). See ESTATE-ORGANIZATION.adoc.
+# ============================================================================
+
+# Regenerate repos.toml from the superproject .gitmodules (groups preserved).
+repos-manifest:
+    @sh scripts/gen-repos-manifest.sh
+    @echo "✓ repos.toml regenerated (epic groups live in repos.groups.toml)"
+
+# Fan a cross-repo thread over the flat clones. Each repo merges via its own PR.
+#   just thread resolve --kind julia
+#   just thread start standards-130 --repos echo-types,affinescript
+#   just thread pr standards-130 --refs standards#130
+#   just thread status standards-130 --repos echo-types,affinescript
+#   just thread land standards-130 --repos echo-types,affinescript --yes
+thread *ARGS:
+    @sh scripts/thread.sh {{ARGS}}
+
+# Regenerate repos-monorepo's submodule index from pushed clone HEADs.
+# DRY-RUN by default; pass --commit or --push.
+#   just sync-aggregator           # preview
+#   just sync-aggregator --push    # record + trigger Pages/mirror
+sync-aggregator *ARGS:
+    @sh scripts/sync-aggregator.sh {{ARGS}}
+
+# Drift alarm: classify submodule-gitlink drift across the estate (A2ML out).
+aggregator-drift:
+    @julia scripts/gitlink-drift.jl "$${REPOS_DIR:-$$HOME/dev/repos}/repos-monorepo"

README.adoc

@@ -27,6 +27,21 @@ just web-serve
 
 Open `http://localhost:801` in a browser, then load `./export.json`.
 
+== Estate Organization
+
+Reposystem is the *coordination layer* for the ~300-repo estate: flat clones
+in `$REPOS_DIR` are the single source of truth, and `repos-monorepo` is a
+generated artifact. The rule, the cross-repo `thread` runner, the generated
+`repos.toml` manifest, and the migration checklist are documented in
+link:ESTATE-ORGANIZATION.adoc[ESTATE-ORGANIZATION.adoc].
+
+[source,bash]
+----
+just repos-manifest                  # regenerate repos.toml from .gitmodules
+just thread status <name> --group X  # cross-repo thread state
+just sync-aggregator --push          # regenerate repos-monorepo (Pages/mirror)
+----
+
 == Web UI (Prototype)
 
 The `web/` folder contains a no-build static UI for the graph:

repos.groups.toml

@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: PMPL-1.0-or-later
+# SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
+#
+# repos.groups.toml — HAND-MAINTAINED. Not generated; survives `just repos-manifest`.
+#
+# A "group" names a set of repos a cross-repo thread spans. The thread runner
+# (scripts/thread.sh) resolves --group <name> to this list, then operates on
+# the flat clones in $REPOS_DIR. Each repo still merges via its own normal PR;
+# this file is just the index of "what a thread touches".
+#
+# Add groups as epics arise. Example shape below — edit freely.
+
+[groups]
+
+# Epic example: an estate standards rollout touching a few repos.
+# standards-130 = ["echo-types", "affinescript", "standards"]
+
+# Convenience buckets (optional; the runner also accepts --kind julia etc.
+# resolved straight from repos.toml, so you only list a group here when the
+# membership is NOT derivable from `kind`).
+# proof-estate = ["echo-types", "affinescript", "my-lang", "gossamer"]