Step 0+1 ground-clearing: ADRs, deletions, test fix, lint cleanup, README/ROADMAP, CI concurrency (#24)

* docs(adr): record octad/verification/justfile decisions, strip empty trees

Adds three ADRs for verisimiser and one for verisimdb-data, plus the
mechanical file deletions they authorise.

- ADR-0001 (octad-ontology): concerns octad is canonical; modalities
  become Tier 2 overlays. Closes #19; sets up #20; closes #21 wontfix.
- ADR-0002 (verification-tree): strip the empty 8-subdirectory tree;
  Idris2 stubs in src/interface/abi/ are unaffected. Closes #15.
- ADR-0003 (justfile-aspirational-recipes): delete recipes that name
  non-existent clap subcommands. Closes #11 (#10 is the mechanical
  follow-up).
- ADR-0001 (verisimdb-data, repo-purpose): repo carries two explicit
  purposes (scan store + ABI dogfood). Lands in the data repo commit.

Deletes:
- examples/SafeDOMExample.res, examples/web-project-deno.json
  (unrelated template flotsam — closes #12)
- root SECURITY.md, root CODE_OF_CONDUCT.md (duplicate; .github/
  versions are canonical — closes #13, #14)
- verification/ subtree (closes #15 via ADR-0002)

Closes #11, #12, #13, #14, #15, #19, #21

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(integration): fix verisim_ vs verisimdb_ prefix + Windows path escaping

Two bugs in tests/integration_test.rs caused 2 of 9 integration tests to
fail (the unit tests were unaffected).

1. Prefix mismatch — codegen emits identifiers prefixed `verisimdb_`
   (see src/codegen/overlay.rs). The integration tests asserted
   substring presence of `verisim_…` which is not a substring of
   `verisimdb_…`. Replaced 11 occurrences in tests/integration_test.rs.

2. Windows path escaping — test_end_to_end_file_workflow interpolates
   `schema_path.display()` into a TOML basic string with `"…"`. On
   Windows the path contains backslashes which TOML treats as escapes,
   producing a malformed manifest and an unwrap-on-Err. Switched the
   embedded path to a TOML literal string (single quotes) which
   suppresses escape interpretation.

Verified: cargo test now reports 26 + 26 + 9 = 61 tests, 0 failed.

Closes #8

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(readme,roadmap): align around concerns octad (ADR-0001)

Per ADR-0001 the canonical octad is concerns
(Data/Metadata/Provenance/Lineage/Constraints/AccessControl/Temporal/
Simulation), not modalities. The previous README led with a modalities
table the codebase no longer supported.

README.adoc rewrites:

- Replace the "Eight Modalities" table with an "Eight Concerns" table
  whose rows match `OctadDimension` enum, OctadConfig fields, and the
  emitted sidecar tables.
- Reframe the eight cross-modal drift categories under Constraints
  (they are symptoms observed by Constraints when Data, Metadata, and
  active Tier 2 overlays disagree). Note explicitly that each category
  still needs a computable definition.
- Tier 1 narrative reorganised around the five Tier 1 concerns
  (Provenance, Temporal, Constraints, Lineage, AccessControl).
- Tier 2 retains modalities but as overlay representations, not as
  "the octad".
- Add a "Related repos" section linking verisimdb-data.
- Add an "ABI" section pointing at src/interface/abi/ and
  src/interface/ffi/ where the Idris2 and Zig stubs actually live.
- Cite ADR-0001 and ADR-0002 inline.

ROADMAP.adoc rewrites:

- Phase 0 marked complete with accurate evidence (ABI types exist in
  three languages; codegen scaffolding ships).
- Phases reordered to match the bottom-up plan: SQLite Tier 1 MVP
  first (cheapest end-to-end), then PostgreSQL, then multi-backend,
  then Constraints/Drift, then AccessControl/Lineage, then Tier 2
  modality overlays, then Simulation, then VCL-total integration,
  then production hardening, then ecosystem.
- Each phase phrased in concerns/modality terms consistent with
  ADR-0001.

Closes #20

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ci(rust-ci): add concurrency cancellation; honour fmt fixes from prior commit

The `rust-ci.yml` lane already runs `cargo check + clippy + fmt` (job
`check`) and `cargo test --all-targets` (job `test`, depends on `check`).
That is exactly what V-L3-A2 asked for — the previous mega-list
incorrectly claimed no cargo-test gate existed.

Real improvement this commit lands:

- Add a `concurrency` block at workflow level so push storms cancel
  older queued runs instead of queueing for minutes. Group keyed on
  `github.ref` so concurrent PR pushes serialize per-PR but parallel
  to main.

Verified locally:
- `cargo fmt --all -- --check` clean
- `cargo clippy --all-targets -- -D warnings` clean
- `cargo test` reports 35 tests (26 lib + 9 integration), 0 failed

The cargo-test gate becomes a required check when branch protection
is configured to require Rust CI / Cargo test — that is a repo
settings change, not a workflow change.

Closes #9

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

Author: hyperpolymath

.github/workflows/rust-ci.yml

@@ -11,6 +11,10 @@ on:
   push:
     branches: [main, master]
 
+concurrency:
+  group: rust-ci-${{ github.ref }}
+  cancel-in-progress: true
+
 permissions:
   contents: read
 

CODE_OF_CONDUCT.md

@@ -7,24 +7,33 @@ Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk>
 :icons: font
 :source-highlighter: rouge
 
+== Related repos
+
+* https://github.com/hyperpolymath/verisimdb-data[`verisimdb-data`] —
+  flat-file store for scan results and drift snapshots, and the dogfood
+  site for the hyperpolymath Idris2 + Zig ABI shared with `proven`,
+  `burble`, `gossamer`. Verisimiser does not require it.
+
 == What Is This?
 
 VeriSimiser augments existing databases with capabilities from
-https://github.com/hyperpolymath/nextgen-databases[VeriSimDB]'s octad model —
-specifically the capabilities that work as genuine piggybacks without requiring
-you to replace your database.
+https://github.com/hyperpolymath/nextgen-databases[VeriSimDB]'s **octad
+model** — eight concerns layered over any backend without forcing a
+schema migration.
 
 **Honest framing**: this is not a pure bolt-on like the language -isers.
 Language -isers generate a separate wrapper alongside your code — one-way
 dependency, your code untouched. Database augmentation is fundamentally
 different because it interacts with shared mutable state. VeriSimiser is
 therefore split into two tiers:
 
-* **Tier 1 (true piggyback)** — capabilities that sit alongside or in front
-  of your database, never touching its storage engine. These are safe bolt-ons.
-* **Tier 2 (augmentation layer)** — capabilities that require additional
-  storage alongside your database. These are honest about being "VeriSimDB
-  with your database as one backend" rather than pretending to be invisible.
+* **Tier 1 (true piggyback)** — concerns that sit alongside or in front
+  of your database, writing only to a sidecar and never touching your
+  storage engine. Safe bolt-ons.
+* **Tier 2 (augmentation layer)** — modality overlays (graph, vector,
+  tensor, semantic, document, spatial) that require additional storage
+  alongside your database. Honest about being "VeriSimDB with your
+  database as one backend" rather than pretending to be invisible.
 
 == VeriSimDB's Octad: Eight Concerns
 
@@ -49,40 +58,19 @@ identity; see "Tier 2" below.)
 | **Simulation**    | "What if we changed this?" — sandbox branches that do not touch the main data. | Tier 2
 |===
 
+The codebase commits to this ontology in
+`src/abi/mod.rs::OctadDimension`, `src/manifest/mod.rs::OctadConfig`,
+and the codegen tables. See `docs/decisions/ADR-0001-octad-ontology.adoc`
+for why "concerns" was chosen over the alternative "modalities" framing.
+
 == Tier 1: True Piggybacks
 
 These work like PostGIS — they add capability without replacing anything.
 
-=== Cross-Modal Drift Detection
-
-The jewel of VeriSimDB. Drift detection monitors whether representations
-of the same entity stay consistent across modalities. VeriSimiser can
-overlay drift detection onto any database:
-
-* Your database stores the primary data (unchanged)
-* VeriSimiser maintains a lightweight **drift index** alongside it
-* Queries pass through to your database normally
-* VeriSimiser intercepts results and checks for cross-modal inconsistencies
-* Alerts when representations drift apart
-
-This is a **read-path augmentation** — it observes query results, it doesn't
-modify them. Safe to add, safe to remove, no data dependency.
-
-VeriSimDB detects eight categories of cross-modal drift:
-
-1. Structural drift — schema changes not reflected across modalities
-2. Semantic drift — meaning divergence between representations
-3. Temporal drift — version skew between modalities
-4. Statistical drift — distribution shift in vector/tensor spaces
-5. Referential drift — broken links between graph and document modalities
-6. Provenance drift — transformation chain inconsistencies
-7. Spatial drift — coordinates inconsistent with other modalities
-8. Embedding drift — vector embeddings stale relative to source documents
-
 === Provenance Tracking
 
 Hash-chain verified origin tracking for every piece of data. VeriSimiser
-can add provenance to any database as a sidecar:
+adds provenance to any database as a sidecar:
 
 * Every write is intercepted and a provenance record created
 * SHA-256 hash chain links provenance records in order
@@ -95,75 +83,103 @@ change what happened. The provenance chain is stored in a separate sidecar
 
 === Temporal Versioning
 
-Automatic version history for entities. VeriSimiser can maintain a temporal
+Automatic version history for entities. VeriSimiser maintains a temporal
 sidecar that records every state change:
 
 * Point-in-time queries: "what did this entity look like at time T?"
 * Diff queries: "what changed between T1 and T2?"
 * Rollback capability: restore any entity to a previous state
 * Retention policies: auto-prune history older than N days
 
-This piggybacks onto write events (triggers, CDC, or application-level hooks)
-and stores version history in a separate sidecar.
+This piggybacks onto write events (triggers, CDC, or application-level
+hooks) and stores version history in a separate sidecar.
+
+=== Constraints & Drift Detection
+
+The Constraints concern enforces cross-dimensional invariants and
+observes *drift* — places where Data, Metadata, and any active Tier 2
+overlays disagree. Drift detection is a **read-path observer**: it
+inspects query results, never modifies them.
+
+VeriSimiser tracks eight symptomatic categories (mapped onto the
+Constraints concern):
+
+. **Structural drift** — schema changes not reflected across active overlays
+. **Semantic drift** — meaning divergence between overlays
+. **Temporal drift** — version skew between overlays
+. **Statistical drift** — distribution shift in vector/tensor overlays
+. **Referential drift** — broken links between the graph overlay and Data
+. **Provenance drift** — transformation-chain inconsistencies
+. **Spatial drift** — coordinates inconsistent with other overlays
+. **Embedding drift** — vector embeddings stale relative to source documents
+
+(Each category needs a computable definition before it can be detected;
+this is on the roadmap, not implemented yet.)
+
+=== Lineage
+
+Directed-edge graph of data derivation: which entity was derived from
+which other entity, and by what transformation. Intended as a DAG (cycle
+prevention enforcement is a future ADR).
+
+=== Access Control
+
+Row-level and column-level policies evaluated at query time. Independent
+of any backend-native role system; verisimiser interprets policies and
+filters/redacts results.
 
 == Tier 2: Augmentation Layer
 
 These capabilities require additional storage alongside your database.
-They're honest about being "VeriSimDB modalities with your database as the
-document/primary store." This is still valuable — it's how you get octad
-capabilities incrementally — but it's not a bolt-on.
+They are honest about being "VeriSimDB modalities with your database as
+the document/primary store." This is still valuable — it's how you get
+extra modality projections incrementally — but it's not invisible.
 
-* **Graph overlay**: Add RDF triples and property graph edges to entities
+* **Graph overlay**: Add RDF triples and property-graph edges to entities
   in your relational database. Stored in a separate graph index.
-* **Vector overlay**: Add embeddings for similarity search. Stored in HNSW
-  index alongside your database.
+* **Vector overlay**: Add embeddings for similarity search. Stored in
+  HNSW index alongside your database.
 * **Tensor overlay**: Add multi-dimensional numeric data. Stored in
   ndarray-backed sidecar.
 * **Semantic overlay**: Add type annotations and proof blobs. Stored in
   CBOR sidecar.
+* **Document overlay**: Add full-text search over text columns.
+  Stored in a Tantivy index.
 * **Spatial overlay**: Add geospatial coordinates. Stored in R-tree sidecar.
+* **Simulation**: Branched copies of data for what-if analysis. Stored
+  in `verisimdb_simulation_branches` + `_deltas`.
 
-Each Tier 2 modality has its own storage and can be enabled independently.
-Your primary database remains the source of truth for its native data.
+Each Tier 2 overlay has its own storage and can be enabled
+independently. Your primary database remains the source of truth for
+its native data.
 
 == The Manifest
 
 [source,toml]
 ----
-[verisimiser]
+[project]
 name = "my-augmented-db"
+version = "0.1.0"
 
 [database]
-target-db = "postgresql"
-connection-string = "postgres://localhost/mydb"
-
-# Tier 1: true piggybacks (no additional storage in your database)
-[tier1]
-drift-detection = true       # cross-modal drift monitoring
-provenance = true             # SHA-256 hash-chain audit trail
-temporal-versioning = true    # automatic version history
-
-[tier1.provenance]
-sidecar = "sqlite"            # sqlite | file | verisim
-sidecar-path = ".verisimiser/provenance.db"
-
-[tier1.temporal]
-sidecar = "sqlite"
-retention-days = 90
-
-# Tier 2: augmentation layer (additional storage alongside your database)
-[tier2]
-graph = false                 # RDF/property graph overlay
-vector = false                # embedding similarity search
-tensor = false                # multi-dimensional numeric
-semantic = false              # type annotations + proof blobs
-spatial = false               # geospatial coordinates
-
-[tier2.vector]
-# model = "sentence-transformers/all-MiniLM-L6-v2"
-# dimensions = 384
+backend = "sqlite"                       # sqlite | postgresql | mongodb
+connection-string-env = "DATABASE_URL"   # env var; never the literal secret
+schema-source = "schema.sql"             # optional; SQL DDL describing target schema
+
+[octad]
+enable-provenance     = true
+enable-lineage        = true
+enable-temporal       = true
+enable-access-control = true
+enable-simulation     = false            # Tier 2: branching/sandboxing
+
+[sidecar]
+storage = "sqlite"                       # sqlite (default) | json
+path    = ".verisim/sidecar.db"
 ----
 
+Run `verisimiser init --database sqlite` to generate a starter manifest.
+
 == Architecture
 
 [source]
@@ -172,52 +188,73 @@ Your Application
       │
       ├──── writes ────► Your Database (unchanged)
       │                       │
-      │                  VeriSimiser intercepts
+      │                  VeriSimiser intercepts (write-path observer)
       │                       │
       │         ┌─────────────┼──────────────┐
       │         │             │              │
-      │    Drift Index   Provenance     Temporal
-      │    (Tier 1)      Sidecar        Sidecar
-      │                  (Tier 1)       (Tier 1)
+      │   Provenance     Temporal       Lineage / AccessControl /
+      │   sidecar        sidecar        Constraints sidecars (Tier 1)
       │
-      └──── optional ──► Tier 2 Sidecars
-                          (Graph, Vector, Tensor,
-                           Semantic, Spatial)
+      └──── optional ──► Tier 2 modality overlays
+                          (Graph, Vector, Tensor, Semantic,
+                           Document, Spatial, Simulation)
 ----
 
-**Interception methods** (configurable per database):
+**Interception methods** (configurable per database — most are still
+roadmap items, see `ROADMAP.adoc`):
 
 * **PostgreSQL**: logical replication / `pg_notify` / triggers
 * **SQLite**: `sqlite3_update_hook` / WAL monitoring
 * **MongoDB**: change streams
 * **Application-level**: middleware / ORM hooks
 
+== ABI
+
+The Application Binary Interface for the augmentation layer is declared
+in two languages:
+
+* `src/interface/abi/` — Idris2 type definitions
+  (`Types.idr`, `Layout.idr`, `Foreign.idr`). These are the canonical
+  shapes; formal proofs of correctness against them are future work
+  (see `docs/decisions/ADR-0002-verification-tree.adoc`).
+* `src/interface/ffi/` — Zig C-compatible FFI implementing the Idris2
+  ABI for use from native code.
+
+The Rust `src/abi/mod.rs` mirrors the Idris2 types as plain structs
+used directly by the CLI and codegen.
+
 == Relationship to VeriSimDB
 
 VeriSimiser is NOT a replacement for VeriSimDB. It's a gateway drug.
 
-* **VeriSimiser Tier 1** gives you drift detection, provenance, and temporal
-  versioning on your existing database. Zero commitment.
-* **VeriSimiser Tier 2** gives you individual octad modalities as sidecars.
-  Incremental adoption.
-* **Full VeriSimDB** gives you the complete octad with native cross-modal
-  querying, VCL, and built-in drift normalisation. Full commitment.
+* **VeriSimiser Tier 1** gives you provenance, temporal, lineage,
+  access control, and constraints/drift detection on your existing
+  database. Zero commitment.
+* **VeriSimiser Tier 2** gives you individual modality overlays
+  (graph/vector/tensor/semantic/document/spatial) and simulation as
+  sidecars. Incremental adoption.
+* **Full VeriSimDB** gives you the complete model with native
+  cross-modal querying, VCL, and built-in drift normalisation. Full
+  commitment.
 
-The migration path is: Tier 1 → Tier 2 → full VeriSimDB (if you want it).
-Most users will be happy at Tier 1 or Tier 2.
+The migration path is: Tier 1 → Tier 2 → full VeriSimDB (if you want
+it). Most users will be happy at Tier 1 or Tier 2.
 
 == Integration with TypedQLiser
 
-VeriSimiser works alongside https://github.com/hyperpolymath/typedqliser[TypedQLiser]:
+VeriSimiser works alongside
+https://github.com/hyperpolymath/typedqliser[TypedQLiser]:
 
 * TypedQLiser type-checks your queries (compile-time, no runtime cost)
-* VeriSimiser augments your database with octad capabilities (runtime)
+* VeriSimiser augments your database with octad concerns (runtime)
 * Together: formally verified queries against an augmented database
 
 == Status
 
-**Pre-alpha.** Architecture defined, tier system designed. Tier 1 (drift
-detection, provenance, temporal versioning) is the priority implementation.
+**Pre-alpha.** Architecture defined, octad ontology pinned (ADR-0001),
+codegen scaffolding in place. The next implementation milestone is V-L1-C1:
+end-to-end SQLite Tier 1 piggyback (`sqlite3_update_hook` →
+`verisimdb_provenance_log` sidecar). See `ROADMAP.adoc`.
 
 Part of the https://github.com/hyperpolymath/iseriser[-iser family].
 **#3 priority** (after TypedQLiser and Chapeliser).