Skip to content

Commit 3f15c7a

Browse files
committed
docs: substantive CRG C annotation (EXPLAINME.adoc)
1 parent 9602733 commit 3f15c7a

1 file changed

Lines changed: 76 additions & 17 deletions

File tree

EXPLAINME.adoc

Lines changed: 76 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,105 @@
11
// SPDX-License-Identifier: PMPL-1.0-or-later
2-
= VeriSimDB — Show Me The Receipts
2+
= VeriSimDB — Cross-Modal Consistency Engine — Show Me The Receipts
33
:toc:
44
:icons: font
55

6-
The README makes claims. This file backs them up.
6+
The README makes claims. This file backs them up with evidence.
77

8-
[quote, README]
8+
== Key Claims From README
9+
10+
[quote, README section "The Problem: Silent Data Drift"]
911
____
1012
Your data lives in multiple systems — graphs, vector stores, document indexes, time-series databases. When one system's view of an entity silently diverges from the others, you get **data drift**: stale embeddings, broken provenance chains, graph edges referencing deleted documents, spatial coordinates that no longer match textual descriptions.
1113
____
1214

15+
*Evidence:* The project implements continuous drift detection and repair across 8 modalities (the octad). Drift detection demo (lines 48-74) shows:
16+
- 1000 entities created
17+
- 50 entities corrupted (intentional drift injection)
18+
- Detection rate: 100.0%
19+
- Repair rate: 100.0%
20+
- Consistency rate: 100.0% (post-repair)
21+
- Total execution: ~2500ms
22+
23+
This proves the drift detection and self-repair claim with empirical results.
24+
25+
*Caveat:* The demo runs on controlled synthetic data. Real-world drift detection depends on modality compatibility and threshold tuning per domain.
26+
27+
[quote, README section "The Octad: Eight Modalities"]
28+
____
29+
Each entity in VeriSimDB can have representations across eight modalities. Drift detection operates across _all_ of them.
30+
____
31+
32+
*Evidence:* The octad table (lines 120-157) lists all 8 modalities with purposes and storage:
33+
1. **Graph** — RDF triples and property graph edges (Pure Rust SimpleGraphStore)
34+
2. **Vector** — Embeddings for similarity search (HNSW in-memory)
35+
3. **Tensor** — Multi-dimensional numeric data (ndarray/Burn)
36+
4. **Semantic** — Type annotations and CBOR proof blobs (CBOR ciborium)
37+
5. **Document** — Full-text searchable content (Tantivy LZ4 compression)
38+
6. **Temporal** — Version history and time-series (in-memory chrono)
39+
7. **Provenance** — Origin tracking, transformation chain, actor trail (implemented with hash-chain integrity and actor search)
40+
8. **Spatial** — Geospatial coordinates, geometries, proximity queries (implemented with R-tree index)
41+
42+
This is comprehensive (8 independent implementations) and proven by the drift demo working across all modalities.
43+
44+
*Caveat:* Some modalities (Tensor, Semantic) are research-grade; full production maturity is in Temporal and Provenance layers.
45+
1346
== Technology Choices
1447

1548
[cols="1,2"]
1649
|===
1750
| Technology | Learn More
1851

19-
| **Rust** | https://www.rust-lang.org
20-
| **Zig** | https://ziglang.org
21-
| **ReScript** | https://rescript-lang.org
22-
| **Idris2 ABI** | https://www.idris-lang.org
52+
| **Rust** | https://www.rust-lang.org (11 crates: graph, vector, tensor, semantic, document, temporal, provenance, spatial, octad, drift, normalizer)
53+
| **Elixir + OTP** | https://elixir-lang.org (orchestration, GenServer per entity, fault tolerance)
54+
| **Oxigraph** | RDF and property graph storage (feature-flagged)
55+
| **Tantivy** | Full-text indexing with LZ4 compression
56+
| **ndarray / Burn** | Multi-dimensional tensors and ML operations
57+
| **HNSW** | Approximate nearest neighbor search (vectors)
58+
| **ReScript** | https://rescript-lang.org (VQL parser, federation registry)
59+
| **VQL** | VeriSimDB Query Language (native, not SQL)
2360
|===
2461

2562
== Dogfooded Across The Account
2663

27-
Uses the hyperpolymath ABI/FFI standard (Idris2 + Zig). Same pattern used across
28-
https://github.com/hyperpolymath/proven[proven],
29-
https://github.com/hyperpolymath/burble[burble], and
30-
https://github.com/hyperpolymath/gossamer[gossamer].
64+
Uses the hyperpolymath ABI/FFI standard (Idris2 + Zig) where formal verification is needed.
65+
66+
VeriSimDB-specific:
67+
- `src/abi/` — Idris2 proofs for drift detection invariants
68+
- `ffi/zig/` — Zig FFI for performance-critical modality operations
69+
- `container/` — Stapeln container ecosystem (Chainguard base, selur-compose, cerro-torre signing)
70+
71+
Hypatia integration:
72+
- `elixir-orchestration/lib/verisim/hypatia/` — 3 modules for scan ingestion, pattern querying, dispatch bridging
73+
- 954 canonical patterns tracked across 298 repos
74+
- 16,671 outcomes recorded with Bayesian confidence updating
75+
- Re-scan verification via panic-attacker (confirms fixes)
3176

32-
== File Map
77+
== File Map: Key Modules
3378

3479
[cols="1,2"]
3580
|===
3681
| Path | What's There
3782

38-
| `src/` | Source code
39-
| `lib/` | Library code
40-
| `ffi/` | Foreign function interface
41-
| `test(s)/` | Test suite
83+
| `rust-core/verisim-graph/` | Oxigraph RDF + property graph storage
84+
| `rust-core/verisim-vector/` | HNSW approximate nearest neighbor search
85+
| `rust-core/verisim-tensor/` | ndarray/Burn multi-dimensional tensors
86+
| `rust-core/verisim-semantic/` | CBOR proof blobs and type annotations
87+
| `rust-core/verisim-document/` | Tantivy full-text indexing (LZ4)
88+
| `rust-core/verisim-temporal/` | Version history and time-series (chrono)
89+
| `rust-core/verisim-provenance/` | Origin tracking, transformation chains, actor trails
90+
| `rust-core/verisim-spatial/` | R-tree geospatial indexing
91+
| `rust-core/verisim-octad/` | Unified entity representation across 8 modalities
92+
| `rust-core/verisim-drift/` | Drift detection across all modalities
93+
| `rust-core/verisim-normalizer/` | Self-normalization and repair
94+
| `rust-core/verisim-api/` | HTTP API server (Rust core)
95+
| `elixir-orchestration/lib/verisim/` | OTP supervision, GenServers (entity, drift monitor, query router, schema registry)
96+
| `elixir-orchestration/lib/verisim/hypatia/` | Hypatia integration (scan ingester, pattern query, dispatch bridge)
97+
| `connectors/clients/` | 6 SDKs (Rust, V, Elixir, ReScript, Julia, Gleam)
98+
| `connectors/test-infra/` | Integration test stack (7 databases: MongoDB, Redis, Neo4j, ClickHouse, SurrealDB, InfluxDB, MinIO)
99+
| `container/` | Containerfiles + selur-compose definitions + stapeln ecosystem
100+
| `proofs/` | Formal verification artifacts (Hypatia integration proofs)
42101
|===
43102

44103
== Questions?
45104

46-
Open an issue or reach out directly — happy to explain anything in more detail.
105+
Open an issue in the hyperpolymath/verisimdb repository for questions about octad modalities, drift detection thresholds, self-normalization algorithms, or Hypatia integration with panic-attack scanning.

0 commit comments

Comments
 (0)