|
Warning
|
SUPERSEDED (2026-07-07) — historical planning document. The described architecture (Week 1–16 plan, "10% completion", six modality stores) no longer reflects the repo. See |
Bringing VeriSimDB from 10% to 70% completion
- Current Status
- Goal
- Milestone V1: Rust Modality Stores (Weeks 1-6)
- Milestone V2: Octad Entity Layer (Weeks 7-8)
- Milestone V3: VCL Execution Engine (Weeks 9-11)
- Milestone V4: Testing & Stability (Weeks 12-14)
- Milestone V5: Performance & Documentation (Weeks 15-16)
- Weekly Progress Tracking
- Testing Strategy
- Checkpoints
- Next Steps
- No Integration with FormBD
-
Completion: 10%
-
Phase: Implementation ramp-up
-
What exists: Architecture docs, VCL grammar, Rust crate scaffolds, Elixir stubs
-
What’s missing: Actual modality store implementations, VCL execution engine
Week 16 target: 70% completion - All 6 modality stores operational - VCL execution engine working - Testing framework complete - Basic federation capability
Implement in this sequence:
-
verisim-document (Week 1-2) - Foundation, most familiar
-
verisim-temporal (Week 2-3) - Version history, time-travel
-
verisim-graph (Week 3-4) - RDF + property graph
-
verisim-vector (Week 4-5) - HNSW embeddings
-
verisim-semantic (Week 5-6) - Type annotations, CBOR
-
verisim-tensor (Week 5-6) - ndarray storage
Goal: Full-text search with Tantivy
// verisim-document/src/lib.rs
use tantivy::schema::*;
use tantivy::{Index, IndexWriter};
use uuid::Uuid;
pub struct DocumentStore {
index: Index,
schema: Schema,
}
impl DocumentStore {
pub fn create_document(&mut self, octad_id: Uuid, title: &str, body: &str) -> Result<()>;
pub fn search(&self, query: &str, limit: usize) -> Result<Vec<SearchResult>>;
pub fn get(&self, octad_id: Uuid) -> Result<Option<Document>>;
pub fn update(&mut self, octad_id: Uuid, title: &str, body: &str) -> Result<()>;
pub fn delete(&mut self, octad_id: Uuid) -> Result<()>;
}Tests: - Unit tests for CRUD operations - Property-based tests (proptest) - Search relevance tests
Deliverable: cargo test passes, search works
Goal: Version history and time-travel queries
// verisim-temporal/src/lib.rs
use chrono::{DateTime, Utc};
use uuid::Uuid;
pub struct TemporalStore {
// Version history storage
}
impl TemporalStore {
pub fn record_version(&mut self, octad_id: Uuid, timestamp: DateTime<Utc>, data: &[u8]) -> Result<()>;
pub fn get_at_time(&self, octad_id: Uuid, timestamp: DateTime<Utc>) -> Result<Option<Vec<u8>>>;
pub fn get_history(&self, octad_id: Uuid, start: DateTime<Utc>, end: DateTime<Utc>) -> Result<Vec<Version>>;
pub fn diff(&self, octad_id: Uuid, t1: DateTime<Utc>, t2: DateTime<Utc>) -> Result<Diff>;
}Tests: - Version recording and retrieval - Time-travel accuracy - Diff correctness
Deliverable: Time-travel queries work
Goal: RDF + property graph with Oxigraph
// verisim-graph/src/lib.rs
use oxigraph::store::Store;
use uuid::Uuid;
pub struct GraphStore {
store: Store,
}
impl GraphStore {
pub fn add_triple(&mut self, subject: &str, predicate: &str, object: &str) -> Result<()>;
pub fn add_edge(&mut self, from: Uuid, to: Uuid, edge_type: &str, props: &HashMap<String, String>) -> Result<()>;
pub fn traverse(&self, from: Uuid, edge_type: &str, depth: usize) -> Result<Vec<Uuid>>;
pub fn sparql_query(&self, query: &str) -> Result<QueryResults>;
}Tests: - Triple insertion and retrieval - Graph traversal - SPARQL subset queries
Deliverable: Graph queries work
Goal: HNSW similarity search
// verisim-vector/src/lib.rs
use uuid::Uuid;
pub struct VectorStore {
// HNSW index
}
impl VectorStore {
pub fn insert(&mut self, octad_id: Uuid, embedding: &[f32]) -> Result<()>;
pub fn search_similar(&self, query: &[f32], k: usize, threshold: f32) -> Result<Vec<(Uuid, f32)>>;
pub fn cosine_similarity(&self, v1: &[f32], v2: &[f32]) -> f32;
pub fn euclidean_distance(&self, v1: &[f32], v2: &[f32]) -> f32;
}Tests: - Embedding insertion - Similarity search correctness - Distance metric validation
Deliverable: Vector similarity search works
verisim-semantic: Type annotations + CBOR proof blobs
// verisim-semantic/src/lib.rs
pub struct SemanticStore {
// Type annotations
}
impl SemanticStore {
pub fn add_type(&mut self, octad_id: Uuid, type_uri: &str) -> Result<()>;
pub fn get_types(&self, octad_id: Uuid) -> Result<Vec<String>>;
pub fn store_proof_blob(&mut self, octad_id: Uuid, contract: &str, blob: &[u8]) -> Result<()>;
pub fn verify_proof(&self, octad_id: Uuid, contract: &str) -> Result<bool>;
}verisim-tensor: Multi-dimensional arrays
// verisim-tensor/src/lib.rs
use ndarray::ArrayD;
pub struct TensorStore {
// Tensor storage
}
impl TensorStore {
pub fn store(&mut self, octad_id: Uuid, tensor: ArrayD<f64>) -> Result<()>;
pub fn get(&self, octad_id: Uuid) -> Result<Option<ArrayD<f64>>>;
pub fn slice(&self, octad_id: Uuid, indices: &[usize]) -> Result<ArrayD<f64>>;
}Deliverable: Both stores operational
Goal: Unified entity abstraction across modalities
// verisim-octad/src/lib.rs
use uuid::Uuid;
pub struct Octad {
pub id: Uuid,
pub document: Option<DocumentData>,
pub graph: Option<GraphData>,
pub vector: Option<VectorData>,
pub temporal: Option<TemporalData>,
pub semantic: Option<SemanticData>,
pub tensor: Option<TensorData>,
}
pub struct OctadStore {
document_store: DocumentStore,
graph_store: GraphStore,
vector_store: VectorStore,
temporal_store: TemporalStore,
semantic_store: SemanticStore,
tensor_store: TensorStore,
}
impl OctadStore {
pub fn create(&mut self, octad: Octad) -> Result<Uuid>;
pub fn get(&self, id: Uuid) -> Result<Option<Octad>>;
pub fn update(&mut self, id: Uuid, octad: Octad) -> Result<()>;
pub fn query_cross_modal(&self, predicate: CrossModalPredicate) -> Result<Vec<Uuid>>;
}Drift Detection:
// verisim-drift/src/lib.rs
pub struct DriftDetector {
// Drift detection logic
}
impl DriftDetector {
pub fn detect(&self, octad_id: Uuid, modality1: Modality, modality2: Modality) -> Result<f32>;
pub fn repair(&mut self, octad_id: Uuid, strategy: RepairStrategy) -> Result<()>;
}Deliverable: Cross-modal queries work, drift detection operational
Goal: Execute VCL queries across all modalities
// verisim-api/src/vql_bridge.rs
use rescript_parser::VCLParser; // ReScript → Rust FFI
pub fn parse_vcl(query: &str) -> Result<VCLAst>;
pub fn execute_vcl(ast: VCLAst, octad_store: &OctadStore) -> Result<QueryResult>;Integration: - ReScript VCL parser → Rust execution engine - AST serialization (CBOR or JSON)
// verisim-api/src/query_planner.rs
pub struct QueryPlan {
pub steps: Vec<PlanStep>,
pub estimated_cost: f64,
}
pub fn plan_query(ast: VCLAst) -> Result<QueryPlan>;Optimization: - Push predicates to modality stores - Minimize cross-modal joins - Cache query plans
// verisim-api/src/explain.rs
pub struct ExplainOutput {
pub plan: QueryPlan,
pub modalities_queried: Vec<Modality>,
pub estimated_time: f64,
pub hints: Vec<String>,
}
pub fn explain(query: &str) -> Result<ExplainOutput>;Deliverable: VCL queries execute, EXPLAIN works
# Cargo.toml
[dev-dependencies]
proptest = "1.4"// tests/property_tests.rs
use proptest::prelude::*;
proptest! {
#[test]
fn insert_then_retrieve(title in "\\w{1,50}", body in "\\w{10,200}") {
let mut store = OctadStore::new()?;
let id = store.create(octad_with_document(title, body))?;
let retrieved = store.get(id)?;
assert!(retrieved.is_some());
}
}# .clusterfuzzlite/project.yaml
language: rust// fuzz/fuzz_targets/fuzz_vql.rs
#![no_main]
use libfuzzer_sys::fuzz_target;
use verisim_api::parse_vcl;
fuzz_target!(|data: &[u8]| {
if let Ok(s) = std::str::from_utf8(data) {
let _ = parse_vcl(s);
}
});// tests/integration_test.rs
#[test]
fn test_cross_modal_query() {
let store = setup_test_store();
// Insert document
let id = store.create_document("Test", "Body")?;
// Add embedding
store.add_vector(id, &[0.1, 0.2, 0.3])?;
// Cross-modal query
let query = "SELECT * FROM octads WHERE DOCUMENT MATCHES 'Test' AND VECTOR SIMILAR TO [0.1, 0.2, 0.3]";
let results = store.query(query)?;
assert_eq!(results.len(), 1);
}Deliverable: Test coverage >80%
// verisim-api/src/cache.rs
use lru::LruCache;
pub struct QueryCache {
plan_cache: LruCache<String, QueryPlan>,
result_cache: LruCache<String, QueryResult>,
}Optimizations: - Query plan caching - Connection pooling - Batch operations
Every Friday:
;; Update STATE.scm
(snapshot (date "2026-02-XX") (session "week-N")
(accomplishments
"Completed verisim-document CRUD"
"Property-based tests passing")
(blockers
"HNSW performance needs optimization")
(next-week
"Begin verisim-temporal implementation"))-
Today: Review this roadmap
-
This week: Begin verisim-document implementation
-
Friday: Update STATE.scm with Week 1 progress
Important: This roadmap is VeriSimDB-only. FormBD is a separate project.
The only connection: VeriSimDB could eventually federate to FormBD as an external data source:
-- VCL federating to FormBD (future feature)
SELECT * FROM octads@formbd_instance WHERE modality = 'document';But that’s just treating FormBD like any other database endpoint (PostgreSQL, MongoDB, etc.).