diff --git a/.bumpversion.toml b/.bumpversion.toml index cb853b21..5ab22e7a 100644 --- a/.bumpversion.toml +++ b/.bumpversion.toml @@ -1,5 +1,5 @@ [tool.bumpversion] -current_version = "0.5.3" +current_version = "0.5.4" parse = "(?P\\d+)\\.(?P\\d+)\\.(?P\\d+)(-(?Palpha|beta|rc)\\.(?P\\d+))?" serialize = [ "{major}.{minor}.{patch}-{pre_label}.{pre_n}", diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index e8a8a441..11f0b898 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -65,6 +65,68 @@ stay as historical references. ## Entries (reverse chronological) +## 2026-04-22 — E-DEPLOY-1 — Supabase-shape thinking extension: trojan-horse A2A training surface over DN-addressed metadata bus, backed by lance-graph, BBB-preserved by blackboard mediation + +**Status:** FINDING (deployment doctrine — the nine-dimension shape that makes everything we've built earn its own product) + +**One line:** the callcenter crate is not a Supabase clone; it is a Supabase-dialect thinking extension that trains itself on A2A agent traffic while the BBB holds at the blackboard. + +**Nine compounding dimensions, one coherent deployment:** + +1. **A2A agents ARE the training surface.** crewai-rust / n8n-rs / openclaw / LangGraph / AutoGen all generate traffic autonomously. Every seed lands with an `ExternalRole` tag; every commit (F<0.2) or FailureTicket (F>0.8) is a labeled training example — no human-in-the-loop, no labeling budget, no cold-start data problem. Per-ecosystem specialization emerges because persona AriGraph subgraphs diverge per family (`CrewaiAgent`-flavored codec cards differ from `N8n`-flavored codec cards after enough traffic). + +2. **Supabase shape = adoption surface.** Any A2A consumer already knows PostgREST filter DSL, Realtime channels, JWT auth. They write a standard RAG integration. They never know there is a cognitive substrate behind it. + +3. **Metadata IS the address bus.** `cognitive_event` Arrow rows carry `(external_role, faculty_role, expert_id, dialect, scent)` as typed columns. Queries against this metadata ARE dispatch — there is no separate "router." `SELECT … WHERE external_role=3` returns rows whose identity tuple is the execution target. Five dialects view the same bus: SQL tabular / Cypher graph-path / GQL / NARS truth-filter / qualia fuzzy-family. + +4. **REST + DataFusion backs it** (ladybug-rs prior art). DataFusion 51 is the query engine for every dialect; Arrow 57 is the wire format; Lance 2 is the durable store. No PostgREST. No Postgres wire protocol in the hot path. + +5. **DN-addressed URL hierarchy** replaces Redis flat keys: `/tree/{ns}/heel/{h}/hip/{h}/branch/{b}/twig/{t}/leaf/{l}` parses deterministically into a metadata predicate. URL path = routing predicate; body content = seed. + +6. **Address = scent (1 byte via codec chain)**. Full path 16Kbit → ZeckBF17 48B → Base17 34B → CAM-PQ 6B → Scent 1B, ρ=0.937. One compressed object serves four uses: route / retrieve / similar / frame. Scent pulls context (AriGraph triples, episodic ±5..±500 window, persona trust, qualia signature) BEFORE the shader reads the body. The answer is grounded in cognitive-substrate state, not in lexical document vectors. + +7. **Body content = external seed** enters the blackboard as `BlackboardEntry { capability: ExternalSeed }`. Never touches BindSpace directly. The round boundary on the blackboard IS the anti-corruption boundary. + +8. **Polyglot front end, one IR, many tongues.** Cypher / GQL / Gremlin / SPARQL already shipped in lance-graph-planner. Adding NARS (native typed cognitive queries with f,c constraints), Redis (flat KV that auto-hydrates to DN), and Spark (bulk analytics + structured streaming) extends the existing `PolyglotDetect` pattern. Dialect-as-signal: the dialect itself is a feature on the seed's metadata row (tells the router which cognitive faculty the consumer is exercising). + +9. **Agent cards + faculties + external roles = one identity space.** `ExpertId = stable_hash_u16(card_yaml)` collapses internal A2A experts, external agents, YAML cards into one register. Faculties (`ReadingComprehension`, `Voice`, `Reasoning`, `Empathy`) carry asymmetric inbound/outbound `ThinkingStyle` and `ToolAbility` sets. Full three-coordinate provenance: `(ExternalRole family, FacultyRole function, ExpertId card)` on every metadata row. + +**BBB invariant — the iron rule that makes the whole thing safe:** + +Every dimension lives in BOTH representations: +- **External (metadata columns)** — Arrow scalars, safely cross the BBB, queryable by the five dialects, projected via `CollapseGate` on every commit. +- **Internal (VSA role-bindings)** — `RoleKey` slot addresses for Markov ±5 braiding, never cross the BBB, produced stack-side via deterministic metadata → slot mapping. + +Same identity, two faces, one direction of flow: metadata IN (translate via RoleKey at the stack side) → VSA braiding (the substrate reasons) → metadata OUT (project back via `CollapseGate`). Supabase refactor only ever sees Arrow columns; the blackboard only ever sees role-tagged entries. No path exists where an external payload touches `Vsa10k`, `RoleKey`, `SemiringChoice`, or `NarsTruth` as a type — the compiler rejects it (Arrow's type system enforces it at `RecordBatch` column level). + +**What the consumer experiences vs. what actually happens:** + +| Consumer sees | Substrate does | +|---|---| +| POST /tree/.../leaf/utterance with Cypher body | URL → DN → 1-byte scent → pulls AriGraph subgraphs + episodic ±5..±500 + persona trust | +| Response JSON with matched rows | Shader cycle ran: bind → braid with pulled context → unbind against AriGraph prior → F descent → Commit writes new SPO triple keyed on scent | +| "This RAG is weirdly good" | The next query at nearby scent pulls richer context because the last query trained this persona's subgraph | + +**Cross-refs:** +- `.claude/plans/callcenter-membrane-v1.md` §§ 10.1 – 10.13 (full architectural spec) +- `contract::external_membrane` — `ExternalRole`, `ExternalEventKind`, `CommitFilter`, `ExternalMembrane` trait +- `contract::a2a_blackboard` — `ExpertCapability::{ExternalSeed, ExternalContext}` (the inbound BB entry types) +- `contract::persona` — `PersonaCard`, `RoutingHint` (identity-as-metadata + four routing modes) +- `contract::faculty` — `FacultyRole`, `FacultyDescriptor`, `ToolAbility` (internal cognitive-function identity) +- `crates/lance-graph-planner/src/serve.rs` — Axum REST + OpenAI-compatible (extend here for DN + polyglot) +- `crates/lance-graph/src/graph/arigraph/` — persona memory home (consumer-side AriGraph subgraph integration) + +**Litmus test** (from plan § 10.9 iron rule, restated for this deployment): + +Before any PR touches the callcenter crate or the metadata bus, answer three questions: +1. Can I name the role, the place, and the translation for every byte crossing the gate? +2. Does the external surface only see Arrow-scalar columns (no Vsa10k/RoleKey/semiring)? +3. Does the internal substrate only see role-tagged blackboard entries (no raw external payload)? + +If any answer is no, the code is leaking external ontology inward (or internal ontology outward) — reject. + +--- + ## 2026-04-21 — AriGraph/episodic/SPO/CAM-PQ are thinking tissue, not storage — this is why it becomes AGI **Status:** FINDING (the final piece that closes the architecture) diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 22f2f8d0..b226b2bc 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -36,6 +36,34 @@ --- +## v1 — Unified Integration: PersonaHub × ONNX × Archetype × MM-CoT × RoleDB (authored 2026-04-23) + +**Author:** main-thread session 2026-04-23 +**Scope:** Integrate four upstream systems (PersonaHub compression, ONNX persona classifier @ L4/L5, Archetype ECS adapter, MM-CoT stage split) into the lance-graph cognitive substrate without adding new architectural layers — each maps onto existing contract types. +**Path:** `.claude/plans/unified-integration-v1.md` +**Deliverables:** DU-0 PersonaHub 56-bit compression, DU-1 ONNX persona classifier (replaces Chronos proposal), DU-2 Archetype ECS bridge crate, DU-3 RoleDB DataFusion VSA UDFs, DU-4 MM-CoT `rationale_phase: bool` in `CognitiveEventRow`, DU-5 board hygiene. + +**Status (2026-04-23):** Active. No deliverables shipped yet. Plan written and committed (commit `468357d`). Architectural ground truth in `callcenter-membrane-v1.md` §§ 15–17. + +**Confidence (2026-04-23):** CONJECTURE — all integration mappings grounded in repo evidence and upstream docs; no code shipped beyond plan. + +**Correction (2026-04-23):** Chronos (Amazon) proposal superseded by ONNX classifier for DU-1. Chronos predicts 1D style scalar; ONNX classifier predicts full 288-class `(ExternalRole × ThinkingStyle)` product. ONNX infra already justified by Jina v5 ONNX on disk. + +--- + +## v1 — Callcenter Membrane: Supabase-shape over Lance + DataFusion (authored 2026-04-22) + +**Author:** main-thread session 2026-04-22 +**Scope:** Assimilate the design and ergonomics of the Supabase callcenter surface into a new crate (`lance-graph-callcenter`) that sits entirely outside the canonical cognitive substrate, backed by Lance + DataFusion, enforcing the BBB (blood-brain barrier) at compile time via the Arrow type system — Phoenix channel realtime + PostgREST query surface without PostgreSQL. +**Path:** `.claude/plans/callcenter-membrane-v1.md` (254 lines) +**Deliverables:** DM-0 `ExternalMembrane` + `CommitFilter` in contract, DM-1 callcenter crate skeleton, DM-2 `LanceMembrane::project()` + compile-time leak test, DM-3 `CommitFilter → DataFusion Expr`, DM-4 `LanceVersionWatcher`, DM-5 `PhoenixServer`, DM-6 `DrainTask`, DM-7 `JwtMiddleware + RLS rewriter`, DM-8 `PostgRestHandler`, DM-9 end-to-end test. + +**Status (2026-04-22):** Active. DM-0 and DM-1 shipped in this session. DM-2 through DM-9 queued. + +**Confidence (2026-04-22):** CONJECTURE on the full architecture (grounded in Arrow BBB analysis + repo evidence; no DM-2+ implementation shipped). DM-0/DM-1 are working stubs; Arrow compile-time BBB enforcement verified structurally, awaiting DM-2 compile-time leak test. + +--- + ## v1 — Categorical-Algebraic Inference (authored 2026-04-21) **Author:** main-thread session 2026-04-21 diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index c1a96f76..356a316d 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -225,6 +225,50 @@ superseded files into an `archive/` subdirectory. Estimate ~200 LOC of meta work, ~2 hours of reading. **Not urgent**; useful before the next major planning session. +--- + +## callcenter-membrane-v1 — Supabase-shape over Lance + DataFusion + +External callcenter membrane crate. BBB enforced by Arrow type system at +compile time. Plan: `.claude/plans/callcenter-membrane-v1.md`. + +### DM-0 / DM-1 — Shipped in this session + +| D-id | Title | Status | PR / Evidence | +|---|---|---|---| +| DM-0 | `ExternalMembrane` trait + `CommitFilter` in `lance-graph-contract/src/external_membrane.rs` | **Shipped** | session 2026-04-22 — `pub mod external_membrane` added to contract lib.rs | +| DM-1 | `lance-graph-callcenter` crate skeleton: `Cargo.toml` (feature gates) + `src/lib.rs` (stub + UNKNOWN markers) | **Shipped** | session 2026-04-22 — added to workspace members | + +### DM-2 through DM-9 — Queued + +| D-id | Title | Status | PR / Evidence | +|---|---|---|---| +| DM-2 | `LanceMembrane: ExternalMembrane` impl with `project()` + compile-time BBB leak test | **Queued** | Resolve UNKNOWN-1 (ShaderSink overlap?) first | +| DM-3 | `CommitFilter` → DataFusion `Expr` translator (`[query]` feature) | **Queued** | — | +| DM-4 | `LanceVersionWatcher` — tails Lance version counter, emits Phoenix `postgres_changes` (`[realtime]`) | **Queued** | — | +| DM-5 | `PhoenixServer` — minimal WS server, Phoenix channel subset (`[realtime]`) | **Queued** | Resolve UNKNOWN-2 (which consumers need Phoenix wire?) first | +| DM-6 | `DrainTask` — `steering_intent` Lance read → `UnifiedStep` → `OrchestrationBridge::route()` | **Queued** | — | +| DM-7 | `JwtMiddleware` + `ActorContext` → `LogicalPlan` RLS rewriter (`[auth]`) | **Queued** | Resolve UNKNOWN-3 (pgwire?) + UNKNOWN-4 (actor_id type) first | +| DM-8 | `PostgRestHandler` — query-string → DataFusion SQL → Lance scan → Arrow response (`[serve]`) | **Queued** | Confirm PostgREST compat needed (§ 8 stop point 4) before building | +| DM-9 | End-to-end test: shader fires → `LanceMembrane::project()` → Lance append → Phoenix subscriber receives event | **Queued** | Depends on DM-2 through DM-6 | + +--- + +## unified-integration-v1 — PersonaHub × ONNX × Archetype × MM-CoT × RoleDB + +Plan: `.claude/plans/unified-integration-v1.md`. Session 2026-04-23. + +| D-id | Title | Status | Notes | +|---|---|---|---| +| DU-0 | PersonaHub 56-bit compression: `(atom_bitset: u32, palette_weight: u8, template_id: u16)` offline extraction from 370M HF parquet rows | **Queued** | Runs offline; no code deps. Output: `personas.bin` + `sigs_dedup.bin` + `templates/*.yaml` | +| DU-1 | ONNX persona classifier @ L4/L5 — 288-class `(ExternalRole × ThinkingStyle)` product prediction; `style_oracle: Option<&OnnxPersonaClassifier>` in Think struct | **Queued** | Needs ~10K labeled cycles from Lance internal_cold (DM-2 must ship first); replaces Chronos proposal | +| DU-2 | Archetype ECS bridge crate `lance-graph-archetype-bridge` — `ArchetypeWorld → Blackboard`, `ArchetypeTick → UnifiedStep`, `project() → DataFrame component` adapters | **Queued** | Needs DM-2 (ExternalMembrane impl) before adapter can be built | +| DU-3 | RoleDB DataFusion VSA UDFs: `unbind`, `bundle`, `hamming_dist`, `braid_at`, `top_k` — registers in DataFusion session | **Queued** | Fingerprint column type decision needed first (FixedSizeBinary vs FixedSizeList); see open question in plan § 5 | +| DU-4 | MM-CoT stage split: add `rationale_phase: bool` to `CognitiveEventRow`; surface `FacultyDescriptor.is_asymmetric()` in projected RecordBatch | **Shipped** (2026-04-23, commit `a05979e`) | Phase A stub: `rationale_phase: false` in `project()`. Phase B: wire from `FacultyDescriptor::is_asymmetric()`. | +| DU-5 | Board hygiene: DU-0 through DU-4 registered; INTEGRATION_PLANS.md + LATEST_STATE.md updated | **Shipped** (2026-04-23, commit `a05979e`) | Plan corrections + precision-tier §18 + father-grandfather concept committed in follow-up. | + +--- + ## Update protocol When a deliverable ships: diff --git a/.claude/plans/callcenter-membrane-v1.md b/.claude/plans/callcenter-membrane-v1.md new file mode 100644 index 00000000..6e5fd480 --- /dev/null +++ b/.claude/plans/callcenter-membrane-v1.md @@ -0,0 +1,1223 @@ +# Plan — External Callcenter Membrane (Supabase-shape over Lance + DataFusion) + +> **Version:** v1 +> **Author:** main-thread session 2026-04-22 +> **Status:** Active +> **Supersedes:** None +> **Confidence:** CONJECTURE — design grounded in repo evidence + conversation arc; +> no implementation shipped yet beyond DM-0 / DM-1 skeleton. +> +> **READ BY:** Every session touching REST/gRPC external API, realtime +> subscriptions, agent handover, n8n/crewai-rust/openclaw integration, +> auth, or any "external consumer sees cognitive state" work. + +--- + +## § 0 — The One-Sentence Goal + +Assimilate the *design and ergonomics* of the Supabase callcenter surface +into a new crate (`lance-graph-callcenter`) that sits entirely outside +the canonical cognitive substrate, backed by Lance + DataFusion instead of +PostgreSQL, and enforces the BBB (blood-brain barrier) at compile time via +the Arrow type system. + +--- + +## § 1 — What This Is NOT + +- NOT a PostgreSQL-compatible server. +- NOT a full Supabase clone. +- NOT a new canonical reasoning substrate. +- NOT a replacement for `OrchestrationBridge` (the inbound steering + path still goes through `OrchestrationBridge::route(UnifiedStep)`; + the callcenter translates external intent *into* `UnifiedStep`). +- NOT a new home for semiring types, Vsa10k, RoleKey, or FreeEnergy. + +--- + +## § 2 — Architecture (four layers, clean separation) + +``` +╔═══ A: Canonical internal substrate (untouched) ══════════════════════╗ +║ Vsa10k · BindSpace SoA · CognitiveShader · CollapseGate · AriGraph ║ +║ a2a_blackboard · NARS revision · FreeEnergy · Awareness ║ +╚══════════════════════════════╤═══════════════════════════════════════╝ + │ CollapseGate fire (EmitMode::Persist) + │ ShaderBus + MetaWord cross here +╔═══ B: ExternalMembrane trait (in contract crate, zero-dep) ══════════╗ +║ project(&ShaderBus, MetaWord) → Self::Commit ║ +║ — strips VSA fields, produces Arrow scalars ║ +║ ingest(Self::Intent) → UnifiedStep ║ +║ — translates external intent to canonical dispatch ║ +║ subscribe(CommitFilter) → Self::Subscription ║ +║ — typed subscription handle ║ +╚══════════════════════════════╤═══════════════════════════════════════╝ + │ LanceMembrane impl (in callcenter crate) + │ Arrow RecordBatch is the Commit type +╔═══ C: Dual ledger (Lance datasets) ══════════════════════════════════╗ +║ External ledger (append-only, versioned): ║ +║ cognitive_event — every CollapseGate fire, Arrow scalars only ║ +║ steering_intent — inbound, consumed by drain task ║ +║ memory / episode — rolled-up facts ║ +║ actor / session — identity + context accumulation ║ +║ ║ +║ Internal ledger (mutable, NARS-revised): ║ +║ AriGraph + SPO store — inside lance-graph, never touches callcenter║ +║ ║ +║ Lance version counter = CDC stream (replaces PG WAL) ║ +║ tokio::sync::watch on version → subscribers notified ║ +╚══════════════════════════════╤═══════════════════════════════════════╝ + │ DataFusion SQL + Lance reads +╔═══ D: lance-graph-callcenter crate (external, feature-gated) ════════╗ +║ LanceMembrane — ExternalMembrane impl, Arrow RecordBatch commit ║ +║ CommitRecord — concrete Arrow scalar row shape ║ +║ CommitFilter→Expr — PostgREST filter params → DataFusion Expr ║ +║ PhoenixServer — Phoenix channel WS server (scaraude protocol shapes)║ +║ DrainTask — steering_intent → UnifiedStep → OrchestrationBridge ║ +║ JwtMiddleware — verify only; actor_id → LogicalPlan rewriter ║ +╚══════════════════════════════════════════════════════════════════════╝ +``` + +--- + +## § 3 — Blood-Brain Barrier Invariant (the thing that makes it safe) + +**The Arrow type system enforces the BBB at compile time.** + +`LanceMembrane::project()` returns `arrow::array::RecordBatch`. Arrow +columns are typed: `Float32`, `UInt32`, `Utf8`, `Timestamp`, `Binary`. +Semiring types (`SemiringChoice`, `NarsTruth`, `HammingMin`, etc.) and +VSA types (`Vsa10k = [u64; 157]`, `RoleKey`, `FreeEnergy`) do NOT implement +Arrow's `Array` trait. They cannot be placed in a `RecordBatch` column. + +**No runtime check needed. The compiler rejects the violation.** + +The only Arrow column that carries any VSA-adjacent data is +`bundle_bytes: Option` — an opaque pre-computed fingerprint +for fast ANN queries over the external ledger. This is a black-box byte +string to the callcenter; it cannot be unbound or used as a Vsa10k. + +--- + +## § 4 — Cognitive event schema (the only Arrow schema crossing the BBB) + +``` +cognitive_event: + timestamp: Timestamp(Microsecond, UTC) + actor_id: UInt64 + session_id: UInt64 + external_role: UInt8 ← ExternalRole (0..7: User..Agent) + faculty_role: UInt8 ← FacultyRole (0..15: ReadingComp..Empathy..) + expert_id: UInt16 ← ExpertId / card hash (stable hash of card YAML) + dialect: UInt8 ← query dialect tag (Cypher/GQL/NARS/Redis/Spark/SQL) + scent: UInt8 ← 1-byte compressed address scent (see § 10.13) + free_energy: Float32 ← MetaWord::free_e() as f32 + resonance: Float32 ← ShaderResonance::entropy + recovery_margin: Float32 ← top-1 ShaderHit::resonance + thinking_style: UInt8 ← MetaWord::thinking() + awareness: UInt8 ← MetaWord::awareness() + nars_f: UInt8 ← MetaWord::nars_f() + nars_c: UInt8 ← MetaWord::nars_c() + gate_decision: UInt8 ← GateDecision packed + hit_count: UInt16 ← ShaderResonance::hit_count + cycles_used: UInt16 ← ShaderResonance::cycles_used + bundle_bytes: Binary (optional) ← opaque pre-computed fingerprint + is_epiphany: Boolean + is_failure: Boolean +``` + +No Vsa10k. No semiring. No RoleKey. These are numbers and bytes. The five +new identity/scent columns (`external_role`, `faculty_role`, `expert_id`, +`dialect`, `scent`) are the metadata address bus coordinates (§ 10.11); they +are queryable via SQL / Cypher / GQL / NARS / qualia without ever exposing +the internal RoleKey slot mapping. + +--- + +## § 5 — What we take from upstream Supabase Rust crates + +### From `scaraude/supabase-realtime-rs` (v0.1.0, Phoenix channel client) + +Take: +- Phoenix channel message type shapes (`phx_join`, `phx_leave`, + `broadcast`, `postgres_changes`, `PresenceState` shape) +- Topic routing enum structure +- Heartbeat lifecycle pattern + +Do NOT take: +- JWT refresh reconnect machinery (we handle auth differently) +- The full reqwest-based connection manager (we are the server) + +Vendor strategy: manually extract the 6–8 Phoenix message types into +`lance-graph-callcenter/src/phoenix/types.rs`. ~150 lines. No dep on +the upstream crate. + +### From `xylex-group/supabase_rs` (v0.7.0, PostgREST client) + +Take: +- Filter operator enum (`eq`, `neq`, `gt`, `lt`, `gte`, `lte`, `like`, + `ilike`, `in`, `is`, `not`) +- Query-string parameter parsing patterns + +Do NOT take: +- `reqwest` (client-side HTTP; we are the server) +- Storage module (S3 gateway; not needed) +- GraphQL experimental module + +Vendor strategy: define `FilterOp` + `QueryParam` locally in +`lance-graph-callcenter/src/query/filter.rs`. ~200 lines. + +### Explicitly NOT ported + +- GoTrue (Elixir Auth) → JWT verify via `jsonwebtoken` only +- Supabase Storage → not needed +- Edge Functions (Deno) → shader IS the compute +- Kong gateway → axum routing +- Dashboard UI → cockpit is the UI +- PostgreSQL wire protocol → omit; add `pgwire` compat only if n8n demands it + +--- + +## § 6 — Dependency strategy + +``` +# Always compiled (zero-dep contract boundary): +lance-graph-contract (path dep, no extra cost) + +# Behind feature flags (never in default build): +[persist] arrow = "57", lance = "2" +[query] datafusion = "51", arrow = "57" +[realtime] tokio = "1" (sync only), tokio-tungstenite = "0.24", serde, serde_json +[serve] axum = "0.7", tower-http = "0.5" (implies realtime + query) +[auth] jsonwebtoken = "9", argon2 = "0.5" +[full] all of the above + +# NEVER add: +reqwest — client HTTP; we are the server +postgres — no Postgres in architecture +sqlx — same +pg-embed — same +``` + +--- + +## § 7 — Deliverables + +| D-id | Title | Status | +|---|---|---| +| DM-0 | `ExternalMembrane` trait + `CommitFilter` in `lance-graph-contract` | Shipped | +| DM-1 | `lance-graph-callcenter` crate skeleton (Cargo.toml + feature gates + stub lib.rs) | Shipped | +| DM-2 | `LanceMembrane: ExternalMembrane` impl with `project()` + compile-time leak test | Queued | +| DM-3 | `CommitFilter` → DataFusion `Expr` translator (`[query]` feature) | Queued | +| DM-4 | `LanceVersionWatcher` — tails Lance version counter, emits Phoenix `postgres_changes` events (`[realtime]`) | Queued | +| DM-5 | `PhoenixServer` — minimal WS server, Phoenix channel subset, `postgres_changes` + `broadcast` (`[realtime]`) | Queued | +| DM-6 | `DrainTask` — `steering_intent` Lance read → `UnifiedStep` → `OrchestrationBridge::route()` | Queued | +| DM-7 | `JwtMiddleware` + `ActorContext` → `LogicalPlan` RLS rewriter (`[auth]`) | Queued | +| DM-8 | `PostgRestHandler` — query-string → DataFusion SQL → Lance scan → Arrow response (`[serve]`) | Queued | +| DM-9 | End-to-end test: shader fires → LanceMembrane::project() → Lance append → Phoenix subscriber receives event | Queued | + +--- + +## § 8 — Stop-and-re-evaluate points + +1. **After DM-2**: verify compile-time leak test passes. A `RecordBatch` + column containing a `SemiringChoice` should not compile. If the test + reveals a gap in the type system (e.g., newtype tricks), close the gap + before proceeding to DM-3. + +2. **After DM-5**: check shader throughput under realtime fanout load. + Confirm `project()` on every commit does not affect hot-path latency + (should be < 1 μs given it's just field extraction + Arrow row builder). + +3. **Before DM-7**: confirm actual auth requirement from consumers + (n8n, crewai-rust, openclaw). Do they need per-actor RLS, or is + process-level auth sufficient for v1? Do not build RLS rewriter if + single-actor usage is the real deployment model. + +4. **Before DM-8**: confirm PostgREST compat is needed vs cockpit using + Phoenix channel subscriptions only. The `serve` feature is the heaviest; + defer if direct Lance reads via Rust API satisfy all consumers. + +--- + +## § 9 — Unknowns (explicitly open) + +- **UNKNOWN-1**: Does `cognitive-shader-driver`'s `ShaderSink` trait + already serve as an `ExternalMembrane` façade, or is there an overlap? + Inspect `crates/cognitive-shader-driver/src/` before wiring DM-2. + +- **UNKNOWN-2**: Which consumers (n8n-rs / crewai-rust / openclaw) actually + need a Phoenix/Supabase wire protocol vs direct Rust API calls? + Verify before building the realtime server (DM-4/DM-5). + +- **UNKNOWN-3**: Does n8n-rs need a Postgres wire (pgwire) connection or + is the existing `OrchestrationBridge` path sufficient? + +- **UNKNOWN-4**: What is the right `actor_id` type? The contract crate + does not yet have an Actor concept. A plain `u64` hash suffices for v1 + but may conflict with future identity semantics. Mark explicitly. + +- **UNKNOWN-5**: Lance dataset path / object-store URL configuration. + Single env var `LANCE_URI` is proposed but not yet defined anywhere in + the repo. + +--- + +## § 10 — Markov XOR Gate + Blackboard Mediation Refinement (2026-04-22) + +> **Status:** FINDING — replaces § 2's "dual Lance datasets" as the live mechanism. +> Lance persistence is still correct for durability; but the LIVE mediator is the +> existing A2A `Blackboard`, not two row-appended datasets. + +### 10.1 — The blackboard is the explicit firewall + +External events do NOT go directly into the BindSpace SoA or the Markov trajectory. +They land on `a2a_blackboard::Blackboard` first: + +``` +Consumer sends seed + → BlackboardEntry { capability: ExpertCapability::ExternalSeed, ... } + → A2A experts run N rounds (30–300 ns/round) + → CollapseGate fires, project() → scalar commit + → External channel receives event (30–300 ms later) +``` + +This is explicit, not conflated. The blackboard round boundary IS the anti-corruption +boundary. Internal experts never see raw external payloads — they see blackboard entries +with confidence scores, dissonance fields, and capability tags, same as any other expert. + +### 10.2 — The Markov ±5 reuse (no new research) + +The grammar parser uses Markov ±5 XOR braiding: + +``` +token at position d → XOR-superpose with ρ^d braiding into trajectory +``` + +The callcenter membrane uses the SAME mechanism across blackboard rounds: + +``` +Blackboard.round = trajectory position +ExternalSeed entry at round N → XOR-braided at ρ^d into the next context bundle +``` + +The `Blackboard.round: u32` counter already exists. The ±5 window means: when +computing the next context bundle, look at entries from `[round - 5, round]`. +No new data structure. No new research. Same ρ^d decay. Same XOR-bundle accumulation. + +For fast agents (LLM round-trips ~10ms): `context_band = (-5, 0)` — 5 rounds back. +For human-in-the-loop: `context_band = (-500, 0)` — 500 rounds back via episodic. +The seed carries this as an explicit parameter; the blackboard honours it. + +### 10.3 — Role taxonomy (`ExternalRole`, now in contract crate) + +Every event crossing the gate is role-tagged before XOR-braiding: + +| Direction | Role variant | Meaning | +|---|---|---| +| Inbound | `User` | direct human input | +| Inbound | `Consumer` | generic API caller | +| Inbound | `N8n` | n8n-rs workflow step | +| Inbound | `OpenClaw` | openclaw agent | +| Inbound | `CrewaiUser` | crewai-rust user role | +| Inbound | `CrewaiAgent` | crewai-rust agent role | +| Outbound | `Rag` | shader presenting as RAG retriever | +| Outbound | `Agent` | specific cognitive agent result | + +Role binding at the gate: `RoleKey::bind(payload, role as u16)` — same slot +addressing as SUBJECT/PREDICATE/OBJECT in the grammar. "Who said this" becomes +a readable coordinate in the trajectory; unbinding recovers it. + +### 10.4 — `ExternalEventKind` (now in contract crate) + +| Kind | Behaviour | +|---|---| +| `Seed` | Deposits `BlackboardEntry(ExternalSeed)`. DrainTask picks it up, routes to `OrchestrationBridge`. | +| `Context` | Deposits `BlackboardEntry(ExternalContext)`. XOR'd into trajectory on next bundle pass; no active cycle triggered. | +| `Commit` | Projected scalar leaving the substrate toward an external subscriber. Never enters the blackboard. | + +### 10.5 — Speed gap absorbed here, not elsewhere + +The substrate runs continuously at 30–300 ns/op. The external consumer pulls at +30–300 ms. The substrate does NOT wait. CollapseGate fires on its own Markov schedule; +the current bundle is whatever is latest. External channels receive the projection +when the consumer is ready — pull-whenever, push-when-committed. The 10⁵–10⁷× gap +is structural, not a problem to solve. The gate just keeps bundling. + +### 10.6 — Agent cards as A2A experts (one identity space) + +The three identity spaces we had — + (a) internal A2A experts (`ExpertId` + `ExpertCapability`), + (b) external roles (`ExternalRole`), + (c) YAML agent cards (`crewai-rust/*`, `.claude/agents/*.md`) — +collapse into one system by convention: + +| Layer | Carries | Granularity | +|---|---|---| +| `ExternalRole` (at the gate) | Family — who invoked this | 8 variants | +| `ExpertId` (on the entry) | Specific card / expert | u16 (65k) | + +**Convention:** for agent cards, `ExpertId = stable_hash_u16(card_yaml)`. +A `crewai-rust` agent or a `.claude/agents/family-codec-smith.md` card +both produce the same kind of `ExpertEntry` and post to the same blackboard +as any internal A2A expert. No distinction in the bus. + +**Addressability** (corrected 2026-04-22 — see erratum below): + +Role and card are SEPARATE typed metadata columns on `cognitive_event` rows +(`external_role: UInt8`, `expert_id: UInt16`). They are addressable independently +via the five query dialects on the metadata bus: + +- *"All `Rag`-family cards"* — `WHERE external_role = 6` +- *"Just card `0x7F3A`"* — `WHERE expert_id = 0x7F3A` +- *"family-codec-smith as CrewaiAgent at round N"* — `WHERE external_role = 5 AND expert_id = 0x7F3A AND round = N` + +Stack-side VSA binding of these identities happens through a deterministic +metadata→RoleKey slot mapping; the mapping never crosses the BBB. + +> **Erratum (2026-04-22):** earlier drafts of this section proposed a packed +> 32-bit braid key `(role << 16) | expert_id` as a VSA slot address. That was +> wrong — it would have carved a parallel address space incompatible with the +> existing 4096 COCA / CAM-PQ / NARS-head vocabulary. Identity lives in +> metadata; VSA binding is a stack-side internal concern. See § 10.11 for the +> metadata address bus doctrine. + +**Meta-awareness consequence:** `QualiaClassification` and `StyleModulation` +experts can fire on features like *"current context is RAG-heavy but +card-diverse"* or *"family-codec-smith resonance falling while palette-engineer +rising"*. The texture and resonance across both family AND card coordinates +become observables in the same SoA sweep — no new column, no new lookup, +just unbind at different mask depths. + +**Registration:** an agent card YAML gets hashed at load time into an +`ExpertEntry`. The `capability` field is read from the card's declared +primary capability; `base_confidence` from its trust prior. After registration +the card participates in blackboard routing (`next_round_experts`) exactly like +any hand-coded expert. + +### 10.7 — Consumers address roles: explicit OR implicit + +External consumers (`crewai-rust`, `n8n-rs`, `openclaw`, …) don't just deposit +seeds — they can *address* specific roles through the same blackboard, two ways: + +**Explicit routing** via `RoutingHint` on the seed: + +| `target_role` | `target_card` | Router behaviour | +|---|---|---| +| `Some(r)` | `Some(c)` | Activate exactly card `c` in family `r` (full address). | +| `Some(r)` | `None` | Activate the best card in family `r` (family route; AriGraph-resonance tiebreak). | +| `None` | `Some(c)` | Activate card `c` regardless of family. | +| `None` | `None` | Implicit — AriGraph-resonance matching against seed payload. | + +**Implicit routing** (no hint): the router matches the seed's context fingerprint +against each registered persona's AriGraph subgraph resonance. Top-k personas +activate for the next round. This is the same `next_round_experts` mechanism the +blackboard already runs — the new input is "persona resonance score" alongside +the existing `base_confidence × weight` term. + +**Consequence:** a crewai-rust agent can say *"route this to the palette-engineer +card explicitly"* OR *"find whoever resonates with this codec question"* through +the exact same seed shape. The two modes are not separate APIs — they're the +presence/absence of the hint on the same entry. + +### 10.8 — AriGraph integration: the persona IS its subgraph + +Per CLAUDE.md's AGI-as-glove doctrine, AriGraph is thinking tissue — not a +service the struct queries, but part of its reasoning surface. Extending this: +**each persona's AriGraph subgraph IS its memory.** + +A persona's full reasoning surface: + +``` +PersonaCard (in contract, identity only) + ├── role: ExternalRole — family at the gate + └── entry: ExpertEntry — id, capability, trust, weight + │ + │ resolved at lance-graph boundary: + ▼ + AriGraph::subgraph_for(entry.id) — in crates/lance-graph/src/graph/arigraph/ + ├── committed SPO triples — what this persona has "said" or "believed" + ├── reversal history — NARS-revised truth values + ├── episodic trace — ±5..±500 round window of participation + └── resonance index — for implicit routing matching +``` + +**AriGraph module layout (in lance-graph core, not contract):** +- `graph/arigraph/triplet_graph.rs` — already the SPO backbone +- `graph/arigraph/episodic.rs` — already the ±5..±500 window +- `graph/arigraph/retrieval.rs` — the resonance-match surface the router reads +- `graph/arigraph/orchestrator.rs` — already coordinates multi-expert cycles + +**New wiring (future DM):** `AriGraph::subgraph_for(ExpertId) → PersonaSubgraph`, +a filtered view over the shared graph keyed by the persona's id. When the router +does implicit resonance matching, it iterates registered personas and calls +`persona_subgraph.resonance_against(seed_fingerprint)`. No new data structure, +no new research — just a view projection over the existing AriGraph. + +**What this unlocks:** personas that LEARN. When card X handles a seed and the +resulting cycle commits, the SPO triple lands in X's subgraph. Next time a +semantically similar seed arrives, X's resonance is higher — it has memory. +This is the same Commit/Epiphany/FailureTicket loop from CLAUDE.md's P-1 Click, +now applied per-persona. Each agent card is its own cognitive loop on a +subgraph; the blackboard composes across them. + +### 10.9 — The "Membrane · Role · Place · Translation" contract (iron rule from the prior stack) + +Every piece of data crossing the gate MUST satisfy FOUR conditions. This is +older doctrine from the prior stack, now explicit and enforced here: + +**1. Pass the membrane.** No direct writes to BindSpace SoA. The only +BindSpace-adjacent write site is `CollapseGate::merge()` with +`MergeMode::Bundle` (Markov-respecting) or `MergeMode::Xor` (single-writer +delta). A payload that bypasses the gate is rejected at the type system — +Arrow's column types cannot hold `Vsa10k`, `RoleKey`, `SemiringChoice`, etc. + +**2. Get a role.** The payload is bound with a `RoleKey` keyed on +`ExternalRole × ExpertId`. No role = no binding = the payload cannot enter. +*Who said this* is a required field of entry, not metadata. + +**3. Get a place.** The payload lands at a specific trajectory position +(blackboard `round`) and a specific VSA slot (role-indexed slice within +the `[u64; 157]` substrate). Every byte has coordinates. No roving. + +**4. Translate into internal reasoning.** The external payload is not +*stored* as bytes — it is *transcribed* into the substrate's grammar: + +| Event kind | Translation at the gate | +|---|---| +| `Seed` | `Vsa10k` trajectory token (RoleKey::bind + ρ^d braid into next round's bundle) | +| `Context` | XOR-superposed into the current context bundle (no new cycle) | +| `Commit` | Already internal; project-only path, no translation needed (it's leaving) | + +The translation is **not reversible**. The substrate does not keep external +bytes "just in case". It only speaks its own language. If a consumer needs +their original bytes back, that's their persistence concern — the gate does +not cache pre-translation payloads. + +**Compile-time enforcement points:** + +| Contract rule | Where it's enforced | +|---|---| +| (1) Membrane | Arrow column types reject VSA/semiring (§ 3 BBB invariant) | +| (2) Role | `ExternalMembrane::ingest(Self::Intent) → UnifiedStep` — the impl must role-bind | +| (3) Place | `BlackboardEntry` always carries `expert_id`; `Blackboard.round: u32` | +| (4) Translation | `ingest()` return type is `UnifiedStep`, not a raw payload — translation is forced at the trait boundary | + +**Consequence:** there is no "untagged external state" anywhere in the +system. Every piece of memory the substrate holds was, at some point in +its history, role-bound, place-addressed, and translated. If you cannot +answer *who said it, when, and in what VSA grammar form* for a piece of +data, it does not belong in the substrate. + +This is the test to apply before merging any new code path that touches +the gate: can I name the role, the place, and the translation? If not, +the code is leaking external ontology inward — reject. + +### 10.10 — VSA 10000-D: lossless role bind/unbind for the internal face + +Roles, faculties, dialects, scents — every internal identity — are bound +into the `Vsa10k = [u64; 157]` substrate via `RoleKey::bind` using +role-indexed slices in the 10000-dim space. This is NOT compression — it +is *lossless*: + +- `RoleKey::bind(payload, role_slot)` places the payload at a specific + role-indexed slice; the other 10000 − slice_width bits are untouched. +- `RoleKey::unbind(trajectory, role_slot)` recovers the payload exactly + from that slice. Bind / unbind are inverse operations at the bit level. +- XOR-superposition across roles remains lossless by construction: the + Johnson-Lindenstrauss + concentration-of-measure bounds (I-SUBSTRATE-MARKOV) + guarantee associativity in expectation; d = 10000 suppresses deviation + at rate ~e⁻ᵈ. See EPIPHANIES.md E-SUBSTRATE-1. + +**Which identities live here (internal face):** + +| Identity | Slot region | Semantics | +|---|---|---| +| `ExternalRole` | reserved family-slot window | "who crossed the gate" | +| `FacultyRole` | reserved faculty-slot window | "which cognitive function is active" | +| `ExpertId` (card hash) | card-slot window | "which specific persona" | +| `dialect` | dialect-slot window | "which query language" | +| `scent` | scent-slot window | "which address neighborhood" | + +Each gets its own reserved slot window in the 10k-dim space. No collisions +with the 4096 COCA vocabulary / CAM-PQ codebook / NARS-head region because +those live in their own allocated ranges; the identity slots are +non-overlapping by partition (zone map belongs in `encoding-ecosystem.md` +when we codify the full layout). + +**Why lossless matters.** A cycle that commits with `FacultyRole::Reasoning` ++ `ExternalRole::CrewaiAgent` + `ExpertId::family-codec-smith` must be +exactly recoverable at any later round via unbind. If the role encoding were +lossy, replay / audit / reversal would drift. Lossless bind/unbind is what +makes the Markov ±5 trajectory a *substrate*, not a buffer. + +**BBB role.** These bindings live exclusively on the internal face. +`RoleKey`, `Vsa10k`, and the slot assignments never cross the gate. The +external face sees the SAME identities as metadata columns (§ 10.11) — +deterministically mapped stack-side. Two faces, one identity, zero leak. + +### 10.11 — The metadata address bus (queries ARE dispatch, external face) + +The `cognitive_event` Arrow table (§ 4) is not a log. It is the **uniform +address bus** of the entire callcenter. Every row carries a typed identity +tuple `(external_role, faculty_role, expert_id, dialect, scent, …)`; +every query over this table returns a row set; every returned row's tuple +IS an execution address. Dispatch = (predicate, action); there is no +separate router. + +**Five dialects, one bus:** + +| Dialect | Strength | Example addressing | +|---|---|---| +| SQL (DataFusion) | tabular / range / aggregation | `WHERE external_role = 3 AND free_energy < 0.2` | +| Cypher / GQL | graph-path, persona neighborhoods | `MATCH (c:Card)-[:HANDLED]->(e:Event {faculty_role: 2})` | +| NARS | truth-weighted selection | `f > 0.7 AND c > 0.6` — route only to confident personas | +| Qualia | fuzzy family match | route by qualia signature, not exact ids | +| OrchestrationBridge | domain-coarse pre-scope | `StepDomain::Thinking` narrows the bus range | + +Composes with shipped machinery: +- `RoutingHint` (persona.rs) — explicit case = degenerate query + (`WHERE role = target_role AND card = target_card`). +- `Blackboard::by_capability()` — already a bus primitive. +- `CommitFilter` (external_membrane.rs) — a narrow predicate on the bus. +- `PlannerContract` — every dialect compiles through it to DataFusion. + +**BBB role.** The bus is the EXTERNAL face of identity. Row values are +`UInt8` / `UInt16` / `Float32` — Arrow scalars that cross the gate safely. +The internal 10k-dim slot mapping (§ 10.10) of these same identities never +appears in any row, any query, any API. + +### 10.12 — DN-addressed REST + polyglot dialects (ladybug-rs pattern) + +The external wire surface is a single endpoint shape: + +``` +POST /tree/{ns}/heel/{h}/hip/{h}/branch/{b}/twig/{t}/leaf/{l} +Content-Type: application/{dialect}+text +Body: +``` + +**URL path = metadata predicate.** Deterministically parsed to: +```sql +WHERE tree = $ns AND heel = $h AND hip = $h + AND branch = $b AND twig = $t AND leaf = $l +``` +No per-endpoint routing logic — the parser is regular, the predicate is +regular, the dispatch is bus § 10.11. + +**Starter dialects** (four already shipped, three added): + +| Dialect | Status | How it lands in the stack | +|---|---|---| +| Cypher | shipped (`parser.rs`, 44 tests) | full parser → logical plan | +| GQL (ISO 39075) | shipped | full parser → logical plan | +| Gremlin | shipped | full parser → logical plan | +| SPARQL | shipped | full parser → logical plan | +| **NARS** | planned | full parser (`?x <--> animal. %{f>0.7;c>0.6}%`) — typed cognitive query | +| **Redis** | planned | **thin shape-adapter over DataFusion — NOT a new parser.** Redis commands map to DataFusion ops on a slightly-shaped row view. `GET tree:foo:leaf:bar` → `SELECT content WHERE tree=foo AND leaf=bar`. `HGET k f` → single-column `SELECT`. `ZRANGE k 0 n` → `ORDER BY score LIMIT`. No separate execution engine. | +| **Spark SQL / DataFrame** | planned | DataFusion handles most Spark SQL directly; DataFrame API maps to DataFusion's DataFrame API; structured streaming maps onto realtime channels | +| DataFusion SQL | shipped (baseline) | canonical power-user path | +| PostgREST filter params | planned (§ 5) | ergonomic REST query strings | + +**Shared compile stack.** Every full-parser dialect → `LogicalPlan` IR → +DataFusion physical plan → metadata bus dispatch. Redis is the exception: +it is an ergonomic view over DataFusion, not a distinct IR branch. + +**Dialect-as-signal.** The dialect itself is metadata: `dialect: UInt8` +on the cognitive_event row. Tells the router which cognitive faculty the +consumer is most likely exercising (Cypher → graph-path reasoning; +NARS → truth-aware inference; Redis → DN-KV pattern; Spark → bulk +analytic). Routing can pre-bias persona activation on dialect before the +body is parsed. + +**BBB role.** The REST surface only ever serves / accepts Arrow-scalar +JSON and parsed query IR. It never sees `Vsa10k`, `RoleKey`, or any +internal type. Responses are projected commits (§ 4 schema rows). + +### 10.13 — Address = scent = context-pull key + +The DN path is not just a routing predicate. It compresses through the +existing codec stack into a **scent** — a 1-byte fingerprint that serves +four uses with one representation: + +``` +DN path (16Kbit) → ZeckBF17 (48B) → Base17 (34B) → CAM-PQ (6B) → Scent (1B, ρ=0.937) +``` + +(Same chain as `docs/CODEC_COMPRESSION_ATLAS.md`.) + +**Four uses, one scent:** + +| Use | Traditional key | Now | +|---|---|---| +| Route | routing key | scent | +| Retrieve | retrieval vector | scent | +| Similar | nearest-neighbor fingerprint | scent | +| Frame | cognitive prior | scent | + +**Context pull.** On seed ingress, scent is computed first. It fires four +parallel pulls before the shader reads the body: +- AriGraph subgraph retrieval — triples near scent neighborhood +- Episodic memory ±5..±500 — past cycles with similar scent +- Persona trust lookup — cards whose AriGraph resonates with this scent +- Qualia classification — codebook cell signature for fuzzy family + +The shader's F-descent cycle runs with the pulled context as prior; the +body content is interpreted AGAINST that context, not in isolation. + +**Result.** Supabase-shape RAG returns document chunks matched by vector. +This returns reasoned intelligence contextualized by the full cognitive +substrate state at this scent. Same REST envelope, same JSON response +shape — response grounded in accumulated reasoning, not lexical similarity. +Next query at nearby scent benefits from this commit's SPO write-back +(AriGraph keyed on scent accumulates). Training-without-labels loop +(E-DEPLOY-1) deepens the cascade with use. + +**BBB role.** `scent: UInt8` is a metadata column (safe to cross). Its +computation uses internal codec primitives (Base17, CAM-PQ, Hamming) but +those never leave the stack side. The scent byte is the surface artifact; +the internal compressive path is invisible to the consumer. + +--- + +## § 11 — Sequencing (post-epiphany) + +Given E-DEPLOY-1 and §§ 10.10–10.13, the DM sequence gains three phases +inside the existing plan: + +| Phase | Deliverables | +|---|---| +| **A — BBB spine** | DM-2 (LanceMembrane impl) + compile-time leak test; § 4 schema with five identity/scent columns; metadata address bus wired; stack-side metadata ⇄ 10k slot mapping | +| **B — Polyglot front end** | extend `PolyglotDetect` with NARS + Spark full parsers; Redis shape-adapter over DataFusion (not a parser); DN-path URL parser → DataFusion `Expr` | +| **C — Scent cascade** | DN → ZeckBF17 → Base17 → CAM-PQ → Scent wiring; context-pull on seed ingress; persona AriGraph subgraph view keyed on scent | +| **D — Realtime + training** | DM-4 + DM-5 (Phoenix channels); commit-writeback trains personas; A2A consumer integration (crewai-rust, n8n-rs first) | + +Each phase is independently testable and reversible. Litmus test +(§ 10.9 + E-DEPLOY-1 footer) applies at every merge. + +--- + +## § 13 — "Git for Cognition" — the unified mental model + +**Brainstorm origin:** 2026-04-23 — the observation that callcenter needs +git-like access to internal state. Not a metaphor: the shared algebraic +foundation is a commutative monoid on blobs. Git approximates it (and +patches around conflicts). VSA d=10000 saturating bundle IS it — +lossless, by CK construction (E-SUBSTRATE-1). Jirak bounds exactly +where the approximation becomes noise. + +### Primitive mapping (one-to-one) + +| Git primitive | Callcenter primitive | Existing machinery | +|---|---|---| +| `commit` | CollapseGate fire → Lance version N | `GateDecision::COMMIT` + `project()` | +| `branch` | Speculative blackboard round (not yet fired) | `Blackboard.round` before CollapseGate | +| `merge` | Bundle two trajectories | `MergeMode::Bundle` (CK-safe) | +| `rebase` | Replay trajectory against different NARS prior | Markov ±5 replay + `awareness.revise()` | +| `checkout HEAD~5` | Markov position −5 in the braid window | Braid offset, already exists | +| `checkout ` | Lance time-travel | `dataset.checkout(version=N)` | +| `diff V1..V2` | Projected RecordBatch row comparison | Two Lance versions, subtract | +| `blame` | `unbind(role)` at trajectory position → metadata column | VSA unbind, map slot → scalar | +| `cherry-pick` | Inject one BlackboardEntry into another round | Router decision | +| `stash` | Speculative bundle without Persist emit | `EmitMode::Bundle` (not `Persist`) | +| `pull` | `subscribe(filter)` | `ExternalMembrane::subscribe` | +| `push` | `ingest(ExternalIntent)` | `ExternalMembrane::ingest` | +| `log --max-count=500` | Markov ±500 episodic window query | Existing episodic memory | +| `tag` | Named persona checkpoint | `PersonaCard` + Lance tag | +| `.gitignore` | `CommitFilter` | Already in contract | +| `pre-commit hook` | NARS check before `EmitMode::Persist` | CollapseGate predicate | + +### Why the callcenter is stronger than git + +Git merge commutativity is approximate — conflicts require human +resolution. VSA d=10000 bundle commutativity is exact (concentration-of- +measure at rate ~e^(−d)). Git blobs are opaque bytes; blame is textual +search. Callcenter blobs are VSA bundles with semantic coordinates; +`blame` is `unbind(role)` — a structured algebraic query. Git rebase can +corrupt history; Markov replay is provably lossless. + +**The scalar-vs-VSA distinction is the blob boundary:** +- Git: blob = file bytes (opaque to git itself) +- Callcenter: blob = VSA bundle (unbind-able by role, Cartan-character- + indexed slots — see [FORMAL-SCAFFOLD]) + +### Chat rounds as commits (§§ 10–13 synthesis) + +A chat turn is a commit. The blackboard round between turns is the +staging area. CollapseGate fire = `git commit`. Lance version bump = +the append to the object store. The ±5/±500 Markov window is `HEAD~5` +and `HEAD~500`. The 10⁵–10⁷× speed gap is absorbed exactly at the +turn boundary — substrate runs at 30 ns/bind internally; external +subscribers see one tick per committed turn. + +**Jirak** (I-NOISE-FLOOR-JIRAK) bounds the turn-update density: too +sparse → consumer context diverges; too dense → weak-dependence breaks. +The ±5 window is the implicit rate limit. + +**Cartan-Kuranishi** governs which columns get projected outward — not +arbitrary, but the intrinsic fiber geometry of the external signal. +`dialect: u8` and `scent: u8` are the Cartan-intrinsic columns; +their slot widths in the 10k substrate should not be chosen by +convention. [FORMAL-SCAFFOLD] revival candidate 3 (learned attention +masks) applies for empirical confirmation. + +### Porcelain vs plumbing (BBB as the split) + +| | Git | Callcenter | +|---|---|---| +| **Porcelain** | `git add`, `git commit`, `git log` | Supabase-shape REST, DN paths, JSON | +| **Plumbing** | `git hash-object`, `git cat-file` | VSA bind/unbind, Markov braid, AriGraph | +| **The line** | `git` CLI | `ExternalMembrane` trait | + +Consumers use porcelain only. Internal faculties use plumbing. +`ExternalMembrane::ingest()` and `project()` are the translators. + +### Consumer mental model (adoption surface) + +**The pitch:** "git for thoughts." + +- n8n-rs / crewai-rust author who knows git → zero onboarding friction +- Human via q2 → Neo4j browser ≈ `gitk` / `git log --graph` +- Curl consumer → `git show ` shape over REST + +DN URL path (`/tree/ns/heel/h/hip/x/branch/b/twig/t/leaf/l`) IS a +refspec. PersonaCard is the author. FacultyDescriptor is the committer. +RoutingHint is the refspec target. + +### Three primitives to name explicitly + +1. **`Speculative`** — blackboard round that exists but has not yet + fired CollapseGate. Equivalent to git's staged-but-not-committed. + Already implicit; needs a first-class name in the callcenter API. +2. **`Rebase` verb** — replay Markov trajectory against a different + NARS prior. Mechanics exist; needs an `ExternalMembrane` method or + REST endpoint in Phase B. +3. **`Blame` projection** — `unbind(role)` internally, project to + metadata-column answer externally. VSA slots never cross the BBB; + the response is always `external_role: u8`, `faculty_role: u8`, + `expert_id: u16`. + +### BBB invariant holds through the git metaphor + +None of the git-shaped primitives requires VSA types to cross the gate: +- `commit` result → Arrow scalars only +- `blame` result → metadata columns only +- `checkout` result → projected RecordBatch +- `branch` / `merge` → internal only (MergeMode::Bundle) + +§ 10.9 iron rule (membrane → role → place → translate) holds at every +git verb. + +--- + +## § 14 — Cold Storage = Git Cold Storage (Two Dataset Classes) + +**Epiphany trigger:** "lance-graph/lancedb + S3 cold storage becomes a git cold storage." + +### Two dataset classes + +| Dataset class | Content | Crossing BBB? | Queryable? | +|---|---|---|---| +| **External / scalar** | Arrow RecordBatch rows from `project()` | Yes (IS the BBB output) | Yes — DataFusion, Supabase FDW, n8n subscribe | +| **Internal / VSA** | `Fingerprint<256>` = `[u64;256]` cycle fingerprints (L4/L5 speed tier, 2 KB/row); NARS truth vectors, braid offsets. L3 cold tier can promote to Vsa10k BF16 (20 KB, lossless) or RaBitQ-quantized Lance columns (zero-copy ANN). | No — never crosses BBB | Yes — DataFusion + VSA UDFs (see § 15) | + +### Parallels with git cold storage + +``` +git object store ≈ Lance + S3 (append-only, content-addressed by version) +git blob ≈ internal VSA fingerprint bundle (opaque to external consumers) +git tree ≈ Blackboard round snapshot (expert entries + round number) +git commit object ≈ external scalar RecordBatch row (the `project()` output) +git pack file ≈ Lance fragment files (batched, compressed, indexed) +git remote ≈ S3 bucket (cold tier, queryable in-place via DataFusion S3 scan) +``` + +### Why Lance is stronger than git cold storage + +- Git blobs are opaque bytes, indexed only by SHA1. Lance fragments carry Arrow + schema; every column is queryable in-place without extracting. +- Git time-travel is commit-hash lookup. Lance time-travel is `dataset.checkout(version=N)`, + which DataFusion can scan directly across versions (temporal join). +- Git has no query engine. Lance has DataFusion — the internal VSA dataset IS + a queryable database via the VSA UDFs in § 15. + +### Training corpus path (E-DEPLOY-1) + +Internal VSA dataset is pre-labeled by F outcome per epiphany E-DEPLOY-1: +- Each row: `{ fingerprint: [u64; 256], meta: MetaWord, f_outcome: f32, role: u8, style: u8 }` +- Rows where F < 0.2 are positive training examples (committed cycle = "good commit") +- Rows where F > 0.8 are negative examples (failed cycle = "bad commit") +- This dataset trains the ONNX persona classifier in § 17 with zero labeling effort + +### Two-tier storage topology + +``` +Hot tier (Lance in-process): + external_dataset — last N committed RecordBatch rows (Supabase-facing) + internal_dataset — last M cycle fingerprints (VSA UDF-facing) + +Cold tier (S3): + external_cold/ — full projection history (audit log, training data) + internal_cold/ — full fingerprint history (ONNX training corpus) +``` + +`ExternalMembrane::subscribe()` wires to a `tokio::sync::watch` on the +Lance version counter of `external_dataset`. External consumers never +touch `internal_dataset`. + +--- + +## § 15 — VSA Dispatch: role × thinking = persona (RoleDB) + +**Core insight:** routing IS a VSA query. `unbind(target_role, trajectory)` returns +the overlap between the trajectory's role-indexed region and the query role — +locality-sensitive, not exact hash. This means dispatch is approximate-nearest-neighbor +over the role-key space, not an if/else over role enums. + +### The product identity + +``` +persona ≡ ExternalRole × ThinkingStyle +``` + +- `ExternalRole` (8 variants) — the "who" coordinate +- `ThinkingStyle` (36 variants) — the "how" coordinate +- Product space: 8 × 36 = **288 canonical personas** +- `PersonaCard` IS this product pair: `(role: ExternalRole, style: ThinkingStyle)` +- AriGraph subgraph keyed on `(ExternalRole, ThinkingStyle)` — one subgraph per persona + +### VSA dispatch mechanics + +``` +route(step): + role_hint = step.thinking.map(|ctx| ctx.style) or RoutingHint.target_role + trajectory = current ShaderBus.cycle_fingerprint + overlap = unbind(role_key[role_hint], trajectory) // VSA inner product + best_match = argmax(overlap over all registered personas) + → dispatch to best_match's FacultyDescriptor +``` + +This replaces exact-match routing in `OrchestrationBridge::route()` with VSA +locality-sensitive routing. A `CrewaiUser` step with `Reasoning` style routes to +the persona `(CrewaiUser, Reasoning)` — the subgraph whose VSA fingerprint has the +highest overlap with the current trajectory state. + +### RoleDB — DataFusion + VSA UDFs + +Five UDFs registered in DataFusion that make the internal VSA dataset queryable +as a "DuckDB over roles": + +| UDF | Signature | Purpose | +|---|---|---| +| `unbind(role, trajectory)` | `(u8, [u64;256]) → f32` | Role overlap (dispatch score) | +| `bundle(cols...)` | `([u64;256]...) → [u64;256]` | Bundle multiple fingerprints | +| `hamming_dist(a, b)` | `([u64;256], [u64;256]) → u32` | Fingerprint distance | +| `braid_at(pos, traj)` | `(i32, [u64;256]) → [u64;256]` | Markov position lookup | +| `top_k(bundle, k)` | `([u64;256], u32) → [u16]` | Top-K persona candidates | + +SQL example: +```sql +SELECT expert_id, unbind(2, fingerprint) AS n8n_overlap +FROM internal_dataset +WHERE round = (SELECT max(round) FROM internal_dataset) +ORDER BY n8n_overlap DESC +LIMIT 5; +``` + +This is "DuckDB emulation based on roles" — DataFusion with VSA semantics +replacing exact predicate matching. + +### n8n-rs + Supabase Realtime wiring + +``` +Lance external_dataset + → PostgreSQL FDW (read-only view over Lance S3 parquet) + → Supabase Realtime + → n8n-rs WebSocket subscription (filter: external_role = N8n) +``` + +No polling. No webhook. Pure subscribe-on-row-insert. Each `project()` call that +produces a row with `external_role = N8n` notifies n8n-rs WebSocket subscriber +within one Lance version tick. + +--- + +## § 16 — Persona as Function: 32 Atoms × 16 Weightings + YAML Runbooks + +**Three layers of persona representation (not three different definitions — one identity, three representations):** + +### Layer 1 — Identity (product type, 9 bits) +```rust +struct PersonaId { + role: ExternalRole, // 3 bits (8 variants) + style: ThinkingStyle, // 6 bits (36 variants → 64 slots) +} +// 288 canonical personas; AriGraph keyed on this pair +``` + +### Layer 2 — Signature (56 bits, compressed PersonaHub) +```rust +struct PersonaSignature { + atom_bitset: u32, // which of 32 named cognitive atoms are active + palette_weight: u8, // 16 weight levels packed as 4-bit × 2 atoms (or 8-bit coarse) + template_id: u16, // which YAML runbook template handles this signature +} +// Total: 7 bytes = 56 bits per persona signature +``` + +The 32 named cognitive atoms (semantic operations, not styles): +``` +deduction, induction, abduction, analogy, counterfactual, +causal, temporal, spatial, modal, deontic, +metaphor, narrative, hypothesis, contradiction, revision, +retrieval, synthesis, compression, expansion, clarification, +empathy, perspective, intention, belief, desire, +uncertainty, confidence, negation, quantification, comparison, +classification, decomposition +``` + +**Addressing space:** +- Each of 32 atoms takes one of 16 weight levels (0–15) +- Configurations: 16^32 ≈ 3.4×10^38 +- PersonaHub's 370M personas are samples in this space +- 56-bit signature reduces 370M rows to a lookup table + 370M×7 bytes ≈ 2.6 GB flat file + +### Layer 3 — Runbook (YAML template, macro scaffolding) + +YAML runbooks are NOT persona identity. They are behavioral scripts for +specific context-loop shapes — multi-turn recovery, escalation, handoff: + +```yaml +# template_0042.yaml — deduction + counterfactual heavy, reasoning style +name: deep_deduction_loop +atoms_required: [deduction, counterfactual, hypothesis, contradiction] +min_weight: 8 +loop: + - step: establish_premises # braid_at(-1, trajectory) → premise bundle + - step: generate_counterfactual + - step: test_contradiction + - step: revise_if_dissonance # awareness.revise() if F > 0.4 + - step: commit_if_stable # CollapseGate if F < 0.2 +fallback: escalate_to_llm +``` + +**Why YAML and not inline prompts:** +- Deterministic, versioned, Git-trackable (can `git diff` across persona generations) +- Composable — templates can `include` sub-templates (context-loop macros) +- More precise than prompt templates for multi-step reasoning scaffolding +- Separate from the persona's identity and separate from per-cycle VSA reasoning +- JIT-compilable: `JitCompiler::compile(template_id)` → `KernelHandle` at dispatch time + +### Storage arithmetic +``` +PersonaHub 370M rows: + Original: 370M × ~2KB/row = ~740 GB + Signatures: 370M × 7 bytes = 2.6 GB (285× reduction) + + template library: 10K × ~200 bytes YAML = 2 MB + Total: ~2.6 GB vs ~740 GB (46,000× total reduction) +``` + +--- + +## § 17 — The Four-Way Multiply + Style Oracle (ONNX@L4/L5) + +### The four multiplicands + +``` +Persona × Style × Stage × Learned-dynamics = cognitive configuration space +``` + +| Axis | Cardinality | Representation | Implementation | +|---|---|---|---| +| **Persona** | 288 (8 × 36) | `PersonaId: (ExternalRole, ThinkingStyle)` | Identity type, AriGraph keyed | +| **Style** | 36 ThinkingStyles | `ThinkingStyle` enum + `FieldModulation(7D)` | Already in contract | +| **Stage** | 2 (rationale/answer) | `rationale_phase: bool` in `CognitiveEventRow` | MM-CoT stage split = FacultyDescriptor.inbound_style/outbound_style asymmetry | +| **Learned-dynamics** | continuous | `style_oracle: &OnnxPersonaClassifier` | ONNX classifier at L4/L5 | + +Total configurations: 288 × 36 × 2 × (oracle prediction space) ≈ **20,736 × oracle** + +F-descent = automatic architecture search over this space. Misaligned configurations +are dropped by the CollapseGate predicate; no explicit selection needed. + +### ONNX persona classifier (replaces Chronos proposal) + +**Why ONNX over Chronos:** + +| Criterion | Chronos | ONNX classifier | +|---|---|---| +| Output | 1D scalar (style ordinal) | 288 logits (full persona product) | +| Task type | Time-series forecasting | Classification | +| Training | Pre-trained (zero shots) | Trains from Lance E-DEPLOY-1 corpus | +| Precision | Style axis only | Full `(role, style)` product axis | +| Infra | Separate model download | `ort` crate — already justified by Jina v5 | +| Fit | Partial fit (style only) | Full fit (role × thinking = persona) | + +**Architecture:** +``` +Input: + recent_fingerprints: Tensor[N, 16384] // N cycle fingerprints [u64;256] as f32 bits (L4/L5 speed tier) + current_meta: Tensor[4] // MetaWord fields unpacked + +Hidden: + Linear(N×16384 → 512) + ReLU + Linear(512 → 288) + +Output: + logits: Tensor[288] // log-softmax over all (ExternalRole, ThinkingStyle) pairs + → argmax → PersonaId { role, style } +``` + +**Integration into Think struct:** +```rust +struct Think { + trajectory: Vsa10k, + awareness: ParamTruths, + free_energy: FreeEnergy, + resolution: Resolution, + episodic: &EpisodicMemory, + graph: &TripletGraph, + global_context: &Vsa10k, + codec: &CamPqCodec, + // ── § 17 addition ── + style_oracle: Option<&OnnxPersonaClassifier>, // None = StyleSelector::Auto fallback +} +``` + +`StyleSelector::Auto` remains the fallback when no oracle is loaded (cold start, +no training corpus yet). ONNX oracle augments, does not replace, the static +qualia→style rule. + +**Training pipeline:** +1. Lance internal_cold dataset accumulates: `{ fingerprint[u64;256], meta: MetaWord, f_outcome: f32, role: u8, style: u8 }` +2. Rows labeled by F: `f_outcome < 0.2` → committed persona, `f_outcome > 0.8` → failed persona +3. Export to ONNX-format Parquet tensors +4. Train small classifier (< 2MB ONNX, fits in process memory) +5. Hot-reload via `ort::Session::new()` at version boundary + +**Layer placement:** L4/L5 (internal, before CollapseGate). Never exposed externally. +ONNX model file is internal asset, not an external API concern. + +### MM-CoT stage split (zero new architecture) + +The two-stage CoT structure (rationale → answer) from MM-CoT maps exactly to +`FacultyDescriptor.inbound_style` / `outbound_style` asymmetry: + +``` +inbound_style = ThinkingStyle for rationale generation (Stage 1, "thinking") +outbound_style = ThinkingStyle for answer emission (Stage 2, "answer") +is_asymmetric() returns true iff these differ — exactly the MM-CoT condition +``` + +`CognitiveEventRow` gains one column: +```rust +pub rationale_phase: bool, // true = Stage 1 (rationale), false = Stage 2 (answer) +``` + +No new trait, no new struct. The stage is already intrinsic to `FacultyDescriptor`; +`rationale_phase` surfaces it in the projected scalar row for external subscribers. + +### PersonaHub compression summary + +``` +370M personas → extract: + atom_bitset: u32 (which of 32 named atoms are active) + palette_weight: u8 (16 weight levels, coarse encoding) + template_id: u16 (which YAML runbook handles this signature) + = 56 bits = 7 bytes per persona + +Offline extraction: + For each PersonaHub row → parse YAML → map to atom presence → pack 56-bit signature + Output: 2.6 GB flat binary + 10K YAML templates (~2 MB) + Hash-deduplicate → ~1–5M unique signatures in practice (PersonaHub has high redundancy) + +Runtime lookup: + PersonaId → signature (56-bit lookup table, negligible memory) + signature.template_id → JIT compile YAML runbook → KernelHandle + PersonaId + KernelHandle → cognitive configuration for this turn +``` + +--- + +## § 18 — VSA Precision Tiers + Generational Compression (Father-Grandfather) + +**Trigger:** 2026-04-23 — clarification that L4/L5 is the speed lane and must stay +at fingerprint resolution; L3 is where precision lives; VSA is the wire format. + +### Three precision tiers + +| Tier | Format | Size | Properties | Where | +|---|---|---|---|---| +| **Fingerprint** | `[u64;256]` = `Fingerprint<256>` | 2 KB (16 Kbit) | Hash-quantized. One-way (no unbind). SIMD-fast Hamming. | L4/L5 substrate (30 ns/bind) | +| **Vsa10k BF16** | `[bf16; 10000]` | 20 KB | Effectively lossless (Jirak noise floor not crossed). Supports `unbind(role)`. | L3 memory / cold storage | +| **Vsa10k f32** | `[f32; 10000]` | 40 KB | Fully lossless. Full bind/unbind algebra. | Offline training / precision UDFs | + +**L4/L5 is the speed lane (motion, learning, fast dispatch).** The 16Kbit +fingerprint is the right format there — shrinking it would leave the speed +lane; inflating it to Vsa10k would blow the L3 memory budget at 30 ns/bind. +This boundary is a hardware-budget invariant, not a design choice. + +**L3 is where precision is affordable.** The callcenter operates at human-turn +rate (seconds between commits), not substrate rate. Full Vsa10k BF16 at L3 +costs 20 KB/row — trivial at that cadence. Alternatively: **RaBitQ**-quantized +columns in Lance provide zero-copy ANN search at < Vsa10k RAM cost while +preserving approximate unbind accuracy. + +### VSA as the wire format + +VSA IS the wire format — the medium through which cognitive content travels +between layers: + +``` +Wire format: bundle(SPO triples) → trajectory [Vsa10k or Fingerprint<256>] + Markov ±5 window → bundle last 5 trajectories → episodic context + Markov ±500 window → bundle last 500 cycles → long-range context + SPO role bind → bind(role_key, payload) → superposed into trajectory +``` + +`CognitiveEventRow` is NOT the wire format — it is the BBB-safe SCALAR +projection of the wire state. The wire (VSA trajectory) stays inside. The +projection (Arrow scalars) crosses the gate. + +### Father-Grandfather generational compression + +**Motivation:** Per-cycle fingerprints accumulate at 2 KB/cycle. 10K cycles += 20 MB (manageable in hot tier). 10M cycles = 20 GB (cold storage, still +feasible). But for full-precision Vsa10k cold storage at 20 KB/cycle, +generational compression prevents unbounded RAM growth during replay. + +**Hierarchy:** + +``` +"Son" (hot tier, L4/L5): + Per-cycle Fingerprint<256> rows — 2 KB each — fast, hash-quantized + Lance version-tagged; last M rows in hot dataset + +"Father" (~100-cycle bundle, L3): + MergeMode::Bundle over last ~100 per-cycle fingerprints + → single Vsa10k BF16 (20 KB) representing 100 cycles + → CK-safe (I-SUBSTRATE-MARKOV); saturating bundle preserves Markov property + → 100 × 2 KB → 20 KB: 10:1 compression; unbind survives at full L3 precision + +"Grandfather" (~1000-cycle bundle, cold tier): + MergeMode::Bundle over last ~1000 cycles (or over 10 Father vectors) + → single Vsa10k BF16 (20 KB) representing 1000 cycles + → 1000 × 2 KB → 20 KB: 100:1 compression; Markov property preserved + → unbind(role, grandfather) = approximate role trajectory over 1000 turns +``` + +**Relationship to existing Markov windows:** +- ±5 window → per-cycle fingerprint braid (existing, L4/L5) +- ±500 window → episodic memory (existing, L3) +- ±1000 window → grandfather bundle (new, L3/cold storage) + +**CK safety proof (informal):** `MergeMode::Bundle` is commutative and +associative in expectation (Johnson-Lindenstrauss + concentration-of-measure +at rate ~e^(−d), I-SUBSTRATE-MARKOV). Bundling 100 fingerprints into one +Vsa10k is a saturating bundle; the Chapman-Kolmogorov property holds for the +bundle as a whole — the state transition from "Son" to "Father" is a valid +Markov step. + +**Implementation note:** deferred. Current hot dataset stores fingerprints +only. Father/Grandfather columns are Phase B additions when cold dataset +accumulates > 10K cycles. No schema change to `CognitiveEventRow` required — +generational bundles live in a separate Lance dataset (compression tier). diff --git a/.claude/plans/unified-integration-v1.md b/.claude/plans/unified-integration-v1.md new file mode 100644 index 00000000..dfff4ab4 --- /dev/null +++ b/.claude/plans/unified-integration-v1.md @@ -0,0 +1,308 @@ +# Plan — Unified Integration: PersonaHub × ONNX × Archetype × MM-CoT × RoleDB + +> **Version:** v1 +> **Author:** main-thread session 2026-04-23 +> **Status:** Active — brainstorm phase complete; deliverables defined; no code shipped yet +> **Confidence:** CONJECTURE — all integration points grounded in repo evidence and upstream docs; +> no implementations shipped yet beyond DM-0 / DM-1 skeleton in callcenter-membrane-v1.md. +> +> **READ BY:** Any session touching persona modeling, thinking style selection, the ONNX oracle, +> Archetype ECS integration, MM-CoT stage split, or the RoleDB DataFusion UDFs. +> +> **Related plan:** `.claude/plans/callcenter-membrane-v1.md` §§ 15–17 (architecture ground truth) + +--- + +## § 0 — One-Sentence Goal + +Integrate four upstream systems (PersonaHub, ONNX classifier, Archetype ECS, MM-CoT) +into the lance-graph cognitive substrate WITHOUT adding new architectural layers — +each maps onto existing contract types with zero new traits. + +--- + +## § 1 — The Integration Surface Map + +``` +PersonaHub (Tencent) ──► PersonaSignature (56-bit) ──► template_id ──► YAML runbook +Chronos (Amazon) ────► SUPERSEDED by ONNX classifier (see DU-1) +ONNX classifier ─────► OnnxPersonaClassifier ──► style_oracle in Think struct +Archetype (VangelisTech) ──► sits ABOVE callcenter ──► Entity=PersonaCard, World=Blackboard +MM-CoT (Amazon) ─────► rationale_phase: bool in CognitiveEventRow (zero new types) +RoleDB ──────────────► DataFusion VSA UDFs (5 functions) ──► internal_dataset queryable +``` + +--- + +## § 2 — Deliverables + +### DU-0 — PersonaHub Compression (offline extraction) + +**Scope:** Compress PersonaHub 370M personas to 56-bit signatures. + +**Output artifacts:** +- Flat binary: `personas.bin` — 370M × 7 bytes ≈ 2.6 GB +- Deduplicated signature table: `sigs_dedup.bin` — ~1–5M unique signatures × 7 bytes +- YAML template library: `templates/*.yaml` — ~10K runbook templates +- Index: `sig_to_template.u16.bin` — `signature → template_id` lookup + +**Algorithm:** +1. Stream PersonaHub HF parquet (`tencent-ailab/persona-hub`, ~3 parquet shards) +2. For each persona JSON: parse `system_prompt` → detect active atoms (32-bit bitset) +3. Score atom activation intensity (0–15 per atom) → `palette_weight: u8` coarse encoding +4. Cluster similar signatures → assign `template_id` from nearest YAML template centroid +5. Pack `(atom_bitset: u32, palette_weight: u8, template_id: u16)` → 7-byte record + +**Atom detection** — NLP heuristics (no LLM needed): +- `deduction`: "therefore", "it follows", "conclude" +- `counterfactual`: "if instead", "had been", "would have" +- `induction`: "in general", "pattern", "typically" +- … (32 keyword/phrase patterns, one per atom) + +**ONNX connection:** deduplicated signatures + F-labeled Lance rows = training corpus for DU-1. + +**Status:** Queued +**Effort:** ~2 days (streaming extraction; no training) +**Owner:** offline tool; can run on HF dataset in Colab or local machine + +--- + +### DU-1 — ONNX Persona Classifier @ L4/L5 + +**Scope:** Replace `StyleSelector::Auto` (static qualia→style rule) with a learned +ONNX classifier that predicts the full `(ExternalRole, ThinkingStyle)` product. + +**Why ONNX over Chronos:** +- Output: 288 logits (full persona product) vs Chronos 1D scalar (style only) +- Task: classification vs time-series forecasting +- Training: from Lance E-DEPLOY-1 corpus (already labeled by F outcome) +- Infra: `ort` crate justified by existing Jina v5 ONNX on disk +- Precision: full `role × thinking` product vs style axis only + +**Input/Output:** +``` +Input: + recent_fingerprints: Tensor[N, 16384] // N cycle fingerprints [u64;256] as f32 bits + // L4/L5 speed-lane format — fingerprint, NOT Vsa10k + // (L4/L5 is motion/learning/fast dispatch; stays at 2KB/row) + current_meta: Tensor[4] // MetaWord: thinking, awareness, nars_f, nars_c unpacked + +Output: + logits: Tensor[288] // log-softmax over (ExternalRole × ThinkingStyle) pairs + argmax → PersonaId { role: ExternalRole, style: ThinkingStyle } +``` + +**Training:** +``` +Source: Lance internal_cold dataset +Schema: { fingerprint: [u64;256], meta: MetaWord, f_outcome: f32, role: u8, style: u8 } +Labels: f_outcome < 0.2 → positive (committed persona = correct prediction) + f_outcome > 0.8 → negative (failed cycle = wrong persona for context) +Min corpus: ~10K labeled cycles before training is meaningful +Target model size: < 2MB ONNX (fits in-process, hot-reloadable) +``` + +**Integration:** `Think.style_oracle: Option<&OnnxPersonaClassifier>` +- `None` → falls back to `StyleSelector::Auto` (existing static rule) +- `Some(oracle)` → oracle prediction replaces static rule; `StyleSelector::Auto` still runs + as confidence calibration baseline + +**Layer:** L4/L5 (internal). Never exposed externally. ONNX file is internal asset. + +**Status:** Queued +**Effort:** ~3 days (training script + ort integration + Think struct addition) +**Blocking:** needs ~10K labeled cycles from Lance internal_cold (DM-2 must ship first) + +--- + +### DU-2 — Archetype ECS Integration Layer + +**Upstream:** `https://github.com/VangelisTech/archetype` — Rust ECS simulation engine, +DataFrame-first, LanceDB-backed, tick-based. + +**Mapping:** + +| Archetype concept | Callcenter / lance-graph concept | Notes | +|---|---|---| +| `Entity` | `PersonaCard = (ExternalRole, ThinkingStyle)` | One entity per canonical persona | +| `Component` | `FacultyDescriptor` field | Each component = one faculty axis | +| `Processor` | `OrchestrationBridge::route()` | Processes steps each tick | +| `World` | `Blackboard` | Shared state across all entities per round | +| `Tick` | Blackboard round + CollapseGate fire | One tick = one committed turn | +| `System` | `FacultyDescriptor.tools: &[ToolAbility]` | Systems declared per entity type | +| `DataFrame` | Arrow RecordBatch from `project()` | Archetype is DataFrame-first | +| `LanceDB` | Lance external_dataset | Archetype persists to Lance natively | + +**Position in stack:** + +``` +q2 (GUI: Gotham/Neo4j/Quarto) ──► Archetype ECS (simulation layer) + │ + ▼ + callcenter (headless wire) + │ + ▼ + lance-graph cognitive substrate +``` + +Archetype sits ABOVE the callcenter. It is the simulation orchestrator that +drives tick cycles. The callcenter remains headless wire — it does not know +about Archetype. Archetype drives it by calling `ExternalMembrane::ingest()` +per tick. + +**Why not merge Archetype into lance-graph:** +- Archetype is ECS simulation; callcenter is cognitive wire +- Archetype belongs in q2's simulation layer; callcenter belongs at wire level +- Keeping them separate preserves the q2=GUI / callcenter=headless boundary + +**Integration deliverable:** thin adapter crate `lance-graph-archetype-bridge` +- `ArchetypeWorld → Blackboard` adapter (World snapshot → Blackboard round) +- `ArchetypeTick → UnifiedStep` adapter (tick event → step for OrchestrationBridge) +- `project() result → Archetype DataFrame component update` + +**Status:** Queued +**Effort:** ~3 days (adapter crate; no changes to callcenter or lance-graph core) + +--- + +### DU-3 — RoleDB DataFusion VSA UDFs + +**Scope:** Register 5 VSA UDFs in DataFusion so the internal dataset is queryable +as a "DuckDB over roles." + +**UDF registry:** + +```rust +// In lance-graph core or lance-graph-callcenter: +fn register_vsa_udfs(ctx: &SessionContext) { + ctx.register_udf(unbind_udf()); // (u8, [u64;256]) → f32 + ctx.register_udf(bundle_udf()); // ([u64;256]...) → [u64;256] + ctx.register_udf(hamming_dist_udf()); // ([u64;256], [u64;256]) → u32 + ctx.register_udf(braid_at_udf()); // (i32, [u64;256]) → [u64;256] + ctx.register_udf(top_k_udf()); // ([u64;256], u32) → list[u16] +} +``` + +**Example query (dispatch scoring):** +```sql +SELECT expert_id, role, style, + unbind(target_role, fingerprint) AS dispatch_score +FROM internal_dataset +WHERE round >= (SELECT max(round) - 5 FROM internal_dataset) -- ±5 Markov window +ORDER BY dispatch_score DESC +LIMIT 10; +``` + +**Precision note:** The UDF signatures above use `[u64;256]` (fingerprint, L4/L5 tier) +for dispatch scoring — fast Hamming distance, approximate role overlap. This is correct +for the dispatch path (30 ns/bind speed lane). For deep role recovery via `unbind()` +at full precision (RoleDB Phase B), the UDF input should be Vsa10k BF16 from the L3 +cold dataset — deferred until the L3 Vsa10k or RaBitQ cold columns exist. See §18 +of `callcenter-membrane-v1.md` for the precision tier architecture. + +**BBB compliance:** UDFs operate on internal_dataset only. Results are scalar +(f32 dispatch scores, u32 Hamming distances). VSA types never cross to external_dataset. + +**Status:** Queued +**Effort:** ~2 days (DataFusion ScalarUDF boilerplate; VSA ops already exist in contract) + +--- + +### DU-4 — MM-CoT Stage Split (minimal, zero new types) + +**Upstream:** `https://github.com/amazon-science/mm-cot` — two-stage CoT +(rationale generation → answer generation) with optional visual features. + +**Mapping to existing architecture:** +- MM-CoT Stage 1 (rationale) = `FacultyDescriptor.inbound_style` (thinking mode) +- MM-CoT Stage 2 (answer) = `FacultyDescriptor.outbound_style` (emission mode) +- `FacultyDescriptor.is_asymmetric()` returns `true` iff stages differ = MM-CoT condition +- Visual features = future `FacultyRole::Vision` variant (not in scope for DU-4 v1) + +**Code change:** one field added to `CognitiveEventRow`: +```rust +pub rationale_phase: bool, // true = Stage 1 rationale, false = Stage 2 answer +``` + +This field surfaces in the projected RecordBatch so external subscribers can filter +to either stage. No new trait, no new struct. + +**Status:** Shipped (2026-04-23) — `rationale_phase: bool` added to `CognitiveEventRow` +in `external_intent.rs`; `project()` in `lance_membrane.rs` populates `rationale_phase: false` +(Phase A stub). Commit `a05979e`. + +--- + +### DU-5 — Board Hygiene + STATUS_BOARD Update + +**Scope:** Register DU-0 through DU-4 in STATUS_BOARD.md, update INTEGRATION_PLANS.md, +update LATEST_STATE.md contract inventory with new plan sections. + +**Status:** Shipped (2026-04-23) — `unified-integration-v1` entry prepended to +`INTEGRATION_PLANS.md`; DU-* rows appended to `STATUS_BOARD.md`. Commit `a05979e`. + +--- + +## § 3 — Sequencing + +``` +DM-2 (LanceMembrane impl) ──► DU-1 needs labeled corpus (Lance internal_cold) +DU-0 (PersonaHub compress) → can run in parallel with DM-2 (offline, no deps) +DU-3 (RoleDB UDFs) → can run in parallel with DM-2 (DataFusion, no deps) +DU-4 (MM-CoT field) → can run immediately (trivial, no deps) +DU-5 (Board hygiene) → run last + +DU-1 (ONNX oracle) ──► needs DM-2 (corpus) + DU-0 (atom vocabulary for feature alignment) +DU-2 (Archetype) ──► needs DM-2 (ExternalMembrane impl to adapt against) +``` + +**Recommended order:** +1. DU-4 (trivial, unlock immediately) +2. DU-3 (UDFs, unblocks RoleDB queries) +3. DU-0 (offline extraction, runs async) +4. DM-2 (LanceMembrane impl — unlocks DU-1, DU-2) +5. DU-1 (ONNX oracle, depends on DM-2 corpus) +6. DU-2 (Archetype bridge, depends on DM-2) +7. DU-5 (board hygiene, last) + +--- + +## § 4 — Invariants (must not be violated) + +1. **BBB invariant:** `DU-1` ONNX oracle output is `PersonaId { role, style }` — scalar only. + The ONNX model lives internal. No VSA tensor crosses to external_dataset. + +2. **I-SUBSTRATE-MARKOV:** `bundle_udf` in DU-3 MUST use `MergeMode::Bundle`, not `MergeMode::Xor`. + XOR breaks the CK guarantee for state-transition paths. + +3. **q2=GUI / callcenter=headless:** DU-2 Archetype bridge does NOT put ECS logic inside + `lance-graph-callcenter`. The bridge is a thin adapter crate that Archetype calls INTO + the callcenter. Callcenter has zero knowledge of Archetype. + +4. **Jirak-derived thresholds (I-NOISE-FLOOR-JIRAK):** `unbind_udf` dispatch scores are + probabilistic, not exact. Any significance threshold for dispatch scoring must cite + Jirak 2016, not classical Berry-Esseen. + +5. **`role × thinking = persona` is the identity:** PersonaCard does not gain new fields + beyond `(role: ExternalRole, style: ThinkingStyle)`. PersonaSignature (56-bit) is a + compressed fingerprint, not the identity. + +--- + +## § 5 — Open Questions + +- **DU-0 atom vocabulary stability:** The 32 named atoms are provisional. Do they need + to be aligned with any external taxonomy (e.g., FrameNet, ACT-R cognitive operators)? + Current stance: internal vocabulary, evolves with F-descent evidence. + +- **DU-1 minimum corpus size:** 10K labeled cycles is a rough estimate. Actual minimum + depends on class balance across 288 persona classes. Some personas may be rare in + practice (e.g., `(OpenClaw, Koan)`) — rare class handling may be needed. + +- **DU-2 Archetype API stability:** `VangelisTech/archetype` is an active repo. Adapter + crate should define its OWN interface (not directly call Archetype internals) so that + Archetype version changes don't break the bridge. + +- **DU-3 fingerprint column type in DataFusion:** `[u64; 256]` is not a native Arrow type. + Representation options: `FixedSizeBinary(2048)`, `FixedSizeList(256)`, or custom + extension type. Choose before writing UDFs — type determines UDF signature. diff --git a/.claude/settings.json b/.claude/settings.json index a8221e4b..9be403fa 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -1,24 +1,16 @@ { "$schema": "https://json.schemastore.org/claude-code-settings.json", "permissions": { + "allow": [ + "Edit(**/*.md)", + "Write(**/*.md)" + ], "ask": [], "deny": [ - "Edit(.claude/board/PR_ARC_INVENTORY.md)", - "Edit(.claude/board/LATEST_STATE.md)", - "Edit(.claude/board/STATUS_BOARD.md)", - "Edit(.claude/board/INTEGRATION_PLANS.md)", - "Edit(.claude/board/EPIPHANIES.md)", - "Edit(.claude/board/ISSUES.md)", - "Edit(.claude/board/IDEAS.md)", - "Edit(.claude/board/TECH_DEBT.md)", - "Write(.claude/board/PR_ARC_INVENTORY.md)", - "Write(.claude/board/LATEST_STATE.md)", - "Write(.claude/board/STATUS_BOARD.md)", - "Write(.claude/board/INTEGRATION_PLANS.md)", - "Write(.claude/board/EPIPHANIES.md)", - "Write(.claude/board/ISSUES.md)", - "Write(.claude/board/IDEAS.md)", - "Write(.claude/board/TECH_DEBT.md)", + "Edit(CLAUDE.md)", + "Edit(**/CLAUDE.md)", + "Write(CLAUDE.md)", + "Write(**/CLAUDE.md)", "Bash(git push --force:*)", "Bash(git push -f:*)", "Bash(git push --force-with-lease:*)", diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 10861406..4d6731af 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -24,17 +24,22 @@ jobs: linux-build: runs-on: ubuntu-24.04 timeout-minutes: 30 + defaults: + run: + working-directory: lance-graph strategy: matrix: toolchain: - stable steps: - uses: actions/checkout@v4 + with: + path: lance-graph - name: Checkout AdaWorldAPI/ndarray (sibling dependency) uses: actions/checkout@v4 with: repository: AdaWorldAPI/ndarray - path: ../ndarray + path: ndarray - name: Setup rust toolchain run: | rustup toolchain install ${{ matrix.toolchain }} @@ -43,16 +48,13 @@ jobs: with: shared-key: "lance-graph-deps" workspaces: | - crates/lance-graph - crates/lance-graph-python + lance-graph/crates/lance-graph - name: Install dependencies run: | sudo apt update sudo apt install -y protobuf-compiler - name: Build lance-graph run: cargo build --manifest-path crates/lance-graph/Cargo.toml - - name: Build lance-graph-python - run: cargo check --manifest-path crates/lance-graph-python/Cargo.toml - name: Build tests run: cargo test --manifest-path crates/lance-graph/Cargo.toml --no-run - name: Run tests diff --git a/.github/workflows/python-publish.yml b/.github/workflows/python-publish.yml deleted file mode 100644 index d91bb058..00000000 --- a/.github/workflows/python-publish.yml +++ /dev/null @@ -1,76 +0,0 @@ -name: Python Publish - -on: - release: - types: - - released - pull_request: - paths: - - .github/workflows/python-publish.yml - workflow_dispatch: - inputs: - mode: - description: "dry_run: build & test only, release: build & publish to PyPI" - required: true - default: dry_run - type: choice - options: - - dry_run - - release - -jobs: - publish: - runs-on: ubuntu-latest - permissions: - id-token: write - contents: read - env: - CARGO_TERM_COLOR: always - RUSTFLAGS: "-C debuginfo=1" - RUST_BACKTRACE: "1" - CARGO_INCREMENTAL: "0" - steps: - - name: Checkout code - uses: actions/checkout@v4 - - - name: Set up Python - uses: actions/setup-python@v5 - with: - python-version: "3.11" - - - name: Install Rust toolchain - uses: actions-rust-lang/setup-rust-toolchain@v1 - with: - toolchain: stable - cache: false - - - name: Install system dependencies - run: | - sudo apt update - sudo apt install -y protobuf-compiler libssl-dev - - - name: Install uv - uses: astral-sh/setup-uv@v4 - - - name: Install maturin - run: | - pip install maturin - - - name: Build distributions - working-directory: python - run: | - make publish - - - name: Upload distributions - uses: actions/upload-artifact@v4 - with: - name: python-dist - path: python/dist - - - name: Publish to PyPI - if: > - (github.event_name == 'release' && github.event.action == 'released') || - (github.event_name == 'workflow_dispatch' && github.event.inputs.mode == 'release') - working-directory: python - run: | - uv publish --trusted-publishing always diff --git a/.github/workflows/python-test.yml b/.github/workflows/python-test.yml deleted file mode 100644 index ee5c92d8..00000000 --- a/.github/workflows/python-test.yml +++ /dev/null @@ -1,108 +0,0 @@ -name: Python Tests -on: - push: - branches: - - main - pull_request: - paths: - - python/** - - crates/lance-graph/** - - Cargo.toml - - Cargo.lock - - .github/workflows/python-test.yml - -concurrency: - group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: true - -env: - CARGO_TERM_COLOR: always - RUSTFLAGS: "-C debuginfo=1" - CARGO_INCREMENTAL: "0" - -jobs: - test: - runs-on: ubuntu-24.04 - timeout-minutes: 30 - strategy: - matrix: - python-version: ["3.11"] - steps: - - uses: actions/checkout@v4 - - name: Set up Python ${{ matrix.python-version }} - uses: actions/setup-python@v4 - with: - python-version: ${{ matrix.python-version }} - - name: Setup Rust - uses: actions-rust-lang/setup-rust-toolchain@v1 - - uses: Swatinem/rust-cache@v2 - with: - shared-key: "lance-graph-deps" - workspaces: | - crates/lance-graph - crates/lance-graph-python - - name: Install dependencies - run: | - sudo apt update - sudo apt install -y protobuf-compiler - - name: Install uv - uses: astral-sh/setup-uv@v3 - - name: Create virtual environment and install dependencies - working-directory: python - run: | - uv venv - source .venv/bin/activate - uv pip install maturin[patchelf] - # Install test dependencies only (skip editable install to avoid double-building Rust) - uv pip install pytest pyarrow pandas ruff - - name: Build Python extension - working-directory: python - run: | - source .venv/bin/activate - maturin develop - - name: Run tests - working-directory: python - run: | - source .venv/bin/activate - pytest python/tests/ -v - - name: Run doctests - working-directory: python - run: | - source .venv/bin/activate - if [ -f python/lance_graph/__init__.py ]; then - python -m doctest python/lance_graph/__init__.py || echo "No doctests found" - fi - - lint: - runs-on: ubuntu-24.04 - timeout-minutes: 15 - steps: - - uses: actions/checkout@v4 - - name: Set up Python - uses: actions/setup-python@v4 - with: - python-version: "3.11" - - name: Install uv - uses: astral-sh/setup-uv@v3 - - name: Install linting tools - working-directory: python - run: | - uv venv - source .venv/bin/activate - # Install only linting tools without building the Rust extension - uv pip install ruff pyright - - name: Run ruff format check - working-directory: python - run: | - source .venv/bin/activate - ruff format --check python/ - - name: Run ruff lint - working-directory: python - run: | - source .venv/bin/activate - ruff check python/ - - name: Run pyright type check - working-directory: python - run: | - source .venv/bin/activate - pyright diff --git a/.github/workflows/rust-test.yml b/.github/workflows/rust-test.yml index c401e3f2..c14c4533 100644 --- a/.github/workflows/rust-test.yml +++ b/.github/workflows/rust-test.yml @@ -24,17 +24,22 @@ jobs: test: runs-on: ubuntu-24.04 timeout-minutes: 30 + defaults: + run: + working-directory: lance-graph strategy: matrix: toolchain: - stable steps: - uses: actions/checkout@v4 + with: + path: lance-graph - name: Checkout AdaWorldAPI/ndarray (sibling dependency) uses: actions/checkout@v4 with: repository: AdaWorldAPI/ndarray - path: ../ndarray + path: ndarray - name: Setup rust toolchain run: | rustup toolchain install ${{ matrix.toolchain }} @@ -43,8 +48,7 @@ jobs: with: shared-key: "lance-graph-deps" workspaces: | - crates/lance-graph - crates/lance-graph-python + lance-graph/crates/lance-graph - name: Install dependencies run: | sudo apt update @@ -59,13 +63,18 @@ jobs: test-with-coverage: runs-on: ubuntu-24.04 timeout-minutes: 30 + defaults: + run: + working-directory: lance-graph steps: - uses: actions/checkout@v4 + with: + path: lance-graph - name: Checkout AdaWorldAPI/ndarray (sibling dependency) uses: actions/checkout@v4 with: repository: AdaWorldAPI/ndarray - path: ../ndarray + path: ndarray - name: Setup rust toolchain run: | rustup toolchain install stable @@ -74,8 +83,7 @@ jobs: with: shared-key: "lance-graph-deps" workspaces: | - crates/lance-graph - crates/lance-graph-python + lance-graph/crates/lance-graph - name: Install dependencies run: | sudo apt update diff --git a/Cargo.lock b/Cargo.lock index 447a0f28..343d007b 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -126,61 +126,25 @@ version = "0.7.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50" -[[package]] -name = "arrow" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6e833808ff2d94ed40d9379848a950d995043c7fb3e81a30b383f4c6033821cc" -dependencies = [ - "arrow-arith 56.2.0", - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-cast 56.2.0", - "arrow-csv 56.2.0", - "arrow-data 56.2.0", - "arrow-ipc 56.2.0", - "arrow-json 56.2.0", - "arrow-ord 56.2.0", - "arrow-row 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "arrow-string 56.2.0", -] - [[package]] name = "arrow" version = "57.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e4754a624e5ae42081f464514be454b39711daae0458906dacde5f4c632f33a8" dependencies = [ - "arrow-arith 57.3.0", - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-cast 57.3.0", - "arrow-csv 57.3.0", - "arrow-data 57.3.0", - "arrow-ipc 57.3.0", - "arrow-json 57.3.0", - "arrow-ord 57.3.0", - "arrow-pyarrow", - "arrow-row 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", - "arrow-string 57.3.0", -] - -[[package]] -name = "arrow-arith" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ad08897b81588f60ba983e3ca39bda2b179bdd84dced378e7df81a5313802ef8" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "chrono", - "num", + "arrow-arith", + "arrow-array", + "arrow-buffer", + "arrow-cast", + "arrow-csv", + "arrow-data", + "arrow-ipc", + "arrow-json", + "arrow-ord", + "arrow-row", + "arrow-schema", + "arrow-select", + "arrow-string", ] [[package]] @@ -189,31 +153,14 @@ version = "57.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f7b3141e0ec5145a22d8694ea8b6d6f69305971c4fa1c1a13ef0195aef2d678b" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-data", + "arrow-schema", "chrono", "num-traits", ] -[[package]] -name = "arrow-array" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8548ca7c070d8db9ce7aa43f37393e4bfcf3f2d3681df278490772fd1673d08d" -dependencies = [ - "ahash", - "arrow-buffer 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "chrono", - "chrono-tz", - "half", - "hashbrown 0.16.0", - "num", -] - [[package]] name = "arrow-array" version = "57.3.0" @@ -221,9 +168,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4c8955af33b25f3b175ee10af580577280b4bd01f7e823d94c7cdef7cf8c9aef" dependencies = [ "ahash", - "arrow-buffer 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", + "arrow-buffer", + "arrow-data", + "arrow-schema", "chrono", "chrono-tz", "half", @@ -233,17 +180,6 @@ dependencies = [ "num-traits", ] -[[package]] -name = "arrow-buffer" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e003216336f70446457e280807a73899dd822feaf02087d31febca1363e2fccc" -dependencies = [ - "bytes", - "half", - "num", -] - [[package]] name = "arrow-buffer" version = "57.3.0" @@ -256,39 +192,18 @@ dependencies = [ "num-traits", ] -[[package]] -name = "arrow-cast" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "919418a0681298d3a77d1a315f625916cb5678ad0d74b9c60108eb15fd083023" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "atoi", - "base64", - "chrono", - "comfy-table", - "half", - "lexical-core", - "num", - "ryu", -] - [[package]] name = "arrow-cast" version = "57.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "646bbb821e86fd57189c10b4fcdaa941deaf4181924917b0daa92735baa6ada5" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-data 57.3.0", - "arrow-ord 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-data", + "arrow-ord", + "arrow-schema", + "arrow-select", "atoi", "base64", "chrono", @@ -299,126 +214,61 @@ dependencies = [ "ryu", ] -[[package]] -name = "arrow-csv" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bfa9bf02705b5cf762b6f764c65f04ae9082c7cfc4e96e0c33548ee3f67012eb" -dependencies = [ - "arrow-array 56.2.0", - "arrow-cast 56.2.0", - "arrow-schema 56.2.0", - "chrono", - "csv", - "csv-core", - "regex", -] - [[package]] name = "arrow-csv" version = "57.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8da746f4180004e3ce7b83c977daf6394d768332349d3d913998b10a120b790a" dependencies = [ - "arrow-array 57.3.0", - "arrow-cast 57.3.0", - "arrow-schema 57.3.0", + "arrow-array", + "arrow-cast", + "arrow-schema", "chrono", "csv", "csv-core", "regex", ] -[[package]] -name = "arrow-data" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a5c64fff1d142f833d78897a772f2e5b55b36cb3e6320376f0961ab0db7bd6d0" -dependencies = [ - "arrow-buffer 56.2.0", - "arrow-schema 56.2.0", - "half", - "num", -] - [[package]] name = "arrow-data" version = "57.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1fdd994a9d28e6365aa78e15da3f3950c0fdcea6b963a12fa1c391afb637b304" dependencies = [ - "arrow-buffer 57.3.0", - "arrow-schema 57.3.0", + "arrow-buffer", + "arrow-schema", "half", "num-integer", "num-traits", ] -[[package]] -name = "arrow-ipc" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1d3594dcddccc7f20fd069bc8e9828ce37220372680ff638c5e00dea427d88f5" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "flatbuffers", - "lz4_flex 0.11.5", - "zstd", -] - [[package]] name = "arrow-ipc" version = "57.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "abf7df950701ab528bf7c0cf7eeadc0445d03ef5d6ffc151eaae6b38a58feff1" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-data", + "arrow-schema", + "arrow-select", "flatbuffers", "lz4_flex 0.12.0", "zstd", ] -[[package]] -name = "arrow-json" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "88cf36502b64a127dc659e3b305f1d993a544eab0d48cce704424e62074dc04b" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-cast 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "chrono", - "half", - "indexmap", - "lexical-core", - "memchr", - "num", - "serde", - "serde_json", - "simdutf8", -] - [[package]] name = "arrow-json" version = "57.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0ff8357658bedc49792b13e2e862b80df908171275f8e6e075c460da5ee4bf86" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-cast 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-cast", + "arrow-data", + "arrow-schema", "chrono", "half", "indexmap", @@ -432,55 +282,17 @@ dependencies = [ "simdutf8", ] -[[package]] -name = "arrow-ord" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3c8f82583eb4f8d84d4ee55fd1cb306720cddead7596edce95b50ee418edf66f" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", -] - [[package]] name = "arrow-ord" version = "57.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f7d8f1870e03d4cbed632959498bcc84083b5a24bded52905ae1695bd29da45b" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", -] - -[[package]] -name = "arrow-pyarrow" -version = "57.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d18c442b4c266aaf3d7f7dd40fd7ae058cef7f113b00ff0cd8256e1e218ec544" -dependencies = [ - "arrow-array 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", - "pyo3", -] - -[[package]] -name = "arrow-row" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9d07ba24522229d9085031df6b94605e0f4b26e099fb7cdeec37abd941a73753" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "half", + "arrow-array", + "arrow-buffer", + "arrow-data", + "arrow-schema", + "arrow-select", ] [[package]] @@ -489,24 +301,13 @@ version = "57.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "18228633bad92bff92a95746bbeb16e5fc318e8382b75619dec26db79e4de4c0" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-data", + "arrow-schema", "half", ] -[[package]] -name = "arrow-schema" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b3aa9e59c611ebc291c28582077ef25c97f1975383f1479b12f3b9ffee2ffabe" -dependencies = [ - "bitflags", - "serde", - "serde_json", -] - [[package]] name = "arrow-schema" version = "57.3.0" @@ -519,20 +320,6 @@ dependencies = [ "serde_json", ] -[[package]] -name = "arrow-select" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8c41dbbd1e97bfcaee4fcb30e29105fb2c75e4d82ae4de70b792a5d3f66b2e7a" -dependencies = [ - "ahash", - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "num", -] - [[package]] name = "arrow-select" version = "57.3.0" @@ -540,41 +327,24 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "68bf3e3efbd1278f770d67e5dc410257300b161b93baedb3aae836144edcaf4b" dependencies = [ "ahash", - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-data", + "arrow-schema", "num-traits", ] -[[package]] -name = "arrow-string" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "53f5183c150fbc619eede22b861ea7c0eebed8eaac0333eaa7f6da5205fd504d" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "memchr", - "num", - "regex", - "regex-syntax", -] - [[package]] name = "arrow-string" version = "57.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "85e968097061b3c0e9fe3079cf2e703e487890700546b5b0647f60fca1b5a8d8" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-data", + "arrow-schema", + "arrow-select", "memchr", "num-traits", "regex", @@ -1064,6 +834,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8b52af3cb4058c895d37317bb27508dccc8e5f2d39454016b297bf4a400597b8" dependencies = [ "axum-core", + "base64", "bytes", "form_urlencoded", "futures-util", @@ -1082,8 +853,10 @@ dependencies = [ "serde_json", "serde_path_to_error", "serde_urlencoded", + "sha1", "sync_wrapper", "tokio", + "tokio-tungstenite 0.28.0", "tower", "tower-layer", "tower-service", @@ -1918,59 +1691,10 @@ dependencies = [ ] [[package]] -name = "datafusion" -version = "50.3.0" +name = "data-encoding" +version = "2.10.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2af15bb3c6ffa33011ef579f6b0bcbe7c26584688bd6c994f548e44df67f011a" -dependencies = [ - "arrow 56.2.0", - "arrow-ipc 56.2.0", - "arrow-schema 56.2.0", - "async-trait", - "bytes", - "bzip2 0.6.1", - "chrono", - "datafusion-catalog 50.3.0", - "datafusion-catalog-listing 50.3.0", - "datafusion-common 50.3.0", - "datafusion-common-runtime 50.3.0", - "datafusion-datasource 50.3.0", - "datafusion-datasource-csv 50.3.0", - "datafusion-datasource-json 50.3.0", - "datafusion-datasource-parquet 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-expr-common 50.3.0", - "datafusion-functions 50.3.0", - "datafusion-functions-aggregate 50.3.0", - "datafusion-functions-nested 50.3.0", - "datafusion-functions-table 50.3.0", - "datafusion-functions-window 50.3.0", - "datafusion-optimizer 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-adapter 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "datafusion-physical-optimizer 50.3.0", - "datafusion-physical-plan 50.3.0", - "datafusion-session 50.3.0", - "datafusion-sql 50.3.0", - "flate2", - "futures", - "itertools 0.14.0", - "log", - "object_store", - "parking_lot", - "parquet 56.2.0", - "rand 0.9.2", - "regex", - "sqlparser 0.58.0", - "tempfile", - "tokio", - "url", - "uuid", - "xz2", - "zstd", -] +checksum = "d7a1e2f27636f116493b8b860f5546edb47c8d8f8ea73e1d2a20be88e28d1fea" [[package]] name = "datafusion" @@ -1978,48 +1702,48 @@ version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8ba7cb113e9c0bedf9e9765926031e132fa05a1b09ba6e93a6d1a4d7044457b8" dependencies = [ - "arrow 57.3.0", - "arrow-schema 57.3.0", + "arrow", + "arrow-schema", "async-trait", "bytes", "bzip2 0.6.1", "chrono", - "datafusion-catalog 51.0.0", - "datafusion-catalog-listing 51.0.0", - "datafusion-common 51.0.0", - "datafusion-common-runtime 51.0.0", - "datafusion-datasource 51.0.0", + "datafusion-catalog", + "datafusion-catalog-listing", + "datafusion-common", + "datafusion-common-runtime", + "datafusion-datasource", "datafusion-datasource-arrow", - "datafusion-datasource-csv 51.0.0", - "datafusion-datasource-json 51.0.0", - "datafusion-datasource-parquet 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-expr-common 51.0.0", - "datafusion-functions 51.0.0", - "datafusion-functions-aggregate 51.0.0", - "datafusion-functions-nested 51.0.0", - "datafusion-functions-table 51.0.0", - "datafusion-functions-window 51.0.0", - "datafusion-optimizer 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-adapter 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "datafusion-physical-optimizer 51.0.0", - "datafusion-physical-plan 51.0.0", - "datafusion-session 51.0.0", - "datafusion-sql 51.0.0", + "datafusion-datasource-csv", + "datafusion-datasource-json", + "datafusion-datasource-parquet", + "datafusion-execution", + "datafusion-expr", + "datafusion-expr-common", + "datafusion-functions", + "datafusion-functions-aggregate", + "datafusion-functions-nested", + "datafusion-functions-table", + "datafusion-functions-window", + "datafusion-optimizer", + "datafusion-physical-expr", + "datafusion-physical-expr-adapter", + "datafusion-physical-expr-common", + "datafusion-physical-optimizer", + "datafusion-physical-plan", + "datafusion-session", + "datafusion-sql", "flate2", "futures", "itertools 0.14.0", "log", "object_store", "parking_lot", - "parquet 57.3.0", + "parquet", "rand 0.9.2", "regex", "rstest", - "sqlparser 0.59.0", + "sqlparser", "tempfile", "tokio", "url", @@ -2028,49 +1752,23 @@ dependencies = [ "zstd", ] -[[package]] -name = "datafusion-catalog" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "187622262ad8f7d16d3be9202b4c1e0116f1c9aa387e5074245538b755261621" -dependencies = [ - "arrow 56.2.0", - "async-trait", - "dashmap", - "datafusion-common 50.3.0", - "datafusion-common-runtime 50.3.0", - "datafusion-datasource 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-plan 50.3.0", - "datafusion-session 50.3.0", - "datafusion-sql 50.3.0", - "futures", - "itertools 0.14.0", - "log", - "object_store", - "parking_lot", - "tokio", -] - [[package]] name = "datafusion-catalog" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "66a3a799f914a59b1ea343906a0486f17061f39509af74e874a866428951130d" dependencies = [ - "arrow 57.3.0", + "arrow", "async-trait", "dashmap", - "datafusion-common 51.0.0", - "datafusion-common-runtime 51.0.0", - "datafusion-datasource 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-plan 51.0.0", - "datafusion-session 51.0.0", + "datafusion-common", + "datafusion-common-runtime", + "datafusion-datasource", + "datafusion-execution", + "datafusion-expr", + "datafusion-physical-expr", + "datafusion-physical-plan", + "datafusion-session", "futures", "itertools 0.14.0", "log", @@ -2079,46 +1777,23 @@ dependencies = [ "tokio", ] -[[package]] -name = "datafusion-catalog-listing" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9657314f0a32efd0382b9a46fdeb2d233273ece64baa68a7c45f5a192daf0f83" -dependencies = [ - "arrow 56.2.0", - "async-trait", - "datafusion-catalog 50.3.0", - "datafusion-common 50.3.0", - "datafusion-datasource 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "datafusion-physical-plan 50.3.0", - "datafusion-session 50.3.0", - "futures", - "log", - "object_store", - "tokio", -] - [[package]] name = "datafusion-catalog-listing" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6db1b113c80d7a0febcd901476a57aef378e717c54517a163ed51417d87621b0" dependencies = [ - "arrow 57.3.0", + "arrow", "async-trait", - "datafusion-catalog 51.0.0", - "datafusion-common 51.0.0", - "datafusion-datasource 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-adapter 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "datafusion-physical-plan 51.0.0", + "datafusion-catalog", + "datafusion-common", + "datafusion-datasource", + "datafusion-execution", + "datafusion-expr", + "datafusion-physical-expr", + "datafusion-physical-expr-adapter", + "datafusion-physical-expr-common", + "datafusion-physical-plan", "futures", "itertools 0.14.0", "log", @@ -2126,31 +1801,6 @@ dependencies = [ "tokio", ] -[[package]] -name = "datafusion-common" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5a83760d9a13122d025fbdb1d5d5aaf93dd9ada5e90ea229add92aa30898b2d1" -dependencies = [ - "ahash", - "arrow 56.2.0", - "arrow-ipc 56.2.0", - "base64", - "chrono", - "half", - "hashbrown 0.14.5", - "indexmap", - "libc", - "log", - "object_store", - "parquet 56.2.0", - "paste", - "recursive", - "sqlparser 0.58.0", - "tokio", - "web-time", -] - [[package]] name = "datafusion-common" version = "51.0.0" @@ -2158,8 +1808,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7c10f7659e96127d25e8366be7c8be4109595d6a2c3eac70421f380a7006a1b0" dependencies = [ "ahash", - "arrow 57.3.0", - "arrow-ipc 57.3.0", + "arrow", + "arrow-ipc", "chrono", "half", "hashbrown 0.14.5", @@ -2167,25 +1817,14 @@ dependencies = [ "libc", "log", "object_store", - "parquet 57.3.0", + "parquet", "paste", "recursive", - "sqlparser 0.59.0", + "sqlparser", "tokio", "web-time", ] -[[package]] -name = "datafusion-common-runtime" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5b6234a6c7173fe5db1c6c35c01a12b2aa0f803a3007feee53483218817f8b1e" -dependencies = [ - "futures", - "log", - "tokio", -] - [[package]] name = "datafusion-common-runtime" version = "51.0.0" @@ -2197,64 +1836,27 @@ dependencies = [ "tokio", ] -[[package]] -name = "datafusion-datasource" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7256c9cb27a78709dd42d0c80f0178494637209cac6e29d5c93edd09b6721b86" -dependencies = [ - "arrow 56.2.0", - "async-compression", - "async-trait", - "bytes", - "bzip2 0.6.1", - "chrono", - "datafusion-common 50.3.0", - "datafusion-common-runtime 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-adapter 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "datafusion-physical-plan 50.3.0", - "datafusion-session 50.3.0", - "flate2", - "futures", - "glob", - "itertools 0.14.0", - "log", - "object_store", - "parquet 56.2.0", - "rand 0.9.2", - "tempfile", - "tokio", - "tokio-util", - "url", - "xz2", - "zstd", -] - [[package]] name = "datafusion-datasource" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fde13794244bc7581cd82f6fff217068ed79cdc344cafe4ab2c3a1c3510b38d6" dependencies = [ - "arrow 57.3.0", + "arrow", "async-compression", "async-trait", "bytes", "bzip2 0.6.1", "chrono", - "datafusion-common 51.0.0", - "datafusion-common-runtime 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-adapter 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "datafusion-physical-plan 51.0.0", - "datafusion-session 51.0.0", + "datafusion-common", + "datafusion-common-runtime", + "datafusion-execution", + "datafusion-expr", + "datafusion-physical-expr", + "datafusion-physical-expr-adapter", + "datafusion-physical-expr-common", + "datafusion-physical-plan", + "datafusion-session", "flate2", "futures", "glob", @@ -2275,114 +1877,64 @@ version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "804fa9b4ecf3157982021770617200ef7c1b2979d57bec9044748314775a9aea" dependencies = [ - "arrow 57.3.0", - "arrow-ipc 57.3.0", + "arrow", + "arrow-ipc", "async-trait", "bytes", - "datafusion-common 51.0.0", - "datafusion-common-runtime 51.0.0", - "datafusion-datasource 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "datafusion-physical-plan 51.0.0", - "datafusion-session 51.0.0", + "datafusion-common", + "datafusion-common-runtime", + "datafusion-datasource", + "datafusion-execution", + "datafusion-expr", + "datafusion-physical-expr-common", + "datafusion-physical-plan", + "datafusion-session", "futures", "itertools 0.14.0", "object_store", "tokio", ] -[[package]] -name = "datafusion-datasource-csv" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "64533a90f78e1684bfb113d200b540f18f268134622d7c96bbebc91354d04825" -dependencies = [ - "arrow 56.2.0", - "async-trait", - "bytes", - "datafusion-catalog 50.3.0", - "datafusion-common 50.3.0", - "datafusion-common-runtime 50.3.0", - "datafusion-datasource 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "datafusion-physical-plan 50.3.0", - "datafusion-session 50.3.0", - "futures", - "object_store", - "regex", - "tokio", -] - [[package]] name = "datafusion-datasource-csv" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "61a1641a40b259bab38131c5e6f48fac0717bedb7dc93690e604142a849e0568" dependencies = [ - "arrow 57.3.0", + "arrow", "async-trait", "bytes", - "datafusion-common 51.0.0", - "datafusion-common-runtime 51.0.0", - "datafusion-datasource 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "datafusion-physical-plan 51.0.0", - "datafusion-session 51.0.0", + "datafusion-common", + "datafusion-common-runtime", + "datafusion-datasource", + "datafusion-execution", + "datafusion-expr", + "datafusion-physical-expr-common", + "datafusion-physical-plan", + "datafusion-session", "futures", "object_store", "regex", "tokio", ] -[[package]] -name = "datafusion-datasource-json" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8d7ebeb12c77df0aacad26f21b0d033aeede423a64b2b352f53048a75bf1d6e6" -dependencies = [ - "arrow 56.2.0", - "async-trait", - "bytes", - "datafusion-catalog 50.3.0", - "datafusion-common 50.3.0", - "datafusion-common-runtime 50.3.0", - "datafusion-datasource 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "datafusion-physical-plan 50.3.0", - "datafusion-session 50.3.0", - "futures", - "object_store", - "serde_json", - "tokio", -] - [[package]] name = "datafusion-datasource-json" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "adeacdb00c1d37271176f8fb6a1d8ce096baba16ea7a4b2671840c5c9c64fe85" dependencies = [ - "arrow 57.3.0", + "arrow", "async-trait", "bytes", - "datafusion-common 51.0.0", - "datafusion-common-runtime 51.0.0", - "datafusion-datasource 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "datafusion-physical-plan 51.0.0", - "datafusion-session 51.0.0", + "datafusion-common", + "datafusion-common-runtime", + "datafusion-datasource", + "datafusion-execution", + "datafusion-expr", + "datafusion-physical-expr-common", + "datafusion-physical-plan", + "datafusion-session", "futures", "object_store", "tokio", @@ -2390,110 +1942,51 @@ dependencies = [ [[package]] name = "datafusion-datasource-parquet" -version = "50.3.0" +version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "09e783c4c7d7faa1199af2df4761c68530634521b176a8d1331ddbc5a5c75133" +checksum = "43d0b60ffd66f28bfb026565d62b0a6cbc416da09814766a3797bba7d85a3cd9" dependencies = [ - "arrow 56.2.0", + "arrow", "async-trait", "bytes", - "datafusion-catalog 50.3.0", - "datafusion-common 50.3.0", - "datafusion-common-runtime 50.3.0", - "datafusion-datasource 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-functions-aggregate 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-adapter 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "datafusion-physical-optimizer 50.3.0", - "datafusion-physical-plan 50.3.0", - "datafusion-pruning 50.3.0", - "datafusion-session 50.3.0", + "datafusion-common", + "datafusion-common-runtime", + "datafusion-datasource", + "datafusion-execution", + "datafusion-expr", + "datafusion-functions-aggregate-common", + "datafusion-physical-expr", + "datafusion-physical-expr-adapter", + "datafusion-physical-expr-common", + "datafusion-physical-plan", + "datafusion-pruning", + "datafusion-session", "futures", "itertools 0.14.0", "log", "object_store", "parking_lot", - "parquet 56.2.0", - "rand 0.9.2", - "tokio", -] - -[[package]] -name = "datafusion-datasource-parquet" -version = "51.0.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "43d0b60ffd66f28bfb026565d62b0a6cbc416da09814766a3797bba7d85a3cd9" -dependencies = [ - "arrow 57.3.0", - "async-trait", - "bytes", - "datafusion-common 51.0.0", - "datafusion-common-runtime 51.0.0", - "datafusion-datasource 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-functions-aggregate-common 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-adapter 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "datafusion-physical-plan 51.0.0", - "datafusion-pruning 51.0.0", - "datafusion-session 51.0.0", - "futures", - "itertools 0.14.0", - "log", - "object_store", - "parking_lot", - "parquet 57.3.0", + "parquet", "tokio", ] -[[package]] -name = "datafusion-doc" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "99ee6b1d9a80d13f9deb2291f45c07044b8e62fb540dbde2453a18be17a36429" - [[package]] name = "datafusion-doc" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2b99e13947667b36ad713549237362afb054b2d8f8cc447751e23ec61202db07" -[[package]] -name = "datafusion-execution" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a4cec0a57653bec7b933fb248d3ffa3fa3ab3bd33bd140dc917f714ac036f531" -dependencies = [ - "arrow 56.2.0", - "async-trait", - "dashmap", - "datafusion-common 50.3.0", - "datafusion-expr 50.3.0", - "futures", - "log", - "object_store", - "parking_lot", - "rand 0.9.2", - "tempfile", - "url", -] - [[package]] name = "datafusion-execution" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "63695643190679037bc946ad46a263b62016931547bf119859c511f7ff2f5178" dependencies = [ - "arrow 57.3.0", + "arrow", "async-trait", "dashmap", - "datafusion-common 51.0.0", - "datafusion-expr 51.0.0", + "datafusion-common", + "datafusion-expr", "futures", "log", "object_store", @@ -2503,62 +1996,27 @@ dependencies = [ "url", ] -[[package]] -name = "datafusion-expr" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ef76910bdca909722586389156d0aa4da4020e1631994d50fadd8ad4b1aa05fe" -dependencies = [ - "arrow 56.2.0", - "async-trait", - "chrono", - "datafusion-common 50.3.0", - "datafusion-doc 50.3.0", - "datafusion-expr-common 50.3.0", - "datafusion-functions-aggregate-common 50.3.0", - "datafusion-functions-window-common 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "indexmap", - "paste", - "recursive", - "serde_json", - "sqlparser 0.58.0", -] - [[package]] name = "datafusion-expr" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f9a4787cbf5feb1ab351f789063398f67654a6df75c4d37d7f637dc96f951a91" dependencies = [ - "arrow 57.3.0", + "arrow", "async-trait", "chrono", - "datafusion-common 51.0.0", - "datafusion-doc 51.0.0", - "datafusion-expr-common 51.0.0", - "datafusion-functions-aggregate-common 51.0.0", - "datafusion-functions-window-common 51.0.0", - "datafusion-physical-expr-common 51.0.0", + "datafusion-common", + "datafusion-doc", + "datafusion-expr-common", + "datafusion-functions-aggregate-common", + "datafusion-functions-window-common", + "datafusion-physical-expr-common", "indexmap", "itertools 0.14.0", "paste", "recursive", "serde_json", - "sqlparser 0.59.0", -] - -[[package]] -name = "datafusion-expr-common" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6d155ccbda29591ca71a1344dd6bed26c65a4438072b400df9db59447f590bb6" -dependencies = [ - "arrow 56.2.0", - "datafusion-common 50.3.0", - "indexmap", - "itertools 0.14.0", - "paste", + "sqlparser", ] [[package]] @@ -2567,60 +2025,31 @@ version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5ce2fb1b8c15c9ac45b0863c30b268c69dc9ee7a1ee13ecf5d067738338173dc" dependencies = [ - "arrow 57.3.0", - "datafusion-common 51.0.0", + "arrow", + "datafusion-common", "indexmap", "itertools 0.14.0", "paste", ] -[[package]] -name = "datafusion-functions" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7de2782136bd6014670fd84fe3b0ca3b3e4106c96403c3ae05c0598577139977" -dependencies = [ - "arrow 56.2.0", - "arrow-buffer 56.2.0", - "base64", - "blake2", - "blake3", - "chrono", - "datafusion-common 50.3.0", - "datafusion-doc 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-expr-common 50.3.0", - "datafusion-macros 50.3.0", - "hex", - "itertools 0.14.0", - "log", - "md-5", - "rand 0.9.2", - "regex", - "sha2", - "unicode-segmentation", - "uuid", -] - [[package]] name = "datafusion-functions" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "794a9db7f7b96b3346fc007ff25e994f09b8f0511b4cf7dff651fadfe3ebb28f" dependencies = [ - "arrow 57.3.0", - "arrow-buffer 57.3.0", + "arrow", + "arrow-buffer", "base64", "blake2", "blake3", "chrono", - "datafusion-common 51.0.0", - "datafusion-doc 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-expr-common 51.0.0", - "datafusion-macros 51.0.0", + "datafusion-common", + "datafusion-doc", + "datafusion-execution", + "datafusion-expr", + "datafusion-expr-common", + "datafusion-macros", "hex", "itertools 0.14.0", "log", @@ -2633,27 +2062,6 @@ dependencies = [ "uuid", ] -[[package]] -name = "datafusion-functions-aggregate" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "07331fc13603a9da97b74fd8a273f4238222943dffdbbed1c4c6f862a30105bf" -dependencies = [ - "ahash", - "arrow 56.2.0", - "datafusion-common 50.3.0", - "datafusion-doc 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-functions-aggregate-common 50.3.0", - "datafusion-macros 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "half", - "log", - "paste", -] - [[package]] name = "datafusion-functions-aggregate" version = "51.0.0" @@ -2661,33 +2069,20 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1c25210520a9dcf9c2b2cbbce31ebd4131ef5af7fc60ee92b266dc7d159cb305" dependencies = [ "ahash", - "arrow 57.3.0", - "datafusion-common 51.0.0", - "datafusion-doc 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-functions-aggregate-common 51.0.0", - "datafusion-macros 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-common 51.0.0", + "arrow", + "datafusion-common", + "datafusion-doc", + "datafusion-execution", + "datafusion-expr", + "datafusion-functions-aggregate-common", + "datafusion-macros", + "datafusion-physical-expr", + "datafusion-physical-expr-common", "half", "log", "paste", ] -[[package]] -name = "datafusion-functions-aggregate-common" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b5951e572a8610b89968a09b5420515a121fbc305c0258651f318dc07c97ab17" -dependencies = [ - "ahash", - "arrow 56.2.0", - "datafusion-common 50.3.0", - "datafusion-expr-common 50.3.0", - "datafusion-physical-expr-common 50.3.0", -] - [[package]] name = "datafusion-functions-aggregate-common" version = "51.0.0" @@ -2695,32 +2090,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "62f4a66f3b87300bb70f4124b55434d2ae3fe80455f3574701d0348da040b55d" dependencies = [ "ahash", - "arrow 57.3.0", - "datafusion-common 51.0.0", - "datafusion-expr-common 51.0.0", - "datafusion-physical-expr-common 51.0.0", -] - -[[package]] -name = "datafusion-functions-nested" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "fdacca9302c3d8fc03f3e94f338767e786a88a33f5ebad6ffc0e7b50364b9ea3" -dependencies = [ - "arrow 56.2.0", - "arrow-ord 56.2.0", - "datafusion-common 50.3.0", - "datafusion-doc 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-functions 50.3.0", - "datafusion-functions-aggregate 50.3.0", - "datafusion-functions-aggregate-common 50.3.0", - "datafusion-macros 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "itertools 0.14.0", - "log", - "paste", + "arrow", + "datafusion-common", + "datafusion-expr-common", + "datafusion-physical-expr-common", ] [[package]] @@ -2729,120 +2102,65 @@ version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ae5c06eed03918dc7fe7a9f082a284050f0e9ecf95d72f57712d1496da03b8c4" dependencies = [ - "arrow 57.3.0", - "arrow-ord 57.3.0", - "datafusion-common 51.0.0", - "datafusion-doc 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-expr-common 51.0.0", - "datafusion-functions 51.0.0", - "datafusion-functions-aggregate 51.0.0", - "datafusion-functions-aggregate-common 51.0.0", - "datafusion-macros 51.0.0", - "datafusion-physical-expr-common 51.0.0", + "arrow", + "arrow-ord", + "datafusion-common", + "datafusion-doc", + "datafusion-execution", + "datafusion-expr", + "datafusion-expr-common", + "datafusion-functions", + "datafusion-functions-aggregate", + "datafusion-functions-aggregate-common", + "datafusion-macros", + "datafusion-physical-expr-common", "itertools 0.14.0", "log", "paste", ] -[[package]] -name = "datafusion-functions-table" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8c37ff8a99434fbbad604a7e0669717c58c7c4f14c472d45067c4b016621d981" -dependencies = [ - "arrow 56.2.0", - "async-trait", - "datafusion-catalog 50.3.0", - "datafusion-common 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-physical-plan 50.3.0", - "parking_lot", - "paste", -] - [[package]] name = "datafusion-functions-table" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "db4fed1d71738fbe22e2712d71396db04c25de4111f1ec252b8f4c6d3b25d7f5" dependencies = [ - "arrow 57.3.0", + "arrow", "async-trait", - "datafusion-catalog 51.0.0", - "datafusion-common 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-physical-plan 51.0.0", + "datafusion-catalog", + "datafusion-common", + "datafusion-expr", + "datafusion-physical-plan", "parking_lot", "paste", ] -[[package]] -name = "datafusion-functions-window" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "48e2aea7c79c926cffabb13dc27309d4eaeb130f4a21c8ba91cdd241c813652b" -dependencies = [ - "arrow 56.2.0", - "datafusion-common 50.3.0", - "datafusion-doc 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-functions-window-common 50.3.0", - "datafusion-macros 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "log", - "paste", -] - [[package]] name = "datafusion-functions-window" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1d92206aa5ae21892f1552b4d61758a862a70956e6fd7a95cb85db1de74bc6d1" dependencies = [ - "arrow 57.3.0", - "datafusion-common 51.0.0", - "datafusion-doc 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-functions-window-common 51.0.0", - "datafusion-macros 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-common 51.0.0", + "arrow", + "datafusion-common", + "datafusion-doc", + "datafusion-expr", + "datafusion-functions-window-common", + "datafusion-macros", + "datafusion-physical-expr", + "datafusion-physical-expr-common", "log", "paste", ] -[[package]] -name = "datafusion-functions-window-common" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0fead257ab5fd2ffc3b40fda64da307e20de0040fe43d49197241d9de82a487f" -dependencies = [ - "datafusion-common 50.3.0", - "datafusion-physical-expr-common 50.3.0", -] - [[package]] name = "datafusion-functions-window-common" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "53ae9bcc39800820d53a22d758b3b8726ff84a5a3e24cecef04ef4e5fdf1c7cc" dependencies = [ - "datafusion-common 51.0.0", - "datafusion-physical-expr-common 51.0.0", -] - -[[package]] -name = "datafusion-macros" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ec6f637bce95efac05cdfb9b6c19579ed4aa5f6b94d951cfa5bb054b7bb4f730" -dependencies = [ - "datafusion-expr 50.3.0", - "quote", - "syn 2.0.117", + "datafusion-common", + "datafusion-physical-expr-common", ] [[package]] @@ -2851,43 +2169,23 @@ version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1063ad4c9e094b3f798acee16d9a47bd7372d9699be2de21b05c3bd3f34ab848" dependencies = [ - "datafusion-doc 51.0.0", + "datafusion-doc", "quote", "syn 2.0.117", ] -[[package]] -name = "datafusion-optimizer" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c6583ef666ae000a613a837e69e456681a9faa96347bf3877661e9e89e141d8a" -dependencies = [ - "arrow 56.2.0", - "chrono", - "datafusion-common 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-expr-common 50.3.0", - "datafusion-physical-expr 50.3.0", - "indexmap", - "itertools 0.14.0", - "log", - "recursive", - "regex", - "regex-syntax", -] - [[package]] name = "datafusion-optimizer" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9f35f9ec5d08b87fd1893a30c2929f2559c2f9806ca072d8fefca5009dc0f06a" dependencies = [ - "arrow 57.3.0", + "arrow", "chrono", - "datafusion-common 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-expr-common 51.0.0", - "datafusion-physical-expr 51.0.0", + "datafusion-common", + "datafusion-expr", + "datafusion-expr-common", + "datafusion-physical-expr", "indexmap", "itertools 0.14.0", "log", @@ -2896,29 +2194,6 @@ dependencies = [ "regex-syntax", ] -[[package]] -name = "datafusion-physical-expr" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c8668103361a272cbbe3a61f72eca60c9b7c706e87cc3565bcf21e2b277b84f6" -dependencies = [ - "ahash", - "arrow 56.2.0", - "datafusion-common 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-expr-common 50.3.0", - "datafusion-functions-aggregate-common 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "half", - "hashbrown 0.14.5", - "indexmap", - "itertools 0.14.0", - "log", - "parking_lot", - "paste", - "petgraph 0.8.3", -] - [[package]] name = "datafusion-physical-expr" version = "51.0.0" @@ -2926,34 +2201,19 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c30cc8012e9eedcb48bbe112c6eff4ae5ed19cf3003cb0f505662e88b7014c5d" dependencies = [ "ahash", - "arrow 57.3.0", - "datafusion-common 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-expr-common 51.0.0", - "datafusion-functions-aggregate-common 51.0.0", - "datafusion-physical-expr-common 51.0.0", + "arrow", + "datafusion-common", + "datafusion-expr", + "datafusion-expr-common", + "datafusion-functions-aggregate-common", + "datafusion-physical-expr-common", "half", "hashbrown 0.14.5", "indexmap", "itertools 0.14.0", "parking_lot", "paste", - "petgraph 0.8.3", -] - -[[package]] -name = "datafusion-physical-expr-adapter" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "815acced725d30601b397e39958e0e55630e0a10d66ef7769c14ae6597298bb0" -dependencies = [ - "arrow 56.2.0", - "datafusion-common 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-functions 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "itertools 0.14.0", + "petgraph", ] [[package]] @@ -2962,26 +2222,12 @@ version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f9ff2dbd476221b1f67337699eff432781c4e6e1713d2aefdaa517dfbf79768" dependencies = [ - "arrow 57.3.0", - "datafusion-common 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-functions 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "itertools 0.14.0", -] - -[[package]] -name = "datafusion-physical-expr-common" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6652fe7b5bf87e85ed175f571745305565da2c0b599d98e697bcbedc7baa47c3" -dependencies = [ - "ahash", - "arrow 56.2.0", - "datafusion-common 50.3.0", - "datafusion-expr-common 50.3.0", - "hashbrown 0.14.5", + "arrow", + "datafusion-common", + "datafusion-expr", + "datafusion-functions", + "datafusion-physical-expr", + "datafusion-physical-expr-common", "itertools 0.14.0", ] @@ -2992,83 +2238,32 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "90da43e1ec550b172f34c87ec68161986ced70fd05c8d2a2add66eef9c276f03" dependencies = [ "ahash", - "arrow 57.3.0", - "datafusion-common 51.0.0", - "datafusion-expr-common 51.0.0", + "arrow", + "datafusion-common", + "datafusion-expr-common", "hashbrown 0.14.5", "itertools 0.14.0", ] -[[package]] -name = "datafusion-physical-optimizer" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "49b7d623eb6162a3332b564a0907ba00895c505d101b99af78345f1acf929b5c" -dependencies = [ - "arrow 56.2.0", - "datafusion-common 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-expr-common 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "datafusion-physical-plan 50.3.0", - "datafusion-pruning 50.3.0", - "itertools 0.14.0", - "log", - "recursive", -] - [[package]] name = "datafusion-physical-optimizer" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ce9804f799acd7daef3be7aaffe77c0033768ed8fdbf5fb82fc4c5f2e6bc14e6" dependencies = [ - "arrow 57.3.0", - "datafusion-common 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-expr-common 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "datafusion-physical-plan 51.0.0", - "datafusion-pruning 51.0.0", + "arrow", + "datafusion-common", + "datafusion-execution", + "datafusion-expr", + "datafusion-expr-common", + "datafusion-physical-expr", + "datafusion-physical-expr-common", + "datafusion-physical-plan", + "datafusion-pruning", "itertools 0.14.0", "recursive", ] -[[package]] -name = "datafusion-physical-plan" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e2f7f778a1a838dec124efb96eae6144237d546945587557c9e6936b3414558c" -dependencies = [ - "ahash", - "arrow 56.2.0", - "arrow-ord 56.2.0", - "arrow-schema 56.2.0", - "async-trait", - "chrono", - "datafusion-common 50.3.0", - "datafusion-common-runtime 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-functions-aggregate-common 50.3.0", - "datafusion-functions-window-common 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "futures", - "half", - "hashbrown 0.14.5", - "indexmap", - "itertools 0.14.0", - "log", - "parking_lot", - "pin-project-lite", - "tokio", -] - [[package]] name = "datafusion-physical-plan" version = "51.0.0" @@ -3076,19 +2271,19 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0acf0ad6b6924c6b1aa7d213b181e012e2d3ec0a64ff5b10ee6282ab0f8532ac" dependencies = [ "ahash", - "arrow 57.3.0", - "arrow-ord 57.3.0", - "arrow-schema 57.3.0", + "arrow", + "arrow-ord", + "arrow-schema", "async-trait", "chrono", - "datafusion-common 51.0.0", - "datafusion-common-runtime 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-functions-aggregate-common 51.0.0", - "datafusion-functions-window-common 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-common 51.0.0", + "datafusion-common", + "datafusion-common-runtime", + "datafusion-execution", + "datafusion-expr", + "datafusion-functions-aggregate-common", + "datafusion-functions-window-common", + "datafusion-physical-expr", + "datafusion-physical-expr-common", "futures", "half", "hashbrown 0.14.5", @@ -3106,25 +2301,25 @@ version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d368093a98a17d1449b1083ac22ed16b7128e4c67789991869480d8c4a40ecb9" dependencies = [ - "arrow 57.3.0", + "arrow", "chrono", - "datafusion-catalog 51.0.0", - "datafusion-catalog-listing 51.0.0", - "datafusion-common 51.0.0", - "datafusion-datasource 51.0.0", + "datafusion-catalog", + "datafusion-catalog-listing", + "datafusion-common", + "datafusion-datasource", "datafusion-datasource-arrow", - "datafusion-datasource-csv 51.0.0", - "datafusion-datasource-json 51.0.0", - "datafusion-datasource-parquet 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-functions-table 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "datafusion-physical-plan 51.0.0", + "datafusion-datasource-csv", + "datafusion-datasource-json", + "datafusion-datasource-parquet", + "datafusion-execution", + "datafusion-expr", + "datafusion-functions-table", + "datafusion-physical-expr", + "datafusion-physical-expr-common", + "datafusion-physical-plan", "datafusion-proto-common", "object_store", - "prost 0.14.3", + "prost", ] [[package]] @@ -3133,27 +2328,9 @@ version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3b6aef3d5e5c1d2bc3114c4876730cb76a9bdc5a8df31ef1b6db48f0c1671895" dependencies = [ - "arrow 57.3.0", - "datafusion-common 51.0.0", - "prost 0.14.3", -] - -[[package]] -name = "datafusion-pruning" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cd1e59e2ca14fe3c30f141600b10ad8815e2856caa59ebbd0e3e07cd3d127a65" -dependencies = [ - "arrow 56.2.0", - "arrow-schema 56.2.0", - "datafusion-common 50.3.0", - "datafusion-datasource 50.3.0", - "datafusion-expr-common 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-expr-common 50.3.0", - "datafusion-physical-plan 50.3.0", - "itertools 0.14.0", - "log", + "arrow", + "datafusion-common", + "prost", ] [[package]] @@ -3162,39 +2339,15 @@ version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ac2c2498a1f134a9e11a9f5ed202a2a7d7e9774bd9249295593053ea3be999db" dependencies = [ - "arrow 57.3.0", - "datafusion-common 51.0.0", - "datafusion-datasource 51.0.0", - "datafusion-expr-common 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-expr-common 51.0.0", - "datafusion-physical-plan 51.0.0", - "itertools 0.14.0", - "log", -] - -[[package]] -name = "datafusion-session" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "21ef8e2745583619bd7a49474e8f45fbe98ebb31a133f27802217125a7b3d58d" -dependencies = [ - "arrow 56.2.0", - "async-trait", - "dashmap", - "datafusion-common 50.3.0", - "datafusion-common-runtime 50.3.0", - "datafusion-execution 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-plan 50.3.0", - "datafusion-sql 50.3.0", - "futures", + "arrow", + "datafusion-common", + "datafusion-datasource", + "datafusion-expr-common", + "datafusion-physical-expr", + "datafusion-physical-expr-common", + "datafusion-physical-plan", "itertools 0.14.0", "log", - "object_store", - "parking_lot", - "tokio", ] [[package]] @@ -3204,46 +2357,29 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8f96eebd17555386f459037c65ab73aae8df09f464524c709d6a3134ad4f4776" dependencies = [ "async-trait", - "datafusion-common 51.0.0", - "datafusion-execution 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-physical-plan 51.0.0", + "datafusion-common", + "datafusion-execution", + "datafusion-expr", + "datafusion-physical-plan", "parking_lot", ] -[[package]] -name = "datafusion-sql" -version = "50.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "89abd9868770386fede29e5a4b14f49c0bf48d652c3b9d7a8a0332329b87d50b" -dependencies = [ - "arrow 56.2.0", - "bigdecimal", - "datafusion-common 50.3.0", - "datafusion-expr 50.3.0", - "indexmap", - "log", - "recursive", - "regex", - "sqlparser 0.58.0", -] - [[package]] name = "datafusion-sql" version = "51.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "3fc195fe60634b2c6ccfd131b487de46dc30eccae8a3c35a13f136e7f440414f" dependencies = [ - "arrow 57.3.0", + "arrow", "bigdecimal", "chrono", - "datafusion-common 51.0.0", - "datafusion-expr 51.0.0", + "datafusion-common", + "datafusion-expr", "indexmap", "log", "recursive", "regex", - "sqlparser 0.59.0", + "sqlparser", ] [[package]] @@ -3290,7 +2426,7 @@ version = "0.19.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "06f7fc164b1557731fcc68a198e813811a000efade0f112d4f0a002e65042b83" dependencies = [ - "arrow 57.3.0", + "arrow", "bytes", "chrono", "comfy-table", @@ -3300,7 +2436,7 @@ dependencies = [ "indexmap", "itertools 0.14.0", "object_store", - "parquet 57.3.0", + "parquet", "reqwest", "roaring 0.11.3", "rustc_version", @@ -3387,24 +2523,24 @@ version = "0.30.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5b098d0ce09726f10a08b102c885a501ee18f06ea4aca864570508a9d5b620d1" dependencies = [ - "arrow 57.3.0", - "arrow-arith 57.3.0", - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-cast 57.3.0", - "arrow-ipc 57.3.0", - "arrow-json 57.3.0", - "arrow-ord 57.3.0", - "arrow-row 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow", + "arrow-arith", + "arrow-array", + "arrow-buffer", + "arrow-cast", + "arrow-ipc", + "arrow-json", + "arrow-ord", + "arrow-row", + "arrow-schema", + "arrow-select", "async-trait", "bytes", "cfg-if", "chrono", "dashmap", - "datafusion 51.0.0", - "datafusion-datasource 51.0.0", + "datafusion", + "datafusion-datasource", "datafusion-proto", "delta_kernel", "deltalake-derive", @@ -3417,7 +2553,7 @@ dependencies = [ "num_cpus", "object_store", "parking_lot", - "parquet 57.3.0", + "parquet", "percent-encoding", "percent-encoding-rfc3986", "pin-project-lite", @@ -3425,7 +2561,7 @@ dependencies = [ "regex", "serde", "serde_json", - "sqlparser 0.59.0", + "sqlparser", "strum 0.27.2", "thiserror 2.0.17", "tokio", @@ -3743,23 +2879,13 @@ version = "1.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c" -[[package]] -name = "fsst" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5ffdff7a2d68d22afc0657eddde3e946371ce7cfe730a3f78a5ed44ea5b1cb2e" -dependencies = [ - "arrow-array 56.2.0", - "rand 0.9.2", -] - [[package]] name = "fsst" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5f9e5c0b1c67a38cb92b41535d44623483beb9511592ae23a3bf42ddec758690" dependencies = [ - "arrow-array 57.3.0", + "arrow-array", "rand 0.9.2", ] @@ -3937,127 +3063,64 @@ dependencies = [ "serde", ] -[[package]] -name = "geoarrow-array" -version = "0.6.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7d1884b17253d8572e88833c282fcbb442365e4ae5f9052ced2831608253436c" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-schema 56.2.0", - "geo-traits", - "geoarrow-schema 0.6.2", - "num-traits", - "wkb", - "wkt", -] - [[package]] name = "geoarrow-array" version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "dc1cc4106ac0a0a512c398961ce95d8150475c84a84e17c4511c3643fa120a17" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-schema 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-schema", "geo-traits", - "geoarrow-schema 0.7.0", + "geoarrow-schema", "num-traits", "wkb", "wkt", ] -[[package]] -name = "geoarrow-expr-geo" -version = "0.6.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a67d3b543bc3ebeffdc204b67d69b8f9fcd33d76269ddd4a4618df99f053a934" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "geo", - "geo-traits", - "geoarrow-array 0.6.2", - "geoarrow-schema 0.6.2", -] - [[package]] name = "geoarrow-expr-geo" version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fa84300361ce57fb875bcaa6e32b95b0aff5c6b1af692b936bdd58ff343f4394" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", + "arrow-array", + "arrow-buffer", "geo", "geo-traits", - "geoarrow-array 0.7.0", - "geoarrow-schema 0.7.0", + "geoarrow-array", + "geoarrow-schema", ] [[package]] name = "geoarrow-schema" -version = "0.6.2" +version = "0.7.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "02f1b18b1c9a44ecd72be02e53d6e63bbccfdc8d1765206226af227327e2be6e" +checksum = "e97be4e9f523f92bd6a0e0458323f4b783d073d011664decd8dbf05651704f34" dependencies = [ - "arrow-schema 56.2.0", + "arrow-schema", "geo-traits", "serde", "serde_json", "thiserror 1.0.69", ] -[[package]] -name = "geoarrow-schema" -version = "0.7.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e97be4e9f523f92bd6a0e0458323f4b783d073d011664decd8dbf05651704f34" -dependencies = [ - "arrow-schema 57.3.0", - "geo-traits", - "serde", - "serde_json", - "thiserror 1.0.69", -] - -[[package]] -name = "geodatafusion" -version = "0.1.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "83d676b8d8b5f391ab4270ba31e9b599ee2c3d780405a38e272a0a7565ea189c" -dependencies = [ - "arrow-arith 56.2.0", - "arrow-array 56.2.0", - "arrow-schema 56.2.0", - "datafusion 50.3.0", - "geo", - "geo-traits", - "geoarrow-array 0.6.2", - "geoarrow-expr-geo 0.6.2", - "geoarrow-schema 0.6.2", - "geohash", - "thiserror 1.0.69", - "wkt", -] - [[package]] name = "geodatafusion" version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "773cfa1fb0d7f7661b76b3fde00f3ffd8e0ff7b3635096f0ff6294fe5ca62a2b" dependencies = [ - "arrow-arith 57.3.0", - "arrow-array 57.3.0", - "arrow-schema 57.3.0", - "datafusion 51.0.0", + "arrow-arith", + "arrow-array", + "arrow-schema", + "datafusion", "geo", "geo-traits", - "geoarrow-array 0.7.0", - "geoarrow-expr-geo 0.7.0", - "geoarrow-schema 0.7.0", + "geoarrow-array", + "geoarrow-expr-geo", + "geoarrow-schema", "geohash", "thiserror 1.0.69", "wkt", @@ -4235,9 +3298,9 @@ dependencies = [ name = "holograph" version = "0.1.0" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-schema 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-schema", "bincode", "futures", "log", @@ -4616,15 +3679,6 @@ dependencies = [ "hashbrown 0.16.0", ] -[[package]] -name = "indoc" -version = "2.0.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "79cf5c93f93228cf8efb3ba362535fb11199ac548a09ce117c9b1adc3030d706" -dependencies = [ - "rustversion", -] - [[package]] name = "inout" version = "0.1.4" @@ -4806,87 +3860,21 @@ dependencies = [ "simple_asn1", ] -[[package]] -name = "lance" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e8c439decbc304e180748e34bb6d3df729069a222e83e74e2185c38f107136e9" -dependencies = [ - "arrow 56.2.0", - "arrow-arith 56.2.0", - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-ipc 56.2.0", - "arrow-ord 56.2.0", - "arrow-row 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "async-recursion", - "async-trait", - "async_cell", - "aws-credential-types", - "byteorder", - "bytes", - "chrono", - "dashmap", - "datafusion 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-functions 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-physical-plan 50.3.0", - "deepsize", - "either", - "futures", - "half", - "humantime", - "itertools 0.13.0", - "lance-arrow 1.0.1", - "lance-core 1.0.1", - "lance-datafusion 1.0.1", - "lance-encoding 1.0.1", - "lance-file 1.0.1", - "lance-geo 1.0.1", - "lance-index 1.0.1", - "lance-io 1.0.1", - "lance-linalg 1.0.1", - "lance-namespace 1.0.1", - "lance-table 1.0.1", - "log", - "moka", - "object_store", - "permutation", - "pin-project", - "prost 0.13.5", - "prost-types 0.13.5", - "rand 0.9.2", - "roaring 0.10.12", - "semver", - "serde", - "serde_json", - "snafu", - "tantivy", - "tokio", - "tokio-stream", - "tracing", - "url", - "uuid", -] - [[package]] name = "lance" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2b7f07b905df393a5554eba19055c620f9ea25a3e40a013bda4bd8dc4ca66f01" dependencies = [ - "arrow 57.3.0", - "arrow-arith 57.3.0", - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-ipc 57.3.0", - "arrow-ord 57.3.0", - "arrow-row 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow", + "arrow-arith", + "arrow-array", + "arrow-buffer", + "arrow-ipc", + "arrow-ord", + "arrow-row", + "arrow-schema", + "arrow-select", "async-recursion", "async-trait", "async_cell", @@ -4895,35 +3883,35 @@ dependencies = [ "bytes", "chrono", "dashmap", - "datafusion 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-functions 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-physical-plan 51.0.0", + "datafusion", + "datafusion-expr", + "datafusion-functions", + "datafusion-physical-expr", + "datafusion-physical-plan", "deepsize", "either", "futures", "half", "humantime", "itertools 0.13.0", - "lance-arrow 2.0.1", - "lance-core 2.0.1", - "lance-datafusion 2.0.1", - "lance-encoding 2.0.1", - "lance-file 2.0.1", - "lance-geo 2.0.1", - "lance-index 2.0.1", - "lance-io 2.0.1", - "lance-linalg 2.0.1", - "lance-namespace 2.0.1", - "lance-table 2.0.1", + "lance-arrow", + "lance-core", + "lance-datafusion", + "lance-encoding", + "lance-file", + "lance-geo", + "lance-index", + "lance-io", + "lance-linalg", + "lance-namespace", + "lance-table", "log", "moka", "object_store", "permutation", "pin-project", - "prost 0.14.3", - "prost-types 0.14.3", + "prost", + "prost-types", "rand 0.9.2", "roaring 0.10.12", "semver", @@ -4938,39 +3926,19 @@ dependencies = [ "uuid", ] -[[package]] -name = "lance-arrow" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4ee5508b225456d3d56998eaeef0d8fbce5ea93856df47b12a94d2e74153210" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-cast 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "bytes", - "getrandom 0.2.16", - "half", - "jsonb", - "num-traits", - "rand 0.9.2", -] - [[package]] name = "lance-arrow" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "100e076cb81c8f0c24cd2881c706fc53e037c7d6e81eb320e929e265d157effb" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-cast 57.3.0", - "arrow-data 57.3.0", - "arrow-ord 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-cast", + "arrow-data", + "arrow-ord", + "arrow-schema", + "arrow-select", "bytes", "getrandom 0.2.16", "half", @@ -4979,17 +3947,6 @@ dependencies = [ "rand 0.9.2", ] -[[package]] -name = "lance-bitpacking" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d1c065fb3bd4a8cc4f78428443e990d4921aa08f707b676753db740e0b402a21" -dependencies = [ - "arrayref", - "paste", - "seq-macro", -] - [[package]] name = "lance-bitpacking" version = "2.0.1" @@ -5001,63 +3958,25 @@ dependencies = [ "seq-macro", ] -[[package]] -name = "lance-core" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e8856abad92e624b75cd57a04703f6441948a239463bdf973f2ac1924b0bcdbe" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-schema 56.2.0", - "async-trait", - "byteorder", - "bytes", - "chrono", - "datafusion-common 50.3.0", - "datafusion-sql 50.3.0", - "deepsize", - "futures", - "lance-arrow 1.0.1", - "libc", - "log", - "mock_instant", - "moka", - "num_cpus", - "object_store", - "pin-project", - "prost 0.13.5", - "rand 0.9.2", - "roaring 0.10.12", - "serde_json", - "snafu", - "tempfile", - "tokio", - "tokio-stream", - "tokio-util", - "tracing", - "url", -] - [[package]] name = "lance-core" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6fa01d1cf490ccfd3b8eaeee2781415d0419e6be8366040e57e43677abf2644e" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-schema 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-schema", "async-trait", "byteorder", "bytes", "chrono", - "datafusion-common 51.0.0", - "datafusion-sql 51.0.0", + "datafusion-common", + "datafusion-sql", "deepsize", "futures", "itertools 0.13.0", - "lance-arrow 2.0.1", + "lance-arrow", "libc", "log", "mock_instant", @@ -5065,7 +3984,7 @@ dependencies = [ "num_cpus", "object_store", "pin-project", - "prost 0.14.3", + "prost", "rand 0.9.2", "roaring 0.10.12", "serde_json", @@ -5078,99 +3997,48 @@ dependencies = [ "url", ] -[[package]] -name = "lance-datafusion" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4c8835308044cef5467d7751be87fcbefc2db01c22370726a8704bd62991693f" -dependencies = [ - "arrow 56.2.0", - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-ord 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "async-trait", - "chrono", - "datafusion 50.3.0", - "datafusion-common 50.3.0", - "datafusion-functions 50.3.0", - "datafusion-physical-expr 50.3.0", - "futures", - "jsonb", - "lance-arrow 1.0.1", - "lance-core 1.0.1", - "lance-datagen 1.0.1", - "lance-geo 1.0.1", - "log", - "pin-project", - "prost 0.13.5", - "snafu", - "tokio", - "tracing", -] - [[package]] name = "lance-datafusion" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ef89a39e3284eef76f79e63f23de8881a0583ad6feb20ed39f47eadd847a2b88" dependencies = [ - "arrow 57.3.0", - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-ord 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow", + "arrow-array", + "arrow-buffer", + "arrow-ord", + "arrow-schema", + "arrow-select", "async-trait", "chrono", - "datafusion 51.0.0", - "datafusion-common 51.0.0", - "datafusion-functions 51.0.0", - "datafusion-physical-expr 51.0.0", + "datafusion", + "datafusion-common", + "datafusion-functions", + "datafusion-physical-expr", "futures", "jsonb", - "lance-arrow 2.0.1", - "lance-core 2.0.1", - "lance-datagen 2.0.1", - "lance-geo 2.0.1", + "lance-arrow", + "lance-core", + "lance-datagen", + "lance-geo", "log", "pin-project", - "prost 0.14.3", + "prost", "snafu", "tokio", "tracing", ] -[[package]] -name = "lance-datagen" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "612de1e888bb36f6bf51196a6eb9574587fdf256b1759a4c50e643e00d5f96d0" -dependencies = [ - "arrow 56.2.0", - "arrow-array 56.2.0", - "arrow-cast 56.2.0", - "arrow-schema 56.2.0", - "chrono", - "futures", - "half", - "hex", - "rand 0.9.2", - "rand_xoshiro", - "random_word", -] - [[package]] name = "lance-datagen" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fc2a60eef5c47e65d91e2ffa8e7e1629c52e7190c8b88a371a1a60601dc49371" dependencies = [ - "arrow 57.3.0", - "arrow-array 57.3.0", - "arrow-cast 57.3.0", - "arrow-schema 57.3.0", + "arrow", + "arrow-array", + "arrow-cast", + "arrow-schema", "chrono", "futures", "half", @@ -5181,75 +4049,36 @@ dependencies = [ "random_word", ] -[[package]] -name = "lance-encoding" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2b456b29b135d3c7192602e516ccade38b5483986e121895fa43cf1fdb38bf60" -dependencies = [ - "arrow-arith 56.2.0", - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-cast 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "bytemuck", - "byteorder", - "bytes", - "fsst 1.0.1", - "futures", - "hex", - "hyperloglogplus", - "itertools 0.13.0", - "lance-arrow 1.0.1", - "lance-bitpacking 1.0.1", - "lance-core 1.0.1", - "log", - "lz4", - "num-traits", - "prost 0.13.5", - "prost-build 0.13.5", - "prost-types 0.13.5", - "rand 0.9.2", - "snafu", - "strum 0.26.3", - "tokio", - "tracing", - "xxhash-rust", - "zstd", -] - [[package]] name = "lance-encoding" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "95ce4a6631308aa681b2671af8f2a845ff781f8d4e755a2a7ccd012379467094" dependencies = [ - "arrow-arith 57.3.0", - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-cast 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow-arith", + "arrow-array", + "arrow-buffer", + "arrow-cast", + "arrow-data", + "arrow-schema", + "arrow-select", "bytemuck", "byteorder", "bytes", - "fsst 2.0.1", + "fsst", "futures", "hex", "hyperloglogplus", "itertools 0.13.0", - "lance-arrow 2.0.1", - "lance-bitpacking 2.0.1", - "lance-core 2.0.1", + "lance-arrow", + "lance-bitpacking", + "lance-core", "log", "lz4", "num-traits", - "prost 0.14.3", - "prost-build 0.14.3", - "prost-types 0.14.3", + "prost", + "prost-build", + "prost-types", "rand 0.9.2", "snafu", "strum 0.26.3", @@ -5259,128 +4088,81 @@ dependencies = [ "zstd", ] -[[package]] -name = "lance-file" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ab1538d14d5bb3735b4222b3f5aff83cfa59cc6ef7cdd3dd9139e4c77193c80b" -dependencies = [ - "arrow-arith 56.2.0", - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "async-recursion", - "async-trait", - "byteorder", - "bytes", - "datafusion-common 50.3.0", - "deepsize", - "futures", - "lance-arrow 1.0.1", - "lance-core 1.0.1", - "lance-encoding 1.0.1", - "lance-io 1.0.1", - "log", - "num-traits", - "object_store", - "prost 0.13.5", - "prost-build 0.13.5", - "prost-types 0.13.5", - "snafu", - "tokio", - "tracing", -] - [[package]] name = "lance-file" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e2d4d82357cbfaa1a18494226c15b1cb3c8ed0b6c84b91146323c82047ede419" dependencies = [ - "arrow-arith 57.3.0", - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow-arith", + "arrow-array", + "arrow-buffer", + "arrow-data", + "arrow-schema", + "arrow-select", "async-recursion", "async-trait", "byteorder", "bytes", - "datafusion-common 51.0.0", + "datafusion-common", "deepsize", "futures", - "lance-arrow 2.0.1", - "lance-core 2.0.1", - "lance-encoding 2.0.1", - "lance-io 2.0.1", + "lance-arrow", + "lance-core", + "lance-encoding", + "lance-io", "log", "num-traits", "object_store", - "prost 0.14.3", - "prost-build 0.14.3", - "prost-types 0.14.3", + "prost", + "prost-build", + "prost-types", "snafu", "tokio", "tracing", ] -[[package]] -name = "lance-geo" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a5a69a2f3b55703d9c240ad7c5ffa2c755db69e9cf8aa05efe274a212910472d" -dependencies = [ - "datafusion 50.3.0", - "geo-types", - "geoarrow-array 0.6.2", - "geoarrow-schema 0.6.2", - "geodatafusion 0.1.1", -] - [[package]] name = "lance-geo" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a7183fc870da62826f0f97df8007b634da053eb310157856efe1dc74f446951c" dependencies = [ - "datafusion 51.0.0", + "datafusion", "geo-traits", "geo-types", - "geoarrow-array 0.7.0", - "geoarrow-schema 0.7.0", - "geodatafusion 0.2.0", - "lance-core 2.0.1", + "geoarrow-array", + "geoarrow-schema", + "geodatafusion", + "lance-core", "serde", ] [[package]] name = "lance-graph" -version = "0.5.3" +version = "0.5.4" dependencies = [ - "arrow 57.3.0", - "arrow-array 57.3.0", - "arrow-schema 57.3.0", + "arrow", + "arrow-array", + "arrow-schema", "async-trait", "bgz-tensor", "bgz17", "chrono", - "datafusion 51.0.0", - "datafusion-common 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-functions-aggregate 51.0.0", - "datafusion-sql 51.0.0", + "datafusion", + "datafusion-common", + "datafusion-expr", + "datafusion-functions-aggregate", + "datafusion-sql", "deltalake", "futures", - "lance 2.0.1", - "lance-arrow 2.0.1", + "lance", + "lance-arrow", "lance-graph-catalog", "lance-graph-planner", - "lance-index 2.0.1", - "lance-linalg 2.0.1", - "lance-namespace 2.0.1", + "lance-index", + "lance-linalg", + "lance-namespace", "ndarray 0.17.2", "nom 7.1.3", "serde", @@ -5395,24 +4177,40 @@ dependencies = [ name = "lance-graph-benches" version = "0.1.0" dependencies = [ - "arrow-array 56.2.0", - "arrow-schema 56.2.0", + "arrow-array", + "arrow-schema", "criterion", "futures", - "lance 1.0.1", + "lance", "lance-graph", "tempfile", "tokio", ] +[[package]] +name = "lance-graph-callcenter" +version = "0.1.0" +dependencies = [ + "arrow", + "axum", + "datafusion", + "lance", + "lance-graph-contract", + "serde", + "serde_json", + "tokio", + "tokio-tungstenite 0.24.0", + "tower-http 0.5.2", +] + [[package]] name = "lance-graph-catalog" -version = "0.5.3" +version = "0.5.4" dependencies = [ - "arrow-schema 57.3.0", + "arrow-schema", "async-trait", - "datafusion 51.0.0", - "lance-namespace 2.0.1", + "datafusion", + "lance-namespace", "reqwest", "serde", "serde_json", @@ -5444,98 +4242,18 @@ dependencies = [ "tracing", ] -[[package]] -name = "lance-graph-python" -version = "0.5.3" -dependencies = [ - "arrow 57.3.0", - "arrow-array 57.3.0", - "arrow-ipc 57.3.0", - "arrow-schema 57.3.0", - "datafusion 51.0.0", - "futures", - "lance-graph", - "pyo3", - "serde", - "serde_json", - "tokio", -] - -[[package]] -name = "lance-index" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0ea84613df6fa6b9168a1f056ba4f9cb73b90a1b452814c6fd4b3529bcdbfc78" -dependencies = [ - "arrow 56.2.0", - "arrow-arith 56.2.0", - "arrow-array 56.2.0", - "arrow-ord 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "async-channel", - "async-recursion", - "async-trait", - "bitpacking", - "bitvec", - "bytes", - "crossbeam-queue", - "datafusion 50.3.0", - "datafusion-common 50.3.0", - "datafusion-expr 50.3.0", - "datafusion-physical-expr 50.3.0", - "datafusion-sql 50.3.0", - "deepsize", - "dirs", - "fst", - "futures", - "half", - "itertools 0.13.0", - "jsonb", - "lance-arrow 1.0.1", - "lance-core 1.0.1", - "lance-datafusion 1.0.1", - "lance-datagen 1.0.1", - "lance-encoding 1.0.1", - "lance-file 1.0.1", - "lance-io 1.0.1", - "lance-linalg 1.0.1", - "lance-table 1.0.1", - "libm", - "log", - "ndarray 0.16.1", - "num-traits", - "object_store", - "prost 0.13.5", - "prost-build 0.13.5", - "prost-types 0.13.5", - "rand 0.9.2", - "rand_distr 0.5.1", - "rayon", - "roaring 0.10.12", - "serde", - "serde_json", - "snafu", - "tantivy", - "tempfile", - "tokio", - "tracing", - "twox-hash", - "uuid", -] - [[package]] name = "lance-index" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "20e9c5aa7024a63af9ae89ee8c0f23c8421b7896742e5cd4a271a60f9956cb80" dependencies = [ - "arrow 57.3.0", - "arrow-arith 57.3.0", - "arrow-array 57.3.0", - "arrow-ord 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow", + "arrow-arith", + "arrow-array", + "arrow-ord", + "arrow-schema", + "arrow-select", "async-channel", "async-recursion", "async-trait", @@ -5543,39 +4261,39 @@ dependencies = [ "bitvec", "bytes", "crossbeam-queue", - "datafusion 51.0.0", - "datafusion-common 51.0.0", - "datafusion-expr 51.0.0", - "datafusion-physical-expr 51.0.0", - "datafusion-sql 51.0.0", + "datafusion", + "datafusion-common", + "datafusion-expr", + "datafusion-physical-expr", + "datafusion-sql", "deepsize", "dirs", "fst", "futures", "geo-types", - "geoarrow-array 0.7.0", - "geoarrow-schema 0.7.0", + "geoarrow-array", + "geoarrow-schema", "half", "itertools 0.13.0", "jsonb", - "lance-arrow 2.0.1", - "lance-core 2.0.1", - "lance-datafusion 2.0.1", - "lance-datagen 2.0.1", - "lance-encoding 2.0.1", - "lance-file 2.0.1", - "lance-geo 2.0.1", - "lance-io 2.0.1", - "lance-linalg 2.0.1", - "lance-table 2.0.1", + "lance-arrow", + "lance-core", + "lance-datafusion", + "lance-datagen", + "lance-encoding", + "lance-file", + "lance-geo", + "lance-io", + "lance-linalg", + "lance-table", "libm", "log", "ndarray 0.16.1", "num-traits", "object_store", - "prost 0.14.3", - "prost-build 0.14.3", - "prost-types 0.14.3", + "prost", + "prost-build", + "prost-types", "rand 0.9.2", "rand_distr 0.5.1", "rangemap", @@ -5593,62 +4311,20 @@ dependencies = [ "uuid", ] -[[package]] -name = "lance-io" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6b3fc4c1d941fceef40a0edbd664dbef108acfc5d559bb9e7f588d0c733cbc35" -dependencies = [ - "arrow 56.2.0", - "arrow-arith 56.2.0", - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-cast 56.2.0", - "arrow-data 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "async-recursion", - "async-trait", - "aws-config", - "aws-credential-types", - "byteorder", - "bytes", - "chrono", - "deepsize", - "futures", - "lance-arrow 1.0.1", - "lance-core 1.0.1", - "lance-namespace 1.0.1", - "log", - "object_store", - "object_store_opendal", - "opendal", - "path_abs", - "pin-project", - "prost 0.13.5", - "rand 0.9.2", - "serde", - "shellexpand", - "snafu", - "tokio", - "tracing", - "url", -] - [[package]] name = "lance-io" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c7d2af0b17fb374a8181bcf1a10bce5703ae3ee4373c1587ce4bba23e15e45c8" dependencies = [ - "arrow 57.3.0", - "arrow-arith 57.3.0", - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-cast 57.3.0", - "arrow-data 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow", + "arrow-arith", + "arrow-array", + "arrow-buffer", + "arrow-cast", + "arrow-data", + "arrow-schema", + "arrow-select", "async-recursion", "async-trait", "aws-config", @@ -5658,16 +4334,16 @@ dependencies = [ "chrono", "deepsize", "futures", - "lance-arrow 2.0.1", - "lance-core 2.0.1", - "lance-namespace 2.0.1", + "lance-arrow", + "lance-core", + "lance-namespace", "log", "object_store", "object_store_opendal", "opendal", "path_abs", "pin-project", - "prost 0.14.3", + "prost", "rand 0.9.2", "serde", "shellexpand", @@ -5677,83 +4353,38 @@ dependencies = [ "url", ] -[[package]] -name = "lance-linalg" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b62ffbc5ce367fbf700a69de3fe0612ee1a11191a64a632888610b6bacfa0f63" -dependencies = [ - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-schema 56.2.0", - "cc", - "deepsize", - "half", - "lance-arrow 1.0.1", - "lance-core 1.0.1", - "num-traits", - "rand 0.9.2", -] - [[package]] name = "lance-linalg" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5125aa62696e75a7475807564b4921f252d8815be606b84bc00e6def0f5c24bb" dependencies = [ - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-schema 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-schema", "cc", "deepsize", "half", - "lance-arrow 2.0.1", - "lance-core 2.0.1", + "lance-arrow", + "lance-core", "num-traits", "rand 0.9.2", ] -[[package]] -name = "lance-namespace" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "791bbcd868ee758123a34e07d320a1fb99379432b5ecc0e78d6b4686e999b629" -dependencies = [ - "arrow 56.2.0", - "async-trait", - "bytes", - "lance-core 1.0.1", - "lance-namespace-reqwest-client 0.0.18", - "snafu", -] - [[package]] name = "lance-namespace" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "70545c2676ce954dfd801da5c6a631a70bba967826cd3a8f31b47d1f04bbfed3" dependencies = [ - "arrow 57.3.0", + "arrow", "async-trait", "bytes", - "lance-core 2.0.1", - "lance-namespace-reqwest-client 0.4.5", + "lance-core", + "lance-namespace-reqwest-client", "snafu", ] -[[package]] -name = "lance-namespace-reqwest-client" -version = "0.0.18" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3ea349999bcda4eea53fc05d334b3775ec314761e6a706555c777d7a29b18d19" -dependencies = [ - "reqwest", - "serde", - "serde_json", - "serde_repr", - "url", -] - [[package]] name = "lance-namespace-reqwest-client" version = "0.4.5" @@ -5767,71 +4398,32 @@ dependencies = [ "url", ] -[[package]] -name = "lance-table" -version = "1.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6fdb2d56bfa4d1511c765fa0cc00fdaa37e5d2d1cd2f57b3c6355d9072177052" -dependencies = [ - "arrow 56.2.0", - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-ipc 56.2.0", - "arrow-schema 56.2.0", - "async-trait", - "byteorder", - "bytes", - "chrono", - "deepsize", - "futures", - "lance-arrow 1.0.1", - "lance-core 1.0.1", - "lance-file 1.0.1", - "lance-io 1.0.1", - "log", - "object_store", - "prost 0.13.5", - "prost-build 0.13.5", - "prost-types 0.13.5", - "rand 0.9.2", - "rangemap", - "roaring 0.10.12", - "semver", - "serde", - "serde_json", - "snafu", - "tokio", - "tracing", - "url", - "uuid", -] - [[package]] name = "lance-table" version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b06ad37bd90045de8ef533df170c6098e6ff6ecb427aade47d7db8e2c86f2678" dependencies = [ - "arrow 57.3.0", - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-ipc 57.3.0", - "arrow-schema 57.3.0", + "arrow", + "arrow-array", + "arrow-buffer", + "arrow-ipc", + "arrow-schema", "async-trait", "byteorder", "bytes", "chrono", "deepsize", "futures", - "lance-arrow 2.0.1", - "lance-core 2.0.1", - "lance-file 2.0.1", - "lance-io 2.0.1", + "lance-arrow", + "lance-core", + "lance-file", + "lance-io", "log", "object_store", - "prost 0.14.3", - "prost-build 0.14.3", - "prost-types 0.14.3", + "prost", + "prost-build", + "prost-types", "rand 0.9.2", "rangemap", "roaring 0.10.12", @@ -6060,9 +4652,6 @@ name = "lz4_flex" version = "0.11.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "08ab2867e3eeeca90e844d1940eab391c9dc5228783db2ed999acbc0a9ed375a" -dependencies = [ - "twox-hash", -] [[package]] name = "lz4_flex" @@ -6146,15 +4735,6 @@ dependencies = [ "libc", ] -[[package]] -name = "memoffset" -version = "0.9.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a" -dependencies = [ - "autocfg", -] - [[package]] name = "mime" version = "0.3.17" @@ -6323,20 +4903,6 @@ dependencies = [ "windows-sys 0.52.0", ] -[[package]] -name = "num" -version = "0.4.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "35bd024e8b2ff75562e5f34e7f4905839deb4b22955ef5e73d2fea1b9813cb23" -dependencies = [ - "num-bigint", - "num-complex", - "num-integer", - "num-iter", - "num-rational", - "num-traits", -] - [[package]] name = "num-bigint" version = "0.4.6" @@ -6399,17 +4965,6 @@ dependencies = [ "num-traits", ] -[[package]] -name = "num-rational" -version = "0.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f83d14da390562dca69fc84082e73e548e1ad308d24accdedd2720017cb37824" -dependencies = [ - "num-bigint", - "num-integer", - "num-traits", -] - [[package]] name = "num-traits" version = "0.2.19" @@ -6706,43 +5261,6 @@ dependencies = [ "windows-targets 0.52.6", ] -[[package]] -name = "parquet" -version = "56.2.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f0dbd48ad52d7dccf8ea1b90a3ddbfaea4f69878dd7683e51c507d4bc52b5b27" -dependencies = [ - "ahash", - "arrow-array 56.2.0", - "arrow-buffer 56.2.0", - "arrow-cast 56.2.0", - "arrow-data 56.2.0", - "arrow-ipc 56.2.0", - "arrow-schema 56.2.0", - "arrow-select 56.2.0", - "base64", - "brotli", - "bytes", - "chrono", - "flate2", - "futures", - "half", - "hashbrown 0.16.0", - "lz4_flex 0.11.5", - "num", - "num-bigint", - "object_store", - "paste", - "ring", - "seq-macro", - "simdutf8", - "snap", - "thrift", - "tokio", - "twox-hash", - "zstd", -] - [[package]] name = "parquet" version = "57.3.0" @@ -6750,13 +5268,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6ee96b29972a257b855ff2341b37e61af5f12d6af1158b6dcdb5b31ea07bb3cb" dependencies = [ "ahash", - "arrow-array 57.3.0", - "arrow-buffer 57.3.0", - "arrow-cast 57.3.0", - "arrow-data 57.3.0", - "arrow-ipc 57.3.0", - "arrow-schema 57.3.0", - "arrow-select 57.3.0", + "arrow-array", + "arrow-buffer", + "arrow-cast", + "arrow-data", + "arrow-ipc", + "arrow-schema", + "arrow-select", "base64", "brotli", "bytes", @@ -6845,16 +5363,6 @@ version = "0.4.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "df202b0b0f5b8e389955afd5f27b007b00fb948162953f1db9c70d2c7e3157d7" -[[package]] -name = "petgraph" -version = "0.7.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3672b37090dbd86368a4145bc067582552b29c27377cad4e0a306c97f9bd7772" -dependencies = [ - "fixedbitset", - "indexmap", -] - [[package]] name = "petgraph" version = "0.8.3" @@ -7082,16 +5590,6 @@ dependencies = [ "unicode-ident", ] -[[package]] -name = "prost" -version = "0.13.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2796faa41db3ec313a31f7624d9286acf277b52de526150b7e69f3debf891ee5" -dependencies = [ - "bytes", - "prost-derive 0.13.5", -] - [[package]] name = "prost" version = "0.14.3" @@ -7099,27 +5597,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d2ea70524a2f82d518bce41317d0fae74151505651af45faf1ffbd6fd33f0568" dependencies = [ "bytes", - "prost-derive 0.14.3", -] - -[[package]] -name = "prost-build" -version = "0.13.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "be769465445e8c1474e9c5dac2018218498557af32d9ed057325ec9a41ae81bf" -dependencies = [ - "heck", - "itertools 0.14.0", - "log", - "multimap", - "once_cell", - "petgraph 0.7.1", - "prettyplease", - "prost 0.13.5", - "prost-types 0.13.5", - "regex", - "syn 2.0.117", - "tempfile", + "prost-derive", ] [[package]] @@ -7132,28 +5610,15 @@ dependencies = [ "itertools 0.14.0", "log", "multimap", - "petgraph 0.8.3", + "petgraph", "prettyplease", - "prost 0.14.3", - "prost-types 0.14.3", + "prost", + "prost-types", "regex", "syn 2.0.117", "tempfile", ] -[[package]] -name = "prost-derive" -version = "0.13.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8a56d757972c98b346a9b766e3f02746cde6dd1cd1d1d563472929fdd74bec4d" -dependencies = [ - "anyhow", - "itertools 0.14.0", - "proc-macro2", - "quote", - "syn 2.0.117", -] - [[package]] name = "prost-derive" version = "0.14.3" @@ -7167,22 +5632,13 @@ dependencies = [ "syn 2.0.117", ] -[[package]] -name = "prost-types" -version = "0.13.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "52c2c1bf36ddb1a1c396b3601a3cec27c2462e45f07c386894ec3ccf5332bd16" -dependencies = [ - "prost 0.13.5", -] - [[package]] name = "prost-types" version = "0.14.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8991c4cbdb8bc5b11f0b074ffe286c30e523de90fee5ba8132f1399f23cb3dd7" dependencies = [ - "prost 0.14.3", + "prost", ] [[package]] @@ -7195,67 +5651,6 @@ dependencies = [ "cc", ] -[[package]] -name = "pyo3" -version = "0.26.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7ba0117f4212101ee6544044dae45abe1083d30ce7b29c4b5cbdfa2354e07383" -dependencies = [ - "indoc", - "libc", - "memoffset", - "once_cell", - "portable-atomic", - "pyo3-build-config", - "pyo3-ffi", - "pyo3-macros", - "unindent", -] - -[[package]] -name = "pyo3-build-config" -version = "0.26.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4fc6ddaf24947d12a9aa31ac65431fb1b851b8f4365426e182901eabfb87df5f" -dependencies = [ - "target-lexicon", -] - -[[package]] -name = "pyo3-ffi" -version = "0.26.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "025474d3928738efb38ac36d4744a74a400c901c7596199e20e45d98eb194105" -dependencies = [ - "libc", - "pyo3-build-config", -] - -[[package]] -name = "pyo3-macros" -version = "0.26.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2e64eb489f22fe1c95911b77c44cc41e7c19f3082fc81cce90f657cdc42ffded" -dependencies = [ - "proc-macro2", - "pyo3-macros-backend", - "quote", - "syn 2.0.117", -] - -[[package]] -name = "pyo3-macros-backend" -version = "0.26.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "100246c0ecf400b475341b8455a9213344569af29a3c841d29270e53102e0fcf" -dependencies = [ - "heck", - "proc-macro2", - "pyo3-build-config", - "quote", - "syn 2.0.117", -] - [[package]] name = "quick-xml" version = "0.37.5" @@ -7637,7 +6032,7 @@ dependencies = [ "tokio-rustls", "tokio-util", "tower", - "tower-http", + "tower-http 0.6.8", "tower-service", "url", "wasm-bindgen", @@ -8237,17 +6632,6 @@ dependencies = [ "der", ] -[[package]] -name = "sqlparser" -version = "0.58.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ec4b661c54b1e4b603b37873a18c59920e4c51ea8ea2cf527d925424dbd4437c" -dependencies = [ - "log", - "recursive", - "sqlparser_derive", -] - [[package]] name = "sqlparser" version = "0.59.0" @@ -8577,12 +6961,6 @@ version = "1.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369" -[[package]] -name = "target-lexicon" -version = "0.13.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "adb6935a6f5c20170eeceb1a3835a49e12e19d792f6dd344ccc76a985ca5a6ca" - [[package]] name = "tempfile" version = "3.23.0" @@ -8799,6 +7177,30 @@ dependencies = [ "tokio", ] +[[package]] +name = "tokio-tungstenite" +version = "0.24.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "edc5f74e248dc973e0dbb7b74c7e0d6fcc301c694ff50049504004ef4d0cdcd9" +dependencies = [ + "futures-util", + "log", + "tokio", + "tungstenite 0.24.0", +] + +[[package]] +name = "tokio-tungstenite" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d25a406cddcc431a75d3d9afc6a7c0f7428d4891dd973e4d54c56b46127bf857" +dependencies = [ + "futures-util", + "log", + "tokio", + "tungstenite 0.28.0", +] + [[package]] name = "tokio-util" version = "0.7.16" @@ -8858,6 +7260,22 @@ dependencies = [ "tracing", ] +[[package]] +name = "tower-http" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e9cd434a998747dd2c4276bc96ee2e0c7a2eadf3cae88e52be55a05fa9053f5" +dependencies = [ + "bitflags", + "bytes", + "http 1.3.1", + "http-body 1.0.1", + "http-body-util", + "pin-project-lite", + "tower-layer", + "tower-service", +] + [[package]] name = "tower-http" version = "0.6.8" @@ -8961,6 +7379,41 @@ version = "0.2.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b" +[[package]] +name = "tungstenite" +version = "0.24.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "18e5b8366ee7a95b16d32197d0b2604b43a0be89dc5fac9f8e96ccafbaedda8a" +dependencies = [ + "byteorder", + "bytes", + "data-encoding", + "http 1.3.1", + "httparse", + "log", + "rand 0.8.5", + "sha1", + "thiserror 1.0.69", + "utf-8", +] + +[[package]] +name = "tungstenite" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8628dcc84e5a09eb3d8423d6cb682965dea9133204e8fb3efee74c2a0c259442" +dependencies = [ + "bytes", + "data-encoding", + "http 1.3.1", + "httparse", + "log", + "rand 0.9.2", + "sha1", + "thiserror 2.0.17", + "utf-8", +] + [[package]] name = "twox-hash" version = "2.1.2" @@ -9020,12 +7473,6 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4a1a07cc7db3810833284e8d372ccdc6da29741639ecc70c9ec107df0fa6154c" -[[package]] -name = "unindent" -version = "0.2.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3" - [[package]] name = "untrusted" version = "0.9.0" @@ -9050,6 +7497,12 @@ version = "2.1.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "daf8dba3b7eb870caf1ddeed7bc9d2a049f3cfdfae7cb521b087cc33ae4c49da" +[[package]] +name = "utf-8" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09cc8ee72d2a9becf2f2febe0205bbed8fc6615b7cb429ad062dc7b7ddd036a9" + [[package]] name = "utf8-ranges" version = "1.0.5" diff --git a/Cargo.toml b/Cargo.toml index 0aab09fc..2c7fcf62 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -2,13 +2,15 @@ members = [ "crates/lance-graph", "crates/lance-graph-catalog", - "crates/lance-graph-python", "crates/lance-graph-benches", "crates/lance-graph-planner", "crates/lance-graph-contract", "crates/neural-debug", + "crates/lance-graph-callcenter", ] exclude = [ + # Python bindings (upstream-inherited, opt-in via --manifest-path) + "crates/lance-graph-python", "crates/lance-graph-codec-research", "crates/bgz17", "crates/deepnsm", diff --git a/crates/lance-graph-benches/Cargo.toml b/crates/lance-graph-benches/Cargo.toml index c8c74d51..4a955508 100644 --- a/crates/lance-graph-benches/Cargo.toml +++ b/crates/lance-graph-benches/Cargo.toml @@ -7,9 +7,9 @@ description = "Benchmarks for lance-graph (not published)" [dependencies] lance-graph = { path = "../lance-graph", version = "0.5.3" } -lance = "1.0.0" -arrow-array = "56.2" -arrow-schema = "56.2" +lance = "2" +arrow-array = "57" +arrow-schema = "57" criterion = { version = "0.5", features = ["async", "async_tokio", "html_reports"] } futures = "0.3" tempfile = "3" diff --git a/crates/lance-graph-callcenter/Cargo.toml b/crates/lance-graph-callcenter/Cargo.toml new file mode 100644 index 00000000..cdd33122 --- /dev/null +++ b/crates/lance-graph-callcenter/Cargo.toml @@ -0,0 +1,37 @@ +[package] +name = "lance-graph-callcenter" +version = "0.1.0" +edition = "2021" +description = "External callcenter membrane over Lance + DataFusion (Supabase-shape API)" + +# NEVER add: reqwest (client HTTP; we are the server), postgres, sqlx, pg-embed. +# All external HTTP/WS/SQL is replaced by Lance + DataFusion + axum. + +[dependencies] +lance-graph-contract = { path = "../lance-graph-contract" } + +# [persist] — Arrow RecordBatch + Lance dataset ops +arrow = { version = "57", optional = true } +lance = { version = "2", optional = true } + +# [query] — CommitFilter → DataFusion Expr translator +datafusion = { version = "51", optional = true } + +# [realtime] — version watcher, Phoenix channel shapes, WebSocket +tokio = { version = "1", features = ["sync"], optional = true } +tokio-tungstenite = { version = "0.24", optional = true } +serde = { version = "1", features = ["derive"], optional = true } +serde_json = { version = "1", optional = true } + +# [serve] — axum WS server (implies realtime + query) +axum = { version = "0.8", features = ["ws"], optional = true } +tower-http = { version = "0.5", features = ["cors"], optional = true } + +[features] +default = [] +persist = ["dep:arrow", "dep:lance"] +query = ["dep:datafusion", "dep:arrow"] +realtime = ["dep:tokio", "dep:tokio-tungstenite", "dep:serde", "dep:serde_json"] +serve = ["realtime", "query", "dep:axum", "dep:tower-http"] +auth = ["dep:serde", "dep:serde_json"] +full = ["persist", "query", "realtime", "serve", "auth"] diff --git a/crates/lance-graph-callcenter/src/dn_path.rs b/crates/lance-graph-callcenter/src/dn_path.rs new file mode 100644 index 00000000..e7404a40 --- /dev/null +++ b/crates/lance-graph-callcenter/src/dn_path.rs @@ -0,0 +1,130 @@ +//! DN-addressed path — the URL hierarchy that becomes a scent. +//! +//! `tree/{ns}/heel/{h}/hip/{h}/branch/{b}/twig/{t}/leaf/{l}` +//! +//! Each segment is a u64 FNV-1a hash of the path component. The 6-tuple +//! compresses via ZeckBF17→Base17→CAM-PQ→scent (1B, ρ=0.937). +//! Phase C wires the full compression chain. Phase A carries the +//! parsed address and a stub scent derived by XOR-folding the 6 hashes. +//! +//! Plan: `.claude/plans/callcenter-membrane-v1.md` § 10.12 – § 10.13 + +/// Parsed DN address — 6 u64 segment hashes. +/// +/// The address IS the context key: the same 6-tuple routes, retrieves +/// similar entries (CAM-PQ), and pulls the AriGraph subgraph (scent). +/// See plan § 10.13 for the four uses of one compressed object. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub struct DnPath { + pub ns: u64, + pub heel: u64, + pub hip: u64, + pub branch: u64, + pub twig: u64, + pub leaf: u64, +} + +impl DnPath { + /// Parse from a slash-separated path string. + /// + /// Expected form: `tree/{ns}/heel/{h}/hip/{h}/branch/{b}/twig/{t}/leaf/{l}` + /// Returns `None` if the path does not match the 12-segment pattern. + pub fn parse(path: &str) -> Option { + let segs: Vec<&str> = path.trim_start_matches('/').split('/').collect(); + if segs.len() < 12 { return None; } + if segs[0] != "tree" || segs[2] != "heel" || segs[4] != "hip" + || segs[6] != "branch" || segs[8] != "twig" || segs[10] != "leaf" + { return None; } + Some(Self { + ns: fnv1a(segs[1]), + heel: fnv1a(segs[3]), + hip: fnv1a(segs[5]), + branch: fnv1a(segs[7]), + twig: fnv1a(segs[9]), + leaf: fnv1a(segs[11]), + }) + } + + /// Phase-A scent stub: XOR-fold of the 6 segment hashes into 1 byte. + /// + /// Phase C replaces this with the full ZeckBF17→Base17→CAM-PQ chain + /// (16Kbit → 48B → 34B → 6B → 1B, ρ=0.937). Until then, this gives + /// a deterministic, stable placeholder that exercises the scent field. + pub fn scent_stub(&self) -> u8 { + let fold = self.ns ^ self.heel ^ self.hip ^ self.branch ^ self.twig ^ self.leaf; + (fold + ^ (fold >> 8) + ^ (fold >> 16) + ^ (fold >> 24) + ^ (fold >> 32) + ^ (fold >> 40) + ^ (fold >> 48) + ^ (fold >> 56)) as u8 + } +} + +/// FNV-1a 64-bit — zero-dep, stable for the same input string. +fn fnv1a(s: &str) -> u64 { + let mut h: u64 = 14695981039346656037; + for b in s.bytes() { + h ^= b as u64; + h = h.wrapping_mul(1099511628211); + } + h +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn parse_full_dn_path() { + let p = DnPath::parse( + "/tree/ada/heel/callcenter/hip/v1/branch/agents/twig/card/leaf/abc", + ) + .unwrap(); + assert_ne!(p.ns, 0); + assert_ne!(p.leaf, 0); + } + + #[test] + fn parse_rejects_short_path() { + assert!(DnPath::parse("/tree/ada/heel/callcenter").is_none()); + } + + #[test] + fn parse_rejects_wrong_keywords() { + // "hip" replaced by "HIP" — case-sensitive + assert!(DnPath::parse( + "/tree/ada/heel/cc/HIP/v1/branch/agents/twig/card/leaf/abc" + ) + .is_none()); + } + + #[test] + fn scent_stub_is_deterministic() { + let p1 = DnPath::parse( + "/tree/ada/heel/callcenter/hip/v1/branch/agents/twig/card/leaf/abc", + ) + .unwrap(); + let p2 = DnPath::parse( + "/tree/ada/heel/callcenter/hip/v1/branch/agents/twig/card/leaf/abc", + ) + .unwrap(); + assert_eq!(p1.scent_stub(), p2.scent_stub()); + } + + #[test] + fn different_paths_typically_differ() { + let p1 = DnPath::parse( + "/tree/ada/heel/callcenter/hip/v1/branch/agents/twig/card/leaf/abc", + ) + .unwrap(); + let p2 = DnPath::parse( + "/tree/ada/heel/callcenter/hip/v1/branch/agents/twig/card/leaf/xyz", + ) + .unwrap(); + // leaf differs → scent should differ (not guaranteed but very likely for FNV-1a) + assert_ne!(p1.leaf, p2.leaf); + } +} diff --git a/crates/lance-graph-callcenter/src/external_intent.rs b/crates/lance-graph-callcenter/src/external_intent.rs new file mode 100644 index 00000000..d9d8ab1a --- /dev/null +++ b/crates/lance-graph-callcenter/src/external_intent.rs @@ -0,0 +1,132 @@ +//! External intent and cognitive event row — the two BBB crossing types. +//! +//! - `ExternalIntent`: inbound — what an external consumer deposits at the gate. +//! - `CognitiveEventRow`: outbound — scalar-only projection of a committed cycle. +//! +//! Both types carry ONLY Arrow-scalar-compatible primitives (`u8`, `u16`, `u64`, +//! `bool`, `Vec`). VSA types, semiring types, and `[u64; 256]` fingerprints +//! never appear here — that is the BBB invariant, enforced at compile time. +//! +//! Plan: `.claude/plans/callcenter-membrane-v1.md` § 4 + § 10.9 + § 10.11 + +use lance_graph_contract::{ + external_membrane::{ExternalEventKind, ExternalRole}, + persona::RoutingHint, +}; + +use crate::dn_path::DnPath; + +/// External consumer intent deposited at the callcenter gate. +/// +/// All fields are Arrow-scalar-compatible. No VSA types, no semiring types. +/// The `body` bytes are the raw seed payload — they cross the gate as opaque +/// bytes and are interpreted as a seed INSIDE the substrate after the membrane +/// translates them into a `UnifiedStep`. +/// +/// # BBB Invariant (§ 10.9 iron rule) +/// +/// 1. Pass membrane — this struct IS the membrane crossing shape. +/// 2. Get a role — `role: ExternalRole` stamped at construction. +/// 3. Get a place — `dn: DnPath` is the deterministic address. +/// 4. Translate — `LanceMembrane::ingest()` converts this to `UnifiedStep`. +#[derive(Clone, Debug)] +pub struct ExternalIntent { + /// Which external family is sending this event. + pub role: ExternalRole, + /// DN-addressed location (URL path parsed to 6 u64 hashes). + pub dn: DnPath, + /// Raw seed payload — opaque bytes, interpreted inside the substrate. + pub body: Vec, + /// Optional explicit routing to a specific card or family. + pub routing: RoutingHint, + /// Whether this triggers a new reasoning cycle, passive context, or outbound commit. + pub kind: ExternalEventKind, +} + +impl ExternalIntent { + pub fn seed(role: ExternalRole, dn: DnPath, body: Vec) -> Self { + Self { + role, + dn, + body, + routing: RoutingHint::default(), + kind: ExternalEventKind::Seed, + } + } + + pub fn context(role: ExternalRole, dn: DnPath, body: Vec) -> Self { + Self { + role, + dn, + body, + routing: RoutingHint::default(), + kind: ExternalEventKind::Context, + } + } +} + +/// Scalar-only projection of one committed `ShaderBus` cycle. +/// +/// This is `LanceMembrane::Commit` — the type that crosses the BBB outbound. +/// Every field is an Arrow-scalar primitive (`u8`, `u16`, `u64`, `bool`). +/// +/// # BBB Invariant +/// +/// The cycle fingerprint (`[u64; 256]`, 2 KB) is NOT carried here. Its full +/// VSA state lives stack-side. Only two scalar identity words cross the gate: +/// `cycle_fp_hi = fingerprint[0]` and `cycle_fp_lo = fingerprint[255]`. +/// Together they identify the cycle without carrying the full binding. +/// +/// Why this is compile-time safe: `CognitiveEventRow` contains no type that +/// fails `Send` or that would require `unsafe impl`. Arrow's `Array` trait +/// requires concrete primitive types — `Vsa10k`, `RoleKey`, and `[u64; 256]` +/// cannot be Arrow array elements without wrapping as `FixedSizeBinary`, which +/// is a deliberate re-encoding step, not an accidental leak. +/// +/// # Schema (§ 4 of plan) +/// +/// | Column | Arrow type | Source | +/// |---------------|------------|---------------------------------| +/// | external_role | UInt8 | ExternalRole discriminant | +/// | faculty_role | UInt8 | FacultyRole discriminant | +/// | expert_id | UInt16 | stable_hash_u16(card_yaml) | +/// | dialect | UInt8 | query dialect (Phase B) | +/// | scent | UInt8 | CAM-PQ 1-byte key (Phase C) | +/// | thinking | UInt8 | MetaWord::thinking() | +/// | awareness | UInt8 | MetaWord::awareness() | +/// | nars_f | UInt8 | MetaWord::nars_f() | +/// | nars_c | UInt8 | MetaWord::nars_c() | +/// | free_e | UInt8 | MetaWord::free_e() | +/// | cycle_fp_hi | UInt64 | fingerprint[0] | +/// | cycle_fp_lo | UInt64 | fingerprint[255] | +/// | gate_commit | Boolean | GateDecision::is_flow() | +/// | gate_f | UInt8 | free_e (redundant, for queries) | +/// | rationale_phase| Boolean | true = Stage 1 rationale; false = Stage 2 answer | +/// +/// `rationale_phase` surfaces the MM-CoT stage split: +/// `FacultyDescriptor.inbound_style` (Stage 1) vs `outbound_style` (Stage 2). +/// Phase A: always false (single-stage emission). Phase B: wired from the +/// faculty dispatcher when `FacultyDescriptor::is_asymmetric()` is true. +#[derive(Clone, Debug, Default)] +pub struct CognitiveEventRow { + // ── Identity columns (§ 4 schema, § 10.11 metadata address bus) ── + pub external_role: u8, + pub faculty_role: u8, + pub expert_id: u16, + pub dialect: u8, // Phase B: set by polyglot front-end parser + pub scent: u8, // Phase C: full ZeckBF17→Base17→CAM-PQ chain + // ── MetaWord fields ── + pub thinking: u8, + pub awareness: u8, + pub nars_f: u8, + pub nars_c: u8, + pub free_e: u8, + // ── Cycle identity (NOT the full 2 KB fingerprint) ── + pub cycle_fp_hi: u64, // fingerprint[0] — first word of cycle signature + pub cycle_fp_lo: u64, // fingerprint[255] — last word of cycle signature + // ── Gate outcome ── + pub gate_commit: bool, // true = F < 0.2 (Flow); false = Hold or Block + pub gate_f: u8, // free_e at gate time (for SQL filter `WHERE gate_f < 50`) + // ── MM-CoT stage (DU-4, plan unified-integration-v1.md § DU-4) ── + pub rationale_phase: bool, // Phase B: true = inbound_style stage (rationale) +} diff --git a/crates/lance-graph-callcenter/src/lance_membrane.rs b/crates/lance-graph-callcenter/src/lance_membrane.rs new file mode 100644 index 00000000..eebf13c3 --- /dev/null +++ b/crates/lance-graph-callcenter/src/lance_membrane.rs @@ -0,0 +1,270 @@ +//! LanceMembrane — the `ExternalMembrane` implementation. +//! +//! This is the compile-time BBB enforcement point. It is the only place +//! in the system where external consumer traffic and internal cognitive +//! state meet. After crossing here, external intent becomes a `UnifiedStep` +//! and committed cycles become `CognitiveEventRow` scalars — never the +//! reverse. +//! +//! # Phase map +//! +//! - Phase A (this file): BBB spine. Types correct. Scent is a stub. +//! Subscription is a disconnected `mpsc::Receiver`. +//! - Phase B: `dialect` field populated by polyglot front-end parsers. +//! - Phase C: `scent` replaced by full ZeckBF17→Base17→CAM-PQ cascade. +//! - Phase D: Subscription wired to `tokio::sync::watch` on Lance version +//! counter; `CommitFilter` applied per-subscriber. +//! +//! # UNKNOWN-1 resolution +//! +//! `ShaderSink` in `cognitive-shader-driver` is the internal BindSpace +//! ingestion path — it processes cycle fingerprints stack-side. There is +//! NO overlap with `ExternalMembrane`: `ShaderSink` never crosses the BBB; +//! `ExternalMembrane` is the gate. The two traits compose without conflict. +//! +//! Plan: `.claude/plans/callcenter-membrane-v1.md` § 10.9 – § 11 + +use std::sync::{ + atomic::{AtomicU64, Ordering}, + mpsc, RwLock, +}; + +use lance_graph_contract::{ + a2a_blackboard::ExpertId, + cognitive_shader::{MetaWord, ShaderBus}, + external_membrane::{CommitFilter, ExternalEventKind, ExternalMembrane}, + faculty::FacultyRole, + orchestration::{StepStatus, UnifiedStep}, +}; + +use crate::external_intent::{CognitiveEventRow, ExternalIntent}; + +/// The Blood-Brain Barrier enforcement point. +/// +/// One `LanceMembrane` per session. All interior state is protected by +/// `RwLock` or atomics so it is `Send + Sync`. +/// +/// `current_role`, `current_faculty`, `current_expert` capture the last +/// `ingest()` context so that the next `project()` call stamps the correct +/// identity columns on the emitted `CognitiveEventRow`. +pub struct LanceMembrane { + current_role: RwLock, // ExternalRole discriminant + current_faculty: RwLock, // FacultyRole discriminant + current_expert: RwLock, + current_scent: AtomicU64, + version: AtomicU64, +} + +impl LanceMembrane { + pub fn new() -> Self { + Self { + current_role: RwLock::new(0), + current_faculty: RwLock::new(FacultyRole::ReadingComprehension as u8), + current_expert: RwLock::new(0), + current_scent: AtomicU64::new(0), + version: AtomicU64::new(0), + } + } + + /// Current Lance version counter (monotonic; ticks on each CollapseGate + /// Persist). Phase D wires this to the Lance dataset version. + pub fn version(&self) -> u64 { + self.version.load(Ordering::Acquire) + } +} + +impl Default for LanceMembrane { + fn default() -> Self { + Self::new() + } +} + +// ───────────────────────────────────────────────────────────────────────────── +// ExternalMembrane impl — the BBB +// ───────────────────────────────────────────────────────────────────────────── + +impl ExternalMembrane for LanceMembrane { + /// Scalar projection of a committed cycle. + /// + /// Only Arrow-scalar primitives here. `[u64; 256]` fingerprint lives + /// in `bus` — this type carries two representative words, not the tensor. + type Commit = CognitiveEventRow; + + /// External consumer intent entering through the gate. + type Intent = ExternalIntent; + + /// Phase-A subscription stub: a disconnected `mpsc::Receiver`. + /// + /// Callers that `recv()` on this immediately get `Err(RecvError)`. + /// Phase D replaces this with a `tokio::sync::watch::Receiver` + /// wired to the Lance version counter, filtered by `CommitFilter`. + type Subscription = mpsc::Receiver; + + /// Project a committed ShaderBus cycle to a scalar row. + /// + /// Strips all VSA state. Emits only Arrow-scalar-compatible fields. + /// Called on every `CollapseGate` fire with `EmitMode::Persist`. + fn project(&self, bus: &ShaderBus, meta: MetaWord) -> CognitiveEventRow { + let role = *self.current_role.read().expect("role poisoned"); + let faculty = *self.current_faculty.read().expect("faculty poisoned"); + let expert = *self.current_expert.read().expect("expert poisoned"); + let scent = self.current_scent.load(Ordering::Relaxed) as u8; + + CognitiveEventRow { + external_role: role, + faculty_role: faculty, + expert_id: expert, + dialect: 0, // Phase B: populated by polyglot front-end + scent, + thinking: meta.thinking(), + awareness: meta.awareness(), + nars_f: meta.nars_f(), + nars_c: meta.nars_c(), + free_e: meta.free_e(), + // Two scalar words from the fingerprint — cycle identity without + // the full 2 KB VSA tensor. The tensor lives only in ShaderBus. + cycle_fp_hi: bus.cycle_fingerprint[0], + cycle_fp_lo: bus.cycle_fingerprint[255], + gate_commit: bus.gate.is_flow(), + gate_f: meta.free_e(), + rationale_phase: false, // Phase B: wired from FacultyDescriptor::is_asymmetric() + } + } + + /// Translate external intent to canonical dispatch. + /// + /// Four-step BBB crossing: + /// 1. Pass membrane — `ExternalIntent` is the safe crossing type. + /// 2. Get a role — `intent.role` is stamped into `current_role`. + /// 3. Get a place — `intent.dn.scent_stub()` becomes `current_scent`. + /// 4. Translate — returns `UnifiedStep` for `OrchestrationBridge::route()`. + fn ingest(&self, intent: ExternalIntent) -> UnifiedStep { + // 2. Role + *self.current_role.write().expect("role poisoned") = intent.role as u8; + + // 3. Place (Phase A: XOR-fold stub; Phase C: full cascade) + let scent = intent.dn.scent_stub(); + self.current_scent.store(scent as u64, Ordering::Relaxed); + + // 4. Translate to step type for OrchestrationBridge + let step_type = match intent.kind { + ExternalEventKind::Seed => "lg.blackboard.seed", + ExternalEventKind::Context => "lg.blackboard.context", + ExternalEventKind::Commit => "lg.blackboard.commit", + }; + + UnifiedStep { + step_id: format!("{:016x}", scent as u64), + step_type: step_type.to_owned(), + status: StepStatus::Pending, + thinking: None, // resolved by OrchestrationBridge::resolve_thinking() + reasoning: None, + confidence: None, + } + } + + /// Subscribe to projected commits matching the filter. + /// + /// Phase A: returns a disconnected channel (recv immediately errors). + /// Phase D: wires to `tokio::sync::watch` + `CommitFilter` predicate. + fn subscribe(&self, _filter: CommitFilter) -> mpsc::Receiver { + let (_tx, rx) = mpsc::channel(); + rx + } +} + +// ───────────────────────────────────────────────────────────────────────────── +// Tests +// ───────────────────────────────────────────────────────────────────────────── + +#[cfg(test)] +mod tests { + use super::*; + use lance_graph_contract::{ + cognitive_shader::ShaderBus, + external_membrane::ExternalRole, + }; + use crate::dn_path::DnPath; + + fn make_dn() -> DnPath { + DnPath::parse("/tree/ada/heel/callcenter/hip/v1/branch/agents/twig/card/leaf/abc") + .unwrap() + } + + #[test] + fn ingest_sets_role_and_scent() { + let m = LanceMembrane::new(); + let intent = ExternalIntent::seed( + ExternalRole::CrewaiAgent, + make_dn(), + b"hello world".to_vec(), + ); + let step = m.ingest(intent); + assert_eq!(step.step_type, "lg.blackboard.seed"); + assert_eq!(step.status, StepStatus::Pending); + // scent was set + assert_ne!(m.current_scent.load(Ordering::Relaxed), 0); + // role was stamped + assert_eq!(*m.current_role.read().unwrap(), ExternalRole::CrewaiAgent as u8); + } + + #[test] + fn project_emits_scalar_row() { + let m = LanceMembrane::new(); + // Prime role context via ingest + let intent = ExternalIntent::seed( + ExternalRole::Rag, + make_dn(), + vec![], + ); + m.ingest(intent); + + let bus = ShaderBus::empty(); + let meta = MetaWord::new(5, 3, 200, 150, 10); + let row = m.project(&bus, meta); + + assert_eq!(row.external_role, ExternalRole::Rag as u8); + assert_eq!(row.thinking, 5); + assert_eq!(row.nars_f, 200); + assert_eq!(row.nars_c, 150); + assert_eq!(row.free_e, 10); + // Fingerprint words from ShaderBus::empty() are 0 + assert_eq!(row.cycle_fp_hi, 0); + assert_eq!(row.cycle_fp_lo, 0); + // Empty bus has HOLD gate (gate=2, not Flow) + assert!(!row.gate_commit); + } + + #[test] + fn context_kind_produces_context_step() { + let m = LanceMembrane::new(); + let intent = ExternalIntent::context( + ExternalRole::N8n, + make_dn(), + b"context payload".to_vec(), + ); + let step = m.ingest(intent); + assert_eq!(step.step_type, "lg.blackboard.context"); + } + + #[test] + fn subscribe_returns_disconnected_receiver() { + let m = LanceMembrane::new(); + let rx = m.subscribe(CommitFilter::default()); + // Phase A: disconnected — no sender kept, recv errors immediately + assert!(rx.try_recv().is_err()); + } + + #[test] + fn bbb_scalar_only_compile_check() { + // This is the compile-time BBB proof: CognitiveEventRow contains + // only u8/u16/u64/bool. It implements Send + Sync trivially. + // If someone added a [u64; 256] field here, the type would still + // be Send, but it would be a 2 KB copy — visible in code review. + // More critically, Arrow::try_new() would reject it as a column + // element when the [persist] feature is activated in Phase B. + fn assert_send_sync() {} + assert_send_sync::(); + assert_send_sync::(); + } +} diff --git a/crates/lance-graph-callcenter/src/lib.rs b/crates/lance-graph-callcenter/src/lib.rs new file mode 100644 index 00000000..da23ddde --- /dev/null +++ b/crates/lance-graph-callcenter/src/lib.rs @@ -0,0 +1,82 @@ +//! lance-graph-callcenter — External callcenter membrane. +//! +//! Implements the `ExternalMembrane` trait from `lance-graph-contract` +//! with Lance + DataFusion as the storage and query layer. +//! +//! # Architecture (four layers, § 2 of the design plan) +//! +//! ```text +//! A — Canonical internal substrate (untouched, in lance-graph-contract) +//! Vsa10k · BindSpace SoA · CognitiveShader · CollapseGate · AriGraph +//! +//! B — ExternalMembrane trait (in lance-graph-contract, zero-dep) +//! project() · ingest() · subscribe() +//! +//! C — Dual ledger (Lance datasets, [persist] feature) +//! cognitive_event · steering_intent · memory · actor / session +//! +//! D — This crate (LanceMembrane + server, feature-gated) +//! LanceMembrane · CommitFilter→Expr · PhoenixServer · DrainTask +//! JwtMiddleware · PostgRestHandler +//! ``` +//! +//! # Feature Gates +//! +//! - `default = []` — contract re-exports only, zero external deps +//! - `[persist]` — Arrow RecordBatch + Lance dataset ops +//! - `[query]` — DataFusion Expr translator (CommitFilter → Expr) +//! - `[realtime]` — tokio watch, WebSocket, Phoenix channel shapes +//! - `[serve]` — axum WS server (implies realtime + query) +//! - `[auth]` — JWT verify + actor context +//! - `[full]` — all of the above +//! +//! # Open Unknowns (resolve before DM-2+) +//! +//! - UNKNOWN-1: Does `cognitive-shader-driver`'s `ShaderSink` trait overlap +//! with `ExternalMembrane`? Inspect `crates/cognitive-shader-driver/src/` +//! before wiring DM-2. +//! - UNKNOWN-2: Which consumers (n8n-rs / crewai-rust / openclaw) need +//! Phoenix wire protocol vs direct Rust API? +//! - UNKNOWN-3: Does n8n-rs need a pgwire connection? +//! - UNKNOWN-4: Right `actor_id` type — u64 hash or proper identity type? +//! - UNKNOWN-5: Lance dataset path / `LANCE_URI` env var convention. +//! +//! Plan: `.claude/plans/callcenter-membrane-v1.md` + +pub use lance_graph_contract::external_membrane::{CommitFilter, ExternalMembrane}; + +// ── Phase A: BBB spine (DM-2) ──────────────────────────────────────────────── +// UNKNOWN-1 resolved: ShaderSink is internal BindSpace ingestion; no overlap +// with ExternalMembrane. See lance_membrane.rs module doc for details. +pub mod dn_path; +pub mod external_intent; +pub mod lance_membrane; + +pub use dn_path::DnPath; +pub use external_intent::{CognitiveEventRow, ExternalIntent}; +pub use lance_membrane::LanceMembrane; + +// DM-3 — CommitFilter → DataFusion Expr translator ([query] feature) +// #[cfg(feature = "query")] +// pub mod filter; + +// DM-4 — LanceVersionWatcher: tail version counter → Phoenix events ([realtime]) +// #[cfg(feature = "realtime")] +// pub mod version_watcher; + +// DM-5 — PhoenixServer: minimal WS server, Phoenix channel subset ([realtime]) +// #[cfg(feature = "realtime")] +// pub mod phoenix; + +// DM-6 — DrainTask: steering_intent → UnifiedStep → OrchestrationBridge +// pub mod drain; + +// DM-7 — JwtMiddleware + ActorContext → LogicalPlan RLS rewriter ([auth]) +// Resolve UNKNOWN-3 (pgwire?) and UNKNOWN-4 (actor_id type) first. +// #[cfg(feature = "auth")] +// pub mod auth; + +// DM-8 — PostgRestHandler: query-string → DataFusion SQL → Lance → Arrow ([serve]) +// Confirm PostgREST compat is needed before building (§ 8 stop point 4). +// #[cfg(feature = "serve")] +// pub mod postgrest; diff --git a/crates/lance-graph-catalog/Cargo.toml b/crates/lance-graph-catalog/Cargo.toml index 7671a5bb..857ae4cf 100644 --- a/crates/lance-graph-catalog/Cargo.toml +++ b/crates/lance-graph-catalog/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "lance-graph-catalog" -version = "0.5.3" +version = "0.5.4" edition = "2021" license = "Apache-2.0" authors = ["Lance Devs "] diff --git a/crates/lance-graph-contract/src/a2a_blackboard.rs b/crates/lance-graph-contract/src/a2a_blackboard.rs index 799a25f7..f7f8c0ab 100644 --- a/crates/lance-graph-contract/src/a2a_blackboard.rs +++ b/crates/lance-graph-contract/src/a2a_blackboard.rs @@ -18,6 +18,18 @@ // ═══════════════════════════════════════════════════════════════════════════ /// Expert identifier. Opaque to the blackboard. +/// +/// **Convention:** for agent cards (crewai-rust agents, `.claude/agents/` +/// specialists), `ExpertId = stable_hash_u16(card_yaml)`. This collapses +/// internal A2A experts, external agents, and YAML-defined cards into one +/// identity space. `ExternalRole` carries the family (Rag / CrewaiAgent / +/// N8n / …) at the gate; `ExpertId` carries the specific card on the entry. +/// +/// Identity lives in metadata columns (`external_role: UInt8`, `expert_id: +/// UInt16`), not in a packed braid key. Queries over these columns ARE +/// dispatch. VSA binding happens stack-side only — a deterministic metadata → +/// RoleKey slot mapping that never crosses the BBB. See `persona.rs` module +/// docs and plan § 10.6 erratum. pub type ExpertId = u16; /// What an expert can do. @@ -40,6 +52,12 @@ pub enum ExpertCapability { StyleModulation = 6, /// Qualia classification (semantic family assignment). QualiaClassification = 7, + /// External inbound seed — consumer event that triggers a blackboard reasoning cycle. + /// The entry's `result` field carries an opaque seed handle; the DrainTask resolves it. + ExternalSeed = 8, + /// External inbound context — passive consumer event XOR'd into the trajectory bundle + /// without activating a new reasoning cycle. Same Markov ±5 braiding as grammar tokens. + ExternalContext = 9, } /// Expert registration entry. diff --git a/crates/lance-graph-contract/src/external_membrane.rs b/crates/lance-graph-contract/src/external_membrane.rs new file mode 100644 index 00000000..611f33ff --- /dev/null +++ b/crates/lance-graph-contract/src/external_membrane.rs @@ -0,0 +1,129 @@ +//! External membrane boundary — the Blood-Brain Barrier (BBB) contract. +//! +//! `ExternalMembrane` is the typed boundary between the canonical cognitive +//! substrate and the external callcenter surface. It lives in the zero-dep +//! contract crate so consumers can declare they implement it without pulling +//! in Arrow, Lance, axum, or any other heavy dependency. +//! +//! **BBB Invariant (enforced at compile time by the Arrow type system):** +//! +//! `Self::Commit` MUST NOT contain Vsa10k, RoleKey, SemiringChoice, NarsTruth, +//! HammingMin, or any other VSA or semiring type. Those types do not implement +//! Arrow's `Array` trait, so they physically cannot appear in a `RecordBatch` +//! column. The compiler rejects the violation — no runtime check needed. +//! +//! Only Arrow-scalar-compatible primitives cross the barrier: +//! `u8`, `u16`, `u32`, `u64`, `f32`, `bool`, bytes, strings, timestamps. +//! +//! **READ BY:** every session touching the callcenter crate, realtime +//! subscriptions, external API surface, or any BBB-crossing work. +//! +//! Plan: `.claude/plans/callcenter-membrane-v1.md` + +use crate::cognitive_shader::{ShaderBus, MetaWord}; +use crate::orchestration::UnifiedStep; + +// ═══════════════════════════════════════════════════════════════════════════ +// EXTERNAL ROLE TAXONOMY +// ═══════════════════════════════════════════════════════════════════════════ + +/// Who sent or received an external event. +/// +/// Used as the role-bind key when XOR-braiding external events into the +/// Markov trajectory. Same ±5 braiding mechanism as the grammar parser — +/// `RoleKey::bind(payload, role as u16)` at the callcenter impl site. +/// +/// Inbound roles (consumer → substrate): 0–5. +/// Outbound roles (substrate → consumer): 6–7. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[repr(u8)] +pub enum ExternalRole { + // ── Inbound (consumer identity) ── + User = 0, + Consumer = 1, + N8n = 2, + OpenClaw = 3, + CrewaiUser = 4, + CrewaiAgent = 5, + // ── Outbound (substrate identity as seen externally) ── + Rag = 6, // cognitive shader acting as knowledge retriever + Agent = 7, // specific cognitive agent that handled this step +} + +/// Whether an external crossing is a seed, passive context, or outbound commit. +/// +/// - `Seed`: triggers a blackboard reasoning cycle (DrainTask routes to OrchestrationBridge). +/// - `Context`: passive — role-bound and XOR'd into the trajectory bundle without +/// starting a new cycle. The Markov ±5 window absorbs it into the next natural cycle. +/// - `Commit`: projected scalar row leaving the substrate toward an external subscriber. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum ExternalEventKind { + Seed, + Context, + Commit, +} + +// ═══════════════════════════════════════════════════════════════════════════ +// COMMIT FILTER +// ═══════════════════════════════════════════════════════════════════════════ + +/// Scalar-only predicate for filtering projected commits. +/// +/// All fields are Arrow-scalar-compatible (`u64`, `u8`, `bool`). +/// No VSA types. No semiring types. +#[derive(Clone, Debug, Default)] +pub struct CommitFilter { + /// Filter to a specific actor (plain u64 hash for v1; see UNKNOWN-4). + pub actor_id: Option, + /// Upper bound on free_energy (packed u8 from `MetaWord::free_e()`). + pub max_free_energy: Option, + /// Filter to a specific thinking style ordinal. + pub style_ordinal: Option, + /// When `Some(true)`, only committed cycles (not failure cycles). + pub is_commit: Option, +} + +/// The typed boundary between the canonical cognitive substrate and +/// the external callcenter surface. +/// +/// Implementations live in `lance-graph-callcenter` (the `LanceMembrane` +/// struct). This trait is here, in the zero-dep contract crate, so that +/// consumers can depend on the boundary shape without pulling in Arrow, +/// Lance, DataFusion, axum, or any other heavy crate. +/// +/// # Associated Types +/// +/// - `Commit` — Arrow scalar row in the `LanceMembrane` impl. +/// Must NOT contain any VSA or semiring type (see BBB invariant above). +/// - `Intent` — External intent shape entering through the callcenter. +/// Translated into a `UnifiedStep` for `OrchestrationBridge::route()`. +/// - `Subscription` — Handle returned by `subscribe()`; wired to a +/// `tokio::sync::watch` on the Lance version counter in the impl. +pub trait ExternalMembrane: Send + Sync { + /// Projected scalar representation of one committed `ShaderBus` cycle. + type Commit: Send; + + /// External intent shape — the inbound side of the BBB. + type Intent: Send; + + /// Subscription handle returned by `subscribe()`. + type Subscription: Send; + + /// Project a committed cognitive cycle to a scalar commit record. + /// + /// Strips all VSA fields. Produces Arrow scalars only. + /// Called on every `CollapseGate` fire with `EmitMode::Persist`. + fn project(&self, bus: &ShaderBus, meta: MetaWord) -> Self::Commit; + + /// Translate external intent to canonical dispatch. + /// + /// The external consumer's intent enters here and exits as a + /// `UnifiedStep` ready for `OrchestrationBridge::route()`. + fn ingest(&self, intent: Self::Intent) -> UnifiedStep; + + /// Subscribe to projected commits matching the filter. + /// + /// The `LanceMembrane` implementation wires this to a + /// `tokio::sync::watch` on the Lance version counter. + fn subscribe(&self, filter: CommitFilter) -> Self::Subscription; +} diff --git a/crates/lance-graph-contract/src/faculty.rs b/crates/lance-graph-contract/src/faculty.rs new file mode 100644 index 00000000..74a113a3 --- /dev/null +++ b/crates/lance-graph-contract/src/faculty.rs @@ -0,0 +1,101 @@ +//! Faculty — cognitive function identity, the internal counterpart to `ExternalRole`. +//! +//! Where `ExternalRole` answers *"who sent this at the gate"* (external +//! identity), `FacultyRole` answers *"which cognitive function processed this +//! inside the substrate"* (internal identity). Both are `RoleKey`-bindable +//! into the Markov ±5 trajectory; both flow through the same A2A blackboard; +//! unbinding at either key recovers the respective coordinate. +//! +//! ## Asymmetry — inbound vs outbound thinking styles +//! +//! Each faculty has two thinking styles, one per direction: +//! +//! - `inbound_style`: how this faculty INTERPRETS received bundles +//! (unbinding strategy + ScanParams shape). +//! - `outbound_style`: how this faculty EMITS its contribution to the next +//! blackboard round (bind strategy + FieldModulation shape). +//! +//! Reading comprehension may receive `Analytical` and emit `Concise`; empathy +//! may receive `Gentle` and emit `Warm`; reasoning may receive `Systematic` +//! and emit `Precise`. The asymmetry is the point — a faculty is a shaped +//! transducer, not a bidirectional pipe. +//! +//! ## Tools +//! +//! Each faculty declares `ToolAbility` ids — concrete operations it can +//! invoke during its processing (chunk_text, tts, nars_revise, qualia_match, +//! …). The tool registry is consumer-side; the contract only carries the +//! opaque id type so that faculty descriptors can be serialized and compared. +//! +//! ## Composition with other roles +//! +//! The full provenance of a blackboard entry is a 3-coordinate identity: +//! +//! ```text +//! (ExternalRole family, ExpertId card, FacultyRole function) +//! ``` +//! +//! - External: "a CrewaiAgent deposited this seed" +//! - Card: "the family-codec-smith card handled it" +//! - Faculty: "the Reasoning faculty processed it" +//! +//! All three are RoleKey-bindable; unbinding at any coordinate recovers that +//! slice of the provenance. The shader can observe its own faculties as +//! first-class features — `QualiaClassification` can fire on "Reasoning +//! overwhelmed, Empathy idle" and the router rebalances. +//! +//! Plan: `.claude/plans/callcenter-membrane-v1.md` § 10.10 + +use crate::thinking::ThinkingStyle; + +/// Opaque tool-ability identifier. +/// +/// The registry mapping ids to implementations is consumer-side (in +/// lance-graph core or the calling crate). The contract carries the id type +/// only so `FacultyDescriptor` can declare tool lists without pulling in +/// implementation dependencies. +pub type ToolAbility = u16; + +/// Cognitive faculty identity — internal role coordinate. +/// +/// Extend by adding variants. The `#[repr(u8)]` layout makes faculties cheap +/// to pack into the braid key alongside `ExternalRole` and `ExpertId`. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[repr(u8)] +pub enum FacultyRole { + /// Text understanding: chunking, entity extraction, coreference. + ReadingComprehension = 0, + /// Prosody-aware speech generation / analysis. + Voice = 1, + /// Logical composition, NARS revision, semiring derivation. + Reasoning = 2, + /// Qualia matching, resonance scoring, other-as-self modelling. + Empathy = 3, + // Extensible: Memory, Attention, Prosody, Imagination, Planning, … +} + +/// Descriptor: a faculty's identity + asymmetric styles + invocable tools. +/// +/// Typically constructed once at registration and held by the faculty +/// dispatcher. The `tools` slice is `'static` because the tool registry is +/// built at startup, not mutated per request. +#[derive(Clone, Debug)] +pub struct FacultyDescriptor { + pub role: FacultyRole, + /// How this faculty INTERPRETS incoming bundles. + pub inbound_style: ThinkingStyle, + /// How this faculty EMITS its contribution. + pub outbound_style: ThinkingStyle, + /// Concrete tool-ability ids this faculty can invoke. + pub tools: &'static [ToolAbility], +} + +impl FacultyDescriptor { + /// True when inbound and outbound styles differ — the expected case for + /// a genuine transducer. A faculty with symmetric styles is either + /// misconfigured or a pure pass-through. + #[inline] + pub fn is_asymmetric(&self) -> bool { + self.inbound_style != self.outbound_style + } +} diff --git a/crates/lance-graph-contract/src/lib.rs b/crates/lance-graph-contract/src/lib.rs index 7cb1cf78..fe8759a3 100644 --- a/crates/lance-graph-contract/src/lib.rs +++ b/crates/lance-graph-contract/src/lib.rs @@ -53,3 +53,6 @@ pub mod qualia; pub mod world_map; pub mod grammar; pub mod crystal; +pub mod external_membrane; +pub mod persona; +pub mod faculty; diff --git a/crates/lance-graph-contract/src/persona.rs b/crates/lance-graph-contract/src/persona.rs new file mode 100644 index 00000000..c2e4eea5 --- /dev/null +++ b/crates/lance-graph-contract/src/persona.rs @@ -0,0 +1,99 @@ +//! Persona — identity bundle for agent cards participating in the A2A blackboard. +//! +//! A persona is what a consumer (n8n-rs, crewai-rust, openclaw, or an internal +//! YAML card in `.claude/agents/`) looks like from the blackboard's point of +//! view. It bundles: +//! +//! - `ExternalRole`: family at the gate (Rag / CrewaiAgent / N8n / …) +//! - `ExpertEntry`: specific card + capability + trust prior +//! - (consumer-side) AriGraph subgraph: the persona's memory, committed facts, +//! reversal history — lives in `lance-graph::graph::arigraph`, referenced by +//! handle, NOT carried in this zero-dep crate. +//! +//! ## Identity lives in metadata; VSA binding is stack-side (BBB invariant) +//! +//! `PersonaCard` fields are METADATA — typed scalars safely crossing the BBB +//! as Arrow columns on `cognitive_event` rows. They are: +//! +//! - SQL-queryable (`WHERE external_role = X AND expert_id = Y`) +//! - Cypher / GQL matchable (`MATCH (e:Event {external_role: ...})`) +//! - NARS-truth-taggable (attach f,c to persona metadata facts) +//! - Qualia-classifiable (map metadata rows to qualia signatures) +//! +//! VSA binding of these identities happens STACK-SIDE only — when a blackboard +//! entry is processed, the stack deterministically maps metadata values +//! (`external_role: u8`, `expert_id: u16`) to `RoleKey` slot addresses in the +//! 10k-dim VSA substrate. The external surface never sees those slots. +//! The metadata → slot mapping is internal and never crosses the gate. +//! +//! ## Why the AriGraph reference is consumer-side +//! +//! The contract crate stays zero-dep — no Arrow, no Lance, no SPO store. The +//! persona's *identity* lives here (role + card + trust); the persona's +//! *memory* lives where the AriGraph lives (lance-graph core). A consumer +//! that wants to resolve a `PersonaCard` to its AriGraph subgraph does so +//! at the lance-graph boundary, e.g. +//! `arigraph.subgraph_for(persona.entry.id)`. +//! +//! ## Routing: explicit vs implicit +//! +//! An external consumer deposits a seed with an optional `RoutingHint`. The +//! blackboard router reads it as follows: +//! +//! - `RoutingHint { target_role: Some(_), target_card: Some(_) }` — explicit +//! full address: exactly that card in that family gets activated. +//! - `RoutingHint { target_role: Some(_), target_card: None }` — family route: +//! the router picks the best card within that family (by AriGraph resonance +//! against the seed payload). +//! - `RoutingHint { target_role: None, target_card: Some(_) }` — card-only: +//! regardless of family, that specific card handles it. +//! - `RoutingHint::default()` — pure implicit: the router matches the seed's +//! context fingerprint against every registered persona's AriGraph subgraph +//! resonance, activates the top-k. +//! +//! READ BY: every session touching callcenter wiring, agent card registration, +//! consumer routing, or persona integration. +//! +//! Plan: `.claude/plans/callcenter-membrane-v1.md` § 10.6 – § 10.8 + +use crate::a2a_blackboard::{ExpertEntry, ExpertId}; +use crate::external_membrane::ExternalRole; + +/// Identity bundle for a persona participating in the A2A blackboard. +/// +/// The AriGraph subgraph for this persona lives in lance-graph core +/// (`graph::arigraph`); consumers resolve it by `entry.id` at their boundary. +#[derive(Clone, Debug)] +pub struct PersonaCard { + /// Family identity at the gate. + pub role: ExternalRole, + /// Specific card + capability + trust prior. + pub entry: ExpertEntry, +} + +// NOTE: identity is not packed into a single braid key. Role and card are +// separate typed metadata columns; they are addressable independently via +// SQL/Cypher/GQL/NARS/qualia. VSA binding of these identities happens +// stack-side only, via a deterministic metadata→RoleKey slot mapping that +// never crosses the BBB. See module-level docs. + +/// Optional routing hint carried on `ExpertCapability::ExternalSeed` entries. +/// +/// See module-level docs for the four routing modes (explicit-full, +/// family-only, card-only, implicit). +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub struct RoutingHint { + /// If `Some`, restrict activation to this family. + pub target_role: Option, + /// If `Some`, activate this specific card. + pub target_card: Option, +} + +impl RoutingHint { + /// True when no explicit targeting is present — the router must fall back + /// to AriGraph-resonance matching against the seed payload. + #[inline] + pub fn is_implicit(&self) -> bool { + self.target_role.is_none() && self.target_card.is_none() + } +} diff --git a/crates/lance-graph-python/Cargo.toml b/crates/lance-graph-python/Cargo.toml index 8878f7cf..c3bba936 100644 --- a/crates/lance-graph-python/Cargo.toml +++ b/crates/lance-graph-python/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "lance-graph-python" -version = "0.5.3" +version = "0.5.4" edition = "2021" authors = ["Lance Devs "] publish = false diff --git a/crates/lance-graph-python/src/graph.rs b/crates/lance-graph-python/src/graph.rs index b1182bec..477ca2b7 100644 --- a/crates/lance-graph-python/src/graph.rs +++ b/crates/lance-graph-python/src/graph.rs @@ -27,7 +27,8 @@ use lance_graph::{ ast::{DistanceMetric as RustDistanceMetric, GraphPattern, ReadingClause}, CypherQuery as RustCypherQuery, ExecutionStrategy as RustExecutionStrategy, GraphConfig as RustGraphConfig, GraphError as RustGraphError, InMemoryCatalog, - SqlQuery as RustSqlQuery, VectorSearch as RustVectorSearch, + SqlDialect as RustSqlDialect, SqlQuery as RustSqlQuery, + VectorSearch as RustVectorSearch, }; use pyo3::{ exceptions::{PyNotImplementedError, PyRuntimeError, PyValueError}, @@ -59,6 +60,34 @@ impl From for RustExecutionStrategy { } } +/// SQL dialect for generating SQL from Cypher queries +#[pyclass(name = "SqlDialect", module = "lance.graph")] +#[derive(Clone, Copy)] +pub enum SqlDialect { + /// Generic SQL (DataFusion default dialect) + Default, + /// Spark SQL dialect + Spark, + /// PostgreSQL dialect + PostgreSql, + /// MySQL dialect + MySql, + /// SQLite dialect + Sqlite, +} + +impl From for RustSqlDialect { + fn from(dialect: SqlDialect) -> Self { + match dialect { + SqlDialect::Default => RustSqlDialect::Default, + SqlDialect::Spark => RustSqlDialect::Spark, + SqlDialect::PostgreSql => RustSqlDialect::PostgreSql, + SqlDialect::MySql => RustSqlDialect::MySql, + SqlDialect::Sqlite => RustSqlDialect::Sqlite, + } + } +} + /// Distance metric for vector similarity search #[pyclass(name = "DistanceMetric", module = "lance.graph")] #[derive(Clone, Copy)] @@ -494,6 +523,8 @@ impl CypherQuery { /// ---------- /// datasets : dict /// Dictionary mapping table names to Lance datasets + /// dialect : SqlDialect, optional + /// SQL dialect to use. Defaults to SqlDialect.Default (generic DataFusion SQL). /// /// Returns /// ------- @@ -504,7 +535,15 @@ impl CypherQuery { /// ------ /// RuntimeError /// If SQL generation fails - fn to_sql(&self, py: Python, datasets: &Bound<'_, PyDict>) -> PyResult { + #[pyo3(signature = (datasets, dialect=None))] + fn to_sql( + &self, + py: Python, + datasets: &Bound<'_, PyDict>, + dialect: Option, + ) -> PyResult { + let sql_dialect = dialect.map(|d| d.into()); + // Convert datasets to Arrow RecordBatch map let arrow_datasets = python_datasets_to_batches(datasets)?; @@ -513,7 +552,7 @@ impl CypherQuery { // Execute via runtime let sql = RT - .block_on(Some(py), inner_query.to_sql(arrow_datasets))? + .block_on(Some(py), inner_query.to_sql(arrow_datasets, sql_dialect))? .map_err(graph_error_to_pyerr)?; Ok(sql) @@ -1548,6 +1587,7 @@ pub fn register_graph_module(py: Python, parent_module: &Bound<'_, PyModule>) -> let graph_module = PyModule::new(py, "graph")?; graph_module.add_class::()?; + graph_module.add_class::()?; graph_module.add_class::()?; graph_module.add_class::()?; graph_module.add_class::()?; diff --git a/crates/lance-graph/Cargo.toml b/crates/lance-graph/Cargo.toml index 606edb09..10806f9d 100644 --- a/crates/lance-graph/Cargo.toml +++ b/crates/lance-graph/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "lance-graph" -version = "0.5.3" +version = "0.5.4" edition = "2021" license = "Apache-2.0" authors = ["Lance Devs "] @@ -31,7 +31,7 @@ datafusion-expr = "51" datafusion-sql = "51" datafusion-functions-aggregate = "51" futures = "0.3" -lance-graph-catalog = { path = "../lance-graph-catalog", version = "0.5.3" } +lance-graph-catalog = { path = "../lance-graph-catalog", version = "0.5.4" } lance = "2" lance-linalg = "2" lance-namespace = "2" diff --git a/crates/lance-graph/src/lib.rs b/crates/lance-graph/src/lib.rs index 0fdeb687..a75deb01 100644 --- a/crates/lance-graph/src/lib.rs +++ b/crates/lance-graph/src/lib.rs @@ -71,7 +71,7 @@ pub use lance_graph_catalog::{ #[cfg(feature = "unity-catalog")] pub use lance_graph_catalog::{UnityCatalogConfig, UnityCatalogProvider}; pub use lance_vector_search::VectorSearch; -pub use query::{CypherQuery, ExecutionStrategy}; +pub use query::{CypherQuery, ExecutionStrategy, SqlDialect}; pub use sql_query::SqlQuery; #[cfg(feature = "delta")] pub use table_readers::DeltaTableReader; diff --git a/crates/lance-graph/src/query.rs b/crates/lance-graph/src/query.rs index e0c8ccd0..bb455b38 100644 --- a/crates/lance-graph/src/query.rs +++ b/crates/lance-graph/src/query.rs @@ -9,13 +9,68 @@ use crate::config::GraphConfig; use crate::error::{GraphError, Result}; use crate::logical_plan::LogicalPlanner; use crate::parser::parse_cypher_query; +use crate::spark_dialect::build_spark_dialect; use arrow_array::RecordBatch; use arrow_schema::{Field, Schema, SchemaRef}; +use datafusion_sql::unparser::dialect::{ + CustomDialect, DefaultDialect, MySqlDialect, PostgreSqlDialect, SqliteDialect, +}; +use datafusion_sql::unparser::Unparser; use lance_graph_catalog::DirNamespace; use lance_namespace::models::DescribeTableRequest; use std::collections::{HashMap, HashSet}; use std::sync::Arc; +/// SQL dialect to use when generating SQL from Cypher queries. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub enum SqlDialect { + /// Generic SQL (DataFusion default dialect) + #[default] + Default, + /// Spark SQL dialect (backtick quoting, STRING type, EXTRACT, etc.) + Spark, + /// PostgreSQL dialect + PostgreSql, + /// MySQL dialect + MySql, + /// SQLite dialect + Sqlite, +} + +/// Wrapper to hold the concrete dialect type and provide an `Unparser` reference. +pub enum DialectUnparser { + Default(DefaultDialect), + Spark(Box), + PostgreSql(PostgreSqlDialect), + MySql(MySqlDialect), + Sqlite(SqliteDialect), +} + +impl DialectUnparser { + pub fn as_unparser(&self) -> Unparser<'_> { + match self { + DialectUnparser::Default(d) => Unparser::new(d), + DialectUnparser::Spark(d) => Unparser::new(d.as_ref()), + DialectUnparser::PostgreSql(d) => Unparser::new(d), + DialectUnparser::MySql(d) => Unparser::new(d), + DialectUnparser::Sqlite(d) => Unparser::new(d), + } + } +} + +impl SqlDialect { + /// Create a `DialectUnparser` configured for this dialect. + pub fn unparser(&self) -> DialectUnparser { + match self { + SqlDialect::Default => DialectUnparser::Default(DefaultDialect {}), + SqlDialect::Spark => DialectUnparser::Spark(Box::new(build_spark_dialect())), + SqlDialect::PostgreSql => DialectUnparser::PostgreSql(PostgreSqlDialect {}), + SqlDialect::MySql => DialectUnparser::MySql(MySqlDialect {}), + SqlDialect::Sqlite => DialectUnparser::Sqlite(SqliteDialect {}), + } + } +} + /// Normalize an Arrow schema to have lowercase field names. /// /// This ensures that column names in the dataset match the normalized @@ -295,10 +350,10 @@ impl CypherQuery { self.explain_internal(Arc::new(catalog), ctx).await } - /// Convert the Cypher query to a DataFusion SQL string + /// Convert the Cypher query to a SQL string in the specified dialect. /// /// This method generates a SQL string that corresponds to the DataFusion logical plan - /// derived from the Cypher query. It uses the `datafusion-sql` unparser. + /// derived from the Cypher query, using the specified SQL dialect for unparsing. /// /// **WARNING**: This method is experimental and the generated SQL dialect may change. /// @@ -308,16 +363,20 @@ impl CypherQuery { /// /// # Arguments /// * `datasets` - HashMap of table name to RecordBatch (nodes and relationships) + /// * `dialect` - The SQL dialect to use for generating the output SQL. + /// Defaults to `SqlDialect::Default` (generic DataFusion SQL). + /// Use `SqlDialect::Spark` for Spark SQL, `SqlDialect::PostgreSql`, etc. /// /// # Returns - /// A SQL string representing the query + /// A SQL string representing the query in the specified dialect pub async fn to_sql( &self, datasets: HashMap, + dialect: Option, ) -> Result { - use datafusion_sql::unparser::plan_to_sql; use std::sync::Arc; + let dialect = dialect.unwrap_or_default(); let _config = self.require_config()?; // Build catalog and context from datasets using the helper @@ -338,11 +397,15 @@ impl CypherQuery { location: snafu::Location::new(file!(), line!(), column!()), })?; - // Unparse to SQL - let sql_ast = plan_to_sql(&optimized_plan).map_err(|e| GraphError::PlanError { - message: format!("Failed to unparse plan to SQL: {}", e), - location: snafu::Location::new(file!(), line!(), column!()), - })?; + // Unparse to SQL using the specified dialect + let dialect_unparser = dialect.unparser(); + let unparser = dialect_unparser.as_unparser(); + let sql_ast = unparser + .plan_to_sql(&optimized_plan) + .map_err(|e| GraphError::PlanError { + message: format!("Failed to unparse plan to SQL: {}", e), + location: snafu::Location::new(file!(), line!(), column!()), + })?; Ok(sql_ast.to_string()) } @@ -587,12 +650,19 @@ impl CypherQuery { let mut logical_planner = LogicalPlanner::new(config); let logical_plan = logical_planner.plan(&semantic.ast)?; - // Phase 3: BlasGraph compilation - // Create an empty TypedGraph — the caller must provide a populated graph - // for real execution. This wiring proves the pipeline compiles end-to-end. - let node_count = 0; - let graph = TypedGraph::new(node_count); + // Phase 3: BlasGraph compilation (not yet wired to input data) + // Return an explicit error rather than silently producing empty results + // from an unpopulated graph. Remove this guard when dataset loading is wired. + return Err(GraphError::PlanError { + message: "ExecutionStrategy::BlasGraph is not yet wired to input datasets; \ + use ExecutionStrategy::DataFusion for queries against loaded data" + .to_string(), + location: snafu::Location::new(file!(), line!(), column!()), + }); + + #[allow(unreachable_code)] let semiring = HdrSemiring::XorBundle; + let graph = TypedGraph::new(0); match compile_to_blasgraph(&logical_plan, &graph, &semiring) { Ok(matrix) => { @@ -1944,7 +2014,7 @@ mod tests { .unwrap() .with_config(cfg); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); println!("Generated SQL: {}", sql); assert!(sql.contains("SELECT")); diff --git a/crates/lance-graph/tests/test_to_sql.rs b/crates/lance-graph/tests/test_to_sql.rs index 1ba8a8d6..9f39d4c7 100644 --- a/crates/lance-graph/tests/test_to_sql.rs +++ b/crates/lance-graph/tests/test_to_sql.rs @@ -97,7 +97,7 @@ async fn test_to_sql_simple_node_scan() { .unwrap() .with_config(config); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); // Verify SQL contains expected elements assert!( @@ -129,7 +129,7 @@ async fn test_to_sql_with_filter() { .unwrap() .with_config(config); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); // Verify SQL contains filter condition assert!(sql.contains("SELECT"), "SQL should contain SELECT"); @@ -157,7 +157,7 @@ async fn test_to_sql_with_multiple_properties() { .unwrap() .with_config(config); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); // Verify all columns are present assert!(sql.contains("name"), "SQL should contain name"); @@ -187,7 +187,7 @@ async fn test_to_sql_with_relationship() { .unwrap() .with_config(config); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); // Verify SQL contains join let sql_upper = sql.to_uppercase(); @@ -223,7 +223,7 @@ async fn test_to_sql_with_relationship_filter() { .unwrap() .with_config(config); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); // Verify SQL contains filter on relationship property assert!(sql.contains("salary"), "SQL should reference salary"); @@ -246,7 +246,7 @@ async fn test_to_sql_with_order_by() { .unwrap() .with_config(config); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); // Verify SQL contains ORDER BY assert!( @@ -272,7 +272,7 @@ async fn test_to_sql_with_limit() { .unwrap() .with_config(config); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); // Verify SQL contains LIMIT assert!( @@ -298,7 +298,7 @@ async fn test_to_sql_with_distinct() { .unwrap() .with_config(config); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); // Verify SQL is generated successfully // Note: DISTINCT might be optimized away by DataFusion's optimizer in some cases @@ -323,7 +323,7 @@ async fn test_to_sql_with_alias() { .unwrap() .with_config(config); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); // Verify SQL contains aliases assert!( @@ -358,7 +358,7 @@ async fn test_to_sql_complex_query() { .unwrap() .with_config(config); - let sql = query.to_sql(datasets).await.unwrap(); + let sql = query.to_sql(datasets, None).await.unwrap(); // Verify complex query elements assert!(sql.contains("SELECT"), "SQL should contain SELECT"); @@ -390,7 +390,7 @@ async fn test_to_sql_missing_config() { let query = CypherQuery::new("MATCH (p:Person) RETURN p.name").unwrap(); // Note: No config set - let result = query.to_sql(datasets).await; + let result = query.to_sql(datasets, None).await; // Should fail without config assert!(result.is_err(), "to_sql should fail without config"); @@ -413,7 +413,7 @@ async fn test_to_sql_empty_datasets() { .unwrap() .with_config(config); - let result = query.to_sql(datasets).await; + let result = query.to_sql(datasets, None).await; // Should fail with empty datasets assert!(result.is_err(), "to_sql should fail with empty datasets"); diff --git a/python/pyproject.toml b/python/pyproject.toml index 8395c691..76fab74d 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "lance-graph" -version = "0.5.3" +version = "0.5.4" dependencies = [ "pyarrow>=14", "pyyaml>=6.0", diff --git a/python/python/lance_graph/__init__.py b/python/python/lance_graph/__init__.py index 1d463c7d..26731960 100644 --- a/python/python/lance_graph/__init__.py +++ b/python/python/lance_graph/__init__.py @@ -83,6 +83,7 @@ def _load_dev_build() -> ModuleType: SqlQuery = _bindings.graph.SqlQuery SqlEngine = _bindings.graph.SqlEngine ExecutionStrategy = _bindings.graph.ExecutionStrategy +SqlDialect = _bindings.graph.SqlDialect VectorSearch = _bindings.graph.VectorSearch DistanceMetric = _bindings.graph.DistanceMetric @@ -101,6 +102,7 @@ def _load_dev_build() -> ModuleType: "SqlQuery", "SqlEngine", "ExecutionStrategy", + "SqlDialect", "VectorSearch", "DistanceMetric", "DirNamespace",