Skip to content

Commit acda642

Browse files
committed
feat(lite-unified): wire additive coexistence feature gate (process, not switch)
Operator constraint: "wire the above but don't deprecate datafusion yet — it's a process, not a switch." - crates/lance-graph/Cargo.toml: add `lite-unified = []` — an ADDITIVE, default-OFF coexistence gate for the future SurrealQL-on-lance query path. Empty for now; datafusion is UNTOUCHED (still a core dep + the default engine, NOT deprecated, NOT made optional). An empty feature gates no code and pulls no deps — cannot affect the default build. - plan: add the load-bearing "Process, not a switch" section — datafusion stays default + is never removed; lite-unified runs the SurrealQL path ALONGSIDE it; promotion is gradual per query-shape (gated on the OQ-LU-2a coverage probe), no cutover, no deprecation milestone. Reframed feature-gate item 3 + increment 4 accordingly; marked Increment 0 (the gate) DONE. The lowering, kv-lance wiring, and any far-future datafusion-optional decision attach to this gate incrementally — none of them are this commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
1 parent 2821a03 commit acda642

2 files changed

Lines changed: 35 additions & 5 deletions

File tree

.claude/plans/lite-unified-surrealql-lance-v1.md

Lines changed: 28 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,19 @@
77
> `lance-graph-contract::action`), `docs/STACK_SCAFFOLD.md`, the
88
> "cold TS + kanban stay Lance-native" ruling.
99
10+
## Process, not a switch (operator constraint, 2026-06-18) — LOAD-BEARING
11+
12+
**datafusion is NOT deprecated and NOT removed.** It stays the **default** query
13+
engine. `lite-unified` is an **additive, default-OFF coexistence gate** (wired as
14+
an empty feature in `crates/lance-graph/Cargo.toml`, datafusion untouched) that
15+
adds the SurrealQL-on-lance path *alongside* datafusion. Promotion is **gradual,
16+
per query-shape** — a shape moves to the SurrealQL path only after the OQ-LU-2a
17+
probe shows SurrealQL covers it; everything else keeps running on datafusion,
18+
indefinitely if need be. There is no flip-the-switch cutover and no
19+
deprecation milestone. The two engines coexist; the feature selects the path
20+
per workload. Any wording below that reads as "drop/replace datafusion" means
21+
*"feature-gate an alternative path,"* never *"remove datafusion."*
22+
1023
## Epiphany (less is more)
1124

1225
Today there are **two query engines over the same lance storage** (lance-graph's
@@ -26,9 +39,13 @@ A `lite-unified` feature that, when ON:
2639
(Cypher/GQL/Gremlin/SPARQL/neo4j) lowers to **SurrealQL** (or the DO-arm
2740
`ActionInvocation`) instead of datafusion SQL. *Missing today:* the
2841
polyglot→SurrealQL lowering (today it's polyglot→datafusion).
29-
3. **datafusion = `optional`, OFF** on this path. Kept behind a separate
30-
`datafusion-analytical` feature for the workloads that genuinely need
31-
vectorized/analytical SQL (joins, aggregations) — SurrealQL's weak spot.
42+
3. **datafusion stays DEFAULT + ON — NOT made optional, NOT deprecated.** Under
43+
`lite-unified` the SurrealQL path runs *alongside* it; a query-shape uses
44+
SurrealQL only once the probe proves coverage, otherwise it stays on
45+
datafusion. Far-future (only if the probe ever shows SurrealQL covers
46+
*everything* a deployment uses, and a deployment opts in): datafusion *could*
47+
become `optional` behind a `datafusion-analytical` feature — but that is a
48+
separate, later, opt-in decision, never part of wiring `lite-unified`.
3249
4. The DO-arm `ExecTarget::SurrealQl` becomes the **primary** exec path, not one
3350
of four.
3451

@@ -66,11 +83,17 @@ is a separate C++ build.
6683
integration; add the `kv-lance` feature + lance dep + `mod lance` in `kvs/mod.rs`).
6784
3. **Polyglot→SurrealQL lowering** — the missing front-end leg (parallel to the
6885
existing polyglot→datafusion).
69-
4. **`datafusion``optional`** + a `datafusion-analytical` feature; default the
70-
common path to SurrealQL-on-lance under `lite-unified`.
86+
4. **Per-shape promotion under the gate** — move covered query-shapes to the
87+
SurrealQL path under `lite-unified`; datafusion stays the default for
88+
everything else. NO `datafusion`→optional flip here (that's a far-future,
89+
separate, opt-in decision — see § Process, not a switch).
7190
5. **Measure** footprint + query-shape coverage; promote CONJECTURE→FINDING or
7291
correct.
7392

93+
**Increment 0 — DONE (this commit):** the `lite-unified` coexistence feature is
94+
wired in `crates/lance-graph/Cargo.toml` (additive, default-OFF, empty, datafusion
95+
untouched). The gate exists; the lowering attaches to it incrementally per above.
96+
7497
## Blockers / open questions
7598

7699
- **OQ-LU-1:** surreal kv-lance feature-wiring (the integration TODOs).

crates/lance-graph/Cargo.toml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,13 @@ planner = ["dep:lance-graph-planner"]
8888
bgz17-codec = ["dep:bgz17"]
8989
# tensor-codec: attention as table lookup, SimilarityTable, Belichtungsmesser, variance audit.
9090
tensor-codec = ["dep:bgz-tensor"]
91+
# lite-unified: ADDITIVE, default-OFF coexistence gate for the SurrealQL-on-lance
92+
# query path (.claude/plans/lite-unified-surrealql-lance-v1.md). PROCESS, NOT A
93+
# SWITCH — datafusion stays the DEFAULT query engine and is NOT deprecated, NOT
94+
# made optional by this feature. `lite-unified` only adds the SurrealQL lowering
95+
# path ALONGSIDE datafusion, promoted gradually per query-shape once OQ-LU-2a
96+
# (the datafusion_planner coverage probe) is green. Empty until the lowering lands.
97+
lite-unified = []
9198

9299
[dev-dependencies]
93100
futures = "0.3"

0 commit comments

Comments
 (0)