feat(node): expose vector + fulltext + hybrid-search APIs to Node.js binding (KMSmcp consumer)

[ryaker]

Why

KMSmcp (the production consumer of SparrowDB's Node.js binding) is hitting a context-injection quality cliff. Audit on 2026-04-12 measured ~22% on-topic injection rate for short queries — Phoenix-related entries dominate unrelated searches because the current ranking is keyword-substring-only and stale high-confidence facts can't be demoted by relevance.

The Rust core fixes for this are tracked in #394 (HNSW vector), #395 (BM25 fulltext), and #396 (RRF hybrid fusion). All three have detailed specs and active worktrees. But none of them ship value to KMSmcp until the Node.js binding exposes the new APIs.

This issue tracks the Node binding work specifically. It blocks on #394 / #395 / #396 stabilizing in the Rust core, but the binding work itself can be staged behind feature flags as each upstream lands.

Current state

• KMSmcp/src/storage/SparrowDBStorage.ts line ~33: comment says "fulltext index requires the Rust API (create_fulltext_index + add_to_fulltext_index). These are not yet exposed to Node.js. Fulltext search falls back to in-process CONTAINS on the content sidecar."
• crates/sparrowdb-node/src/lib.rs has only one mention: `fulltext_pending: HashMap<String, FulltextIndex>` — a placeholder, no actual NAPI methods.
• KMSmcp's search path is entirely in-process JavaScript filtering against a JSON sidecar (`content-index.json`) because the graph node properties truncate strings >7 chars (the SPA bug).

Node.js API surface needed

Vector index (depends on #394)

```typescript
// On the SparrowDB instance (existing class)
db.execute(`CREATE VECTOR INDEX FOR (n:Knowledge) ON (n.embedding) OPTIONS { dimensions: 768, similarity: 'cosine' }`)

// Vector property write — Float32Array round-trips through NAPI as a typed list
db.execute(`MATCH (k:Knowledge {id: $id}) SET k.embedding = $vec`, {
id: 'abc-123',
vec: new Float32Array([0.123, -0.456, ...]) // 768 dims
})

// Similarity query
const result = db.execute(`
MATCH (n:Knowledge)
WHERE vector_similarity(n.embedding, $query_vec) > 0.7
RETURN n.id, n.content, vector_similarity(n.embedding, $query_vec) AS score
ORDER BY score DESC LIMIT 10
`, { query_vec: queryEmbedding })
```

Fulltext index (depends on #395)

```typescript
db.execute(`CREATE FULLTEXT INDEX FOR (n:Knowledge) ON (n.content)`)

// BM25-ranked search
const result = db.execute(`
MATCH (n:Knowledge)
WHERE full_text_search('Knowledge', 'content', $query)
RETURN n.id, bm25_score(n.content, $query) AS score
ORDER BY score DESC LIMIT 20
`, { query: 'corrective tools supersede' })
```

Hybrid RRF (depends on #396)

```typescript
const result = db.execute(`
MATCH (n:Knowledge)
WITH n, reciprocal_rank_fusion(
vector_similarity(n.embedding, $query_vec),
bm25_score(n.content, $query_text)
) AS hybrid_score
RETURN n.id, n.content, hybrid_score
ORDER BY hybrid_score DESC LIMIT 10
`, { query_vec: queryEmbedding, query_text: 'corrective tools supersede' })
```

NAPI type marshalling concerns

1. `Float32Array` → `Value::Vector` — Node's typed arrays need to map cleanly to Rust `Vec` without per-element conversion overhead. PackStream encoding already handles this for Bolt; the NAPI binding needs the same.
2. String truncation (SPA bug) — Vectors must NOT route through the existing string property serialization path. They need a dedicated value type from day one or they'll get mangled like floats currently do.
3. Result row marshalling — `vector_similarity()` returns `Float64`. Confirm the existing Float64 pathway works under NAPI (the existing `confidence` field has to be stored as a string today as a workaround).

KMSmcp-side migration plan (out of scope for this issue but for context)

Once the binding ships:

1. Add `embedding: Float32Array` to `UnifiedKnowledge` and `ContentEntry`
2. Add an Ollama-backed `EmbeddingService` (uses `nomic-embed-text` — already in the local Ollama stack)
3. Embed-on-store hook in `SparrowDBStorage.store()` — best-effort, non-blocking
4. Backfill job for ~5000 existing entries — single batch via Ollama, ~10 minutes
5. Update `SparrowDBStorage.search()` to use hybrid RRF instead of in-process CONTAINS
6. Update `KMSmcp/src/tools/UnifiedSearchTool.ts` to merge SparrowDB hybrid results with Mongo and Mem0
7. Drop the JSON sidecar entirely once vectors live natively in SparrowDB graph properties

Acceptance criteria

• db.execute() accepts `Float32Array` parameters and round-trips them as vector property values
• CREATE VECTOR INDEX and CREATE FULLTEXT INDEX succeed via `db.execute()` from Node
• `vector_similarity()`, `full_text_search()`, `bm25_score()`, `reciprocal_rank_fusion()` callable from Cypher executed via Node binding
• Result rows containing `Vec` properties marshal to `Float32Array` on the Node side without precision loss
• No regression in existing KMSmcp test suite (`cd ~/Dev/KMSmcp && npm test`)
• Smoke test: end-to-end embed → store → vector search → hybrid search → return matching results, against a 100-entry fixture corpus

Performance budget

• Single vector query (768 dims, 5000 nodes, top-10): <50ms
• Fulltext query (5000 nodes): <30ms
• Hybrid RRF query: <80ms (i.e. roughly the sum, not 2x)
• Embedding (Ollama nomic-embed-text, single string): <200ms — outside SparrowDB's control but informs the Node-side embed-on-store strategy

Dependencies

• feat(vector): HNSW vector index + cosine/dot/euclidean similarity search #394 (HNSW vector index) — core implementation
• feat(fts): BM25 full-text index + CONTAINS/STARTS WITH acceleration #395 (BM25 fulltext) — core implementation
• feat(hybrid-search): RRF fusion of vector + BM25 results in single Cypher query #396 (RRF hybrid fusion) — core implementation

This issue can begin as soon as #394 stabilizes — vector exposure can ship first, then fts, then hybrid. Doesn't have to be one big PR.

Consumer reference

Production consumer: https://github.com/ryaker/KMSmcp

• Storage layer: `src/storage/SparrowDBStorage.ts`
• Routing: `src/tools/UnifiedSearchTool.ts`
• Context injection hook: `~/.claude/hooks/kms-context-inject.sh` (calls `unified_search`)
• Audit baseline (KMS entry id `0565ef46`): 22% on-topic rate, 6/9 results dominated by stale Phoenix entries
• Improvement plan (KMS entry id `6737fe3a`): supersede wrongs → project-scope filter → LLM re-ranker → semantic embedding (this issue is the foundation for steps 3 and 4)

🤖 Filed by Claude on behalf of @ryaker after audit of KMSmcp injection quality 2026-04-12