feat: engraph v2.0 — hybrid search, smart chunking, vault profiles

[devwhodevs]

Summary

engraph v2.0 upgrades the search engine from pure semantic search to a hybrid system with significantly improved relevance and new vault management capabilities.

What's new

• Smart chunking — Break-point scoring algorithm replaces heading-only splitting. Finds optimal split points at headings, code fences, blank lines, and thematic breaks. Code fence protection prevents splitting inside code blocks.
• Docid system — Every indexed file gets a deterministic 6-char hex ID (#abc123) shown in search results for quick reference.
• FTS5 search lane — SQLite FTS5 full-text search runs alongside semantic search. BM25-ranked keyword matching catches exact terms (ticket IDs, names, dates) that embeddings miss.
• RRF fusion engine — Reciprocal Rank Fusion merges semantic + FTS5 results with configurable lane weights. --explain flag shows per-lane score breakdown.
• Vault profiles — engraph init auto-detects vault structure (PARA/Folders/Flat), type (Obsidian/Logseq/Plain), wikilinks, frontmatter, and tags. Writes vault.toml for future configuration.
• Pluggable model layer — ModelBackend trait enables future model swapping. engraph models list/info for model management. Registry ships with known-good models.

New CLI commands

engraph init [path]          # Auto-detect vault and generate vault.toml
engraph configure            # Interactive configuration (placeholder for v2.1)
engraph models list          # List available embedding models
engraph models info <name>   # Show model details
engraph search --explain     # Show per-lane RRF score breakdown

Architecture changes

• 7 modules → 11 modules (docid, fts, fusion, model, profile added)
• SQLite schema adds docid column on files, chunks_fts FTS5 virtual table
• Search pipeline: query → [semantic lane, FTS lane] → RRF fusion → ranked results
• Automatic schema migration for existing databases

Test plan

• 91 unit tests passing (cargo test --lib)
• Clippy clean (cargo clippy -- -D warnings)
• Formatted (cargo fmt --check)
• Integration tests compile (cargo test --test integration --no-run)
• Manual: engraph index ~/vault && engraph search "test query" --explain
• Manual: engraph init ~/vault on an Obsidian vault
• Manual: engraph models list
• Verify existing databases migrate seamlessly (docid column, FTS5 table)

🤖 Generated with Claude Code