v0.6.0: Write pipeline with sqlite-vec migration#7
Merged
devwhodevs merged 20 commits intomainfrom Mar 25, 2026
Merged
Conversation
Wrap sqlite-vec for vector search, replacing HNSW-based approach. Provides init, insert, delete, search (with tombstone filtering), and clear operations on a vec0 virtual table. Includes 5 unit tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All search code paths now use store.search_vec() instead of HnswIndex::search(). The hnsw module remains but is unused — deletion is deferred to Task 5. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove HnswIndex import and HNSW rebuild steps (11-12) - Insert vectors into vec0 table during chunk write loop - Delete from vec0 when files are deleted or changed - Clear vec0 on full rebuild - Use store.next_vector_id() instead of scanning all vectors - Add folder centroid computation and storage after indexing - Add folder_centroids table migration and upsert/get methods in Store Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…sqlite-vec Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds `migrate_vectors_to_vec0()` which copies BLOB vectors from `chunks.vector` into the `chunks_vec` vec0 virtual table. Called from `init()` after `init_vec_table()` so the virtual table is guaranteed to exist. No-ops when vec0 is already populated or no BLOBs are present. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Scans note content for potential wikilink targets using exact filename and alias matching. Supports case-insensitive search, word boundary checking, existing wikilink skipping, and longest-match-first priority. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three-strategy cascade: type-based rules (person/daily/workout + content pattern detection) → semantic centroid matching against precomputed folder embeddings → inbox fallback. 12 tests covering all strategies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…d move Implements the writer module that ties together content analysis, tag resolution, link discovery, folder placement, and atomic write+index. - CreateNoteInput: 5-step pipeline (filename, tags, links, placement, write) - AppendInput: append content with mtime conflict detection - UpdateMetadataInput: frontmatter-only updates without re-chunking - move_note: relocate files with store record updates - All writes use temp+rename for atomicity with transaction rollback - Pre-computes embeddings before holding DB lock - Adds Store::resolve_file() for path/basename/#docid resolution - Adds time crate for date formatting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extends the MCP server with 4 write tools that expose the writer module pipeline to Claude Code clients, completing the read-write tool surface. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds `engraph write create` and `engraph write append` subcommands backed by the writer module pipeline. Both support --content flag or stdin for content input, with --json output mode. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Scans the vault for leftover `.md.tmp` files on both `engraph index` and `engraph serve` startup, removing any that survived a previous crash mid-write. Logs the count if any are removed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three #[ignore] tests covering create_note searchability, append index update, and mtime conflict detection. Run with: cargo test --test write_pipeline -- --ignored Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove redundant tombstone writes from indexer (delete_vec handles it). Replace tombstone loading in search with empty set. Fix clippy warning in writer.rs. Apply cargo fmt across all modules. Bump version to 0.6.0. Update CLAUDE.md with 19 modules, 190 tests, write pipeline docs, and sqlite-vec architecture. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- archive: moves note to 04-Archive/, adds archived frontmatter, removes from index - unarchive: restores to original location (via archived_from), re-indexes - indexer auto-excludes archive folder during walks - MCP tools: archive, unarchive (13 total tools now) - CLI: engraph write archive/unarchive Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…r, incremental centroids, orphan cleanup, tag queries - Gap 1: Add suggestion field to PlacementResult; add ticket ID detection (BRE-XXXX/DRIFT-XXX), meeting note detection, decision type_hint - Gap 2: Inject suggested_folder frontmatter when semantic placement finds a below-threshold match during inbox fallback - Gap 3: Incrementally update folder centroids after each note creation (weighted merge with existing centroid) - Gap 4: Add verify_index_integrity() to clean orphan DB entries for files that no longer exist on disk; called on index and serve startup - Gap 5: Add agent_created_tags(), low_usage_tags(), stale_tags() queries to store for tag hygiene tooling Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
hnsw_rswithsqlite-vecfor vector search — all data (metadata, chunks, vectors, FTS5, edges) unified in a single SQLite database. DB size dropped from 18MB to 9.6MB. HNSW rebuild eliminated — vectors are immediately queryable after insert.create,append,update_metadata,move_note,archive,unarchive[[wikilinks]]04-Archive/, removes from index. Indexer auto-excludes archive folder. Unarchive restores to original location and re-indexes.BEGIN IMMEDIATESQLite transactions + temp-file-then-rename. Mtime conflict detection prevents overwriting external edits. Crash recovery cleans up orphan.tmpfiles on startup.Stats
Test plan
-D warnings)cargo fmt --checkclean🤖 Generated with Claude Code