Skip to content

Commit d3619f9

Browse files
committed
fix(core): dialect-gate sqlite-vec embeddings purge
The previous fix issued an unconditional `DELETE FROM search_vector_embeddings WHERE rowid IN (...)` during project deletion, which 500'd on Postgres: column "rowid" does not exist Postgres uses `chunk_id` (a real column with an FK to search_vector_chunks.id ON DELETE CASCADE), so it doesn't need or accept the SQLite query at all. The cascade picture by backend: - search_index → project: Postgres has FK CASCADE; SQLite FTS5 virtual table can't carry FKs and needs explicit cleanup. - search_vector_chunks → project: neither backend has an FK, so both need an explicit DELETE. - search_vector_embeddings → search_vector_chunks: Postgres has FK CASCADE on chunk_id; SQLite vec0 virtual table is keyed by rowid with no cascade, so embeddings must be purged before the chunks. Branch is now: chunks DELETE runs on both backends; search_index and the vec0 embeddings DELETE only run on SQLite. Verified locally against both backends (SQLite: 31 passed, Postgres via testcontainers: 26 passed) for tests/services/test_project_removal_bug.py and the CLI/MCP project-management integration suites. Signed-off-by: phernandez <paul@basicmachines.co>
1 parent 2bd6552 commit d3619f9

1 file changed

Lines changed: 30 additions & 21 deletions

File tree

src/basic_memory/repository/project_repository.py

Lines changed: 30 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -125,23 +125,24 @@ async def set_as_default(self, project_id: int) -> Optional[Project]:
125125
async def delete(self, entity_id: int) -> bool:
126126
"""Delete a project and its derived search rows in one transaction.
127127
128-
Postgres carries an ON DELETE CASCADE FK from search_index.project_id to
129-
project.id, so the search rows go with the project automatically there.
130-
SQLite stores search_index as an FTS5 virtual table, which cannot hold
131-
foreign keys — without an explicit purge here the FTS rows survive as
132-
orphans, and a later project that reuses the same auto-increment id
133-
inherits the previous tenant's content. search_vector_chunks is a real
134-
table on both backends but only carries the FK on Postgres.
135-
136-
sqlite-vec stores embeddings in a separate vec0 virtual table keyed by
137-
chunk rowid with no cascade, so embeddings must be purged before the
138-
chunk rows or `_run_vector_query` will keep returning stale vectors
139-
that crowd out live results.
140-
141-
Each derived table — search_index, search_vector_chunks,
142-
search_vector_embeddings — is created lazily on first use, so any of
143-
them may be absent on minimal test DBs or installs without semantic
144-
search. Inspect the connection once and skip whichever is missing.
128+
The cascade picture differs by backend:
129+
130+
- search_index → project: Postgres has ON DELETE CASCADE FK; SQLite
131+
stores search_index as an FTS5 virtual table and can't carry FKs,
132+
so it needs explicit cleanup.
133+
- search_vector_chunks → project: neither backend has an FK here, so
134+
both need an explicit DELETE.
135+
- search_vector_embeddings → search_vector_chunks: Postgres has an FK
136+
(chunk_id REFERENCES … ON DELETE CASCADE); SQLite stores embeddings
137+
in a vec0 virtual table keyed by rowid with no cascade. On SQLite
138+
the embeddings must be purged before the chunk rows, otherwise
139+
`_run_vector_query` keeps returning stale vectors that crowd out
140+
live results.
141+
142+
Each derived table is created lazily (search_index by
143+
SearchRepository.init_search_index, the vector tables once semantic
144+
search initializes), so any of them may be absent on minimal test DBs.
145+
Inspect the connection once and skip whichever is missing.
145146
"""
146147
async with db.scoped_session(self.session_maker) as session:
147148
try:
@@ -152,20 +153,28 @@ async def delete(self, entity_id: int) -> bool:
152153
except NoResultFound:
153154
return False
154155

156+
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
157+
is_sqlite = dialect_name == "sqlite"
158+
155159
existing_tables = await session.run_sync(
156160
lambda sync_session: set(sa_inspect(sync_session.connection()).get_table_names())
157161
)
158162

159-
if "search_index" in existing_tables:
163+
# search_index: SQLite has no FK on the FTS5 virtual table; Postgres
164+
# cascades from the project FK, so the explicit DELETE is redundant.
165+
if is_sqlite and "search_index" in existing_tables:
160166
await session.execute(
161167
text("DELETE FROM search_index WHERE project_id = :project_id"),
162168
{"project_id": entity_id},
163169
)
164170

171+
# search_vector_chunks: no FK to project on either backend, so both
172+
# backends need this. SQLite must purge vec0 embeddings first
173+
# (rowid pseudocolumn — Postgres uses chunk_id and would 500 here);
174+
# Postgres' chunk_id FK CASCADE handles its embeddings cleanup when
175+
# we delete the chunk rows below.
165176
if "search_vector_chunks" in existing_tables:
166-
if "search_vector_embeddings" in existing_tables:
167-
# sqlite-vec has no CASCADE — drop embeddings first while the
168-
# chunk rows that name them still exist.
177+
if is_sqlite and "search_vector_embeddings" in existing_tables:
169178
await session.execute(
170179
text(
171180
"DELETE FROM search_vector_embeddings WHERE rowid IN ("

0 commit comments

Comments
 (0)