Summary
search_graph with semantic_query returns the same result set for any query whatsoever — different phrases, single terms, different languages all return identical hits, in what appears to be alphabetical order. The vector data in the index is fine (verified by computing cosine similarity manually against the SQLite DB); the bug is in the query executor: it returns exactly the nodes that have NO row in node_vectors, which suggests a LEFT JOIN + ORDER BY similarity DESC where the NULL/NaN similarity of vector-less nodes sorts above every real cosine score.
Environment
- v0.8.1, darwin-arm64 (release binary, checksum verified)
- Large TypeScript + Java monorepo (~10k files, 154k nodes after
mode: "moderate" indexing)
- Reproduced identically via
cli one-shot mode AND via the stdio MCP server (tools/call)
Reproduction
codebase-memory-mcp cli index_repository '{"repo_path":"<repo>","mode":"moderate"}'
# Any of these return the SAME hits (JSON keys from a registry.json, alphabetical):
codebase-memory-mcp cli search_graph '{"project":"<slug>","semantic_query":["where is client debt calculated"],"limit":6}'
codebase-memory-mcp cli search_graph '{"project":"<slug>","semantic_query":["debt"],"limit":6}'
codebase-memory-mcp cli search_graph '{"project":"<slug>","semantic_query":["cost basis FIFO when selling stocks"],"limit":6}'
Every call returns (excerpt):
$behaviors_note | apps/.../ui/registry.json
$bulkZipDownload_note | apps/.../ui/registry.json
$candidateExemplar_note| apps/.../ui/registry.json
...
Note the results are query-independent and alphabetically ordered — the classic signature of sorting on NULL/NaN similarity.
Root-cause evidence (from the index DB)
The data side is healthy:
node_vectors: 46,606 rows × 768d int8 — populated.
token_vectors: 18,049 tokens with vector + idf; all my query terms (debt, overdue, balance, fifo, …) are present.
Manual cosine check (Python over the SQLite DB, query vector = mean of its token vectors):
query "client debt overdue balance" vs CcmaSaldoCalculator → cos = 0.9492 (the CORRECT hit)
query "client debt overdue balance" vs $behaviors_note → node has NO row in node_vectors
So the correct ranking exists in the data — CcmaSaldoCalculator (a debt-calculation class) scores 0.949 and should be the top hit. Instead, the executor returns precisely the nodes that are absent from node_vectors (JSON-key nodes from a registry.json, which apparently never get vectors).
Suggested fix
Exclude vector-less nodes from the semantic ranking — INNER JOIN node_vectors (or WHERE nv.vector IS NOT NULL), and/or treat missing similarity as -inf rather than letting NULL/NaN win the sort.
Side note
BM25 (query) and search_code work great on the same index — this report is only about the semantic_query path.
Summary
search_graphwithsemantic_queryreturns the same result set for any query whatsoever — different phrases, single terms, different languages all return identical hits, in what appears to be alphabetical order. The vector data in the index is fine (verified by computing cosine similarity manually against the SQLite DB); the bug is in the query executor: it returns exactly the nodes that have NO row innode_vectors, which suggests aLEFT JOIN+ORDER BY similarity DESCwhere the NULL/NaN similarity of vector-less nodes sorts above every real cosine score.Environment
mode: "moderate"indexing)clione-shot mode AND via the stdio MCP server (tools/call)Reproduction
Every call returns (excerpt):
Note the results are query-independent and alphabetically ordered — the classic signature of sorting on NULL/NaN similarity.
Root-cause evidence (from the index DB)
The data side is healthy:
node_vectors: 46,606 rows × 768d int8 — populated.token_vectors: 18,049 tokens with vector + idf; all my query terms (debt,overdue,balance,fifo, …) are present.Manual cosine check (Python over the SQLite DB, query vector = mean of its token vectors):
So the correct ranking exists in the data —
CcmaSaldoCalculator(a debt-calculation class) scores 0.949 and should be the top hit. Instead, the executor returns precisely the nodes that are absent fromnode_vectors(JSON-key nodes from aregistry.json, which apparently never get vectors).Suggested fix
Exclude vector-less nodes from the semantic ranking —
INNER JOIN node_vectors(orWHERE nv.vector IS NOT NULL), and/or treat missing similarity as-infrather than letting NULL/NaN win the sort.Side note
BM25 (
query) andsearch_codework great on the same index — this report is only about thesemantic_querypath.