@@ -83,7 +83,7 @@ which captures everything needed to search, understand, and locate it without re
8383What a ` CodeSymbol ` carries:
8484
8585** name / symbol_type / language** — These uniquely describe what kind of thing this is (save,
86- method, java) so retrieval can filter by language or type before even looking at embeddings .
86+ method, java), and are stored on the point so results can be displayed and grouped by language or type.
8787
8888** signature** — The declaration line only, e.g. * def save(self, db: Session) -> User* . This is what you'd see in an
8989IDE's autocomplete popup — compact enough to show in search results without including the full body.
@@ -173,7 +173,7 @@ docstring and the full signature. Finally, the raw source body is appended, capp
173173tokens). The goal is to give the embedding model everything it would need to understand the symbol's role, not just
174174its implementation.
175175The fields that are useful for * displaying* results (like ` start_line ` , ` end_line ` , ` file_path ` , ` signature ` , ` source ` )
176- or * filtering* them (like ` language ` , ` service ` , ` symbol_type ` ) are stored separately as the Qdrant ** payload** —
176+ or * filtering* them (like ` service ` ) are stored separately as the Qdrant ** payload** —
177177they sit next to the vector but are never embedded.
178178
179179How does ** semcode** build the sparse input?
@@ -223,8 +223,9 @@ Payload is a JSON object with the following fields:
223223
224224- ** Identity & filtering** — ` symbol_name ` , ` symbol_type ` , ` language ` , ` service ` ,
225225 ` file_path ` , ` package ` , ` parent_name ` . These uniquely place the symbol in
226- the repo, and three of them — ` language ` , ` service ` , ` symbol_type ` — are
227- wired as active query-time filters.
226+ the repo. Only one of them — ` service ` — is wired as an active query-time
227+ filter on semantic search; the others are kept on the payload for display,
228+ scoped lookups (e.g. exact-name search), and future use.
228229- ** Display** — ` signature ` , ` source ` , ` docstring ` , ` start_line ` , ` end_line ` ,
229230 ` annotations ` , ` extras ` (HTTP method, route, Spring stereotype). These are
230231 what the MCP client renders back to the user — they are never filtered on,
@@ -241,7 +242,9 @@ symbol — then throw away the ones that don't match.
241242
242243Payload indexes flip this order. ** semcode** indexes six fields — ` language ` , ` service ` , ` symbol_type ` , ` chunk_tier ` ,
243244` parent_name ` , ` file_path ` — so Qdrant can narrow the candidate set * before* any vector math happens. The
244- vector search then runs only over the matching symbols, not the whole collection.
245+ vector search then runs only over the matching symbols, not the whole collection. In practice the semantic search
246+ path only filters on ` service ` ; the other indexes still pay off for direct symbol lookups and the incremental
247+ reindex flow, which scrolls the collection by ` service ` and ` file_path ` .
245248
246249### A second, simpler collection
247250
@@ -324,8 +327,8 @@ it requires rethinking every layer of the pipeline, from how you chunk (by symbo
324327to how you embed (rich context for dense vectors, exact tokens for sparse vectors) to how you store
325328(named vectors with a payload that carries as much signal as the vectors themselves). Hybrid
326329dense+sparse retrieval with server-side RRF bridges the gap between intent-based queries and exact identifier lookups,
327- giving you both in a single round-trip. The payload is half the system: without language, service, and type fields
328- indexed as filters , every search scans the entire collection regardless of how good the vectors are. And without
330+ giving you both in a single round-trip. The payload is half the system: without a ` service ` filter indexed on the
331+ payload , every search scans the entire collection regardless of how good the vectors are. And without
329332incremental indexing via blob SHAs, the embedding cost alone would make continuous reindexing impractical at any serious
330333repository scale. Together these choices form a pipeline that stays accurate, stays fast, and stays affordable as the
331334codebase grows.
0 commit comments