Skip to content

Commit 480ca22

Browse files
committed
📝 docs(spec): add exploratory v2 design drafts for hybrid retrieval
- Add v2 outline, runtime resurfacing, and config drafts to spec/ - Add embedding storage, conformance scenarios, migration sequencing drafts - Add maintainer v2 hybrid retrieval roadmap and semantic interfaces drafts - Add operator v2 embedding backfill and health draft - Update development tracker with 2026-03-20 session progress - Link all new drafts in respective README indexes
1 parent 5b1127f commit 480ca22

14 files changed

Lines changed: 2881 additions & 1 deletion

docs/go/README.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,8 @@ If you are deploying or operating the MCP server:
2525
- [Client Examples](./operator/client-examples.md)
2626
- [Import Ingestion](./operator/import-ingestion.md)
2727
Batch import, long-lived follow mode, cleanup/audit hygiene commands, and packaged example-catalog lookup.
28+
- [V2 Embedding Backfill And Health Draft](./operator/v2-embedding-backfill-health-draft.md)
29+
Draft operator workflow for semantic backfill, health checks, and rebuild-oriented recovery.
2830
- [Release Readiness](./operator/release-readiness.md)
2931
- [Troubleshooting](./operator/troubleshooting.md)
3032

@@ -35,6 +37,18 @@ If you are maintaining or testing the Go implementation:
3537
- [Development Tracker](./maintainer/development-tracker.md)
3638
- [Development Kickoff](./maintainer/dev-kickoff.md)
3739
- [Implementation Plan](./maintainer/implementation-plan.md)
40+
- [V2 Hybrid Retrieval Roadmap](./maintainer/v2-hybrid-retrieval-roadmap.md)
41+
Early maintainer draft for possible v2 retrieval evolution.
42+
- [V2 Semantic Interfaces Draft](./maintainer/v2-semantic-interfaces-draft.md)
43+
Draft package-level interfaces for embedding providers, semantic indexes, and hybrid retrieval wiring.
44+
- [Spec V2 Runtime Resurfacing Draft](../spec/v2-runtime-resurfacing.md)
45+
Draft runtime algorithm and working-context shaping for implicit memory resurfacing.
46+
- [Spec V2 Embedding Storage Draft](../spec/v2-embedding-storage-draft.md)
47+
Draft storage and backfill design for local-first semantic retrieval.
48+
- [Spec V2 Conformance Scenarios Draft](../spec/v2-conformance-scenarios-draft.md)
49+
Draft scenario matrix for proving lexical fallback, policy guardrails, and bounded resurfacing behavior.
50+
- [Spec V2 Migration Sequencing Draft](../spec/v2-migration-sequencing-draft.md)
51+
Draft rollout order for additive schema changes, sidecar bootstrap, and config-gated hybrid enablement.
3852

3953
## Planning And Spec References
4054

docs/go/maintainer/README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,18 @@ Start here:
1414
Language-neutral execution backlog used by the Go plan.
1515
- [MCP Integration](./mcp-integration.md)
1616
Source-tree MCP smoke tests and transport behavior.
17+
- [V2 Hybrid Retrieval Roadmap](./v2-hybrid-retrieval-roadmap.md)
18+
Early maintainer roadmap for possible v2 retrieval work.
19+
- [V2 Semantic Interfaces Draft](./v2-semantic-interfaces-draft.md)
20+
Draft package and interface boundaries for semantic retrieval implementation.
21+
- [Spec V2 Runtime Resurfacing Draft](../../spec/v2-runtime-resurfacing.md)
22+
Draft trigger, threshold, and working-context rules for implicit memory resurfacing.
23+
- [Spec V2 Embedding Storage Draft](../../spec/v2-embedding-storage-draft.md)
24+
Draft storage, sidecar-index, and backfill direction for semantic retrieval.
25+
- [Spec V2 Conformance Scenarios Draft](../../spec/v2-conformance-scenarios-draft.md)
26+
Draft verification matrix for lexical fallback, hybrid retrieval, degraded semantic states, and resurfacing controls.
27+
- [Spec V2 Migration Sequencing Draft](../../spec/v2-migration-sequencing-draft.md)
28+
Draft rollout order for embedding metadata migration and staged hybrid enablement.
1729

1830
For packaged-binary usage and deployment, see [operator/](../operator/README.md).
1931
For end-user concepts and prompts, see [user/](../user/README.md).

docs/go/maintainer/development-tracker.md

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# codex-mem Go Development Tracker
22

3-
Last updated: 2026-03-19
3+
Last updated: 2026-03-20
44
Status: active
55

66
## Purpose
@@ -149,6 +149,34 @@ Immediate next tasks:
149149

150150
## Decisions Log
151151

152+
### 2026-03-20 Session Update
153+
154+
- Completed: Continued the exploratory v2 documentation set by adding a maintainer-facing semantic-interfaces draft plus a migration-sequencing draft. The interface doc maps the v2 design into the current Go package boundaries, proposes additive retrieval, memory-domain, semantic-index, and runtime-resurfacing interfaces, and keeps `retrieval.Service` on a nil-safe lexical-first path. The migration doc then turns the same design into a rollout order: interfaces first, additive note-embedding metadata migration next, sidecar/backfill tooling after that, notes-only hybrid retrieval later, and implicit resurfacing only after hybrid retrieval itself is stable.
155+
- In progress: none.
156+
- Blockers: none.
157+
- Next step: decide whether to keep extending docs with an implementation-spike plan, or start the first Go slice for additive metadata fields and nil-safe semantic interfaces.
158+
159+
### 2026-03-20 Session Update
160+
161+
- Completed: Continued the exploratory v2 documentation set with two more design artifacts: a v2 conformance-scenarios draft and an operator-facing embedding backfill/health draft. The conformance doc defines the minimum scenario matrix for lexical-only baseline behavior, happy-path hybrid retrieval, degraded semantic states, runtime resurfacing controls, privacy/lifecycle guardrails, and sidecar rebuild recovery. The operator doc turns the storage draft into a rollout sequence, health-state model, expected summary fields, and repair-oriented workflows, while the roadmap and readmes now link both drafts.
162+
- In progress: none.
163+
- Blockers: none.
164+
- Next step: decide whether the next v2 design pass should sketch concrete Go interfaces and migration ordering, or stop design expansion and begin implementation spikes behind config gates.
165+
166+
### 2026-03-20 Session Update
167+
168+
- Completed: Continued the exploratory v2 documentation set by adding a dedicated embedding-storage and backfill draft. The new doc compares direct-in-SQLite, local sidecar, and external-backend shapes; recommends SQLite metadata plus a pluggable local sidecar index for the first rollout; defines minimal embedding lifecycle metadata, explicit one-shot backfill flow, repair semantics, and degraded retrieval rules; and links that design back into the v2 outline, roadmap, and doc indexes.
169+
- In progress: none.
170+
- Blockers: none.
171+
- Next step: decide whether the next v2 draft follow-up should add operator-facing backfill/health guidance or a conformance-scenarios draft for lexical-only, hybrid, and degraded retrieval modes.
172+
173+
### 2026-03-20 Session Update
174+
175+
- Completed: Continued the exploratory v2 documentation set by adding a dedicated runtime resurfacing draft plus a companion v2 config draft. The new docs define a suggested consult-scoring pipeline, prototype confidence thresholds, session-local suppression-cache cooling rules, a shaped working-context payload for implicit memory injection, and a notes-first early-v2 boundary that keeps handoffs lexical-only at first. The v2 outline, roadmap, and doc indexes now link to the new drafts so the design direction is easier to re-enter.
176+
- In progress: none.
177+
- Blockers: none.
178+
- Next step: decide whether the next v2 draft follow-up should focus on embedding storage/backfill design, conformance scenarios for hybrid retrieval, or pause until implementation work begins.
179+
152180
### 2026-03-19 Session Update
153181

154182
- Completed: Added `--fail-on-partial` to `ingest-imports` so operators can keep `--continue-on-error` partial-success behavior, failed-output exports, and failed-manifest exports while still receiving a non-zero exit when any line in the batch failed. The ingest report now surfaces `fail_on_partial`, CLI coverage verifies that the partial report is still written before the command exits with an error, and the reusable `App.IngestImports` path now honors the same flag so embedded callers can receive the same report-plus-error behavior instead of only the CLI getting that control. Operator docs plus the README describe the automation-oriented workflow.

0 commit comments

Comments
 (0)