You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: sync CONFIG/MEMORY/EXTENDED_MEMORY/AGENTS with roadmap changes
- CONFIG.md: add semantic_dedup_threshold and consolidate_similarity_threshold
to the extended config example and field reference
- MEMORY.md: combined session-end extraction call, bounded drain at close
(episode extraction no longer fire-and-forget), quarantine-not-reject
guard behavior with the honest add_atom report, stats in the CLI surface
and observability sections, complete memory tool action enum
- EXTENDED_MEMORY.md: consolidate/stats now have CLI commands
- AGENTS.md: PIGuard E2E and fuzz soak commands in Testing
go test -fuzz=FuzzExtractJSON -fuzztime=30s ./internal/memory/extended/
245
+
go test -fuzz=FuzzParseSkillContent -fuzztime=30s ./internal/skills/
246
+
go test -fuzz=FuzzSessionLoad -fuzztime=30s ./internal/session/
238
247
```
239
248
240
249
Note: MCP client E2E tests build the fakeserver from `internal/mcpclient/testdata/main.go` at test time (no pre-compiled binary). macOS temp dirs are classified as `LocalWrite` (not `SystemWrite`), and the Docker availability check verifies daemon reachability before running sandbox tests.
Copy file name to clipboardExpand all lines: docs/CONFIG.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -375,6 +375,8 @@ The `memory` section controls the persistent memory system (see [docs/MEMORY.md]
375
375
"semantic_search_overfetch": 4,
376
376
"semantic_search_min_score": 0.55,
377
377
"semantic_search_rerank": true,
378
+
"semantic_dedup_threshold": 0.92,
379
+
"consolidate_similarity_threshold": 0.9,
378
380
"atom_max_chars": 300,
379
381
"memory_budget_chars": 2000,
380
382
"decay_half_life_days": 30,
@@ -409,6 +411,8 @@ The `memory` section controls the persistent memory system (see [docs/MEMORY.md]
409
411
|`semantic_search_overfetch`|`4`| — | — | Candidate multiplier before filtering and reranking. |
410
412
|`semantic_search_min_score`|`0.55`| — | — | Minimum cosine similarity for a candidate to be considered. |
411
413
|`semantic_search_rerank`|`true`| — | — | Use the memory LLM to rerank candidates. |
414
+
|`semantic_dedup_threshold`|`0.92`| — | — | Cosine similarity at or above which an incoming atom is treated as a paraphrase of an existing live atom and refreshes it instead of appending. `0` disables the semantic tier (exact-match dedup always runs). |
415
+
|`consolidate_similarity_threshold`|`0.9`| — | — | Pairwise cosine similarity at or above which live atoms are grouped for LLM merging by `odek memory extended consolidate` / `ConsolidateAtoms`. |
412
416
|`atom_max_chars`|`300`|`ODEK_MEMORY_EXTENDED_ATOM_MAX_CHARS`|`--memory-extended-atom-max-chars`| Maximum stored text length per atom. |
413
417
|`memory_budget_chars`|`2000`|`ODEK_MEMORY_EXTENDED_MEMORY_BUDGET_CHARS`|`--memory-extended-memory-budget-chars`| Maximum injected Extended Memory context per turn. |
414
418
|`decay_half_life_days`|`30`| — | — | Days until an atom's recall/eviction weight halves. |
Copy file name to clipboardExpand all lines: docs/EXTENDED_MEMORY.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -174,11 +174,11 @@ Atoms are deduplicated in two tiers at persistence time:
174
174
175
175
### Atom Consolidation
176
176
177
-
`ExtendedMemory.ConsolidateAtoms(ctx)`(no CLI yet) merges groups of live atoms that are near-duplicates (pairwise cosine similarity at or above `consolidate_similarity_threshold`). For each group it asks the memory LLM — one call per group — to merge the texts into a single concise atom, stores the merged atom through the normal add path (scan and size cap apply) keeping the group's highest confidence, a refreshed `CreatedAt`, and the union of the group's outward associations, then removes the originals. Quarantined atoms are never consolidated. On any failure for a group (LLM error, empty response, scan rejection) the originals are kept untouched.
177
+
`ExtendedMemory.ConsolidateAtoms(ctx)`— exposed as `odek memory extended consolidate` — merges groups of live atoms that are near-duplicates (pairwise cosine similarity at or above `consolidate_similarity_threshold`). For each group it asks the memory LLM — one call per group — to merge the texts into a single concise atom, stores the merged atom through the normal add path (scan and size cap apply) keeping the group's highest confidence, a refreshed `CreatedAt`, and the union of the group's outward associations, then removes the originals. Quarantined atoms are never consolidated. On any failure for a group (LLM error, empty response, scan rejection) the originals are kept untouched.
178
178
179
179
### Observability
180
180
181
-
`ExtendedMemory.Stats()` returns a `Stats` snapshot for operators and monitoring: live/quarantined atom counts, quarantine entries grouped by reason prefix (`tainted`, `scan_rejected`), index vector count and dirty flag, on-disk store size, and recall error counters (`RecallTimeouts` for context-deadline-exceeded errors, `RecallFailures` for all other recall errors).
181
+
`ExtendedMemory.Stats()` returns a `Stats` snapshot — surfaced as `odek memory extended stats` — for operators and monitoring: live/quarantined atom counts, quarantine entries grouped by reason prefix (`tainted`, `scan_rejected`), index vector count and dirty flag, on-disk store size, and recall error counters (`RecallTimeouts` for context-deadline-exceeded errors, `RecallFailures` for all other recall errors). The CLI prints a degradation warning when either counter is non-zero.
After sessions with ≥3 turns, the MemoryManager runs SimpleCall to extract 1-3 durable facts. Written to `episodes/<session-id>.md`. Searchable via `memory(search=...)` which uses **RandomProjections** (go-vector) to rank episodes by cosine similarity to the query — zero LLM calls per search. Set `llm_search: true` in config to use LLM-based ranking instead.
48
+
After sessions with ≥3 turns, the MemoryManager extracts a session summary. When both episode extraction (`extract_on_end`) and fact extraction (`extract_facts`) are enabled, a **single combined LLM call** produces the episode summary and the durable facts in one JSON response (falling back to the two single-purpose calls if the combined response is unparseable). Written to `episodes/<session-id>.md`. Searchable via `memory(search=...)` which uses **RandomProjections** (go-vector) to rank episodes by cosine similarity to the query — zero LLM calls per search. Set `llm_search: true` in config to use LLM-based ranking instead.
49
49
50
-
Episode extraction runs **asynchronously** — it does not block process exit. The session summary is a best-effort post-processing step that completes in a background goroutine.
50
+
Episode extraction runs **asynchronously** — it does not block the agent loop. Session-end work is tracked by the MemoryManager and drained with a bounded wait (~15s) in `Agent.Close`, so episodes survive CLI exit without hanging the process.
51
51
52
52
## Memory Tool — Unified API
53
53
@@ -56,7 +56,7 @@ Episode extraction runs **asynchronously** — it does not block process exit. T
56
56
"name": "memory",
57
57
"description": "Manage persistent memory across sessions.",
"old_text": { "type": "string", "description": "Unique substring for replace/remove" },
@@ -169,7 +169,7 @@ Key properties:
169
169
-**Trust boundary**: per-turn extraction only produces `user_said` atoms. Tainted source classes (`tool_output`, `file_read`, `web`, `mcp`, `subagent`, `agent_generated`, `inferred`) can be stored but are quarantined and excluded from recall until promoted.
170
170
-**Size cap**: defaults to 100 MB with `retention_decay` eviction; pinned atoms are never evicted.
When enabled, Extended Memory atoms are injected as a separate system message after the legacy memory block and episode summaries on each turn. For the full design, config reference, and implementation status, see [docs/EXTENDED_MEMORY.md](EXTENDED_MEMORY.md).
175
175
@@ -189,10 +189,12 @@ The optional prompt-injection guard subsystem ([docs/CONFIG.md](CONFIG.md#prompt
189
189
- the session buffer
190
190
- Extended Memory atom extraction, `add_atom`, and recall paths
191
191
192
-
The guard runs the local rule scan first, then optionally consults a configured `piguard` sidecar. If the guard flags content, the write is rejected and the agent receives an error.
192
+
The guard runs the local rule scan first, then optionally consults a configured `piguard` sidecar. Legacy fact writes that fail the scan are rejected with an error. Extended Memory atoms that fail the scan are instead **quarantined** with a `scan_rejected` reason (visible via `odek memory extended quarantine`) so guard false positives can be reviewed and promoted instead of silently lost; the `add_atom` tool result then reports "quarantined for human review" rather than "added".
193
193
194
194
## Observability (lifecycle events)
195
195
196
+
`odek memory extended stats` prints a snapshot of the Extended Memory store: live/quarantined atom counts, quarantine reason breakdown (`tainted` vs `scan_rejected` — the guard false-positive signal), index vector count and dirty flag, store size, and recall timeout/failure counters, with a degradation warning when recall errors have occurred in the process.
197
+
196
198
Every memory lifecycle moment emits a `memory.MemoryEvent` so operators can see
197
199
activity that was previously silent. Events fan out (via `MultiMemoryNotifier`)
0 commit comments