Skip to content

Commit 597a439

Browse files
committed
docs: sync CONFIG/MEMORY/EXTENDED_MEMORY/AGENTS with roadmap changes
- CONFIG.md: add semantic_dedup_threshold and consolidate_similarity_threshold to the extended config example and field reference - MEMORY.md: combined session-end extraction call, bounded drain at close (episode extraction no longer fire-and-forget), quarantine-not-reject guard behavior with the honest add_atom report, stats in the CLI surface and observability sections, complete memory tool action enum - EXTENDED_MEMORY.md: consolidate/stats now have CLI commands - AGENTS.md: PIGuard E2E and fuzz soak commands in Testing
1 parent db74fbf commit 597a439

4 files changed

Lines changed: 22 additions & 7 deletions

File tree

AGENTS.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,15 @@ ODEK_E2E=true go test -v -count=1 ./cmd/odek/ -run "TestMCPE2E_"
235235

236236
# Sandbox integration tests (requires Docker)
237237
go test -v -count=1 ./cmd/odek/ -run "TestSandbox"
238+
239+
# PIGuard sidecar E2E (requires the docker piguard stack running;
240+
# runs in CI via .github/workflows/piguard-e2e.yml)
241+
ODEK_E2E_GUARD=1 go test -v -count=1 ./internal/guard/ -run "TestE2E_"
242+
243+
# Fuzz soaks (extractJSON, SKILL.md parsing, session loading)
244+
go test -fuzz=FuzzExtractJSON -fuzztime=30s ./internal/memory/extended/
245+
go test -fuzz=FuzzParseSkillContent -fuzztime=30s ./internal/skills/
246+
go test -fuzz=FuzzSessionLoad -fuzztime=30s ./internal/session/
238247
```
239248

240249
Note: MCP client E2E tests build the fakeserver from `internal/mcpclient/testdata/main.go` at test time (no pre-compiled binary). macOS temp dirs are classified as `LocalWrite` (not `SystemWrite`), and the Docker availability check verifies daemon reachability before running sandbox tests.

docs/CONFIG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -375,6 +375,8 @@ The `memory` section controls the persistent memory system (see [docs/MEMORY.md]
375375
"semantic_search_overfetch": 4,
376376
"semantic_search_min_score": 0.55,
377377
"semantic_search_rerank": true,
378+
"semantic_dedup_threshold": 0.92,
379+
"consolidate_similarity_threshold": 0.9,
378380
"atom_max_chars": 300,
379381
"memory_budget_chars": 2000,
380382
"decay_half_life_days": 30,
@@ -409,6 +411,8 @@ The `memory` section controls the persistent memory system (see [docs/MEMORY.md]
409411
| `semantic_search_overfetch` | `4` ||| Candidate multiplier before filtering and reranking. |
410412
| `semantic_search_min_score` | `0.55` ||| Minimum cosine similarity for a candidate to be considered. |
411413
| `semantic_search_rerank` | `true` ||| Use the memory LLM to rerank candidates. |
414+
| `semantic_dedup_threshold` | `0.92` ||| Cosine similarity at or above which an incoming atom is treated as a paraphrase of an existing live atom and refreshes it instead of appending. `0` disables the semantic tier (exact-match dedup always runs). |
415+
| `consolidate_similarity_threshold` | `0.9` ||| Pairwise cosine similarity at or above which live atoms are grouped for LLM merging by `odek memory extended consolidate` / `ConsolidateAtoms`. |
412416
| `atom_max_chars` | `300` | `ODEK_MEMORY_EXTENDED_ATOM_MAX_CHARS` | `--memory-extended-atom-max-chars` | Maximum stored text length per atom. |
413417
| `memory_budget_chars` | `2000` | `ODEK_MEMORY_EXTENDED_MEMORY_BUDGET_CHARS` | `--memory-extended-memory-budget-chars` | Maximum injected Extended Memory context per turn. |
414418
| `decay_half_life_days` | `30` ||| Days until an atom's recall/eviction weight halves. |

docs/EXTENDED_MEMORY.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -174,11 +174,11 @@ Atoms are deduplicated in two tiers at persistence time:
174174

175175
### Atom Consolidation
176176

177-
`ExtendedMemory.ConsolidateAtoms(ctx)` (no CLI yet) merges groups of live atoms that are near-duplicates (pairwise cosine similarity at or above `consolidate_similarity_threshold`). For each group it asks the memory LLM — one call per group — to merge the texts into a single concise atom, stores the merged atom through the normal add path (scan and size cap apply) keeping the group's highest confidence, a refreshed `CreatedAt`, and the union of the group's outward associations, then removes the originals. Quarantined atoms are never consolidated. On any failure for a group (LLM error, empty response, scan rejection) the originals are kept untouched.
177+
`ExtendedMemory.ConsolidateAtoms(ctx)` — exposed as `odek memory extended consolidate` merges groups of live atoms that are near-duplicates (pairwise cosine similarity at or above `consolidate_similarity_threshold`). For each group it asks the memory LLM — one call per group — to merge the texts into a single concise atom, stores the merged atom through the normal add path (scan and size cap apply) keeping the group's highest confidence, a refreshed `CreatedAt`, and the union of the group's outward associations, then removes the originals. Quarantined atoms are never consolidated. On any failure for a group (LLM error, empty response, scan rejection) the originals are kept untouched.
178178

179179
### Observability
180180

181-
`ExtendedMemory.Stats()` returns a `Stats` snapshot for operators and monitoring: live/quarantined atom counts, quarantine entries grouped by reason prefix (`tainted`, `scan_rejected`), index vector count and dirty flag, on-disk store size, and recall error counters (`RecallTimeouts` for context-deadline-exceeded errors, `RecallFailures` for all other recall errors).
181+
`ExtendedMemory.Stats()` returns a `Stats` snapshot — surfaced as `odek memory extended stats`for operators and monitoring: live/quarantined atom counts, quarantine entries grouped by reason prefix (`tainted`, `scan_rejected`), index vector count and dirty flag, on-disk store size, and recall error counters (`RecallTimeouts` for context-deadline-exceeded errors, `RecallFailures` for all other recall errors). The CLI prints a degradation warning when either counter is non-zero.
182182

183183
## Semantic Search
184184

docs/MEMORY.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -45,9 +45,9 @@ HH:MM agent pushed 19 tests, tagged v0.8.19
4545

4646
### Tier 3 — Episodes (on-disk, searchable)
4747

48-
After sessions with ≥3 turns, the MemoryManager runs SimpleCall to extract 1-3 durable facts. Written to `episodes/<session-id>.md`. Searchable via `memory(search=...)` which uses **RandomProjections** (go-vector) to rank episodes by cosine similarity to the query — zero LLM calls per search. Set `llm_search: true` in config to use LLM-based ranking instead.
48+
After sessions with ≥3 turns, the MemoryManager extracts a session summary. When both episode extraction (`extract_on_end`) and fact extraction (`extract_facts`) are enabled, a **single combined LLM call** produces the episode summary and the durable facts in one JSON response (falling back to the two single-purpose calls if the combined response is unparseable). Written to `episodes/<session-id>.md`. Searchable via `memory(search=...)` which uses **RandomProjections** (go-vector) to rank episodes by cosine similarity to the query — zero LLM calls per search. Set `llm_search: true` in config to use LLM-based ranking instead.
4949

50-
Episode extraction runs **asynchronously** — it does not block process exit. The session summary is a best-effort post-processing step that completes in a background goroutine.
50+
Episode extraction runs **asynchronously** — it does not block the agent loop. Session-end work is tracked by the MemoryManager and drained with a bounded wait (~15s) in `Agent.Close`, so episodes survive CLI exit without hanging the process.
5151

5252
## Memory Tool — Unified API
5353

@@ -56,7 +56,7 @@ Episode extraction runs **asynchronously** — it does not block process exit. T
5656
"name": "memory",
5757
"description": "Manage persistent memory across sessions.",
5858
"parameters": {
59-
"action": { "enum": ["add", "replace", "remove", "consolidate", "read", "search"] },
59+
"action": { "enum": ["add", "replace", "remove", "consolidate", "read", "search", "view", "add_atom", "search_atoms", "forget_atom", "list_quarantine", "pin_atom", "confirm_pending_review", "reject_pending_review", "list_pending_review"] },
6060
"target": { "enum": ["user", "env"], "description": "For add/replace/remove/consolidate" },
6161
"content": { "type": "string", "description": "For add/replace" },
6262
"old_text": { "type": "string", "description": "Unique substring for replace/remove" },
@@ -169,7 +169,7 @@ Key properties:
169169
- **Trust boundary**: per-turn extraction only produces `user_said` atoms. Tainted source classes (`tool_output`, `file_read`, `web`, `mcp`, `subagent`, `agent_generated`, `inferred`) can be stored but are quarantined and excluded from recall until promoted.
170170
- **Size cap**: defaults to 100 MB with `retention_decay` eviction; pinned atoms are never evicted.
171171
- **Tool surface**: `memory` tool actions `add_atom`, `search_atoms`, `forget_atom`, `pin_atom`, `list_quarantine`, `confirm_pending_review`, `reject_pending_review`, and `list_pending_review`.
172-
- **CLI surface**: `odek memory extended forget|promote|pin|quarantine|compact|pending|confirm|reject`.
172+
- **CLI surface**: `odek memory extended forget|promote|pin|quarantine|compact|stats|consolidate|pending|confirm|reject`.
173173

174174
When enabled, Extended Memory atoms are injected as a separate system message after the legacy memory block and episode summaries on each turn. For the full design, config reference, and implementation status, see [docs/EXTENDED_MEMORY.md](EXTENDED_MEMORY.md).
175175

@@ -189,10 +189,12 @@ The optional prompt-injection guard subsystem ([docs/CONFIG.md](CONFIG.md#prompt
189189
- the session buffer
190190
- Extended Memory atom extraction, `add_atom`, and recall paths
191191

192-
The guard runs the local rule scan first, then optionally consults a configured `piguard` sidecar. If the guard flags content, the write is rejected and the agent receives an error.
192+
The guard runs the local rule scan first, then optionally consults a configured `piguard` sidecar. Legacy fact writes that fail the scan are rejected with an error. Extended Memory atoms that fail the scan are instead **quarantined** with a `scan_rejected` reason (visible via `odek memory extended quarantine`) so guard false positives can be reviewed and promoted instead of silently lost; the `add_atom` tool result then reports "quarantined for human review" rather than "added".
193193

194194
## Observability (lifecycle events)
195195

196+
`odek memory extended stats` prints a snapshot of the Extended Memory store: live/quarantined atom counts, quarantine reason breakdown (`tainted` vs `scan_rejected` — the guard false-positive signal), index vector count and dirty flag, store size, and recall timeout/failure counters, with a degradation warning when recall errors have occurred in the process.
197+
196198
Every memory lifecycle moment emits a `memory.MemoryEvent` so operators can see
197199
activity that was previously silent. Events fan out (via `MultiMemoryNotifier`)
198200
to whichever surfaces are wired:

0 commit comments

Comments
 (0)