feat(memory): path-aware episode/skill provenance + human-gated promote (#10)

* feat(memory): path-aware episode/skill provenance + human-gated promote

Fix the "episode dead-end": provenance tainted episodes by tool *name* only,
so read_file/search_files/multi_grep — used in essentially every coding
session — always produced an untrusted episode. Untrusted episodes are
excluded from recall, and nothing ever set UserApproved, so for normal work
the episode tier was written but never replayed (dead weight).

Provenance is now argument-aware via a single shared predicate
`memory.ToolCallTaints(name, argsJSON)`:

- Always untrusted: browser, http_batch, transcribe, and any MCP tool (__).
- Path-scoped (read_file, search_files, multi_grep): taint only when the
  `path` argument resolves to a sensitive location (danger.ClassifyPath →
  system_write/destructive, e.g. /etc, ~/.ssh, ~/.aws). Workspace and other
  non-sensitive local reads stay trusted. Malformed/absent path → conservative
  taint. This precisely re-targets the original "covers /etc/, $HOME/.ssh"
  concern while letting ordinary sessions stay recallable.

The skills provenance (internal/skills/selfimprove.go) now delegates to the
same predicate, so learned skills stop being over-flagged for review too.

Reusing danger.ClassifyPath means DeriveProvenance keeps its (messages)
signature — no cwd plumbing into the async session-end call sites.

Escape hatch for genuinely-external episodes: new human-gated CLI
`odek memory list` / `odek memory promote <session_id>` sets UserApproved.
It is intentionally NOT an agent tool — a prompt-injected agent must not be
able to self-approve its own poisoned memory.

Also fixes two stale descriptions found while mapping the system: the
extract_on_end docs ("durable facts" → episode summary) and the
FormatEpisodeContext comment ("recency-based ranker, no LLM" → it uses the
configured, LLM-by-default ranker).

Tests: path-aware taint (workspace trusted, sensitive/system tainted, malformed
conservative, always-external), the ToolCallTaints table, episode
promote/pending lifecycle + persistence, skills mirror (guards the
llm.Message→LlmMessage Arguments plumbing), and the CLI command end to end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(memory): close provenance bypasses found by verification (D-01/D-02/D-03/D-05)

Remediates the findings from the AI Verification Protocol run on this PR. The
v1 taint check used a sensitive-path *denylist* (danger.ClassifyPath) over only
three tools, which left several bypasses:

- D-03 (regression): macOS /etc and /var are symlinks to /private/etc, /private
  /var. ClassifyPath only matched /etc,/var, so read_file("/private/etc/...")
  classified local_write and did NOT taint — a sensitive read that tainted
  before this PR. Confirmed live.
- D-01: sibling whole-file readers (batch_read, json_query, head_tail, sort,
  tr, diff, count_lines, checksum, word_count, file_info, glob, tree, base64)
  were in no taint set at all, so reading /etc/shadow via batch_read produced a
  TRUSTED, recallable episode.
- D-02: home credential files outside ClassifyPath's hardcoded subdir list
  (e.g. ~/secrets.env) classified local_write and did not taint.
- D-05: session_search re-surfaces prior-session transcripts but never tainted.

Fix — flip the model from a sensitive-path denylist to a workspace-containment
allowlist, and broaden coverage:

- ToolCallTaints now taints a path-reading tool when ANY of its path arguments
  (path / path_a / path_b / files[].path) resolves OUTSIDE the workspace trust
  zone (cwd, the sandbox /workspace mount, or ~/.odek). Symlinks are resolved
  so /etc -> /private/etc can't disguise an escape. Workspace reads stay
  trusted (the original goal); everything else taints. Malformed args still
  taint conservatively; absent/empty path defaults to the workspace.
- PathReadingTools expanded to every file-reading tool; session_search added to
  AlwaysExternalTools. Provenance no longer depends on ClassifyPath.
- Defense in depth: ClassifyPath now strips the macOS /private prefix so
  /private/etc|var classify the same as /etc|var for its other callers
  (transcribe risk, write confinement).

Also adds the missing manager-wrapper tests (PromoteEpisode /
PendingReviewEpisodes were at 0% coverage — the report's coverage finding;
now 100%) and updates docs/SECURITY.md to describe the containment model
accurately (the report's doc-accuracy finding).

Verified: batch_read/json_query/diff of /etc, read_file of /private/etc and
~/secrets.env, and session_search all taint; workspace reads (relative + abs)
and the /workspace mount stay trusted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(memory): optional auto-approval of episode learnings (default off)

Add `memory.auto_approve_episodes` (default false). When enabled, untrusted
episodes are stamped approved at session end so they are recalled without a
manual `odek memory promote` — an explicit opt-in that trades the human review
gate for convenience in trusted/sandboxed deployments.

- New EpisodeProvenance.AutoApproved field, kept DISTINCT from UserApproved so
  the audit trail shows the approval was automatic (policy), while Untrusted +
  Sources are still recorded. Episode Search and PendingReview treat
  AutoApproved as approved.
- The stamp is applied at write time in OnSessionEndWithProvenance only when
  the flag is on and the episode is untrusted; trusted episodes are unaffected.
- Config plumbed through MemoryConfig (default false), NewMemoryManager merge,
  and resolveMemory.
- docs/CONFIG.md and docs/SECURITY.md document it as a security trade-off that
  disables the persistence-injection protection for episodes when on.

Tests: store-level recall of an AutoApproved episode + exclusion from pending;
the secure default is false; and the full OnSessionEnd stamping path with the
flag on (AutoApproved set, recallable, not pending) vs off (excluded, pending).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: jkyberneees

cmd/odek/dispatch.go

@@ -53,6 +53,8 @@ func dispatch(args []string) int {
 		return cliExit(telegramCmd(rest))
 	case "schedule":
 		return cliExit(scheduleCmd(rest))
+	case "memory":
+		return cliExit(memoryCmd(rest))
 	default:
 		fmt.Fprintf(os.Stderr, "odek: unknown command %q\n", cmd)
 		printUsage()

cmd/odek/main.go

@@ -452,6 +452,7 @@ func printUsage() {
   odek mcp [--sandbox]
   odek telegram
   odek schedule <list|add|rm|enable|disable|run|next|daemon>
+  odek memory <list|promote <session_id>>
   odek version
 
 Commands:
@@ -477,6 +478,10 @@ Commands:
                        Subcommands: list, add, rm, enable, disable, run, next, daemon
                        The daemon (or the Telegram bot) fires jobs and delivers
                        results to stdout, a log, or a Telegram chat.
+  memory              Review and promote past-session memory episodes
+                       list: show episodes excluded from recall (untrusted)
+                       promote <session_id>: approve one so it can be recalled.
+                       Human-gated on purpose — not available to the agent.
   init                Create a config file (default: ./odek.json)
   version             Print version and exit
 

cmd/odek/memory_cmd.go

@@ -0,0 +1,66 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	"github.com/BackendStack21/odek/internal/memory"
+)
+
+// memoryCmd handles `odek memory <list|promote> [args]`.
+//
+// This is the human-gated surface for the episode-memory trust control.
+// Episodes whose originating session touched external content (web/http/MCP/
+// audio, or reads of sensitive paths) are stored but excluded from recall
+// until a human promotes them. Promotion lives HERE — on the CLI — and is
+// deliberately NOT exposed as an agent tool, so a prompt-injected agent cannot
+// approve its own poisoned memory.
+func memoryCmd(args []string) error {
+	if len(args) == 0 {
+		fmt.Fprintf(os.Stderr, "Usage: odek memory <list|promote> [args]\n")
+		return nil
+	}
+
+	dir := expandHome("~/.odek/memory")
+	store := memory.NewEpisodeStore(dir, nil)
+
+	sub := args[0]
+	subArgs := args[1:]
+
+	switch sub {
+	case "list", "ls", "pending":
+		pending, err := store.PendingReview()
+		if err != nil {
+			return err
+		}
+		if len(pending) == 0 {
+			fmt.Println("No episodes pending review — all stored episodes are recallable.")
+			return nil
+		}
+		fmt.Printf("%d episode(s) pending review (excluded from recall until promoted):\n\n", len(pending))
+		for _, ep := range pending {
+			fmt.Printf("• %s  (%d turns, %s)\n", ep.SessionID, ep.Turns, ep.CreatedAt.Format("2006-01-02 15:04"))
+			if len(ep.Provenance.Sources) > 0 {
+				fmt.Printf("    sources: %s\n", strings.Join(ep.Provenance.Sources, ", "))
+			}
+			fmt.Printf("    %s\n\n", ep.Summary)
+		}
+		fmt.Println("Review the summary above, then promote with:  odek memory promote <session_id>")
+		return nil
+
+	case "promote":
+		if len(subArgs) == 0 {
+			return fmt.Errorf("usage: odek memory promote <session_id>")
+		}
+		id := subArgs[0]
+		if err := store.Promote(id); err != nil {
+			return err
+		}
+		fmt.Printf("odek: promoted episode %q — it can now be recalled into future sessions\n", id)
+		return nil
+
+	default:
+		return fmt.Errorf("unknown memory subcommand %q (expected: list, promote)", sub)
+	}
+}

cmd/odek/memory_cmd_test.go

@@ -0,0 +1,53 @@
+package main
+
+import (
+	"path/filepath"
+	"testing"
+
+	"github.com/BackendStack21/odek/internal/memory"
+)
+
+// TestMemoryCmd_ListAndPromote exercises the human-gated promote path end to
+// end through the CLI command: a seeded untrusted episode is pending, the
+// command promotes it, and the approval is persisted to the on-disk index.
+func TestMemoryCmd_ListAndPromote(t *testing.T) {
+	home := setupTestHome(t)
+	dir := filepath.Join(home, ".odek", "memory")
+
+	es := memory.NewEpisodeStore(dir, nil)
+	if err := es.WriteWithProvenance("20260108-web", "researched a library", 5,
+		memory.EpisodeProvenance{Untrusted: true, Sources: []string{"browser"}}); err != nil {
+		t.Fatalf("seed: %v", err)
+	}
+
+	if err := memoryCmd([]string{"list"}); err != nil {
+		t.Fatalf("memory list: %v", err)
+	}
+	if err := memoryCmd([]string{"promote", "20260108-web"}); err != nil {
+		t.Fatalf("memory promote: %v", err)
+	}
+
+	fresh := memory.NewEpisodeStore(dir, nil)
+	idx, err := fresh.ReadIndex()
+	if err != nil {
+		t.Fatalf("read index: %v", err)
+	}
+	if len(idx) != 1 || !idx[0].Provenance.UserApproved {
+		t.Errorf("episode not approved after promote: %+v", idx)
+	}
+
+	if err := memoryCmd([]string{"promote", "does-not-exist"}); err == nil {
+		t.Error("promoting an unknown id should error")
+	}
+	if err := memoryCmd([]string{"bogus"}); err == nil {
+		t.Error("unknown subcommand should error")
+	}
+}
+
+// TestMemoryCmd_ListEmpty: list on a clean home must not error.
+func TestMemoryCmd_ListEmpty(t *testing.T) {
+	setupTestHome(t)
+	if err := memoryCmd([]string{"list"}); err != nil {
+		t.Fatalf("memory list on empty home: %v", err)
+	}
+}

docs/CONFIG.md

@@ -200,7 +200,8 @@ The `memory` section controls the persistent memory system (see [docs/MEMORY.md]
     "llm_extract": true,
     "llm_consolidate": true,
     "merge_threshold": 0.7,
-    "add_threshold": 0.3
+    "add_threshold": 0.3,
+    "auto_approve_episodes": false
   }
 }
 ```
@@ -213,12 +214,13 @@ The `memory` section controls the persistent memory system (see [docs/MEMORY.md]
 | `buffer_lines` | 20 | Max turn summaries in session buffer |
 | `buffer_enabled` | true | Enable the turn-level buffer |
 | `merge_on_write` | true | Use go-vector RP similarity to auto-merge related entries |
-| `extract_on_end` | true | Extract durable facts via LLM at session end (≥3 turns) |
+| `extract_on_end` | true | At session end (≥3 turns), extract a narrative episode summary via LLM for later recall |
 | `llm_search` | true | Use LLM to rank episode search results by relevance |
 | `llm_extract` | true | Use LLM for end-of-session fact extraction |
 | `llm_consolidate` | true | Use LLM to merge related fact entries |
 | `merge_threshold` | 0.7 | go-vector cosine threshold for auto-merge (0.0–1.0) |
 | `add_threshold` | 0.3 | go-vector cosine threshold for auto-add (0.0–1.0) |
+| `auto_approve_episodes` | false | **Security trade-off.** When true, untrusted episodes (sessions that touched web/MCP/out-of-workspace content) are auto-approved at session end so they are recalled without a manual `odek memory promote`. Leaving it `false` keeps the human review gate (recommended). |
 
 ## Sub-agent configuration
 

docs/SECURITY.md

@@ -100,11 +100,25 @@ Both:
 
 `internal/memory` tracks `EpisodeProvenance{Untrusted, Sources, UserApproved}` for every episode. An episode derived from a session that ingested untrusted content is **stored on disk for audit but never auto-replayed** into future sessions. This stops a single successful injection from becoming a persistent backdoor through the episode pipeline.
 
-To use a tainted episode anyway, the user must explicitly promote it (set `UserApproved=true`).
+Taint is decided per tool call by `memory.ToolCallTaints` (the single source of truth, shared with skills):
+
+- **Always untrusted:** `browser`, `http_batch`, `transcribe` (network / opaque-audio content), `session_search` (recall of prior-session transcripts, which may carry earlier-injected text), and any MCP tool (`server__tool`).
+- **Path-reading tools** (`read_file`, `search_files`, `multi_grep`, `batch_read`, `json_query`, `head_tail`, `count_lines`, `checksum`, `word_count`, `sort`, `tr`, `diff`, `file_info`, `glob`, `tree`, `base64`) taint when **any** of their path arguments resolves **outside the workspace trust zone** — the workspace dir, the sandbox `/workspace` mount, or `~/.odek`. Reads confined to the workspace stay trusted, so ordinary coding sessions remain recallable; reads of anything else (system/credential paths, home files, sibling repos) taint. The check is a workspace-containment allowlist rather than a sensitive-path denylist, and it resolves symlinks (so e.g. `/etc` → `/private/etc` on macOS cannot disguise an escape). A malformed argument string is treated conservatively as untrusted. When adding a new file-reading tool, add it to `PathReadingTools`.
+
+To use a tainted episode anyway, the user explicitly promotes it (sets `UserApproved=true`) from the CLI:
+
+```
+odek memory list                    # episodes excluded from recall, with their sources
+odek memory promote <session_id>    # approve one after reviewing its summary
+```
+
+Promotion is **CLI-only and human-gated** — it is deliberately *not* exposed as an agent tool, so a prompt-injected agent cannot self-approve its own poisoned memory.
+
+**Opt-out of the gate (`memory.auto_approve_episodes`, default `false`).** Operators who accept the risk (e.g. a fully sandboxed, single-tenant deployment) can set `auto_approve_episodes: true` to have untrusted episodes stamped `AutoApproved` at session end so they are recalled without a manual promote. This **disables the persistence-injection protection** for episodes — a single successful injection can then influence future sessions automatically — so it is off by default and should stay off in any environment exposed to untrusted input. The on-disk record still keeps `Untrusted=true` and `Sources`, and uses a distinct `AutoApproved` flag (never `UserApproved`) so the audit trail shows the approval was automatic.
 
 ### 6. Skill provenance gate
 
-`internal/skills` carries the same provenance model. Skills auto-saved from sessions that touched `browser` / `http_batch` / `read_file` / `search_files` / `multi_grep` / `transcribe` / any MCP tool are tagged with `Provenance.Untrusted=true` and `NeedsReview=true`. The skill loader pins those skills to the Lazy set regardless of their `auto_load` flag.
+`internal/skills` carries the same provenance model and shares the exact taint decision (`memory.ToolCallTaints`). Skills auto-saved from sessions that crossed the trust boundary — `browser` / `http_batch` / `transcribe` / any MCP tool, or a `read_file` / `search_files` / `multi_grep` of a **sensitive** path — are tagged with `Provenance.Untrusted=true` and `NeedsReview=true`. The skill loader pins those skills to the Lazy set regardless of their `auto_load` flag.
 
 After reviewing the skill body, promote it:
 

internal/config/loader.go

@@ -649,22 +649,22 @@ func LoadConfig(cli CLIFlags) ResolvedConfig {
 		MaxIter:  cfg.MaxIter,
 		System:   cfg.System,
 
-		SandboxImage:        cfg.SandboxImage, // empty = resolve at call site (Dockerfile.odek or alpine:latest)
-		SandboxNetwork:      ifZero(cfg.SandboxNetwork, DefaultSandboxNetwork),
-		SandboxMemory:       cfg.SandboxMemory,
-		SandboxCPUs:         cfg.SandboxCPUs,
-		SandboxUser:         cfg.SandboxUser,
-		SandboxEnv:          cfg.SandboxEnv,
-		SandboxVolumes:      cfg.SandboxVolumes,
-		Skills:              resolveSkills(cfg.Skills),
-		Dangerous:           resolveDangerous(cfg.Dangerous),
-		Memory:              resolveMemory(cfg.Memory),
-		MCPServers:          cfg.MCPServers,
-		Telegram:            resolveTelegram(cfg.Telegram),
-		Transcription:       resolveTranscription(cfg.Transcription),
-		Schedules:           resolveSchedules(cfg.Schedules),
-		InteractionMode:     ifZero(cfg.InteractionMode, "engaging"),
-		ToolProgress:        ifZero(cfg.ToolProgress, "all"),
+		SandboxImage:    cfg.SandboxImage, // empty = resolve at call site (Dockerfile.odek or alpine:latest)
+		SandboxNetwork:  ifZero(cfg.SandboxNetwork, DefaultSandboxNetwork),
+		SandboxMemory:   cfg.SandboxMemory,
+		SandboxCPUs:     cfg.SandboxCPUs,
+		SandboxUser:     cfg.SandboxUser,
+		SandboxEnv:      cfg.SandboxEnv,
+		SandboxVolumes:  cfg.SandboxVolumes,
+		Skills:          resolveSkills(cfg.Skills),
+		Dangerous:       resolveDangerous(cfg.Dangerous),
+		Memory:          resolveMemory(cfg.Memory),
+		MCPServers:      cfg.MCPServers,
+		Telegram:        resolveTelegram(cfg.Telegram),
+		Transcription:   resolveTranscription(cfg.Transcription),
+		Schedules:       resolveSchedules(cfg.Schedules),
+		InteractionMode: ifZero(cfg.InteractionMode, "engaging"),
+		ToolProgress:    ifZero(cfg.ToolProgress, "all"),
 	}
 
 	// MaxConcurrency: default to 3 if not set
@@ -850,6 +850,9 @@ func resolveMemory(cfg *memory.MemoryConfig) memory.MemoryConfig {
 	if cfg.MinTurnsForExtraction > 0 {
 		def.MinTurnsForExtraction = cfg.MinTurnsForExtraction
 	}
+	if cfg.AutoApproveEpisodes != nil {
+		def.AutoApproveEpisodes = cfg.AutoApproveEpisodes
+	}
 	return def
 }
 

internal/danger/classifier.go

@@ -145,6 +145,15 @@ func ClassifyPath(path string) RiskClass {
 	}
 	abs = filepath.Clean(abs)
 
+	// macOS canonicalizes /etc, /var, and /tmp as symlinks under /private.
+	// Strip the /private prefix so the sensitivity checks below match
+	// consistently — e.g. /private/etc/master.passwd must classify the same
+	// as /etc/master.passwd (system_write), and /private/var/folders/... must
+	// still resolve to the temp dir (local_write).
+	if strings.HasPrefix(abs, "/private/") {
+		abs = strings.TrimPrefix(abs, "/private")
+	}
+
 	for _, prefix := range []string{"/boot", "/dev", "/proc", "/sys", "/mnt", "/media"} {
 		if strings.HasPrefix(abs, prefix) {
 			return Destructive

internal/memory/episodes.go

@@ -217,7 +217,7 @@ func (e *EpisodeStore) Search(query string, limit int) ([]EpisodeMeta, error) {
 	// EpisodeProvenance exists to close.
 	filtered := idx[:0:len(idx)]
 	for _, ep := range idx {
-		if ep.Provenance.Untrusted && !ep.Provenance.UserApproved {
+		if ep.Provenance.Untrusted && !ep.Provenance.UserApproved && !ep.Provenance.AutoApproved {
 			continue
 		}
 		filtered = append(filtered, ep)
@@ -244,6 +244,59 @@ func (e *EpisodeStore) Search(query string, limit int) ([]EpisodeMeta, error) {
 	return ranked, nil
 }
 
+// ── Promotion (human-gated escape hatch) ──────────────────────────────
+
+// Promote marks a tainted episode as user-approved so it can be replayed
+// into future sessions. This is the human-gated escape hatch for episodes
+// whose originating session legitimately touched external content. It is
+// intentionally NOT exposed to the agent (only via `odek memory promote`) so
+// that a prompt-injected agent cannot self-approve poisoned memory.
+//
+// Returns an error if the session is unknown or already approved.
+func (e *EpisodeStore) Promote(sessionID string) error {
+	if err := session.ValidateSessionID(sessionID); err != nil {
+		return fmt.Errorf("memory: episodes promote: %w", err)
+	}
+	e.mu.Lock()
+	defer e.mu.Unlock()
+
+	idx, err := e.ReadIndex()
+	if err != nil {
+		return err
+	}
+	found := false
+	for i := range idx {
+		if idx[i].SessionID == sessionID {
+			found = true
+			if idx[i].Provenance.UserApproved {
+				return fmt.Errorf("memory: episode %q is already approved", sessionID)
+			}
+			idx[i].Provenance.UserApproved = true
+		}
+	}
+	if !found {
+		return fmt.Errorf("memory: episode %q not found", sessionID)
+	}
+	return e.writeIndex(idx)
+}
+
+// PendingReview returns the episodes that are untrusted and not yet
+// user-approved — the ones currently excluded from recall that a user may
+// want to promote. Ordered newest-first (as ReadIndex returns them).
+func (e *EpisodeStore) PendingReview() ([]EpisodeMeta, error) {
+	idx, err := e.ReadIndex()
+	if err != nil {
+		return nil, err
+	}
+	var pending []EpisodeMeta
+	for _, ep := range idx {
+		if ep.Provenance.Untrusted && !ep.Provenance.UserApproved && !ep.Provenance.AutoApproved {
+			pending = append(pending, ep)
+		}
+	}
+	return pending, nil
+}
+
 // ── Index helpers ─────────────────────────────────────────────────────
 
 // addToIndex appends an entry to the index and writes it.