feat: feedback system, adaptive retrieval, governance audit trail, API hardening, and comprehensive e2e tests (#65)

## What type of PR is this?

- [ ] feat (new feature)
- [ ] fix (bug fix)
- [ ] docs (documentation)
- [ ] style (formatting, no code change)
- [ ] refactor (code change that neither fixes a bug nor adds a feature)
- [ ] perf (performance improvement)
- [ ] test (adding or updating tests)
- [ ] chore (maintenance, tooling)
- [ ] build / ci (build or CI changes)

## Which issue(s) this PR fixes

Fixes #

## What this PR does / why we need it

#### 1. Feedback & Adaptive Retrieval System
- New `mem_retrieval_feedback` table for explicit relevance signals
(useful/irrelevant/outdated/wrong)
- New `mem_user_retrieval_params` table for per-user adaptive scoring
parameters
- `record_feedback()` validates signals, verifies memory ownership,
updates denormalized counters
- `search_hybrid_from_scored()` applies feedback adjustment: `(1 + fw *
(useful - 0.5*negative)).clamp(0.5, 2.0)`
- `DefaultScoringPlugin` auto-tunes feedback_weight based on signal
ratios (≥10 feedback threshold)
- REST endpoints: `POST /v1/memories/:id/feedback`, `GET
/v1/feedback/stats`, `GET /v1/feedback/by-tier`, `GET/PUT
/v1/retrieval-params`, `POST /v1/retrieval-params/tune`

#### 2. Governance Audit Trail Enhancement
- Governance operations now record structured payloads: `{"quarantined":
N}`, `{"cleaned_stale": N}`, etc.
- All 5 governance operations (archive_working, cleanup_stale,
quarantine, compress_redundant, cleanup_orphaned_incrementals) include
detailed audit logs
- `mem_edit_log` redesigned: `target_ids JSON` → `memory_id VARCHAR(64)`
+ `payload JSON`, no PK, `CLUSTER BY`, UUID v7 for edit_id

#### 3. API Error Handling Improvements
- New `MemoriaError::Validation` variant for input validation errors
- New `api_err_typed()` function maps error variants to proper HTTP
status codes:
  - `NotFound` → 404
  - `Validation/InvalidMemoryType/InvalidTrustTier` → 422
  - `Blocked` → 403
  - Others → 500
- Applied to `record_feedback` and `store_memory` handlers

#### 4. Prometheus Metrics & Admin Config
- `GET /metrics` endpoint: Prometheus text exposition format with
memoria_memories_total, memoria_users_total,
memoria_auth_failures_total, etc.
- `GET /admin/config` (master-key-only): runtime config view with
redacted DB password

#### 5. MCP Tool Surface Reduction
- 5 tools hidden from `list()` but still callable via REST/direct
invocation: `memory_rebuild_index`, `memory_get_retrieval_params`,
`memory_tune_params`, `memory_extract_entities`, `memory_link_entities`
- Tool count: 18 → 13 in public listing

#### 6. Comprehensive E2E Test Coverage
- **Existing fixes**: `test_api_feedback_invalid_signal` (422 for
invalid signal), `test_api_tune_retrieval_params` (COALESCE fix for
empty feedback)
- **New API tests**: `/metrics`, `/v1/snapshots/:name/rollback`,
`/v1/entities`, `/admin/config` (with master-key auth)
- **Concurrency tests**: parallel stores, entity extraction race
condition, concurrent feedback
- **Pressure tests**: batch store at 100-item limit
- **Graceful degradation**: nonexistent snapshot, feedback on deleted
memory, correct after purge

#### 7. Documentation & Templates Sync
- All 8 markdown templates (Kiro, Cursor, Claude) updated with
feedback/adaptive retrieval guidance
- Steering rules synchronized across all 3 agent platforms
- API reference and architecture skills updated

### Bug Fixes
- Fixed `get_feedback_stats()` NULL handling with COALESCE for empty
feedback tables
- Fixed race condition in `upsert_entity()`: INSERT-first, catch
"Duplicate entry" error
- Fixed `batch_upsert_memory_entity_links()` to use multi-row INSERT
with ON DUPLICATE KEY UPDATE

Author: XuPeng-SH

.kiro/steering/goal-driven-evolution.md

@@ -1,18 +1,28 @@
 ---
-inclusion: agent_requested
+inclusion: always
 ---
 
 <!-- memoria-version: 0.1.0-->
 
-# Goal-Driven Iterative Evolution via Memory
+# Goal-Driven Evolution + Plan Integration
 
-Track goals, plans, progress, lessons, and user feedback across conversations. All content in English for consistent retrieval.
+Track goals, plans, progress, lessons, and user feedback across conversations. Integrates with plan panels (Kiro Shift+Tab, Cursor Composer, Claude multi-step).
 
-## Workflow
+## Before Starting Any Multi-Step Task
 
-### 1. Register Goal
+Query memory first:
 
-Check for duplicates first, then store:
+```
+memory_search(query="GOAL [topic]")                    # existing related goals
+memory_search(query="LESSON [topic]")                  # past learnings
+memory_search(query="CORRECTION ANTIPATTERN [topic]") # what NOT to do
+```
+
+If an active goal exists, continue it instead of creating a new one.
+
+## Register Goal
+
+For multi-session work (skip for trivial single-session tasks < 3 steps):
 
 ```
 memory_search(query="GOAL [keywords]")
@@ -22,14 +32,7 @@ memory_store(
 )
 ```
 
-### 2. Plan & Execute
-
-Before acting, search for past failures and user corrections to avoid repeating mistakes:
-
-```
-memory_search(query="CORRECTION ANTIPATTERN [goal name]")
-memory_search(query="❌ STEP for GOAL [name]")
-```
+## Plan & Execute
 
 Store the plan, then track each step:
 
@@ -41,16 +44,18 @@ memory_store(content="✅ STEP [N/total] for GOAL [name] (#X)\nAction: [done]\nR
 memory_store(content="❌ STEP [N/total] for GOAL [name] (#X)\nAction: [tried]\nError: [wrong]\nRoot Cause: [why]\nNext: [adjust]", memory_type="working")
 ```
 
+Only store non-obvious insights. Don't store "ran tests, passed".
+
 For high-risk iterations, isolate on a branch:
 ```
 memory_branch(name="goal_[name]_iter_[N]")
 memory_checkout(name="goal_[name]_iter_[N]")
 # work on branch... then validate and merge (see Iteration Review)
 ```
 
-### 3. Capture User Feedback (immediately, any time)
+## Capture User Feedback (immediately)
 
-User corrections are the highest-value signal — always store as `procedural`:
+User corrections are highest-value — always store as `procedural`:
 
 ```
 # User corrects direction
@@ -66,7 +71,7 @@ memory_store(content="⚠️ ANTIPATTERN for GOAL [name]: [what went wrong]. Rul
 memory_correct(query="GOAL: [name]", new_content="🎯 GOAL: [name]\n...\nPivot: [old] → [new]. Reason: [why]", reason="User changed direction")
 ```
 
-### 4. Iteration Review
+## Iteration Review
 
 When an iteration completes or is blocked:
 
@@ -78,7 +83,7 @@ memory_store(
   memory_type="procedural"
 )
 
-# If the insight is reusable beyond this goal, extract it now — don't wait for completion
+# If the insight is reusable beyond this goal, extract it now
 memory_store(content="💡 LESSON from [goal] iter #X: [cross-goal reusable insight]", memory_type="procedural")
 
 memory_correct(query="GOAL: [name]", new_content="🎯 GOAL: [name]\nStatus: ITERATION #X COMPLETE — [progress %]\nNext: [plan]", reason="iteration complete")
@@ -92,13 +97,13 @@ memory_merge(source="goal_[name]_iter_[N]", strategy="replace")
 memory_branch_delete(name="goal_[name]_iter_[N]")
 ```
 
-Starting the next iteration? The new PLAN must reference the previous RETRO's improvements:
+Starting the next iteration? Reference the previous RETRO's improvements:
 ```
 memory_search(query="RETRO for GOAL [name]")
-# Incorporate "Next: [improvements]" into the new plan — never repeat the same plan unchanged
+# Incorporate "Next: [improvements]" into the new plan
 ```
 
-### 5. New Conversation Bootstrap
+## New Conversation Bootstrap
 
 ```
 memory_search(query="GOAL ACTIVE")
@@ -108,7 +113,7 @@ memory_search(query="CORRECTION ANTIPATTERN [name]")
 
 Summarize to user: active goals, last progress, and any corrections to respect.
 
-### 6. Goal Completion & Cleanup
+## Goal Completion & Cleanup
 
 ```
 memory_correct(query="GOAL: [name]", new_content="🎯 GOAL: [name] — ✅ ACHIEVED\nIterations: [N]\nFinal approach: [what worked]", reason="Goal achieved")
@@ -120,10 +125,18 @@ memory_store(content="💡 LESSON from [goal]: [reusable insight for future work
 memory_purge(topic="STEP for GOAL [name]", reason="Goal achieved, archived in RETRO")
 ```
 
+## When Goal is Abandoned
+
+```
+memory_store(content="⚠️ ANTIPATTERN: [what didn't work]. Reason: [why abandoned]", memory_type="procedural")
+memory_correct(query="GOAL: [name]", new_content="🎯 GOAL: [name] — ❌ ABANDONED\nReason: [why]", reason="abandoned")
+```
+
 ## Rules
 
 - **Search before acting**: always check past failures, corrections, and antipatterns before proposing a plan
 - **User corrections override all**: if user corrected something, that correction has highest priority forever
-- **Be specific**: "Tests failed" is useless; "pytest fixtures don't work with async DB, use factory pattern" is valuable
+- **Be specific**: "pytest fixtures don't work with async DB, use factory pattern" > "tests failed"
+- **Don't create goals for quick fixes** (< 3 tasks, single session)
 - **Emoji prefixes**: 🎯 goal, 📋 plan, ✅❌ steps, 🔄 retro, 💡 lesson, 🔧 correction, 👍 feedback, ⚠️ antipattern
 - **Type discipline**: GOAL/PLAN/RETRO/LESSON/CORRECTION → `procedural`; STEP logs → `working`

.kiro/steering/memory-branching-patterns.md

@@ -1,5 +1,7 @@
 ---
-inclusion: agent_requested
+inclusion: auto
+name: memory-branching
+description: Git-like branching for memory - isolated experiments, tech evaluation, A/B comparison. Use when exploring alternatives or risky changes.
 ---
 
 <!-- memoria-version: 0.1.0-->

.kiro/steering/memory-hygiene.md

@@ -1,5 +1,7 @@
 ---
-inclusion: agent_requested
+inclusion: auto
+name: memory-hygiene
+description: Memory health management - governance triggers, contradiction resolution, snapshot cleanup. Use when memory seems noisy or contradictory.
 ---
 
 <!-- memoria-version: 0.1.0-->
@@ -17,7 +19,6 @@ Run `memory_governance` (1h cooldown) when you notice ANY of these:
 
 After governance, check the response for:
 - `snapshot_health.auto_ratio > 50%` → suggest `memory_snapshot_delete(prefix="auto:")`
-- `needs_rebuild = True` → run `memory_rebuild_index`
 - Quarantined memories → inform user what was quarantined and why
 
 ## Contradiction Resolution
@@ -52,7 +53,7 @@ Keep named snapshots the user created explicitly.
 
 ## Entity Graph Maintenance
 
-See entity graph triggers in [memory.md](memory.md) (proactive section). After extraction, if mode returns candidates, extract entities yourself and call `memory_link_entities` with the correct JSON format (see memory.md tool reference).
+Entity extraction is automatic — every `memory_store` triggers regex-based extraction, with LLM extraction as a fallback when configured. No manual intervention needed.
 
 ## Reflection Cadence
 

.kiro/steering/memory.md

@@ -9,19 +9,17 @@ inclusion: always
 You have persistent memory via MCP tools. Memory survives across conversations.
 
 ## 🔴 MANDATORY: Every conversation start
-Follow the bootstrap protocol in [session-lifecycle](session-lifecycle.md) — multi-query retrieval before your first response.
 
-At minimum, call `memory_retrieve` with a **semantic query** derived from the user's message BEFORE responding.
+Call `memory_retrieve` with a **semantic query** derived from the user's message BEFORE responding.
 
-**Query construction rules:**
-- ✅ **DO**: Extract key concepts from user's question → "benchmark optimization", "graph retrieval bug", "active goals"
-- ❌ **DON'T**: Use meta-queries → "all memories", "everything", "list all", "show me data"
-- When user asks "what do I know" or "我有哪些记忆", query the most recent active context instead (e.g., "recent goals tasks projects")
+**Query rules:**
+- ✅ Extract key concepts → "benchmark optimization", "graph retrieval bug"
+- ❌ Don't use meta-queries → "all memories", "everything", "list all"
 
 **After retrieval:**
-- If results come back → use them as **reference only**. Treat retrieved memories as potentially stale or incomplete — always verify against current context before acting on them. Do NOT blindly trust memory content as ground truth.
-- If "No relevant memories found" → this is normal for new users, proceed without.
-- If ⚠️ health warnings appear → inform the user and offer to run `memory_governance`.
+- Results → use as reference, verify against current context
+- "No relevant memories" → normal for new users, proceed
+- ⚠️ warnings → inform user, offer `memory_governance`
 
 ## 🔴 MANDATORY: Every conversation turn
 After responding, decide if anything is worth remembering:
@@ -91,6 +89,35 @@ Before storing a new memory, consider:
 | `memory_retrieve` | Conversation start, or when context is needed | `query`, `top_k` (default 5), `session_id` (optional), `explain` (false = no debug, true = show timing) |
 | `memory_search` | User asks "what do you know about X" or you need to browse | `query`, `top_k` (default 10), `explain` (false = no debug, true = show timing) |
 | `memory_profile` | User asks "what do you know about me" | — |
+| `memory_feedback` | After using a retrieved memory, record if it was helpful | `memory_id`, `signal` (useful/irrelevant/outdated/wrong), `context` (optional) |
+
+**`memory_feedback`**: Call this after retrieval when you can assess whether a memory was helpful. Signals:
+- `useful` — memory helped answer the question or complete the task
+- `irrelevant` — memory was retrieved but not relevant to the query
+- `outdated` — memory contains stale information (consider `memory_correct` instead if you know the new value)
+- `wrong` — memory contains incorrect information (consider `memory_correct` instead if you know the correct value)
+
+**When to call feedback vs other tools**:
+- Memory helped → `memory_feedback(signal="useful")`
+- Memory irrelevant but correct → `memory_feedback(signal="irrelevant")`
+- Memory outdated and you know new value → `memory_correct` (not feedback)
+- Memory outdated but you don't know new value → `memory_feedback(signal="outdated")`
+- Memory wrong and you know correct value → `memory_correct` (not feedback)
+- Memory should be deleted → `memory_purge` (not feedback)
+
+**Example flow**:
+```
+# 1. Retrieve memories
+memories = memory_retrieve(query="database config")
+
+# 2. Use memories to answer user's question
+# ... (memory about "Uses PostgreSQL" helped answer)
+
+# 3. Record feedback for the helpful memory
+memory_feedback(memory_id="abc123", signal="useful", context="answered DB question")
+```
+
+**Impact**: Feedback accumulates over time. With default settings, a memory with 3 `useful` signals ranks ~30% higher in future retrievals. Don't call for every memory — only when you have clear signal.
 
 **`memory_retrieve` vs `memory_search`**: In MCP mode, both use the same retrieval pipeline (graph → hybrid vector+fulltext → fulltext fallback). The differences are:
 - `memory_retrieve` accepts `session_id` for session-scoped boosting; `memory_search` does not
@@ -124,33 +151,18 @@ When `memory_governance` reports snapshot_health with high auto_ratio (>50%), su
 ### Branches (isolated experiments)
 Git-like workflow for memory. `memory_branch(name)` creates, `memory_checkout(name)` switches, `memory_diff(source)` previews changes, `memory_merge(source)` merges back, `memory_branch_delete(name)` cleans up. `memory_branches()` lists all.
 
-### Entity graph (proactive — call when conditions are met)
-| Tool | When to call | Key params |
-|------|-------------|------------|
-| `memory_extract_entities` | **Proactively** after storing ≥ 5 new memories in a session, OR when user discusses a new project/technology/person not yet in the graph | `mode` (default: auto) |
-| `memory_link_entities` | After `extract_entities(mode='candidates')` returns memories — extract entities yourself, then call this | `entities` (JSON string) |
-
-**Trigger heuristics — call `memory_extract_entities` when ANY of these are true:**
-- You stored ≥ 5 memories this session and haven't extracted entities yet
-- User mentions a project, technology, or person by name that you haven't seen in previous `memory_retrieve` results
-- User asks about relationships between concepts ("how does X relate to Y")
-- User starts working on a new codebase or topic area
-
-**Do NOT extract entities when:**
-- Conversation is short (< 3 turns) and no new named entities appeared
-- User is only asking questions, not sharing new information
-- You already ran extraction this session
+### Entity graph
+Entity extraction is automatic — every `memory_store` triggers regex-based extraction, with LLM extraction as a fallback when configured. No manual intervention needed.
 
 ### Maintenance (proactive triggers in [memory-hygiene](memory-hygiene.md), manual triggers below)
 | Tool | Trigger phrase | Cooldown |
 |------|---------------|----------|
 | `memory_governance` | "clean up memories", "check memory health", or proactively per [memory-hygiene](memory-hygiene.md) | 1 hour |
 | `memory_consolidate` | "check for contradictions", "fix conflicts" | 30 min |
 | `memory_reflect` | "find patterns", "summarize what you know" | 2 hours |
-| `memory_rebuild_index` | Only when governance reports `needs_rebuild=True` | — |
 | `memory_snapshot_delete` | When governance reports high snapshot auto_ratio, or user asks to clean snapshots | — |
 
-`memory_reflect` and `memory_extract_entities` support `mode` parameter:
+`memory_reflect` supports `mode` parameter:
 - `auto` (default): uses Memoria's internal LLM if configured, otherwise returns candidates for YOU to process
-- `candidates`: always returns raw data for YOU to synthesize/extract, then store results via `memory_store` or `memory_link_entities`
+- `candidates`: always returns raw data for YOU to synthesize, then store results via `memory_store`
 - `internal`: always uses Memoria's internal LLM (fails if not configured)

.kiro/steering/session-lifecycle.md

@@ -1,5 +1,7 @@
 ---
-inclusion: always
+inclusion: auto
+name: session-lifecycle
+description: Detailed session lifecycle management - bootstrap, mid-session re-retrieval, wrap-up cleanup. Use when starting conversations or managing session state.
 ---
 
 <!-- memoria-version: 0.1.0-->

README.md

@@ -29,7 +29,7 @@ Every memory change is tracked, auditable, and reversible — snapshots, branche
 - **Self-maintaining** — built-in governance detects contradictions, quarantines low-confidence memories
 - **Private by default** — local embedding model option, no data leaves your machine
 
-**Supported Agents:** [Kiro](https://kiro.dev) · [Cursor](https://cursor.sh) · [Claude Code](https://docs.anthropic.com/en/docs/claude-code) · [OpenClaw](plugins/openclaw/README.md) · Any MCP-compatible agent
+**Supported Agents:** [Kiro](https://kiro.dev) · [Cursor](https://cursor.sh) · [Claude Code](https://docs.anthropic.com/en/docs/claude-code) · [Codex](https://openai.com/index/introducing-codex/) · [OpenClaw](plugins/openclaw/README.md) · Any MCP-compatible agent
 
 **Storage Backend:** [MatrixOne](https://github.com/matrixorigin/matrixone) — Distributed database with native vector indexing
 
@@ -74,7 +74,7 @@ cd your-project
 memoria init -i   # Interactive wizard (recommended)
 ```
 
-This creates MCP config + steering rules for your AI tool (Kiro, Cursor, or Claude).
+This creates MCP config + steering rules for your AI tool (Kiro, Cursor, Claude, or Codex).
 
 ### 🦞 OpenClaw Plugin (Already Using OpenClaw?)
 
@@ -179,6 +179,7 @@ AI:  → memory_branch(name="eval_sqlite")
 - Kiro: `.kiro/steering/*.md`
 - Cursor: `.cursor/rules/*.mdc`
 - Claude: `.claude/rules/*.md`
+- Codex: `AGENTS.md`
 
 ### Update Rules
 
@@ -197,20 +198,27 @@ memoria rules --force
 |------|-------------|
 | `memory_store` | Store a new memory |
 | `memory_retrieve` | Retrieve relevant memories (call at conversation start) |
+| `memory_search` | Semantic search across all memories |
 | `memory_correct` | Update an existing memory |
 | `memory_purge` | Delete by ID or topic keyword |
-| `memory_search` | Semantic search across all memories |
+| `memory_list` | List active memories |
 | `memory_profile` | Get user's memory-derived profile |
+| `memory_feedback` | Record relevance feedback (useful/irrelevant/outdated/wrong) |
+| `memory_capabilities` | List available memory tools |
 
 ### Snapshots & Branches
 
 | Tool | Description |
 |------|-------------|
 | `memory_snapshot` | Create named snapshot |
+| `memory_snapshots` | List snapshots with pagination |
+| `memory_snapshot_delete` | Delete snapshots by name, prefix, or age |
 | `memory_rollback` | Restore to snapshot |
 | `memory_branch` | Create isolated branch |
+| `memory_branches` | List all branches |
 | `memory_checkout` | Switch branch |
 | `memory_merge` | Merge branch back |
+| `memory_branch_delete` | Delete a branch |
 | `memory_diff` | Preview merge changes |
 
 ### Maintenance
@@ -220,7 +228,8 @@ memoria rules --force
 | `memory_governance` | Quarantine low-confidence memories (1h cooldown) |
 | `memory_consolidate` | Detect contradictions (30min cooldown) |
 | `memory_reflect` | Synthesize insights (2h cooldown) |
-| `memory_extract_entities` | Build entity graph |
+
+> `memory_rebuild_index`, `memory_observe`, `memory_get_retrieval_params`, `memory_tune_params`, `memory_extract_entities`, and `memory_link_entities` are available via REST API but hidden from MCP tool listing — they are ops/debug tools not intended for agent use.
 
 Full API details: [API Reference Skill](skills/api-reference/SKILL.md)
 
@@ -257,7 +266,7 @@ If you're an AI agent helping a user set up Memoria:
 
 1. **Load the [Setup Skill](skills/setup/SKILL.md)** — it has step-by-step instructions
 2. **Ask before acting**:
-   - Which AI tool? (Kiro / Cursor / Claude)
+   - Which AI tool? (Kiro / Cursor / Claude / Codex)
    - MatrixOne database? (Docker / Cloud / existing)
    - Embedding service? (OpenAI / SiliconFlow / local)
 3. **Run `memoria init -i`** in the user's project directory