Skip to content

Commit b6743b9

Browse files
committed
具备可运行的 InMemory vs SQLite replay harness
1 parent 0fa1a63 commit b6743b9

5 files changed

Lines changed: 944 additions & 171 deletions

File tree

docs/replay_consistency_design.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
# Session / Memory / Summary 回放一致性设计
22

3-
本测试以 InMemory 作为基准,以 SQLite 驱动 `SqlSessionService``SqlMemoryService` 作为无需外部服务的持久化后端。每条 JSONL 轨迹使用相同的事件、state、memory、summary 操作分别回放,再通过公开读取接口取得快照。比较前会归一化自动生成 ID、时间戳和字典字段顺序;事件列表顺序、消息内容、state、工具调用、记忆检索结果以及 summary 的归属、位置和覆盖关系均严格比较。Summary event 的回放元数据包含版本、更新时间和关联 Session ID;摘要文本可做结构化归一化,但这些元数据、替换后的活动事件和历史事件不允许模糊匹配。`allowed_diff` 采用字段路径白名单,默认空集,报告会保留允许差异而非静默跳过。失败恢复轨迹模拟“写入成功但调用方收到异常”后重试,用于检查重复 memory、脏 summary 或错误 state。报告固定覆盖写入 `tests/sessions/session_memory_summary_diff_report.json`,逐条记录 case、后端、对象、字段路径及两侧值。当前默认运行 InMemory 与 SQLite;Redis/MySQL 后续仅需增加环境变量驱动的后端工厂,并复用同一轨迹、快照和比较逻辑
3+
测试以 InMemory 为基准,将同一 JSONL 轨迹回放到 SQLiteMock Redis;配置 REPLAY_REDIS_URL 时追加真实 Redis,轻量模式仅运行 InMemory。快照忽略时间戳和自动 ID,规范化字典顺序及摘要文本;事件顺序、state、memory、工具调用和摘要元数据严格比较。摘要由确定性模型驱动 SessionSummarizer 生成,session 归属、版本、更新时间和覆盖事件不得忽略。allowed_diff 仅接受明确路径,报告保留 session、event index、字段路径和两侧值。故障注入比较同一后端变异前后的快照,检测重复事件、脏状态、记忆重复及摘要丢失、覆盖和错绑
Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1 @@
1-
{"id":"06_memory","operations":[
2-
{"op":"append","author":"user","text":"I prefer tea in the morning"},
3-
{"op":"append","author":"agent","text":"I will remember your morning drink preference"},
4-
{"op":"append","author":"user","text":"I dislike very sweet drinks"},
5-
{"op":"append","author":"agent","text":"I will avoid sweet drink recommendations"},
6-
{"op":"append","author":"user","text":"Recommend something for tomorrow"},
7-
{"op":"append","author":"agent","text":"I will recommend unsweetened tea"},
8-
{"op":"summary","text":"User prefers unsweetened tea in the morning and dislikes very sweet drinks.","keep_recent":2},
9-
{"op":"store_memory"},{"op":"search_memory","query":"tea"}]}
1+
{"id":"06_memory","operations":[{"op":"append","author":"user","text":"I prefer tea in the morning"},{"op":"append","author":"agent","text":"I will remember your morning drink preference"},{"op":"append","author":"user","text":"I dislike very sweet drinks"},{"op":"append","author":"agent","text":"I will avoid sweet drink recommendations"},{"op":"append","author":"user","text":"Recommend something for tomorrow"},{"op":"append","author":"agent","text":"I will recommend unsweetened tea"},{"op":"summary","text":"User prefers unsweetened tea in the morning and dislikes very sweet drinks.","keep_recent":2},{"op":"store_memory"},{"op":"search_memory","query":"tea"}]}
Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1 @@
1-
{"id":"10_failure_recovery","operations":[
2-
{"op":"append","author":"user","text":"I need to recover my previous travel preferences"},
3-
{"op":"append","author":"agent","text":"I will restore your saved travel context"},
4-
{"op":"append","author":"user","text":"My preference is budget train travel"},
5-
{"op":"append","author":"agent","text":"Budget train preference recorded"},
6-
{"op":"append","author":"user","text":"Keep hotels near stations"},
7-
{"op":"append","author":"agent","text":"Station hotel preference recorded"},
8-
{"op":"summary_retry","text":"Recovery context: user prefers budget train travel and hotels near stations.","keep_recent":2},
9-
{"op":"store_memory_retry"},{"op":"search_memory","query":"train"}]}
1+
{"id":"10_failure_recovery","operations":[{"op":"append","author":"user","text":"I need to recover my previous travel preferences"},{"op":"append","author":"agent","text":"I will restore your saved travel context"},{"op":"append","author":"user","text":"My preference is budget train travel"},{"op":"append","author":"agent","text":"Budget train preference recorded"},{"op":"append","author":"user","text":"Keep hotels near stations"},{"op":"append","author":"agent","text":"Station hotel preference recorded","state_delta":{"recovery_status":"complete"}},{"op":"summary_retry","text":"Recovery context: user prefers budget train travel and hotels near stations.","keep_recent":2},{"op":"store_memory_retry"},{"op":"search_memory","query":"train"}]}

0 commit comments

Comments
 (0)