Skip to content

Commit 2d5d433

Browse files
committed
test(sessions): tighten replay harness coverage
Extend replay snapshots to retain all session aliases, preserve memory query observations across restarts, and allow runtime faults to target non-active sessions. Add cross-user isolation coverage plus targeted harness tests for duplicate query names and restart-stable memory observations.
1 parent c83f127 commit 2d5d433

5 files changed

Lines changed: 478 additions & 79 deletions

File tree

tests/sessions/replay_acceptance_design.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Replay Consistency Design Note
22

33
## 设计说明
4-
本框架以统一 `ReplayCase` 协议驱动 `InMemory`、文件型 `SQLite` 与可选 `Redis` 后端,先生成 `session/state/memory/summary` 四类快照,再做归一化比较。归一化只移除非业务噪声,不掩盖事件顺序、state 最终值、memory 检索结果及 summary 归属与覆盖关系;其中普通时间戳做精度收敛,`summary_timestamp` 使用 case 级确定性时钟参与比较。summary 比较分为两层:一层比较摘要文本与压缩后的事件窗口,另一层比较 `summary_id/version/replaces/session_id` 等 lineage 元数据。为验证保存/读取语义,summary 元数据会随 summary event 一起持久化,`SQLite/Redis` 适配器在抓取快照前执行一次关服务后重开读回,并支持在 case 中显式插入中途重启步骤验证“重启后继续写”。协议还支持同一 case 内用 `session_alias` 驱动多个 session,以覆盖跨 sessionmemory 聚合和 `app:/user:/temp:` 状态可见性差异。负例分为 snapshot mutation 与 runtime fault 两类,前者验证比较器精度,后者验证重复写入、中途失败和运行时污染场景。轻量模式默认运行 `InMemory + SQLite`,集成模式通过环境变量启用 `Redis`。
4+
本框架以统一 `ReplayCase` 协议驱动 `InMemory`、文件型 `SQLite` 与可选 `Redis` 后端,先生成 `session/state/memory/summary` 四类快照,再做归一化比较。归一化只移除非业务噪声,不掩盖事件顺序、state 最终值、memory 检索结果及 summary 归属与覆盖关系;其中普通时间戳做精度收敛,`summary_timestamp` 使用 case 级确定性时钟参与比较。summary 比较分为两层:一层比较摘要文本与压缩后的事件窗口,另一层比较 `summary_id/version/replaces/session_id` 等 lineage 元数据。为验证保存/读取语义,summary 元数据会随 summary event 一起持久化,`SQLite/Redis` 适配器在抓取快照前执行一次关服务后重开读回,并支持在 case 中显式插入中途重启步骤验证“重启后继续写”。协议支持用 `session_alias` 在同一 case 内驱动多 session / 多 user,并在最终快照中同时保留 `active session` 与 `sessions_by_alias` 视图,避免非活跃 session 损坏被漏检。memory 检索按 step 记录为独立 observation,不会在重启后被重算覆盖;同名 query 可跨 session 重复使用。负例分为 snapshot mutation 与 runtime fault 两类,二者都支持 alias 级注入,前者验证比较器精度,后者验证重复写入、中途失败、非活跃 session 污染和运行时 summary 破坏。轻量模式默认运行 `InMemory + SQLite`,集成模式通过环境变量启用 `Redis`。
55

66
## 官方 10 条验收 Case
77
| Case ID | 场景 | 验收点 |
@@ -23,6 +23,10 @@
2323
- `duplicate_event_runtime_fault`:补充重复写入异常。
2424
- `runtime_state_corruption_fault`:补充运行时 state 污染。
2525
- `runtime_summary_loss_fault`:补充运行时 summary 丢失。
26+
- `non_active_session_summary_loss_fault`:补充非活跃 session 的 summary 损坏检测。
2627
- `cross_session_memory_aggregation`:补充同一 app/user 下跨 session 的 memory 聚合语义。
2728
- `restart_mid_replay_after_summary`:补充 summary 持久化后中途重启再续写的恢复语义。
2829
- `state_namespace_roundtrip`:补充 `app:/user:/temp:` 状态命名空间在跨 session 和重启后的可见性语义。
30+
- `cross_user_memory_isolation`:补充同一 app 下不同 user 的长期记忆隔离语义。
31+
- `duplicate_memory_query_name_across_sessions`:用定向测试覆盖跨 alias 的同名 memory query 不应互相覆盖。
32+
- `memory_query_observation_survives_restart`:用定向测试覆盖重启后 memory query 观测不得被后续结果回填。

tests/sessions/replay_cases.py

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -463,6 +463,47 @@
463463
),
464464
),
465465
),
466+
ReplayCase(
467+
case_id="non_active_session_summary_loss_fault",
468+
description="A runtime summary deletion on a non-active session alias must still be detected through alias-scoped snapshots.",
469+
session_id="replay-negative-target-session",
470+
enable_summary=True,
471+
summary_keep_recent_count=2,
472+
store_historical_events=True,
473+
expected_diff_paths=("sessions_by_alias.source.summary",),
474+
runtime_faults=(
475+
RuntimeFault(
476+
backend_name=_PERSISTENT_BACKEND,
477+
after_step=6,
478+
operation=RuntimeFaultOperation.DELETE_SUMMARY,
479+
session_alias="source",
480+
),
481+
),
482+
steps=(
483+
ReplayStep.create_session(session_alias="source", session_id="replay-negative-source-session"),
484+
ReplayStep.append_event(
485+
EventSpec(author="user", role="user", text="Please summarize my architecture notes."),
486+
session_alias="source",
487+
),
488+
ReplayStep.append_event(
489+
EventSpec(author="assistant", role="model", text="Sure, continue with the details."),
490+
session_alias="source",
491+
),
492+
ReplayStep.append_event(
493+
EventSpec(author="user", role="user", text="We run SQLite for local replay and Redis in production."),
494+
session_alias="source",
495+
),
496+
ReplayStep.append_event(
497+
EventSpec(author="assistant", role="model", text="I will summarize the backend setup."),
498+
session_alias="source",
499+
),
500+
ReplayStep.create_summary(force=True, session_alias="source"),
501+
ReplayStep.create_session(session_alias="default", session_id="replay-negative-target-session"),
502+
ReplayStep.append_event(
503+
EventSpec(author="user", role="user", text="Switch focus to the current session."),
504+
),
505+
),
506+
),
466507
)
467508

468509

@@ -575,6 +616,92 @@
575616
),
576617
),
577618
),
619+
ReplayCase(
620+
case_id="duplicate_memory_query_name_across_sessions",
621+
description="Two sessions may reuse the same query name without overwriting earlier memory observations.",
622+
session_id="replay-duplicate-query-target",
623+
steps=(
624+
ReplayStep.create_session(session_alias="source", session_id="replay-duplicate-query-source"),
625+
ReplayStep.append_event(
626+
EventSpec(author="user", role="user", text="Please remember that I prefer dragon well tea."),
627+
session_alias="source",
628+
),
629+
ReplayStep.append_event(
630+
EventSpec(author="assistant", role="model", text="Dragon well tea preference recorded."),
631+
session_alias="source",
632+
),
633+
ReplayStep.store_memory(session_alias="source"),
634+
ReplayStep.search_memory(
635+
name="shared_preference_search",
636+
query="dragon well",
637+
limit=5,
638+
session_alias="source",
639+
),
640+
ReplayStep.restart_services(),
641+
ReplayStep.create_session(session_alias="default", session_id="replay-duplicate-query-target"),
642+
ReplayStep.search_memory(
643+
name="shared_preference_search",
644+
query="dragon well",
645+
limit=5,
646+
),
647+
),
648+
),
649+
ReplayCase(
650+
case_id="memory_query_observation_survives_restart",
651+
description="Memory query observations should retain their original step-time results across restarts and later writes.",
652+
session_id="replay-memory-observation",
653+
steps=(
654+
ReplayStep.create_session(),
655+
ReplayStep.append_event(
656+
EventSpec(author="user", role="user", text="Please remember that my favorite tea is oolong."),
657+
),
658+
ReplayStep.append_event(
659+
EventSpec(author="assistant", role="model", text="I will remember your oolong preference."),
660+
),
661+
ReplayStep.store_memory(),
662+
ReplayStep.search_memory(name="tea_preference", query="oolong", limit=5),
663+
ReplayStep.restart_services(),
664+
ReplayStep.append_event(
665+
EventSpec(author="user", role="user", text="Also remember that I enjoy jasmine tea."),
666+
),
667+
ReplayStep.append_event(
668+
EventSpec(author="assistant", role="model", text="I will remember the jasmine preference too."),
669+
),
670+
ReplayStep.store_memory(),
671+
ReplayStep.search_memory(name="tea_preference", query="jasmine", limit=5),
672+
),
673+
),
674+
ReplayCase(
675+
case_id="cross_user_memory_isolation",
676+
description="Memory saved for one user must not become visible to another user in the same application.",
677+
session_id="replay-cross-user-target",
678+
steps=(
679+
ReplayStep.create_session(
680+
session_alias="source",
681+
session_id="replay-cross-user-source",
682+
user_id="replay-user-a",
683+
),
684+
ReplayStep.append_event(
685+
EventSpec(author="user", role="user", text="Please remember that I prefer matcha desserts."),
686+
session_alias="source",
687+
),
688+
ReplayStep.append_event(
689+
EventSpec(author="assistant", role="model", text="Matcha dessert preference recorded."),
690+
session_alias="source",
691+
),
692+
ReplayStep.store_memory(session_alias="source"),
693+
ReplayStep.create_session(
694+
session_alias="default",
695+
session_id="replay-cross-user-target",
696+
user_id="replay-user-b",
697+
),
698+
ReplayStep.search_memory(
699+
name="cross_user_matcha_search",
700+
query="matcha",
701+
limit=5,
702+
),
703+
),
704+
),
578705
)
579706

580707

@@ -602,9 +729,17 @@
602729
_NEGATIVE_CASES[4], # duplicate_event_runtime_fault
603730
_NEGATIVE_CASES[5], # runtime_state_corruption_fault
604731
_NEGATIVE_CASES[6], # runtime_summary_loss_fault
732+
_NEGATIVE_CASES[9], # non_active_session_summary_loss_fault
605733
_ROBUSTNESS_CASES[0], # cross_session_memory_aggregation
606734
_ROBUSTNESS_CASES[1], # restart_mid_replay_after_summary
607735
_ROBUSTNESS_CASES[2], # state_namespace_roundtrip
736+
_ROBUSTNESS_CASES[5], # cross_user_memory_isolation
737+
)
738+
739+
740+
REPLAY_TARGETED_CASES: tuple[ReplayCase, ...] = (
741+
_ROBUSTNESS_CASES[3], # duplicate_memory_query_name_across_sessions
742+
_ROBUSTNESS_CASES[4], # memory_query_observation_survives_restart
608743
)
609744

610745

0 commit comments

Comments
 (0)