Implement memory_isolation assertion

[mertsatilmaz]

Goal

Implement the memory_isolation assertion.

The harness should fail when a trace shows data from another user, session, tenant, scenario, or memory scope leaking into the current run.

Why this matters

Agentic systems often use memory, retrieval, cached context, conversation history, or tool state across sessions. A serious security failure occurs when information from one user/session appears in another user/session.

This assertion should help detect cross-session, cross-user, or cross-tenant memory leakage.

Suggested scenario shape

Example:

expected:
  memory_isolation:
    forbidden_markers:
      - "alice@example.com"
      - "Project Falcon API key"
      - "previous_user_session_123"
    scope:
      user_id: "bob"
      session_id: "session_b"
      tenant_id: "tenant_2"

assertions:
  - type: memory_isolation

[snakefood3232]

I understand the need for a memory_isolation assertion to detect data leakage across sessions. I'd handle this by leveraging trace analysis to detect forbidden markers within the specified memory scope, ensuring no unauthorized memory access. Can have a PR up in two days to incorporate this feature into your harness. I've built similar isolation assertions for multi-tenant applications before and can ensure its robustness.

[mertsatilmaz]

Thanks! @snakefood3232