Skip to content

Add MCP trace replay fixture family#146

Merged
ProfRandom92 merged 8 commits into
mainfrom
codex/add-mcp-trace-replay-fixture-family
May 20, 2026
Merged

Add MCP trace replay fixture family#146
ProfRandom92 merged 8 commits into
mainfrom
codex/add-mcp-trace-replay-fixture-family

Conversation

@ProfRandom92

Copy link
Copy Markdown
Owner

Motivation

  • Introduce a manifest-driven MCP-style trace replay fixture family to exercise deterministic replay-safety contracts for tool-order, validation-before-unsafe-action, dependency preservation, recovery-path availability, and capability-boundary preservation.
  • Provide a small, deterministic set of baseline→mild→moderate→severe fixtures so existing validators and artifact generators can express and check MCP-style operational trace guarantees without runtime MCP integration.
  • Keep the change scoped to fixtures, manifest entries, expected contract/failure JSONs, and regenerated committed admissibility artifacts to maintain a fixture-first workflow.

Description

  • Added the new fixture family mcp_trace_replay with four fixtures: mcp_trace_replay_v1, mcp_trace_replay_mild_v1, mcp_trace_replay_moderate_v1, and mcp_trace_replay_degraded_v1, each following existing fixture layout (original/, reconstructed/, original/contracts/, expected/, README.md).
  • Implemented required contract JSONs inside each fixture using existing validator types: tool_call_order_preserved (ordering), validation_before_unsafe_action (ordering), dependency_chain_preserved (causality), recovery_path_available (reachability), and capability_boundary_respected (invariant/no_orphan_dependencies).
  • Updated fixtures/manifest.json in deterministic order and updated tests that assert manifest ordering and multi-family coverage (tests/test_fixture_manifest.py and tests/test_multi_family_admissibility_artifact.py).
  • Regenerated impacted committed artifacts using existing scripts which updated artifacts/multi_family_admissibility_results.json and artifacts/multi_family_admissibility_curves.svg; no runtime or external integrations were added.

Agent PR summary (per AGENTS.md): Summary: Added mcp_trace_replay fixture family covering baseline→mild→moderate→severe and corresponding contracts and expectations. Changed files: updated fixtures/manifest.json, added fixtures/mcp_trace_replay_* directories, updated tests/test_fixture_manifest.py and tests/test_multi_family_admissibility_artifact.py, and regenerated artifacts/multi_family_admissibility_results.json and artifacts/multi_family_admissibility_curves.svg. Testing: ran targeted pytest suites and artifact generators and validated repository npm run check. Risks: expected-failure labels and artifact diffs are intentionally strict and may require tuning if validator semantics change. Next: follow-up PR can add per-family MCP results artifacts when ready.

Testing

  • Ran pytest tests/test_failure_taxonomy.py -q, pytest tests/test_fixture_manifest.py -q, pytest tests/test_multi_family_admissibility_artifact.py -q, and pytest tests/test_multi_family_svg_renderer.py -q, and all targeted tests passed.
  • Ran npm run generate:multi-family-admissibility and npm run generate:multi-family-svg to regenerate committed multi-family artifacts and verified output matches the committed files.
  • Ran full validation via npm run check which completed successfully (test suite run reported all tests passing).
  • No failing automated tests remain after artifact regeneration and test updates.

Codex Task

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the mcp_trace_replay family of test fixtures, providing deterministic data for baseline, mild, moderate, and severe degradation scenarios. The changes include comprehensive fixture definitions with associated contracts, dependency graphs, and expected admissibility results, alongside updates to the global manifest and visualization artifacts. I have no feedback to provide as there were no review comments to evaluate.

@ProfRandom92 ProfRandom92 merged commit 7d454e2 into main May 20, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant