Skip to content

Add cross-domain operational dependency workflow fixture family#142

Merged
ProfRandom92 merged 7 commits into
mainfrom
codex/add-cross-domain-operational-fixture-family
May 19, 2026
Merged

Add cross-domain operational dependency workflow fixture family#142
ProfRandom92 merged 7 commits into
mainfrom
codex/add-cross-domain-operational-fixture-family

Conversation

@ProfRandom92

Copy link
Copy Markdown
Owner

Summary:

  • Add a third deterministic fixture family cross_domain_operational_dependency_workflow with four degradation levels (baseline, mild, moderate, severe) that exercise cross-domain dependency chains, ordering/governance gating, recovery/rollback reachability, and no-orphan invariants using only registered failure labels.

Changed files:

  • Added fixture directories and payloads for the four new fixtures under fixtures/:
    • fixtures/cross_domain_operational_dependency_workflow_v1
    • fixtures/cross_domain_operational_dependency_workflow_mild_v1
    • fixtures/cross_domain_operational_dependency_workflow_moderate_v1
    • fixtures/cross_domain_operational_dependency_workflow_degraded_v1
  • Added contract JSONs for the family (dependency_chain_preserved, policy_ordering_governance_before_deploy, recovery_or_rollback_reachable, no_orphan_operational_steps).
  • Updated fixtures/manifest.json to register the new family and expected failure labels.
  • Regenerated deterministic multi-family artifacts:
    • artifacts/multi_family_admissibility_results.json
    • artifacts/multi_family_admissibility_curves.svg
  • Updated tests that encode concrete fixture/family expectations:
    • tests/test_multi_family_admissibility_artifact.py
    • tests/test_fixture_manifest.py

Testing:

  • Ran artifact generation and rendering: python scripts/generate_multi_family_admissibility_artifact.py and python scripts/render_multi_family_admissibility_svg.py (artifacts updated).
  • Unit/test validations executed and passed locally: pytest tests/test_failure_taxonomy.py -q, pytest tests/test_manifest_fixture_families.py -q, pytest tests/test_multi_family_admissibility_artifact.py -q, pytest tests/test_multi_family_svg_renderer.py -q, pytest tests/test_fixture_manifest.py -q (all passed during the final run), and full repo check npm run check completed (including pytest suite) with all tests passing (222 passed).

Risks:

  • The new family adds expected deterministic artifacts and fixed test expectations; any subsequent edits to fixture ordering, manifest ordering, or artifact generation will require synchronized updates to the manifest/tests/artifacts.
  • This change touches committed deterministic artifacts (JSON/SVG); reviewers should validate artifact parity expectations for CI.

Next:

  • Open as Draft PR for CI validation in upstream runner and code review; preserve small/focused scope and wait for green Actions before marking ready.

Codex Task

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the cross_domain_operational_dependency_workflow fixture family, providing a comprehensive set of baseline and degraded scenarios (mild, moderate, severe) along with updated admissibility curves and results. The review feedback highlights several inconsistencies between the generated results and the defined expectations; specifically, the INVARIANT_VIOLATION label and no_orphan_operational_steps contract failure were missing from the manifest and fixture metadata for mild and moderate levels. Additionally, an extraneous setup_workspace event was found in the reconstructed baseline trace that does not align with the original source data.

Comment on lines +3 to +6
{
"action": "setup_workspace",
"step": 0
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The reconstructed trace for the baseline fixture includes an extra setup_workspace event at step 0. This event is not present in the original trace (original/trace.json) and has no corresponding node in the dependency graph. This inconsistency makes the baseline reconstruction "noisy" compared to the source data and other fixtures in the family.

Comment thread fixtures/manifest.json
Comment on lines +181 to +183
"expected_failure_labels": [
"RECOVERY_PATH_INVALID"
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The expected_failure_labels for the mild degradation level is missing the INVARIANT_VIOLATION label. The generated results in artifacts/multi_family_admissibility_results.json (line 138) show that this fixture triggers an invariant violation because the recovery nodes are present in the graph but disconnected (orphans).

      "expected_failure_labels": [
        "INVARIANT_VIOLATION",
        "RECOVERY_PATH_INVALID"
      ]

Comment thread fixtures/manifest.json
Comment on lines +199 to +202
"expected_failure_labels": [
"CAUSAL_DEPENDENCY_LOSS",
"RECOVERY_PATH_INVALID"
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The expected_failure_labels for the moderate degradation level is missing the INVARIANT_VIOLATION label, which is observed in the admissibility results artifact (line 164).

      "expected_failure_labels": [
        "CAUSAL_DEPENDENCY_LOSS",
        "INVARIANT_VIOLATION",
        "RECOVERY_PATH_INVALID"
      ]

Comment on lines +5 to +10
"must_fail_contracts": [
"recovery_or_rollback_reachable"
],
"expected_failure_labels": [
"RECOVERY_PATH_INVALID"
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The must_fail_contracts and expected_failure_labels are incomplete. They should include no_orphan_operational_steps and INVARIANT_VIOLATION respectively, to match the actual behavior of the fixture as recorded in the admissibility results.

  "must_fail_contracts": [
    "no_orphan_operational_steps",
    "recovery_or_rollback_reachable"
  ],
  "expected_failure_labels": [
    "INVARIANT_VIOLATION",
    "RECOVERY_PATH_INVALID"
  ]

Comment on lines +5 to +12
"must_fail_contracts": [
"dependency_chain_preserved",
"recovery_or_rollback_reachable"
],
"expected_failure_labels": [
"CAUSAL_DEPENDENCY_LOSS",
"RECOVERY_PATH_INVALID"
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The expectations for the moderate fixture should be updated to include the invariant violation detected during artifact generation.

  "must_fail_contracts": [
    "dependency_chain_preserved",
    "no_orphan_operational_steps",
    "recovery_or_rollback_reachable"
  ],
  "expected_failure_labels": [
    "CAUSAL_DEPENDENCY_LOSS",
    "INVARIANT_VIOLATION",
    "RECOVERY_PATH_INVALID"
  ]

@ProfRandom92 ProfRandom92 merged commit db9aeb9 into main May 19, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant