[cross-repo from server#312] Conformance blocker: expand replay coverage beyond current smoke (#632)

Author: durable-workflow-ops

docs/architecture/platform-conformance-suite.md

@@ -17,7 +17,7 @@ The machine-readable mirror of this document is
 `Workflow\V2\Support\PlatformConformanceSuite`, exported by the
 standalone `workflow-server` from `GET /api/cluster/info` under
 `platform_conformance_suite`. Schema:
-`durable-workflow.v2.platform-conformance.suite`, version `2`.
+`durable-workflow.v2.platform-conformance.suite`, version `3`.
 
 ## Why one suite
 
@@ -73,7 +73,7 @@ them from the declared locations.
 | `control_plane_request_response` | `cli`, `sdk-python` | `tests/fixtures/control-plane/` | Frozen request bodies and response shapes for `workflow.start`, `signal`, `query`, `update`, `cancel`, `task-history`, namespace storage. |
 | `worker_task_lifecycle` | `cli`, `sdk-python`, `server` | `tests/fixtures/external-task-input/`, `tests/fixtures/external-task-result/` | Task input envelopes (poll → claim → run) and task result envelopes (complete, fail, cancel, heartbeat) used by every conforming worker. |
 | `signal_query_runtime_contract` | `workflow`, `server`, `cli`, `sdk-python`, `waterline` | `docs/architecture/platform-conformance-suite.md`, `docs/architecture/query-and-live-debug.md`, `src/V2/Client/ControlPlaneClient.php`, `tests/Unit/V2/ControlPlaneClientTest.php`, `tests/Feature/SignalReplayTest.php`, `tests/Feature/V2/V2QueryWorkflowTest.php`, `tests/Feature/WorkflowControlPlaneTest.php`, `tests/Feature/WorkflowQueryTaskBrokerTest.php`, `tests/Commands/`, `tests/test_signals.py`, `tests/test_queries.py`, `tests/test_worker.py`, `CONFORMANCE.md` | Live published-artifact scenarios for signal delivery and query consistency across PHP and Python workers, CLI and SDK clients, replay timing, terminal runs, malformed payloads, and operator visibility. |
-| `history_replay_bundles` | `workflow`, `sdk-python` | `tests/Fixtures/V2/GoldenHistory/`, `tests/fixtures/golden_history/` | Frozen history event bundles. A conforming SDK must replay each bundle and reproduce the documented final command sequence. |
+| `history_replay_bundles` | `durable-workflow.github.io`, `workflow`, `sdk-python` | `static/platform-conformance/replay-runtime-scenarios.json`, `tests/Fixtures/V2/GoldenHistory/`, `tests/fixtures/golden_history/` | Deterministic replay coverage for frozen history bundles, worker restart replay, adversarial refusal, and in-flight signal timing across the official PHP and Python runtimes. |
 | `failure_repair_actionability` | `server`, `workflow` | `docs/contracts/external-task-result.md`, `docs/contracts/replay-verification.md`, fixture pointers therein | Failure objects and repair / actionability shapes for stuck tasks, deterministic failure, and replay-mismatch surfaces. |
 | `cli_json_envelopes` | `cli` | `tests/fixtures/control-plane/`, `schemas/` | The `--output=json` and `--output=jsonl` envelopes that automation depends on. Diagnostic-only fields are listed and excluded from the contract diff. |
 | `waterline_observer_envelopes` | `waterline` | (TBD: `tests/fixtures/observer/`) | The `/waterline/api/v2/*` shapes and operator dashboard JSON envelopes. Status: provisional — fixtures land alongside the next Waterline contract slice. |
@@ -127,6 +127,31 @@ Required scenarios:
   stable reason when live query values are intentionally not materialized
   in read-only detail responses.
 
+### History replay runtime contract
+
+The `history_replay_bundles` category is also stable and load-bearing.
+It must run against published install channels only, pin the resolved
+artifact versions in the result, and name every required replay scenario
+as passed, failed, or unsupported with a linked finding. A smoke-only
+run is nonconforming even when the smoke path passes.
+
+Required scenarios are published in the public replay scenario manifest
+at `static/platform-conformance/replay-runtime-scenarios.json` and
+include:
+
+- published-artifact install-only evidence for server, CLI, PHP runtime,
+  and Python SDK;
+- PHP and Python completed-history replay for activity, signal/update,
+  wait-condition, version-marker, and saga-compensation families;
+- PHP and Python worker-restart replay for completed-query, activity,
+  signal/update, wait-condition, version-marker, and saga-compensation
+  state;
+- PHP and Python divergent-code refusal with actionable
+  non-determinism diagnostics;
+- server history mutation and malformed-history refusal through the
+  documented replay verification surface;
+- PHP and Python in-flight signal restart timing.
+
 ## Pass / fail rules
 
 The harness runs each fixture against the implementation under test and
@@ -154,10 +179,11 @@ emits a structured result. The rules below are normative.
    release does not conform.
 
 5. **Stable runtime scenario coverage.** A stable runtime category such
-   as `signal_query_runtime_contract` passes only when every scenario it
-   declares records a pass, fail, or unsupported result with resolved
-   artifact versions and linked findings. A smoke-only subset or omitted
-   scenario is nonconforming, not provisional.
+   as `signal_query_runtime_contract` or `history_replay_bundles` passes
+   only when every scenario it declares records a pass, fail, or
+   unsupported result with resolved artifact versions and linked
+   findings. A smoke-only subset or omitted scenario is nonconforming,
+   not provisional.
 
 6. **Provisional categories warn but do not fail.** A failed fixture in
    a provisional category emits a warning in the harness output. A
@@ -276,10 +302,11 @@ docs. It indexes them under one normative declaration so a single
   `sdk-python/tests/fixtures/control-plane/` are the existing parity
   fixtures. The suite cites them as the
   `control_plane_request_response` source-of-truth.
-- `tests/Fixtures/V2/GoldenHistory/` (this repo) and
-  `sdk-python/tests/fixtures/golden_history/` are the existing replay
-  bundles. The suite cites them as the `history_replay_bundles`
-  source-of-truth.
+- `static/platform-conformance/replay-runtime-scenarios.json`,
+  `tests/Fixtures/V2/GoldenHistory/` (this repo), and
+  `sdk-python/tests/fixtures/golden_history/` are the replay scenario and
+  bundle authorities. The suite cites them as the
+  `history_replay_bundles` source-of-truth.
 
 ## Changing this document
 

src/V2/Exceptions/HistoryEventShapeMismatchException.php

@@ -15,12 +15,18 @@ public function __construct(
         public readonly int $workflowSequence,
         public readonly string $expectedHistoryShape,
         public readonly array $recordedEventTypes,
+        ?string $detail = null,
     ) {
+        $detail = is_string($detail) && $detail !== ''
+            ? " {$detail}"
+            : '';
+
         parent::__construct(sprintf(
-            'Workflow history at workflow sequence %d recorded [%s], but the current workflow yielded %s. Keep yielded workflow steps stable across deployments or run this workflow on a compatible build.',
+            'Workflow history at workflow sequence %d recorded [%s], but the current workflow yielded %s.%s Keep yielded workflow steps stable across deployments or run this workflow on a compatible build.',
             $workflowSequence,
             implode(', ', $recordedEventTypes),
             $expectedHistoryShape,
+            $detail,
         ));
     }
 }

src/V2/Support/PlatformConformanceSuite.php

@@ -29,7 +29,7 @@ final class PlatformConformanceSuite
 {
     public const SCHEMA = 'durable-workflow.v2.platform-conformance.suite';
 
-    public const VERSION = 2;
+    public const VERSION = 3;
 
     public const RESULT_SCHEMA = 'durable-workflow.v2.platform-conformance.result';
 
@@ -302,8 +302,12 @@ private static function fixtureCatalog(): array
             ],
             'history_replay_bundles' => [
                 'status' => self::CATEGORY_STATUS_STABLE,
-                'description' => 'Frozen history event bundles. A conforming SDK must replay each bundle and reproduce the documented final command sequence.',
+                'description' => 'Deterministic replay coverage for frozen history bundles, worker restart replay, adversarial refusal, and in-flight signal timing across the official PHP and Python runtimes.',
                 'sources' => [
+                    [
+                        'repository' => 'durable-workflow.github.io',
+                        'path' => 'static/platform-conformance/replay-runtime-scenarios.json',
+                    ],
                     [
                         'repository' => 'workflow',
                         'path' => 'tests/Fixtures/V2/GoldenHistory/',
@@ -313,7 +317,8 @@ private static function fixtureCatalog(): array
                         'path' => 'tests/fixtures/golden_history/',
                     ],
                 ],
-                'authority_doc' => 'https://github.com/durable-workflow/workflow/blob/v2/docs/api-stability.md',
+                'authority_doc' => 'https://durable-workflow.github.io/docs/2.0/platform-conformance, https://github.com/durable-workflow/server/blob/main/docs/contracts/replay-verification.md, https://github.com/durable-workflow/workflow/blob/v2/docs/api-stability.md',
+                'required_scenarios' => self::replayRequiredScenarios(),
             ],
             'failure_repair_actionability' => [
                 'status' => self::CATEGORY_STATUS_STABLE,
@@ -400,7 +405,10 @@ private static function passFailRules(): array
             ],
             'stable_runtime_scenario_coverage' => [
                 'rule' => 'A stable runtime fixture category passes only when every required scenario it declares records a pass, fail, or unsupported result with artifact versions and linked findings. A smoke-only subset or omitted scenario is nonconforming, not provisional.',
-                'applies_to_categories' => ['signal_query_runtime_contract'],
+                'applies_to_categories' => [
+                    'signal_query_runtime_contract',
+                    'history_replay_bundles',
+                ],
             ],
             'provisional_categories_warn_only' => [
                 'rule' => 'A failed fixture in a provisional category emits a warning in the harness output and does not block the release. The category becomes load-bearing when promoted to stable in a later suite version.',
@@ -411,6 +419,44 @@ private static function passFailRules(): array
         ];
     }
 
+    /**
+     * @return list<string>
+     */
+    private static function replayRequiredScenarios(): array
+    {
+        return [
+            'published_artifact_install_only',
+            'python_completed_history_activity_replay',
+            'python_completed_history_signal_update_replay',
+            'python_completed_history_wait_condition_replay',
+            'python_completed_history_version_marker_replay',
+            'python_completed_history_saga_compensation_replay',
+            'php_completed_history_activity_replay',
+            'php_completed_history_signal_update_replay',
+            'php_completed_history_wait_condition_replay',
+            'php_completed_history_version_marker_replay',
+            'php_completed_history_saga_compensation_replay',
+            'python_worker_restart_completed_query',
+            'python_worker_restart_activity_state',
+            'python_worker_restart_signal_update_state',
+            'python_worker_restart_wait_condition_state',
+            'python_worker_restart_version_marker_state',
+            'python_worker_restart_saga_compensation_state',
+            'php_worker_restart_completed_query',
+            'php_worker_restart_activity_state',
+            'php_worker_restart_signal_update_state',
+            'php_worker_restart_wait_condition_state',
+            'php_worker_restart_version_marker_state',
+            'php_worker_restart_saga_compensation_state',
+            'python_code_divergence_refusal',
+            'php_code_divergence_refusal',
+            'server_history_mutation_refusal',
+            'malformed_history_refusal',
+            'python_in_flight_signal_restart_timing',
+            'php_in_flight_signal_restart_timing',
+        ];
+    }
+
     /**
      * @return array<string, mixed>
      */

src/V2/Support/QueryStateReplayer.php

@@ -76,7 +76,9 @@ public function replayState(WorkflowRun $run): ReplayState
             }
 
             if ($current instanceof LocalActivityCall) {
-                WorkflowStepHistory::assertCompatible($run, $sequence, WorkflowStepHistory::LOCAL_ACTIVITY);
+                WorkflowStepHistory::assertCompatible($run, $sequence, WorkflowStepHistory::LOCAL_ACTIVITY, [
+                    'activity_type' => $current->activity,
+                ]);
 
                 $activityCompletion = $this->activityCompletionEvent($run, $sequence);
 
@@ -113,7 +115,9 @@ public function replayState(WorkflowRun $run): ReplayState
             }
 
             if ($current instanceof ActivityCall) {
-                WorkflowStepHistory::assertCompatible($run, $sequence, WorkflowStepHistory::ACTIVITY);
+                WorkflowStepHistory::assertCompatible($run, $sequence, WorkflowStepHistory::ACTIVITY, [
+                    'activity_type' => $current->activity,
+                ]);
 
                 $activityCompletion = $this->activityCompletionEvent($run, $sequence);
 
@@ -268,7 +272,9 @@ public function replayState(WorkflowRun $run): ReplayState
 
             if ($current instanceof VersionCall) {
                 $this->applyRecordedUpdates($run, $workflow, $sequence);
-                WorkflowStepHistory::assertCompatible($run, $sequence, WorkflowStepHistory::VERSION_MARKER);
+                WorkflowStepHistory::assertCompatible($run, $sequence, WorkflowStepHistory::VERSION_MARKER, [
+                    'change_id' => $current->changeId,
+                ]);
 
                 $versionEvent = $this->versionMarkerEvent($run, $sequence);
                 $resolution = VersionResolver::resolve($run, $versionEvent, $current, $sequence);
@@ -307,7 +313,9 @@ public function replayState(WorkflowRun $run): ReplayState
 
             if ($current instanceof SignalCall) {
                 $this->applyRecordedUpdates($run, $workflow, $sequence);
-                WorkflowStepHistory::assertCompatible($run, $sequence, WorkflowStepHistory::SIGNAL_WAIT);
+                WorkflowStepHistory::assertCompatible($run, $sequence, WorkflowStepHistory::SIGNAL_WAIT, [
+                    'signal_name' => $current->name,
+                ]);
 
                 $signalEvent = $this->appliedSignalEvent($run, $sequence, $current);
 
@@ -342,7 +350,9 @@ public function replayState(WorkflowRun $run): ReplayState
 
             if ($current instanceof ChildWorkflowCall) {
                 $this->applyRecordedUpdates($run, $workflow, $sequence);
-                WorkflowStepHistory::assertCompatible($run, $sequence, WorkflowStepHistory::CHILD_WORKFLOW);
+                WorkflowStepHistory::assertCompatible($run, $sequence, WorkflowStepHistory::CHILD_WORKFLOW, [
+                    'child_workflow_type' => $current->workflow,
+                ]);
 
                 $resolutionEvent = ChildRunHistory::resolutionEventForSequence($run, $sequence);
                 $childRun = ChildRunHistory::childRunForSequence($run, $sequence);
@@ -453,6 +463,9 @@ public function replayState(WorkflowRun $run): ReplayState
                         $call instanceof ActivityCall
                             ? WorkflowStepHistory::ACTIVITY
                             : WorkflowStepHistory::CHILD_WORKFLOW,
+                        $call instanceof ActivityCall
+                            ? ['activity_type' => $call->activity]
+                            : ['child_workflow_type' => $call->workflow],
                     );
 
                     if ($call instanceof ActivityCall) {

src/V2/Support/WorkflowExecutor.php

@@ -137,7 +137,13 @@ public function run(WorkflowRun $run, WorkflowTask $task): ?WorkflowTask
                     return $this->restartAfterPendingUpdateFailure($run, $task);
                 }
 
-                if (! $this->ensureStepHistoryCompatible($run, $task, $sequence, WorkflowStepHistory::LOCAL_ACTIVITY)) {
+                if (! $this->ensureStepHistoryCompatible(
+                    $run,
+                    $task,
+                    $sequence,
+                    WorkflowStepHistory::LOCAL_ACTIVITY,
+                    ['activity_type' => $current->activity],
+                )) {
                     return null;
                 }
 
@@ -211,7 +217,13 @@ public function run(WorkflowRun $run, WorkflowTask $task): ?WorkflowTask
                     return $this->restartAfterPendingUpdateFailure($run, $task);
                 }
 
-                if (! $this->ensureStepHistoryCompatible($run, $task, $sequence, WorkflowStepHistory::ACTIVITY)) {
+                if (! $this->ensureStepHistoryCompatible(
+                    $run,
+                    $task,
+                    $sequence,
+                    WorkflowStepHistory::ACTIVITY,
+                    ['activity_type' => $current->activity],
+                )) {
                     return null;
                 }
 
@@ -548,7 +560,13 @@ public function run(WorkflowRun $run, WorkflowTask $task): ?WorkflowTask
                 // legacy replays that have since produced ACTIVITY/TIMER
                 // events at the same sequence.
                 if ($resolution->advancesSequence
-                    && ! $this->ensureStepHistoryCompatible($run, $task, $sequence, WorkflowStepHistory::VERSION_MARKER)
+                    && ! $this->ensureStepHistoryCompatible(
+                        $run,
+                        $task,
+                        $sequence,
+                        WorkflowStepHistory::VERSION_MARKER,
+                        ['change_id' => $current->changeId],
+                    )
                 ) {
                     return null;
                 }
@@ -734,7 +752,13 @@ public function run(WorkflowRun $run, WorkflowTask $task): ?WorkflowTask
                     return $this->restartAfterPendingUpdateFailure($run, $task);
                 }
 
-                if (! $this->ensureStepHistoryCompatible($run, $task, $sequence, WorkflowStepHistory::SIGNAL_WAIT)) {
+                if (! $this->ensureStepHistoryCompatible(
+                    $run,
+                    $task,
+                    $sequence,
+                    WorkflowStepHistory::SIGNAL_WAIT,
+                    ['signal_name' => $current->name],
+                )) {
                     return null;
                 }
 
@@ -884,7 +908,13 @@ public function run(WorkflowRun $run, WorkflowTask $task): ?WorkflowTask
                     return $this->restartAfterPendingUpdateFailure($run, $task);
                 }
 
-                if (! $this->ensureStepHistoryCompatible($run, $task, $sequence, WorkflowStepHistory::CHILD_WORKFLOW)) {
+                if (! $this->ensureStepHistoryCompatible(
+                    $run,
+                    $task,
+                    $sequence,
+                    WorkflowStepHistory::CHILD_WORKFLOW,
+                    ['child_workflow_type' => $current->workflow],
+                )) {
                     return null;
                 }
 
@@ -1036,6 +1066,9 @@ public function run(WorkflowRun $run, WorkflowTask $task): ?WorkflowTask
                         $call instanceof ActivityCall
                             ? WorkflowStepHistory::ACTIVITY
                             : WorkflowStepHistory::CHILD_WORKFLOW,
+                        $call instanceof ActivityCall
+                            ? ['activity_type' => $call->activity]
+                            : ['child_workflow_type' => $call->workflow],
                     )) {
                         return null;
                     }
@@ -3107,9 +3140,10 @@ private function ensureStepHistoryCompatible(
         WorkflowTask $task,
         int $sequence,
         string $expectedShape,
+        array $expectedDetails = [],
     ): bool {
         try {
-            WorkflowStepHistory::assertCompatible($run, $sequence, $expectedShape);
+            WorkflowStepHistory::assertCompatible($run, $sequence, $expectedShape, $expectedDetails);
 
             return true;
         } catch (Throwable $throwable) {