Pin verification failure evidence producer (#170)

* Pin verification failure evidence producer

* Pin stable completion validator revision

* Pin stable native package revision

Author: chubes4

.github/workflows/README.md

@@ -16,7 +16,7 @@ The reusable workflow declares the expected typed review artifacts for Docs Agen
 
 The target repository grants `contents: write`, `pull-requests: write`, and `issues: write`. Docs Agent forwards the caller-scoped `${{ github.token }}` to WP Codebox for same-repository publication, so consumers do not configure `ACCESS_TOKEN`. `OPENAI_API_KEY` is an optional workflow secret and is required only for a live OpenAI run; skipped and dry-run calls do not require it. `EXTERNAL_PACKAGE_SOURCE_POLICY` remains a separate required v1 JSON secret. Migrate its value to the exact one-line JSON in the root README: it authorizes the selected Docs Agent package, the pinned Agents API component, the pinned PHP AI Client overlay, and the checksum-pinned OpenAI provider artifact. Both secrets are forwarded to WP Codebox without serialization into the task descriptor.
 
-Docs Agent consumes the reusable WP Codebox workflow at accepted immutable revision `12a5bb19a97b89d0a78b502fc71adede5b122359` and passes `wp_codebox_release_ref: v0.12.29` only for released packaged runtime assets. The release tag resolves to immutable revision `bc982947ec33c78160125026e16d357b7ece3ea1`, distinct from both the workflow producer revision and the fixed Docs Agent package revision. WP Codebox materializes the declared native runtime closure, executes the selected package, applies bounded workspace changes, runs verification, and publishes approved host changes. The released assets are `01-wp-codebox.zip` and `02-wp-codebox-workspace-0.12.29.tgz`; the private package-recovery manifest is intentionally not a reviewer asset.
+Docs Agent consumes the reusable WP Codebox workflow at accepted immutable revision `0227ca7551d8cd98f14242b31e142b50f78dff13` and passes `wp_codebox_release_ref: v0.12.29` only for released packaged runtime assets. The release tag resolves to immutable revision `bc982947ec33c78160125026e16d357b7ece3ea1`, distinct from both the workflow producer revision and the fixed Docs Agent package revision. WP Codebox materializes the declared native runtime closure, executes the selected package, applies bounded workspace changes, runs verification, and publishes approved host changes. The released assets are `01-wp-codebox.zip` and `02-wp-codebox-workspace-0.12.29.tgz`; the private package-recovery manifest is intentionally not a reviewer asset.
 
 Provider artifacts retain only allowlisted review artifacts, controlled workflow envelopes, and canonical runtime-source and seed provenance. WP Codebox v0.12.24 uploads a reviewer-safe workflow-result projection: public control, verification, publication, output-projection, access, failure, artifact-declaration, and canonical transcript-provenance fields remain, while raw `runtime_result`, `outputs.engine_data`, model/provider/tool payloads, source content, private paths, and secrets are excluded. Before persistence or upload, it sanitizes private runtime paths from nested values, object keys, diagnostics, and command arguments. WP Codebox captures one trusted canonical `codebox-transcript` through a pre-sanitization reviewer-evidence descriptor, validates its artifact-root containment, schema, digest, and size, then emits the compact reviewer transcript as a `wp-codebox/reviewer-agent-transcript/v1` projection. Before apply-back, it compares the seed identity with the host workspace identity; rejected patches preserve the expected and actual identities plus patch and changed-file evidence for upload. Its pre-redaction trusted apply input preserves machine-applicable patch bytes in a private channel that is removed before durable artifact sanitization, as fixed in [WP Codebox #1842](https://github.com/Automattic/wp-codebox/pull/1842). The canonical transcript remains the bounded tool-observability surface. Lifecycle and downstream failures retain runtime evidence in their internal normalized failed result even when artifact preparation cannot complete. Diagnostic messages that name runtime classes remain reviewable, while PHP-shaped runtime source remains blocked. See [WP Codebox #1767](https://github.com/Automattic/wp-codebox/issues/1767), [WP Codebox #1814](https://github.com/Automattic/wp-codebox/issues/1814), [WP Codebox #1815](https://github.com/Automattic/wp-codebox/pull/1815), and [WP Codebox #1823](https://github.com/Automattic/wp-codebox/pull/1823). Successful canonical reviewer-evidence reference: [run `29350690551`](https://github.com/Automattic/build-with-wordpress/actions/runs/29350690551). The [hosted failure `29468956857`](https://github.com/Automattic/agents-api/actions/runs/29468956857) executed WP Codebox v0.12.20, published [Agents API #430](https://github.com/Automattic/agents-api/pull/430), then failed during upload preparation; it motivated #1814/#1815 and is not a successful v0.12.24 reviewer-safe workflow-result run.
 

.github/workflows/maintain-docs.yml

@@ -2,9 +2,9 @@ name: Maintain Docs
 
 env:
   # Advance this revision and all package digests together when package bytes change.
-  DOCS_AGENT_PACKAGE_REVISION: 85f0d162a7d499fdc1286891371342727d084c88
+  DOCS_AGENT_PACKAGE_REVISION: a39d9db230eb9e0b72ed84465f4d61bd8dda1bab
   # Advance independently when the post-command completion validator changes.
-  DOCS_AGENT_COMPLETION_CONTRACT_REVISION: cc6ec369d1e47ffdf541e3ec2c72bce5eed5f685
+  DOCS_AGENT_COMPLETION_CONTRACT_REVISION: a39d9db230eb9e0b72ed84465f4d61bd8dda1bab
 
 'on':
   workflow_call:
@@ -344,7 +344,7 @@ jobs:
       contents: write
       pull-requests: write
       issues: write
-    uses: Automattic/wp-codebox/.github/workflows/run-agent-task.yml@12a5bb19a97b89d0a78b502fc71adede5b122359
+    uses: Automattic/wp-codebox/.github/workflows/run-agent-task.yml@0227ca7551d8cd98f14242b31e142b50f78dff13
     with:
       wp_codebox_release_ref: v0.12.29
       external_package_source: ${{ needs.prepare.outputs.external_package_source }}

.github/workflows/validate.yml

@@ -17,7 +17,7 @@ jobs:
         uses: actions/checkout@v4
         with:
           repository: Automattic/wp-codebox
-          ref: 12a5bb19a97b89d0a78b502fc71adede5b122359
+          ref: 0227ca7551d8cd98f14242b31e142b50f78dff13
           path: .wp-codebox
       - name: Validate Docs Agent
         env:

README.md

@@ -134,7 +134,7 @@ Docs Agent declares the review artifacts it expects the runner to materialize as
 
 `maintain-docs.yml` writes `expected_artifacts` and `artifact_declarations` into a portable Docs Agent recipe and exposes the same declaration objects as `declared_artifacts_json`. The runtime declaration for `docs_agent_completion_report` remains `required: false` because WP Codebox may evaluate runtime-level required artifacts before post-command validation. The final completion drift entry is mandatory and declares `{name,type,path}`; after successful semantic validation, the validator atomically writes canonical JSON to `.codebox/agent-task-artifacts/docs-agent-completion-report.json`, and WP Codebox stages that declared command artifact for reviewers.
 
-The reusable workflow producer is pinned to accepted revision `12a5bb19a97b89d0a78b502fc71adede5b122359`; `v0.12.29` remains only the `wp_codebox_release_ref` for released packaged runtime assets at `bc982947ec33c78160125026e16d357b7ece3ea1`.
+The reusable workflow producer is pinned to accepted revision `0227ca7551d8cd98f14242b31e142b50f78dff13`; `v0.12.29` remains only the `wp_codebox_release_ref` for released packaged runtime assets at `bc982947ec33c78160125026e16d357b7ece3ea1`.
 
 WP Codebox v0.12.29 at `bc982947ec33c78160125026e16d357b7ece3ea1` uploads a reviewer-safe workflow-result projection with public control and publication fields plus canonical transcript provenance. This released workflow revision remains distinct from the fixed Docs Agent package revision. It excludes raw `runtime_result`, `outputs.engine_data`, model/provider/tool payloads, source content, private paths, secrets, Git-ignored verification artifacts such as pnpm's symlinked `node_modules` tree, and mutable `.codebox` runtime control files. Tracked or otherwise publishable symlinks remain rejected. Integrity failures retain bounded added, modified, and deleted path evidence, and non-Git workspaces retain a bounded filesystem snapshot fallback. Canonical GitHub repository identity is compared case-insensitively while pull-request URL syntax, pull number, target binding, and API resolution remain strict. Successful publication verification returns `{ valid: true }` without a failure-only `error`, while repository mismatches retain their exact diagnostic as fixed by [WP Codebox #1885](https://github.com/Automattic/wp-codebox/pull/1885). The canonical `codebox-transcript` remains the bounded tool-observability surface: its pre-sanitization reviewer-evidence descriptor records the trusted artifact-relative path, schema, verified source digest, and size, which the uploader revalidates before producing the `wp-codebox/reviewer-agent-transcript/v1` projection. Before apply-back, it validates that the runner seed and host workspace identities match; rejected patches retain identity, patch, and changed-file evidence for review. The pre-redaction trusted apply input fixed by [WP Codebox #1842](https://github.com/Automattic/wp-codebox/pull/1842) retains machine-applicable patch bytes privately, then removes them before durable artifact sanitization. The publication snapshot fixes are tracked in [WP Codebox #1845](https://github.com/Automattic/wp-codebox/pull/1845), [WP Codebox #1848](https://github.com/Automattic/wp-codebox/pull/1848), [WP Codebox #1852](https://github.com/Automattic/wp-codebox/pull/1852), [WP Codebox #1875](https://github.com/Automattic/wp-codebox/pull/1875), and [WP Codebox #1885](https://github.com/Automattic/wp-codebox/pull/1885). The published release assets are `01-wp-codebox.zip` and `02-wp-codebox-workspace-0.12.29.tgz`; its private package-recovery manifest is intentionally not a reviewer asset.
 
@@ -230,7 +230,7 @@ Docs Agent has one canonical architecture: five standalone native Agents API pac
 - `bundles/user-docs-agent/native/user-docs-maintenance-agent.agent.json`
 - `bundles/skills-agent/native/skills-maintenance-agent.agent.json`
 
-These five `.agent.json` files are the complete executable package surface. Each package is the sole executable instruction authority for its lane. The reusable workflow maps `audience` and `run_kind` to exactly one package and its canonical agent slug; it does not select a separate manifest, flow, pipeline, or memory envelope. Every descriptor uses the package-source revision `85f0d162a7d499fdc1286891371342727d084c88`, independently of the revision that invokes the reusable workflow, and supplies a byte-level `sha256-bytes-v1` digest.
+These five `.agent.json` files are the complete executable package surface. Each package is the sole executable instruction authority for its lane. The reusable workflow maps `audience` and `run_kind` to exactly one package and its canonical agent slug; it does not select a separate manifest, flow, pipeline, or memory envelope. Every descriptor uses the package-source revision `a39d9db230eb9e0b72ed84465f4d61bd8dda1bab`, independently of the revision that invokes the reusable workflow, and supplies a byte-level `sha256-bytes-v1` digest.
 
 Package updates advance the package-source revision and all five declared digests atomically. The immutable-source validator reads each package blob from that Git revision, recomputes its digest and canonical slug, and rejects a descriptor that does not match those historical bytes.
 

ci/docs-agent-runner-recipe.json

@@ -5,7 +5,7 @@
   "docsAgent": {
     "externalPackageSource": {
       "repository": "Automattic/docs-agent",
-      "revision": "85f0d162a7d499fdc1286891371342727d084c88",
+      "revision": "a39d9db230eb9e0b72ed84465f4d61bd8dda1bab",
       "path": "bundles/technical-docs-agent/native/technical-docs-maintenance-agent.agent.json",
       "digest": "sha256-bytes-v1:975c7b0a0a7aff52897c52be5ac903a7fb110ea3c33e16227f8694c74c932519"
     },

examples/runner-recipe.example.json

@@ -5,7 +5,7 @@
   "docsAgent": {
     "externalPackageSource": {
       "repository": "Automattic/docs-agent",
-      "revision": "85f0d162a7d499fdc1286891371342727d084c88",
+      "revision": "a39d9db230eb9e0b72ed84465f4d61bd8dda1bab",
       "path": "bundles/technical-docs-agent/native/technical-docs-maintenance-agent.agent.json",
       "digest": "sha256-bytes-v1:975c7b0a0a7aff52897c52be5ac903a7fb110ea3c33e16227f8694c74c932519"
     },

tests/reusable-workflow-empty-called-workflow-context.fixture.json

@@ -3,5 +3,5 @@
   "called_workflow_context": {
     "job_workflow_sha": ""
   },
-  "expected_package_revision": "85f0d162a7d499fdc1286891371342727d084c88"
+  "expected_package_revision": "a39d9db230eb9e0b72ed84465f4d61bd8dda1bab"
 }

tests/validate-docs-agent-packages.php

@@ -45,7 +45,7 @@
 $assert( 'docs-agent/runner-recipe/v1' === ( $recipe['schema'] ?? null ), 'Runner recipe must use the portable Docs Agent runner recipe schema.' );
 $expected_package_source = array(
 	'repository' => 'Automattic/docs-agent',
-	'revision'   => '85f0d162a7d499fdc1286891371342727d084c88',
+	'revision'   => 'a39d9db230eb9e0b72ed84465f4d61bd8dda1bab',
 	'path'       => 'bundles/technical-docs-agent/native/technical-docs-maintenance-agent.agent.json',
 	'digest'     => 'sha256-bytes-v1:975c7b0a0a7aff52897c52be5ac903a7fb110ea3c33e16227f8694c74c932519',
 );
@@ -71,7 +71,7 @@
 $assert( str_contains( $maintain_docs_workflow, 'artifact_declarations<<EOF' ), 'maintain-docs.yml must prepare typed artifact declarations without caller-specific projections.' );
 $assert( str_contains( $maintain_docs_workflow, 'artifact_declarations<<EOF' ), 'maintain-docs.yml must expose artifact declarations through workflow outputs.' );
 
-$wp_codebox_producer_revision        = '12a5bb19a97b89d0a78b502fc71adede5b122359';
+$wp_codebox_producer_revision        = '0227ca7551d8cd98f14242b31e142b50f78dff13';
 $generic_codebox_agent_task_workflow = 'uses: Automattic/wp-codebox/.github/workflows/run-agent-task.yml@' . $wp_codebox_producer_revision;
 $assert( str_contains( $maintain_docs_workflow, $generic_codebox_agent_task_workflow ), 'maintain-docs.yml must call the immutable generic Codebox agent-task workflow candidate.' );
 $assert( str_contains( $maintain_docs_workflow, 'wp_codebox_release_ref: v0.12.29' ), 'maintain-docs.yml must pass the matching WP Codebox release tag.' );
@@ -82,7 +82,7 @@
 $assert( str_contains( $maintain_docs_workflow, 'sourceDelta:$sourceDelta' ), 'maintain-docs.yml must retain caller-known bounded source deltas in the portable recipe.' );
 $assert( str_contains( $maintain_docs_workflow, 'source_delta must contain at least one caller-bounded item for maintenance.' ), 'maintain-docs.yml must reject empty maintenance source deltas before execution.' );
 $assert( str_contains( $maintain_docs_workflow, 'validate-docs-agent-completion.php' ), 'maintain-docs.yml must execute the Docs Agent-owned completion validator.' );
-$completion_contract_revision = 'cc6ec369d1e47ffdf541e3ec2c72bce5eed5f685';
+$completion_contract_revision = 'a39d9db230eb9e0b72ed84465f4d61bd8dda1bab';
 $assert( str_contains( $maintain_docs_workflow, 'DOCS_AGENT_COMPLETION_CONTRACT_REVISION: ' . $completion_contract_revision ), 'maintain-docs.yml must pin the completion validator to its immutable implementation commit.' );
 $assert( str_contains( $maintain_docs_workflow, 'Automattic/docs-agent/$DOCS_AGENT_COMPLETION_CONTRACT_REVISION/scripts/validate-docs-agent-completion.php' ), 'maintain-docs.yml must fetch the validator independently of native package provenance.' );
 $historical_validator = shell_exec( 'git -C ' . escapeshellarg( $root ) . ' show ' . escapeshellarg( $completion_contract_revision . ':scripts/validate-docs-agent-completion.php' ) );

tests/validate-external-native-package-sources.php

@@ -30,7 +30,7 @@
 	return $contents;
 };
 
-$package_revision = '85f0d162a7d499fdc1286891371342727d084c88';
+$package_revision = 'a39d9db230eb9e0b72ed84465f4d61bd8dda1bab';
 $packages = array(
 	'technical:bootstrap' => array( 'path' => 'bundles/technical-docs-agent/native/technical-docs-bootstrap-agent.agent.json', 'slug' => 'technical-docs-bootstrap-agent', 'digest' => '0230e0e3fd8a4f045a545407d3d01c22df537b05f031260e29d6b531285b7839' ),
 	'technical:maintenance' => array( 'path' => 'bundles/technical-docs-agent/native/technical-docs-maintenance-agent.agent.json', 'slug' => 'technical-docs-maintenance-agent', 'digest' => '975c7b0a0a7aff52897c52be5ac903a7fb110ea3c33e16227f8694c74c932519' ),

tests/validate-wp-codebox-run-agent-task-contract.php

@@ -230,7 +230,7 @@
 
 $assert( isset( $caller_inputs['external_package_source'] ), 'Docs Agent must provide the required external package descriptor.' );
 $assert( ! isset( $caller_inputs['agent_bundle'] ), 'Docs Agent must not pass the removed agent_bundle input.' );
-$assert( str_contains( $workflow, 'DOCS_AGENT_PACKAGE_REVISION: 85f0d162a7d499fdc1286891371342727d084c88' ), 'Docs Agent must use the fixed native package source revision.' );
+$assert( str_contains( $workflow, 'DOCS_AGENT_PACKAGE_REVISION: a39d9db230eb9e0b72ed84465f4d61bd8dda1bab' ), 'Docs Agent must use the fixed native package source revision.' );
 $assert( ! str_contains( $workflow, 'github.job_workflow_sha' ), 'Docs Agent must not depend on unavailable called-workflow provenance.' );
 
 preg_match( "/runtime_sources='(?<json>[^']+)'/", $workflow, $runtime_sources_match );