Merge pull request #1954 from Automattic/feat/cloudflare-incremental-publication

Publish affected Cloudflare routes automatically

Author: chubes4

packages/runtime-cloudflare/README.md

@@ -10,27 +10,27 @@ On cold start, the entry Worker uses the acquired pointer to rebuild PHP-WASM's
 
 Canonical browser, health, and mutation boots require the separately managed `WORDPRESS_AUTH_SECRET` Worker secret. The entry Worker derives the eight WordPress auth keys and salts from that secret with a versioned, site-scoped (`default`) SHA-256 domain separator before `wp-load.php`. It never logs, persists, or returns the secret or derived values. Configure it independently from the bootstrap password with `wrangler secret put WORDPRESS_AUTH_SECRET --config packages/runtime-cloudflare/wrangler.jsonc`; rotating `WORDPRESS_ADMIN_PASSWORD` does not rotate authentication salts or invalidate sessions.
 
-After each mutating HTTP request, the entry runtime invokes MDI's explicit request-boundary flush, collects canonical files, stores immutable content-addressed R2 objects and a revision manifest, then commits the new pointer through the held lease. GET, HEAD, and asset requests release without promotion. Failed requests abort their leases; stale leases recover by token/version/expiry checks. The entry isolate can cache one runtime only for the exact acquired pointer revision and exits it after promotion or when another isolate advances the pointer. It does not persist SQLite. Existing manifests are reused when canonical file hashes have not changed.
+After each mutating HTTP request, the entry runtime invokes MDI's explicit request-boundary flush, collects canonical files, stores immutable content-addressed R2 objects and a revision manifest, then commits the new pointer through the held lease. GET, HEAD, and asset requests release without promotion. Failed requests abort their leases; stale leases recover by token/version/expiry checks. The entry isolate can cache one runtime only for the exact acquired pointer revision and exits it after promotion or when another isolate advances the pointer. It does not persist SQLite. Existing manifests are reused when canonical file hashes have not changed. Authenticated requests that produce no canonical MDI changes do not rebuild a publication.
 
-Anonymous HTML responses are stored as create-once, host-independent R2 artifacts under their canonical revision. An authorized `POST ?phase=operator-publish` promotes a bounded, sorted set of already-rendered routes by writing an immutable publication descriptor and then replacing one strongly consistent `sites/default/publications/current.json` pointer. Published GET and HEAD requests check a 60-second edge entry and then the R2 publication before constructing the Durable Object stub; authenticated, preview, admin, REST, and unpublished routes continue through WordPress. Promotion rejects missing, stale, duplicate, malformed, or conflicting artifacts. This is the coordinator-free reader boundary, not yet a complete route compiler or global CDN purge pipeline.
+Anonymous HTML responses are stored as create-once, host-independent R2 artifacts under their canonical revision. An authorized `POST ?phase=operator-publish` establishes or replaces a bounded route set. Once that publication exists, WordPress hooks report the public routes affected by post, term, menu, theme, plugin, and relevant option mutations. The Worker renders only those routes in the already-mutated PHP runtime, stages immutable page artifacts and a version 2 publication descriptor, commits canonical state through the coordinator, then conditionally promotes `sites/default/publications/current.json` against its prior R2 ETag. Unchanged route entries retain their older canonical artifacts. A promotion conflict or post-commit R2 failure leaves the prior publication readable and is exposed through `x-wp-codebox-publication`; it never rolls back already-committed WordPress state. Published GET and HEAD requests check a 60-second edge entry and then R2 before coordinator construction, while authenticated, preview, admin, REST, and unpublished routes continue through WordPress. The version 2 reader accepts existing version 1 descriptors during migration. This provides bounded automatic route compilation and local edge invalidation, not a global CDN purge pipeline.
 
 The same revision transaction persists bounded user-managed files under `wp-content/plugins`, `themes`, `languages`, and `mu-plugins`. Runtime-owned MDI, SQLite integration, and Codebox adapter files remain reconstructable artifacts and are excluded. Unchanged bundled-theme files are omitted by release hash, while modified files become canonical overrides. Public canonical plugin and theme assets serve directly from revision-addressed R2 cache entries before the immutable release corpus. Nonce-protected `wp-admin` GET actions such as plugin activation are classified as mutations so their filesystem and MDI option changes commit atomically. Existing revisions without `wpContent` remain valid.
 
 The Worker forwards browser cookies directly to Playground and disables Playground's internal cookie store, preventing an empty per-isolate store from replacing a valid browser session after cold restart.
 
-A minute Cloudflare Cron Trigger drains at most five due WordPress events or 25 seconds of work per invocation. Each event receives its own Durable Object lease and canonical R2 revision transaction. Direct `/wp-cron.php` requests are disabled so callbacks cannot bypass that transaction boundary.
+A minute Cloudflare Cron Trigger drains at most five due WordPress events or 25 seconds of work per invocation. Each event receives its own coordinator lease and canonical R2 revision transaction. Scheduled publication callbacks use the same affected-route compiler and post-commit conditional promotion as browser mutations. Direct `/wp-cron.php` requests are disabled so callbacks cannot bypass that transaction boundary.
 
 The bundled MDI source is pinned to immutable commit `bf6d434d1673fdd86d777501f7eaec292d32ad1f`, including [MDI PR #141](https://github.com/Automattic/markdown-database-integration/pull/141). The bundle generator, worker provenance, and source-contract test use the same revision.
 
 WordPress server files, browser assets, and pinned runtime dependencies are separate deployment artifacts, never Worker-module imports. `generate:cloudflare-wordpress-runtime-corpus` emits a deterministic server ZIP containing only `isWordPressRuntimeFile()` entries, one concatenated browser-asset blob containing the complete supported `wp-admin`, `wp-includes`, and bundled-theme surface, and the pinned SQLite integration ZIP. Checked-in manifests record content-addressed R2 keys, hashes, budgets, provenance, and each browser asset's exact byte range. Cold boot validates and streams the server ZIP into PHP MEMFS while loading SQLite integration from R2. Browser requests use one bounded R2 range read, validate the selected bytes, and populate an immutable artifact-versioned Worker cache. Runtime requests never fetch WordPress or SQLite integration release archives from third-party origins.
 
-Canonical Cloudflare boots patch only the assembled PHP MEMFS copy of `/wordpress/wp-settings.php` after the R2 WordPress corpus is materialized and before WordPress executes. The disabled-cron scheduling policy requires exactly one canonical `do_action( 'init' );` needle and fails closed otherwise. When `DISABLE_WP_CRON` is true, it removes only core's `wp_cron`, `wp_schedule_delete_old_privacy_export_files`, and `wp_schedule_update_checks` callbacks through public `remove_action()` before executing the original `init` call once. This prevents browser requests from recreating cron work that cannot execute in the PHP-WASM/MDI shutdown path. Durable cron execution belongs to a separate explicit queue or scheduled-Worker contract.
+Canonical Cloudflare boots patch only the assembled PHP MEMFS copy of `/wordpress/wp-settings.php` after the R2 WordPress corpus is materialized and before WordPress executes. The disabled-cron scheduling policy requires exactly one canonical `do_action( 'init' );` needle and fails closed otherwise. When `DISABLE_WP_CRON` is true, it removes only core's `wp_cron`, `wp_schedule_delete_old_privacy_export_files`, and `wp_schedule_update_checks` callbacks through public `remove_action()` before executing the original `init` call once. This prevents browser requests from recreating cron work outside the explicit scheduled-Worker transaction.
 ## Verification
 
 1. Run `npm run generate:cloudflare-canonical-mdi-seed` and `npm run generate:cloudflare-wordpress-runtime-corpus` to regenerate deterministic runtime artifacts and manifests.
 2. Run `npm run provision:cloudflare-wordpress-runtime-corpus -- --local --persist-to <directory>` to verify and upload all exact content-addressed artifacts into isolated local R2 storage. For an authorized deployment, run the provisioner with `--remote` and require every upload to succeed before deploying the Worker that imports their manifests.
 3. Run `npm run test:cloudflare-runtime` for routing, coordinator composition, canonical-state, artifact validation, source contract, and TypeScript coverage.
 4. Run `npm run cloudflare:dry-run` and `npm run cloudflare:dry-run:d1` to compile the Durable Object and D1 profiles without creating Cloudflare resources. The placeholder D1 database ID is for local/dry-run verification; an implementation supplies its provisioned binding at deployment.
-5. Run `npm run cloudflare:local-gate` and `npm run cloudflare:local-gate:d1` for the same isolated workerd workflow through both coordinator implementations. Each gate generates and provisions all artifacts, verifies the selected backend through the state envelope, injects stable test-only admin-password, auth-secret, and operator-token values, uploads and activates a real plugin ZIP, promotes anonymous homepage and canonical-permalink artifacts, and proves an R2 publication read after Worker restart. They also cover login, concurrent canonical writes, authenticated REST publication, media, representative frontend/admin/editor assets, PHP diagnostics, session recovery, cron, and a fresh login after restart.
+5. Run `npm run cloudflare:local-gate` and `npm run cloudflare:local-gate:d1` for the same isolated workerd workflow through both coordinator implementations. Each gate generates and provisions all artifacts, verifies the selected backend through the state envelope, injects stable test-only admin-password, auth-secret, and operator-token values, uploads and activates a real plugin ZIP, establishes an anonymous homepage and canonical-permalink publication, updates a published post through authenticated REST, and proves that the affected permalink and homepage promote automatically. The cron gate also proves that a scheduled post becomes readable from the publication path without an operator call. Restart verification requires the immutable R2 publication, while the remaining coverage includes login, concurrent canonical writes, media, representative frontend/admin/editor assets, PHP diagnostics, session recovery, and a fresh login after restart.
 
 This document describes local candidate verification only. It does not claim remote deployment.

packages/runtime-cloudflare/src/published-reader.ts

@@ -1,4 +1,5 @@
-export const PUBLISHED_REVISION_SCHEMA = "wp-codebox/published-revision/v1" as const
+export const PUBLISHED_REVISION_SCHEMA = "wp-codebox/published-revision/v2" as const
+const LEGACY_PUBLISHED_REVISION_SCHEMA = "wp-codebox/published-revision/v1"
 export const PUBLISHED_PAGE_SCHEMA = "wp-codebox/wordpress-page/v2" as const
 export const R2_PUBLISHED_CURRENT_KEY = "sites/default/publications/current.json"
 export const R2_PUBLISHED_REVISION_PREFIX = "sites/default/publications/revisions"
@@ -9,6 +10,7 @@ export const MAX_PUBLISHED_PAGE_BYTES = 8 * 1024 * 1024
 export interface PublishedRoute {
   route: string
   objectKey: string
+  canonicalRevision: string
 }
 
 export interface PublishedRevision {
@@ -40,17 +42,22 @@ export function publishedRevisionObjectKey(revision: string): string {
 export function validatePublishedRevision(value: unknown): PublishedRevision {
   if (!value || typeof value !== "object") throw new Error("Published revision is invalid.")
   const revision = value as Partial<PublishedRevision>
-  if (revision.schema !== PUBLISHED_REVISION_SCHEMA || !isRevision(revision.revision) || !isRevision(revision.canonicalRevision)
+  const legacy = (value as { schema?: unknown }).schema === LEGACY_PUBLISHED_REVISION_SCHEMA
+  if (![PUBLISHED_REVISION_SCHEMA, LEGACY_PUBLISHED_REVISION_SCHEMA].includes(revision.schema as string) || !isRevision(revision.revision) || !isRevision(revision.canonicalRevision)
     || typeof revision.publishedAt !== "string" || !Number.isFinite(Date.parse(revision.publishedAt)) || !Array.isArray(revision.routes)
     || revision.routes.length === 0 || revision.routes.length > MAX_PUBLISHED_ROUTES) throw new Error("Published revision is invalid.")
   let previous = ""
+  const normalizedRoutes: PublishedRoute[] = []
   for (const route of revision.routes) {
+    const routeRevision = legacy ? revision.canonicalRevision : route.canonicalRevision
     if (!route || typeof route !== "object" || !isCanonicalRoute(route.route) || route.route <= previous
-      || route.objectKey !== `sites/default/pages/${revision.canonicalRevision}/${route.objectKey.split("/").at(-1)}`
+      || !isRevision(routeRevision)
+      || route.objectKey !== `sites/default/pages/${routeRevision}/${route.objectKey.split("/").at(-1)}`
       || !/^sites\/default\/pages\/[a-f0-9-]{36}\/[a-f0-9]{64}\.json$/.test(route.objectKey)) throw new Error("Published revision route is invalid.")
     previous = route.route
+    normalizedRoutes.push({ route: route.route, objectKey: route.objectKey, canonicalRevision: routeRevision })
   }
-  return revision as PublishedRevision
+  return { ...revision, schema: PUBLISHED_REVISION_SCHEMA, routes: normalizedRoutes } as PublishedRevision
 }
 
 export function normalizePublishedRoutes(value: unknown): string[] {