Skip to content

Commit 483353a

Browse files
authored
Merge pull request #1952 from Automattic/feat/cloudflare-published-reader
Make Cloudflare publication and coordination composable
2 parents 52ecc5f + 62c95ec commit 483353a

14 files changed

Lines changed: 697 additions & 145 deletions

package.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,9 @@
8585
"scripts": {
8686
"build": "node ./node_modules/typescript/bin/tsc -b packages/runtime-core packages/runtime-playground packages/cli && node scripts/ensure-cli-bin-executable.mjs",
8787
"cloudflare:dry-run": "npm exec -- wrangler deploy --dry-run --config packages/runtime-cloudflare/wrangler.jsonc",
88+
"cloudflare:dry-run:d1": "npm exec -- wrangler deploy --dry-run --config packages/runtime-cloudflare/wrangler.d1.jsonc",
8889
"cloudflare:local-gate": "node scripts/cloudflare-local-gate.mjs",
90+
"cloudflare:local-gate:d1": "node scripts/cloudflare-local-gate.mjs --coordinator=d1",
8991
"postinstall": "patch-package",
9092
"generate:cloudflare-wordpress-runtime-corpus": "tsx scripts/generate-cloudflare-wordpress-runtime-corpus.ts",
9193
"provision:cloudflare-wordpress-runtime-corpus": "node scripts/provision-cloudflare-wordpress-runtime-corpus.mjs",

packages/runtime-cloudflare/README.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,16 @@ This candidate integration for [wp-codebox#1838](https://github.com/Automattic/w
44

55
## Runtime Architecture
66

7-
The entry Worker executes PHP-WASM and WordPress. The named `WordPressStateCoordinator` Durable Object remains lightweight: it serializes a bounded lease and atomically promotes the current canonical R2 pointer with token, base-revision, and version checks. It never imports or instantiates PHP-WASM. State reads query the coordinator directly; every request that can observe canonical WordPress state acquires a lease first.
7+
The entry Worker executes PHP-WASM and WordPress. Runtime behavior depends on the typed `RevisionCoordinator` contract rather than a Cloudflare storage product. The standard `worker-do.ts` entrypoint injects the existing `WordPressStateCoordinator` Durable Object adapter. The ChatGPT Sites-compatible `worker-d1.ts` entrypoint injects a D1 adapter that stores only the current pointer, version, lease token, lease base, and expiry in one conditional-update row. Both adapters serialize the same bounded lease and CAS promotion semantics; neither imports or instantiates PHP-WASM. MDI, disposable SQLite, and canonical R2 storage are unchanged.
88

99
On cold start, the entry Worker uses the acquired pointer to rebuild PHP-WASM's disposable SQLite index from canonical MDI Markdown and JSON files. A missing pointer materializes the packaged canonical MDI seed and boots one PHP-WASM primary runtime. The build-time PHP CLI generator creates that archive from `wordpress-install-seed.sqlite` through MDI's public `bootstrap_existing_cache()` API, validates its pinned MDI revision and input digest, and never packages SQLite. The runtime updates `siteurl` and `home` through WordPress APIs using the request origin and sets the admin password from `WORDPRESS_ADMIN_PASSWORD`; only WordPress's password hash is canonical. Bootstrap persists and CAS-promotes this mutation before serving the next request.
1010

1111
Canonical browser, health, and mutation boots require the separately managed `WORDPRESS_AUTH_SECRET` Worker secret. The entry Worker derives the eight WordPress auth keys and salts from that secret with a versioned, site-scoped (`default`) SHA-256 domain separator before `wp-load.php`. It never logs, persists, or returns the secret or derived values. Configure it independently from the bootstrap password with `wrangler secret put WORDPRESS_AUTH_SECRET --config packages/runtime-cloudflare/wrangler.jsonc`; rotating `WORDPRESS_ADMIN_PASSWORD` does not rotate authentication salts or invalidate sessions.
1212

1313
After each mutating HTTP request, the entry runtime invokes MDI's explicit request-boundary flush, collects canonical files, stores immutable content-addressed R2 objects and a revision manifest, then commits the new pointer through the held lease. GET, HEAD, and asset requests release without promotion. Failed requests abort their leases; stale leases recover by token/version/expiry checks. The entry isolate can cache one runtime only for the exact acquired pointer revision and exits it after promotion or when another isolate advances the pointer. It does not persist SQLite. Existing manifests are reused when canonical file hashes have not changed.
1414

15+
Anonymous HTML responses are stored as create-once, host-independent R2 artifacts under their canonical revision. An authorized `POST ?phase=operator-publish` promotes a bounded, sorted set of already-rendered routes by writing an immutable publication descriptor and then replacing one strongly consistent `sites/default/publications/current.json` pointer. Published GET and HEAD requests check a 60-second edge entry and then the R2 publication before constructing the Durable Object stub; authenticated, preview, admin, REST, and unpublished routes continue through WordPress. Promotion rejects missing, stale, duplicate, malformed, or conflicting artifacts. This is the coordinator-free reader boundary, not yet a complete route compiler or global CDN purge pipeline.
16+
1517
The same revision transaction persists bounded user-managed files under `wp-content/plugins`, `themes`, `languages`, and `mu-plugins`. Runtime-owned MDI, SQLite integration, and Codebox adapter files remain reconstructable artifacts and are excluded. Unchanged bundled-theme files are omitted by release hash, while modified files become canonical overrides. Public canonical plugin and theme assets serve directly from revision-addressed R2 cache entries before the immutable release corpus. Nonce-protected `wp-admin` GET actions such as plugin activation are classified as mutations so their filesystem and MDI option changes commit atomically. Existing revisions without `wpContent` remain valid.
1618

1719
The Worker forwards browser cookies directly to Playground and disables Playground's internal cookie store, preventing an empty per-isolate store from replacing a valid browser session after cold restart.
@@ -27,8 +29,8 @@ Canonical Cloudflare boots patch only the assembled PHP MEMFS copy of `/wordpres
2729

2830
1. Run `npm run generate:cloudflare-canonical-mdi-seed` and `npm run generate:cloudflare-wordpress-runtime-corpus` to regenerate deterministic runtime artifacts and manifests.
2931
2. Run `npm run provision:cloudflare-wordpress-runtime-corpus -- --local --persist-to <directory>` to verify and upload all exact content-addressed artifacts into isolated local R2 storage. For an authorized deployment, run the provisioner with `--remote` and require every upload to succeed before deploying the Worker that imports their manifests.
30-
3. Run `npm run test:cloudflare-runtime` for routing, canonical-state, artifact validation, source contract, and TypeScript coverage.
31-
4. Run `npm run cloudflare:dry-run` to compile the Worker without creating Cloudflare resources.
32-
5. Run `npm run cloudflare:local-gate` for isolated local workerd evidence. It generates and provisions all artifacts before workerd starts, then injects stable test-only admin-password and auth-secret values, uploads and activates a real plugin ZIP, verifies that plugin's REST route before and after Worker restart, and covers login, authenticated REST publication, media, public rendering, representative frontend/admin/editor assets, PHP diagnostics, session recovery, and a fresh login after restart.
32+
3. Run `npm run test:cloudflare-runtime` for routing, coordinator composition, canonical-state, artifact validation, source contract, and TypeScript coverage.
33+
4. Run `npm run cloudflare:dry-run` and `npm run cloudflare:dry-run:d1` to compile the Durable Object and D1 profiles without creating Cloudflare resources. The placeholder D1 database ID is for local/dry-run verification; an implementation supplies its provisioned binding at deployment.
34+
5. Run `npm run cloudflare:local-gate` and `npm run cloudflare:local-gate:d1` for the same isolated workerd workflow through both coordinator implementations. Each gate generates and provisions all artifacts, verifies the selected backend through the state envelope, injects stable test-only admin-password, auth-secret, and operator-token values, uploads and activates a real plugin ZIP, promotes anonymous homepage and canonical-permalink artifacts, and proves an R2 publication read after Worker restart. They also cover login, concurrent canonical writes, authenticated REST publication, media, representative frontend/admin/editor assets, PHP diagnostics, session recovery, cron, and a fresh login after restart.
3335

3436
This document describes local candidate verification only. It does not claim remote deployment.
Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
import { RevisionConflict, type MarkdownPointer, type RevisionCoordinator, type RevisionLease, type RevisionState } from "./revision-coordinator.js"
2+
3+
interface StateRow {
4+
revision: string | null
5+
manifest_key: string | null
6+
persisted_at: string | null
7+
version: number
8+
lease_token: string | null
9+
lease_base_revision: string | null
10+
lease_version: number | null
11+
lease_expires_at: number | null
12+
}
13+
14+
const SITE_ID = "default"
15+
const LEASE_MS = 90_000
16+
const schemaReady = new WeakMap<object, Promise<void>>()
17+
18+
export class D1RevisionCoordinator implements RevisionCoordinator {
19+
constructor(private readonly database: D1Database, private readonly leaseMs = LEASE_MS) {}
20+
21+
state(): Promise<RevisionState> {
22+
return readWordPressState(this.database)
23+
}
24+
25+
acquire(): Promise<RevisionLease> {
26+
return beginStateLease(this.database, this.leaseMs)
27+
}
28+
29+
async release(lease: RevisionLease): Promise<void> {
30+
await releaseStateLease(this.database, lease)
31+
}
32+
33+
async abort(lease: RevisionLease): Promise<void> {
34+
await abortStateLease(this.database, lease)
35+
}
36+
37+
commit(lease: RevisionLease, pointer: MarkdownPointer): Promise<{ pointer: MarkdownPointer; version: number }> {
38+
return commitStateLease(this.database, lease, pointer)
39+
}
40+
41+
async reset(): Promise<void> {
42+
await resetWordPressState(this.database)
43+
}
44+
}
45+
46+
async function readWordPressState(database: D1Database): Promise<RevisionState> {
47+
await ensureSchema(database)
48+
const row = await readRow(database)
49+
return {
50+
schema: "wp-codebox/cloudflare-wordpress-state/v2",
51+
store: "d1",
52+
pointer: pointerFromRow(row),
53+
version: row.version,
54+
}
55+
}
56+
57+
async function beginStateLease(database: D1Database, leaseMs = LEASE_MS): Promise<RevisionLease> {
58+
await ensureSchema(database)
59+
const now = Date.now()
60+
const token = crypto.randomUUID()
61+
const expiresAt = now + leaseMs
62+
const result = await database.prepare(`UPDATE wp_codebox_state
63+
SET lease_token = ?, lease_base_revision = revision, lease_version = version, lease_expires_at = ?
64+
WHERE site_id = ? AND (lease_token IS NULL OR lease_expires_at <= ?)`)
65+
.bind(token, expiresAt, SITE_ID, now).run()
66+
if (result.meta.changes !== 1) {
67+
const active = await readRow(database)
68+
throw new RevisionConflict("A canonical WordPress lease is active.", active.lease_expires_at ?? undefined)
69+
}
70+
const row = await readRow(database)
71+
if (row.lease_token !== token || row.lease_version === null) throw new RevisionConflict("The canonical WordPress lease was not acquired.")
72+
return { token, pointer: pointerFromRow(row), version: row.lease_version, expiresAt }
73+
}
74+
75+
async function releaseStateLease(database: D1Database, lease: RevisionLease): Promise<{ released: true }> {
76+
await finishLease(database, lease, "release")
77+
return { released: true }
78+
}
79+
80+
async function abortStateLease(database: D1Database, lease: RevisionLease): Promise<{ aborted: true }> {
81+
await finishLease(database, lease, "abort")
82+
return { aborted: true }
83+
}
84+
85+
async function commitStateLease(database: D1Database, lease: RevisionLease, pointer: MarkdownPointer): Promise<{ pointer: MarkdownPointer; version: number }> {
86+
validatePointer(pointer)
87+
await ensureSchema(database)
88+
const baseRevision = lease.pointer?.revision ?? null
89+
const result = await database.prepare(`UPDATE wp_codebox_state
90+
SET revision = ?, manifest_key = ?, persisted_at = ?, version = version + 1,
91+
lease_token = NULL, lease_base_revision = NULL, lease_version = NULL, lease_expires_at = NULL
92+
WHERE site_id = ? AND lease_token = ? AND lease_expires_at > ? AND version = ? AND lease_version = ?
93+
AND ((revision IS NULL AND ? IS NULL) OR revision = ?)
94+
AND ((lease_base_revision IS NULL AND ? IS NULL) OR lease_base_revision = ?)`)
95+
.bind(pointer.revision, pointer.manifestKey, pointer.persistedAt, SITE_ID, lease.token, Date.now(), lease.version, lease.version,
96+
baseRevision, baseRevision, baseRevision, baseRevision).run()
97+
if (result.meta.changes !== 1) throw new RevisionConflict("The canonical pointer changed before D1 promotion.")
98+
return { pointer, version: lease.version + 1 }
99+
}
100+
101+
async function resetWordPressState(database: D1Database): Promise<{ reset: true }> {
102+
await ensureSchema(database)
103+
await database.prepare(`UPDATE wp_codebox_state
104+
SET revision = NULL, manifest_key = NULL, persisted_at = NULL, version = version + 1,
105+
lease_token = NULL, lease_base_revision = NULL, lease_version = NULL, lease_expires_at = NULL
106+
WHERE site_id = ?`).bind(SITE_ID).run()
107+
return { reset: true }
108+
}
109+
110+
async function finishLease(database: D1Database, lease: RevisionLease, action: "release" | "abort"): Promise<void> {
111+
await ensureSchema(database)
112+
const result = await database.prepare(`UPDATE wp_codebox_state
113+
SET lease_token = NULL, lease_base_revision = NULL, lease_version = NULL, lease_expires_at = NULL
114+
WHERE site_id = ? AND lease_token = ? AND lease_expires_at > ?`)
115+
.bind(SITE_ID, lease.token, Date.now()).run()
116+
if (result.meta.changes !== 1) throw new RevisionConflict(`The canonical WordPress lease cannot ${action} because it expired or changed.`)
117+
}
118+
119+
async function readRow(database: D1Database): Promise<StateRow> {
120+
const row = await database.prepare(`SELECT revision, manifest_key, persisted_at, version,
121+
lease_token, lease_base_revision, lease_version, lease_expires_at
122+
FROM wp_codebox_state WHERE site_id = ?`).bind(SITE_ID).first<StateRow>()
123+
if (!row) throw new Error("D1 WordPress state row is unavailable.")
124+
return row
125+
}
126+
127+
function pointerFromRow(row: StateRow): MarkdownPointer | null {
128+
if (row.revision === null && row.manifest_key === null && row.persisted_at === null) return null
129+
const pointer = { revision: row.revision, manifestKey: row.manifest_key, persistedAt: row.persisted_at }
130+
validatePointer(pointer)
131+
return pointer as MarkdownPointer
132+
}
133+
134+
function validatePointer(pointer: unknown): asserts pointer is MarkdownPointer {
135+
if (!pointer || typeof pointer !== "object") throw new RevisionConflict("A complete canonical pointer is required for D1 promotion.")
136+
const candidate = pointer as Partial<MarkdownPointer>
137+
if (typeof candidate.revision !== "string" || typeof candidate.manifestKey !== "string" || typeof candidate.persistedAt !== "string") {
138+
throw new RevisionConflict("A complete canonical pointer is required for D1 promotion.")
139+
}
140+
}
141+
142+
function ensureSchema(database: D1Database): Promise<void> {
143+
const key = database as object
144+
const existing = schemaReady.get(key)
145+
if (existing) return existing
146+
const pending = (async () => {
147+
await database.prepare(`CREATE TABLE IF NOT EXISTS wp_codebox_state (
148+
site_id TEXT PRIMARY KEY,
149+
revision TEXT,
150+
manifest_key TEXT,
151+
persisted_at TEXT,
152+
version INTEGER NOT NULL DEFAULT 0,
153+
lease_token TEXT,
154+
lease_base_revision TEXT,
155+
lease_version INTEGER,
156+
lease_expires_at INTEGER
157+
)`).run()
158+
await database.prepare(`INSERT OR IGNORE INTO wp_codebox_state (site_id, version) VALUES (?, 0)`).bind(SITE_ID).run()
159+
})()
160+
schemaReady.set(key, pending)
161+
pending.catch(() => schemaReady.delete(key))
162+
return pending
163+
}
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
export const PUBLISHED_REVISION_SCHEMA = "wp-codebox/published-revision/v1" as const
2+
export const PUBLISHED_PAGE_SCHEMA = "wp-codebox/wordpress-page/v2" as const
3+
export const R2_PUBLISHED_CURRENT_KEY = "sites/default/publications/current.json"
4+
export const R2_PUBLISHED_REVISION_PREFIX = "sites/default/publications/revisions"
5+
export const MAX_PUBLISHED_ROUTES = 1_000
6+
export const MAX_PUBLISHED_REVISION_BYTES = 512 * 1024
7+
export const MAX_PUBLISHED_PAGE_BYTES = 8 * 1024 * 1024
8+
9+
export interface PublishedRoute {
10+
route: string
11+
objectKey: string
12+
}
13+
14+
export interface PublishedRevision {
15+
schema: typeof PUBLISHED_REVISION_SCHEMA
16+
revision: string
17+
canonicalRevision: string
18+
publishedAt: string
19+
routes: PublishedRoute[]
20+
}
21+
22+
export function canonicalPublicRoute(input: Request | URL | string): string {
23+
const url = input instanceof Request ? new URL(input.url) : input instanceof URL ? new URL(input) : new URL(input, "https://wp-codebox-runtime.invalid")
24+
url.searchParams.sort()
25+
return `${url.pathname}${url.search}`
26+
}
27+
28+
export async function publishedPageObjectKey(canonicalRevision: string, route: string): Promise<string> {
29+
if (!isRevision(canonicalRevision) || !isCanonicalRoute(route)) throw new Error("Published page identity is invalid.")
30+
const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(route))
31+
const hash = Array.from(new Uint8Array(digest), (byte) => byte.toString(16).padStart(2, "0")).join("")
32+
return `sites/default/pages/${canonicalRevision}/${hash}.json`
33+
}
34+
35+
export function publishedRevisionObjectKey(revision: string): string {
36+
if (!isRevision(revision)) throw new Error("Published revision identity is invalid.")
37+
return `${R2_PUBLISHED_REVISION_PREFIX}/${revision}.json`
38+
}
39+
40+
export function validatePublishedRevision(value: unknown): PublishedRevision {
41+
if (!value || typeof value !== "object") throw new Error("Published revision is invalid.")
42+
const revision = value as Partial<PublishedRevision>
43+
if (revision.schema !== PUBLISHED_REVISION_SCHEMA || !isRevision(revision.revision) || !isRevision(revision.canonicalRevision)
44+
|| typeof revision.publishedAt !== "string" || !Number.isFinite(Date.parse(revision.publishedAt)) || !Array.isArray(revision.routes)
45+
|| revision.routes.length === 0 || revision.routes.length > MAX_PUBLISHED_ROUTES) throw new Error("Published revision is invalid.")
46+
let previous = ""
47+
for (const route of revision.routes) {
48+
if (!route || typeof route !== "object" || !isCanonicalRoute(route.route) || route.route <= previous
49+
|| route.objectKey !== `sites/default/pages/${revision.canonicalRevision}/${route.objectKey.split("/").at(-1)}`
50+
|| !/^sites\/default\/pages\/[a-f0-9-]{36}\/[a-f0-9]{64}\.json$/.test(route.objectKey)) throw new Error("Published revision route is invalid.")
51+
previous = route.route
52+
}
53+
return revision as PublishedRevision
54+
}
55+
56+
export function normalizePublishedRoutes(value: unknown): string[] {
57+
if (!Array.isArray(value) || value.length === 0 || value.length > MAX_PUBLISHED_ROUTES) throw new Error("Publication requires a bounded routes array.")
58+
const routes = value.map((route) => {
59+
if (typeof route !== "string" || !route.startsWith("/")) throw new Error("Publication route is invalid.")
60+
return canonicalPublicRoute(route)
61+
}).sort()
62+
if (routes.some((route, index) => index > 0 && route === routes[index - 1])) throw new Error("Publication routes must be unique.")
63+
return routes
64+
}
65+
66+
function isCanonicalRoute(route: string): boolean {
67+
if (!route.startsWith("/") || route.includes("#") || route.includes("\\") || route.length > 2_048) return false
68+
try {
69+
return canonicalPublicRoute(route) === route
70+
} catch {
71+
return false
72+
}
73+
}
74+
75+
function isRevision(revision: unknown): revision is string {
76+
return typeof revision === "string" && /^[a-f0-9-]{36}$/.test(revision)
77+
}

packages/runtime-cloudflare/src/request-routing.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ export type WorkerRequestRoute =
55
| { kind: "r2-mutate" }
66
| { kind: "operator-reset" }
77
| { kind: "operator-restore" }
8+
| { kind: "operator-publish" }
89
| { kind: "probe"; phase: string }
910

1011
export function routeWorkerRequest(request: Request): WorkerRequestRoute {
@@ -15,5 +16,6 @@ export function routeWorkerRequest(request: Request): WorkerRequestRoute {
1516
if (phase === "r2-mutate") return { kind: "r2-mutate" }
1617
if (phase === "operator-reset") return { kind: "operator-reset" }
1718
if (phase === "operator-restore") return { kind: "operator-restore" }
19+
if (phase === "operator-publish") return { kind: "operator-publish" }
1820
return { kind: "probe", phase }
1921
}

0 commit comments

Comments
 (0)