feat(runtime-services): add native ephemeral MariaDB provider (#2062)

* feat(runtime-services): add native ephemeral MariaDB provider

* fix(runtime-services): harden native MariaDB containment

* fix(runtime-services): close native MariaDB isolation gaps

Author: chubes4

docs/native-mariadb-runtime-service.md

@@ -0,0 +1,86 @@
+# Native MariaDB Runtime Service
+
+`provider: "native"` provisions a short-lived MariaDB process for MySQL-compatible workloads on Docker-free hosts. The generic runtime-service layer owns its complete lifecycle. Recipes may select only `provider: "native"` and `engine: "mariadb"`; host, port, credentials, sockets, configuration files, and filesystem paths are not configurable.
+
+## Isolation Contract
+
+- The provider rejects UID 0 and resolves and validates `mariadbd`, `mariadb-install-db`, `mariadb`, util-linux `prlimit`, `truncate`, `mkfs.ext4`, `fuse2fs`, and `fusermount3` identities before allocation.
+- Default executable discovery ignores the caller's `PATH`, searches fixed system directories, resolves symlinks, and requires every executable and ancestor to be UID-0-owned and not group/other writable. Child processes receive a fixed minimal `PATH`.
+- Every run gets a mode-`0700` `mkdtemp` root. MariaDB's data directory, temporary directory, socket, PID file, error log, plugin directory, secure-file directory, home, and working directory are all inside the bounded image.
+- Initialization, daemon, and administrative clients run through fixed hard rlimits; each database command itself begins with `--no-defaults`. No shell or default socket is used.
+- Administration uses only the private Unix socket. Workloads receive a generated least-privilege `runtime` account over loopback TCP through the existing ephemeral connector-secret channel.
+- Initialization, FUSE, and daemon commands each run in a new owned process group. Cleanup addresses the retained group, not a PID file; it waits for the complete group to disappear after graceful shutdown, then applies group-wide `SIGTERM` and `SIGKILL` as needed. Linux captures the leader start-time token and revalidates it while the leader is alive. Root device/inode identity and a symlink-free tree are revalidated before recursive removal.
+- Failures, aborts, timeouts, startup crashes, and partial initialization all enter the same graceful-shutdown, forced-shutdown, wait, and verified-removal state machine. Cleanup failure is terminal and retained in bounded lifecycle evidence.
+- Address space is capped at 2 GiB, CPU at 300 seconds, individual daemon files at 128 MiB, open files at 512, and processes/threads at 512. Core files and locked memory are disabled. The datadir is a provider-owned 256 MiB ext4 image formatted with 4,096 inodes and mounted through unprivileged FUSE; device, byte, and inode geometry must be proven before initialization. Hosts without this containment fail closed.
+- Recipes may declare at most two native services, bounding the aggregate native ceiling to two 2-GiB address spaces, two 256-MiB images, and the corresponding process/file limits.
+- The daemon uses an empty bounded plugin directory and a bounded `secure-file-priv` directory. Startup fails unless every enabled storage engine is on the fixed local-only allowlist; FEDERATED, CONNECT, SPIDER, S3, and unknown enabled engines are rejected. The runtime account has privileges only on its generated database and cannot install plugins.
+- Runtime descriptor discovery distinguishes package support from host availability. It advertises the active native capability only after trusted tools and a real create, mount, geometry, write, unmount, process-group-exit, and removal probe succeed; unavailable reasons are stable codes without private paths.
+- Cleanup is single-flight for concurrent callers. A failed attempt may be retried, and evidence transitions from `teardown: failed` to a consistent released/completed state only after the retry proves cleanup.
+- Evidence contains service ID, engine/provider version, lifecycle state, and memory measurements only. It never contains credentials or private absolute paths.
+
+## Daemon Arguments
+
+The provider starts `mariadbd` through `prlimit` with these fixed controls plus provider-owned absolute paths and a freshly allocated loopback port:
+
+```text
+--no-defaults
+--datadir=<bounded-image>/database
+--socket=<bounded-image>/runtime/server.sock
+--pid-file=<bounded-image>/runtime/server.pid
+--log-error=<bounded-image>/runtime/server.log
+--tmpdir=<bounded-image>/tmp
+--plugin-dir=<bounded-image>/plugins
+--secure-file-priv=<bounded-image>/files
+--bind-address=127.0.0.1
+--port=<ephemeral>
+--skip-name-resolve
+--skip-log-bin
+--skip-host-cache
+--skip-slave-start
+--skip-symbolic-links
+--local-infile=OFF
+--performance-schema=OFF
+--skip-feedback
+--innodb-buffer-pool-size=32M
+--innodb-buffer-pool-size-max=32M
+--innodb-log-buffer-size=4M
+--key-buffer-size=8M
+--aria-pagecache-buffer-size=8M
+--thread-handling=pool-of-threads
+--thread-pool-size=1
+--aria-log-file-size=16M
+--innodb-file-per-table=OFF
+--innodb-data-file-path=ibdata1:32M:autoextend:max:96M
+--innodb-temp-data-file-path=ibtmp1:16M:autoextend:max:32M
+--innodb-log-file-size=32M
+--max-connections=10
+--max-prepared-stmt-count=256
+--max-session-mem-used=32M
+--open-files-limit=512
+--thread-cache-size=0
+--table-open-cache=128
+--table-definition-cache=128
+--tmp-table-size=8M
+--max-heap-table-size=8M
+```
+
+The hard address-space ceiling is 2 GiB. Linux runs record the owned daemon's post-readiness RSS in evidence; the MariaDB 10.11 integration gate additionally requires observed RSS to remain at or below 128 MiB.
+
+## Integration Contract
+
+Homeboy Extensions issue #2412 may select this provider by emitting the generic runtime-service recipe declaration below. WP Codebox performs all provisioning and cleanup; callers do not supply or manage native process details.
+
+```json
+{
+  "id": "wordpress-database",
+  "kind": "mysql",
+  "configuration": { "provider": "native", "engine": "mariadb" },
+  "outputs": {
+    "host": "DB_HOST",
+    "port": "DB_PORT",
+    "username": "DB_USER",
+    "password": "DB_PASSWORD",
+    "database": "DB_NAME"
+  }
+}
+```

package.json

@@ -250,8 +250,9 @@
     "test:generic-ability-runtime-run": "tsx tests/generic-ability-runtime-run.test.ts",
     "test:provider-runtime-contracts": "tsx tests/provider-runtime-contracts.test.ts",
     "test:runtime-requirements-readiness": "tsx tests/runtime-requirements-readiness.test.ts",
-    "test:runtime-services": "tsx tests/runtime-services.test.ts && tsx tests/external-mysql-runtime-service.test.ts",
+    "test:runtime-services": "tsx tests/runtime-services.test.ts && tsx tests/external-mysql-runtime-service.test.ts && tsx tests/native-mariadb-runtime-service.test.ts",
     "test:runtime-services-lifecycle": "tsx tests/runtime-services-lifecycle.test.ts",
+    "test:native-mariadb-runtime-service-integration": "tsx tests/native-mariadb-runtime-service.integration.test.ts",
     "test:disposable-mysql-mysqli-e2e": "tsx tests/disposable-mysql-mysqli.integration.test.ts",
     "test:runtime-contract-manifest": "tsx tests/runtime-contract-manifest.test.ts",
     "test:runtime-contract-package-exports": "tsx tests/runtime-contract-package-exports.test.ts",

packages/cli/src/commands/discovery.ts

@@ -2,6 +2,7 @@ import { createWorkspaceRecipeJsonSchema, runtimeDescriptor, type RuntimeDescrip
 import { commandRegistry, type CommandDefinition } from "@automattic/wp-codebox-core/contracts"
 import { printCommandCatalogHumanOutput, printRecipeSchemaHumanOutput, printRuntimeDescriptorHumanOutput } from "../output.js"
 import { cliRuntimeBackendRecipePolicy, listCliRecipeCommandDefinitions, listCliRuntimeBackendKinds } from "../runtime-backends.js"
+import { nativeMariaDbHostReadiness } from "../runtime-services.js"
 
 interface CommandCatalogOutput {
   schema: "wp-codebox/command-catalog/v1"
@@ -40,7 +41,7 @@ export async function runRecipeSchemaCommand(args: string[]): Promise<number> {
 
 export async function runRuntimeDescriptorCommand(args: string[]): Promise<number> {
   const json = parseDiscoveryJsonOption(args)
-  const output = runtimeDescriptorOutput()
+  const output = await runtimeDescriptorOutput()
   if (!json) {
     printRuntimeDescriptorHumanOutput(output)
     return 0
@@ -109,6 +110,6 @@ function recipeSchemaOutput(): RecipeSchemaOutput {
   }
 }
 
-function runtimeDescriptorOutput(): RuntimeDescriptor {
-  return runtimeDescriptor()
+async function runtimeDescriptorOutput(): Promise<RuntimeDescriptor> {
+  return runtimeDescriptor({ nativeMariaDb: await nativeMariaDbHostReadiness() })
 }

packages/cli/src/commands/recipe-run-artifact-pointers.ts

@@ -4,6 +4,7 @@ import type { ArtifactBundle, RecipeRunSummary, RuntimeInfo } from "@automattic/
 import { stripUndefined } from "@automattic/wp-codebox-core/internals"
 import type { RunOutput } from "../runtime-command-wrappers.js"
 import type { RecipeArtifactPointerCommandStatus, RecipeArtifactPointerState, RecipeBrowserEvidence, RecipeDiagnosticArtifactRef, RecipePhaseEvidence } from "./recipe-run-types.js"
+import type { RuntimeServiceEvidence } from "../runtime-services.js"
 
 export class RecipeArtifactPointerTracker {
   private command: string | undefined
@@ -15,6 +16,7 @@ export class RecipeArtifactPointerTracker {
   private browserEvidence: RecipeBrowserEvidence[] = []
   private diagnosticArtifacts: RecipeDiagnosticArtifactRef[] = []
   private result: RecipeRunSummary | undefined
+  private managedRuntimeServices: RuntimeServiceEvidence[] = []
 
   constructor(private readonly directory: string | undefined, private readonly runId: string, private readonly recipePath: string, private readonly startedAt: string) {}
 
@@ -32,6 +34,7 @@ export class RecipeArtifactPointerTracker {
     this.browserEvidence = state.browserEvidence ?? this.browserEvidence
     this.diagnosticArtifacts = state.diagnosticArtifacts ?? this.diagnosticArtifacts
     this.result = state.result ?? this.result
+    this.managedRuntimeServices = state.managedRuntimeServices ?? this.managedRuntimeServices
 
     const pointer = stripUndefined({
       schema: "wp-codebox/recipe-run-artifact-pointer/v1",
@@ -47,6 +50,7 @@ export class RecipeArtifactPointerTracker {
       failure: this.failure,
       failurePhase: recipeArtifactPointerFailurePhase(this.failure, this.phases),
       result: this.result,
+      managedRuntimeServices: this.managedRuntimeServices.length > 0 ? this.managedRuntimeServices : undefined,
       browserEvidence: this.browserEvidence.length > 0 ? this.browserEvidence : undefined,
       diagnosticArtifacts: this.diagnosticArtifacts.length > 0 ? this.diagnosticArtifacts : undefined,
       ...await recipeArtifactPointerArtifactState(this.directory, this.runtime, this.artifacts),

packages/cli/src/commands/recipe-run-finalizer.ts

@@ -67,6 +67,7 @@ interface RecipeRunCommonOutputFields {
   replayStatus?: RecipeRunOutput["replayStatus"]
   fuzzRun?: RecipeRunOutput["fuzzRun"]
   provenance?: RecipeRunOutput["provenance"]
+  managedRuntimeServices?: RecipeRunOutput["managedRuntimeServices"]
   artifacts?: ArtifactBundle
   interruption?: RecipeRunOutput["interruption"]
 }
@@ -194,7 +195,7 @@ export async function finalizeCompletedRecipeRun(args: FinalizeCompletedRecipeRu
   }) as RecipeRunOutput)
   runRecord = await args.runRegistry.update(args.runRecord.runId, { result: runtimeRunResultFromRecipeSummary(output.result!) })
   output.run = runRecord
-  await args.artifactPointer.update({ commandStatus: args.success ? "completed" : "failed", runtime: args.runtime, artifacts: args.artifacts, failure: args.failure, phases: args.phaseEvidence, stepFailures: args.output.stepFailures, browserEvidence: args.browserEvidence, result: output.result })
+  await args.artifactPointer.update({ commandStatus: args.success ? "completed" : "failed", runtime: args.runtime, artifacts: args.artifacts, failure: args.failure, phases: args.phaseEvidence, stepFailures: args.output.stepFailures, browserEvidence: args.browserEvidence, managedRuntimeServices: args.output.managedRuntimeServices, result: output.result })
   return output
 }
 
@@ -220,19 +221,19 @@ export async function finalizeRecoveredRecipeFailure(args: FinalizeRecoveredReci
   }) as RecipeRunOutput)
   runRecord = await args.runRegistry.update(args.runRecord.runId, { result: runtimeRunResultFromRecipeSummary(output.result!) })
   output.run = runRecord
-  await args.artifactPointer.update({ commandStatus: "failed", ...(args.runtime ? { runtime: args.runtime } : {}), ...(args.artifacts ? { artifacts: args.artifacts } : {}), failure: args.serializedError, phases: args.phaseEvidence, stepFailures: args.output.stepFailures, browserEvidence: args.browserEvidence, diagnosticArtifacts: args.diagnosticArtifacts, result: output.result })
+  await args.artifactPointer.update({ commandStatus: "failed", ...(args.runtime ? { runtime: args.runtime } : {}), ...(args.artifacts ? { artifacts: args.artifacts } : {}), failure: args.serializedError, phases: args.phaseEvidence, stepFailures: args.output.stepFailures, browserEvidence: args.browserEvidence, diagnosticArtifacts: args.diagnosticArtifacts, managedRuntimeServices: args.output.managedRuntimeServices, result: output.result })
   return output
 }
 
-export async function runRecipeCleanup(runRegistry: RuntimeRunRegistry, runRecord: RuntimeRunRecord, cleanup: () => Promise<void>): Promise<RunResourceCleanupEvidence> {
+export async function runRecipeCleanup(runRegistry: RuntimeRunRegistry, runRecord: RuntimeRunRecord, cleanup: () => Promise<void>, finalMetadata?: () => Record<string, unknown>): Promise<RunResourceCleanupEvidence> {
   const startedAtMs = Date.now()
   await runRegistry.update(runRecord.runId, { cleanup: { status: "running" } })
   try {
     await cleanup()
-    const updatedRunRecord = await runRegistry.update(runRecord.runId, { cleanup: { status: "succeeded" } })
+    const updatedRunRecord = await runRegistry.update(runRecord.runId, { cleanup: { status: "succeeded" }, ...(finalMetadata ? { metadata: finalMetadata() } : {}) })
     return cleanupEvidenceFromRunRecord(updatedRunRecord, Date.now() - startedAtMs)
   } catch (error) {
-    const updatedRunRecord = await runRegistry.update(runRecord.runId, { cleanup: { status: "failed", error: serializeError(error) } })
+    const updatedRunRecord = await runRegistry.update(runRecord.runId, { cleanup: { status: "failed", error: serializeError(error) }, ...(finalMetadata ? { metadata: finalMetadata() } : {}) })
     const cleanupError = serializeRecipeRunError(error)
     throw new RunResourceCleanupError(
       cleanupEvidenceFromRunRecord(updatedRunRecord, Date.now() - startedAtMs, cleanupError),

packages/cli/src/commands/recipe-run-types.ts

@@ -5,6 +5,7 @@ import type { RecipeExternalServiceBoundaryHostCorrelation } from "../recipe-ext
 import type { RecipeValidationIssue, RecipeWorkflowPhase } from "../recipe-validation.js"
 import type { RunOutput } from "../runtime-command-wrappers.js"
 import type { RecipeAdversarialCampaignOutput } from "../adversarial-recipe.js"
+import type { RuntimeServiceEvidence } from "../runtime-services.js"
 
 export interface RecipeRunOptions {
   recipePath: string
@@ -83,6 +84,7 @@ export interface RecipeRunOutput {
   fuzzRun?: RecipeFuzzRunResult
   adversarialCampaigns?: RecipeAdversarialCampaignOutput[]
   provenance?: RecipeRunProvenance
+  managedRuntimeServices?: RuntimeServiceEvidence[]
   result?: RecipeRunSummary
   artifacts?: ArtifactBundle
   run?: RuntimeRunRecord
@@ -263,6 +265,7 @@ export interface RecipeArtifactPointerState {
   browserEvidence?: RecipeBrowserEvidence[]
   diagnosticArtifacts?: RecipeDiagnosticArtifactRef[]
   result?: RecipeRunSummary
+  managedRuntimeServices?: RuntimeServiceEvidence[]
 }
 
 export interface RecipeEffectiveRecipeArtifact {

packages/cli/src/commands/recipe-run.ts

@@ -445,7 +445,7 @@ export async function runRecipe(options: RecipeRunOptions, interruption?: Recipe
         browserEvidence,
         replayStatus: evidence.replayStatus ? recipeReplayStatusOutput(evidence.replayStatus) : undefined,
         failure: recipeFailure,
-        output: { ...completedRecipeOutputFields({ executions, componentContracts: componentContractResults(recipe, extraPlugins, phaseTracker.list(), executions), stagedFiles: stagedFiles.map(recipeRunStagedFile), fixtureDatabases, siteSeeds, distributionSetupArtifacts, distributionStartupProbes, probes, declaredArtifacts, stepFailures, phaseEvidence: phaseTracker.list(), advisoryFailures, browserEvidence, benchResultsList, fuzzRun: fuzzRunResult, evidence }), ...(adversarialCampaigns.length > 0 ? { adversarialCampaigns } : {}), provenance: recipeRunProvenance(recipe, recipePath) },
+        output: { ...completedRecipeOutputFields({ executions, componentContracts: componentContractResults(recipe, extraPlugins, phaseTracker.list(), executions), stagedFiles: stagedFiles.map(recipeRunStagedFile), fixtureDatabases, siteSeeds, distributionSetupArtifacts, distributionStartupProbes, probes, declaredArtifacts, stepFailures, phaseEvidence: phaseTracker.list(), advisoryFailures, browserEvidence, benchResultsList, fuzzRun: fuzzRunResult, evidence }), ...(adversarialCampaigns.length > 0 ? { adversarialCampaigns } : {}), provenance: recipeRunProvenance(recipe, recipePath), managedRuntimeServices: serviceEvidence },
       })
     }
 
@@ -464,7 +464,7 @@ export async function runRecipe(options: RecipeRunOptions, interruption?: Recipe
       phaseEvidence: phaseTracker.list(),
       browserEvidence,
       replayStatus: evidence.replayStatus ? recipeReplayStatusOutput(evidence.replayStatus) : undefined,
-      output: { ...completedRecipeOutputFields({ executions, componentContracts: componentContractResults(recipe, extraPlugins, phaseTracker.list(), executions), stagedFiles: stagedFiles.map(recipeRunStagedFile), fixtureDatabases, siteSeeds, distributionSetupArtifacts, distributionStartupProbes, probes, declaredArtifacts, stepFailures, phaseEvidence: phaseTracker.list(), advisoryFailures, browserEvidence, benchResultsList, fuzzRun: fuzzRunResult, evidence }), ...(adversarialCampaigns.length > 0 ? { adversarialCampaigns } : {}), provenance: recipeRunProvenance(recipe, recipePath) },
+      output: { ...completedRecipeOutputFields({ executions, componentContracts: componentContractResults(recipe, extraPlugins, phaseTracker.list(), executions), stagedFiles: stagedFiles.map(recipeRunStagedFile), fixtureDatabases, siteSeeds, distributionSetupArtifacts, distributionStartupProbes, probes, declaredArtifacts, stepFailures, phaseEvidence: phaseTracker.list(), advisoryFailures, browserEvidence, benchResultsList, fuzzRun: fuzzRunResult, evidence }), ...(adversarialCampaigns.length > 0 ? { adversarialCampaigns } : {}), provenance: recipeRunProvenance(recipe, recipePath), managedRuntimeServices: serviceEvidence },
     })
   } catch (error) {
     const failedServiceEvidence = runtimeServiceEvidenceFromError(error)
@@ -592,6 +592,7 @@ export async function runRecipe(options: RecipeRunOptions, interruption?: Recipe
         ...(adversarialCampaigns.length > 0 ? { adversarialCampaigns } : {}),
         diagnostics: recipeRuntimeDiagnostics(recipe, executions, error),
         provenance: recipeRunProvenance(recipe, recipePath, diagnosticArtifacts),
+        managedRuntimeServices: serviceEvidence,
       },
     })
   } finally {
@@ -611,16 +612,12 @@ export async function runManagedServiceCleanup(
   cleanup: () => Promise<void>,
 ): Promise<RunResourceCleanupEvidence> {
   try {
-    return await runRecipeCleanup(runRegistry, runRecord, cleanup)
+    return await runRecipeCleanup(runRegistry, runRecord, cleanup, () => ({ managedRuntimeServices: serviceEvidence }))
   } catch (error) {
     if (preservePrimaryFailure && error instanceof RunResourceCleanupError) {
       return error.evidence
     }
     throw error
-  } finally {
-    await runRegistry.update(runRecord.runId, {
-      metadata: { managedRuntimeServices: serviceEvidence },
-    })
   }
 }