Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -725,7 +725,7 @@ Supported runtime commands today:
- `wordpress.server-page-load`: load `surface=admin|frontend` plus `path=<path>` or `url=<path-or-url>` through the live preview HTTP server without a browser. Its performance observation is marked `source=server-http` and `kind=server-page-load`.
- `wordpress.browser-page-load`: load `surface=admin|frontend` plus `path=<path>` or `url=<path-or-url>` in Playwright by wrapping `wordpress.browser-probe`.
- `wordpress.browser-probe`: boot the live preview, visit `url=<path-or-url>` with Playwright, and capture generic browser replay/audit evidence under `files/browser/`.
- `wordpress.browser-actions`: boot the live preview, drive it with an ordered interaction script (`steps-json`), assert browser behavior, and capture replay/audit evidence under `files/browser/`.
- `wordpress.browser-actions`: boot the live preview, drive it with an ordered interaction script (`steps-json`) or bounded adaptive exploration, optionally classify accessibility/keyboard/focus failures, and capture replay/audit evidence under `files/browser/`.

`wordpress.run-php` loads `/wordpress/wp-load.php` by default. Use `--arg bootstrap=none` for raw PHP.

Expand Down
38 changes: 38 additions & 0 deletions docs/adversarial-runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ and replay surfaces; they do not replace those surfaces.
does not assume a browser, HTTP library, application, or transport.
- `adversarial-browser.ts` owns DOM-derived journey planning, hostile generic
input generation, journey minimization, and user-facing oracle contracts.
- `browser-accessibility.ts` owns provider-neutral accessibility configuration,
normalized findings, capability status, and stable privacy-safe fingerprints.
The Playground adapter collects bounded DOM, focus, keyboard, geometry, and
redacted ARIA-tree evidence through the existing adaptive browser runner.
- `adversarial-artifacts.ts` writes bounded manifested finding and replay bundles,
removes declared secrets and machine-specific paths, and seals corpus/finding
identity with a deterministic content digest.
Expand Down Expand Up @@ -166,6 +170,40 @@ controls are repeated to expose duplicate side effects. Generic oracles cover:
Journey minimization uses deterministic subset replay to retain the shortest
sequence that preserves the oracle failure.

### Accessibility and keyboard exploration

Add `accessibility` to `adaptive-exploration-json` to scan the initial state,
each novel state, and the final state. The contract accepts `ruleTags`,
`includeScopes`, `excludeScopes`, `impactThreshold`, `cadence`, required or
optional `rules`, `focus`, and `accessibilityTree` capabilities, plus explicit
scan, violation, focus-history, tree-size, and keyboard-action budgets.

```json
{
"schema": "wp-codebox/browser-adaptive-exploration/v1",
"seed": "accessible-settings",
"startUrl": "/settings/",
"accessibility": {
"ruleTags": ["accessible-name", "keyboard-reachable", "focus-visible", "dialog-focus", "aria-state"],
"impactThreshold": "serious",
"cadence": ["initial", "novel-state", "final"],
"capabilities": { "rules": "required", "focus": "required", "accessibilityTree": "optional" },
"budgets": { "maxScans": 24, "maxViolationsPerScan": 25, "maxFocusTransitions": 100, "maxTreeChars": 20000, "maxKeyboardActions": 8 }
}
}
```

The collector classifies unnamed controls, keyboard-unreachable custom
controls, unexpected tab stops, hidden/inert/clipped/offscreen focus, focus loss,
dialog entry/containment/restoration failures, and observable ARIA state drift.
Every finding records its oracle, rule, impact, structural target, state and
transition context, screenshot, correlated bounded DOM snapshot, and stable
fingerprint. ARIA-tree names are redacted and target evidence excludes text.
Unavailable tree capabilities produce an explicit `unsupported` scan. Required
capabilities make the exploration incomplete; optional capabilities allow the
campaign to continue without turning the unsupported check into a pass.
Existing caller-supplied `accessibilityViolations` remain supported.

## Evidence and safety

`writeAdversarialEvidenceBundle()` writes a manifest, campaign result, one
Expand Down
2 changes: 1 addition & 1 deletion examples/recipes/adversarial-adaptive-browser.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
{
"command": "wordpress.browser-actions",
"args": [
"adaptive-exploration-json={\"schema\":\"wp-codebox/browser-adaptive-exploration/v1\",\"seed\":\"adaptive-browser-v1\",\"startUrl\":\"/\",\"budgets\":{\"maxActions\":24,\"maxStates\":16,\"maxTransitions\":32,\"maxDurationMs\":60000,\"maxArtifactBytes\":2097152,\"maxErrors\":10},\"actionFamilies\":[\"click\",\"fill\",\"select\",\"submit\",\"keyboard\",\"back\",\"reload\",\"repeat\",\"double-submit\"],\"resetPolicy\":{\"mode\":\"start-url\"}}",
"adaptive-exploration-json={\"schema\":\"wp-codebox/browser-adaptive-exploration/v1\",\"seed\":\"adaptive-browser-v1\",\"startUrl\":\"/\",\"budgets\":{\"maxActions\":24,\"maxStates\":16,\"maxTransitions\":32,\"maxDurationMs\":60000,\"maxArtifactBytes\":2097152,\"maxErrors\":10},\"actionFamilies\":[\"click\",\"fill\",\"select\",\"submit\",\"keyboard\",\"back\",\"reload\",\"repeat\",\"double-submit\"],\"resetPolicy\":{\"mode\":\"start-url\"},\"accessibility\":{\"ruleTags\":[\"accessible-name\",\"keyboard-reachable\",\"focus-visible\",\"dialog-focus\",\"aria-state\"],\"impactThreshold\":\"serious\",\"cadence\":[\"initial\",\"novel-state\",\"final\"],\"capabilities\":{\"rules\":\"required\",\"focus\":\"required\",\"accessibilityTree\":\"optional\"},\"budgets\":{\"maxScans\":24,\"maxViolationsPerScan\":25,\"maxFocusTransitions\":100,\"maxTreeChars\":20000,\"maxKeyboardActions\":8}}}",
"capture=steps,console,errors,network"
]
}
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,7 @@
"test:fuzz-run-recipe": "tsx tests/fuzz-run-recipe.test.ts",
"test:fuzz-suite-runner": "tsx tests/fuzz-suite-runner.test.ts",
"test:adversarial-runtime": "tsx --test tests/transport-faults.test.ts tests/adversarial-campaign.test.ts tests/adversarial-browser.test.ts tests/browser-clock-control.test.ts tests/adversarial-recipe-orchestration.test.ts tests/wordpress-adversarial-adapter.test.ts",
"test:browser-accessibility-oracles": "tsx --test tests/browser-accessibility-oracles.test.ts tests/browser-adaptive-exploration.test.ts tests/adversarial-browser.test.ts",
"test:wordpress-adversarial-adapter": "tsx tests/wordpress-adversarial-adapter.test.ts",
"test:playground-fuzz-suite-public": "tsx tests/playground-fuzz-suite-public.test.ts",
"test:nested-fuzz-suite-recipe-command": "tsx tests/nested-fuzz-suite-recipe-command.test.ts",
Expand Down
3 changes: 3 additions & 0 deletions packages/runtime-core/src/adversarial-browser.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { createHash } from "node:crypto"

import type { BrowserActionCorpusDescriptor, BrowserInteractionStep } from "./browser-interaction.js"
import type { BrowserAccessibilityFinding } from "./browser-accessibility.js"

export const ADVERSARIAL_BROWSER_PLAN_SCHEMA = "wp-codebox/adversarial-browser-plan/v1" as const
export const ADVERSARIAL_BROWSER_ORACLE_RESULT_SCHEMA = "wp-codebox/adversarial-browser-oracle-result/v1" as const
Expand All @@ -26,6 +27,7 @@ export interface AdversarialBrowserObservation {
loadingIndicators?: Array<{ id: string; visibleForMs: number }>
boxes?: Array<{ id: string; x: number; y: number; width: number; height: number; viewportWidth: number; viewportHeight: number; focused?: boolean; visible?: boolean; clipped?: boolean }>
accessibilityViolations?: Array<{ rule: string; target: string; impact?: string }>
accessibilityFindings?: BrowserAccessibilityFinding[]
effects?: Array<{ id: string; count: number; expectedMaximum?: number }>
}

Expand Down Expand Up @@ -91,6 +93,7 @@ export function evaluateAdversarialBrowserOracles(observation: AdversarialBrowse
if (box.focused && box.visible === false) failures.push({ oracle: "layout", code: "browser-invisible-focus", message: `Focused element ${box.id} is not visible.`, target: box.id })
}
for (const violation of observation.accessibilityViolations ?? []) failures.push({ oracle: "accessibility", code: `browser-a11y-${violation.rule}`, message: `Accessibility rule ${violation.rule} failed.`, target: violation.target })
for (const finding of observation.accessibilityFindings ?? []) failures.push({ oracle: finding.oracle === "accessibility" ? "accessibility" : finding.oracle === "focus" ? "layout" : "dead-control", code: finding.code, message: `${finding.classification} (${finding.impact}).`, target: finding.target.locator })
for (const effect of observation.effects ?? []) {
if (effect.count > (effect.expectedMaximum ?? 1)) failures.push({ oracle: "duplicate-effect", code: "browser-duplicate-effect", message: `Effect ${effect.id} occurred ${effect.count} times.`, target: effect.id })
}
Expand Down
167 changes: 167 additions & 0 deletions packages/runtime-core/src/browser-accessibility.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
import { createHash } from "node:crypto"

import type { BrowserAdaptiveAction } from "./browser-adaptive-exploration.js"
import { isPlainObject, stableJson, stripUndefined } from "./object-utils.js"

export const BROWSER_ACCESSIBILITY_SCHEMA = "wp-codebox/browser-accessibility/v1" as const
export const BROWSER_ACCESSIBILITY_RULES = [
"accessible-name",
"keyboard-reachable",
"tab-order",
"focus-visible",
"focus-loss",
"dialog-focus",
"aria-state",
] as const

export type BrowserAccessibilityRule = typeof BROWSER_ACCESSIBILITY_RULES[number]
export type BrowserAccessibilityImpact = "minor" | "moderate" | "serious" | "critical"
export type BrowserAccessibilityScanPhase = "initial" | "novel-state" | "final" | "replay"

export interface BrowserAccessibilityContract {
schema: typeof BROWSER_ACCESSIBILITY_SCHEMA
ruleTags: BrowserAccessibilityRule[]
includeScopes: string[]
excludeScopes: string[]
impactThreshold: BrowserAccessibilityImpact
cadence: Array<"initial" | "novel-state" | "final">
capabilities: {
rules: "required" | "optional"
focus: "required" | "optional"
accessibilityTree: "required" | "optional" | "disabled"
}
budgets: {
maxScans: number
maxViolationsPerScan: number
maxTargetsPerViolation: number
maxFocusTransitions: number
maxTreeChars: number
maxKeyboardActions: number
maxFrames: number
}
}

export interface BrowserAccessibilityTargetEvidence {
locator: string
frameId: string
tag: string
role?: string
states?: Partial<Record<"expanded" | "selected" | "checked" | "busy" | "pressed" | "hidden" | "inert", string>>
box?: { x: number; y: number; width: number; height: number; viewportWidth: number; viewportHeight: number }
}

export interface BrowserAccessibilityFinding {
oracle: "accessibility" | "keyboard" | "focus"
rule: BrowserAccessibilityRule
code: string
impact: BrowserAccessibilityImpact
classification: string
fingerprint: string
target: BrowserAccessibilityTargetEvidence
stateDigest?: string
transitionId?: string
actionId?: string
expected?: string
actual?: string
}

export interface BrowserAccessibilityFocusTransition {
index: number
phase: BrowserAccessibilityScanPhase
actionId?: string
from?: string
to: string
visible: boolean
insideDialog: boolean
}

export interface BrowserAccessibilityScan {
index: number
phase: BrowserAccessibilityScanPhase
status: "passed" | "findings" | "unsupported" | "inconclusive"
stateDigest?: string
transitionId?: string
actionId?: string
findings: BrowserAccessibilityFinding[]
accessibilityTree?: { status: "captured" | "unsupported" | "inconclusive"; snapshot?: string; reason?: string; truncated?: boolean }
diagnostics: Array<{ code: string; message: string }>
artifacts?: { screenshot?: string; domSnapshot?: string }
}

export interface BrowserAccessibilityEvidence {
schema: typeof BROWSER_ACCESSIBILITY_SCHEMA
collector: { name: string; version: string; capabilities: { rules: "supported" | "unsupported"; focus: "supported" | "unsupported"; accessibilityTree: "supported" | "unsupported" } }
contract: BrowserAccessibilityContract
scans: BrowserAccessibilityScan[]
focusHistory: BrowserAccessibilityFocusTransition[]
diagnostics: Array<{ code: string; message: string }>
summary: { scans: number; findings: number; passed: number; unsupported: number; inconclusive: number; truncated: boolean }
}

export interface BrowserAccessibilityCollector {
beforeAction(action: BrowserAdaptiveAction): Promise<void>
scan(input: { phase: BrowserAccessibilityScanPhase; stateDigest?: string; transitionId?: string; action?: BrowserAdaptiveAction; record?: boolean }): Promise<BrowserAccessibilityScan>
reset(): Promise<void>
evidence(): BrowserAccessibilityEvidence
}

export function browserAccessibilityContract(input: unknown): BrowserAccessibilityContract | undefined {
if (!isPlainObject(input) || input.enabled === false) return undefined
const budgets = object(input.budgets)
const capabilities = object(input.capabilities)
const requestedRules = array(input.ruleTags ?? input.rule_tags).filter((value): value is BrowserAccessibilityRule => (BROWSER_ACCESSIBILITY_RULES as readonly unknown[]).includes(value))
const requestedCadence = array(input.cadence).filter((value): value is "initial" | "novel-state" | "final" => value === "initial" || value === "novel-state" || value === "final")
return {
schema: BROWSER_ACCESSIBILITY_SCHEMA,
ruleTags: requestedRules.length > 0 ? [...new Set(requestedRules)] : [...BROWSER_ACCESSIBILITY_RULES],
includeScopes: boundedStrings(input.includeScopes ?? input.include_scopes, 20),
excludeScopes: boundedStrings(input.excludeScopes ?? input.exclude_scopes, 20),
impactThreshold: impact(input.impactThreshold ?? input.impact_threshold),
cadence: requestedCadence.length > 0 ? [...new Set(requestedCadence)] : ["initial", "novel-state", "final"],
capabilities: {
rules: requirement(capabilities.rules),
focus: requirement(capabilities.focus),
accessibilityTree: capabilities.accessibilityTree === "disabled" || capabilities.accessibility_tree === "disabled" ? "disabled" : requirement(capabilities.accessibilityTree ?? capabilities.accessibility_tree),
},
budgets: {
maxScans: integer(budgets.maxScans ?? budgets.max_scans, 32, 1, 500),
maxViolationsPerScan: integer(budgets.maxViolationsPerScan ?? budgets.max_violations_per_scan, 25, 1, 200),
maxTargetsPerViolation: integer(budgets.maxTargetsPerViolation ?? budgets.max_targets_per_violation, 3, 1, 20),
maxFocusTransitions: integer(budgets.maxFocusTransitions ?? budgets.max_focus_transitions, 100, 1, 1_000),
maxTreeChars: integer(budgets.maxTreeChars ?? budgets.max_tree_chars, 20_000, 256, 200_000),
maxKeyboardActions: integer(budgets.maxKeyboardActions ?? budgets.max_keyboard_actions, 12, 0, 50),
maxFrames: integer(budgets.maxFrames ?? budgets.max_frames, 16, 1, 100),
},
}
}

export function browserAccessibilityFindingFingerprint(finding: Omit<BrowserAccessibilityFinding, "fingerprint">): string {
const stable = stripUndefined({
oracle: finding.oracle,
rule: finding.rule,
code: finding.code,
impact: finding.impact,
classification: finding.classification,
target: {
locator: normalizeLocator(finding.target.locator),
frameId: finding.target.frameId,
tag: finding.target.tag,
role: finding.target.role,
states: finding.target.states,
},
expected: finding.expected,
actual: finding.actual,
})
return createHash("sha256").update("wp-codebox/browser-accessibility-finding/v1\n").update(stableJson(stable)).digest("hex")
}

function normalizeLocator(value: string): string {
return value.replace(/([_:-](?:[a-z]{0,4})?)[a-f0-9]{8,}/gi, "$1<generated>").slice(0, 240)
}

function object(value: unknown): Record<string, unknown> { return isPlainObject(value) ? value : {} }
function array(value: unknown): unknown[] { return Array.isArray(value) ? value : [] }
function boundedStrings(value: unknown, maximum: number): string[] { return array(value).filter((item): item is string => typeof item === "string" && item.trim().length > 0).slice(0, maximum).map((item) => item.trim().slice(0, 240)) }
function requirement(value: unknown): "required" | "optional" { return value === "required" ? "required" : "optional" }
function impact(value: unknown): BrowserAccessibilityImpact { return value === "minor" || value === "moderate" || value === "critical" ? value : "serious" }
function integer(value: unknown, fallback: number, minimum: number, maximum: number): number { const numeric = Number(value); return Number.isFinite(numeric) ? Math.max(minimum, Math.min(maximum, Math.floor(numeric))) : fallback }
Loading
Loading