Merge branch 'codex/native-input-conformance'

Author: Roy Lin

.github/workflows/ci.yml

@@ -49,12 +49,15 @@ jobs:
           - name: AppKit
             os: macos-latest
             feature: appkit-native
+            backend: appkit
           - name: GTK4
             os: ubuntu-latest
             feature: gtk4-native
+            backend: gtk4
           - name: WinUI
             os: windows-latest
             feature: winui-native
+            backend: winui
     steps:
       - name: Checkout
         uses: actions/checkout@v5
@@ -85,6 +88,44 @@ jobs:
         shell: bash
         run: cargo check --locked --all-targets --features "${{ matrix.feature }}"
 
+      - name: Generate required native input matrix
+        shell: bash
+        run: |
+          cargo run --locked --bin a3s-gui-native-input-conformance -- manifest "${{ matrix.backend }}" > "native-input-${{ matrix.backend }}.json"
+
+      - name: Upload required native input matrix
+        uses: actions/upload-artifact@v6
+        with:
+          name: native-input-requirements-${{ matrix.backend }}
+          path: native-input-${{ matrix.backend }}.json
+          if-no-files-found: error
+
+      - name: Install Windows App Runtime 1.7 for OS-input smoke
+        if: runner.os == 'Windows' && github.event_name == 'workflow_dispatch'
+        shell: pwsh
+        run: |
+          $installer = Join-Path $env:RUNNER_TEMP 'WindowsAppRuntimeInstall-x64.exe'
+          Invoke-WebRequest 'https://aka.ms/windowsappsdk/1.7/1.7.260224002/windowsappruntimeinstall-x64.exe' -OutFile $installer
+          & $installer --quiet
+          if ($LASTEXITCODE -ne 0) {
+            throw "Windows App Runtime installer exited with code $LASTEXITCODE"
+          }
+
+      - name: Capture WinUI button-backed role OS-input smoke evidence
+        if: runner.os == 'Windows' && github.event_name == 'workflow_dispatch'
+        shell: pwsh
+        run: >-
+          cargo run --locked --no-default-features --features winui-native
+          --bin a3s-gui-winui-input-smoke -- native-input-winui-smoke.json
+
+      - name: Upload WinUI button-backed role OS-input smoke evidence
+        if: always() && runner.os == 'Windows' && github.event_name == 'workflow_dispatch'
+        uses: actions/upload-artifact@v6
+        with:
+          name: native-input-smoke-winui
+          path: native-input-winui-smoke.json
+          if-no-files-found: error
+
   bundle-smoke:
     name: Bundle Smoke (${{ matrix.name }})
     runs-on: ${{ matrix.os }}

Cargo.toml

@@ -14,6 +14,15 @@ categories = ["gui", "rendering", "accessibility"]
 name = "a3s_gui"
 path = "src/lib.rs"
 
+[[bin]]
+name = "a3s-gui-native-input-conformance"
+path = "src/bin/native_input_conformance.rs"
+
+[[bin]]
+name = "a3s-gui-winui-input-smoke"
+path = "src/bin/winui_input_smoke.rs"
+required-features = ["winui-native"]
+
 [[example]]
 name = "component_playground"
 required-features = ["authoring"]
@@ -178,8 +187,13 @@ gtk4_crate = { package = "gtk4", version = "0.11.4", optional = true }
 [target.'cfg(target_os = "windows")'.dependencies]
 windows = { version = "0.62.0", optional = true, features = [
     "Foundation",
+    "Win32_System_Com",
+    "Win32_System_SystemInformation",
+    "Win32_UI_Controls",
+    "Win32_UI_Accessibility",
     "Win32_Foundation",
     "Win32_UI_Input_KeyboardAndMouse",
+    "Win32_UI_Input_Pointer",
     "Win32_UI_Shell",
     "Win32_UI_WindowsAndMessaging",
 ] }

README.md

@@ -27,8 +27,18 @@ object graph at the host boundary.
   background inertness, nested focus containment/restoration, typed anchored
   overlay placement, inherited locale/direction, versioned capability
   reporting, shared native event-source state machines, and accessibility
-  conformance checks.
-  Role-edge input parity, layout-aware collection page movement, IME/dead-key
+  conformance checks. Versioned native-input manifests expand every role marked
+  native into strict OS-automation cases; adapter and headless traces cannot
+  satisfy that evidence gate.
+  A WinUI smoke runner records real XAML Button mouse and keyboard input through
+  `SendInput`, pen and touch through Windows synthetic pointer injection, and
+  assistive activation through UI Automation `InvokePattern`. It covers
+  activation, mouse/pen/touch cancellation, keyed-rerender cancellation, and
+  disabled input for all 14 cases of each button-backed `Button`,
+  `DisclosureSummary`, `Link`, `ImageMapArea`, and `MenuItem` role (70 of 98
+  WinUI cases); `ListBoxItem` and `TreeItem` remain outstanding.
+  Populating the remaining manifest cases with real platform automation,
+  role-edge input parity, layout-aware collection page movement, IME/dead-key
   conformance, measured overlay collision/arrow conformance and scroll locking,
   remaining native focus conformance, and locale formatting are still in
   progress.
@@ -175,6 +185,7 @@ platform-matched typography, for example `w-[396px]`, `rounded-[12px]`, and
 | AppKit native surface | Usable for macOS smoke apps with role-aware press/hover/focus/key translation, controls, menus, close actions, and typed post-mount `autoFocus`. |
 | GTK4 native surface | Usable for Linux smoke apps with role-aware press/hover/focus/key translation, controls, menus, dialogs, and scroll containers. |
 | WinUI native surface | Usable for Windows smoke apps with role-aware press/hover/focus/key translation, typed programmatic focus and post-mount `autoFocus`, core controls, resize bounds, close actions, and root-window exit. |
+| Native input conformance | Versioned requirement/run/report artifacts and a strict verifier are available. The WinUI smoke harness covers all 70 button-backed role cases (70 of 98); full AppKit, GTK4, and WinUI evidence is not complete. |
 | Product app shell | Dogfood-ready. Production distribution still needs signed installers and longer real-world focus/input hardening. |
 
 ## Roadmap
@@ -255,6 +266,25 @@ denied, library and example tests, cross-platform planning adapters, and diff
 whitespace. On macOS, Linux, or Windows, `just native-ci` additionally runs the
 matching native feature's library tests and an all-target compile check.
 
+Generate the exact input cases implied by a backend's native capability claims,
+then verify an artifact emitted by an operating-system automation runner:
+
+```bash
+just native-input-manifest winui
+just native-input-conformance path/to/winui-evidence.json
+just winui-input-smoke path/to/winui-button-backed-smoke.json
+```
+
+The verifier exits nonzero for missing or duplicate cases, incorrect semantic
+event order, modality/target mismatches, incomplete environment identity, or
+evidence produced only by an adapter kernel or the portable runtime.
+The WinUI smoke command opens a real XAML window and emits a partial artifact;
+passing that command validates its 70 captured button-backed role cases, not
+the full WinUI manifest. It requires a supported Windows desktop session with
+the Windows App Runtime 1.7 framework package available. WinUI examples enter
+the XAML-owned application lifecycle with `run_winui_application_staged_async`
+and keep native layout and input responsive through the asynchronous app loop.
+
 The default feature set includes RSX authoring and the built-in design system.
 The runtime/protocol core is independently buildable without SWC or `rsx_ui`:
 

docs/architecture.md

@@ -154,6 +154,46 @@ The application or host owns ACL parsing and validation, then passes typed,
 immutable capability grants across the outer boundary. The GUI core owns the
 capability types but must not depend on the ACL parser or its AST.
 
+## Native Input Evidence Boundary
+
+`NativeCapabilities` describes the behavior a backend claims for each role.
+`NativeInputConformanceManifestV1` turns native press claims into a canonical
+role/scenario matrix, including activation, cancellation, disabled-state, and
+keyed-rerender cases for each applicable input modality. This derivation keeps
+capability declarations and test scope in one source of truth.
+
+An operating-system runner records semantic events with
+`NativeInputConformanceObservationV1::capture` and submits a versioned
+`NativeInputConformanceRunV1`. Capture excludes event values and raw key data.
+The verifier compares observations against the generated manifest, checks
+provenance and environment identity, and produces a serializable
+`NativeInputConformanceReportV1`. Headless and adapter-kernel traces are
+explicitly ineligible to prove a native claim even when their event sequence is
+identical.
+
+File I/O remains outside the GUI core. The
+`a3s-gui-native-input-conformance` binary is a thin adapter that prints current
+manifests and reads evidence JSON for CI. Native backends own the OS automation
+driver and event-loop integration; the portable verifier owns only the shared
+semantic contract.
+
+The Windows-only `a3s-gui-winui-input-smoke` harness keeps automation outside
+the runtime library. It mounts each button-backed semantic role as a real WinUI
+Button, confirms that the mounted native role matches the manifest case,
+locates and verifies its enabled state through the OS UI Automation tree,
+injects mouse and Enter-key input with `SendInput`, injects pen and touch with
+`CreateSyntheticPointerDevice` / `InjectSyntheticPointerInput`, and invokes its
+assistive `InvokePattern`. Mouse cancellation releases the real XAML pointer
+capture after the injected press; pen and touch cancellation inject Windows'
+cancelled-up pointer state. The production WinUI message loop records successful,
+cancelled, keyed-rerender, and disabled responses. The harness validates all 70
+observations for `Button`, `DisclosureSummary`, `Link`, `ImageMapArea`, and
+`MenuItem` with the strict manifest verifier while allowing only `ListBoxItem`
+and `TreeItem` to remain `MissingObservation`. Its output therefore remains an
+explicitly partial run artifact rather than a conformance report. The harness
+requires an interactive Windows desktop and the Windows App Runtime 1.7
+framework package used by the WinUI backend's dynamic dependency.
+
 ## NativeHost Boundary
 
 Every platform adapter implements the same host operations:
@@ -1003,31 +1043,35 @@ Feature-gated platform executor surfaces:
   its native `Switch` semantic in the IR; with `winio-winui3` 0.4.2, the native
   surface temporarily backs that state with a WinUI `CheckBox` because the
   generated bindings do not expose `ToggleSwitch` yet.
-  The WinUI message pump also maps `WM_KEYDOWN`, `WM_KEYUP`,
-  `WM_SYSKEYDOWN`, and `WM_SYSKEYUP` records into A3S keyboard events. Since
-  `winio-winui3` 0.4.2 does not expose strong `KeyRoutedEvent` registration
-  methods, keyboard events target the currently focused A3S node tracked from
-  WinUI focus callbacks, falling back to the root surface for window-level
-  routes. The same binding version leaves programmatic `Focus` unwrapped; the
-  native surface bridges its fixed WinRT ABI from an adjacent public vtable
-  method and still uses WinUI focus callbacks as the observable event source.
+  WinUI pointer handlers opt into handled routed events so Button-backed
+  controls preserve mouse, pen, and touch press phases that XAML class handling
+  would otherwise hide. Preview key-down and key-up handlers likewise feed the
+  shared keyboard state machine before Button activation, avoiding duplicate
+  semantic presses from the later `Click` callback. `winio-winui3` 0.4.2 does
+  not generate these registration methods, so the native surface isolates the
+  fixed WinRT ABI calls in a small adapter. The same binding version leaves
+  programmatic `Focus` unwrapped; that call uses the same isolated ABI approach
+  while WinUI focus callbacks remain the observable event source.
   Window close requests are observed through the HWND message path: the surface
   installs a close-event subclass on each WinUI window and enqueues
   `NativeEventKind::Close` when `WM_CLOSE` arrives. Content dialogs register
   WinUI's `Closing` callback and enqueue the same event for dialog `onClose`
   actions while clearing local open-dialog tracking. Draining those native close
   events releases the retained `ShowAsync` operation; programmatic hides are
   suppressed during render-driven teardown.
-  `WinUiRuntimeApp` provides the embedded app loop for this backend: it renders
-  into `WinUiNativeSurface`, pumps the Windows message queue, drains queued A3S
-  native events, runs the application reducer, rerenders the next frame, and
-  observes the root WinUI window handle so `run_winui_while` can stop when the
-  user closes the surface.
+  WinUI surfaces are created inside the XAML-owned `Application::Start`
+  lifecycle. `run_winui_application_staged_async` renders and activates the
+  first window during `OnLaunched`, then polls the application future through
+  `DispatcherQueue` turns after launch returns. `WinUiRuntimeApp` drains queued
+  A3S native events, runs the reducer, rerenders the next frame, and observes
+  the root window so `run_winui_while_async` can stop when the user closes the
+  surface without blocking XAML layout or input dispatch.
   The `winui_controls` example runs the same shared native controls smoke
   frame against real WinUI widgets.
   The `winui_dogfood` example runs the shared task editor and review workflow
   through the same reducer loop and WinUI event queue, including window close
-  lifecycle actions and state-driven app loop exit through `run_winui_while`.
+  lifecycle actions and state-driven app loop exit through
+  `run_winui_while_async`.
 - `gtk4`: maps classes such as `gtk::Button`, `gtk::Entry`,
   `gtk::SearchEntry`, `gtk::PasswordEntry`, `gtk::SpinButton`, and
   `gtk::TextView` into GTK object kinds behind `Gtk4WidgetDriver` and replays

docs/react-aria-native.md

@@ -305,14 +305,68 @@ conservative and role overrides opt into verified behavior. This prevents a
 native wrapper, menu model, or logical collection item from being advertised as
 interactive merely because another role on that backend is interactive.
 
+## Native Input Evidence Gate
+
+`NativeInputConformanceManifestV1::from_capabilities` expands each role whose
+press support is marked `Native` into a machine-readable automation matrix. A
+complete press-lifecycle role requires separate mouse and pen activation,
+cancellation, and disabled cases, plus keyboard, virtual assistive activation,
+and keyed-rerender cancellation. GTK4 and WinUI also require touch activation,
+cancellation, and disabled-touch cases. AppKit and GTK4 menu items retain an
+explicit terminal-activation exception because their menu models do not expose
+the generic view event source. The expected successful activation order follows
+React Aria's [`usePress`](https://react-aria.adobe.com/usePress) contract:
+press start, press up, press end, then terminal press.
+
+The evidence boundary is intentionally strict:
+
+- `NativeInputConformanceObservationV1::capture` retains only redacted semantic
+  press events; raw key values and native event payload values are excluded.
+- A verifier derives expectations from the current capability manifest rather
+  than accepting expectations supplied by an evidence file.
+- Exactly one observation is required per case. Event order, target identity,
+  modality, keyboard activation deduplication, and click count are checked.
+- Only `OperatingSystemAutomation` evidence with matching OS, OS version,
+  toolkit version, and automation-driver identity is eligible. Adapter-kernel
+  and portable-runtime traces remain useful tests but cannot prove native
+  support.
+- The native CI matrix publishes the generated AppKit, GTK4, and WinUI
+  requirement artifacts so missing platform evidence is explicit and
+  reviewable.
+
+Generate or verify artifacts with:
+
+```bash
+just native-input-manifest appkit
+just native-input-manifest gtk4
+just native-input-manifest winui
+just native-input-conformance path/to/evidence.json
+just winui-input-smoke path/to/winui-button-backed-smoke.json
+```
+
+The WinUI smoke harness covers the XAML Button-backed `Button`,
+`DisclosureSummary`, `Link`, `ImageMapArea`, and `MenuItem` roles. Mouse and
+keyboard use Windows `SendInput`, pen and touch use synthetic pointer injection,
+and assistive activation uses UI Automation `InvokePattern`. It exercises
+successful activation, mouse/pen/touch cancellation, keyed-rerender
+cancellation, and disabled input inside the production XAML application
+lifecycle. Handled routed pointer events and preview key events feed the same
+portable press state machine while the asynchronous dispatcher loop leaves
+WinUI layout and input dispatch unblocked.
+The strict verifier accepts no semantic defect in the complete 70-case slice,
+but the 28 `ListBoxItem` and `TreeItem` WinUI cases remain. The P0 gap is
+therefore still open. Native automation drivers still need to execute every
+generated case and submit passing run artifacts on real macOS, Linux, and
+Windows runners.
+
 ## Known Gaps
 
 The following behavior systems are still incomplete or only represented as
 props:
 
 | Priority | Area | Required outcome |
 | --- | --- | --- |
-| P0 | Native input conformance | Add platform-run pointer, keyboard, assistive activation, and cancellation fixtures for every role currently marked native, then close or document remaining menu/item exceptions. |
+| P0 | Native input conformance | Populate the generated V1 manifests with platform-run mouse, pen, touch where applicable, keyboard, assistive activation, disabled, cancellation, and keyed-rerender fixtures for every role currently marked native; then close or retain evidence-backed menu/item exceptions. |
 | P1 | Event propagation | Add platform-run conformance fixtures for conditional `Stop`/`Continue` across nested native controls. |
 | P1 | Focus management | Add platform-run conformance fixtures for post-mount `autoFocus`, nested containment, and restoration. |
 | P1 | Collections and selection | Add layout-aware page navigation and complete IME/dead-key typeahead conformance. |
@@ -341,3 +395,5 @@ A behavior is complete only when all of the following are true:
    asserted semantically.
 5. The public documentation describes the supported contract without implying
    broader parity.
+6. A real operating-system automation run satisfies the generated versioned
+   manifest; portable or adapter-only tests do not count as native evidence.

docs/roadmap.md

@@ -82,6 +82,13 @@ The following foundations are implemented and are not roadmap placeholders:
   versioned `positionOverlay` commands, recovery replay, and AppKit, GTK4,
   WinUI, and headless projections
 - AppKit, GTK4, and WinUI build/test matrices with real native dogfood surfaces
+- versioned native-input requirement, run, and report artifacts derived from
+  role capabilities; a strict verifier rejects adapter/headless traces as
+  native evidence, and CI publishes each backend's outstanding matrix
+- a Windows OS-input smoke harness that drives real XAML Button-backed
+  `Button`, `DisclosureSummary`, `Link`, `ImageMapArea`, and `MenuItem` roles
+  through `SendInput`, synthetic pen/touch injection, and UI Automation,
+  currently covering all 70 of their cases (70 of 98 required WinUI cases)
 
 ## Delivery Dependency
 
@@ -357,8 +364,9 @@ Target: days 151-180. Status: planned after M5.
   surfaces for menus, dialogs, keyboard input, focus, text input, collection
   selection, table navigation, close, and recovery flows.
 - Generate or validate the documented platform support matrix from backend
-  declarations and real automation results. Placeholder behavior does not count
-  as supported.
+  declarations and real automation results. Feed each platform run through the
+  generated `NativeInputConformanceManifestV1`; placeholder, adapter-kernel, and
+  portable-runtime behavior does not count as supported.
 - Replace thread park/unpark fallbacks with real AppKit, GLib, and WinUI event
   loop wake messages for externally completed work.
 - Add native failure injection for partial commit, teardown, fresh-executor