fix: harden Windows WHPX and E2B pause lifecycle#134
Merged
Conversation
This was referenced Jul 21, 2026
added 11 commits
July 22, 2026 05:16
ZhiXiao-Lin
marked this pull request as ready for review
July 22, 2026 15:28
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary - Stabilize native Windows WHPX execution across exit handling, logs, ports, bind mounts, stopped-box commits, persistent rootfs recovery, snapshots, and lifecycle concurrency. - Fail closed on unsupported or unsafe requests, including Windows health checks/bridge networking/SMP, stale snapshot policy, unsafe live host-path snapshots, conflicting lifecycle operations, and non-Unix session/content APIs. - Harden OCI, cache, archive, rootfs metadata, and commit boundaries against traversal, malformed digests, symlinks/reparse points, special files, metadata mismatches, and TOCTOU races. - Make libkrun firmware/source builds, release archives, WinGet publication, and RuntimeClass installation reproducible and checksum-verified. - Add deterministic pinned E2B preview compatibility across Linux and Windows hosts: LF-stable contracts, official Python/TypeScript client fixtures, Windows executable resolution, and a no-symlink-privilege TypeScript harness fallback. - Implement ordered E2B Process
StreamInputwith incremental bounded Connect framing, ordered backend writes, keepalives, process reselection, and malformed-frame rejection. - Deliver pinned E2BSIGNAL_SIGTERMandSIGNAL_SIGKILLacross the broker, shared execution sessions, Exec/PTY transports, and Linux guest process groups, retaining cancel compatibility for SIGKILL and signal-derived exit codes. - Implement durable E2B filesystem-only pause (pause(memory=false)) with rootfs preservation, fresh runtime generations, EnvD reinitialization, lifecycle locking, idempotent snapshot cleanup, and crash recovery. - Build the E2B runtime image on relevant pull requests without logging in to GHCR or publishing an image; keep publication limited to trusted push/manual events. - Keep guest mount parsing platform-independent with OCI/POSIX semantics, while allowing non-Unix volume control-plane setup and rejecting unsupported content operations explicitly. - Document the validated native WHPX and E2B support boundaries and source/release requirements. ## Root causes and impact Windows-specific gaps allowed final guest results to be lost, persistent data to be removed after a failed restart, snapshot policy to be dropped, and concurrent lifecycle commands to overwrite replacement PIDs or leave orphan VMs. Guest-controlled filesystem state also crossed host trust boundaries without uniform no-follow, confinement, durability, and lifecycle fencing. OCI image manifests were re-serialized after registry resolution, so valid descriptor digests could point at different bytes. The pull path now stores and parses the exact digest-verified registry payload. CRI test layouts now use canonical, content-derived SHA-256 descriptors so the hardened validation is exercised rather than bypassed by placeholders. The E2B fixture harness previously inherited CRLF and Windows command/symlink behavior from the host. Contracts are now LF-stable, npm/tsc are resolved to executable paths, and temporary pinned dependencies are copied when ordinary Windows users cannot create directory symlinks. Non-Unix filesystem content access remains fail-closed because descriptor-relative safety requires Unix APIs. This PR claims the documented pinned preview subset, not full E2B compatibility. Filesystem-only pause was previously rejected at the compatibility boundary and had no durable pause-mode/recovery path. Cold resume and restart crash windows could lose the retained rootfs, misreport transitional state, or publish a replacement runtime before EnvD was initialized. Pause mode and transitions are now persisted; lifecycle operations are serialized; retained rootfs state is reconciled idempotently; and replacement runtimes are published only after EnvD initialization succeeds. The supported single-vCPU WHPX path is deterministic, persistent data survives failed restarts, policies round-trip through snapshots, stopped-rootfs traversal is serialized, and unsafe or unsupported operations are rejected before side effects. The corresponding libkrun WHPX and firmware work is tracked in A3S-Lab/libkrun#4. ## Testing -cargo fmt --all -- --checkand strict workspace Clippy in CI. -cargo check -p a3s-box-runtime -p a3s-box-compat --all-targets. - Core library tests: 461 passed. - Runtime library tests: 1,149 passed, 1 ignored. - Guest-init library tests: 102 passed. - Shim tests passed. - CLI library tests: 770 passed, plus all non-ignored integration suites. - Compatibility library tests: 132 passed; the checked-in fixture regeneration test was filtered locally because this host has noprotoc. - E2B contract verification and official Python sync/async plus TypeScript client lifecycle fixtures. The newly added filesystem-only pause assertions still require the next certified A3S OS host run. - Python and TypeScript SDK package build/test checks; local Python production-harness tests: 2 passed. - Containerd shim tests and RuntimeClass installer checks. - Cross-platform build checks for Linux x86_64, Linux arm64, and macOS arm64. - Windows WHPX native binary build, bundle verification, and artifact upload. - Current-head real WHPX validation on Windows, using a regular-file-only static rootfs because the host lacks symlink privilege: - #129: detached and create/start logs were visible while running; unterminated final stdout/stderr appeared exactly once; foreground and recorded exit status were both7. - #130: the published port served two sequential requests on the same mapping in each of two independent runs. - #131: drive-letter directory and single-file read-only bind mounts were readable in the guest and rejected writes withEROFS, without modifying the host. - #132: a filesystem snapshot restored with no OCI metadata, retained the original command, observed the pre-snapshot marker, and started successfully. - Final current-head WHPX regression pass covered lifecycle/log collection, persistent commit and snapshot restore, repeated published-port requests, drive-letter bind mounts, graceful stop, forced cleanup, and an exact process-name check confirming no residuala3s-box-shim.exe. - Isolation selection on the same host defaulted deterministically tomicrovm; explicit unsupported isolation requests failed before persisting a box, with no backend fallback. - The conditional real-microVM Linux KVM job is intentionally limited to trusted push/tag/manual events and requires an enrolled runner plusKVM_CI=true; neither is configured for this repository, so it remains an external host gate rather than a PR result. ## Managed CLI lifecycle - Route managed CLI pause, unpause, stop, terminating signals, and forced removal through the canonical execution manager; persist signal/timeout intent so crash recovery replays the same termination policy. Fixes #129 Fixes #130 Fixes #131 Fixes #132 ## Ordered process input - Implement the pinned JSON client-streaming Process contract with bounded per-envelope decoding, arbitrary transport fragmentation, ordered stdin/PTY writes, upstream-compatible keepalives and process reselection, and explicit truncated, invalid-flag, oversized, or malformed request failures. ## Pinned process signals - Map the pinned enum names and numeric protobuf JSON forms for SIGTERM and SIGKILL after generation-fenced process lookup, matching upstream lookup/error ordering. - Deliver SIGTERM to Exec and PTY process groups without closing the control stream; retain the established hard-cancel wire behavior for SIGKILL. - Cover broker routing, missing/unsupported/default enum behavior, Exec and PTY wire frames, guest parsing, Unix process-group delivery, and signal-derived exit status. Production official-client signal validation remains a certified-host gate. ## Protobuf Process Connect - Accept pinned unaryapplication/protoand streamingapplication/connect+protorequests in Broker/MicroVM mode while retaining JSON behavior. - Preserve the selected encoding for successful unary and streaming messages, emit protocol-mandated JSON EndStream/error envelopes, and reject unsupported media types with HTTP 415. - Cover all eight pinned Process procedures, raw non-UTF-8 stdio, fragmented and oversized frames, malformed protobuf, enum/bytes/sint32 mappings, and independent golden wire tags. Runtime-envd Sandbox mode continues to proxy upstream envd directly. ## Brokered envd file transfer - Route generation-fenced Broker/MicroVMGET /filesand raw or multipartPOST /filesthroughExecutionSessionManager, with binary-safe bodies, selected-user POSIX paths, parent creation/ownership, and envd numeric errors. - Add a discriminated guest-session file envelope so file requests cannot be parsed as legacy bare JSON exec requests; cover core wire shape, runtime framing, guest binary round trips, generation forwarding, raw upload, multi-file multipart, and errors. - Keep the bounded transfer scope explicit: 11 MiB per file; compression, ranges, signed URLs, and large-file streaming remain gates, andfull_compatibility=falseis unchanged.Brokered Filesystem RPCs
full_compatibility=falseremains unchanged.