Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .github/scripts/native-linux-smoke.sh
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ run_multi_container_smoke() {
fi
jq --exit-status \
--argjson expected "$expected_kvm_present" \
'.schema_version == "a3s.oci.native-linux-multi-container-smoke.v2"
'.schema_version == "a3s.oci.native-linux-multi-container-smoke.v3"
and .platform == "linux" and .status == "available"
and .kvm_device_present == $expected
and .bundles_loaded
Expand Down Expand Up @@ -235,6 +235,14 @@ run_multi_container_smoke() {
and .lifecycle.wait_b_replayed
and .lifecycle.delete_b_replayed
and .lifecycle.b_missing_after_delete
and (.namespace_join.donor_pid > 0)
and .namespace_join.wrong_type_rejected_before_state
and .namespace_join.joined_non_mount_namespaces
and .namespace_join.joined_pid_time_workload_verified
and .namespace_join.joined_mount_namespace
and .namespace_join.retained_rootfs_verified
and .namespace_join.donor_unchanged_after_joins
and .namespace_join.all_state_removed
and .markers_removed
and .executor_runtime_clean
and .session_root_clean
Expand Down
12 changes: 10 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -496,7 +496,7 @@ jobs:
if [[ "$support" == "1" ]]; then
test "$status" -eq 0
jq --exit-status \
'.schema_version == "a3s.oci.oci-vm-multi-container-smoke.v2"
'.schema_version == "a3s.oci.oci-vm-multi-container-smoke.v3"
and .platform == "macos" and .status == "available"
and .bundles_loaded
and .lifecycle.distinct_bundle_directories
Expand Down Expand Up @@ -541,6 +541,14 @@ jobs:
and .lifecycle.wait_b_replayed
and .lifecycle.delete_b_replayed
and .lifecycle.b_missing_after_delete
and (.namespace_join.donor_pid > 0)
and .namespace_join.wrong_type_rejected_before_state
and .namespace_join.joined_non_mount_namespaces
and .namespace_join.joined_pid_time_workload_verified
and .namespace_join.joined_mount_namespace
and .namespace_join.retained_rootfs_verified
and .namespace_join.donor_unchanged_after_joins
and .namespace_join.all_state_removed
and .markers_removed and .guest_runtime_clean
and .bridge.platform == "macos"
and .bridge.status == "available"
Expand All @@ -563,7 +571,7 @@ jobs:
else
test "$status" -eq 2
jq --exit-status \
'.schema_version == "a3s.oci.oci-vm-multi-container-smoke.v2"
'.schema_version == "a3s.oci.oci-vm-multi-container-smoke.v3"
and .platform == "macos" and .status == "unavailable"
and .bundles_loaded
and .bridge.platform == "macos"
Expand Down
130 changes: 79 additions & 51 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
<a href="#runtime-model">Runtime Model</a> •
<a href="#platform-status">Platform Status</a> •
<a href="#architecture">Architecture</a> •
<a href="#conformance-and-security">Conformance</a> •
<a href="#development">Development</a>
</p>

Expand Down Expand Up @@ -350,12 +351,13 @@ target/debug/a3s-oci oci-vm-multi-container-smoke \
--console "$rootfs_dir/oci-multi-container.log"
```

`a3s.oci.oci-vm-multi-container-smoke.v2` requires that starting, killing,
`a3s.oci.oci-vm-multi-container-smoke.v3` requires that starting, killing,
waiting for, and deleting A never changes or blocks B; both waits return and
replay the exact normal exit status; recreating A advances generation 1 to 2;
stale and cross-container replay requests fail; B then completes
independently; and VM shutdown restores guest-runtime, endpoint, descriptor,
shim, and worker inventories.
independently; existing namespace descriptors are type-checked and joined
across the shared executor; and VM shutdown restores guest-runtime, endpoint,
descriptor, shim, and worker inventories.

Fault cleanup reuses the same signed shim and bundle but stops after each
successful lifecycle boundary:
Expand Down Expand Up @@ -433,6 +435,8 @@ The current executor implements a reviewed bootstrap vertical slice:
- parent-authenticated rootful UID/GID mappings plus read-back verification,
with normalized monotonic and boottime offsets applied before the first
time-namespace child;
- type-checked joins for existing UTS, mount, IPC, network, cgroup, PID, user,
and time namespaces, including retained rootfs access after a mount join;
- hostname and domain name configuration;
- recursively private mount propagation and `pivot_root`;
- ordered existing-target OCI mounts with bind/rbind and common VFS options;
Expand All @@ -443,8 +447,9 @@ The current executor implements a reviewed bootstrap vertical slice:

The supported user-namespace slice is rootful: it requires both UID and GID
mappings, coverage for every configured process ID, and an `allow` setgroups
policy. Unimplemented OCI fields are rejected instead of ignored. Rootless
mapping policy, namespace joins, complete mount semantics, cgroup resources,
policy. Mount entries remain unsupported when joining an existing mount
namespace. Other unimplemented OCI fields are rejected instead of ignored.
Rootless mapping policy, complete mount semantics, cgroup resources,
capabilities, hooks, seccomp, full I/O, recovery, and the remaining SDK
operations are still release gates.

Expand Down Expand Up @@ -473,9 +478,9 @@ boundary.

| Host | Execution path | Retained evidence | Current readiness |
| --- | --- | --- | --- |
| Linux x86_64/aarch64 | Native Linux executor | Kernel pidfd signaling probe, real rootful lifecycle with exact SIGKILL status and repeated wait, two-container wait and mutation isolation, plus shutdown cleanup after create, start, and kill without delete; `/dev/kvm` absent and present-but-unusable | Default inventory `probe-only`; explicitly opened development instance `experimental` |
| Linux x86_64/aarch64 | Native Linux executor | Kernel pidfd signaling probe, real rootful lifecycle with exact SIGKILL status and repeated wait, two-container isolation, type-checked existing-namespace joins, plus shutdown cleanup after create, start, and kill without delete; `/dev/kvm` absent and present-but-unusable | Default inventory `probe-only`; explicitly opened development instance `experimental` |
| Linux x86_64/aarch64 | libkrun + KVM utility VM | Device access, ioctl result, and KVM API version | `probe-only`; VM driver not implemented |
| macOS arm64 | libkrun + HVF utility VM | Direct HVF VM create/destroy, checksum-pinned context lifecycle, authenticated protocol-v2 arm64 guest agent, pidfd-backed fixed and two-container OCI lifecycles with exact repeated exit status and nonblocking wait evidence, and no-delete cleanup after create, start, and kill | `probe-only`; complete enforcement and recovery pending |
| macOS arm64 | libkrun + HVF utility VM | Direct HVF VM create/destroy, checksum-pinned context lifecycle, authenticated protocol-v2 arm64 guest agent, pidfd-backed fixed and two-container OCI lifecycles, type-checked existing-namespace joins, exact repeated exit status and nonblocking wait evidence, and no-delete cleanup after create, start, and kill | `probe-only`; complete enforcement and recovery pending |
| Windows x86_64 | libkrun + WHPX utility VM | Partition, context, guest command, authenticated agent, and fixed OCI core lifecycle | `probe-only`; complete enforcement and recovery pending |

Linux installation, feature inspection, and the native SDK path must work when
Expand All @@ -484,47 +489,69 @@ runtime prerequisite.

## Architecture

The platform-neutral control plane is independent of the host isolation
mechanism. Native libraries and Linux-specific execution stay behind explicit
driver, shim, and guest-agent boundaries:

```text
A3S Box / a3s-oci CLI / Rust SDK consumers
RuntimeClient / OciRuntimeService
in-process call or bounded local IPC
OCI validation and lifecycle
HostRuntimeService
exact bundle · generations · journal · reconciliation
RuntimeDriver
┌─────────┴──────────┐
│ │
Native Linux host Utility VM qualification
NativeLinuxDriver a3s-oci-krun-shim → libkrun
│ KVM · HVF · WHPX
│ │
│ authenticated guest protocol
│ │
│ a3s-oci-agent
│ │
└─────────┬──────────┘
LinuxExecutor
namespaces · mounts · PID 1 · pidfds · wait · cleanup
The public contract is separated from replaceable infrastructure. The SDK and
lifecycle control plane remain platform-neutral, while native isolation and
hypervisor libraries stay behind explicit driver, shim, and guest boundaries:

```mermaid
flowchart TB
subgraph consumers["Consumers and SDK"]
box["A3S Box"]
cli["a3s-oci CLI"]
containerd["containerd shim<br/>(planned)"]
client["RuntimeClient<br/>in-process service or bounded local IPC"]
box --> client
cli --> client
containerd -.-> client
end

subgraph control["Platform-neutral host control plane"]
service["OciRuntimeService<br/>HostRuntimeService"]
validation["OCI schema and semantic validation"]
lifecycle["Durable lifecycle<br/>generations · replay · fencing · reconciliation"]
state[("Runtime-owned state<br/>exact config · operation journal")]
selection{"RuntimeDriver<br/>explicit isolation selection"}

service --> validation --> lifecycle --> selection
lifecycle <--> state
end

subgraph native["Native Linux — experimental opt-in"]
native_driver["NativeLinuxDriver<br/>shared host kernel"]
end

subgraph utility["Utility VM — qualification path"]
utility_driver["Utility VM RuntimeDriver<br/>(integration pending)"]
shim["a3s-oci-krun-shim<br/>checksum-pinned native loading"]
hypervisor["libkrun<br/>HVF · WHPX<br/>KVM probe only"]
bridge["Authenticated host/guest bridge<br/>Unix socket or named pipe → AF_VSOCK"]
agent["a3s-oci-agent<br/>static Linux guest service"]

utility_driver -.-> shim
shim --> hypervisor --> bridge --> agent
end

executor["Shared LinuxExecutor<br/>new and joined namespaces · mounts · create/start barrier<br/>PID 1 · pidfds · wait · scoped cleanup"]

client --> service
selection -->|"shared-host-kernel"| native_driver
selection -.->|"dedicated-vm / shared-guest-kernel"| utility_driver
native_driver --> executor
agent --> executor
```

The same `LinuxExecutor` is called directly on Linux and through the guest
agent in a utility VM. The utility-VM branch represents the qualification
architecture; readiness remains defined by the
[platform status](#platform-status), not by presence in the diagram.
The two paths compile and place the same `LinuxExecutor` differently: directly
behind `NativeLinuxDriver` on Linux, or inside `a3s-oci-agent` in a utility VM.
The utility-VM driver edge is dashed because it is a qualification path and is
not yet wired into `HostRuntimeService`. Solid edges show implemented or
directly exercised boundaries, not `supported` readiness; the
[platform status](#platform-status) remains authoritative.

A3S Box owns product-level images, builds, volumes, networks, and policy. A3S
OCI Runtime owns the validated OCI lifecycle, platform execution, durable
state, guest protocol, and runtime-scoped cleanup.
| Boundary | Owns | Deliberately leaves outside |
| --- | --- | --- |
| A3S Box product plane | Images, builds, volumes, networks, and product policy | OCI process and isolation enforcement |
| OCI Runtime control plane | Exact OCI validation, lifecycle state, replay, reconciliation, capability reporting, and driver selection | Silent isolation fallback and product policy |
| Platform execution plane | Hypervisor bridge, Linux namespaces and mounts, PID 1 lifecycle, signaling, and runtime-scoped cleanup | Image distribution and workload orchestration |

The main runtime, CLI, and SDK do not link libkrun. Only
`a3s-oci-krun-shim` loads the checksum-verified native runtime bundle, keeping
Expand Down Expand Up @@ -597,6 +624,7 @@ Run checks from the OCI Runtime repository root:
cargo fmt --all -- --check
cargo test --workspace --all-targets
cargo clippy --workspace --all-targets -- -D warnings
RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps
```

Cross-check Linux compilation without treating the monorepo root as a Rust
Expand All @@ -615,13 +643,13 @@ Platform CI covers:

- the 237-point durable commit matrix and all 14 `RuntimeDriver` call
boundaries on Linux, macOS, and Windows;
- Ubuntu x86_64 native pidfd probe, lifecycle, multi-container isolation, and
three-phase no-delete cleanup without KVM;
- Ubuntu aarch64 native pidfd probe, lifecycle, multi-container isolation, and
three-phase no-delete cleanup without KVM;
- Ubuntu x86_64 native pidfd probe, lifecycle, multi-container and
existing-namespace isolation, and three-phase no-delete cleanup without KVM;
- Ubuntu aarch64 native pidfd probe, lifecycle, multi-container and
existing-namespace isolation, and three-phase no-delete cleanup without KVM;
- macOS HVF, isolated libkrun context, guest-marker, authenticated-agent,
pidfd-backed fixed and multi-container OCI lifecycles, three-phase no-delete
cleanup, and missing-entitlement fail-closed gates;
pidfd-backed fixed, multi-container, and existing-namespace OCI lifecycles,
three-phase no-delete cleanup, and missing-entitlement fail-closed gates;
- Windows WHPX and libkrun context gates;
- static x86_64 and aarch64 musl guest-agent output.

Expand Down
10 changes: 9 additions & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,10 @@ Completed:
- real native Linux create/state/start/kill/wait/delete SDK evidence on x86_64
and aarch64, including exact repeated SIGKILL status and bounded running
wait, repeated with `/dev/kvm` absent and present but unusable;
- type-checked joins for existing UTS, mount, IPC, network, cgroup, PID, user,
and time namespaces, including retained rootfs execution after a mount join,
three-pass user-namespace permission recovery, and shared native Linux/macOS
utility-VM lifecycle evidence;
- real native Linux no-delete cleanup after create, start, and kill on x86_64
and aarch64, including init-PID reap and executor, durable-state, marker, and
session-root removal;
Expand Down Expand Up @@ -305,7 +309,11 @@ then may HVF become `experimental`.
UID/GID mappings through the authenticated parent, apply and verify
monotonic/boottime offsets before the first child, and prove the path through
native Linux and the macOS utility VM.
- [ ] Join existing namespaces.
- [x] Open and type-check all existing namespace descriptors before mutation,
join non-user namespaces around the user-namespace capability transition,
preserve PID/time next-child semantics, and prove UTS, mount, IPC, network,
cgroup, PID, user, and time joins through native Linux and the macOS
utility-VM path.
- [ ] Mount-target creation, rootfs propagation overrides, idmapped and
recursive-attribute mounts, masked paths, read-only paths, and read-only
rootfs.
Expand Down
Loading
Loading