Skip to content

Commit 7f57bed

Browse files
authored
feat(k8s): add dockur windows pool exmaple (opensandbox-group#878)
* feat(k8s): add dockur windows pool exmaple * feat(server): fix windows reboot
1 parent deffb33 commit 7f57bed

11 files changed

Lines changed: 888 additions & 5 deletions

File tree

examples/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ Examples for common OpenSandbox use cases. Each subdirectory contains runnable c
2121
- 🦞 [**nullclaw**](nullclaw): Launch a Nullclaw Gateway inside a sandbox
2222
- 🦞 [**openclaw**](openclaw): Run an OpenClaw Gateway inside a sandbox
2323
- 🖥️ [**desktop**](desktop): Launch VNC desktop (Xvfb + x11vnc) for VNC client connections
24+
- 🪟 [**windows**](windows): Run a Windows guest VM via KVM/QEMU with RDP and web console access
2425
- <img src="https://playwright.dev/img/playwright-logo.svg" alt="Playwright" width="16" height="16" style="display:inline-block;width:16px;height:16px;vertical-align:middle;margin-right:4px;" /> [**playwright**](playwright): Launch headless browser (Playwright + Chromium) to scrape web content
2526
- <img src="https://code.visualstudio.com/assets/favicon.ico" alt="VS Code" width="16" height="16" style="display:inline-block;width:16px;height:16px;vertical-align:middle;margin-right:4px;" /> [**vscode**](vscode): Launch code-server (VS Code Web) to provide browser access
2627
- <img src="https://www.google.com/chrome/static/images/chrome-logo.svg" alt="Google Chrome" width="16" height="16" style="display:inline-block;width:16px;height:16px;vertical-align:middle;margin-right:4px;" /> [**chrome**](chrome): Launch headless Chromium with DevTools port exposed for remote debugging

examples/windows/README.md

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
# Windows Sandbox Example
2+
3+
Run a Windows guest in an OpenSandbox sandbox via KVM/QEMU using the [`dockur/windows`](https://github.com/dockur/windows) image.
4+
5+
## How it works
6+
7+
OpenSandbox creates a Linux container running KVM/QEMU, which boots a Windows guest OS inside it. The Windows profile (`platform.os=windows`) automatically configures the required devices, capabilities, OEM scripts, and port mappings — you only need to specify `platform` and `resource` in the SDK call.
8+
9+
## Prerequisites
10+
11+
- OpenSandbox server running (e.g. `http://localhost:8080`)
12+
- Host with `/dev/kvm` and `/dev/net/tun` present
13+
- Server `storage.allowed_host_paths` configured for any host bind mounts
14+
15+
## Start OpenSandbox server [local]
16+
17+
```shell
18+
uv pip install opensandbox-server
19+
opensandbox-server init-config ~/.sandbox.toml --example docker
20+
opensandbox-server
21+
```
22+
23+
## Run the example
24+
25+
```shell
26+
uv pip install opensandbox
27+
python main.py
28+
```
29+
30+
The script will:
31+
32+
1. Create a Windows sandbox with `dockurr/windows:latest` and Windows 11
33+
2. Wait until the sandbox is healthy (first boot can take several minutes)
34+
3. Print the execd, RDP (3389), and web console (8006) endpoints
35+
4. Execute a test command and print the output
36+
37+
## Environment Variables
38+
39+
- `SANDBOX_DOMAIN`: Sandbox service address (default: `localhost:8080`)
40+
- `SANDBOX_API_KEY`: API key if your server requires authentication (optional for local)
41+
42+
## Customization
43+
44+
### Resource limits
45+
46+
The Windows profile enforces minimum resources: **cpu >= 2, memory >= 4G, disk >= 64G**. The example uses 4 CPU, 8G RAM, and 64G disk. You can adjust these in the `main.py` `resource` dict.
47+
48+
### Persistent storage
49+
50+
Bind a host directory to `/storage` for a persistent system disk (add to the `SandboxSync.create` call):
51+
52+
```python
53+
from opensandbox.models.sandboxes import Host, Volume
54+
55+
volumes = [
56+
Volume(
57+
name="win-storage",
58+
host=Host(path="/data/opensandbox/windows-storage"),
59+
mount_path="/storage",
60+
read_only=False,
61+
),
62+
]
63+
```
64+
65+
### Local ISO
66+
67+
Bind a Windows install ISO to `/boot.iso` to avoid repeated downloads:
68+
69+
```python
70+
volumes = [
71+
Volume(
72+
name="win-iso",
73+
host=Host(path="/data/iso/Win11_23H2.iso"),
74+
mount_path="/boot.iso",
75+
read_only=True,
76+
),
77+
]
78+
```
79+
80+
### Windows guest configuration
81+
82+
Pass [dockur/windows environment variables](https://github.com/dockur/windows) through the `env` parameter:
83+
84+
```python
85+
env = {
86+
"VERSION": "11l",
87+
"USERNAME": "Docker",
88+
"PASSWORD": "your-secure-password",
89+
"LANGUAGE": "Chinese",
90+
"REGION": "zh-CN",
91+
"KEYBOARD": "zh-CN",
92+
}
93+
```
94+
95+
Do not manually set `CPU_CORES`, `RAM_SIZE`, or `DISK_SIZE` — they are derived from `resourceLimits` automatically.
96+
97+
## Exposed ports
98+
99+
| Port | Service |
100+
|------|---------|
101+
| 44772 | execd (sandbox execution API) |
102+
| 8080 | HTTP service |
103+
| 3389 | RDP (native Remote Desktop) |
104+
| 8006 | Web console (noVNC) |
105+
106+
## Troubleshooting
107+
108+
- **`Unsupported platform.os 'windows'`**: Server build has no Windows profile; upgrade OpenSandbox server.
109+
- **`INVALID_PARAMETER` for resourceLimits**: Ensure cpu >= 2, memory >= 4G, disk >= 64G.
110+
- **Stays Pending a long time**: First Windows install is slow; check host resources and `/storage` space, increase `ready_timeout`.
111+
- **Status Running but endpoint unreachable**: Verify endpoint resolution returns a valid address; check `USER_PORTS` if you need additional ports forwarded.
112+
113+
### ENI CNI network issue (Alibaba Cloud ACK)
114+
115+
On clusters using ENI-based CNIs (e.g. Alibaba Cloud ACK Terway in ENI mode), dockur/windows fails at startup with:
116+
117+
```
118+
❯ ERROR: This container does not support host mode networking!
119+
```
120+
121+
or:
122+
123+
```
124+
❯ ERROR: Status 1 while: ethtool -i "$VM_NET_DEV"
125+
```
126+
127+
**Root cause**: The image's `network.sh` uses `ethtool -i` to check the network interface. ENI interfaces have real PCI bus-info, which triggers a false "host mode" detection. Standard veth-based CNIs (Calico, Flannel, Cilium) do NOT have this problem.
128+
129+
**Solution**: Use the provided `main_fix_net.py` example, which patches the script at runtime and sets `NETWORK=slirp` for QEMU user-mode NAT:
130+
131+
```shell
132+
python main_fix_net.py
133+
```
134+
135+
See [`main_fix_net.py`](./main_fix_net.py) for the full implementation.
136+
137+
**How it works**:
138+
139+
1. `sed` replaces three lines in `/run/network.sh` with empty variable assignments (`result=""`, `nic=""`, `bus=""`), preventing the ethtool check from aborting the script.
140+
2. `NETWORK=slirp` tells the script to use QEMU's SLIRP networking (user-mode NAT), which doesn't require a real NIC.
141+
3. `exec /usr/bin/tini -s /run/entry.sh` launches the original image entrypoint after patching.
142+
143+
This approach keeps the Pod's independent IP and requires no image rebuild or `hostNetwork`.
144+
145+
## Windows Sandbox from pool
146+
147+
Use a pre-warmed K8s pool for faster Windows sandbox startup.
148+
149+
### 1. Create the pool
150+
151+
Apply the pool manifest (the image, resources, device mounts, and OEM scripts are pre-configured):
152+
153+
```shell
154+
kubectl apply -f pool-win-example.yaml
155+
```
156+
157+
### 2. Start the OpenSandbox server [k8s]
158+
159+
```shell
160+
uv pip install opensandbox-server
161+
opensandbox-server init-config ~/.sandbox.toml --example k8s
162+
opensandbox-server
163+
```
164+
165+
### 3. Run the pool example
166+
167+
```shell
168+
uv pip install opensandbox
169+
python main_use_pool.py
170+
```
171+
172+
The script acquires a sandbox from `pool-win-example`, prints endpoints, and runs a command.
173+
174+
### Environment variables (pool)
175+
176+
- `SANDBOX_DOMAIN`: Sandbox service address (default: `localhost:8080`)
177+
- `SANDBOX_API_KEY`: API key if your server requires authentication
178+
179+
## References
180+
181+
- [Windows sandbox guide](../../docs/windows-sandbox.md)
182+
- [dockur/windows](https://github.com/dockur/windows)

examples/windows/main.py

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Copyright 2026 Alibaba Group Holding Ltd.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""Minimal Windows sandbox example using dockur/windows."""
16+
17+
import os
18+
from datetime import timedelta
19+
20+
from opensandbox import SandboxSync
21+
from opensandbox.config import ConnectionConfigSync
22+
from opensandbox.models.sandboxes import PlatformSpec
23+
24+
25+
def main() -> None:
26+
cfg = ConnectionConfigSync(
27+
domain=os.getenv("SANDBOX_DOMAIN", "localhost:8080"),
28+
api_key=os.getenv("SANDBOX_API_KEY") or None,
29+
request_timeout=timedelta(minutes=3),
30+
use_server_proxy=True,
31+
)
32+
33+
sbx = SandboxSync.create(
34+
image="dockurr/windows:latest",
35+
timeout=timedelta(hours=12),
36+
ready_timeout=timedelta(minutes=30),
37+
resource={
38+
"cpu": "4",
39+
"memory": "8G",
40+
"disk": "64G",
41+
},
42+
env={"VERSION": "11"},
43+
platform=PlatformSpec(os="windows", arch="amd64"),
44+
connection_config=cfg,
45+
)
46+
47+
try:
48+
print(f"Created: {sbx.id}")
49+
print(f"execd: {sbx.get_endpoint(44772).endpoint}")
50+
print(f"RDP: {sbx.get_endpoint(3389).endpoint}")
51+
print(f"Web: {sbx.get_endpoint(8006).endpoint}")
52+
53+
exec = sbx.commands.run("cmd /c echo Hello from Windows sandbox")
54+
print(f"Command output: {exec.logs.stdout[0].text}")
55+
finally:
56+
sbx.kill()
57+
sbx.close()
58+
59+
60+
if __name__ == "__main__":
61+
main()

examples/windows/main_fix_net.py

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# Copyright 2026 Alibaba Group Holding Ltd.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
"""
16+
Windows sandbox example with ENI CNI network fix.
17+
18+
Use this example on clusters with ENI-based CNIs (e.g. Alibaba Cloud ACK
19+
Terway in ENI mode) where dockur/windows fails with:
20+
21+
ERROR: This container does not support host mode networking!
22+
23+
or:
24+
25+
ERROR: Status 1 while: ethtool -i "$VM_NET_DEV"
26+
27+
The fix patches /run/network.sh at container startup to bypass the
28+
ethtool/bus-info check, then uses NETWORK=slirp for QEMU user-mode NAT.
29+
Standard veth-based CNIs (Calico, Flannel, Cilium) do NOT need this fix.
30+
"""
31+
32+
import os
33+
from datetime import timedelta
34+
35+
from opensandbox import SandboxSync
36+
from opensandbox.config import ConnectionConfigSync
37+
from opensandbox.models.sandboxes import PlatformSpec
38+
39+
# sed command to bypass the ethtool/grep checks in network.sh.
40+
# Replaces three lines with empty variable assignments so that:
41+
# - ethtool -i (would fail on ENI with real PCI bus-info) is skipped
42+
# - grep on empty result (would fail with pipefail) is skipped
43+
_NETWORK_PATCH_CMD = (
44+
"sed -i"
45+
" -e 's/result=$(ethtool -i \"$VM_NET_DEV\")/result=\"\"/'"
46+
" -e '/grep.*driver:/s/.*/ nic=\"\"/'"
47+
" -e '/grep.*bus-info:/s/.*/ bus=\"\"/'"
48+
" /run/network.sh"
49+
)
50+
51+
# Original dockur/windows ENTRYPOINT
52+
_WINDOWS_ENTRYPOINT = "/usr/bin/tini -s /run/entry.sh"
53+
54+
55+
def main() -> None:
56+
cfg = ConnectionConfigSync(
57+
domain=os.getenv("SANDBOX_DOMAIN", "localhost:8080"),
58+
api_key=os.getenv("SANDBOX_API_KEY") or None,
59+
request_timeout=timedelta(minutes=3),
60+
use_server_proxy=True,
61+
)
62+
63+
sbx = SandboxSync.create(
64+
image="dockurr/windows:latest",
65+
timeout=timedelta(hours=12),
66+
ready_timeout=timedelta(minutes=120),
67+
resource={"cpu": "8", "memory": "16G", "disk": "64G"},
68+
env={
69+
"VERSION": "11",
70+
"NETWORK": "slirp", # Use QEMU built-in user-mode NAT
71+
},
72+
# Patch network.sh then exec the original entrypoint
73+
entrypoint=["/bin/sh", "-c", f"{_NETWORK_PATCH_CMD} && exec {_WINDOWS_ENTRYPOINT}"],
74+
platform=PlatformSpec(os="windows", arch="amd64"),
75+
connection_config=cfg,
76+
)
77+
78+
try:
79+
print(f"Created: {sbx.id}")
80+
print(f"execd: {sbx.get_endpoint(44772).endpoint}")
81+
print(f"RDP: {sbx.get_endpoint(3389).endpoint}")
82+
print(f"Web: {sbx.get_endpoint(8006).endpoint}")
83+
84+
result = sbx.commands.run("cmd /c echo Hello from Windows sandbox")
85+
print(f"Command output: {result.logs.stdout[0].text}")
86+
finally:
87+
sbx.kill()
88+
sbx.close()
89+
90+
91+
if __name__ == "__main__":
92+
main()

0 commit comments

Comments
 (0)