You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+118-1Lines changed: 118 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,9 +10,12 @@ Self-hosted worker for Oz cloud agents.
10
10
11
11
## Requirements
12
12
13
-
- Docker daemon (accessible via socket or TCP)
14
13
- Service account API key with team scope
15
14
- Network egress to warp-server
15
+
- One supported execution backend:
16
+
- Docker daemon access for the Docker backend
17
+
- Local `oz` CLI access plus a writable workspace root for the Direct backend
18
+
- Kubernetes API access plus cluster credentials for the Kubernetes backend
16
19
17
20
## Usage
18
21
@@ -28,6 +31,118 @@ docker run -v /var/run/docker.sock:/var/run/docker.sock \
28
31
29
32
> **Note:** Mounting the Docker socket gives the container access to the host's Docker daemon. This is required for the worker to create and manage task containers.
30
33
34
+
### Direct
35
+
36
+
The direct backend executes tasks directly on the host instead of inside Docker or Kubernetes. It requires the `oz` CLI to be available on `PATH` (or configured explicitly with `backend.direct.oz_path`) and stores per-task workspaces under `backend.direct.workspace_root` (default: `/var/lib/oz/workspaces`).
37
+
38
+
Example config:
39
+
40
+
```yaml
41
+
worker_id: "my-worker"
42
+
backend:
43
+
direct:
44
+
workspace_root: "/var/lib/oz/workspaces"
45
+
oz_path: "/usr/local/bin/oz"
46
+
```
47
+
48
+
### Kubernetes
49
+
50
+
The Kubernetes backend creates one Job per task. Cluster selection is controlled by the Kubernetes client config:
51
+
52
+
- `backend.kubernetes.kubeconfig` points to an explicit kubeconfig file
53
+
- if `kubeconfig` is omitted, the worker uses in-cluster config when running inside Kubernetes
54
+
- otherwise it falls back to the default kubeconfig loading rules and uses the current context
55
+
56
+
Example config:
57
+
58
+
```yaml
59
+
worker_id: "my-worker"
60
+
backend:
61
+
kubernetes:
62
+
kubeconfig: "/path/to/kubeconfig"
63
+
namespace: "agents"
64
+
unschedulable_timeout: "2m"
65
+
pod_template:
66
+
nodeSelector:
67
+
kubernetes.io/os: linux
68
+
containers:
69
+
- name: task
70
+
resources:
71
+
requests:
72
+
cpu: "2"
73
+
memory: 4Gi
74
+
```
75
+
76
+
Notes:
77
+
78
+
- `namespace`selects the namespace inside the chosen cluster; it does not choose the cluster itself, and defaults to `default` when omitted
79
+
- `unschedulable_timeout`controls how long a Pod may remain unschedulable before the task is failed early; it defaults to `30s`, and `0s` disables that fail-fast behavior
80
+
- `image_pull_policy`defaults to `IfNotPresent`
81
+
- the Kubernetes backend requires creating Pods with a root init container to materialize sidecars into `emptyDir` volumes
82
+
- the worker runs a short-lived startup preflight Job and waits for either preflight Pod creation or an early controller failure, so incompatible Pod Security or admission policy failures surface before the worker starts accepting tasks
83
+
- `preflight_image`defaults to `busybox:1.36`; set it if your cluster only allows pulling startup-preflight images from an internal or allowlisted registry
84
+
- `pod_template`accepts standard Kubernetes PodSpec YAML and is the declarative way to configure task pod scheduling, service accounts, image pull secrets, resources, and environment
85
+
- when using `pod_template`, define a container named `task` if you want to customize the main task container directly; otherwise the worker appends its own `task` container to the PodSpec
86
+
- use `valueFrom.secretKeyRef` inside `pod_template` to inject Kubernetes Secret values into task container environment variables:
87
+
88
+
```yaml
89
+
pod_template:
90
+
containers:
91
+
- name: task
92
+
env:
93
+
- name: MY_SECRET
94
+
valueFrom:
95
+
secretKeyRef:
96
+
name: my-k8s-secret
97
+
key: secret-key
98
+
```
99
+
100
+
101
+
### Helm Chart
102
+
103
+
This repo includes a namespace-scoped Helm chart at `charts/oz-agent-worker`.
104
+
105
+
The chart deploys:
106
+
107
+
- a long-lived `Deployment` for `oz-agent-worker`
108
+
- a namespaced `ServiceAccount`
109
+
- a namespaced `Role` / `RoleBinding`
110
+
- a `ConfigMap` containing the worker config
111
+
- an optional `Secret` for `WARP_API_KEY` (or a reference to an existing `Secret`)
112
+
113
+
At runtime, the deployed worker connects outbound to Warp and creates one Kubernetes `Job` per task. The built-in Kubernetes Job controller then manages the task Pod lifecycle.
The chart assumes the worker runs inside the target cluster and uses in-cluster Kubernetes auth by default. It does not create CRDs or cluster-scoped RBAC. Set `image.tag` explicitly for each install so the worker image is pinned instead of defaulting to `latest`.
130
+
131
+
The chart always deploys a single replica for a given `worker.workerId`. If you want multiple workers, deploy multiple releases with distinct worker IDs rather than scaling one release horizontally.
132
+
133
+
The chart defaults the long-lived worker `Deployment` to a non-root security context and conservative starting resource requests of `100m` CPU and `128Mi` memory. Tune `worker.resources` for your workload and cluster policy.
134
+
135
+
The Deployment includes a default `exec` liveness probe that checks the worker process is still running (`kill -0 1`). If the worker becomes unresponsive, Kubernetes will restart the pod after three consecutive failures. Override `worker.livenessProbe` in your values to use a custom probe (e.g. `httpGet` if you add a health endpoint), or set it to `null` to disable.
136
+
137
+
Recommended namespace-scoped permissions for the worker are:
138
+
139
+
- create, get, list, watch, delete `jobs`
140
+
- get, list, watch `pods`
141
+
- get `pods/log`
142
+
- list `events`
143
+
144
+
The worker Deployment's `ServiceAccount` is separate from the task Job `serviceAccountName` you may set inside `backend.kubernetes.pod_template` / `kubernetesBackend.podTemplate`. The worker `Deployment` defaults to non-root, but the task namespace must still allow creating Jobs with a root init container, since sidecar materialization currently depends on that pattern. If your cluster restricts image sources for admission or policy reasons, set `kubernetesBackend.preflightImage` in the chart to an allowlisted image for the startup preflight Job, and configure task `imagePullSecrets` inside `podTemplate` when needed.
145
+
31
146
### Go Install
32
147
33
148
```bash
@@ -68,6 +183,8 @@ docker run -v /var/run/docker.sock:/var/run/docker.sock \
When configuring the Kubernetes backend via YAML or Helm, declarative task-container env belongs in `backend.kubernetes.pod_template` / `kubernetesBackend.podTemplate` rather than a separate top-level Kubernetes env list. The `-e` / `--env` flags remain available as backend-agnostic runtime overrides.
187
+
71
188
## Docker Connectivity
72
189
73
190
The worker automatically discovers the Docker daemon using standard Docker client mechanisms, in this order:
0 commit comments