You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(agent-workflows): rescope runner self-hosting cleanup to v2
Adopt owner review round 1: defer the capability endpoint, the bootstrap
manifest, and fail-loud mounts; add the hosting-layout workstream, per-page
docs dispositions, and the agenta_cloud migration delta.
Claude-Session: https://claude.ai/code/session_01XhENr63WL9npkKrJGnzDc1
Copy file name to clipboardExpand all lines: docs/design/agent-workflows/projects/runner-selfhosting-cleanup/README.md
+16-13Lines changed: 16 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,10 @@
1
1
# Runner self-hosting cleanup
2
2
3
-
Status: DESIGNED, awaiting owner review
3
+
Status: DESIGNED v2 after owner review round 1, awaiting second-round review
4
4
Date: 2026-07-13
5
5
Target branch: `big-agents`
6
6
7
-
This project makes the agent runner understandable from its deployment configuration alone. It replaces shared and overlapping environment variables with one runner-owned provider contract, removes automatic subscription credential upload, adds explicit bootstrap assets, and makes required mounts fail loudly.
7
+
This project makes the agent runner understandable from its deployment configuration alone. It replaces shared and overlapping environment variables with one runner-owned provider contract, removes automatic subscription credential upload, and documents explicit operator-owned subscription mounts. A capability endpoint, a bootstrap manifest, and fail-loud mount semantics are deferred open issues, not release work.
8
8
9
9
## Outcome
10
10
@@ -23,23 +23,25 @@ The normal deployment uses one runner service. That runner can support both loca
23
23
24
24
- The runner is a deployment-level service, not one sidecar per harness or sandbox provider.
25
25
- The operator enables sandbox providers explicitly. Unset means local only, not every provider known to the binary.
26
+
- The enabled-provider list is also the deployment security posture: `daytona` only means no user code ever runs inside the runner container. That is the right posture for any multi-tenant or exposed deployment.
27
+
- API and web get provider availability from the same environment value, set once per deployment by the hosting templates. A discovery endpoint is deferred (RSH-7).
26
28
- Daytona configuration belongs under the runner namespace. Code-evaluator Daytona configuration is separate.
27
29
- The runner never discovers an operator's home directory or automatically uploads Pi or Claude OAuth state.
28
-
- Subscription authentication is an explicit local self-hosting setuptaught in a tutorial.
29
-
-Bootstrap assets are declarative, typed, and implemented by both local and Daytona adapters. Version 1 does not execute arbitrary hooks.
30
+
- Subscription authentication is an explicit local self-hosting setup: a read-only Compose volume mount taught in a tutorial. There is no bootstrap manifest in version 1 (RSH-4).
31
+
-Runtime customization (extra binaries, certificates, dependencies) happens through operator-built images and the shipped Daytona snapshot scripts, not runner configuration.
30
32
- Remote subscription authentication is unsupported until the relevant provider explicitly approves the integration and Agenta defines a safe product contract.
31
33
- Pi installation is detected and repaired by the runtime. There is no deployment-wide "Pi is installed" boolean.
32
-
-Durable mounts required by a session or workflow artifact fail the run when they cannot be signed or mounted. Sessionless runs remain ephemeral.
34
+
-Mount failure behavior stays best-effort in version 1, with one structured warning when a durable mount degrades. The fail-loud contract is deferred (RSH-11).
33
35
- The runner receives a narrow environment and no static Agenta API key. Per-run caller credentials authorize callbacks and trace export. It does not inherit a shared application environment or database and cryptographic secrets.
34
36
35
37
## Reading order
36
38
37
39
-[Context](./context.md) explains today's architecture and why the current variables are confusing.
38
40
-[Research](./research.md) records repository evidence and the review of PRs #5274, #5285, and #5286.
39
-
-[Interface](./interface.md) defines the target environment, Helm, capability, bootstrap, and failure contracts.
41
+
-[Interface](./interface.md) defines the target environment, Helm, provider-availability, subscription-mount, and failure contracts.
40
42
-[Plan](./plan.md) splits the cleanup into reviewable implementation PRs.
41
43
-[QA](./qa.md) defines the environment, harness, and credential matrix.
42
-
-[Documentation plan](./documentation-plan.md) maps the final public docs through Diátaxis.
44
+
-[Documentation plan](./documentation-plan.md) maps the final public docs through Diátaxis, with per-page dispositions for the existing self-host tree.
43
45
-[Open issues](./open-issues.md) parks work that should not block the cleanup.
44
46
-[Status](./status.md) is the owner review checklist and progress log.
45
47
@@ -48,20 +50,21 @@ The normal deployment uses one runner service. That runner can support both loca
48
50
- Runner provider selection and configuration.
49
51
- Runner-specific Daytona names and validation.
50
52
- Runner-to-Services routing and authentication.
51
-
- Declarative bootstrap assets for local and Daytona environments.
52
-
- Local subscription setup.
53
+
- Local subscription setup through explicit read-only mounts.
53
54
- Removal of automatic Daytona OAuth upload.
54
55
- Removal of deployment-wide runtime facts and hidden policy switches.
55
-
-Required-mount failure behavior.
56
-
- Compose, Helm, Railway, self-hosting docs, and the local 8280 QA deployment.
57
-
-Capability discovery needed to keep the API and UI honest.
56
+
-A structured warning when a durable mount degrades.
57
+
- Compose, Helm, Railway, self-hosting docs, a `hosting/README.md` layout map, and the local 8280 QA deployment.
58
+
-One shared provider-availability value read by API and runner.
58
59
59
60
## Out of scope
60
61
61
62
- Implementing the Docker or E2B providers.
62
63
- Sharing a personal subscription between users or tenants.
63
64
- Remote subscription support.
64
-
- Arbitrary bootstrap scripts, VPN hooks, or network overlays.
65
+
- A runner capability-discovery endpoint (RSH-7).
66
+
- A declarative bootstrap-asset manifest, scripts, VPN hooks, or network overlays (RSH-4).
67
+
- Fail-loud required-mount semantics (RSH-11).
65
68
- Reworking code-evaluator execution in the same change.
66
69
- Backward-compatible aliases for unpublished pre-production variables.
67
70
- A custom-role migration bridge. Default roles and tests still need the final mount permissions.
Scope: staging and the preview stages in `agenta_cloud`. Agenta keeps its current cloud security posture in this release; broader hardening is [RSH-12](./open-issues.md).
4
+
5
+
## Current state
6
+
7
+
The runner already ships in `agenta_cloud`: one `runner` service in `hosting/docker-compose/docker-compose.application.yml` (all stages share this file), built and pushed like the other images, with a `--runner-tag` in the deploy scripts and workflows. Staging is itself a preview-account stage; the only per-stage difference is which `${stage}.env` AWS secret gets pulled. Railway is not involved in `agenta_cloud` at all; the Railway scripts live in the public repo.
8
+
9
+
Nothing needs to be created or moved. The migration is a rename inside one compose file plus new keys in the per-stage secrets.
10
+
11
+
## The migration delta
12
+
13
+
Two places change, in lockstep with the first runner image that reads the new names:
14
+
15
+
1.**Compose runner env block** (about 10 lines in `docker-compose.application.yml`):
- The five `DAYTONA_*` mappings become their `AGENTA_RUNNER_DAYTONA_*` names.
18
+
- Delete the `AGENTA_AGENT_SANDBOX_PI_INSTALLED` line.
19
+
2.**Per-stage secret (`${stage}.env`)** for staging and each preview stage with agents enabled:
20
+
- Add `AGENTA_RUNNER_ENABLED_SANDBOX_PROVIDERS=daytona` and `AGENTA_RUNNER_DEFAULT_SANDBOX_PROVIDER=daytona`. The shared env_file already delivers these to both the API and the runner, so the API-side gate needs no compose change.
21
+
- Add the `AGENTA_RUNNER_DAYTONA_*` keys. Keep the generic `DAYTONA_*` keys: the code evaluator still reads them (`api/oss/src/utils/env.py`). The Daytona key is temporarily present under two names; that is accepted.
22
+
- Mint one random `AGENTA_RUNNER_TOKEN` per stage when the token PR lands; the shared env_file delivers it to services and runner with no compose change.
23
+
- Delete `AGENTA_SANDBOX_LOCAL_ALLOWED`, `AGENTA_SESSION_HARNESS_MOUNTS`, and other removed keys if they exist in the secret blobs.
24
+
25
+
Effort: one small `agenta_cloud` PR, minutes per stage of secret edits, one paired deploy with a smoke test. Roughly half a day for staging plus preview, not weeks.
26
+
27
+
## Why cloud needs no hardening work for this release
28
+
29
+
Cloud sets `AGENTA_RUNNER_ENABLED_SANDBOX_PROVIDERS=daytona`. With local disabled, no user code ever executes inside the runner container, so nothing user-controlled can read the runner's process environment, the shared env_file contents, or the Daytona provisioning credential. The enabled-provider list is the posture switch; disabling local is the entire cloud-specific security story for this release.
30
+
31
+
The impact report's remaining items (a dedicated `${stage}.runner.env` secret, removing the runner's shared env_file, dropping FUSE privileges, runner-first rollout ordering, evaluator namespace migration) describe pre-existing properties of the shared-secret model, not deltas this design imposes. They are deliberately out of scope; revisit them as their own hardening project after the release.
32
+
33
+
## Operational cautions
34
+
35
+
-**Lockstep deploy**: the new runner image stops reading the old names. The compose rename, the secret keys, and the runner tag must land in the same deploy, or Daytona config silently drops and, with local disabled, agent runs fail.
36
+
-**Rollback window**: keep the legacy keys in the secret until rolling back to an old runner image is no longer possible; delete them last.
Copy file name to clipboardExpand all lines: docs/design/agent-workflows/projects/runner-selfhosting-cleanup/context.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -50,7 +50,7 @@ DAYTONA_JWT_TOKEN and DAYTONA_ORGANIZATION_ID.
50
50
51
51
Nothing was wrong with the user's Daytona account. The request reached a runner deployment that was not configured for Daytona. The setup made it easy to split capabilities accidentally.
52
52
53
-
The target local QA deployment is one trusted development runner with both providers enabled and explicit local subscription bootstrap assets. This is a development convenience, not a production multi-tenant topology. A local harness shares the runner container and can inspect other same-user processes through `/proc`.
53
+
The target local QA deployment is one trusted development runner with both providers enabled and explicit read-only local subscription mounts. This is a development convenience, not a production multi-tenant topology. A local harness shares the runner container and can inspect other same-user processes through `/proc`.
54
54
55
55
## Trust boundaries
56
56
@@ -71,7 +71,7 @@ The runner stays in the deployment. The daemon and harness execute in a Daytona
71
71
- the resolved model credential for the run;
72
72
- workspace and instruction files;
73
73
- scoped durable-mount credentials;
74
-
- explicitly configured bootstrap assets;
74
+
- explicitly configured runtime files;
75
75
- runner-owned support binaries and configuration.
76
76
77
77
The runner's Daytona organization credential provisions the sandbox. It is infrastructure control-plane material and must never enter the sandbox environment.
@@ -80,19 +80,19 @@ The runner's Daytona organization credential provisions the sandbox. It is infra
80
80
81
81
### Configuration describes intent
82
82
83
-
Operators configure facts they own: enabled providers, a default, credentials, lifecycle values, and mounted bootstrap inputs. The runner detects runtime facts such as whether Pi is already installed.
83
+
Operators configure facts they own: enabled providers, a default, credentials, lifecycle values, and mounted subscription inputs. The runner detects runtime facts such as whether Pi is already installed.
84
84
85
85
### One owner per value
86
86
87
87
A Daytona API key used to provision agent sandboxes belongs to the runner. A Daytona key used by code evaluation belongs to the code evaluator. A shared ambient variable is not a simplification when the two consumers have different images, snapshots, permissions, and lifecycles.
88
88
89
89
### Explicit data movement
90
90
91
-
The runner should never infer that a missing model API key means "copy whatever login I happen to have." Runtime authentication is a declared mode, and bootstrap assets are a declared deployment input.
91
+
The runner should never infer that a missing model API key means "copy whatever login I happen to have." Runtime authentication is a declared mode, and subscription state is a declared deployment mount.
92
92
93
93
### Fail before side effects
94
94
95
-
Unknown providers, disabled providers, missing provider credentials, invalid bootstrap paths, and contradictory defaults fail at startup or before sandbox creation. Required mount failures fail the run. There is no silent provider fallback and no silent durable-to-ephemeral downgrade.
95
+
Unknown providers, disabled providers, missing provider credentials, and contradictory defaults fail at startup or before sandbox creation. There is no silent provider fallback. Mount behavior stays best-effort in version 1, with a structured warning on durable-to-ephemeral degradation (the fail-loud contract is RSH-11).
0 commit comments