Skip to content

Commit e593343

Browse files
committed
docs(agent-workflows): rescope runner self-hosting cleanup to v2
Adopt owner review round 1: defer the capability endpoint, the bootstrap manifest, and fail-loud mounts; add the hosting-layout workstream, per-page docs dispositions, and the agenta_cloud migration delta. Claude-Session: https://claude.ai/code/session_01XhENr63WL9npkKrJGnzDc1
1 parent 1c3e0bd commit e593343

10 files changed

Lines changed: 329 additions & 423 deletions

File tree

docs/design/agent-workflows/projects/runner-selfhosting-cleanup/README.md

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# Runner self-hosting cleanup
22

3-
Status: DESIGNED, awaiting owner review
3+
Status: DESIGNED v2 after owner review round 1, awaiting second-round review
44
Date: 2026-07-13
55
Target branch: `big-agents`
66

7-
This project makes the agent runner understandable from its deployment configuration alone. It replaces shared and overlapping environment variables with one runner-owned provider contract, removes automatic subscription credential upload, adds explicit bootstrap assets, and makes required mounts fail loudly.
7+
This project makes the agent runner understandable from its deployment configuration alone. It replaces shared and overlapping environment variables with one runner-owned provider contract, removes automatic subscription credential upload, and documents explicit operator-owned subscription mounts. A capability endpoint, a bootstrap manifest, and fail-loud mount semantics are deferred open issues, not release work.
88

99
## Outcome
1010

@@ -23,23 +23,25 @@ The normal deployment uses one runner service. That runner can support both loca
2323

2424
- The runner is a deployment-level service, not one sidecar per harness or sandbox provider.
2525
- The operator enables sandbox providers explicitly. Unset means local only, not every provider known to the binary.
26+
- The enabled-provider list is also the deployment security posture: `daytona` only means no user code ever runs inside the runner container. That is the right posture for any multi-tenant or exposed deployment.
27+
- API and web get provider availability from the same environment value, set once per deployment by the hosting templates. A discovery endpoint is deferred (RSH-7).
2628
- Daytona configuration belongs under the runner namespace. Code-evaluator Daytona configuration is separate.
2729
- The runner never discovers an operator's home directory or automatically uploads Pi or Claude OAuth state.
28-
- Subscription authentication is an explicit local self-hosting setup taught in a tutorial.
29-
- Bootstrap assets are declarative, typed, and implemented by both local and Daytona adapters. Version 1 does not execute arbitrary hooks.
30+
- Subscription authentication is an explicit local self-hosting setup: a read-only Compose volume mount taught in a tutorial. There is no bootstrap manifest in version 1 (RSH-4).
31+
- Runtime customization (extra binaries, certificates, dependencies) happens through operator-built images and the shipped Daytona snapshot scripts, not runner configuration.
3032
- Remote subscription authentication is unsupported until the relevant provider explicitly approves the integration and Agenta defines a safe product contract.
3133
- Pi installation is detected and repaired by the runtime. There is no deployment-wide "Pi is installed" boolean.
32-
- Durable mounts required by a session or workflow artifact fail the run when they cannot be signed or mounted. Sessionless runs remain ephemeral.
34+
- Mount failure behavior stays best-effort in version 1, with one structured warning when a durable mount degrades. The fail-loud contract is deferred (RSH-11).
3335
- The runner receives a narrow environment and no static Agenta API key. Per-run caller credentials authorize callbacks and trace export. It does not inherit a shared application environment or database and cryptographic secrets.
3436

3537
## Reading order
3638

3739
- [Context](./context.md) explains today's architecture and why the current variables are confusing.
3840
- [Research](./research.md) records repository evidence and the review of PRs #5274, #5285, and #5286.
39-
- [Interface](./interface.md) defines the target environment, Helm, capability, bootstrap, and failure contracts.
41+
- [Interface](./interface.md) defines the target environment, Helm, provider-availability, subscription-mount, and failure contracts.
4042
- [Plan](./plan.md) splits the cleanup into reviewable implementation PRs.
4143
- [QA](./qa.md) defines the environment, harness, and credential matrix.
42-
- [Documentation plan](./documentation-plan.md) maps the final public docs through Diátaxis.
44+
- [Documentation plan](./documentation-plan.md) maps the final public docs through Diátaxis, with per-page dispositions for the existing self-host tree.
4345
- [Open issues](./open-issues.md) parks work that should not block the cleanup.
4446
- [Status](./status.md) is the owner review checklist and progress log.
4547

@@ -48,20 +50,21 @@ The normal deployment uses one runner service. That runner can support both loca
4850
- Runner provider selection and configuration.
4951
- Runner-specific Daytona names and validation.
5052
- Runner-to-Services routing and authentication.
51-
- Declarative bootstrap assets for local and Daytona environments.
52-
- Local subscription setup.
53+
- Local subscription setup through explicit read-only mounts.
5354
- Removal of automatic Daytona OAuth upload.
5455
- Removal of deployment-wide runtime facts and hidden policy switches.
55-
- Required-mount failure behavior.
56-
- Compose, Helm, Railway, self-hosting docs, and the local 8280 QA deployment.
57-
- Capability discovery needed to keep the API and UI honest.
56+
- A structured warning when a durable mount degrades.
57+
- Compose, Helm, Railway, self-hosting docs, a `hosting/README.md` layout map, and the local 8280 QA deployment.
58+
- One shared provider-availability value read by API and runner.
5859

5960
## Out of scope
6061

6162
- Implementing the Docker or E2B providers.
6263
- Sharing a personal subscription between users or tenants.
6364
- Remote subscription support.
64-
- Arbitrary bootstrap scripts, VPN hooks, or network overlays.
65+
- A runner capability-discovery endpoint (RSH-7).
66+
- A declarative bootstrap-asset manifest, scripts, VPN hooks, or network overlays (RSH-4).
67+
- Fail-loud required-mount semantics (RSH-11).
6568
- Reworking code-evaluator execution in the same change.
6669
- Backward-compatible aliases for unpublished pre-production variables.
6770
- A custom-role migration bridge. Default roles and tests still need the final mount permissions.
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Cloud impact: what changes in agenta_cloud
2+
3+
Scope: staging and the preview stages in `agenta_cloud`. Agenta keeps its current cloud security posture in this release; broader hardening is [RSH-12](./open-issues.md).
4+
5+
## Current state
6+
7+
The runner already ships in `agenta_cloud`: one `runner` service in `hosting/docker-compose/docker-compose.application.yml` (all stages share this file), built and pushed like the other images, with a `--runner-tag` in the deploy scripts and workflows. Staging is itself a preview-account stage; the only per-stage difference is which `${stage}.env` AWS secret gets pulled. Railway is not involved in `agenta_cloud` at all; the Railway scripts live in the public repo.
8+
9+
Nothing needs to be created or moved. The migration is a rename inside one compose file plus new keys in the per-stage secrets.
10+
11+
## The migration delta
12+
13+
Two places change, in lockstep with the first runner image that reads the new names:
14+
15+
1. **Compose runner env block** (about 10 lines in `docker-compose.application.yml`):
16+
- `SANDBOX_AGENT_PROVIDER` becomes `AGENTA_RUNNER_DEFAULT_SANDBOX_PROVIDER` (value `daytona`).
17+
- The five `DAYTONA_*` mappings become their `AGENTA_RUNNER_DAYTONA_*` names.
18+
- Delete the `AGENTA_AGENT_SANDBOX_PI_INSTALLED` line.
19+
2. **Per-stage secret (`${stage}.env`)** for staging and each preview stage with agents enabled:
20+
- Add `AGENTA_RUNNER_ENABLED_SANDBOX_PROVIDERS=daytona` and `AGENTA_RUNNER_DEFAULT_SANDBOX_PROVIDER=daytona`. The shared env_file already delivers these to both the API and the runner, so the API-side gate needs no compose change.
21+
- Add the `AGENTA_RUNNER_DAYTONA_*` keys. Keep the generic `DAYTONA_*` keys: the code evaluator still reads them (`api/oss/src/utils/env.py`). The Daytona key is temporarily present under two names; that is accepted.
22+
- Mint one random `AGENTA_RUNNER_TOKEN` per stage when the token PR lands; the shared env_file delivers it to services and runner with no compose change.
23+
- Delete `AGENTA_SANDBOX_LOCAL_ALLOWED`, `AGENTA_SESSION_HARNESS_MOUNTS`, and other removed keys if they exist in the secret blobs.
24+
25+
Effort: one small `agenta_cloud` PR, minutes per stage of secret edits, one paired deploy with a smoke test. Roughly half a day for staging plus preview, not weeks.
26+
27+
## Why cloud needs no hardening work for this release
28+
29+
Cloud sets `AGENTA_RUNNER_ENABLED_SANDBOX_PROVIDERS=daytona`. With local disabled, no user code ever executes inside the runner container, so nothing user-controlled can read the runner's process environment, the shared env_file contents, or the Daytona provisioning credential. The enabled-provider list is the posture switch; disabling local is the entire cloud-specific security story for this release.
30+
31+
The impact report's remaining items (a dedicated `${stage}.runner.env` secret, removing the runner's shared env_file, dropping FUSE privileges, runner-first rollout ordering, evaluator namespace migration) describe pre-existing properties of the shared-secret model, not deltas this design imposes. They are deliberately out of scope; revisit them as their own hardening project after the release.
32+
33+
## Operational cautions
34+
35+
- **Lockstep deploy**: the new runner image stops reading the old names. The compose rename, the secret keys, and the runner tag must land in the same deploy, or Daytona config silently drops and, with local disabled, agent runs fail.
36+
- **Rollback window**: keep the legacy keys in the secret until rolling back to an old runner image is no longer possible; delete them last.

docs/design/agent-workflows/projects/runner-selfhosting-cleanup/context.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ DAYTONA_JWT_TOKEN and DAYTONA_ORGANIZATION_ID.
5050

5151
Nothing was wrong with the user's Daytona account. The request reached a runner deployment that was not configured for Daytona. The setup made it easy to split capabilities accidentally.
5252

53-
The target local QA deployment is one trusted development runner with both providers enabled and explicit local subscription bootstrap assets. This is a development convenience, not a production multi-tenant topology. A local harness shares the runner container and can inspect other same-user processes through `/proc`.
53+
The target local QA deployment is one trusted development runner with both providers enabled and explicit read-only local subscription mounts. This is a development convenience, not a production multi-tenant topology. A local harness shares the runner container and can inspect other same-user processes through `/proc`.
5454

5555
## Trust boundaries
5656

@@ -71,7 +71,7 @@ The runner stays in the deployment. The daemon and harness execute in a Daytona
7171
- the resolved model credential for the run;
7272
- workspace and instruction files;
7373
- scoped durable-mount credentials;
74-
- explicitly configured bootstrap assets;
74+
- explicitly configured runtime files;
7575
- runner-owned support binaries and configuration.
7676

7777
The runner's Daytona organization credential provisions the sandbox. It is infrastructure control-plane material and must never enter the sandbox environment.
@@ -80,19 +80,19 @@ The runner's Daytona organization credential provisions the sandbox. It is infra
8080

8181
### Configuration describes intent
8282

83-
Operators configure facts they own: enabled providers, a default, credentials, lifecycle values, and mounted bootstrap inputs. The runner detects runtime facts such as whether Pi is already installed.
83+
Operators configure facts they own: enabled providers, a default, credentials, lifecycle values, and mounted subscription inputs. The runner detects runtime facts such as whether Pi is already installed.
8484

8585
### One owner per value
8686

8787
A Daytona API key used to provision agent sandboxes belongs to the runner. A Daytona key used by code evaluation belongs to the code evaluator. A shared ambient variable is not a simplification when the two consumers have different images, snapshots, permissions, and lifecycles.
8888

8989
### Explicit data movement
9090

91-
The runner should never infer that a missing model API key means "copy whatever login I happen to have." Runtime authentication is a declared mode, and bootstrap assets are a declared deployment input.
91+
The runner should never infer that a missing model API key means "copy whatever login I happen to have." Runtime authentication is a declared mode, and subscription state is a declared deployment mount.
9292

9393
### Fail before side effects
9494

95-
Unknown providers, disabled providers, missing provider credentials, invalid bootstrap paths, and contradictory defaults fail at startup or before sandbox creation. Required mount failures fail the run. There is no silent provider fallback and no silent durable-to-ephemeral downgrade.
95+
Unknown providers, disabled providers, missing provider credentials, and contradictory defaults fail at startup or before sandbox creation. There is no silent provider fallback. Mount behavior stays best-effort in version 1, with a structured warning on durable-to-ephemeral degradation (the fail-loud contract is RSH-11).
9696

9797
### Pre-production cleanup
9898

0 commit comments

Comments
 (0)