OSEP: substrate as the backend POC and proposal#1310
Conversation
github-actions
Bot
added
component/server
documentation
There was a problem hiding this comment.
Pull request overview
This PR introduces an experimental Agent Substrate backend proof-of-concept for OpenSandbox’s Kubernetes runtime by adding a new agent-substrate workload provider, documenting it as OSEP-0017, and extending the public lifecycle contract to support selecting an administrator-registered template via extensions["agent-substrate.template"].
Changes:
- Extend the lifecycle API/spec and server schema to support Agent Substrate template mode (new startup source via
extensions["agent-substrate.template"]). - Add a new Kubernetes workload provider implementation (
agent-substrate) backed by a pinned gRPC API, including generated protobuf stubs and a client wrapper. - Add tests, configuration docs, and an end-to-end/manual POC smoke test + docs page.
Reviewed changes
Copilot reviewed 34 out of 36 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| specs/sandbox-lifecycle.yml | Documents new create-sandbox startup mode using extensions["agent-substrate.template"] and clarifies mutually exclusive sources. |
| server/uv.lock | Adds/pins grpc/protobuf-related dependencies needed for the Agent Substrate gRPC client and stub generation. |
| server/pyproject.toml | Declares runtime/dev dependencies for grpc/protobuf tooling and adds ruff ignores for generated protobuf code. |
| server/opensandbox_server/well_known_extensions.py | Introduces a safe-to-import constant for the new well-known extension key. |
| server/opensandbox_server/extensions/keys.py | Re-exports the Agent Substrate extension key for existing schema/import patterns. |
| server/opensandbox_server/api/schema.py | Enforces Agent Substrate template-mode request validation (mutual exclusivity with template-owned fields). |
| server/opensandbox_server/services/k8s/workload_provider.py | Adds provider hooks for preflight validation and PVC-management capability signaling. |
| server/opensandbox_server/services/k8s/provider_factory.py | Registers the new agent-substrate workload provider type. |
| server/opensandbox_server/services/k8s/init.py | Exposes the new provider type constant from the k8s services package. |
| server/opensandbox_server/services/k8s/kubernetes_service.py | Integrates provider preflight validation and skips image resolution when in Agent Substrate template mode; adjusts PVC cleanup behavior by provider capability. |
| server/opensandbox_server/services/k8s/agent_substrate_client.py | Implements a synchronous gRPC client wrapper with TLS and optional token-file auth. |
| server/opensandbox_server/services/k8s/agent_substrate_provider.py | Implements the Agent Substrate provider, mapping sandboxes to Actors and routing endpoints via atenet. |
| server/opensandbox_server/services/k8s/agent_substrate_proto/init.py | Initializes the generated gRPC proto package. |
| server/opensandbox_server/services/k8s/agent_substrate_proto/UPSTREAM.md | Documents the pinned upstream proto source and regeneration command. |
| server/opensandbox_server/services/k8s/agent_substrate_proto/ateapi.proto | Adds the pinned upstream proto definition used to generate Python stubs. |
| server/opensandbox_server/services/k8s/agent_substrate_proto/ateapi_pb2.py | Generated protobuf messages for the pinned Agent Substrate API. |
| server/opensandbox_server/services/k8s/agent_substrate_proto/ateapi_pb2.pyi | Generated typing stubs for protobuf messages. |
| server/opensandbox_server/services/k8s/agent_substrate_proto/ateapi_pb2_grpc.py | Generated gRPC client/server stub bindings with version checks. |
| server/opensandbox_server/services/docker/docker_service.py | Rejects Agent Substrate template mode on the Docker runtime with a clear 400 error. |
| server/opensandbox_server/config.py | Adds Agent Substrate runtime configuration models and validates provider/config coupling. |
| server/opensandbox_server/examples/example.config.agent-substrate.toml | Adds a sample server config for the Agent Substrate POC. |
| server/configuration.md | Documents the new provider and the [agent_substrate] configuration block and constraints. |
| server/tests/test_schema.py | Adds schema-level validation tests for Agent Substrate template mode constraints. |
| server/tests/test_config.py | Adds config validation tests for Agent Substrate config structure and invariants. |
| server/tests/test_docker_service.py | Adds a Docker runtime test ensuring Agent Substrate template mode is rejected. |
| server/tests/k8s/test_provider_factory.py | Adds factory registration/creation tests for the new provider type. |
| server/tests/k8s/test_kubernetes_service.py | Adds Kubernetes service test ensuring image resolution is skipped in template mode. |
| server/tests/k8s/test_agent_substrate_provider.py | Adds unit/integration-style tests for Actor lifecycle mapping and endpoint routing behavior. |
| oseps/README.md | Adds OSEP-0017 to the index. |
| oseps/0017-agent-substrate-workload-provider.md | Adds the full proposal/specification for the Agent Substrate provider POC and future production considerations. |
| examples/agent-substrate/README.md | Adds a small entrypoint README for running the POC check. |
| examples/agent-substrate/test_poc.py | Adds a runnable smoke test that exercises create/get/endpoint/pause/auto-resume/delete via REST + atenet. |
| docs/getting-started/configuration.md | Adds a getting-started section for the Agent Substrate POC configuration and usage constraints. |
| docs/examples/index.md | Adds Agent Substrate POC to the examples index. |
| docs/examples/agent-substrate.md | Adds the full step-by-step POC validation guide. |
| docs/.vitepress/config.mts | Adds the Agent Substrate example page to the docs sidebar/navigation. |
Files not reviewed (1)
- server/opensandbox_server/services/k8s/agent_substrate_proto/ateapi_pb2.py: Generated file
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if self.credential_proxy is not None: | ||
| conflicting_fields.append("credentialProxy") | ||
| if self.secure_access: | ||
| conflicting_fields.append("secureAccess") |
| "pytest>=7.0.0", | ||
| "pytest-asyncio>=0.21.0", | ||
| "pytest-cov>=4.0.0", | ||
| "ruff>=0.14.8", | ||
| "pyright>=1.1.0", | ||
| "grpcio-tools>=1.75", | ||
| ] |
| "grpcio>=1.75", | ||
| "protobuf>=6.31", |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ed9dda8233
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| "grpcio>=1.75", | ||
| "protobuf>=6.31", |
There was a problem hiding this comment.
The checked-in generated files require grpcio>=1.82.1 (ateapi_pb2_grpc.py sets GRPC_GENERATED_VERSION = '1.82.1') and a protobuf runtime at least 7.35.0 (ateapi_pb2.py validates that generated version), but these bounds allow grpcio==1.75.x and protobuf==6.31.x. In installs that do not use uv.lock or have compatible transitive constraints, importing the new Agent Substrate client can raise before server startup, so the provider can be configured but cannot load. Please raise the minimums to match the generated code.
Useful? React with 👍 / 👎.
|
|
||
| export OPENSANDBOX_API_URL=http://127.0.0.1:8080 | ||
| export ATENET_URL=http://127.0.0.1:8000 | ||
| export AGENT_SUBSTRATE_TEMPLATE_KEY=execd |
There was a problem hiding this comment.
This guide exports execd, and the POC script defaults to the same key, but the bundled Agent Substrate config registers only key = "python" under [[agent_substrate.templates]]. Following the guide against that sample config makes create fail in validate_create_request as an unknown template instead of exercising the POC; align the exported/default key or the sample config.
Useful? React with 👍 / 👎.
Signed-off-by: Ryan Zhang <zhangryan@microsoft.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 51a897b578
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| (request.extensions or {}).get(AGENT_SUBSTRATE_TEMPLATE_KEY, "").strip() | ||
| ) | ||
|
|
||
| self.workload_provider.validate_create_request(request) |
There was a problem hiding this comment.
When the Kubernetes service is configured for agent-substrate, requests with a missing or unknown extensions["agent-substrate.template"] reach validate_create_request, which raises ValueError; because this call is before the existing try block that converts ValueError to a 400 response, those client errors escape as unhandled server errors. The same happens when a template-mode request is sent to a non-Agent-Substrate Kubernetes provider, so move this validation under the existing error mapping or raise an HTTPException here.
Useful? React with 👍 / 👎.
| ateapi_pb2.Actor.STATUS_CRASHED: "Failed", | ||
| } | ||
| state = state_map.get(actor.status, "Pending") | ||
| status_name = ateapi_pb2.Actor.Status.Name(actor.status) |
There was a problem hiding this comment.
If a newer Agent Substrate server returns an Actor status enum value this generated client does not know, state_map.get(...) correctly falls back to Pending, but Actor.Status.Name(actor.status) then raises ValueError. That makes get/list/create-wait paths fail instead of reporting a safe pending/unknown reason for forward-compatible enum values.
Useful? React with 👍 / 👎.
Signed-off-by: Ryan Zhang <zhangryan@microsoft.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4ef9cd9909
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| "annotations": dict(record.annotations), | ||
| "creationTimestamp": create_time, | ||
| }, | ||
| "spec": {}, |
There was a problem hiding this comment.
When GET/list reads an Agent Substrate sandbox, the service passes this dict into _build_sandbox_from_workload, whose dict path looks for containers under spec.template/spec.podTemplate and then creates ImageSpec(uri="unknown") when there is no snapshot and no image. Because actor-backed workloads return an empty spec here, every Agent Substrate sandbox is reported as an image sandbox with an "unknown" image and empty entrypoint instead of the template-owned startup source described by the new API contract. Add a mapper/provider branch so actor-backed sandboxes leave image/entrypoint unset or expose the registered template.
Useful? React with 👍 / 👎.
| key = "python" | ||
| namespace = "opensandbox-templates" | ||
| name = "python-execd" | ||
| logical_port = 44772 |
There was a problem hiding this comment.
Why is port information needed here?
Summary
Testing
Breaking Changes
Checklist