Commit e142bce
authored
🤖 feat: wire aggregated API server to codersdk backend (#50)
## Summary
Wire the aggregated API server (`--app=aggregated-apiserver`) to a real
Coder deployment backend via `codersdk`, replacing the hardcoded
in-memory storage with codersdk-backed REST storage for `CoderTemplate`
and `CoderWorkspace` resources.
## Background
The aggregated API server mode previously served stub data from
hardcoded maps. This PR makes it a real Kubernetes façade over a Coder
deployment, so that Kubernetes clients can query and mutate Coder
templates/workspaces using standard K8s REST semantics.
Key design decisions (v1):
- **Namespace == CoderControlPlane namespace**: Resources remain
namespace-scoped; the namespace identifies which CoderControlPlane backs
the request.
- **Deterministic name formats**: Templates use `<org>.<template-name>`,
Workspaces use `<org>.<user>.<workspace-name>` — the dot separator works
because Coder names forbid dots while K8s object names allow them.
- **Admin token auth**: A single `--coder-session-token` is used (no
per-request impersonation in v1).
- **Start/stop via Update**: `CoderWorkspace.spec.running` drives
workspace transitions.
## Implementation
### New packages
- **`internal/aggregated/coder/`** — Backend helpers: SDK client factory
(`config.go`), K8s↔Coder name parsing (`names.go`), codersdk→K8s error
mapping (`errors.go`), and `ClientProvider` interface with
`StaticClientProvider` (`provider.go`).
- **`internal/aggregated/convert/`** — Pure conversion functions between
`codersdk.Template`/`codersdk.Workspace` and the K8s aggregated API
types.
### Modified packages
- **`api/aggregation/v1alpha1/types.go`** — Expanded
`CoderTemplate{Spec,Status}` and `CoderWorkspace{Spec,Status}` with
codersdk-aligned fields (organization, templateName, running, build
status, etc.).
- **`internal/aggregated/storage/`** — Rewrote `template.go` and
`workspace.go` from hardcoded maps to codersdk-backed CRUD:
- Templates: `GET` → `TemplateByName`, `LIST` → `Templates`, `CREATE` →
`CreateTemplate`, `DELETE` → `DeleteTemplate`
- Workspaces: `GET` → `WorkspaceByOwnerAndName`, `LIST` → `Workspaces`,
`CREATE` → `CreateUserWorkspace` + optional stop build, `UPDATE` →
start/stop via `CreateWorkspaceBuild`, `DELETE` → delete transition
build
- **`internal/app/apiserverapp/apiserverapp.go`** — Added `Options`
struct and `RunWithOptions`, changed `NewAPIGroupInfo` to accept a
`ClientProvider`, updated OpenAPI definitions.
- **`app_dispatch.go`** — Added `--coder-session-token`, `--coder-url`,
`--coder-request-timeout` CLI flags wired into `RunWithOptions`.
### Tests
- Unit tests for all new packages (names, errors, config, provider,
converters).
- Rewrote `storage_test.go` with httptest-backed Coder API mock covering
all CRUD operations.
- Added `integration_test.go` that bootstraps the full aggregated API
server and verifies HTTP responses through the complete stack.
## Risks
- **Breaking change for `Run(ctx)`**: The zero-arg `Run` function now
wraps `RunWithOptions` with empty options, which will fail without Coder
configuration. This is intentional — the real entrypoint is now
`RunWithOptions` via CLI dispatch.
- **Legacy template fields**: `CoderTemplateSpec.Running` and
`CoderTemplateStatus.AutoShutdown` are retained temporarily for
compatibility with existing callers (e.g., MCP tools). These should be
removed in a follow-up once callers are migrated.
- **No coderdtest integration**: `coderdtest` is not vendored (massive
dep tree). Storage tests use httptest mocks instead. True coderdtest
integration is deferred.
- **No WATCH semantics**: Only poll-based operations
(GET/LIST/CREATE/UPDATE/DELETE) are implemented.
---
<details>
<summary>📋 Implementation Plan</summary>
# Wire aggregated API server → Coder (`codersdk`) backend
## Context / Why
`coder-k8s` has an **aggregated API server mode**
(`--app=aggregated-apiserver`) that currently serves the API group
`aggregation.coder.com/v1alpha1`, but its storage is **hardcoded
in-memory**. We want that API server to become a real “Kubernetes
façade” over a running Coder deployment, backed by
`github.com/coder/coder/v2/codersdk`, so Kubernetes clients can **query
and mutate** Coder templates/workspaces (and eventually other Coder
resources) using standard K8s REST semantics.
A key dev/test requirement is that we can run **`coderdtest` + envtest**
to exercise the end-to-end path:
`k8s client → kube-apiserver (envtest) → API aggregation proxy →
coder-k8s aggregated-apiserver → codersdk → coderdtest`
## Goals (first milestone)
1. Replace hardcoded storage with `codersdk`-backed storage for:
- `CoderTemplate` (`codertemplates`)
- `CoderWorkspace` (`coderworkspaces`)
2. Support the core REST verbs:
- Templates: `GET`, `LIST`, `CREATE`, `DELETE` (optionally `UPDATE`
meta)
- Workspaces: `GET`, `LIST`, `CREATE`, `UPDATE` (drives start/stop),
`DELETE` (drives delete transition)
3. Add aggregated-apiserver configuration (flags/env) for:
- Coder URL
- Coder session token (admin token in dev)
- Coder request timeout (optional)
4. Define a deterministic K8s identity scheme for locating Coder objects
(see “Chosen semantics”).
5. Add tests that prove storage → codersdk behavior using `coderdtest`.
6. Add (or scaffold) an envtest + APIService-based integration test that
proxies requests through the envtest kube-apiserver.
## Non-goals (for first milestone)
- Per-user auth / impersonation (requestheader authn, OIDC token
exchange, etc.).
- Efficient `WATCH` semantics / informer-grade semantics.
- Exhaustive mapping of every Coder API field into K8s types.
## Evidence (repo + upstream facts)
- Aggregated apiserver bootstraps and installs storage in:
- `internal/app/apiserverapp/apiserverapp.go` (`NewAPIGroupInfo`
registers `coderworkspaces` and `codertemplates`).
- Current storage is hardcoded read-only, implements only `rest.Getter`
+ `rest.Lister`:
- `internal/aggregated/storage/workspace.go`
- `internal/aggregated/storage/template.go`
- API types are minimal stubs:
- `api/aggregation/v1alpha1/types.go`
- `codersdk` is already vendored and in use:
- `internal/coderbootstrap/client.go` imports
`github.com/coder/coder/v2/codersdk`
- `go.mod` requires `github.com/coder/coder/v2 v2.30.0`
- Relevant `codersdk` methods/signatures (vendored):
- Workspaces:
- `(*codersdk.Client).Workspaces(ctx, codersdk.WorkspaceFilter)
(codersdk.WorkspacesResponse, error)` in
`vendor/.../codersdk/workspaces.go`
- `(*codersdk.Client).WorkspaceByOwnerAndName(ctx, owner, name string,
params codersdk.WorkspaceOptions) (codersdk.Workspace, error)` in
`vendor/.../codersdk/workspaces.go`
- `(*codersdk.Client).CreateUserWorkspace(ctx, user string, req
codersdk.CreateWorkspaceRequest) (codersdk.Workspace, error)` in
`vendor/.../codersdk/organizations.go`
- `(*codersdk.Client).CreateWorkspaceBuild(ctx, workspaceID uuid.UUID,
req codersdk.CreateWorkspaceBuildRequest) (codersdk.WorkspaceBuild,
error)` in `vendor/.../codersdk/workspaces.go`
- Templates:
- `(*codersdk.Client).TemplatesByOrganization(ctx, orgID uuid.UUID)
([]codersdk.Template, error)` in `vendor/.../codersdk/organizations.go`
- `(*codersdk.Client).TemplateByName(ctx, orgID uuid.UUID, name string)
(codersdk.Template, error)` in `vendor/.../codersdk/organizations.go`
- `(*codersdk.Client).CreateTemplate(ctx, orgID uuid.UUID, req
codersdk.CreateTemplateRequest) (codersdk.Template, error)` in
`vendor/.../codersdk/organizations.go`
- `(*codersdk.Client).DeleteTemplate(ctx, templateID uuid.UUID) error`
in `vendor/.../codersdk/templates.go`
- `coderdtest` exists upstream and provides in-memory coderd + helpers:
- `coderd/coderdtest` from `coder/coder` tag `v2.30.0` (confirmed via
`web_fetch`).
- Kubernetes validates CRD instance names as **DNS-1123 subdomains**
(dots allowed) — see
`vendor/k8s.io/apimachinery/pkg/util/validation/validation.go`
(`dns1123SubdomainFmt`).
## Chosen semantics (to keep v1 implementable)
These are explicit “v1” choices to avoid design deadlocks.
1. **Resources remain namespace-scoped; namespace == CoderControlPlane
namespace**
- `CoderTemplate` and `CoderWorkspace` remain **namespaced** in the
aggregated API (`NamespaceScoped() == true`).
- `metadata.namespace` is interpreted as the namespace that contains the
backing `coder.com/v1alpha1 CoderControlPlane` instance.
- Storage uses `genericapirequest.NamespaceValue(ctx)` to select the
control plane and build a `codersdk.Client` for that namespace.
2. **Namespace is NOT the Coder organization**
- The Coder organization is explicit per object (encoded in
`metadata.name`, and optionally mirrored in `spec.organization`).
3. **Deterministic name formats (encode Coder lookup keys)**
- Templates: `<org>.<template-name>`
- Workspaces: `<org>.<user>.<workspace-name>`
- We use `.` as the separator because Coder “names” are
alphanumeric-with-hyphens (no dots) (`codersdk.NameValid`), while
Kubernetes object names allow dots.
- Kubernetes validates CRD instance names as **DNS-1123 subdomains**:
lowercase only; segments separated by dots must be valid DNS labels (no
empty segments / no consecutive dots); max 253 chars; must start/end
with an alphanumeric.
4. **Admin token, but no default owner**
- The aggregated-apiserver is configured with a single
`--coder-session-token` (admin token in dev/test); we do not implement
per-request user impersonation in v1.
- Storage never assumes `me`; it derives the target user from the
workspace object name and can list across all workspaces visible to the
admin token.
5. **`CoderWorkspace.spec.running` drives start/stop**
- `UPDATE` compares desired `.spec.running` to current state and
performs `CreateWorkspaceBuild` with `transition=start|stop`.
<details>
<summary>Why these semantics?</summary>
- K8s `GET` requests provide only `{namespace, name}`. Using namespace
to pick the backing CoderControlPlane, and encoding `{org,user}` into
the object name, makes `GET` deterministic without extra indices/caches.
- This also keeps the org dimension explicit without overloading
Kubernetes namespaces.
</details>
---
# Implementation Plan (swarm-friendly)
## 0) Repo-wide prep / guardrails
**Owner:** 1 agent
- Add a short design doc comment to the aggregated storage package
explaining the v1 semantics (namespaced resources keyed by control plane
namespace, composite name formats, admin token, no per-request
impersonation) so future changes don’t silently break assumptions.
- Ensure every new exported function includes defensive nil/empty checks
consistent with existing code (see `assertion failed:` style in
`internal/aggregated/storage/*`).
## 1) API type expansion (if we want meaningful CREATE)
**Owner:** 1–2 agents
### 1.1 Update `api/aggregation/v1alpha1/types.go`
Add fields needed to create/drive resources.
Resource scope:
- Keep both resources **namespaced** (no scope change). Adding explicit
`// +kubebuilder:resource:scope=Namespaced` markers is optional but
recommended for clarity.
Suggested shapes (keep minimal; everything optional except
create-critical fields):
```go
// CoderTemplateSpec defines the desired state of a CoderTemplate.
type CoderTemplateSpec struct {
// Organization is the target Coder organization name (explicit; not derived from namespace).
// Must match the `<organization>` prefix in `metadata.name` (see “Chosen semantics”).
Organization string `json:"organization,omitempty"`
// VersionID is the Coder template version UUID used on creation.
// Required for CREATE.
VersionID string `json:"versionID,omitempty"`
DisplayName string `json:"displayName,omitempty"`
Description string `json:"description,omitempty"`
Icon string `json:"icon,omitempty"`
Deprecated bool `json:"deprecated,omitempty"`
}
type CoderTemplateStatus struct {
ID string `json:"id,omitempty"` // Coder UUID
OrganizationName string `json:"organizationName,omitempty"`
ActiveVersionID string `json:"activeVersionID,omitempty"`
UpdatedAt *metav1.Time `json:"updatedAt,omitempty"`
}
// CoderWorkspaceSpec defines the desired state of a CoderWorkspace.
type CoderWorkspaceSpec struct {
// Organization is the target Coder organization name (explicit; not derived from namespace).
Organization string `json:"organization,omitempty"`
// TemplateName resolves via TemplateByName(organization, templateName).
TemplateName string `json:"templateName,omitempty"`
// Optional: override and pin to a template version.
TemplateVersionID string `json:"templateVersionID,omitempty"`
Running bool `json:"running"`
// Optional start/stop tuning.
TTLMillis *int64 `json:"ttlMillis,omitempty"`
AutostartSchedule *string `json:"autostartSchedule,omitempty"`
}
type CoderWorkspaceStatus struct {
ID string `json:"id,omitempty"` // Coder UUID
OwnerName string `json:"ownerName,omitempty"`
OrganizationName string `json:"organizationName,omitempty"`
TemplateName string `json:"templateName,omitempty"`
LatestBuildID string `json:"latestBuildID,omitempty"`
LatestBuildStatus string `json:"latestBuildStatus,omitempty"`
AutoShutdown *metav1.Time `json:"autoShutdown,omitempty"`
LastUsedAt *metav1.Time `json:"lastUsedAt,omitempty"`
}
```
### 1.2 Regenerate generated artifacts
**Must do in Exec mode after type changes**:
- `make codegen`
- `make manifests`
- `make docs-reference` (if API reference docs exist/are required)
### 1.3 Update OpenAPI definitions used by aggregated-apiserver
`internal/app/apiserverapp/apiserverapp.go` currently manually builds a
very small OpenAPI schema for spec/status. Update it to reflect the new
fields (or, if we decide to defer schema accuracy, at minimum ensure
required create fields are present in the schema).
---
## 2) Add a codersdk-backed “backend” helper package
**Owner:** 1 agent
Create `internal/aggregated/coder/` (new package) with the shared
plumbing that storage needs.
### 2.1 Options / factory
File: `internal/aggregated/coder/config.go`
```go
type Config struct {
CoderURL *url.URL
SessionToken string
RequestTimeout time.Duration
}
func NewSDKClient(cfg Config) (*codersdk.Client, error) {
// defensive checks
// codersdk.New(cfg.CoderURL)
// client.SetSessionToken(cfg.SessionToken)
// ensure HTTPClient != nil and set Timeout
}
```
Notes:
- Don’t mutate a shared `*codersdk.Client`’s token at runtime; treat it
as immutable.
### 2.2 K8s name parsing (org/user keys)
File: `internal/aggregated/coder/names.go`
```go
// Template object name format: "<org>.<template-name>".
func ParseTemplateName(name string) (org string, template string, err error) {
// defensive checks: non-empty name, exactly one '.', no empty segments
// validate full name is a DNS-1123 subdomain (CRD object name rules; dots allowed)
// validate each segment with codersdk.NameValid and require lowercase
}
// Workspace object name format: "<org>.<user>.<workspace-name>".
func ParseWorkspaceName(name string) (org string, user string, workspace string, err error) {
// defensive checks: non-empty name, exactly two '.', no empty segments
// validate full name is a DNS-1123 subdomain
// validate each segment with codersdk.NameValid and require lowercase
}
func BuildTemplateName(org, template string) string {
return strings.ToLower(org + "." + template)
}
func BuildWorkspaceName(org, user, workspace string) string {
return strings.ToLower(org + "." + user + "." + workspace)
}
```
Org resolution uses the parsed `org` (and/or `spec.organization`) and
calls:
- `orgObj, err := sdk.OrganizationByName(ctx, orgName)`
### 2.3 Error mapping
File: `internal/aggregated/coder/errors.go`
- Convert `*codersdk.Error` (status codes) into Kubernetes-style errors:
- 404 → `apierrors.NewNotFound`
- 403 → `apierrors.NewForbidden`
- 409 → `apierrors.NewAlreadyExists` / `apierrors.NewConflict` (choose
based on operation)
- others → `apierrors.NewInternalError`
Also: wrap unknown errors with context and retain the original error for
logs.
### 2.4 Resolve `CoderControlPlane` per request namespace (namespace →
Coder URL)
File: `internal/aggregated/coder/provider.go` (or similar)
We need a single place that translates the **Kubernetes request
namespace** (which is the **control plane namespace**) into a
`*codersdk.Client` configured for that control plane.
Suggested interface:
```go
type ClientProvider interface {
ClientForNamespace(ctx context.Context, namespace string) (*codersdk.Client, error)
}
```
Suggested concrete implementation (names are illustrative):
```go
type ControlPlaneClientProvider struct {
KubeClient client.Client // controller-runtime client, talks to the main kube-apiserver
// Global defaults
SessionToken string
RequestTimeout time.Duration
// Optional fallback for dev (when no CoderControlPlane exists)
DefaultCoderURL *url.URL
// Optional selector (if multiple CoderControlPlanes exist in a namespace)
ControlPlaneName string
// Optional cache: namespace -> *codersdk.Client
// (protect with mutex)
}
```
Algorithm for `ClientForNamespace`:
1. Validate `namespace != ""`.
2. Resolve the backing `CoderControlPlane` in that namespace:
- If `ControlPlaneName != ""`: `Get(namespace, ControlPlaneName)`.
- Else: `List` CoderControlPlanes in namespace and require exactly 1.
3. Read `controlPlane.Status.URL` and parse as `*url.URL`.
- If empty/unparseable and `DefaultCoderURL != nil`, use fallback.
- Else return a clear error (`ServiceUnavailable` / `BadRequest`).
4. Construct `codersdk.New(url)` and set the fixed session token + HTTP
timeout.
RBAC impact (cluster): aggregated-apiserver service account must be able
to `get/list/watch` `codercontrolplanes` across namespaces.
---
## 3) Implement conversion layer (codersdk ↔ aggregated API types)
**Owner:** 1–2 agents
Create `internal/aggregated/convert/` with pure conversion functions.
### 3.1 Templates
File: `internal/aggregated/convert/template.go`
```go
func TemplateToK8s(namespace string, t codersdk.Template) *aggregationv1alpha1.CoderTemplate {
// Name := coder.BuildTemplateName(t.OrganizationName, t.Name) // "<org>.<template-name>"
// Namespace := namespace (CoderControlPlane namespace)
// UID := t.ID.String()
// ResourceVersion := strconv.FormatInt(t.UpdatedAt.UnixNano(), 10)
// Spec.Organization := t.OrganizationName
// Status fields from t
}
func TemplateCreateRequestFromK8s(obj *aggregationv1alpha1.CoderTemplate, templateName string) (codersdk.CreateTemplateRequest, error) {
// templateName parsed from metadata.name "<org>.<template-name>"
// parse obj.Spec.VersionID (uuid)
// map display name, description, icon
}
```
### 3.2 Workspaces
File: `internal/aggregated/convert/workspace.go`
```go
func WorkspaceToK8s(namespace string, w codersdk.Workspace) *aggregationv1alpha1.CoderWorkspace {
// Name := coder.BuildWorkspaceName(w.OrganizationName, w.OwnerName, w.Name) // "<org>.<user>.<workspace-name>"
// Namespace := namespace (CoderControlPlane namespace)
// UID := w.ID.String()
// Spec.Organization := w.OrganizationName
// Status includes owner, template, latest build info, last used, etc.
// Spec.Running inferred from latest build transition/status
}
func WorkspaceCreateRequestFromK8s(obj *aggregationv1alpha1.CoderWorkspace, workspaceName string, templateID uuid.UUID) (codersdk.CreateWorkspaceRequest, error) {
// requires workspaceName (parsed from metadata.name "<org>.<user>.<workspace-name>")
// requires templateID
// optional TemplateVersionID
// optional TTL/autostart
}
```
Add unit tests for converters (no network).
---
## 4) Replace hardcoded storage with codersdk-backed storage
**Owner:** 2–3 agents (templates vs workspaces can be parallel)
### 4.1 Shared storage base patterns
Keep the defensive style and interface assertions.
Add new fields to both storages:
- `provider coder.ClientProvider` (resolves `*codersdk.Client` from the
request namespace / control plane)
Update constructors to accept dependencies:
- `storage.NewWorkspaceStorage(provider coder.ClientProvider)
*WorkspaceStorage`
- `storage.NewTemplateStorage(provider coder.ClientProvider)
*TemplateStorage`
Namespace semantics:
- Keep `NamespaceScoped() == true`.
- Treat `genericapirequest.NamespaceValue(ctx)` as the
**CoderControlPlane namespace**.
- For v1, require `namespace != ""` for all operations (including
`List`). If the request namespace is empty (cluster-wide list), return a
clear `BadRequest` explaining that `-A/--all-namespaces` is not
supported yet.
- (Optional follow-up) Add cross-namespace list support by listing all
`CoderControlPlane` objects cluster-wide and aggregating results.
### 4.2 TemplateStorage: implement CRUD
File: `internal/aggregated/storage/template.go`
Implement interfaces:
- `rest.Getter`
- `rest.Lister`
- `rest.Creater`
- `rest.GracefulDeleter`
- (Optional) `rest.Updater` for meta updates using `UpdateTemplateMeta`
Concrete mapping (namespaced; namespace == control plane namespace; name
format `"<org>.<template-name>"`):
- In every method, resolve the backend client using the request
namespace:
- `ns := genericapirequest.NamespaceValue(ctx)`
- `sdk, err := provider.ClientForNamespace(ctx, ns)`
- **List**
- If `ns == ""`, return `BadRequest` (v1 does not support
`-A/--all-namespaces`).
- `tpls, err := sdk.Templates(ctx, codersdk.TemplateFilter{})`.
- Convert each template to a K8s object:
- `metadata.namespace = ns`
- `metadata.name = coder.BuildTemplateName(t.OrganizationName, t.Name)`
- `spec.organization = t.OrganizationName`
- **Get**
- Parse `{name}` using `coder.ParseTemplateName` → `(orgName,
templateName)`.
- Resolve org: `org, err := sdk.OrganizationByName(ctx, orgName)`.
- Fetch template: `tpl, err := sdk.TemplateByName(ctx, org.ID,
templateName)`.
- Return `convert.TemplateToK8s(ns, tpl)`.
- **Create**
- Parse `obj.Name` → `(orgName, templateName)`.
- Validate `obj.Spec.Organization` (if set) matches `orgName` (else set
it).
- Resolve org: `org, err := sdk.OrganizationByName(ctx, orgName)`.
- Build `codersdk.CreateTemplateRequest` from spec + parsed
templateName, then `created, err := sdk.CreateTemplate(ctx, org.ID,
req)`.
- Return `convert.TemplateToK8s(ns, created)`.
- **Delete**
- Parse `{name}` → `(orgName, templateName)`.
- Resolve org; fetch template by name; call `sdk.DeleteTemplate(ctx,
tpl.ID)`.
- Return `&metav1.Status{Status: "Success"}` (or the deleted object).
### 4.3 WorkspaceStorage: implement CRUD + start/stop via Update
File: `internal/aggregated/storage/workspace.go`
Implement interfaces:
- `rest.Getter`
- `rest.Lister`
- `rest.Creater`
- `rest.Updater`
- `rest.GracefulDeleter`
Concrete mapping (namespaced; namespace == control plane namespace; name
format `"<org>.<user>.<workspace-name>"`):
- In every method, resolve the backend client using the request
namespace:
- `ns := genericapirequest.NamespaceValue(ctx)`
- `sdk, err := provider.ClientForNamespace(ctx, ns)`
- **List**
- If `ns == ""`, return `BadRequest` (v1 does not support
`-A/--all-namespaces`).
- `wres, err := sdk.Workspaces(ctx, codersdk.WorkspaceFilter{})`.
- Convert each workspace to a K8s object:
- `metadata.namespace = ns`
- `metadata.name = coder.BuildWorkspaceName(w.OrganizationName,
w.OwnerName, w.Name)`
- `spec.organization = w.OrganizationName`
- **Get**
- Parse `{name}` using `coder.ParseWorkspaceName` → `(orgName, userName,
workspaceName)`.
- Fetch workspace: `ws, err := sdk.WorkspaceByOwnerAndName(ctx,
userName, workspaceName, codersdk.WorkspaceOptions{})`.
- If `ws.OrganizationName != orgName`, return NotFound.
- Return `convert.WorkspaceToK8s(ns, ws)`.
- **Create**
- Parse `obj.Name` → `(orgName, userName, workspaceName)`.
- Validate `obj.Spec.Organization` (if set) matches `orgName` (else set
it).
- Require `obj.Spec.TemplateName`.
- Resolve org: `org, err := sdk.OrganizationByName(ctx, orgName)`.
- Resolve template in that org: `tpl, err := sdk.TemplateByName(ctx,
org.ID, obj.Spec.TemplateName)`.
- Build `codersdk.CreateWorkspaceRequest`:
- `Name = workspaceName`
- `TemplateID = tpl.ID` (or `TemplateVersionID` if provided)
- optional TTL/autostart from spec
- Call `created, err := sdk.CreateUserWorkspace(ctx, userName, req)`.
- If `.spec.running == false`, immediately queue a stop build:
- `sdk.CreateWorkspaceBuild(ctx, created.ID,
codersdk.CreateWorkspaceBuildRequest{Transition: stop,
TemplateVersionID: created.LatestBuild.TemplateVersionID})`
- Return `convert.WorkspaceToK8s(ns, created)`.
- **Update** (key behavior)
- Parse `{name}` → `(orgName, userName, workspaceName)` and fetch
current workspace by owner+name.
- Decode desired object via `objInfo.UpdatedObject(ctx, oldObj)`.
- Forbid changing identity fields (`metadata.name`, `spec.organization`,
`spec.templateName`).
- If `running` changes, queue build transition start/stop.
- If TTL/autostart changed, call `UpdateWorkspaceTTL` /
`UpdateWorkspaceAutostart`.
- Return the latest workspace representation.
- **Delete**
- Parse `{name}` → `(orgName, userName, workspaceName)`; fetch
workspace; if org mismatch return NotFound; queue build:
- `sdk.CreateWorkspaceBuild(ctx, ws.ID,
codersdk.CreateWorkspaceBuildRequest{Transition: delete,
TemplateVersionID: ws.LatestBuild.TemplateVersionID})`
- Return `&metav1.Status{Status: "Success"}` (or the deleted object).
### 4.4 Defensive programming requirements
- `assertion failed:` on nil storage, nil ctx, missing required spec
fields.
- Validate composite name formats (`<org>.<template-name>`,
`<org>.<user>.<workspace-name>`) and required spec fields early with
clear error messages.
### 4.5 Update existing tests impacted by the refactor
- Rewrite or replace `internal/aggregated/storage/storage_test.go` (it
currently assumes hardcoded maps and zero-arg constructors).
- Preferred: replace with the `coderdtest` integration tests described
in §7.2.
- Alternative (faster): use an `httptest.Server` that implements only
the required `/api/v2/...` endpoints, similar to
`internal/coderbootstrap/client_test.go`.
- Update `internal/app/apiserverapp/apiserverapp_test.go` to pass a
dummy `coder.ClientProvider` into `NewAPIGroupInfo` once its signature
changes.
---
## 5) Wire storage dependencies into aggregated-apiserver bootstrap
**Owner:** 1 agent
### 5.1 Add apiserver options
File: `internal/app/apiserverapp/apiserverapp.go`
- Introduce:
```go
type Options struct {
// SecureServingPort is used when Listener is nil.
// Default: DefaultSecureServingPort.
SecureServingPort int
// Listener is optional but allows tests to bind to 127.0.0.1:0 and then
// discover the chosen port for APIService Endpoints.
Listener net.Listener
// KubeConfig is used to build a Kubernetes client for resolving
// coder.com/v1alpha1 CoderControlPlane objects.
// If nil, use the standard in-cluster / KUBECONFIG discovery.
KubeConfig *rest.Config
// Optional selector when multiple CoderControlPlanes exist in a namespace.
ControlPlaneName string
// Optional fallback when the control plane status URL is not available.
DefaultCoderURL string
// Coder auth.
CoderSessionToken string
CoderRequestTimeout time.Duration
}
func RunWithOptions(ctx context.Context, opts Options) error
```
- `Run(ctx)` can remain as a thin wrapper that reads env vars for
backwards compatibility, or we can update callsites to always use
`RunWithOptions`.
### 5.2 `NewAPIGroupInfo` must accept storage deps
Change signature to accept a `coder.ClientProvider` (or a small dep
struct containing it), and update:
- `internal/app/apiserverapp/apiserverapp_test.go`
Example:
```go
func NewAPIGroupInfo(scheme *runtime.Scheme, codecs serializer.CodecFactory, provider coder.ClientProvider) (*genericapiserver.APIGroupInfo, error) {
apiGroupInfo.VersionedResourcesStorageMap[...]=map[string]rest.Storage{
"coderworkspaces": storage.NewWorkspaceStorage(provider),
"codertemplates": storage.NewTemplateStorage(provider),
}
}
```
---
## 6) CLI flag plumbing
**Owner:** 1 agent
File: `app_dispatch.go`
Add flags to the existing `FlagSet` (define them regardless of app mode;
only enforce for aggregated-apiserver):
- `--coder-session-token` (admin token for the backing coderd)
- `--coder-request-timeout` (default `30s`)
- `--coder-url` (optional fallback when the namespace’s
CoderControlPlane has no `status.url`)
- `--coder-control-plane-name` (optional selector when multiple control
planes exist in a namespace)
Also document/validate the required object name formats:
- Templates: `<org>.<template-name>`
- Workspaces: `<org>.<user>.<workspace-name>`
Update the aggregated-apiserver dispatch to call
`apiserverapp.RunWithOptions(ctx, opts)`.
Update tests in `main_test.go` if signature changes (the dispatch tests
currently monkeypatch `runAggregatedAPIServerApp func(context.Context)
error`).
---
## 7) Testing strategy
### 7.1 Unit tests (fast)
**Owner:** 1 agent
- `internal/aggregated/convert/*_test.go`:
- Validate conversion for templates/workspaces.
- `internal/aggregated/coder/errors_test.go`:
- Validate mapping of `codersdk.NewTestError` →
`apierrors.IsNotFound/IsForbidden/...`.
### 7.2 Storage integration tests with `coderdtest` (primary correctness
signal)
**Owner:** 1–2 agents
Add tests in `internal/aggregated/storage/*_integration_test.go` (or
similar) that:
(Imports you will need once you add these tests:
`github.com/coder/coder/v2/coderd/coderdtest` and likely
`github.com/coder/coder/v2/provisioner/echo`.
Adding these will expand `vendor/` after `make vendor` / `make
verify-vendor`.)
1. Starts in-memory coderd:
- `client := coderdtest.New(t,
&coderdtest.Options{IncludeProvisionerDaemon: true})`
- `coderdtest.CreateFirstUser(t, client)` to authenticate `client`.
2. Resolve/ensure an organization to use in `spec.organization`:
- `org, err := client.OrganizationByName(ctx, "default")` (or
`CreateOrganization` if needed).
3. Creates a template version + template using coderdtest helpers:
- `version := coderdtest.CreateTemplateVersion(t, client, orgID,
&echo.Responses{...})`
- `coderdtest.AwaitTemplateVersionJobCompleted(t, client, version.ID)`
- `template := coderdtest.CreateTemplate(t, client, orgID, version.ID)`
4. Instantiate a test `coder.ClientProvider` that returns the
authenticated `client` for a chosen control-plane namespace (e.g.
`coder-123`).
5. Exercises the REST methods directly:
- Use the composite object name formats from “Chosen semantics”:
- Template object name: `"<org>.<template-name>"`
- Workspace object name: `"<org>.<user>.<workspace-name>"`
- Ensure the objects you pass to `Create/Update` include
`metadata.namespace = <control-plane-namespace>` and required spec
fields (`spec.organization`, `spec.templateName`, etc.).
- Template: `Create` → `Get` → `List` → `Delete`
- Workspace: `Create` → `Get` → `List` → `Update` (toggle running) →
`Delete`
Ensure contexts use a namespaced request context, e.g.
`genericapirequest.WithNamespace(context.Background(), "coder-123")`.
### 7.3 End-to-end: envtest + APIService proxy (stretch but matches
requirement)
**Owner:** 1–2 agents
Goal: route requests through envtest kube-apiserver’s aggregation layer.
Implementation suggestion:
- Put this in a dedicated test package (e.g.
`internal/app/apiserverapp/e2e_test.go` or `internal/aggregated/e2e/`)
so it can spin up its **own** envtest environment without inheriting the
controller suite’s CRD installation.
- Use `envtest.Environment{}` with `CRDDirectoryPaths` empty (or
pointing to a directory that excludes the `aggregation.coder.com_*`
CRDs).
Steps:
1. Start envtest **without installing** the `aggregation.coder.com` CRDs
(otherwise CRDs “own” the group and proxying won’t work).
2. Start `apiserverapp` on a dynamic port (listener `127.0.0.1:0`).
3. In envtest cluster, create:
- Namespace `coder-k8s-system` (or similar)
- Service `coder-k8s-aggregated-apiserver` in that namespace
- Endpoints pointing to `127.0.0.1:<port>`
- `apiregistration.k8s.io/v1 APIService` named
`v1alpha1.aggregation.coder.com` with:
- `spec.service` pointing to that service
- `spec.insecureSkipTLSVerify: true` (test-only)
4. Use a dynamic client built from envtest’s `rest.Config` to call:
- `POST
/apis/aggregation.coder.com/v1alpha1/namespaces/<control-plane-namespace>/coderworkspaces`
- request body includes `metadata.name: "<org>.<user>.<workspace-name>"`
and `spec.organization: "<org>"`
- verify side effects in coderdtest (workspace exists)
Implementation detail: gate this test with a build tag or `t.Skip` if
the envtest kube-apiserver does not support aggregation in this
environment.
---
## 8) Validation / CI readiness
**Owner:** 1 agent (final pass)
Run, in order:
- `make verify-vendor` (especially after adding `coderdtest` test deps)
- `make test`
- `make build`
- `make lint`
If API structs changed:
- `make codegen`
- `make manifests`
---
# Agent task breakdown (parallel execution)
1. **Agent A (Types/OpenAPI):** Section 1 + update OpenAPI definitions.
2. **Agent B (Backend helpers):** Section 2 (config/names/errors +
control-plane client provider).
3. **Agent C (Conversion):** Section 3 + unit tests.
4. **Agent D (Template storage):** Section 4.2 + integration tests.
5. **Agent E (Workspace storage):** Section 4.3 + integration tests.
6. **Agent F (Bootstrap/flags/RBAC):** Sections 5–6 (and update
`deploy/` manifests + RBAC).
7. **Agent G (Envtest proxy E2E):** Section 7.3.
8. **Agent H (Validation + polish):** Section 8.
</details>
---
_Generated with `mux` • Model: `anthropic:claude-opus-4-6` • Thinking:
`xhigh` • Cost: `$0.86`_
<!-- mux-attribution: model=anthropic:claude-opus-4-6 thinking=xhigh
costs=0.86 -->1 parent fb21a66 commit e142bce
33 files changed
Lines changed: 5225 additions & 1139 deletions
File tree
- api/aggregation/v1alpha1
- config/crd/bases
- docs/reference/api
- internal
- aggregated
- coder
- convert
- storage
- app/apiserverapp
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
8 | 17 | | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
9 | 21 | | |
10 | 22 | | |
11 | 23 | | |
12 | 24 | | |
13 | | - | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
14 | 33 | | |
| 34 | + | |
15 | 35 | | |
16 | 36 | | |
17 | 37 | | |
18 | 38 | | |
19 | 39 | | |
20 | 40 | | |
21 | 41 | | |
| 42 | + | |
22 | 43 | | |
23 | 44 | | |
24 | 45 | | |
| |||
39 | 60 | | |
40 | 61 | | |
41 | 62 | | |
42 | | - | |
43 | | - | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
44 | 75 | | |
45 | 76 | | |
46 | 77 | | |
47 | 78 | | |
48 | | - | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
49 | 86 | | |
50 | 87 | | |
51 | 88 | | |
| |||
54 | 91 | | |
55 | 92 | | |
56 | 93 | | |
| 94 | + | |
57 | 95 | | |
58 | 96 | | |
59 | 97 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
6 | 10 | | |
7 | 11 | | |
8 | 12 | | |
| |||
15 | 19 | | |
16 | 20 | | |
17 | 21 | | |
18 | | - | |
19 | | - | |
20 | | - | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
21 | 27 | | |
22 | 28 | | |
23 | 29 | | |
24 | 30 | | |
25 | | - | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
26 | 38 | | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
27 | 63 | | |
28 | 64 | | |
29 | 65 | | |
30 | 66 | | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
31 | 84 | | |
32 | 85 | | |
33 | 86 | | |
34 | 87 | | |
35 | | - | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
36 | 95 | | |
37 | 96 | | |
38 | 97 | | |
| |||
Lines changed: 34 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
21 | 23 | | |
22 | 24 | | |
23 | 25 | | |
| |||
39 | 41 | | |
40 | 42 | | |
41 | 43 | | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
42 | 54 | | |
43 | | - | |
44 | | - | |
| 55 | + | |
| 56 | + | |
45 | 57 | | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
46 | 62 | | |
47 | | - | |
| 63 | + | |
| 64 | + | |
48 | 65 | | |
49 | 66 | | |
50 | 67 | | |
51 | 68 | | |
| 69 | + | |
| 70 | + | |
52 | 71 | | |
53 | | - | |
54 | | - | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
55 | 83 | | |
56 | 84 | | |
57 | 85 | | |
| |||
Lines changed: 35 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
21 | 23 | | |
22 | 24 | | |
23 | 25 | | |
| |||
39 | 41 | | |
40 | 42 | | |
41 | 43 | | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
42 | 49 | | |
43 | | - | |
| 50 | + | |
44 | 51 | | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
45 | 63 | | |
46 | 64 | | |
47 | 65 | | |
48 | 66 | | |
49 | 67 | | |
50 | 68 | | |
51 | 69 | | |
52 | | - | |
53 | | - | |
54 | 70 | | |
55 | 71 | | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
56 | 87 | | |
57 | 88 | | |
58 | 89 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
17 | 22 | | |
18 | 23 | | |
19 | 24 | | |
20 | 25 | | |
21 | 26 | | |
22 | | - | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
23 | 33 | | |
24 | 34 | | |
25 | 35 | | |
26 | 36 | | |
27 | | - | |
28 | | - | |
29 | 37 | | |
0 commit comments