🤖 feat: expose template source files via CoderTemplate spec.files (#69)

## Summary

Expose Coder template source files via the aggregated API by adding
`spec.files` (a `map[string]string` of relative paths → UTF-8 contents)
to `CoderTemplate`. This enables `kubectl get codertemplate <org>.<name>
-o yaml` to include template source, and enables GitOps controllers to
create/update templates by specifying source files inline.

## Background

The aggregated API server already exposes `CoderTemplate` resources with
template metadata, but users and GitOps tooling couldn't view or manage
the template's actual source tree (Terraform files, Dockerfiles, etc.)
through the Kubernetes API. This change bridges that gap.

## Implementation

### API type (`api/aggregation/v1alpha1/types.go`)
- Added `Files map[string]string` field to `CoderTemplateSpec` with
`json:"files,omitempty"`
- Regenerated deepcopy via `make codegen`

### Storage helper (`internal/aggregated/storage/template_files.go`)
- `fetchTemplateSourceFiles()`: downloads the active version's source
zip via `codersdk.TemplateVersion` → `codersdk.DownloadWithFormat`,
unpacks, validates paths/sizes, filters non-UTF8 files
- `buildSourceZip()`: creates deterministic zip from file map for upload
- Enforced safety limits: 20 MiB compressed, 40 MiB total extracted,
2000 files max, 2 MiB per file
- Full defensive programming: nil assertions, zip-slip path protection,
UTF-8 validation

### Storage Get/Create/Update
(`internal/aggregated/storage/template.go`)
- **Get**: populates `spec.files` from the active version's source
archive
- **List**: intentionally omits `spec.files` to keep responses small
- **Create with files**: builds zip → uploads → creates template version
→ creates template
- **Update with files**: diffs current vs desired files; if changed,
uploads new zip → creates version → promotes to active. If identical,
no-op (GitOps idempotency)
- **Update metadata**: reconciles `displayName`, `description`, `icon`
via `UpdateTemplateMeta`

### Convert helper (`internal/aggregated/convert/template.go`)
- Added `TemplateUpdateMetaRequestFromK8s()` for building metadata
update payloads

### Tests (`internal/aggregated/storage/storage_test.go`)
- Extended mock Coder server with file upload/download, template version
creation, metadata update, and active version promotion handlers
- Added tests: `TestTemplateStorageGetPopulatesSpecFiles`,
`TestTemplateStorageListOmitsSpecFiles`,
`TestTemplateStorageCreateWithFiles`,
`TestTemplateStorageUpdateWithChangedFiles`,
`TestTemplateStorageUpdateWithIdenticalFilesIsNoOp`,
`TestTemplateStorageUpdateMetadata`
- Updated existing update tests to match relaxed immutability (metadata
now mutable)

### Docs
- Regenerated `docs/reference/api/codertemplate.md` to include the new
field

## Validation

- `make verify-vendor` ✅
- `make build` ✅
- `make test` ✅
- `make lint` ✅
- `make docs-check` ✅

## Risks

- **Non-atomic update**: metadata patch and source update are sequential
codersdk operations. If source step fails after metadata succeeds, the
template may have updated metadata but old source. This is inherent to
the codersdk API design.
- **Memory pressure**: GET requests now download + unpack source
archives inline. Bounded by safety limits (20 MiB compressed) but
concurrent GETs on large templates could increase memory usage. A future
`/source` subresource could mitigate this.

---

<details>
<summary>📋 Implementation Plan</summary>

# Plan: Expose Coder template source files via
`CoderTemplate.spec.files`

## Context / Why
Today the aggregated API server exposes `CoderTemplate` and
`CoderWorkspace` as Kubernetes-native resources, but it only returns
**template metadata** (name, description, version IDs, etc.). In the
Coder UI, users can view a template version’s source tree (Terraform,
Dockerfile, etc.).

We want `kubectl get codertemplate <org>.<name> -o yaml` to include the
template’s source files so downstream tooling/UIs can display the same
information via the aggregated API.

**Chosen approach (Option A):** Add an inline `spec.files` map (path →
file contents) to `CoderTemplate` and have the aggregated API server
populate it from the active template version’s source archive stored in
Coder.

## Evidence (repo + SDK)
- Aggregated API types:
- `api/aggregation/v1alpha1/types.go` defines
`CoderTemplateSpec`/`Status` but has no source fields.
- `api/aggregation/v1alpha1/zz_generated.deepcopy.go` is generated and
will need regen after API changes.
- Aggregated API storage:
- `internal/aggregated/storage/template.go` `Get()` calls Coder
(`sdk.TemplateByName`) and returns `convert.TemplateToK8s(...)`.
- `List()` calls `sdk.Templates(...)` and converts each template; we
should avoid per-template source downloads here.
- Conversion layer:
- `internal/aggregated/convert/template.go` maps
`codersdk.Template.ActiveVersionID` → `CoderTemplate.spec.versionID`.
- Coder SDK supports fetching template version + archive:
- `(*codersdk.Client).TemplateVersion(ctx, id)` in
`vendor/github.com/coder/coder/v2/codersdk/templateversions.go`.
- `TemplateVersion.Job.FileID` (source archive ID) in
`vendor/.../codersdk/provisionerdaemons.go`.
- `(*codersdk.Client).DownloadWithFormat(ctx, fileID,
codersdk.FormatZip)` in `vendor/.../codersdk/files.go`.
- API reference docs are generated and include `CoderTemplate` fields:
  - `docs/reference/api/codertemplate.md` (generated).

## Implementation details

### 1) API: add `spec.files` field (text-only)
**File:** `api/aggregation/v1alpha1/types.go`

Add a new optional field to `CoderTemplateSpec`:

```go
// CoderTemplateSpec defines the desired state of a CoderTemplate.
type CoderTemplateSpec struct {
    // ...existing fields...

    // Files is the template source tree for the template’s *active* version.
    //
    // - Keys are relative, slash-delimited paths (e.g. "main.tf", "modules/vpc/main.tf").
    // - Values are UTF-8 file contents.
    // - Callers may set this field on CREATE/UPDATE to manage template source (GitOps).
    // - The aggregated API server populates this field on GET requests.
    // - This field is intentionally omitted from LIST responses to keep them small.
    // - Binary / non-UTF8 files are rejected or skipped (future: add encoding support).
    Files map[string]string `json:"files,omitempty"`
}
```

Notes/constraints to bake into the implementation:
- Keep this **best-effort** and bounded: do not allow a single request
to download/unpack arbitrarily large archives.
- If we later want binary support, we should migrate to a richer type
(e.g. `map[string]CoderTemplateFile{content, encoding}`) rather than
stuffing base64 into a string.

**Generated artifacts:**
- Regenerate `api/aggregation/v1alpha1/zz_generated.deepcopy.go` (via
`make codegen`).
- Regenerate `docs/reference/api/codertemplate.md` (via `make
docs-reference` or the repo’s canonical docs target).

### 2) Storage: populate `spec.files` in `TemplateStorage.Get()` only
**File:** `internal/aggregated/storage/template.go`

Change `Get()` from returning `convert.TemplateToK8s(...)` directly to:
1. Convert template → k8s object.
2. Fetch active template version (`sdk.TemplateVersion(ctx,
template.ActiveVersionID)`).
3. Download the archive from the files API (`sdk.DownloadWithFormat(...,
codersdk.FormatZip)`).
4. Unzip and build the map.
5. Set `obj.Spec.Files = files`.

Shape:

```go
obj := convert.TemplateToK8s(namespace, template)

files, err := fetchTemplateSourceFiles(ctx, sdk, template.ActiveVersionID)
if err != nil {
    return nil, err
}
obj.Spec.Files = files

return obj, nil
```

#### Helper: unzip + safety limits
Add a helper function (either near `TemplateStorage` or in a small
internal helper file like
`internal/aggregated/storage/template_files.go`) to keep `Get()`
readable:

```go
const (
    // NOTE: These limits gate how much data we’ll pull back and inline into a single
    // Kubernetes API response. Keep them bounded to avoid memory/latency issues.
    maxTemplateSourceZipBytes               = 20 << 20 // 20 MiB compressed archive cap
    maxTemplateSourceTotalUncompressedBytes = 40 << 20 // 40 MiB across all extracted files
    maxTemplateSourceFiles                  = 2000     // cap number of files
    maxTemplateSourceFileBytes              = 2 << 20  // 2 MiB per file
)

func fetchTemplateSourceFiles(
    ctx context.Context,
    sdk *codersdk.Client,
    versionID uuid.UUID,
) (map[string]string, error) {
    // 1) sdk.TemplateVersion(...)
    // 2) sdk.DownloadWithFormat(..., codersdk.FormatZip)
    // 3) validate size + content-type
    // 4) unzip and read each file with limits
    // 5) validate paths are relative + clean (no ../, no leading /)
    // 6) utf8.Valid(...) check; skip invalid
}
```

<details>
<summary>Trade-offs of raising limits (e.g. 20–40 MiB
archives)</summary>

- `codersdk.DownloadWithFormat` buffers the entire archive in memory;
larger limits increase per-request allocations and GC pressure.
- Unzipped content is returned inline as JSON/YAML. A 40 MiB extracted
template means a 40+ MiB API response, which can:
  - be slow / time out through the kube-apiserver aggregation proxy,
  - be expensive for `kubectl`/GitOps controllers to render/diff,
  - increase OOM risk under concurrent GETs.
- GitOps CREATE/UPDATE has an additional constraint: kube-apiserver
enforces a max request body size (flag `--max-request-bytes`), so very
large YAML manifests may be rejected before reaching the aggregated API
server.

If we need to support very large templates, we should consider:
- making these limits configurable via aggregated-apiserver flags,
and/or
- a future `/source` subresource that streams zip/tar instead of
inlining all file contents.

</details>

Defensive programming expectations:
- Assert required inputs are non-nil/non-empty.
- Explicitly error if Coder returns an unexpected content-type.
- Reject pathological archives (too many files, total zip bytes too
large, per-file too large).
- Protect against zip-slip style paths even though we’re not writing to
disk.

#### Keep LIST fast
**File:** `internal/aggregated/storage/template.go`

Do **not** populate `spec.files` in `List()`:
- It would multiply Coder API calls by N templates.
- It would make `kubectl get codertemplates -o yaml` huge.

So `List()` continues to call `convert.TemplateToK8s(...)` and returns
objects with `spec.files` unset.

### 3) GitOps: support creating/updating templates from `spec.files`
**Files:** `internal/aggregated/storage/template.go`,
`internal/aggregated/convert/template.go`

Goal: enable GitOps controllers (ArgoCD/Flux) and `kubectl apply` to
create and reconcile Coder templates from YAML by treating `spec.files`
as the desired template source of truth.

#### CREATE behavior (template does not exist yet)
In `TemplateStorage.Create(...)`:
- If `spec.files` is provided (len > 0):
1. Validate and normalize the file map (paths are clean/relative; UTF-8;
enforce size limits).
  2. Build a zip archive in-memory from `spec.files`.
3. Upload archive to Coder: `sdk.Upload(ctx, codersdk.ContentTypeZip,
reader)` → `fileID`.
  4. Create a new template version (unattached):
- `sdk.CreateTemplateVersion(ctx, org.ID,
codersdk.CreateTemplateVersionRequest{ StorageMethod: file, FileID:
fileID, Provisioner: terraform })`
5. Create the template using that version as the initial active version:
- `sdk.CreateTemplate(ctx, org.ID, codersdk.CreateTemplateRequest{ Name:
templateName, VersionID: templateVersion.ID, ...meta })`
- Else (no files): keep existing behavior that creates a template from a
pre-existing `spec.versionID`.

#### UPDATE behavior (reconcile template to match YAML)
Expand `TemplateStorage.Update(...)` beyond the current “`spec.running`
only” legacy behavior:

- **Metadata reconcile**
- Allow `spec.displayName`, `spec.description`, and `spec.icon` to be
updated via `sdk.UpdateTemplateMeta(...)`.
- Keep `metadata.name` + `spec.organization` immutable (still derived
from name).

- **Source reconcile (`spec.files`)**
  - If the update request includes `spec.files`:
1. Fetch current source for the template’s active version (use the same
`fetchTemplateSourceFiles` helper).
2. If desired files are byte-for-byte equal to current files: **no-op**
(prevents creating a new template version every sync).
    3. Otherwise:
       - Zip desired files → upload (`/api/v2/files`).
- Create a new template version attached to the template (`TemplateID:
template.ID`).
- Promote it to active via `sdk.UpdateActiveTemplateVersion(template.ID,
codersdk.UpdateActiveTemplateVersion{ID: newVersion.ID})`.

- **Patch / server-side apply compatibility**
- Verify `kubectl apply` and a JSON merge-patch update flow work
end-to-end.
- Confirm server-side apply / merge-patch work end-to-end.
`TemplateStorage` already satisfies `rest.Patcher` (Getter+Updater), but
we should verify managedFields/resourceVersion behavior in practice.

<details>
<summary>Notes on idempotency + drift</summary>

- The “compare desired files to current files” step is important;
without it, GitOps controllers that re-apply manifests could create a
new Coder template version on every sync loop.
- For very large templates, fetching + unpacking current source during
UPDATE can be expensive. If this becomes a bottleneck, we can add a
status-level checksum (or reuse Coder’s `job.file_id`) to short-circuit
equality checks.

</details>

### 4) Tests: extend mock Coder server to serve template source archive
**File:** `internal/aggregated/storage/storage_test.go`

The existing `newMockCoderServer()` already supports template CRUD plus:
- `GET /api/v2/templateversions/{id}`

Extend it to also support GitOps + source flows:
- `GET /api/v2/files/{fileID}?format=zip` (download source archive)
- `POST /api/v2/files` (upload source archive)
- `POST /api/v2/organizations/{orgID}/templateversions` (create template
version)
- `PATCH /api/v2/templates/{templateID}` (update template metadata)
- `PATCH /api/v2/templates/{templateID}/versions` (promote active
version)

Implementation approach:
- Keep an in-memory `filesByID map[uuid.UUID][]byte` in mock state and
seed it with a deterministic zip for the starter template version.
- Implement upload to stash request bytes and return a new UUID in the
`hash` JSON field.
- Implement create template version to persist
`TemplateVersion.Job.FileID` from the request’s `FileID` and optionally
attach it to a template when `TemplateID` is set.

Add/extend tests:
- `Get()` returns `spec.files` with expected contents (e.g. `main.tf`).
- `List()` omits `spec.files`.
- CREATE with `spec.files` succeeds and subsequent GET returns those
files.
- UPDATE with changed `spec.files` creates a new template version +
promotes it; UPDATE with identical files is a no-op (GitOps
idempotency).
- Failure-mode tests for archive/file limits.

### 5) Docs + examples
- Run docs regeneration so `docs/reference/api/codertemplate.md`
includes the new `spec.files` field.
- Consider adding a short `docs/` page or README note explaining:
- `spec.files` is the desired template source for GitOps. The server
returns it on GET but omits it on LIST.
- size/encoding limitations and current hard limits (20MiB zip / 40MiB
extracted / 2MiB per file).
- GitOps caveats: kube-apiserver max request size may need tuning for
large manifests; `spec.versionID` reflects the active version and may
change when source changes (configure Argo/Flux ignore rules if needed).
- security implications (source code becomes readable to anyone with
RBAC `get` on `codertemplates`).

## Validation checklist (what to run in Exec mode)
- `make codegen`
- `make docs-reference` (or repo docs target)
- `make test`
- `make lint`

</details>

---
_Generated with `mux` • Model: `anthropic:claude-opus-4-6` • Thinking:
`xhigh` • Cost: `$0.92`_

<!-- mux-attribution: model=anthropic:claude-opus-4-6 thinking=xhigh
costs=0.92 -->

Author: ThomasK33

api/aggregation/v1alpha1/types.go

@@ -70,6 +70,15 @@ type CoderTemplateSpec struct {
 	Description string `json:"description,omitempty"`
 	Icon        string `json:"icon,omitempty"`
 
+	// Files is the template source tree for the active template version.
+	//
+	// Keys are slash-delimited relative paths (e.g. "main.tf").
+	// Values are UTF-8 file contents.
+	//
+	// Populated on GET; intentionally omitted from LIST to keep responses small.
+	// On CREATE/UPDATE with files, the server uploads source and creates a new template version.
+	Files map[string]string `json:"files,omitempty"`
+
 	// Running is a legacy flag retained temporarily for in-repo callers that still read template run-state directly.
 	Running bool `json:"running,omitempty"`
 }

api/aggregation/v1alpha1/zz_generated.deepcopy.go

@@ -18,6 +18,7 @@
 | `displayName` | string |  |
 | `description` | string |  |
 | `icon` | string |  |
+| `files` | object (keys:string, values:string) | Files is the template source tree for the active template version. Keys are slash-delimited relative paths (e.g. "main.tf"). Values are UTF-8 file contents. Populated on GET; intentionally omitted from LIST to keep responses small. On CREATE/UPDATE with files, the server uploads source and creates a new template version. |
 | `running` | boolean | Running is a legacy flag retained temporarily for in-repo callers that still read template run-state directly. |
 
 ## Status

docs/reference/api/codertemplate.md

@@ -72,3 +72,20 @@ func TemplateCreateRequestFromK8s(obj *aggregationv1alpha1.CoderTemplate, templa
 		Icon:        obj.Spec.Icon,
 	}, nil
 }
+
+// TemplateUpdateMetaRequestFromK8s builds a codersdk.UpdateTemplateMeta request.
+func TemplateUpdateMetaRequestFromK8s(obj *aggregationv1alpha1.CoderTemplate) codersdk.UpdateTemplateMeta {
+	if obj == nil {
+		panic("assertion failed: template object must not be nil")
+	}
+
+	displayName := obj.Spec.DisplayName
+	description := obj.Spec.Description
+	icon := obj.Spec.Icon
+
+	return codersdk.UpdateTemplateMeta{
+		DisplayName: &displayName,
+		Description: &description,
+		Icon:        &icon,
+	}
+}