Skip to content

Commit c864ea3

Browse files
igoramfclaude
andauthored
feat: Deco CRD and BuildReconciler for cfworkers (#5)
* feat: add DecoBuild CRD and BuildReconciler for cfworkers builds - DecoBuild CRD: represents a cfworkers build request; operator creates K8s Jobs - BuildReconciler: watches DecoBuild, generates S3 presigned URLs, creates Job - build/job.go: Job spec builder (cfworkers-builder image, env vars, TTL 24h) - build/s3presign.go: generates presigned URLs for logs/cache using aws-sdk-go-v2 - RBAC: batch/jobs + deco.sites/decobuilds permissions - Chart bumped to v0.3.0; cfworkers env vars injected from ExternalSecret * chore: go mod tidy — add aws-sdk-go-v2 go.sum entries * fix: gofmt formatting in build/job.go constants * fix: wire DecoBuild CRD and cfworkers env vars into Helm generator - Add deco.sites_decobuilds.yaml to config/crd/kustomization.yaml so helm-generator picks it up - Add cfworkers env vars block to helm-generator addEnvVarsToDeployment - Regenerate chart templates via make manifests helm * feat: replace DecoBuild CRD with Deco CR for cfworkers builds and previews Replaces the per-build DecoBuild CRD with a site-scoped Deco CR that owns both production and preview builds. The operator reconciles spec.build.source for production deploys and spec.previews.active[] for concurrent PR previews, fixing the concurrent PR overwrite bug. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: also set WORKER_NAME env var for backwards compat with existing builder images Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * revert: remove WORKER_NAME backwards compat, use DECO_SITE_NAME only Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: remove DecoBuild CRD — using Deco CR directly for cfworkers builds Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(operator): decouple reconciler from cfworkers via JobFactory The DecoReconciler no longer holds cfworkers-specific credentials (CfApiToken, CfAccountId, S3Config). A JobFactory function type is injected at startup, keeping the reconciler platform-agnostic. Future serving types just need a new factory wired in main.go. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(operator): move platform-specific constants into JobFactory BuilderImage, TTLSeconds, LogsBucket and CacheBucket are now fields on JobOpts/S3Config instead of hardcoded constants. The cfworkers factory in main.go owns these values. spec.build.builder in the CR still takes precedence over the platform default for BuilderImage. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(operator): support multiple serving types via factory registry Replace single JobFactory field with Factories map[string]JobFactory keyed by spec.serving.type. createJob looks up the factory by type and returns an error for unknown types. A new platform just registers its factory in main.go. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * rename: JobOpts → CfWorkersJobOpts, NewJob → NewCfWorkersJob Makes the cfworkers-specific types explicit in the build package so future platforms can add their own types alongside without ambiguity. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(operator): introduce build.Registry for platform dispatch Move serving-type dispatch out of the controller into build.Registry. The controller now holds a *build.Registry and calls registry.NewJob() without knowing about platforms. New platforms register via registry.Register() in main.go. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * rename: build/job.go → build/cfworkers.go Clarifies that this file is cfworkers-specific. Generic build helpers (registry, s3presign) stay at the package level; platform-specific implementations get their own file. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: bump version to 0.2.7 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: revert version to 0.2.6 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: move s3Region from values to secret Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add cfworkers build duration and count metrics Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address CI lint failures and move S3_REGION to secretKeyRef - Fix goconst: extract Succeeded/Failed as constants - Fix prealloc: pre-allocate newStatuses slice - Fix lll: break long factory func signature in cmd/main.go - Fix helm-generator to emit S3_REGION as secretKeyRef (key: s3-region) - Regenerate CRD template (alphabetical field ordering from controller-gen) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: pin CRD plural to 'decos' via +kubebuilder:resource:path=decos Without this annotation controller-gen auto-pluralises 'Deco' as 'decoes', creating a second deco.sites_decoes.yaml alongside the committed deco.sites_decos.yaml. envtest loads every yaml in config/crd/bases/ so both CRDs were registered, causing the BeforeSuite context deadline timeout. Also regenerates deco.sites_decos.yaml from current types (updated schema, alphabetical field ordering, adds previews/branchRef fields). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: regenerate helm chart CRD template from updated deco.sites_decos.yaml Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(build): replace Factory func type with Builder interface * feat(build): add CfWorkersConfig, cfWorkersBuilder, NewCloudflareFactory * refactor(controller): hold build.Builder interface; update default bucket names * refactor(main): remove CF/S3 specifics — wire via build.NewCloudflareFactory * refactor(build): unexport internal types NewCfWorkersJob, CfWorkersJobOpts, PresignedURLs, GeneratePresignedURLs * refactor(build): remove defaults test — buckets are env-configurable * chore: remove cfworkers_test.go — follow-up PR * refactor(build): remove hardcoded default bucket/image names — repo is public * feat(helm): move S3_REGION to values; add builderImage, s3LogsBucket, s3CacheBucket as plain values * refactor(helm): split S3 into dedicated s3.existingSecret — shared across build platforms * refactor(build): rename Registry to BuilderRegistry for clarity * fix(build): update compile-time check to BuilderRegistry after rename * feat(crd): add envs and secrets fields to DecoSpecBuild for custom Job env injection * refactor(build): read envs/secrets directly from Deco spec in newCfWorkersJob * fix(build): prealloc envFrom, make TTLSecondsAfterFinished configurable via CRD (default 10m) * fix(build): fix prealloc lint — use make+index instead of make+append for envFrom * refactor(s3): rename cacheBucket to artifactsBucket — build artifacts not just npm cache * chore: remove values-local.yaml from .gitignore * refactor: move artifactsBucket from s3 to cfworkers in chart values artifactsBucket is platform-specific (Cloudflare Workers), not shared S3 infrastructure, so it belongs under the cfworkers section. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: use Status().Patch() to avoid optimistic concurrency conflicts on status update Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: resolve helm-verify and lint CI failures - Regenerate CRDs with controller-gen v0.18.0 to match Makefile pin - Fix gofmt: remove extra alignment spaces in cfworkers.go - Add phaseRunning constant; replace all "Running" literals in deco_controller - Add condTypePodsNotified constant in decofile_controller - Add metricsNamespace/metricsSubsystemValkey constants in metrics.go - Add valkeyReservedDefault constant in namespace_controller - Exclude test files from goconst in .golangci.yml Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 99217c4 commit c864ea3

23 files changed

Lines changed: 1992 additions & 28 deletions

.golangci.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ linters:
2222
- unparam
2323
- unused
2424
settings:
25+
goconst:
26+
min-occurrences: 3
2527
revive:
2628
rules:
2729
- name: comment-spacings
@@ -36,6 +38,9 @@ linters:
3638
- dupl
3739
- lll
3840
path: internal/*
41+
- linters:
42+
- goconst
43+
path: _test\.go$
3944
paths:
4045
- third_party$
4146
- builtin$

api/v1alpha1/deco_types.go

Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
package v1alpha1
2+
3+
import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
4+
5+
// DecoSpec defines the desired state of a Deco workload.
6+
type DecoSpec struct {
7+
// Type is the workload type. site | server | admin | preview
8+
// +optional
9+
Type string `json:"type,omitempty"`
10+
11+
// Site is the site/repository name.
12+
// +kubebuilder:validation:Required
13+
Site string `json:"site"`
14+
15+
// Org is the GitHub organization or owner.
16+
// +kubebuilder:validation:Required
17+
Org string `json:"org"`
18+
19+
// Framework is the site framework. deno | tanstack | next | remix | static
20+
// +optional
21+
Framework string `json:"framework,omitempty"`
22+
23+
// Build describes the production build pipeline.
24+
// +optional
25+
Build *DecoSpecBuild `json:"build,omitempty"`
26+
27+
// Serving describes the runtime serving configuration.
28+
// +optional
29+
Serving *DecoSpecServing `json:"serving,omitempty"`
30+
31+
// Previews configures preview builds for this site.
32+
// Admin adds entries to Previews.Active on PR open and removes on PR close.
33+
// +optional
34+
Previews *DecoPreviewPolicy `json:"previews,omitempty"`
35+
}
36+
37+
// DecoEnvVar is a plain environment variable injected into the build Job.
38+
type DecoEnvVar struct {
39+
// Name is the environment variable name.
40+
// +kubebuilder:validation:Required
41+
Name string `json:"name"`
42+
// Value is the literal value.
43+
// +optional
44+
Value string `json:"value,omitempty"`
45+
}
46+
47+
// DecoSecretRef mounts all keys of a K8s Secret as environment variables in the build Job.
48+
type DecoSecretRef struct {
49+
// Name is the name of the K8s Secret in the same namespace as the Job.
50+
// +kubebuilder:validation:Required
51+
Name string `json:"name"`
52+
// Optional specifies whether the Secret must exist. Defaults to false.
53+
// +optional
54+
Optional *bool `json:"optional,omitempty"`
55+
}
56+
57+
// DecoSpecBuild describes the build pipeline for a workload.
58+
type DecoSpecBuild struct {
59+
// Type is the build mechanism. Currently only k8s-job is supported.
60+
// +optional
61+
Type string `json:"type,omitempty"`
62+
63+
// Source identifies the code revision to build.
64+
// Repository and owner come from spec.site and spec.org.
65+
Source DecoSpecBuildSource `json:"source"`
66+
67+
// Builder overrides the builder image (repository:tag).
68+
// +optional
69+
Builder string `json:"builder,omitempty"`
70+
71+
// Envs are additional plain environment variables injected into the build Job.
72+
// +optional
73+
Envs []DecoEnvVar `json:"envs,omitempty"`
74+
75+
// Secrets are K8s Secrets whose keys are mounted as environment variables in the build Job.
76+
// The secrets must exist in the same namespace as the Job (the site namespace).
77+
// +optional
78+
Secrets []DecoSecretRef `json:"secrets,omitempty"`
79+
80+
// TTLSecondsAfterFinished controls how long the Job is kept after completion.
81+
// Defaults to 600 (10 minutes) when not set.
82+
// +optional
83+
// +kubebuilder:validation:Minimum=0
84+
TTLSecondsAfterFinished *int32 `json:"ttlSecondsAfterFinished,omitempty"`
85+
}
86+
87+
// DecoSpecBuildSource identifies the code revision to build.
88+
type DecoSpecBuildSource struct {
89+
// CommitSha is the git commit SHA to build.
90+
// Updating this field triggers a new build.
91+
// +kubebuilder:validation:Required
92+
CommitSha string `json:"commitSha"`
93+
94+
// Production indicates whether this is a production deploy.
95+
// +optional
96+
Production bool `json:"production,omitempty"`
97+
98+
// BranchRef is the branch name for preview aliases (non-production only).
99+
// +optional
100+
BranchRef string `json:"branchRef,omitempty"`
101+
}
102+
103+
// DecoSpecServing describes the runtime serving configuration.
104+
type DecoSpecServing struct {
105+
// Type is the serving runtime. Drives both serving and build job selection.
106+
// Supported: cloudflare-worker | knative | deployment
107+
// +kubebuilder:validation:Required
108+
Type string `json:"type"`
109+
}
110+
111+
// DecoPreviewPolicy configures the preview system for this site.
112+
type DecoPreviewPolicy struct {
113+
// Type is the preview runtime. cloudflare-preview | statefulset | sandbox
114+
// +kubebuilder:validation:Required
115+
Type string `json:"type"`
116+
117+
// MaxActive is the maximum number of concurrent previews the operator will build.
118+
// Operator processes only the most recent MaxActive entries in Active.
119+
// +optional
120+
MaxActive int32 `json:"maxActive,omitempty"`
121+
122+
// TTL is the duration after which completed previews are eligible for cleanup (e.g. "48h").
123+
// +optional
124+
TTL string `json:"ttl,omitempty"`
125+
126+
// Active is the list of preview builds currently requested.
127+
// Admin adds entries on PR open, removes on PR close.
128+
// +optional
129+
Active []DecoPreviewRequest `json:"active,omitempty"`
130+
}
131+
132+
// DecoPreviewRequest identifies a single preview build request.
133+
type DecoPreviewRequest struct {
134+
// CommitSha is the git commit SHA to build.
135+
// +kubebuilder:validation:Required
136+
CommitSha string `json:"commitSha"`
137+
138+
// BranchRef is the branch or PR ref (used as the wrangler preview alias).
139+
// +kubebuilder:validation:Required
140+
BranchRef string `json:"branchRef"`
141+
142+
// PrId is the pull request ID, for tracking.
143+
// +optional
144+
PrId string `json:"prId,omitempty"`
145+
}
146+
147+
// DecoStatus defines the observed state of a Deco workload.
148+
type DecoStatus struct {
149+
// Build tracks the current production build lifecycle.
150+
// +optional
151+
Build *DecoStatusBuild `json:"build,omitempty"`
152+
153+
// Previews tracks the build status of each active preview.
154+
// +optional
155+
Previews []DecoPreviewStatus `json:"previews,omitempty"`
156+
}
157+
158+
// DecoStatusBuild tracks the production build lifecycle.
159+
type DecoStatusBuild struct {
160+
// Phase is the current build phase: Running | Succeeded | Failed
161+
// +optional
162+
Phase string `json:"phase,omitempty"`
163+
164+
// CommitSha is the commit currently being built (or last attempted).
165+
// +optional
166+
CommitSha string `json:"commitSha,omitempty"`
167+
168+
// LastBuiltCommit is the commit SHA of the last successful build.
169+
// +optional
170+
LastBuiltCommit string `json:"lastBuiltCommit,omitempty"`
171+
172+
// JobName is the K8s Job name for the current build.
173+
// +optional
174+
JobName string `json:"jobName,omitempty"`
175+
176+
// StartTime is when the current build started.
177+
// +optional
178+
StartTime *metav1.Time `json:"startTime,omitempty"`
179+
180+
// CompletionTime is when the current build finished.
181+
// +optional
182+
CompletionTime *metav1.Time `json:"completionTime,omitempty"`
183+
}
184+
185+
// DecoPreviewStatus tracks the build status of a single preview.
186+
type DecoPreviewStatus struct {
187+
CommitSha string `json:"commitSha"`
188+
BranchRef string `json:"branchRef"`
189+
PrId string `json:"prId,omitempty"`
190+
JobName string `json:"jobName,omitempty"`
191+
Phase string `json:"phase"`
192+
StartTime *metav1.Time `json:"startTime,omitempty"`
193+
CompletionTime *metav1.Time `json:"completionTime,omitempty"`
194+
}
195+
196+
// +kubebuilder:object:root=true
197+
// +kubebuilder:subresource:status
198+
// +kubebuilder:resource:path=decos
199+
// +kubebuilder:printcolumn:name="Site",type=string,JSONPath=`.spec.site`
200+
// +kubebuilder:printcolumn:name="Serving",type=string,JSONPath=`.spec.serving.type`
201+
// +kubebuilder:printcolumn:name="Commit",type=string,JSONPath=`.spec.build.source.commitSha`
202+
// +kubebuilder:printcolumn:name="Phase",type=string,JSONPath=`.status.build.phase`
203+
// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp`
204+
205+
// Deco is the Schema for the decos API.
206+
type Deco struct {
207+
metav1.TypeMeta `json:",inline"`
208+
metav1.ObjectMeta `json:"metadata,omitempty"`
209+
210+
Spec DecoSpec `json:"spec,omitempty"`
211+
Status DecoStatus `json:"status,omitempty"`
212+
}
213+
214+
// +kubebuilder:object:root=true
215+
216+
// DecoList contains a list of Deco.
217+
type DecoList struct {
218+
metav1.TypeMeta `json:",inline"`
219+
metav1.ListMeta `json:"metadata,omitempty"`
220+
Items []Deco `json:"items"`
221+
}
222+
223+
func init() {
224+
SchemeBuilder.Register(&Deco{}, &DecoList{})
225+
}

0 commit comments

Comments
 (0)