|
| 1 | +--- |
| 2 | +title: Aspire Kubernetes Operator Model |
| 3 | +description: Run, publish, and deploy KubeOps operators with Aspire Kubernetes environments. |
| 4 | +sidebar_position: 12 |
| 5 | +--- |
| 6 | + |
| 7 | +# Aspire Kubernetes Operator Model |
| 8 | + |
| 9 | +This document describes the implemented Aspire model for running, publishing, and deploying KubeOps operators. |
| 10 | + |
| 11 | +Aspire has two relevant execution modes: |
| 12 | + |
| 13 | +- **Run mode** starts the AppHost locally for development and debugging. |
| 14 | +- **Publish/deploy mode** generates and applies deployment artifacts for a target environment. |
| 15 | + |
| 16 | +KubeOps operators need both modes, but they need different behavior in each mode. |
| 17 | + |
| 18 | +## Design |
| 19 | + |
| 20 | +- Keep `aspire run` as local process orchestration. |
| 21 | +- Keep `aspire publish` and `aspire deploy` as deployment artifact and cluster deployment flows. |
| 22 | +- Reuse Aspire Kubernetes and AKS compute environments instead of inventing a separate KubeOps target model. |
| 23 | +- Make local CRD handling explicit, safe, and predictable. |
| 24 | +- Avoid automatic destructive changes to a developer's current cluster. |
| 25 | + |
| 26 | +## AppHost Shape |
| 27 | + |
| 28 | +The AppHost can express a local run target and a publish target separately: |
| 29 | + |
| 30 | +```csharp |
| 31 | +var builder = DistributedApplication.CreateBuilder(args); |
| 32 | + |
| 33 | +var dev = builder.AddKubernetesEnvironment("dev") |
| 34 | + .WithHelm(helm => helm.WithNamespace("operator-dev")); |
| 35 | + |
| 36 | +var aks = builder.AddAzureKubernetesEnvironment("aks") |
| 37 | + .WithHelm(helm => helm.WithNamespace("operator-system")); |
| 38 | + |
| 39 | +builder.AddKubeOps<Projects.Operator>("operator") |
| 40 | + .RunWithKubernetes(dev, run => |
| 41 | + { |
| 42 | + run.WithPersistentCrds(); |
| 43 | + }) |
| 44 | + .PublishAsKubernetesOperator(aks, publish => |
| 45 | + { |
| 46 | + publish.WithServiceAccount("operator"); |
| 47 | + }); |
| 48 | + |
| 49 | +builder.Build().Run(); |
| 50 | +``` |
| 51 | + |
| 52 | +`RunWithKubernetes(...)` configures the local project process. The operator still runs on the developer machine under Aspire, but its Kubernetes client is configured for the selected Kubernetes target. |
| 53 | + |
| 54 | +`PublishAsKubernetesOperator(...)` configures the Kubernetes deployment. The operator becomes a Kubernetes workload, and KubeOps-generated CRDs, RBAC, and service-account resources are incorporated into the generated Aspire chart. |
| 55 | + |
| 56 | +If `RunWithKubernetes(...)` is not configured, `AddKubeOps<TProject>(...)` keeps the operator resource in Aspire's explicit-start mode during `aspire run`. The project remains in the AppHost model and publish graph, but the local operator process does not start by default. A KubeOps operator without a Kubernetes run target would otherwise fail against an accidental kube context or mutate the wrong cluster. |
| 57 | + |
| 58 | +## Run Mode |
| 59 | + |
| 60 | +Run mode defaults to a productive inner loop once you opt in: |
| 61 | + |
| 62 | +```csharp |
| 63 | +var dev = builder.AddKubernetesEnvironment("dev"); |
| 64 | + |
| 65 | +builder.AddKubeOps<Projects.Operator>("operator") |
| 66 | + .RunWithKubernetes(dev); |
| 67 | +``` |
| 68 | + |
| 69 | +Default run behavior: |
| 70 | + |
| 71 | +- If run mode is not configured, do not start the operator automatically. |
| 72 | +- Create missing CRDs before starting the local operator process. |
| 73 | +- Track which CRDs were created by this Aspire run. |
| 74 | +- Remove only the CRDs that this run created when the AppHost shuts down. |
| 75 | +- Leave pre-existing CRDs alone. |
| 76 | +- Configure the operator process with the target namespace and Kubernetes client hints. |
| 77 | + |
| 78 | +Run mode exposes explicit CRD lifecycle choices: |
| 79 | + |
| 80 | +```csharp |
| 81 | +run.WithEphemeralCrds(); // default: create missing CRDs, remove only those created by this run |
| 82 | +run.WithPersistentCrds(); // create or update CRDs, leave them after shutdown |
| 83 | +run.RequireExistingCrds(); // fail before operator start if CRDs are missing |
| 84 | +run.SkipCrds(); // do not check or manage CRDs |
| 85 | +``` |
| 86 | + |
| 87 | +RBAC and service accounts are not required for the default local process path because the process normally authenticates with the developer's kubeconfig credentials. They can be added later for parity scenarios, for example a `RunAsServiceAccount(...)` option that creates a service account, binds RBAC, obtains a token, and configures the local process to use it. |
| 88 | + |
| 89 | +## Publish And Deploy Mode |
| 90 | + |
| 91 | +Publish mode defaults to production-ready Kubernetes output: |
| 92 | + |
| 93 | +```csharp |
| 94 | +var k8s = builder.AddKubernetesEnvironment("k8s") |
| 95 | + .WithHelm(helm => |
| 96 | + { |
| 97 | + helm.WithChartName("my-operator"); |
| 98 | + helm.WithReleaseName("my-operator"); |
| 99 | + helm.WithNamespace("operator-system"); |
| 100 | + }); |
| 101 | + |
| 102 | +builder.AddKubeOps<Projects.Operator>("operator") |
| 103 | + .PublishAsKubernetesOperator(k8s); |
| 104 | +``` |
| 105 | + |
| 106 | +Default publish behavior: |
| 107 | + |
| 108 | +- Generate CRDs. |
| 109 | +- Generate RBAC. |
| 110 | +- Generate a service account using the operator resource name. |
| 111 | +- Bind generated RBAC to that service account. |
| 112 | +- Let Aspire own the operator `Deployment` so image publishing, Helm values, service discovery, configuration, and telemetry wiring remain part of the full Aspire model. |
| 113 | +- Merge KubeOps-generated deployment settings into the Aspire-owned deployment instead of emitting a duplicate deployment. |
| 114 | + |
| 115 | +Publish options make each part explicit: |
| 116 | + |
| 117 | +```csharp |
| 118 | +publish.GenerateCrds(); // default |
| 119 | +publish.GenerateRbac(); // default |
| 120 | +publish.WithServiceAccount("operator"); // default name: resource name |
| 121 | +publish.SkipCrds(); |
| 122 | +publish.SkipRbac(); |
| 123 | +``` |
| 124 | + |
| 125 | +With a Kubernetes environment, `aspire publish` writes the generated Helm chart. `aspire deploy` uses the same Kubernetes environment and deployment engine to install it. The KubeOps integration hooks into Aspire's Kubernetes resource customization path (`PublishAsKubernetesService`) rather than running a separate KubeOps deployment. |
| 126 | + |
| 127 | +The integration filters KubeOps' generated `Deployment` out of the additional resource set because Aspire owns the workload. KubeOps deployment settings such as replicas, termination grace period, environment variables, resource requirements, `POD_NAMESPACE`, and the service account are merged into the Aspire-owned deployment instead. The chart therefore deploys one operator deployment, not a service-only placeholder and not a duplicate operator. |
| 128 | + |
| 129 | +## Standalone Publish |
| 130 | + |
| 131 | +You can also generate KubeOps manifests without an Aspire Kubernetes environment: |
| 132 | + |
| 133 | +```csharp |
| 134 | +builder.AddKubeOps<Projects.Operator>("operator") |
| 135 | + .PublishAsKubernetesOperator(publish => |
| 136 | + { |
| 137 | + publish.Namespace = "operator-system"; |
| 138 | + publish.WithServiceAccount("operator"); |
| 139 | + }); |
| 140 | +``` |
| 141 | + |
| 142 | +This path is useful when the AppHost should participate in Aspire publish, but the Kubernetes installation is handled by another workflow. It does not call Aspire's Kubernetes publishing integration and does not require `AddKubernetesEnvironment(...)`, Helm, or a live cluster. `aspire publish` writes the raw KubeOps-generated YAML under the operator resource's output directory. |
| 143 | + |
| 144 | +## AKS And Existing Resources |
| 145 | + |
| 146 | +AKS follows Aspire's Azure resource conventions. |
| 147 | + |
| 148 | +When the AppHost declares: |
| 149 | + |
| 150 | +```csharp |
| 151 | +var aks = builder.AddAzureKubernetesEnvironment("aks"); |
| 152 | +``` |
| 153 | + |
| 154 | +then `aspire deploy` provisions the AKS cluster, ACR, identity, and dependent Azure resources according to Aspire's AKS integration, then builds images, pushes them to ACR, generates Helm charts, and installs them into AKS. |
| 155 | + |
| 156 | +For local run mode, Azure integrations may provision Azure resources unless configured as existing resources. Users who want local run mode to attach to an existing AKS cluster should use Aspire's existing-resource APIs: |
| 157 | + |
| 158 | +```csharp |
| 159 | +var aks = builder.AddAzureKubernetesEnvironment("aks") |
| 160 | + .RunAsExisting(aksName, aksResourceGroup); |
| 161 | + |
| 162 | +builder.AddKubeOps<Projects.Operator>("operator") |
| 163 | + .RunWithKubernetes(aks); |
| 164 | +``` |
| 165 | + |
| 166 | +This keeps KubeOps aligned with Aspire instead of inventing a KubeOps-specific `UseExistingAks` switch. |
| 167 | + |
| 168 | +## Scenario Examples |
| 169 | + |
| 170 | +### Local Only |
| 171 | + |
| 172 | +Run the operator as a local process against the selected Kubernetes environment. The operator is not automatically published to Kubernetes unless you also call `PublishAsKubernetesOperator(...)`. |
| 173 | + |
| 174 | +```csharp |
| 175 | +var dev = builder.AddKubernetesEnvironment("dev"); |
| 176 | + |
| 177 | +builder.AddKubeOps<Projects.Operator>("operator") |
| 178 | + .RunWithKubernetes(dev, run => run.WithPersistentCrds()); |
| 179 | +``` |
| 180 | + |
| 181 | +### Azure Only |
| 182 | + |
| 183 | +Deploy the operator into AKS without running it as a local process during `aspire run`. |
| 184 | + |
| 185 | +```csharp |
| 186 | +var aks = builder.AddAzureKubernetesEnvironment("aks"); |
| 187 | + |
| 188 | +builder.AddKubeOps<Projects.Operator>("operator") |
| 189 | + .PublishAsKubernetesOperator(aks, publish => |
| 190 | + { |
| 191 | + publish.Namespace = "operator-system"; |
| 192 | + publish.WithServiceAccount("operator"); |
| 193 | + }); |
| 194 | +``` |
| 195 | + |
| 196 | +### Local Run And Azure Deploy |
| 197 | + |
| 198 | +Use a local Kubernetes environment for the development loop and AKS for publish/deploy. |
| 199 | + |
| 200 | +```csharp |
| 201 | +var dev = builder.AddKubernetesEnvironment("dev"); |
| 202 | +var aks = builder.AddAzureKubernetesEnvironment("aks"); |
| 203 | + |
| 204 | +builder.AddKubeOps<Projects.Operator>("operator") |
| 205 | + .RunWithKubernetes(dev) |
| 206 | + .PublishAsKubernetesOperator(aks, publish => publish.WithServiceAccount("operator")); |
| 207 | +``` |
| 208 | + |
| 209 | +### Publish Only Without A Kubernetes Environment |
| 210 | + |
| 211 | +Generate standalone KubeOps manifests without registering an Aspire Kubernetes environment. |
| 212 | + |
| 213 | +```csharp |
| 214 | +builder.AddKubeOps<Projects.Operator>("operator") |
| 215 | + .PublishAsKubernetesOperator(publish => |
| 216 | + { |
| 217 | + publish.Namespace = "operator-system"; |
| 218 | + publish.WithServiceAccount("operator"); |
| 219 | + }); |
| 220 | +``` |
| 221 | + |
| 222 | +## Mock And Local Cluster Targets |
| 223 | + |
| 224 | +KubeOps runtime watchers require a Kubernetes API server. Unit tests can replace `IKubernetesClient`, but the normal operator runtime is not an offline simulator. |
| 225 | + |
| 226 | +Additional targets can be modeled as Kubernetes environments if they provide Kubernetes API semantics: |
| 227 | + |
| 228 | +```csharp |
| 229 | +var k3s = builder.AddK3sCluster("k3s"); |
| 230 | +var mock = builder.AddKubeOpsMockKubernetes("mock-kube"); |
| 231 | + |
| 232 | +builder.AddKubeOps<Projects.Operator>("operator") |
| 233 | + .RunWithKubernetes(k3s); |
| 234 | +``` |
| 235 | + |
| 236 | +A mock target would require operator-side runtime support to replace the Kubernetes client and watcher behavior. It should not be represented as a normal Kubernetes environment unless it provides Kubernetes API semantics. |
| 237 | + |
| 238 | +## Non-Goals |
| 239 | + |
| 240 | +- Do not hide cluster mutations behind `#if DEBUG`. |
| 241 | +- Do not deploy a second KubeOps-generated operator deployment beside Aspire's deployment. |
| 242 | +- Do not require users to use AKS; plain Kubernetes environments, kind, k3s, and existing kubeconfig targets should work. |
| 243 | +- Do not make local run mode require RBAC/service-account generation unless the user explicitly asks to run as a service account. |
0 commit comments