|
| 1 | +--- |
| 2 | +title: Pod Configuration |
| 3 | +description: Configure Kubernetes pods launched by LocalStack services. |
| 4 | +template: doc |
| 5 | +sidebar: |
| 6 | + order: 6 |
| 7 | +tags: ["Enterprise"] |
| 8 | +--- |
| 9 | + |
| 10 | +## Introduction |
| 11 | + |
| 12 | +When LocalStack runs inside Kubernetes with the Kubernetes executor enabled, some services create child pods for workloads such as Lambda invocations and ECS tasks. |
| 13 | +In heterogeneous clusters, these child pods may need additional Kubernetes configuration so they are scheduled onto the right node pools, carry the right resource requests, or integrate with cluster policies. |
| 14 | + |
| 15 | +Use the `LOCALSTACK_K8S_POD_CONFIG` environment variable to configure Kubernetes metadata, scheduling, and resource settings for LocalStack-spawned pods. |
| 16 | +The variable accepts a JSON object with reusable `profiles` and optional per-service mappings. |
| 17 | +The value must be valid JSON. LocalStack validates this configuration at startup and refuses to start if the value is not valid JSON or contains unknown fields, so misconfigurations surface before any child pods are created. |
| 18 | + |
| 19 | +::::note |
| 20 | +This feature currently applies to Lambda and ECS pods created by the Kubernetes executor. |
| 21 | +Support for additional service integrations is being added incrementally. |
| 22 | +:::: |
| 23 | + |
| 24 | +## Supported fields |
| 25 | + |
| 26 | +Each profile can define the following fields: |
| 27 | + |
| 28 | +| Field | Description | |
| 29 | +|---|---| |
| 30 | +| `tolerations` | Kubernetes tolerations applied to the pod spec | |
| 31 | +| `nodeSelector` | Node selector applied to the pod spec | |
| 32 | +| `affinity` | Kubernetes affinity configuration applied to the pod spec | |
| 33 | +| `topologySpreadConstraints` | Topology spread constraints applied to the pod spec | |
| 34 | +| `priorityClassName` | Priority class name applied to the pod spec | |
| 35 | +| `resources` | Resource requests and limits applied to every non-init container in the pod | |
| 36 | +| `labels` | Labels merged into the pod metadata | |
| 37 | +| `annotations` | Annotations merged into the pod metadata | |
| 38 | + |
| 39 | +Scheduling fields such as `nodeSelector`, `tolerations`, and `affinity` replace the corresponding values on the generated pod spec. |
| 40 | +They are not merged with existing spec values. |
| 41 | +Resource settings are applied to the pod's application containers; init containers keep their generated defaults. |
| 42 | + |
| 43 | +## Configure default profiles |
| 44 | + |
| 45 | +The simplest configuration is to define default profiles. |
| 46 | +LocalStack uses the architecture-specific defaults for Lambda and ECS when the workload architecture is known, and falls back to `default` otherwise. |
| 47 | + |
| 48 | +```json |
| 49 | +{ |
| 50 | + "profiles": { |
| 51 | + "default": { |
| 52 | + "nodeSelector": { |
| 53 | + "pool": "general" |
| 54 | + } |
| 55 | + }, |
| 56 | + "defaultArm64": { |
| 57 | + "nodeSelector": { |
| 58 | + "pool": "arm-nodes" |
| 59 | + } |
| 60 | + }, |
| 61 | + "defaultAmd64": { |
| 62 | + "nodeSelector": { |
| 63 | + "pool": "amd-nodes" |
| 64 | + } |
| 65 | + } |
| 66 | + } |
| 67 | +} |
| 68 | +``` |
| 69 | + |
| 70 | +With this configuration: |
| 71 | + |
| 72 | +- Lambda and ECS ARM workloads use the `defaultArm64` profile. |
| 73 | +- Lambda and ECS x86 workloads use the `defaultAmd64` profile. |
| 74 | +- Workloads without architecture information use the `default` profile. |
| 75 | + |
| 76 | +To use this with the Helm chart, pass the JSON as an environment variable in your `values.yaml`: |
| 77 | + |
| 78 | +```yaml |
| 79 | +extraEnvVars: |
| 80 | + - name: LOCALSTACK_K8S_POD_CONFIG |
| 81 | + value: | |
| 82 | + { |
| 83 | + "profiles": { |
| 84 | + "default": { |
| 85 | + "nodeSelector": { |
| 86 | + "pool": "general" |
| 87 | + } |
| 88 | + }, |
| 89 | + "defaultArm64": { |
| 90 | + "nodeSelector": { |
| 91 | + "pool": "arm-nodes" |
| 92 | + } |
| 93 | + }, |
| 94 | + "defaultAmd64": { |
| 95 | + "nodeSelector": { |
| 96 | + "pool": "amd-nodes" |
| 97 | + } |
| 98 | + } |
| 99 | + } |
| 100 | + } |
| 101 | +``` |
| 102 | +
|
| 103 | +If you deploy LocalStack with the [LocalStack Operator](/aws/enterprise/kubernetes/kubernetes-operator/), set the same configuration as structured YAML under `spec.podSchedulingConfig` in your `LocalStack` resource instead of passing JSON: |
| 104 | + |
| 105 | +```yaml |
| 106 | +apiVersion: api.localstack.cloud/v1alpha1 |
| 107 | +kind: LocalStack |
| 108 | +spec: |
| 109 | + # other fields |
| 110 | + podSchedulingConfig: |
| 111 | + profiles: |
| 112 | + default: |
| 113 | + nodeSelector: |
| 114 | + pool: general |
| 115 | + defaultArm64: |
| 116 | + nodeSelector: |
| 117 | + pool: arm-nodes |
| 118 | + defaultAmd64: |
| 119 | + nodeSelector: |
| 120 | + pool: amd-nodes |
| 121 | +``` |
| 122 | + |
| 123 | +## Configure per-service profiles |
| 124 | + |
| 125 | +Use the `services` block when a service needs a dedicated profile. |
| 126 | +A service can reference a single profile with `profile`, or it can map `arm64` and `amd64` workloads to different profiles. |
| 127 | + |
| 128 | +```json |
| 129 | +{ |
| 130 | + "profiles": { |
| 131 | + "default": { |
| 132 | + "nodeSelector": { |
| 133 | + "pool": "general" |
| 134 | + } |
| 135 | + }, |
| 136 | + "defaultAmd64": { |
| 137 | + "nodeSelector": { |
| 138 | + "pool": "amd-nodes" |
| 139 | + } |
| 140 | + }, |
| 141 | + "lambda-arm": { |
| 142 | + "nodeSelector": { |
| 143 | + "pool": "lambda-arm-nodes" |
| 144 | + }, |
| 145 | + "resources": { |
| 146 | + "requests": { |
| 147 | + "cpu": "500m", |
| 148 | + "memory": "512Mi" |
| 149 | + }, |
| 150 | + "limits": { |
| 151 | + "cpu": "2", |
| 152 | + "memory": "2Gi" |
| 153 | + } |
| 154 | + } |
| 155 | + }, |
| 156 | + "ecs-tasks": { |
| 157 | + "nodeSelector": { |
| 158 | + "pool": "ecs-nodes" |
| 159 | + } |
| 160 | + } |
| 161 | + }, |
| 162 | + "services": { |
| 163 | + "lambda": { |
| 164 | + "arm64": "lambda-arm" |
| 165 | + }, |
| 166 | + "ecs": { |
| 167 | + "profile": "ecs-tasks" |
| 168 | + } |
| 169 | + } |
| 170 | +} |
| 171 | +``` |
| 172 | + |
| 173 | +In this example, ARM Lambda pods use `lambda-arm`. |
| 174 | +AMD64 Lambda pods use `defaultAmd64`. |
| 175 | +ECS pods always use `ecs-tasks`, regardless of architecture, because `services.ecs.profile` bypasses architecture-based routing. |
| 176 | +Workloads without architecture information use `default`. |
| 177 | + |
| 178 | +## Profile resolution |
| 179 | + |
| 180 | +LocalStack resolves the profile for a child pod in this order: |
| 181 | + |
| 182 | +1. `services.<service>.profile` |
| 183 | +2. `services.<service>.arm64` or `services.<service>.amd64` |
| 184 | +3. `profiles.defaultArm64` or `profiles.defaultAmd64` |
| 185 | +4. `profiles.default` |
| 186 | +5. No scheduling, resource, or metadata profile is applied. System labels are still added. |
| 187 | + |
| 188 | +`defaultArm64`, `defaultAmd64`, and `default` are reserved profile names. |
| 189 | +Use them only for global defaults. |
| 190 | + |
| 191 | +Architecture values are normalized before lookup. |
| 192 | +For example, `ARM64` is treated as `arm64`, and `x86_64` or `X86_64` are treated as `amd64`. |
| 193 | +The keys in `LOCALSTACK_K8S_POD_CONFIG` should still be `arm64` and `amd64`. |
| 194 | + |
| 195 | +If a service references a profile that does not exist, LocalStack logs a warning and applies no pod configuration for that request. |
| 196 | +It does not silently fall back to `default`. |
| 197 | + |
| 198 | +## Labels and annotations |
| 199 | + |
| 200 | +`labels` and `annotations` in `LOCALSTACK_K8S_POD_CONFIG` are merged into the metadata of the generated child pod. |
| 201 | +Profile labels override labels set through `LOCALSTACK_K8S_LABELS`, and profile annotations override annotations set through `LOCALSTACK_K8S_ANNOTATIONS`. |
| 202 | + |
| 203 | +LocalStack also injects the following system labels: |
| 204 | + |
| 205 | +```yaml |
| 206 | +app.kubernetes.io/managed-by: localstack |
| 207 | +localstack.cloud/service: <service> |
| 208 | +``` |
| 209 | + |
| 210 | +System labels are applied last and cannot be overridden. |
| 211 | +You can use these labels to target LocalStack-spawned pods from Kubernetes tooling such as network policies, admission controllers, or monitoring agents. |
| 212 | + |
| 213 | +## Example with scheduling and metadata |
| 214 | + |
| 215 | +The following example schedules Lambda pods on dedicated nodes, adds tolerations, sets resource limits, and attaches metadata used by platform tooling: |
| 216 | + |
| 217 | +```json |
| 218 | +{ |
| 219 | + "profiles": { |
| 220 | + "lambda": { |
| 221 | + "nodeSelector": { |
| 222 | + "workload": "localstack-lambda" |
| 223 | + }, |
| 224 | + "tolerations": [ |
| 225 | + { |
| 226 | + "key": "dedicated", |
| 227 | + "operator": "Equal", |
| 228 | + "value": "localstack", |
| 229 | + "effect": "NoSchedule" |
| 230 | + } |
| 231 | + ], |
| 232 | + "resources": { |
| 233 | + "requests": { |
| 234 | + "cpu": "250m", |
| 235 | + "memory": "256Mi" |
| 236 | + }, |
| 237 | + "limits": { |
| 238 | + "cpu": "1", |
| 239 | + "memory": "1Gi" |
| 240 | + } |
| 241 | + }, |
| 242 | + "labels": { |
| 243 | + "team": "platform" |
| 244 | + }, |
| 245 | + "annotations": { |
| 246 | + "prometheus.io/scrape": "true" |
| 247 | + } |
| 248 | + } |
| 249 | + }, |
| 250 | + "services": { |
| 251 | + "lambda": { |
| 252 | + "profile": "lambda" |
| 253 | + } |
| 254 | + } |
| 255 | +} |
| 256 | +``` |
| 257 | + |
| 258 | +## Related configuration |
| 259 | + |
| 260 | +- Use `LOCALSTACK_K8S_NAMESPACE` to choose the namespace for child pods. |
| 261 | +- Use `LOCALSTACK_K8S_LABELS` and `LOCALSTACK_K8S_ANNOTATIONS` for simple labels and annotations that apply to all child pods. |
| 262 | +- Use `K8S_CONTAINER_SECURITY_CONTEXT` to configure the security context for child pod containers. |
| 263 | + |
| 264 | +For the complete list of Kubernetes executor configuration variables, see the [Kubernetes configuration reference](/aws/enterprise/kubernetes/configuration/). |
0 commit comments