Skip to content

Commit 0cea6d1

Browse files
potiukclaude
andcommitted
Add draft project security threat-model document
Adds a draft project-level security threat-model document (draft-THREAT-MODEL.md) at repo root, improving discoverability for automated security scanners running against this repository. The file follows the rubric format used by several other ASF projects piloting security-model discoverability. The "draft-" prefix signals this is a proposal for the PMC to review, correct, or reject — not a finalised maintainer-blessed model. Every claim carries a provenance tag (documented / inferred / maintainer) so reviewers can see where each claim originates; §14 collects open questions for the maintainers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 4740dbc commit 0cea6d1

1 file changed

Lines changed: 357 additions & 0 deletions

File tree

draft-THREAT-MODEL.md

Lines changed: 357 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,357 @@
1+
<!--
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
20+
# Apache CloudStack Kubernetes Provider Security Threat Model — delta (draft)
21+
22+
> **Delta document.** Inherits §3, §4 B1, and §7 from
23+
> `cloudstack-threat-model-draft.md`. Read the main model first.
24+
25+
## §1 Header
26+
27+
- **Project:** Apache CloudStack Kubernetes Provider
28+
(`apache/cloudstack-kubernetes-provider`) — Kubernetes Cloud
29+
Controller Manager (CCM) for CloudStack-managed clusters.
30+
- **Commit:** `4740dbc` (HEAD of `main` at draft time).
31+
- **Date:** 2026-05-29.
32+
- **Authors:** ASF Security team draft.
33+
- **Status:** Draft delta over `cloudstack-threat-model-draft.md`.
34+
- **Version binding:** as of the commit above.
35+
- **Reporting:** as in the main model.
36+
- **Provenance legend:** as in the main model.
37+
- **Draft confidence:** 11 documented / 0 maintainer / 12 inferred.
38+
39+
**About the project.** A Kubernetes Cloud Controller Manager (CCM)
40+
written in Go that lets a Kubernetes cluster running on CloudStack
41+
discover its node instances, sync labels, and provision load
42+
balancers via the CloudStack API *(documented: `README.md`)*. Replaces
43+
the old in-tree `kubernetes/kubernetes` CloudStack provider that was
44+
removed *(documented: `README.md` — references to
45+
`kubernetes/enhancements` issues #672 and #88)*. Deployed automatically
46+
when CloudStack 4.16+ creates a Kubernetes cluster; can also be
47+
deployed manually. Runs as a Pod in the `kube-system` namespace.
48+
49+
**Crucially**, *(documented: `README.md` "Deployment")*: a separate
50+
service user **`kubeadmin`** is created in the same CloudStack account
51+
as the cluster owner, and the CCM uses **that user's API keys** to
52+
talk to CloudStack. The CCM holds long-lived CloudStack credentials in
53+
a Kubernetes Secret.
54+
55+
## §2 Scope and intended use
56+
57+
**Primary intended use.** Run as the Kubernetes Cloud Controller
58+
Manager for a Kubernetes cluster whose nodes are CloudStack-managed
59+
VMs. Discovers node identity, propagates zone / region labels,
60+
and provisions CloudStack load balancers in response to `Service
61+
type=LoadBalancer` declarations *(documented: `cloudstack.go`,
62+
`cloudstack_loadbalancer.go`, `cloudstack_instances.go`)*.
63+
64+
**Deployment shape.** A long-running Pod inside `kube-system`. Has
65+
network access to (a) the Kubernetes API server (in-cluster
66+
ServiceAccount auth), and (b) the CloudStack management server
67+
(`api-url` from the mounted `cloud-config` Secret).
68+
69+
**Caller expectations.** The Kubernetes cluster operator is trusted
70+
to:
71+
72+
- create the `cloudstack-secret` containing `cloud-config` with
73+
`api-url`, `api-key`, `secret-key`, optional `project-id`, `zone`,
74+
`region`, and **`ssl-no-verify`** *(documented: `README.md`)*,
75+
- not regenerate the `kubeadmin` user's API keys after deployment
76+
(the CCM relies on a stable identity) *(documented: `README.md`)*,
77+
- restrict access to the `cloudstack-secret` per Kubernetes RBAC.
78+
79+
**Component-family table.**
80+
81+
| Family | Representative entry | Touches outside the process? | In this delta? |
82+
| --- | --- | --- | --- |
83+
| Provider config + CloudStack client (`cloudstack.go` `CSConfig`, `newCSCloud`) | reads `cloud-config` Secret, builds `cloudstack-go` client | **yes — network + creds** | yes |
84+
| Instance discovery (`cloudstack_instances.go`) | maps node VM IDs to CloudStack VMs | inherited | yes |
85+
| Load-balancer reconciler (`cloudstack_loadbalancer.go`) | provisions CloudStack LB rules in response to K8s Service changes | inherited | yes |
86+
| Protocol helpers (`protocol.go`) | TCP / UDP / TCP-Proxy LoadBalancer support *(documented: `README.md`)* | n/a | yes |
87+
| `cmd/cloudstack-ccm` | CCM entry point | binds in-cluster ServiceAccount | yes |
88+
| `deployment.yaml`, `nginx-ingress-controller-patch.yml`, Dockerfile | packaging | n/a at runtime | yes — these define the *deployment posture* |
89+
| Tests `*_test.go` | unit / integration tests | n/a | **out of model** *(§3)* |
90+
| `get_kubernetes_deps.sh` | dev script | n/a | **out of model** *(§3)* |
91+
92+
## §3 Out of scope (explicit non-goals)
93+
94+
The main model's §3 applies. **Additional** out-of-scope items
95+
specific to the K8s provider:
96+
97+
1. **Kubernetes Secret confidentiality.** The CCM expects `cloud-config`
98+
to be in a Kubernetes Secret. Whether that Secret is encrypted at
99+
rest (etcd encryption, KMS provider), bound to a specific Pod, or
100+
exposed via in-cluster reads is the Kubernetes cluster's own
101+
responsibility. *(inferred — Q1)*
102+
2. **Server-side correctness of CloudStack responses.** Same as
103+
sibling deltas.
104+
3. **TLS verification when `ssl-no-verify = true`.** When the cluster
105+
operator sets `ssl-no-verify = true` in `cloud-config`, the CCM
106+
passes `verifyssl=false` to `cloudstack-go` (via the equivalent
107+
constructor). That is operator choice.
108+
4. **In-cluster RBAC for the CCM's ServiceAccount.** What Kubernetes
109+
API permissions the CCM holds via its ServiceAccount and Role
110+
bindings is the cluster operator's job.
111+
5. **Kubernetes itself.** Bugs in `kube-apiserver`, `kubelet`, etcd,
112+
the CRI runtime are upstream.
113+
6. **The `kubeadmin` user creation flow on the CloudStack side.** The
114+
`kubeadmin` user is created by CloudStack's `cloudstack-management`
115+
when the cluster is provisioned; that flow is in the main model.
116+
This delta covers only the CCM's *consumption* of those credentials.
117+
7. **The four sibling repos.**
118+
119+
## §4 Trust boundaries and data flow
120+
121+
The CCM stitches together two main-model trust transitions:
122+
123+
- **K1: K8s controller-manager → Kubernetes API server.** In-cluster
124+
ServiceAccount token, mounted into the Pod. Authn / authz are
125+
Kubernetes-side.
126+
- **K2: CCM → CloudStack management server.** Main-model B1 — HMAC-
127+
SHA1 signed JSON API calls. Credentials come from the
128+
`cloudstack-secret`'s `cloud-config`.
129+
130+
The boundary is **the Kubernetes Pod sandbox** plus the mounted
131+
Secret. Bytes inside the Pod that come from either the K8s API server
132+
or the CloudStack management server are trusted control-plane content.
133+
134+
**`ssl-no-verify` behaviour** *(documented: `cloudstack.go` `CSConfig`
135+
gcfg tag `ssl-no-verify`)*: when `true`, TLS verification of the
136+
CloudStack management-server cert is disabled.
137+
138+
## §5 Assumptions about the environment
139+
140+
- **Host**: Kubernetes 1.16+ (the in-tree provider was removed in
141+
1.15) *(documented: `README.md`)*.
142+
- **Container**: `apache/cloudstack-kubernetes-provider` published on
143+
Docker Hub *(documented: `README.md`)*.
144+
- **Filesystem**: reads `cloud-config` from a mounted Secret;
145+
otherwise no host filesystem writes *(inferred — Q2)*.
146+
- **Network**: outbound HTTPS to `api-url` (CloudStack) and in-cluster
147+
HTTPS to `kube-apiserver`.
148+
- **Auth**: stdlib TLS, opt-out via `ssl-no-verify=true`; in-cluster
149+
K8s auth via projected ServiceAccount token.
150+
- **What the CCM does not do**: no host-network listener, no privileged
151+
Pod requirement *(inferred — Q3)*, no write to the host filesystem.
152+
153+
## §5a Build-time and configuration variants
154+
155+
| Knob | Default | Stance | Effect |
156+
| --- | --- | --- | --- |
157+
| `api-url` | none | operator config | endpoint |
158+
| `api-key`, `secret-key` | none — provisioned automatically as `kubeadmin` | operator must not regenerate after deployment *(documented: `README.md`)* | identity |
159+
| `ssl-no-verify` | `false` *(inferred — Q4)* | dev / self-signed-CA | flips TLS verification off |
160+
| `project-id`, `zone`, `region` | unset | optional scoping | constrains LB / instance discovery to a project |
161+
| LoadBalancer protocols | TCP, UDP, TCP-Proxy *(documented: `README.md`)* | supported set | not security-relevant |
162+
163+
## §6 Assumptions about inputs
164+
165+
| Entry point | Parameter | Attacker-controllable in the model? | Caller must enforce |
166+
| --- | --- | --- | --- |
167+
| `cloud-config` Secret | `api-url`, `api-key`, `secret-key`, `ssl-no-verify`, etc. | **no** — Kubernetes cluster operator config | restrict Secret read RBAC; consider etcd-at-rest encryption |
168+
| Kubernetes Service / Node / Endpoint events | watch payloads | **trusted from kube-apiserver** | bytes are control-plane |
169+
| CloudStack JSON responses | typed-decoded | trusted (B1) | bytes are control-plane |
170+
171+
## §7 Adversary model
172+
173+
Main-model §7 applies. **Adjustments specific to the K8s provider**:
174+
175+
- "Unauthenticated network peer reaching `:8080`" is upstream.
176+
- An additional adversary worth naming: **any Kubernetes principal
177+
with `get`/`list` on Secrets in `kube-system`**. Such a principal
178+
recovers the CloudStack `kubeadmin` user's API key and secret in
179+
plaintext and can drive the CloudStack API as `kubeadmin` (with
180+
whatever CloudStack-side role that user holds). This is in scope:
181+
the CCM cannot prevent in-cluster Secret read, but the documented
182+
*(README)* deployment recommendation that `kubeadmin` keys must not
183+
be regenerated means a leaked `cloud-config` cannot be invalidated
184+
by rotation without breaking the CCM. *(inferred — Q5 — high-
185+
priority.)*
186+
- **A passive observer on the in-cluster network** between the CCM
187+
Pod and the CloudStack management server when `ssl-no-verify=true`
188+
is the same shape as in the Terraform-provider delta: TLS verify is
189+
off, MitM is possible undetected.
190+
191+
## §8 Security properties the CCM provides
192+
193+
### K1 — HMAC-SHA1 signature via `cloudstack-go`
194+
195+
- As main-model §8 P1.
196+
197+
### K2 — In-cluster ServiceAccount-bound CCM identity
198+
199+
- **Property.** The CCM authenticates to the Kubernetes API server as
200+
its own ServiceAccount; no shared secret is sent to kube-apiserver.
201+
- **Conditions.** Pod is deployed per `deployment.yaml` with a
202+
ServiceAccount.
203+
- **Violation symptom.** CCM speaks to kube-apiserver as a different
204+
identity.
205+
- **Severity.** Security-critical, `VALID`.
206+
207+
### K3 — TLS verification toggle (`ssl-no-verify`)
208+
209+
- As Go-SDK delta §8 S2 / CloudMonkey delta §8 C2.
210+
211+
## §9 Security properties the CCM does *not* provide
212+
213+
- **No protection of the `cloud-config` Secret.** A K8s principal with
214+
Secret read in `kube-system` recovers CloudStack credentials.
215+
- **No key rotation.** The README *requires* operators **not** to
216+
rotate `kubeadmin` API keys after deployment — meaning a compromise
217+
cannot be remediated by rotation without breaking the CCM. *(See
218+
§14 Q6 for the proposed-rotation question.)*
219+
- **No defence when `ssl-no-verify = true`.**
220+
- **No least-privilege constraint** on the `kubeadmin` CloudStack user's
221+
scope beyond what CloudStack's RBAC + project / zone scoping gives.
222+
*(inferred — Q7)*
223+
- **No defence against an attacker with read access to the CCM Pod's
224+
in-cluster network namespace** (e.g. a sidecar in the same Pod).
225+
*(inferred — Q8)*
226+
227+
### False-friend properties
228+
229+
- **`ssl-no-verify` is not "test mode."** Same wording as the
230+
Terraform delta.
231+
- **A Kubernetes Secret is not a hardened secret store** — it is
232+
base64-encoded by default in etcd, encrypted only when etcd
233+
encryption is enabled.
234+
- **HMAC-SHA1** — see main model §11a.
235+
236+
## §10 Downstream responsibilities
237+
238+
The Kubernetes cluster operator MUST:
239+
240+
1. Enable etcd at-rest encryption (or KMS-backed Secret encryption)
241+
for the `cloudstack-secret`.
242+
2. Restrict Kubernetes RBAC so that only the CCM ServiceAccount can
243+
read `cloudstack-secret`.
244+
3. Set `ssl-no-verify = false` (the recommended posture) unless the
245+
CloudStack management server is unreachable over TLS in the
246+
cluster's network.
247+
4. Scope the CloudStack-side `kubeadmin` user to the smallest
248+
CloudStack role that can list VMs and create/update load
249+
balancers — not a root admin or domain admin.
250+
5. Treat the CCM Pod as a credential-bearing workload: no co-tenant
251+
sidecars, no shared PID namespace with untrusted workloads.
252+
6. Match CCM container image to the CloudStack API contract (the
253+
image embeds a specific `cloudstack-go` version).
254+
7. Monitor CloudStack audit logs for `kubeadmin`-account anomalies.
255+
256+
## §11 Known misuse patterns
257+
258+
- Storing `cloud-config` in a Kubernetes Secret without etcd
259+
encryption.
260+
- Granting `secret get` on `kube-system` to non-admin Kubernetes
261+
principals.
262+
- Running the CCM with a `kubeadmin` user that is a root admin or
263+
domain admin of the CloudStack account (over-privileged).
264+
- Setting `ssl-no-verify = true` in production.
265+
- Sharing the `cloud-config` across clusters of different trust
266+
levels — a compromise of one cluster's Secret compromises the
267+
others.
268+
- Regenerating the `kubeadmin` API key — breaks the CCM until the
269+
`cloud-config` Secret is updated *(documented: `README.md`)*.
270+
271+
## §11a Known non-findings (recurring false positives)
272+
273+
- **"Secret contains plaintext `api-key` / `secret-key`."** That is
274+
the documented deployment shape. → `BY-DESIGN: property-disclaimed`.
275+
- **"Pod has access to a long-lived credential."** Same. → `BY-DESIGN:
276+
property-disclaimed`.
277+
- **"HMAC-SHA1 — SHA1 is deprecated."**`KNOWN-NON-FINDING` per
278+
main model.
279+
- **"`InsecureSkipVerify` is reachable from `ssl-no-verify`."**
280+
Operator choice. → `OUT-OF-MODEL: trusted-input`.
281+
- **"`kubeadmin` user has broad CloudStack privileges."** Depends on
282+
operator's CloudStack RBAC posture per §10 item 4. → `OUT-OF-MODEL:
283+
trusted-input` against the operator's CloudStack config.
284+
- **"Tests in `*_test.go` have weak input handling."** Out of model.
285+
`OUT-OF-MODEL: unsupported-component`.
286+
287+
## §12 Conditions that would change this delta
288+
289+
- A move from static `cloud-config` Secret to a secret-manager
290+
pattern (External Secrets Operator, CSI Secret Driver, projected
291+
short-lived token from the CloudStack side).
292+
- Support for short-lived / rotating credentials at the CloudStack
293+
side (would change the §9 "no key rotation" bullet).
294+
- Addition of a CCM-side TLS-verify default that overrides
295+
`ssl-no-verify=false` to `true`.
296+
- Change in signing algorithm at the main-model layer.
297+
298+
## §13 Triage dispositions
299+
300+
Use the same table as the main model.
301+
302+
## §14 Open questions for the maintainers
303+
304+
**Q1.** Out-of-scope: Kubernetes-side Secret confidentiality (etcd
305+
encryption, KMS, RBAC). Confirm. *(maps to §3, §9)*
306+
307+
**Q2.** Confirm the CCM has no host-filesystem writes and no
308+
ephemeral credential cache that survives Pod restart.
309+
310+
**Q3.** Does the upstream `deployment.yaml` require any host
311+
privileges (host network, host PID, privileged container)? Proposed:
312+
**no**. Confirm.
313+
314+
**Q4.** `ssl-no-verify` default — proposed: **`false`**. Confirm.
315+
*(maps to §5a, §10)*
316+
317+
**Q5.** What is the recommended Kubernetes Secret protection posture
318+
for `cloudstack-secret`? Proposed: etcd-at-rest encryption + RBAC
319+
restricting `get`/`list` on Secrets in `kube-system` to cluster
320+
admins + the CCM ServiceAccount only. *(maps to §3, §10)*
321+
322+
**Q6.** **Highest-leverage question in this delta.** The README says
323+
*"It is imperative that this user is not altered or have its keys
324+
regenerated."* — meaning credential rotation is not supported.
325+
326+
- Is rotation actually impossible, or is it documented-not-supported
327+
because the operator-side workflow is non-trivial?
328+
- Is there a path to rotate the `kubeadmin` API key with controlled
329+
CCM downtime?
330+
- Should the threat model state explicitly that the CCM's
331+
CloudStack credential is *non-rotatable* and treat any leak as
332+
requiring redeployment of the cluster's CloudStack account?
333+
*(maps to §9, §10, §11)*
334+
335+
**Q7.** What CloudStack-side RBAC scope is `kubeadmin` *expected* to
336+
have? Proposed: the smallest set of API commands that covers `listVMs`
337+
+ LoadBalancer rule CRUD. Is there a published recommended role?
338+
*(maps to §9, §10)*
339+
340+
**Q8.** What is the CCM's posture against a malicious sidecar in the
341+
same Pod, or against a malicious DaemonSet sharing the same node? Is
342+
that out of scope (delegated to Pod isolation / Pod Security
343+
Standards)? Proposed: out of scope. *(maps to §7, §9)*
344+
345+
**Q9.** Meta — should this delta live at `docs/threat-model.md` in
346+
`apache/cloudstack-kubernetes-provider`, or in the website tree?
347+
348+
**Q10.** When the main model's signing algorithm changes, what is the
349+
release-cadence commitment for an updated CCM image?
350+
351+
**Q11.** Confirm the unsupported-component list (tests, dev scripts,
352+
old in-tree references in the README).
353+
354+
**Q12.** TCP-Proxy LoadBalancer support pulls in HAProxy PROXY
355+
protocol *(documented: `README.md`)*. Is the CCM responsible for any
356+
authentication of the PROXY-protocol header, or is that downstream of
357+
the actual workload Pod? Proposed: downstream of the workload Pod.

0 commit comments

Comments
 (0)