Skip to content

Commit 2fa9214

Browse files
oliverbaehlerSvarrogh1337adberger
authored
feat: add recommended kyverno rules (#46)
* feat: add versioning Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * feat: add recommended kyverno policies Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * feat(docs): add opencost integration (#41) * feat(docs): add opencost integration Signed-off-by: Hristo Hristov <me@hhristov.info> * feat(docs): add opencost integration Signed-off-by: Hristo Hristov <me@hhristov.info> * feat(docs): add opencost integration Signed-off-by: Hristo Hristov <me@hhristov.info> * feat(docs): add opencost integration Signed-off-by: Hristo Hristov <me@hhristov.info> * feat(docs): add opencost integration Signed-off-by: Hristo Hristov <me@hhristov.info> * feat(docs): add opencost integration Signed-off-by: Hristo Hristov <me@hhristov.info> * feat(docs): add opencost integration Signed-off-by: Hristo Hristov <me@hhristov.info> * feat(docs): add opencost integration Signed-off-by: Hristo Hristov <me@hhristov.info> --------- Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * fix(website): separate preview and production deployments (#42) Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * chore(actions): fix action image Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * chore(docs): make apidocs-capsule-proxy and apidocs-capsule Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * chore(docs): fix docs lint Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * feat(docs): add integration for teleport (#44) Signed-off-by: Adrian Berger <adrian.berger@bedag.ch> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * feat(docs): add versioned documentation Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * feat(docs): add versioned documentation Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * feat(docs): add versioned documentation Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * feat(docs): add versioned documentation Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * feat(docs): add versioned documentation Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * feat(docs): add versioned documentation Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> * feat: align with fixed metrics --------- Signed-off-by: Oliver Bähler <oliverbaehler@hotmail.com> Signed-off-by: Hristo Hristov <me@hhristov.info> Signed-off-by: Adrian Berger <adrian.berger@bedag.ch> Co-authored-by: Hristo Hristov <me@hhristov.info> Co-authored-by: Adrian Berger <43774417+adberger@users.noreply.github.com>
1 parent a72318e commit 2fa9214

6 files changed

Lines changed: 496 additions & 31 deletions

File tree

content/en/docs/operating/monitoring.md

Lines changed: 29 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -169,29 +169,48 @@ capsule_pool_usage_percentage{pool="solar-compute",resource="requests.memory"} 4
169169
capsule_pool_usage_percentage{pool="solar-size",resource="pods"} 42.857142857142854
170170
```
171171

172+
## Tenants
172173

173-
## Quotas
174-
175-
Instrumentation for [Quotas](../tenants/quotas/).
174+
Instrumentation for [Tenants](../tenants/).
176175

177176
### Metrics
178177

179178
The following Metrics are exposed and can be used for monitoring:
180179

181180
```shell
181+
# HELP capsule_tenant_condition Provides per tenant condition status for each condition
182+
# TYPE capsule_tenant_condition gauge
183+
capsule_tenant_condition{condition="Cordoned",tenant="solar"} 0
184+
capsule_tenant_condition{condition="Ready",tenant="solar"} 1
185+
186+
187+
# HELP capsule_tenant_namespace_condition Provides per namespace within a tenant condition status for each condition
188+
# TYPE capsule_tenant_namespace_condition gauge
189+
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="earth",tenant="solar"} 0
190+
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="fire",tenant="solar"} 0
191+
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="foild",tenant="solar"} 0
192+
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="green",tenant="solar"} 0
193+
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="solar",tenant="solar"} 0
194+
capsule_tenant_namespace_condition{condition="Cordoned",target_namespace="wind",tenant="solar"} 0
195+
capsule_tenant_namespace_condition{condition="Ready",target_namespace="earth",tenant="solar"} 1
196+
capsule_tenant_namespace_condition{condition="Ready",target_namespace="fire",tenant="solar"} 1
197+
capsule_tenant_namespace_condition{condition="Ready",target_namespace="foild",tenant="solar"} 1
198+
capsule_tenant_namespace_condition{condition="Ready",target_namespace="green",tenant="solar"} 1
199+
capsule_tenant_namespace_condition{condition="Ready",target_namespace="solar",tenant="solar"} 1
200+
capsule_tenant_namespace_condition{condition="Ready",target_namespace="wind",tenant="solar"} 1
201+
182202
# HELP capsule_tenant_namespace_count Total number of namespaces currently owned by the tenant
183203
# TYPE capsule_tenant_namespace_count gauge
184204
capsule_tenant_namespace_count{tenant="solar"} 6
185205

186206
# HELP capsule_tenant_namespace_relationship Mapping metric showing namespace to tenant relationships
187207
# TYPE capsule_tenant_namespace_relationship gauge
188-
capsule_tenant_namespace_relationship{namespace="earth",tenant="solar"} 1
189-
capsule_tenant_namespace_relationship{namespace="wind",tenant="solar"} 1
190-
capsule_tenant_namespace_relationship{namespace="fire",tenant="solar"} 1
191-
192-
# HELP capsule_tenant_status Tenant cordon state indicating if tenant operations are restricted (1) or allowed (0) for resource creation and modification
193-
# TYPE capsule_tenant_status gauge
194-
capsule_tenant_status{tenant="limiting-resources"} 0
208+
capsule_tenant_namespace_relationship{target_namespace="earth",tenant="solar"} 1
209+
capsule_tenant_namespace_relationship{target_namespace="fire",tenant="solar"} 1
210+
capsule_tenant_namespace_relationship{target_namespace="soil",tenant="solar"} 1
211+
capsule_tenant_namespace_relationship{target_namespace="green",tenant="solar"} 1
212+
capsule_tenant_namespace_relationship{target_namespace="solar",tenant="solar"} 1
213+
capsule_tenant_namespace_relationship{target_namespace="wind",tenant="solar"} 1
195214

196215
# HELP capsule_tenant_resource_limit Current resource limit for a given resource in a tenant
197216
# TYPE capsule_tenant_resource_limit gauge

content/en/docs/operating/setup/configuration.md

Lines changed: 89 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -7,57 +7,131 @@ description: >
77

88
The configuration for the capsule controller is done via it's dedicated configration Custom Resource. You can explain the configuration options and how to use them:
99

10-
11-
1210
## CapsuleConfiguration
1311

1412
The configuration for Capsule is done via it's dedicated configration Custom Resource. You can explain the configuration options and how to use them:
1513

16-
```bash
14+
```shell
1715
kubectl explain capsuleConfiguration.spec
1816
```
1917

20-
### enableTLSReconciler
18+
### `enableTLSReconciler`
2119
Toggles the TLS reconciler, the controller that is able to generate CA and certificates for the webhooks when not using an already provided CA and certificate, or when these are managed externally with Vault, or cert-manager.
2220

23-
### forceTenantPrefix
21+
```yaml
22+
tls:
23+
enableController: true
24+
```
25+
26+
### `forceTenantPrefix`
2427
Enforces the Tenant owner, during Namespace creation, to name it using the selected Tenant name as prefix, separated by a dash. This is useful to avoid Namespace name collision in a public CaaS environment.
2528

26-
### nodeMetadata
29+
```yaml
30+
manager:
31+
options:
32+
forceTenantPrefix: true
33+
```
34+
35+
### `nodeMetadata`
2736
Allows to set the forbidden metadata for the worker nodes that could be patched by a Tenant. This applies only if the Tenant has an active NodeSelector, and the Owner have right to patch their nodes.
2837

29-
### overrides
38+
```yaml
39+
manager:
40+
options:
41+
nodeMetadata:
42+
forbiddenLabels:
43+
denied:
44+
- "node-role.kubernetes.io/*"
45+
deniedRegex: ""
46+
forbiddenAnnotations:
47+
denied:
48+
- "node.alpha.kubernetes.io/*"
49+
deniedRegex: ""
50+
```
51+
52+
[Read More](/docs/tenants/enforcement/#nodes)
53+
54+
55+
### `overrides`
3056
Allows to set different name rather than the canonical one for the Capsule configuration objects, such as webhook secret or configurations.
3157

32-
### protectedNamespaceRegex
58+
### `protectedNamespaceRegex`
3359
Disallow creation of namespaces, whose name matches this regexp
3460

35-
### userGroups
61+
```yaml
62+
manager:
63+
options:
64+
protectedNamespaceRegex: "^(kube|default|capsule|admin|system|com|org|local|localhost|io)$"
65+
```
66+
67+
### `userGroups`
3668
Names of the groups for Capsule users. Users must have this group to be considered for the Capsule tenancy. If a user does not have any group mentioned here, they are not recognized as a Capsule user.
3769

38-
### userNames
70+
```yaml
71+
manager:
72+
options:
73+
capsuleUserGroups:
74+
- system:serviceaccounts:tenants-gitops
75+
- company:org:users
76+
```
77+
78+
### `userNames`
3979
Names of the users for Capsule users. Users must have this name to be considered for the Capsule tenancy. If userGroups are set, the properties are ORed, meaning that a user can be recognized as a Capsule user if they have one of the groups or one of the names.
4080

41-
### ignoreUserWithGroups
81+
```yaml
82+
manager:
83+
options:
84+
userNames:
85+
- system:serviceaccount:crossplane-system:crossplane-k8s-provider
86+
```
87+
88+
### `ignoreUserWithGroups`
4289
Define groups which when found in the request of a user will be ignored by the Capsule. This might be useful if you have one group where all the users are in, but you want to separate administrators from normal users with additional groups.
4390

91+
```yaml
92+
manager:
93+
options:
94+
ignoreUserWithGroups:
95+
- company:org:administrators
96+
```
97+
98+
### `allowServiceAccountPromotion`
99+
100+
ServiceAccounts within tenant namespaces can be promoted to owners of the given tenant this can be achieved by labeling the serviceaccount and then they are considered owners. This can only be done by other owners of the tenant. However ServiceAccounts which have been promoted to owner can not promote further serviceAccounts.
101+
102+
[Read More](/docs/tenants/permissions/#serviceaccount-promotion)
103+
104+
```yaml
105+
manager:
106+
options:
107+
allowServiceAccountPromotion: true
108+
```
109+
44110
## Controller Options
45111

46112
Depending on the version of the Capsule Controller, the configuration options may vary. You can view the options for the latest version of the Capsule Controller or by executing the controller locally:
47113

48114
```bash
49-
$ docker run ghcr.io/projectcapsule/capsule:v0.6.0-rc0 -h
50-
2024/02/25 13:21:21 maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined
51-
Usage of /ko-app/capsule:
115+
$ go run ./cmd/. --zap-log-level 7 -h
116+
2025/09/13 23:50:30 maxprocs: Leaving GOMAXPROCS=8: CPU quota undefined
117+
Usage of /var/folders/ts/43yg7sk56ls3r3xjf66npgpm0000gn/T/go-build2624543463/b001/exe/cmd:
52118
--configuration-name string The CapsuleConfiguration resource name to use (default "default")
53119
--enable-leader-election Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
54120
--metrics-addr string The address the metric endpoint binds to. (default ":8080")
55121
--version Print the Capsule version and exit
56122
--webhook-port int The port the webhook server binds to. (default 9443)
57123
--zap-devel Development Mode defaults(encoder=consoleEncoder,logLevel=Debug,stackTraceLevel=Warn). Production Mode defaults(encoder=jsonEncoder,logLevel=Info,stackTraceLevel=Error)
58124
--zap-encoder encoder Zap log encoding (one of 'json' or 'console')
59-
--zap-log-level level Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', or any integer value > 0 which corresponds to custom debug levels of increasing verbosity
125+
--zap-log-level level Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', 'panic'or any integer value > 0 which corresponds to custom debug levels of increasing verbosity
60126
--zap-stacktrace-level level Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic').
61127
--zap-time-encoding time-encoding Zap time encoding (one of 'epoch', 'millis', 'nano', 'iso8601', 'rfc3339' or 'rfc3339nano'). Defaults to 'epoch'.
62128
```
63129

130+
Define additional options in the `values.yaml` when installing via Helm:
131+
132+
```yaml
133+
manager:
134+
extraArgs:
135+
- "--enable-leader-election=true"
136+
```
137+

content/en/docs/operating/setup/installation.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,12 @@ Perform the following steps to install the capsule Operator:
5858

5959
Here are some key considerations to keep in mind when installing Capsule. Also check out the **[Best Practices](/docs/operating/best-practices)** for more information.
6060

61+
### Admission Policies
62+
63+
While Capsule provides a robust framework for managing multi-tenancy in Kubernetes, it does not include built-in admission policies for enforcing specific security or operational standards for all possible aspects of a Kubernetes cluster. Therefore, it is recommended to use additional tools like [Kyverno](https://kyverno.io/) to enforce admission policies that align with your organization's requirements.
64+
65+
[We provide policy recommendations for Kyverno here](/ecosystem/integrations/kyverno/#recommended-policies).
66+
6167
### Certificate Management
6268

6369
We recommend using [cert-manager](https://cert-manager.io/) to manage the TLS certificates for Capsule. This will ensure that your Capsule installation is secure and that the certificates are automatically renewed. Capsule requires a valid TLS certificate for it's Admission Webserver. By default Capsule reconciles it's own TLS certificate. To use cert-manager, you can set the following values:

content/en/docs/tenants/permissions.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,28 @@ system:serviceaccounts:{service-account-namespace}
164164

165165
You have to add `system:serviceaccounts:{service-account-namespace}` to the CapsuleConfiguration [Group Scope](#group-scope) to make it work.
166166

167+
### ServiceAccount Promotion
168+
169+
Within a tenant, a ServiceAccount can be promoted to a Tenant Owner. For example, Alice can create a ServiceAccount called robot in the solar tenant and promote it to be a Tenant Owner (This requires Alice to be an owner of the tenant as well):
170+
171+
```yaml
172+
kubectl label sa gitops-reconcile -n green-test owner.projectcapsule.dev/promote=true --as alice --as-group projectcapsule.dev
173+
```
174+
175+
Now the ServiceAccount robot can create namespaces in the solar tenant:
176+
177+
```bash
178+
kubectl create ns green-valkey--as system:serviceaccount:green-test:gitops-reconcile
179+
```
180+
181+
To revoke the promotion, Alice can just remove the label:
182+
183+
```yaml
184+
kubectl label sa gitops-reconcile -n green-test owner.projectcapsule.dev/promote- --as alice --as-group projectcapsule.dev
185+
```
186+
187+
This feature must be enabled in the [CapsuleConfiguration](/docs/operating/setup/configuration/#allowserviceaccountpromotion).
188+
167189

168190
### Owner Roles
169191

content/en/docs/whats-new.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,22 +5,21 @@ description: >
55
weight: 1
66
---
77

8-
98
## Features
109

11-
* includes a new approach to how Resources (ResourceQuotas) should be handled across multiple namespaces. With this release, we are introducing the concept of ResourcePools and ResourcePoolClaims. Essentially, you can now define Resources and the audience (namespaces) that can claim these Resources from a ResourcePool. This introduces a shift-left in resource management, where Tenant Owners themselves are responsible for organizing their resources. Comes with a Queuing-Mechanism already in place. This new feature works with all namespaces — not just exclusive Capsule namespaces. [Read More](/docs/resourcepools/)
12-
13-
* Added support for GatewayAPI v1 (Gateway-Class control). [Read More](/docs/tenants/enforcement/#gatewayclasses)
10+
* Owners can promote ServiceAccounts from their Tenant namespaces to Owners of the Tenant [Read More](/docs/tenants/permissions/#serviceaccount-promotion)
1411

15-
- Added a more sophisticated way to control metadata for namespaces within a tenant. This allows you to distribute labels and annotations to namespaces based on more specific conditions. It's now also possible so use simple templating to assign metadata. [Read More](/docs/tenants/enforcement/#namespaces)
12+
* Reworked Metrics based on improved Tenant state management via Conditions. [Read More](/docs/operating/monitoring/#metrics-1)
1613

14+
* Includes a new approach to how Resources (ResourceQuotas) should be handled across multiple namespaces. With this release, we are introducing the concept of ResourcePools and ResourcePoolClaims. Essentially, you can now define Resources and the audience (namespaces) that can claim these Resources from a ResourcePool. This introduces a shift-left in resource management, where Tenant Owners themselves are responsible for organizing their resources. Comes with a Queuing-Mechanism already in place. This new feature works with all namespaces — not just exclusive Capsule namespaces. [Read More](/docs/resourcepools/)
1715

1816
## Documentation
1917

2018
We have added new documentation for a better experience. See the following Topics:
2119

2220
* **[Best Practices](/docs/operating/best-practices/)**
2321
* **[Installation](/docs/operating/setup/installation/)**
22+
* **[Kyverno Policy Recommendations](/ecosystem/integrations/kyverno/#recommended-policies)**
2423

2524
## Ecosystem
2625

@@ -29,4 +28,5 @@ Newly added documentation to integrate Capsule with other applications:
2928
* [OpenCost](/ecosystem/integrations/opencost/)
3029
* [Headlamp](/ecosystem/integrations/headlamp/)
3130
* [Gangplank](/ecosystem/integrations/gangplank/)
32-
* [Openshift](/docs/operating/setup/openshift/)
31+
* [Teleport](/ecosystem/integrations/teleport/)
32+
* [Openshift](/docs/operating/setup/openshift/)

0 commit comments

Comments
 (0)