Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ WATCH_NAMESPACE ?= ""

IMG ?= $(IMAGE_REGISTRY)/$(REGISTRY_NAMESPACE)/$(IMAGE_NAME):$(IMAGE_TAG)

# NETWORK_POLICY controls whether NetworkPolicy manifests are included
# in the generated installer YAMLs. Set to true to enable.
NETWORK_POLICY ?= false

# ENVTEST_K8S_VERSION refers to the version of kubebuilder assets to be downloaded by envtest binary.
ENVTEST_K8S_VERSION = 1.29.0

Expand Down Expand Up @@ -74,6 +78,15 @@ resources:
endef
export BUILD_CSI_RBAC_OVERLAY

# Helpers to toggle the network-policy kustomize resource.
# Called from build targets when NETWORK_POLICY=true.
define enable-network-policy
$(if $(filter true,$(NETWORK_POLICY)),cd config/default && sed -i 's|#- ../network-policy|- ../network-policy|' kustomization.yaml)
endef
define disable-network-policy
$(if $(filter true,$(NETWORK_POLICY)),cd config/default && sed -i 's|^- ../network-policy|#- ../network-policy|' kustomization.yaml)
endef

.PHONY: all
all: build

Expand Down Expand Up @@ -192,19 +205,23 @@ docker-buildx: ## Build and push docker image for the manager for cross-platform

.PHONY: build-installer
build-installer: manifests generate kustomize ## Generate a consolidated YAML with CRDs and deployment.
$(call enable-network-policy)
mkdir -p build deploy/all-in-one
cd build && echo "$$BUILD_INSTALLER_OVERLAY" > kustomization.yaml
cd build && $(KUSTOMIZE) edit add resource ../config/default/
$(KUSTOMIZE) build build > deploy/all-in-one/install.yaml
rm -rf build
$(call disable-network-policy)

.PHONY: build-openshift-installer
build-openshift-installer: manifests generate kustomize ## Generate a consolidated YAML with CRDs, deployment, and OpenShift SCC.
$(call enable-network-policy)
mkdir -p build deploy/all-in-one
cd build && echo "$$BUILD_INSTALLER_OVERLAY" > kustomization.yaml
cd build && $(KUSTOMIZE) edit add resource ../config/openshift/
$(KUSTOMIZE) build build > deploy/all-in-one/install-openshift.yaml
rm -rf build
$(call disable-network-policy)

.PHONY: build-helm-installer
build-helm-installer: manifests generate kustomize helmify ## Generate helm charts for the operator.
Expand Down Expand Up @@ -275,11 +292,13 @@ uninstall: manifests kustomize ## Uninstall CRDs from the K8s cluster specified

.PHONY: deploy
deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config.
$(call enable-network-policy)
mkdir -p build
cd build && echo "$$BUILD_INSTALLER_OVERLAY" > kustomization.yaml
cd build && $(KUSTOMIZE) edit add resource ../config/default/
$(KUSTOMIZE) build build | $(KUBECTL) apply -f -
rm -rf build
$(call disable-network-policy)

.PHONY: undeploy
undeploy: kustomize ## Undeploy controller from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.
Expand Down
1 change: 1 addition & 0 deletions PendingReleaseNotes.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@
- Fencing can now be enabled in the ceph-csi-drivers Helm chart either globally for all drivers or on a per-driver basis.
- Added `enabled` field to log rotation configuration in the ceph-csi-drivers Helm chart. Users can now disable log rotation by setting `log.rotation.enabled: false` either globally in `operatorConfig.driverSpecDefaults` or per-driver in `drivers.<driver-type>`. This resolves Helm warnings when disabling file logging from parent charts.
- Added support to set the priorityClass in the operator helm chart.
- Added NetworkPolicies for the operator pod and CSI driver pods (controller-plugin, csi-addons node agent). Disabled by default; enabled with `NETWORK_POLICY=true` at build time. Driver NPs are created by the operator when `networking.k8s.io/v1` is available on the cluster. Node-plugin pods are exempt (`hostNetwork: true`).
## NOTE
6 changes: 2 additions & 4 deletions config/default/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,8 @@ resources:
#- ../prometheus
# [METRICS] Expose the controller manager metrics service.
# - metrics_service.yaml
# [NETWORK POLICY] Protect the /metrics endpoint and Webhook Server with NetworkPolicy.
# Only Pod(s) running a namespace labeled with 'metrics: enabled' will be able to gather the metrics.
# Only CR(s) which requires webhooks and are applied on namespaces labeled with 'webhooks: enabled' will
# be able to communicate with the Webhook Server.
# [NETWORK POLICY] Protect the operator pod with NetworkPolicy.
# Denies all ingress and allows open egress for API server access.
#- ../network-policy

#patches:
Expand Down
27 changes: 0 additions & 27 deletions config/network-policy/allow-metrics-traffic.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion config/network-policy/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
resources:
- allow-metrics-traffic.yaml
- operator-network-policy.yaml
14 changes: 14 additions & 0 deletions config/network-policy/operator-network-policy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: ceph-csi-controller-manager
spec:
podSelector:
matchLabels:
control-plane: ceph-csi-op-controller-manager
# Ingress is not specified — deny all inbound traffic.
egress:
- {}
policyTypes:
- Ingress
- Egress
12 changes: 12 additions & 0 deletions config/rbac/role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,18 @@ rules:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- networkpolicies
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- storage.k8s.io
resources:
Expand Down
12 changes: 12 additions & 0 deletions deploy/all-in-one/install-openshift.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15802,6 +15802,18 @@ rules:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- networkpolicies
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- storage.k8s.io
resources:
Expand Down
12 changes: 12 additions & 0 deletions deploy/all-in-one/install.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15788,6 +15788,18 @@ rules:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- networkpolicies
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- storage.k8s.io
resources:
Expand Down
12 changes: 12 additions & 0 deletions deploy/charts/ceph-csi-operator/templates/manager-rbac.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,18 @@ rules:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- networkpolicies
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- storage.k8s.io
resources:
Expand Down
12 changes: 12 additions & 0 deletions deploy/multifile/operator.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,18 @@ rules:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- networkpolicies
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- storage.k8s.io
resources:
Expand Down
86 changes: 86 additions & 0 deletions docs/design/network-policy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# NetworkPolicy for Ceph-CSI Operator and Driver Pods

This document describes the NetworkPolicy implementation for the
ceph-csi-operator and the CSI driver pods it manages.

## Component Inventory

| Component | Pod Labels | hostNetwork | NP Coverage |
|---|---|---|---|
| Operator pod | `control-plane: ceph-csi-op-controller-manager` | No | Static manifest |
| Controller-plugin (per driver) | `app: {driver}-ctrlplugin` | Configurable | Code-generated |
| csi-addons node agent (per driver) | `app: {driver}-nodeplugin-csi-addons` | No | Code-generated |
| Node-plugin DaemonSet | `app: {driver}-nodeplugin` | Yes (hardcoded) | Exempt |

## Policies

### Operator Pod

- **Ingress:** Deny all. No exposed ports (metrics disabled by default,
kubelet health probes bypass NetworkPolicy).
- **Egress:** Open (`- {}`). Required for API server access. The API server
ClusterIP varies per cluster (10.96.0.1 on kubeadm, 172.30.0.1 on OpenShift)
and CNI ipBlock behavior is inconsistent across implementations, so
restricting egress to a specific IP is not portable.

### Controller-Plugin

- **Ingress:** Two rules:
1. csi-addons controller-manager → csi-addons gRPC port (9070 for RBD,
9080 for CephFS). Source restricted to pods with labels
`app.kubernetes.io/name: csi-addons` + `control-plane: controller-manager`
in any namespace.
2. (RBD only) Any pod → snapshot-metadata gRPC port 50051. Open source
because backup applications (Velero, etc.) can run in any namespace
with any labels. See [CSI snapshot-metadata](https://kubernetes-csi.github.io/docs/external-snapshot-metadata.html).
- **Egress:** Open. Required for API server and Ceph cluster access.

### csi-addons Node Agent

- **Ingress:** csi-addons controller-manager → port 9071 only.
- **Egress:** Open.
- **Lifecycle:** Mirrors the csi-addons DaemonSet — created when
`DeployCsiAddons` is true, deleted when false. Not created for NFS drivers.

### Node-Plugin DaemonSet

No NetworkPolicy. Runs with `hostNetwork: true`, which makes it exempt
from NetworkPolicy enforcement.

## Gating

### Static Manifest (Operator Pod)

The operator pod NP is in `config/network-policy/` and referenced from
`config/default/kustomization.yaml` as a commented resource. It is
**disabled by default**. Set `NETWORK_POLICY=true` when building to include it:

```console
NETWORK_POLICY=true make build-installer
```

### Code-Generated NPs (Driver Pods)

Driver pod NPs are created by the operator at runtime. The operator checks
for the `networking.k8s.io/v1` API group at startup via the discovery client.
If the API is not available, no NPs are created and a log message is emitted.

## API Server Egress Rationale

Open egress (`- {}`) is used instead of restricting to a specific API server
IP because:

1. The API server ClusterIP varies per cluster and is unknown at manifest
authoring time.
2. CNI implementations handle `ipBlock` inconsistently with DNAT (OVN-K8s
timing, Cilium quirks).
3. HyperShift clusters may use non-standard API server ports.

This follows the precedent set by
[operator-marketplace PR #723](https://github.com/operator-framework/operator-marketplace/pull/723).

## References

- [CSI snapshot-metadata sidecar](https://kubernetes-csi.github.io/docs/external-snapshot-metadata.html)
- [kubernetes-csi/external-snapshot-metadata](https://github.com/kubernetes-csi/external-snapshot-metadata)
- [operator-marketplace NP precedent](https://github.com/operator-framework/operator-marketplace/pull/723)
30 changes: 30 additions & 0 deletions docs/kubernetes-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -495,3 +495,33 @@ Expected output:
```text
No resources found in ceph-csi-operator-system namespace.
```

## 6. Optional: Enable NetworkPolicies

NetworkPolicies for the operator pod and CSI driver pods are included but
**disabled by default**. To include them in the generated manifests, pass
`NETWORK_POLICY=true` to the build target:

```bash
NETWORK_POLICY=true make build-installer
# or for OpenShift:
NETWORK_POLICY=true make build-openshift-installer
```

When enabled:

- The **operator pod** gets a policy that denies all ingress and allows open
egress (required for API server access).
- Each **controller-plugin** pod gets a policy allowing ingress only from the
csi-addons controller-manager on the csi-addons gRPC port, and (for RBD)
from any pod on port 50051 for snapshot-metadata gRPC.
- Each **csi-addons node agent** pod (when `deployCsiAddons: true`) gets a
policy allowing ingress only from the csi-addons controller-manager on
port 9071.
- The **node-plugin** DaemonSet runs with `hostNetwork: true` and is exempt
from NetworkPolicies.

Driver pod NetworkPolicies are created by the operator when the cluster
supports the `networking.k8s.io/v1` API group (checked at startup).

For design details, see [docs/design/network-policy.md](design/network-policy.md).
Loading
Loading