Skip to content

Security Context Capabilities Merge Bug - Cannot Override Default Capabilities #174

@shivanikothari

Description

@shivanikothari

Company or project name

Describe what's wrong

When deploying ClickHouseCluster or KeeperCluster resources in namespaces with restricted PodSecurity policies, pods fail to start.

The operator's ApplyDefault merge logic makes it impossible to remove default capabilities (IPC_LOCK, PERFMON, SYS_PTRACE) using the standard Kubernetes pattern of setting capabilities.drop: [ ALL]. This violates restricted PodSecurity policies and creates a security footgun.

Problem sequence:

  1. User provides capabilities: { drop: [ALL] } (no add field)
  2. capabilities.add is nil in user config
  3. ApplyDefault function sees nil and fills it with [IPC_LOCK, PERFMON, SYS_PTRACE]
  4. Result: Both add and drop are set

Does it reproduce on the most recent release?

Yes

How to reproduce

Environment

  • ClickHouse Operator version: [latest from main branch]
  • Kubernetes version: 1.28+
  • PodSecurity enforcement: restricted:latest
  1. Create namespace with restricted PodSecurity:
kubectl create ns test-restricted
kubectl label ns test-restricted pod-security.kubernetes.io/enforce=restricted
  1. Apply this manifest:
apiVersion: clickhouse.com/v1alpha1
kind: KeeperCluster
metadata:
  name: test
  namespace: test-restricted
spec:
  replicas: 3
  containerTemplate:
    securityContext:
      capabilities:
        drop:
          - ALL
      allowPrivilegeEscalation: false
      runAsNonRoot: true
      runAsUser: 101
  1. Observe pod creation failure with PodSecurity violation

Expected behavior

Setting containerTemplate.securityContext.capabilities.drop: [ALL] should override the operator's default capabilities and result in no capabilities being added:

spec:
  containerTemplate:
    securityContext:
      capabilities:
        drop:
          - ALL

Error message and/or stacktrace

pods "keeper-0" is forbidden: violates PodSecurity "restricted:latest":
unrestricted capabilities (container "clickhouse-keeper" must not include
"IPC_LOCK", "PERFMON", "SYS_PTRACE" in securityContext.capabilities.add)

Additional context

Users must explicitly set add to an empty array to prevent the merge:

spec:
  containerTemplate:
    securityContext:
      capabilities:
        add: [] 
        drop:
          - ALL

This is non-intuitive and not documented.

While these capabilities improve performance, they should be:

  1. Easy to disable for security-conscious deployments
  2. Documented in user-facing documentation with security trade-offs
  3. Not silently enforced when users try to override them

Metadata

Metadata

Assignees

No one assigned

    Labels

    potential bugTo be reviewed by developers and confirmed/rejected.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions