Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 118 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,124 @@ The TrustedProfileAnalyzer CR spec uses `x-kubernetes-preserve-unknown-fields: t
- `metrics.enabled`: Enable metrics collection
- `tracing.enabled`: Enable distributed tracing

## Cloud Credential Operator (CCO) Integration

The operator supports OpenShift Cloud Credential Operator integration for automatic cloud credential provisioning. CCO eliminates the need to manually create and manage S3 access keys by delegating credential lifecycle to the platform.

### Enabling CCO

Set `cloudProvider` in the CR spec. This is the master toggle — when absent, no CCO resources are created and credentials must be supplied manually via `storage.accessKey`/`storage.secretKey`.

```yaml
spec:
cloudProvider: aws # "aws" or "gcp"
ccoMode: mint # optional: "default", "mint", "passthrough", or "manual"
cloudCredentials:
aws:
statementEntries:
- effect: Allow
action: ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket"]
resource: "*"
```

When `cloudProvider` is set:
- A `CredentialsRequest` resource is created in the `openshift-cloud-credential-operator` namespace
- CCO provisions a Secret named `<release-name>-cloud-creds` in the deployment namespace
- `storage.accessKey` and `storage.secretKey` become **optional** (auto-populated from the CCO secret)

### Disabling CCO

Remove or leave `cloudProvider` unset in the CR spec. When unset:
- No `CredentialsRequest` is created
- No CCO volumes or environment variables are injected into pods
- S3 credentials must be provided explicitly via `storage.accessKey` and `storage.secretKey`

### CCO Modes

| Mode | `ccoMode` value | Description |
|------|----------------|-------------|
| **Default** | `default` | CCO auto-determines provisioning method. |
| **Mint** | `mint` | CCO creates new IAM credentials with least-privilege permissions from `statementEntries`. |
| **Passthrough** | `passthrough` | CCO copies cluster admin credentials to the target namespace. |
| **Manual** | `manual` | Credentials pre-provisioned via `ccoctl` tool (STS/WIF). Requires `stsIAMRoleARN` for AWS. |

### Manual Mode (STS)

Manual mode uses short-lived token-based authentication instead of static access keys. It requires additional configuration:

```yaml
spec:
cloudProvider: aws
ccoMode: manual
cloudCredentials:
aws:
statementEntries:
- effect: Allow
action: ["s3:*"]
resource: "*"
stsIAMRoleARN: "arn:aws:iam::123456789012:role/trustify-s3-role"
```

When manual mode is active, the operator automatically:
- Mounts a projected ServiceAccount token at `/var/run/secrets/openshift/serviceaccount`
- Mounts the CCO credentials secret at `/var/run/secrets/cloud`
- Sets `AWS_SHARED_CREDENTIALS_FILE`, `AWS_WEB_IDENTITY_TOKEN_FILE`, and `AWS_ROLE_ARN` environment variables
- Omits `TRUSTD_S3_ACCESS_KEY` / `TRUSTD_S3_SECRET_KEY` (the AWS SDK uses STS instead)

### RDS IAM Authentication via CCO

The operator supports using CCO-provisioned credentials for RDS IAM authentication, eliminating the need for static database passwords. This is controlled by `ccoRds.enabled` and requires `cloudProvider` to be set.

```yaml
spec:
cloudProvider: aws
ccoRds:
enabled: true
region: us-east-1
cloudCredentials:
aws:
statementEntries:
- effect: Allow
action: ["s3:*", "rds-db:connect"]
resource: "*"
database:
host: mydb.cluster-xyz.us-east-1.rds.amazonaws.com
name: trustify
username: trustify_user
# password is NOT required when ccoRds.enabled is true
```

When `ccoRds.enabled` is true:
- `TRUSTD_DB_IAM_AUTH=true` is set on all trustd pods
- `TRUSTD_DB_REGION` is set from `ccoRds.region`
- `database.password` becomes optional (omitted from env vars)
- SSL mode is forced to `require` (RDS IAM auth mandates TLS)
- For **mint/passthrough/default** modes: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are populated from the CCO secret
- For **manual** (STS) mode: the existing STS volumes/env vars are sufficient — no extra credentials needed

When `ccoRds.enabled` is false or absent, behavior is unchanged — `database.password` is required and SSL mode uses the configured value.

**Note:** The `create-database` and `create-importers` init jobs use `psql` directly and cannot generate RDS IAM tokens. These jobs continue requiring static credentials via `createDatabase.database`. The `migrate-database` job runs `trustd db migrate` and supports RDS IAM auth.

### Key Files

| File | Purpose |
|------|---------|
| `helm-charts/.../templates/credentialrequest.yaml` | CredentialsRequest template (conditional on `cloudProvider`) |
| `helm-charts/.../templates/helpers/_cco.tpl` | Helper templates for manual mode volumes/mounts/env vars and RDS IAM auth |
| `helm-charts/.../templates/helpers/_storage.tpl` | S3 env vars — branches on CCO vs manual credentials |
| `helm-charts/.../templates/helpers/_postgres.tpl` | Database env vars — branches on ccoRds for password/SSL |
| `config/rbac/clusterrole.yaml` | ClusterRole granting access to `credentialsrequests` API |
| `config/rbac/clusterrolebinding_cco.yaml` | Binds the CCO ClusterRole to the operator ServiceAccount |
| `test/fixtures/aws_cco_*.yaml` | Example CRs for each CCO mode |
| `test/fixtures/aws_cco_rds_cr.yaml` | Example CR for RDS IAM auth (mint mode) |
| `test/fixtures/aws_cco_manual_rds_cr.yaml` | Example CR for RDS IAM auth (manual/STS mode) |
| `test/e2e/cco_helm_rendering_test.go` | E2E tests for CCO template rendering |

### RBAC

The operator requires a ClusterRole with permissions on `cloudcredential.openshift.io/credentialsrequests` (create, delete, get, list, patch, update, watch). This is configured in `config/rbac/clusterrole.yaml` and bound via `config/rbac/clusterrolebinding_cco.yaml`.

## Linting

The project uses golangci-lint with configuration in `.golangci.yml`. Enabled linters include:
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.rhtpa-operator.rh
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ LABEL features.operators.openshift.io/proxy-aware="false"
LABEL features.operators.openshift.io/cnf="false"
LABEL features.operators.openshift.io/csi="false"
LABEL features.operators.openshift.io/tls-profiles="false"
LABEL features.operators.openshift.io/token-auth-aws="false"
LABEL features.operators.openshift.io/token-auth-aws="true"
LABEL features.operators.openshift.io/token-auth-azure="false"
LABEL features.operators.openshift.io/token-auth-gcp="false"

Expand Down
2 changes: 1 addition & 1 deletion bundle.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ LABEL features.operators.openshift.io/proxy-aware="false"
LABEL features.operators.openshift.io/cnf="false"
LABEL features.operators.openshift.io/csi="false"
LABEL features.operators.openshift.io/tls-profiles="false"
LABEL features.operators.openshift.io/token-auth-aws="false"
LABEL features.operators.openshift.io/token-auth-aws="true"
LABEL features.operators.openshift.io/token-auth-azure="false"
LABEL features.operators.openshift.io/token-auth-gcp="false"
# Core bundle labels.
Expand Down
2 changes: 1 addition & 1 deletion bundle/manifests/rhtpa-operator.clusterserviceversion.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ metadata:
features.operators.openshift.io/fips-compliant: "false"
features.operators.openshift.io/proxy-aware: "false"
features.operators.openshift.io/tls-profiles: "false"
features.operators.openshift.io/token-auth-aws: "false"
features.operators.openshift.io/token-auth-aws: "true"
features.operators.openshift.io/token-auth-azure: "false"
features.operators.openshift.io/token-auth-gcp: "false"
operators.openshift.io/valid-subscription: '["Red Hat Trusted Profile Analyzer"]'
Expand Down
2 changes: 1 addition & 1 deletion bundle/metadata/annotations.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ annotations:
features.operators.openshift.io/cnf: "false"
features.operators.openshift.io/csi: "false"
features.operators.openshift.io/tls-profiles: "false"
features.operators.openshift.io/token-auth-aws: "false"
features.operators.openshift.io/token-auth-aws: "true"
features.operators.openshift.io/token-auth-azure: "false"
features.operators.openshift.io/token-auth-gcp: "false"
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ metadata:
features.operators.openshift.io/fips-compliant: "false"
features.operators.openshift.io/proxy-aware: "false"
features.operators.openshift.io/tls-profiles: "false"
features.operators.openshift.io/token-auth-aws: "false"
features.operators.openshift.io/token-auth-aws: "true"
features.operators.openshift.io/token-auth-azure: "false"
features.operators.openshift.io/token-auth-gcp: "false"
operators.openshift.io/valid-subscription: '["Red Hat Trusted Profile Analyzer"]'
Expand Down
8 changes: 8 additions & 0 deletions config/rbac/clusterrole.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cco-credentialsrequest-access
rules:
- apiGroups: ["cloudcredential.openshift.io"]
resources: ["credentialsrequests"]
verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]
15 changes: 15 additions & 0 deletions config/rbac/clusterrolebinding_cco.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/name: rhtpa-operator
app.kubernetes.io/managed-by: kustomize
name: cco-credentialsrequest-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cco-credentialsrequest-access
subjects:
- kind: ServiceAccount
name: rhtpa-operator-controller-manager
namespace: placeholder
2 changes: 2 additions & 0 deletions config/rbac/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ resources:
- rolebinding_job.yaml
- role_cluster_ingress.yaml
- role_cluster_ingress_binding.yaml
- clusterrole.yaml
- clusterrolebinding_cco.yaml
# The following RBAC configurations are used to protect
# the metrics endpoint with authn/authz. These configurations
# ensure that only authorized users and service accounts
Expand Down
24 changes: 24 additions & 0 deletions devel/cco/credentialRequest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: cloudcredential.openshift.io/v1
kind: CredentialsRequest
metadata:
name: my-operator-credentials
namespace: openshift-cloud-credential-operator
spec:
secretRef:
name: my-cloud-creds
namespace: my-operator-namespace
providerSpec:
apiVersion: cloudcredential.openshift.io/v1
kind: AWSProviderSpec # AWS
statementEntries:
- effect: Allow
action:
- "s3:GetObject"
- "s3:PutObject"
- "s3:DeleteObject"
- "s3:ListBucket"
- "s3:GetBucketLocation"
- "s3:ListBucketMultipartUploads"
- "s3:AbortMultipartUpload"
- "s3:ListMultipartUploadParts"
resource: "*"
100 changes: 100 additions & 0 deletions extensions/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Tokenize with Cloud Credentials Operator

Cloud Credentials Operator (CCO) is installed by default on OCP.
To check the CCO status

```console
oc get clusteroperator cloud-credential
```
it shows something like

```console
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE
cloud-credential 4.x.x True False False ...
```
Pod status
```console
oc get pods -n openshift-cloud-credential-operator
```

Credential requests checks

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (typo): Rephrase this heading for clearer grammar (singular vs plural).

The phrase "Credential requests checks" is awkward. Consider "CredentialRequest checks" or "Credential request checks" for clearer, grammatically correct wording.

Suggested change
Credential requests checks
CredentialRequest checks

```console
oc get credentialsrequests -n openshift-cloud-credential-operator
```

CCO details
```console
oc describe clusteroperator cloud-credential
```

CCO Modality
```console
oc get cloudcredential cluster -o yaml
```
On spec.credentialsMode will be the configured setting (Mint,
Passthrough, Manual, or empty for default).

## How the Operator interacts with the CCO

1. Operator declares permissions needed in a CredentialsRequest CR in the namespace openshift-cloud-credential-operator

```console
apiVersion: cloudcredential.openshift.io/v1
kind: CredentialsRequest
metadata:
name: my-operator-credentials
namespace: openshift-cloud-credential-operator
spec:
secretRef:
name: my-cloud-creds
namespace: my-operator-namespace
providerSpec:
apiVersion: cloudcredential.openshift.io/v1
kind: AWSProviderSpec # esempio per AWS
statementEntries:
Comment thread
desmax74 marked this conversation as resolved.
- effect: Allow
action:
- "s3:GetObject"
- "s3:PutObject"
resource: "*"
```

2. CCO processes the CR and create a Kubernetes Secret with the cloud credentails
in the namespace specified in spec.secretRef.

3. Operator reads the Secret and uses the credentials to interact with the cloud API.
The Operator must tolerate the non-immediate availability of the Secret because it takes time to create.

## How to integrate the CCO with the Helm Chart Operator

1. Define Credential Request in the chart
2.
2. Configure Deployment to use the secret created by the CCO

3. Handling the delay of the creation of the secret
first approach: init container
```console
initContainers:
- name: wait-for-creds
image: registry.redhat.io/openshift4/ose-cli
command:
- /bin/bash
- -c
- |
until oc get secret {{ .Release.Name }}-cloud-creds -n {{
.Release.Namespace }} 2>/dev/null; do
echo "Waiting for cloud credentials..."
sleep 5
done
```
second approach Retry in the Operator's code, but this is available on a full Go Operator

4. Support different cloud providers with values configurations

5. RBAC needed to create the CredentialsRequest
6. Support Manual Mode (STS/WIF): if the cluster uses STS mode, CCO doesn’t create the secret automatically.
In STS mode the user must :
1. Extract CredentialsRequest from chart
2. Use ccoctl tool to generate the credentials
3. Create the secrets manually before installing the chart.

Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{{- if .Values.cloudProvider }}
apiVersion: cloudcredential.openshift.io/v1
kind: CredentialsRequest
metadata:
name: {{ .Release.Name }}-cloud-creds
namespace: openshift-cloud-credential-operator
spec:
secretRef:
name: {{ .Release.Name }}-cloud-creds
namespace: {{ .Release.Namespace }}
{{- if eq (toString .Values.ccoMode) "manual" }}
cloudTokenPath: /var/run/secrets/openshift/serviceaccount/token
{{- end }}
{{- if eq .Values.cloudProvider "aws" }}
providerSpec:
apiVersion: cloudcredential.openshift.io/v1
kind: AWSProviderSpec
statementEntries:
{{- toYaml .Values.cloudCredentials.aws.statementEntries | nindent 6 }}
{{- if eq (toString .Values.ccoMode) "manual" }}
stsIAMRoleARN: {{ .Values.cloudCredentials.aws.stsIAMRoleARN | default "" | quote }}
{{- end }}
{{- else if eq .Values.cloudProvider "gcp" }}
providerSpec:
apiVersion: cloudcredential.openshift.io/v1
kind: GCPProviderSpec
predefinedRoles:
{{- toYaml .Values.cloudCredentials.gcp.permissions | nindent 6 }}
{{- end }}
{{- end }}
Loading
Loading