Skip to content

Commit 29b4fd7

Browse files
sebrandon1sebrandon1
committed
Add self-signed/CA issuers, hub-spoke CA trust, and root CA monitoring
Addresses four cert-manager trust chain gaps for the telco RDS: - OCPBUGS-85773: Add self-signed and CA ClusterIssuer alternatives for disconnected environments (hub, core, ran profiles) - OCPBUGS-85776: Add kubeconfig trust update guidance after API server cert replacement with custom CA - OCPBUGS-85774: Add hub-spoke bidirectional CA trust configuration (spoke CA trust ConfigMap + hub CA distribution ACM policy) - OCPBUGS-85777: Add root CA expiration monitoring via PrometheusRule ACM policy (90-day warning, 30-day critical alerts) Also extends the existing CertificatePolicy to monitor the cert-manager namespace and adds kube-compare templates for the new issuers.
1 parent ef013bd commit 29b4fd7

30 files changed

Lines changed: 638 additions & 14 deletions

‎telco-core/configuration/reference-crs-kube-compare/optional/cert-manager/certManagerClusterIssuerCA.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
apiVersion: cert-manager.io/v1
3+
kind: ClusterIssuer
4+
metadata:
5+
name: {{ .metadata.name | default "ca-issuer" }}
6+
spec:
7+
ca:
8+
secretName: {{ .spec.ca.secretName | default "root-ca-secret" }}

‎telco-core/configuration/reference-crs-kube-compare/optional/cert-manager/certManagerClusterIssuerSelfSigned.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
apiVersion: cert-manager.io/v1
3+
kind: ClusterIssuer
4+
metadata:
5+
name: {{ .metadata.name | default "selfsigned-issuer" }}
6+
spec:
7+
selfSigned: {}

‎telco-core/configuration/reference-crs-kube-compare/optional/cert-manager/certManagerRootCACertificate.yaml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
apiVersion: cert-manager.io/v1
3+
kind: Certificate
4+
metadata:
5+
name: {{ .metadata.name | default "root-ca" }}
6+
namespace: {{ .metadata.namespace | default "cert-manager" }}
7+
spec:
8+
isCA: true
9+
commonName: {{ .spec.commonName | default "root-ca" }}
10+
secretName: {{ .spec.secretName | default "root-ca-secret" }}
11+
duration: {{ .spec.duration | default "87600h" }}
12+
renewBefore: {{ .spec.renewBefore | default "720h" }}
13+
privateKey:
14+
algorithm: {{ .spec.privateKey.algorithm | default "ECDSA" }}
15+
size: {{ .spec.privateKey.size | default 256 }}
16+
issuerRef:
17+
name: {{ .spec.issuerRef.name | default "selfsigned-issuer" }}
18+
kind: {{ .spec.issuerRef.kind | default "ClusterIssuer" }}
19+
group: {{ .spec.issuerRef.group | default "cert-manager.io" }}

‎telco-core/configuration/reference-crs/optional/cert-manager/README.md‎

Lines changed: 57 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,14 @@ Cert-manager automates the management and issuance of TLS certificates from vari
1616
- `certManagerOperatorgroup.yaml` - Creates the OperatorGroup for cert-manager
1717
- `certManagerSubscription.yaml` - Installs the OpenShift cert-manager operator
1818

19-
### Certificate Issuer
20-
- `certManagerClusterIssuer.yaml` - Configures an ACME ClusterIssuer using Let's Encrypt with DNS-01 challenge
19+
### Certificate Issuers
20+
21+
Three ClusterIssuer types are available. Choose one based on your environment:
22+
23+
- `certManagerClusterIssuer.yaml` - **ACME issuer** using Let's Encrypt with DNS-01 challenge (requires internet connectivity)
24+
- `certManagerClusterIssuerSelfSigned.yaml` - **Self-signed issuer** for disconnected or air-gapped environments
25+
- `certManagerClusterIssuerCA.yaml` - **CA issuer** using an internal certificate authority (disconnected environments with existing PKI)
26+
- `certManagerRootCACertificate.yaml` - Bootstraps a self-signed root CA for use with the CA issuer
2127

2228
### Certificate Resources
2329
- `apiServerCertificate.yaml` - Creates a certificate for the API Server endpoint
@@ -31,11 +37,24 @@ Cert-manager automates the management and issuance of TLS certificates from vari
3137

3238
Before applying these configurations, you must customize the following:
3339

34-
1. **ClusterIssuer** (`certManagerClusterIssuer.yaml`):
40+
1. **ClusterIssuer** — choose one of the following issuer configurations:
41+
42+
**Option A: ACME issuer** (`certManagerClusterIssuer.yaml`) — for connected environments:
3543
- Update `email` with your contact email
3644
- Configure the appropriate DNS provider for DNS-01 challenge (example shows Route53)
3745
- Add necessary credentials for your DNS provider
3846

47+
**Option B: Self-signed CA** (disconnected environments) — deploy these three files in order:
48+
1. `certManagerClusterIssuerSelfSigned.yaml` — creates the self-signed bootstrap issuer
49+
2. `certManagerRootCACertificate.yaml` — creates a root CA certificate (10-year duration)
50+
3. `certManagerClusterIssuerCA.yaml` — creates the CA issuer using the root CA
51+
- Update the Certificate `apiServerCertificate.yaml` and `ingressCertificate.yaml` to reference `ca-issuer` instead of `acme-issuer` in the `issuerRef` field
52+
53+
**Option C: Existing internal CA** (disconnected environments with existing PKI):
54+
- Create a Secret named `root-ca-secret` in the `cert-manager` namespace containing your CA's `tls.crt` and `tls.key`
55+
- Deploy only `certManagerClusterIssuerCA.yaml`
56+
- Update the Certificate resources to reference `ca-issuer` in the `issuerRef` field
57+
3958
2. **Certificates** (`apiServerCertificate.yaml` and `ingressCertificate.yaml`):
4059
- Update `commonName` and `dnsNames` to match your cluster's domain
4160
- Example: Replace `api.example.com` with your actual API endpoint
@@ -51,7 +70,8 @@ Before applying these configurations, you must customize the following:
5170
3. Deploy the ClusterIssuer
5271
4. Deploy the Certificate resources
5372
5. Wait for certificates to be issued and secrets created
54-
6. Apply the APIServer and IngressController configurations
73+
6. **(Option B & C only)** Update kubeconfig to trust the new root CA (see "Kubeconfig Trust" section below)
74+
7. Apply the APIServer and IngressController configurations
5575

5676
## Certificate Verification
5777

@@ -61,6 +81,39 @@ After applying these configurations, verify that:
6181
- API Server is using the certificate: Test HTTPS connection to API endpoint
6282
- Ingress is using the certificate: Test HTTPS connection to any route
6383

84+
## Important: Kubeconfig Trust After API Server Cert Replacement
85+
86+
> **Note:** For Option B (Self-signed CA) and Option C (Existing internal CA), you must complete
87+
> this kubeconfig update *before* applying the APIServer configuration (step 7 in the deployment
88+
> order above). Applying the APIServer configuration first will lock you out.
89+
90+
> **Warning:** When cert-manager replaces the API server certificate with one signed by a custom CA
91+
> (self-signed or internal), existing kubeconfig files become invalid. The embedded
92+
> `certificate-authority-data` still references the original cluster CA and cannot verify the new
93+
> certificate. All `oc` and API client commands will fail with `x509: certificate signed by unknown authority`.
94+
95+
### Updating kubeconfig
96+
97+
1. Extract the new root CA certificate:
98+
```bash
99+
oc get secret root-ca-secret -n cert-manager -o jsonpath='{.data.ca\.crt}' | base64 -d > /tmp/root-ca.crt
100+
```
101+
102+
2. Update your kubeconfig to trust the new CA:
103+
```bash
104+
oc config set-cluster $(oc config current-context | cut -d/ -f2) \
105+
--certificate-authority=/tmp/root-ca.crt --embed-certs
106+
```
107+
108+
3. Verify connectivity:
109+
```bash
110+
oc cluster-info
111+
```
112+
113+
### Best practice for PKI environments
114+
115+
Generate a root CA once and use it as the root for your PKI (the ACME issuer or CA issuer your clusters will use). Add the root CA PEM to `/etc/pki/ca-trust/source/anchors/` on your workstation and run `update-ca-trust`. All certificates issued from that root CA will then be trusted without per-cluster kubeconfig updates.
116+
64117
## References
65118

66119
- [OpenShift Cert-Manager Operator Documentation](https://docs.openshift.com/container-platform/latest/security/cert_manager_operator/index.html)

‎telco-core/configuration/reference-crs/optional/cert-manager/certManagerClusterIssuerCA.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
apiVersion: cert-manager.io/v1
3+
kind: ClusterIssuer
4+
metadata:
5+
name: ca-issuer
6+
spec:
7+
ca:
8+
# References a Secret in the cert-manager namespace containing tls.crt and tls.key
9+
# for your internal CA. Create this Secret before applying the ClusterIssuer.
10+
# To bootstrap a self-signed root CA, apply certManagerRootCACertificate.yaml first,
11+
# which creates the root-ca-secret automatically.
12+
secretName: root-ca-secret

‎telco-core/configuration/reference-crs/optional/cert-manager/certManagerClusterIssuerSelfSigned.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
apiVersion: cert-manager.io/v1
3+
kind: ClusterIssuer
4+
metadata:
5+
name: selfsigned-issuer
6+
spec:
7+
selfSigned: {}

‎telco-core/configuration/reference-crs/optional/cert-manager/certManagerRootCACertificate.yaml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
---
2+
# Bootstrap a self-signed root CA for the ca-issuer ClusterIssuer.
3+
# This creates a root-ca-secret in the cert-manager namespace.
4+
# Apply certManagerClusterIssuerSelfSigned.yaml first to provide the
5+
# selfsigned-issuer needed to sign this root CA certificate.
6+
apiVersion: cert-manager.io/v1
7+
kind: Certificate
8+
metadata:
9+
name: root-ca
10+
namespace: cert-manager
11+
spec:
12+
isCA: true
13+
commonName: root-ca
14+
secretName: root-ca-secret
15+
duration: 87600h # 10 years
16+
renewBefore: 720h # 30 days
17+
privateKey:
18+
algorithm: ECDSA
19+
size: 256
20+
issuerRef:
21+
name: selfsigned-issuer
22+
kind: ClusterIssuer
23+
group: cert-manager.io

‎telco-hub/configuration/reference-crs-kube-compare/compare_ignore‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,19 @@ required/gitops/addPluginsPolicy.yaml
3333
# not include the full policy content due to policy templating.
3434
required/acm/observabilityRoutePolicy.yaml
3535

36+
# cert-manager config CRs (user-specific, and policies with hub-side templating)
37+
optional/cert-manager/certManagerClusterIssuerCA.yaml
38+
optional/cert-manager/certManagerClusterIssuerSelfSigned.yaml
39+
optional/cert-manager/certManagerRootCACertificate.yaml
40+
optional/cert-manager/certManagerSpokeCATrustCM.yaml
41+
optional/cert-manager/certManagerHubCATrustPolicy.yaml
42+
optional/cert-manager/certManagerHubCATrustPolicyPlacement.yaml
43+
optional/cert-manager/certManagerHubCATrustPolicyPlacementBinding.yaml
44+
optional/cert-manager/certManagerRootCAExpirationPolicy.yaml
45+
optional/cert-manager/certManagerRootCAExpirationPolicyPlacement.yaml
46+
optional/cert-manager/certManagerRootCAExpirationPolicyPlacementBinding.yaml
47+
optional/cert-manager/kustomization.yaml
48+
3649
required/gitops/extra-manifests-policy.yaml
3750
# ArgoCD files
3851
kustomization.yaml

‎telco-hub/configuration/reference-crs-kube-compare/optional/cert-manager/certManagerCertificatePolicy.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ spec:
1919
spec:
2020
minimumDuration: {{ .spec.minimumDuration | default "720h" }}
2121
namespaceSelector:
22-
include:{{- template "unorderedListAllowExtra" (list .spec.namespaceSelector.include (list "openshift-ingress" "openshift-config") ) }}
22+
include:{{- template "unorderedListAllowExtra" (list .spec.namespaceSelector.include (list "openshift-ingress" "openshift-config" "cert-manager") ) }}
2323
remediationAction: inform
2424
severity: low
2525
remediationAction: inform

‎telco-hub/configuration/reference-crs-kube-compare/optional/cert-manager/certManagerClusterIssuerCA.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
apiVersion: cert-manager.io/v1
3+
kind: ClusterIssuer
4+
metadata:
5+
name: {{ .metadata.name | default "ca-issuer" }}
6+
annotations:
7+
argocd.argoproj.io/sync-wave: "-30"
8+
argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
9+
spec:
10+
ca:
11+
secretName: {{ .spec.ca.secretName | default "root-ca-secret" }}

0 commit comments

Comments
 (0)