Commit 587ee8e
Clean up certificate handling
There were multiple issues with how the operator handled certificates:
1) The lightspeed-stack pod used REQUESTS_CA_BUNDLE and SSL_CERT_FILE
environment variables, which bypassed the system-configured
certificates.
2) When a user provided a custom CA certificate, it was expected under
the cert.crt key in their ConfigMap. This was undocumented and the
required key name was not obvious.
3) PostgresDB appeared to be configured for mTLS because ssl_ca_file
was set in postgres.conf and the openshift-service-ca.crt was
mounted into the PostgresDB pod. This created a false sense of mTLS
being in place, but with the default pg_hba.conf, client certificate
verification [1] is not enabled. Neither OGX [3][4] nor Lightspeed
Stack [5] supports providing client certificates to PostgresDB.
4) PostgresDB SSL connection settings were configured for OGX even
though they have no effect. OGX does not support configuring the
SSL mode for its PostgresDB connection [3][4], so the PostgresDB
certificate verification cannot be strictly enforced on the OGX
side (the default is "prefer" [2], which does not enforce
certificate verification and can fall back to unencrypted
communication). OGX uses a non-strict config mode, so unrecognized
options are silently ignored.
5) The operator did not watch for changes to ConfigMaps. When the
content of the CA bundle ConfigMap was updated, the operator did
not automatically reconcile.
6) Not a bug strictly speaking, but Lightspeed Stack used ssl_mode
"require" when it could have used "verify-full", which checks both
that the certificate is signed by a trusted CA and that the server
hostname matches the CN field in the certificate.
This commit simplifies certificate handling with the following changes:
- Introduce a single CA bundle ConfigMap
(openstack-lightspeed-ca-bundle) containing the system CAs,
user-provided CA certificates from the OpenStackLightspeed CRD,
kube-root-ca.crt, and openshift-service-ca.crt. This bundle is
mounted into all containers in the lightspeed-stack-deployment pod,
eliminating the need for REQUESTS_CA_BUNDLE and SSL_CERT_FILE.
- When a user specifies a ConfigMap with custom CA certificates,
iterate over all keys, validate that each holds a valid certificate,
and append it to the CA bundle (resolves #2).
- Stop mounting openshift-service-ca into the Postgres pod and remove
ssl_ca_file from postgres.conf. These gave a false sense of client
certificate validation; actually enforcing it requires configuring
pg_hba.conf [1], and neither OGX [3][4] nor Lightspeed Stack [5]
currently supports providing client certificates.
- Remove ssl_mode, ca_cert_path, and gss_encmode from
storage.backends.postgres_backend in ogx_config.yaml. These options
are not supported by OGX [3][4] and gave a false sense of SSL being
configured.
- Add a Watch() on ConfigMaps to the reconciler so that whenever a
user updates the CA bundle ConfigMap, the reconcile loop runs
automatically.
- Configure Lightspeed Stack with ssl_mode "verify-full" for its
PostgreSQL connection, ensuring both CA trust and hostname
verification.
[1]
https://www.postgresql.org/docs/current/ssl-tcp.html#SSL-CLIENT-CERTIFICATES
[2]
https://magicstack.github.io/asyncpg/current/api/index.html#asyncpg.connection.connect
[3]
https://github.com/ogx-ai/ogx/blob/34d7901/src/ogx/core/storage/datatypes.py#L200
[4]
https://github.com/ogx-ai/ogx/blob/34d7901/src/ogx/core/storage/sqlalchemy_sqlstore.py#L125
[5]
https://github.com/lightspeed-core/lightspeed-stack/blob/7503ebd/src/models/config.py#L181
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>1 parent dbe8c44 commit 587ee8e
20 files changed
Lines changed: 1169 additions & 257 deletions
File tree
- internal/controller
- assets
- test/kuttl
- common
- expected-configs
- openstack-lightspeed-instance
- tests/update-openstacklightspeed
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
0 commit comments