[DPE-10615] Migrate TLS to the single-kernel library (VM) (#1816)

* feat(tls): migrate TLS to the single-kernel library

Consume the single-kernel library's operator-certificate TLS handler
(events.tls.TLS) and TLSManager instead of the charm-side src/relations/tls.py,
stacked on the DPE-10062 Patroni/cluster port (#1788). The TLS events handler
owns the two certificate requirers and is constructor-injected into TLSManager,
whose live-fetch getters read cert/key from them. A charm-side reload bridge
(_reload_tls_after_push) reloads PostgreSQL after the lib handler stores+pushes
certs; it also fires on relation_broken so detaching the TLS operator re-renders
Patroni with TLS disabled.

Removes src/relations/tls.py and push_tls_files_to_workload (now owned by the
lib), routes internal-cert regeneration through _regenerate_internal_cert,
redirects the operator-cert getters to tls_manager, and pins the library to the
TLS stack tip via archive URL (16.3.2 is not yet released).

Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

* build(deps): bump single-kernel lib to the 16.3.3 stack tip

The TLS lib stack was rebased onto current 16/edge and re-bumped to
16.3.3 because 16/edge had meanwhile shipped its own 16.3.2, and the
rebase dropped the dead workload parameter from the TLS events handler
constructor, so the pin and the construction call move together.

Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

* test(tls): assert relation_broken bridge ordering and exception defer

The relation_broken bridge registrations reference only self.on[...], so
an __init__ reorder above the lib handler's construction would silently
run the reload before the lib clears state and pushes files; the prior
test asserted membership without order. The bridge's except-and-defer
branch also had no coverage, so a refactor could drop the broad guard
without failing CI.

Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

* build(deps): bump single-kernel pin to a1d2178 (cleaned lib)

Re-pin the unreleased single-kernel library from 5a9fa05 to a1d2178,
the rebased TLS-stack tip. a1d2178 includes the operator-cert manager/
events handler (the migration payload) plus the docstring/comment cleanup
that dropped stale operator-repo file-path references and #168/V3-V4 idiom
mentions from the lib. The 5a9fa05→a1d2178 diff is docstring/comment-only,
so no charm-side code changes are required.

Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

* test(tls): drop observer-wiring tests from test_tls.py

Remove _platform_machine, _observers_for, and the three
reload-bridge observer-ordering tests that exercised charm-side
__init__ wiring now that the bridge itself is transitional and
scheduled to leave the charm when update_config ports to the lib.

_platform_machine was redundant: refresh_versions.toml keys both
x86_64 and aarch64, so the platform.machine() snap-revision lookup
succeeds natively on either arch without the monkeypatch (proven by
the suite passing on aarch64 after removal). Its harness dependency
and the now-unused TLS_CLIENT/TLS_PEER_RELATION imports go with it.

Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

* build(deps): pin single-kernel lib to published 16.3.3

The TLS flip branch pinned postgresql-charms-single-kernel to a GitHub
archive URL (commit a1d2178) to test unreleased lib changes. 16.3.3 is
now published to PyPI carrying the same TLS state + manager/events
content, so restore the published-version pin form and regenerate
poetry.lock against the PyPI wheel/tarball.

Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

* build(deps): pin single-kernel to PITR-fix archive on TLS branch

Point the lib dependency at the #179 fix archive (16.3.4 = 16.3.3 plus the
get_member_status RetryError guard) so this TLS-migration branch's CI proves
the PITR restore integration test stops failing before 16.3.4 is published.
The archive carries the same 4-arg TLSManager this branch already targets, so
no charm code changes are needed. Temporary: revert to the published version
once the lib release lands.

Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

* build(deps): pin single-kernel lib to published 16.3.4

Move the TLS branch off the pre-release GitHub-archive commit pin to the
released PyPI 16.3.4 package. 16.3.4 ships the DPE-10663 fix for
Patroni get_member_status when the cluster is unreachable — the path
test_tls_enabled exercises when patroni is stopped mid-test — which the
archive commit pin was capturing ahead of release; the published release
now supersedes that stopgap.

Drop the redundant `vm` extra: the charm already declares charmlibs-snap,
charmlibs-systemd, psutil, and pysyncobj as direct dependencies, so the
lib extra only re-declared them.

Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

* build(deps): restore the vm extra on the single-kernel pin

Keep both extras on the lib pin to match the branch's original pin
shape. The vm extra carries the lower-bound constraints for the
VM-only deps (charmlibs-snap, charmlibs-systemd, psutil, pysyncobj)
that the lib expects, which the charm also declares directly.

Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

---------

Signed-off-by: Marcelo Henrique Neppel <marcelo.neppel@canonical.com>

Author: marceloneppel

poetry.lock

@@ -21,7 +21,7 @@ charm-refresh = "^3.1.0.2"
 charmlibs-snap = "^1.0.1"
 charmlibs-systemd = "^1.0.0.post0"
 charmlibs-interfaces-tls-certificates = "^1.8.3"
-postgresql-charms-single-kernel = {extras = ["postgresql", "vm"], version = "16.3.2"}
+postgresql-charms-single-kernel = {extras = ["postgresql", "vm"], version = "16.3.4"}
 
 [tool.poetry.group.charm-libs.dependencies]
 # data_platform_libs/v0/data_interfaces.py

pyproject.toml

@@ -109,10 +109,8 @@
     SNAP_USER,
     SPI_MODULE,
     SYSTEM_USERS,
-    TLS_CA_BUNDLE_FILE,
-    TLS_CA_FILE,
-    TLS_CERT_FILE,
-    TLS_KEY_FILE,
+    TLS_CLIENT_RELATION,
+    TLS_PEER_RELATION,
     TRACING_PROTOCOL,
     TRACING_RELATION_NAME,
     UNIT_SCOPE,
@@ -121,12 +119,13 @@
 )
 from single_kernel_postgresql.core.config import CharmConfig
 from single_kernel_postgresql.core.state import CharmState
+from single_kernel_postgresql.events.tls import TLS
 from single_kernel_postgresql.events.tls_transfer import TLSTransfer
 from single_kernel_postgresql.managers.cluster import ClusterManager
 from single_kernel_postgresql.managers.config import ConfigManager
 from single_kernel_postgresql.managers.patroni import PatroniManager
 from single_kernel_postgresql.managers.tls import TLSManager
-from single_kernel_postgresql.utils import label2name, new_password, render_file
+from single_kernel_postgresql.utils import label2name, new_password
 from single_kernel_postgresql.utils.postgresql import (
     ACCESS_GROUP_IDENTITY,
     ACCESS_GROUPS,
@@ -155,7 +154,6 @@
 )
 from constants import (
     MONITORING_SNAP_SERVICE,
-    PATRONI_CONF_PATH,
     PGBACKREST_MONITORING_SNAP_SERVICE,
     POSTGRESQL_DATA_DIR,
     RAFT_PARTNER_PREFIX,
@@ -167,7 +165,6 @@
 from ldap import PostgreSQLLDAP
 from relations.async_replication import PostgreSQLAsyncReplication
 from relations.postgresql_provider import PostgreSQLProvider
-from relations.tls import TLS
 from relations.watcher import PostgreSQLWatcherRelation
 from rotate_logs import RotateLogs
 
@@ -379,11 +376,10 @@ def __init__(self, *args):
 
         # TODO switch to the abstract class base
         # State
-        self.state = CharmState(charm=self, substrate=self.substrate)  # type: ignore
+        self.state = CharmState(charm=self, substrate=self.substrate)
 
         # Managers
         self.patroni_manager = PatroniManager(state=self.state, workload=self.workload)
-        self.tls_manager = TLSManager(state=self.state, workload=self.workload)
         self.cluster_manager = ClusterManager(state=self.state, workload=self.workload)
         self.config_manager = ConfigManager(state=self.state, workload=self.workload)
 
@@ -421,7 +417,31 @@ def __init__(self, *args):
         self.postgresql_client_relation = PostgreSQLProvider(self)
         self.backup = PostgreSQLBackups(self, "s3-parameters")
         self.ldap = PostgreSQLLDAP(self, "ldap")
-        self.tls = TLS(self, PEER_RELATION)
+        # TLS events handler owns the two cert requirers; build it before the TLS
+        # manager so the manager can constructor-inject them for its live-fetch getters.
+        self.tls = TLS(self, self.state)
+        self.tls_manager = TLSManager(
+            state=self.state,
+            workload=self.workload,
+            client_certificate=self.tls.client_certificate,
+            peer_certificate=self.tls.peer_certificate,
+        )
+        # Bridge the lib TLS handler's requirer events back into a PostgreSQL reload:
+        # the lib handler stores+pushes certs on certificate_available, then we reload.
+        # Also fires on relation_broken so detaching the TLS operator re-renders Patroni
+        # with TLS disabled.
+        self.framework.observe(
+            self.tls.client_certificate.on.certificate_available, self._reload_tls_after_push
+        )
+        self.framework.observe(
+            self.tls.peer_certificate.on.certificate_available, self._reload_tls_after_push
+        )
+        self.framework.observe(
+            self.on[TLS_CLIENT_RELATION].relation_broken, self._reload_tls_after_push
+        )
+        self.framework.observe(
+            self.on[TLS_PEER_RELATION].relation_broken, self._reload_tls_after_push
+        )
         self.tls_transfer = TLSTransfer(self, PEER_RELATION)
         self.async_replication = PostgreSQLAsyncReplication(self)
         self.watcher_offer = PostgreSQLWatcherRelation(self)
@@ -498,6 +518,49 @@ def substrate(self) -> Substrates:
         """
         return Substrates.VM
 
+    def _reload_tls_after_push(self, event) -> None:
+        """Reload PostgreSQL after the lib TLS handler stores+pushes certs.
+
+        Also fires on the TLS relations' relation_broken so detaching the TLS
+        operator re-renders Patroni with TLS disabled and reloads.
+
+        Mirror the handler's readiness guard: when the internal CA is absent the
+        handler defers its push (no files on disk), so skip the reload to avoid
+        rendering ssl:on against missing TLS files on an already-running unit.
+
+        A transient config-apply failure (Patroni API unreachable, member not
+        started) defers and retries rather than leaving stale TLS state or failing
+        the hook.
+        """
+        if not self.get_secret(APP_SCOPE, "internal-ca"):
+            return
+        # Don't enable TLS in the config until the lib has written the cert files to
+        # disk (its push can defer while this local render would still succeed, which
+        # would start Patroni ssl:on against missing files).
+        if self.is_tls_enabled and not self.tls_manager.client_tls_files_on_disk():
+            event.defer()
+            return
+        try:
+            if not self.update_config():
+                event.defer()
+        except Exception:
+            logger.exception("TLS reload (update_config) failed; deferring")
+            event.defer()
+
+    def _regenerate_internal_cert(self, *, reload: bool = True) -> None:
+        """Generate the internal peer cert, push it to the workload, and (optionally) reload.
+
+        reload=False is used at cluster bootstrap: the leader renders patroni.yml on
+        leader-elected and each replica renders it in _on_peer_relation_changed just
+        before starting Patroni, so a reload here would be redundant. The internal peer
+        cert does not toggle ssl in the config -- only the operator/client cert does, via
+        is_tls_enabled -- so skipping the reload cannot leave a stale ssl setting.
+        """
+        self.tls_manager.generate_internal_peer_cert()
+        self.tls_manager.push_tls_files()
+        if reload:
+            self.update_config()
+
     def _check_and_update_internal_cert(self) -> None:
         """Check if the internal cert CN matches the unit IP and regenerate if needed."""
         try:
@@ -509,7 +572,7 @@ def _check_and_update_internal_cert(self) -> None:
                     != self.state.unit_ip
                 )
             ):
-                self.tls.generate_internal_peer_cert()
+                self._regenerate_internal_cert()
         except Exception:
             logger.exception("Unable to check or update internal cert")
 
@@ -1548,7 +1611,7 @@ def is_standby_cluster(self) -> bool:
     @property
     def is_tls_enabled(self) -> bool:
         """Return whether TLS is enabled."""
-        return all(self.tls.get_client_tls_files())
+        return all(self.tls_manager.get_client_tls_files())
 
     @property
     def _peer_members_ips(self) -> set[str]:
@@ -1759,7 +1822,7 @@ def _on_leader_elected(self, event: LeaderElectedEvent) -> None:  # noqa: C901
             logger.debug("On leader elected failed to reconfigure cluster.")
 
         if not self.get_secret(APP_SCOPE, "internal-ca"):
-            self.tls.generate_internal_peer_ca()
+            self.tls_manager.generate_internal_peer_ca()
         self.update_config()
 
         # Don't update connection endpoints in the first time this event run for
@@ -1946,7 +2009,7 @@ def _on_start(self, event: StartEvent) -> None:
             event.defer()
             return
         if not self.get_secret(UNIT_SCOPE, "internal-cert"):
-            self.tls.generate_internal_peer_cert()
+            self._regenerate_internal_cert(reload=False)
 
         self.unit_peer_data.update({"ip": self.state.unit_ip})
         self._ensure_storage_layout()
@@ -2497,10 +2560,12 @@ def _update_certificate(self) -> None:
         # Request the certificate only if there is already one. If there isn't,
         # the certificate will be generated in the relation joined event when
         # relating to the TLS Certificates Operator.
-        if all(self.tls.get_client_tls_files()) or all(self.tls.get_peer_tls_files()):
+        if all(self.tls_manager.get_client_tls_files()) or all(
+            self.tls_manager.get_peer_tls_files()
+        ):
             self.tls.refresh_tls_certificates_event.emit()
         if self.get_secret(UNIT_SCOPE, "internal-cert"):
-            self.tls.generate_internal_peer_cert()
+            self._regenerate_internal_cert()
 
     @property
     def is_blocked(self) -> bool:
@@ -2582,37 +2647,6 @@ def _peers(self) -> Relation | None:
         """
         return self.model.get_relation(PEER_RELATION)
 
-    def push_tls_files_to_workload(self) -> bool:
-        """Move TLS files to the PostgreSQL storage path and enable TLS."""
-        key, ca, cert = self.tls.get_client_tls_files()
-        if key is not None:
-            render_file(Substrates.VM, f"{PATRONI_CONF_PATH}/{TLS_KEY_FILE}", key, 0o600)
-        if ca is not None:
-            render_file(Substrates.VM, f"{PATRONI_CONF_PATH}/{TLS_CA_FILE}", ca, 0o600)
-        if cert is not None:
-            render_file(Substrates.VM, f"{PATRONI_CONF_PATH}/{TLS_CERT_FILE}", cert, 0o600)
-
-        key, ca, cert = self.tls.get_peer_tls_files()
-        if key is not None:
-            render_file(Substrates.VM, f"{PATRONI_CONF_PATH}/peer_{TLS_KEY_FILE}", key, 0o600)
-        if ca is not None:
-            render_file(Substrates.VM, f"{PATRONI_CONF_PATH}/peer_{TLS_CA_FILE}", ca, 0o600)
-        if cert is not None:
-            render_file(Substrates.VM, f"{PATRONI_CONF_PATH}/peer_{TLS_CERT_FILE}", cert, 0o600)
-
-        render_file(
-            Substrates.VM,
-            f"{PATRONI_CONF_PATH}/{TLS_CA_BUNDLE_FILE}",
-            self.tls.get_peer_ca_bundle(),
-            0o600,
-        )
-
-        try:
-            return self.update_config()
-        except Exception:
-            logger.exception("TLS files failed to push. Error in config update")
-            return False
-
     def push_ca_file_into_workload(self, secret_name: str) -> bool:
         """Move CA certificates file into the PostgreSQL storage path."""
         certs = self.get_secret(UNIT_SCOPE, secret_name)

src/charm.py

@@ -485,7 +485,7 @@ def update_endpoints(self, event: DatabaseRequestedEvent | None = None) -> None:
         tls = "True" if self.charm.is_tls_enabled else "False"
         ca = None
         if tls == "True":
-            _, ca, _ = self.charm.tls.get_client_tls_files()
+            _, ca, _ = self.charm.tls_manager.get_client_tls_files()
         if not ca:
             ca = ""