Skip to content

kubernetes-cnpg.md

Latest commit

 

History

History
287 lines (228 loc) · 8.18 KB

kubernetes-cnpg.md

File metadata and controls

287 lines (228 loc) · 8.18 KB
Raw
title Kubernetes with CloudNativePG
description Deploy the agent as a per-Pod sidecar against a CloudNativePG cluster, wired up via the CNPG-I provider.
tags
kubernetes
cnpg
sidecar

Kubernetes with CloudNativePG

Stands up pg_hardstorage against a CloudNativePG cluster on a kind node. The agent runs as a sidecar in each PG Pod, talks to the local Unix socket, and ships chunks + WAL to a MinIO repo running in-cluster. About 30 minutes from a clean kind node.

!!! warning "Roadmap — not yet shipped" The CNPG-I provider this tutorial uses lands with v0.5. The kubectl apply of the provider manifest shown below does not work against a released build yet. Until then, the verified path is the sidecar Helm chart plus a CronJob. This page documents the target shape.

CloudNativePG (CNPG) ships its own backup story; this tutorial wires the cluster to pg_hardstorage via the CNPG-I provider so the operator manages backups through the familiar Cluster.spec.backup surface. The agent runs alongside PG in the same Pod, talks over the local Unix socket (no network hop), and writes to whatever S3-API storage you point at.

What you need

  • kind 0.22+ or any reachable Kubernetes 1.28+ cluster.
  • kubectl, helm 3.13+, and mc (the MinIO client) on $PATH.
  • 4 GB free RAM for the kind node.

The pg-hardstorage-sidecar Helm chart and the CNPG-I provider manifests live in charts/; this tutorial pins them to the published v0.2.x stream.

Steps

1. Spin up the cluster

kind create cluster --name hs-cnpg

Install the CloudNativePG operator:

helm repo add cnpg https://cloudnative-pg.github.io/charts
helm repo update
helm install cnpg cnpg/cloudnative-pg \
    --namespace cnpg-system --create-namespace \
    --version 0.22.x

2. Stand up an in-cluster MinIO as the repo

helm repo add minio https://charts.min.io/
helm install hs-minio minio/minio \
    --namespace hs-storage --create-namespace \
    --set rootUser=pg_hardstorage,rootPassword=hs-tutorial-secret \
    --set persistence.enabled=false

Create the bucket:

kubectl -n hs-storage port-forward svc/hs-minio 9000:9000 &
mc alias set local http://127.0.0.1:9000 pg_hardstorage hs-tutorial-secret
mc mb local/hs-cnpg-repo

3. Install the pg_hardstorage CNPG-I provider

Provider manifests register a CRD that the CNPG operator reconciles into per-cluster sidecars. Install with:

kubectl apply -f https://raw.githubusercontent.com/cybertec-postgresql/pg_hardstorage/v0.2.x/deploy/cnpg-i/provider.yaml

This installs the provider Deployment in the cnpg-system namespace and registers the pghardstorage.org/v1.HSDeployment CRD.

4. Create credentials and the repo URL secret

kubectl create namespace pg-tutorial

kubectl -n pg-tutorial create secret generic hs-repo-creds \
    --from-literal=AWS_ACCESS_KEY_ID=pg_hardstorage \
    --from-literal=AWS_SECRET_ACCESS_KEY=hs-tutorial-secret

kubectl -n pg-tutorial create secret generic hs-repo-url \
    --from-literal=url='s3://hs-cnpg-repo/?endpoint=http://hs-minio.hs-storage:9000&region=us-east-1&use_path_style=true'

A kek.bin for envelope encryption is generated by the provider on first-run and stored as a Secret named hs-<cluster>-kek in the cluster's namespace. Treat it like any other Kubernetes secret — back it up to your KMS of record.

5. Apply a CNPG Cluster with a pg_hardstorage backup spec

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: pg-tutorial
  namespace: pg-tutorial
spec:
  instances: 3
  storage:
    size: 2Gi
  bootstrap:
    initdb:
      database: app
      owner: app
  plugins:
    - name: pghardstorage.org
      enabled: true
      parameters:
        repo_secret: hs-repo-url
        repo_credentials_secret: hs-repo-creds
        deployment: pg-tutorial
        # Inherits sensible defaults: encryption=on, schedule=every 6h,
        # retention=7d/4w/12m, WAL streaming=enabled.
kubectl apply -f cluster.yaml

The CNPG operator picks up the pghardstorage.org plugin entry and asks the provider to inject a sidecar container into every PG Pod. The sidecar runs pg_hardstorage agent against the local Unix socket; backups, WAL streaming, and retention all live inside the cluster.

6. Watch the sidecars come up

kubectl -n pg-tutorial get pods
NAME                       READY   STATUS    RESTARTS   AGE
pg-tutorial-1              2/2     Running   0          2m
pg-tutorial-2              2/2     Running   0          1m30s
pg-tutorial-3              2/2     Running   0          1m

2/2 is the PG container plus the pg_hardstorage sidecar. Inspect the sidecar's view of the cluster:

kubectl -n pg-tutorial exec -c pg-hardstorage pg-tutorial-1 -- \
    pg_hardstorage doctor pg-tutorial
pg-tutorial — PG 17.x — primary @ /controller/run/.s.PGSQL.5432
  ✓ PostgreSQL reachable (Unix socket)
  ✓ Replication slot 'pg_hardstorage_pg_tutorial' active
  ✓ Repository s3://hs-cnpg-repo/ writable
  ✓ KMS keyring present (mounted at /etc/pg_hardstorage/keyring)

7. Take a backup the cluster-native way

CNPG's Backup resource triggers the provider:

apiVersion: postgresql.cnpg.io/v1
kind: Backup
metadata:
  name: pg-tutorial-1
  namespace: pg-tutorial
spec:
  cluster:
    name: pg-tutorial
  method: plugin
  pluginConfiguration:
    name: pghardstorage.org
kubectl apply -f backup.yaml
kubectl -n pg-tutorial get backups -w
NAME              AGE   CLUSTER       METHOD   PHASE       ERROR
pg-tutorial-1     30s   pg-tutorial   plugin   completed

Or run a backup imperatively from inside the sidecar:

kubectl -n pg-tutorial exec -c pg-hardstorage pg-tutorial-1 -- \
    pg_hardstorage backup pg-tutorial

Either path produces the same artefact in the repo — they share the data plane.

8. Restore into a new CNPG cluster

CNPG's Cluster resource has a bootstrap.recovery block; the provider hooks the same restore CLI under the cover:

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: pg-tutorial-restored
  namespace: pg-tutorial
spec:
  instances: 1
  storage:
    size: 2Gi
  bootstrap:
    recovery:
      source: pg-tutorial
      recoveryTarget:
        targetTime: "5 minutes ago"
  externalClusters:
    - name: pg-tutorial
      plugin:
        name: pghardstorage.org
        parameters:
          deployment: pg-tutorial
          repo_secret: hs-repo-url
          repo_credentials_secret: hs-repo-creds

The same natural-language --to semantics from the PITR tutorial apply.

9. Tear down

kind delete cluster --name hs-cnpg

What just happened

You ran pg_hardstorage as a per-Pod sidecar inside a CloudNativePG cluster, wired up through the CNPG-I provider so the cluster's existing Cluster / Backup / Restore CRDs drive the agent. The sidecar talks to PG over the local Unix socket (no network hop, no auth surface), pushes chunks and WAL to MinIO, and surfaces health through pg_hardstorage doctor like any other deployment.

The Kubernetes-only differences from the Patroni tutorial:

  • Coordination uses Kubernetes Leases, not PG advisory locks. The agent resolves "am I the leader for this deployment" by asking the control-plane API or coordination.k8s.io/Lease.
  • The KEK is a Secret, not a file on disk. kek.bin is mounted read-only into the sidecar; rotation goes through kms rotate plus a Secret update.
  • Operator-driven schedule — backup cadence is the Cluster.spec.backup.method=plugin resource, not the agent's internal schedule engine.

Next steps