Logical backup tries to connect to local unix socket instead of pg host

[vitobotta]

Hi, I am trying to test logical backups, but the jobs fail because the backup tries to connect to a local unix socket that doesn't exist. I would expect it to connect to the pg host via host and port, right? The PGHOST variable was missing in the cronjob so I tried adding it manually but it doesn't work. Any idea of how to get logical backups to work? Thanks

[vitobotta]

I think I found the problem from looking at the script at https://github.com/zalando/postgres-operator/blob/master/docker/logical-backup/dump.sh

For some reason it tries to find the pods with labelSelector=version, because CLUSTER_NAME_LABEL is somehow set to version, but the CLUSTER_NAME_LABEL for the cluster's pods is cluster-name by default. So The backup script cannot find a pod to use for the dump and therefore tries to connect to the unix socket because PGHOST is set to an empty string. At least I think that's what happens.

Where should I set CLUSTER_NAME_LABEL so that the dump script uses cluster-name instead of version in the selector? I could try to set it to version in the operator and cluster pod configmap instead, but I would like to understand how to properly set it for the cron job. Thanks

[vitobotta]

It looks like when installing the operator in configmap mode the cluster name label is set to version by default

postgres-operator/manifests/configmap.yaml

Line 13 in 107334f

cluster_name_label: version

I am using the CRD installation mode and have set cluster_name_label to cluster-name correctly, so it seems that setting is being ignored.

[vitobotta]

I can confirm that if I set it to version in the operator's config the dump script can find the pod and the logical backups work. So there is a bug somewhere and the setting is used to label the cluster's pods, but is ignored by the logical backups somehow.

[FxKu]

This is already fixed with #725. The OS-version of the logical backup container, can already deal with arbitrary cluster name labels. It simply hasn't been merged in our internal repo which is behind the registry.opensource.zalan.do/acid/logical-backup image that is referenced in the configuration. We should also get rid of the version label. Have done it in #696, but might create a separate PR for this.

[vitobotta]

Hi @FxKu so for now the workaround of setting the cluster name label to version is OK?

[FxKu]

yes!. In the meantime, we have merge the open PRs in our internal repo. So hopefully, nobody will run into this again.

[vitobotta]

Cool, thanks! :)

[vitobotta]

Hi @FxKu , I have deleted everything and started from scratch to ensure my notes are valid, but apparently something is different now... I set the cluster_name_label in the operator config to version, but when I create the cluster I see this in the logs:

could not create cluster: could not create statefulset: the server could not find the requested resource" cluster-name=postgres-cluster-dev/postgres-cluster

So it looks like somewhere it's still using cluster-name as the label. I can't find any other settings about this and I don't remember I have set it elsewhere. The only difference from yesterday is that I have deleted the git repo and have re-cloned it now. But from the git log it doesn't seem like anything has changed. Any idea of what I may be missing? Thanks!

[vitobotta]

Found the difference: I had to use the latest tag for the operator. Which tag should I be using?