Skip to content

Commit 7386243

Browse files
TineoChorsegegunesmayankshah1607
authored
K8SPSMDB-1602: support Workload Identity for GCS backup storage (#2315)
* K8SPSMDB-1602: add GCS workload identity support for backup/restore - Add WorkloadIdentity bool field to BackupStorageGCSSpec - Make CredentialsSecret omitempty (not required when using workload identity) - Skip credential secret loading when WorkloadIdentity is true, allowing PBM to use GKE Application Default Credentials - Add unit test for GCS workload identity storage config - Update CRD schemas for all three resource types * Regenerate CRDs and manifests - Add credentials.workloadIdentity (boolean) to BackupStorageGCSSpec schema - Drop credentialsSecret from required list in all three CRDs (PerconaServerMongoDB, PerconaServerMongoDBBackup, PerconaServerMongoDBRestore) Generated with `make generate manifests`. * use image tag main * revert * revert image tag to main in deploy/cw-bundle.yaml * K8SPSMDB-1602: address PR review comments - Remove explicit workloadIdentity field from API - Follow AWS S3 pattern: empty credentialsSecret triggers ADC fallback (hors feedback) - Remove workloadIdentity from all CRD YAMLs via make generate manifests - Add E2E test: demand-backup-gcs-workload-identity (mayankshah1607 feedback) - Keep PBM-side ADC fallback for when credentials are not provided * K8SPSMDB-1602: set WorkloadIdentity in PBM config for GCS ADC fallback - Set WorkloadIdentity: true in GCS credentials when credentialsSecret is empty so PBM uses ADC instead of erroring - Update unit test to expect WorkloadIdentity: true in the no-credentials GCS config --------- Co-authored-by: tineoc <tineoc@users.noreply.github.com> Co-authored-by: Viacheslav Sarzhan <slava.sarzhan@percona.com> Co-authored-by: Ege Güneş <ege.gunes@percona.com> Co-authored-by: Mayank Shah <mayank.shah@percona.com>
1 parent 58757da commit 7386243

16 files changed

Lines changed: 181 additions & 16 deletions

File tree

‎config/crd/bases/psmdb.percona.com_perconaservermongodbbackups.yaml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,6 @@ spec:
141141
type: object
142142
required:
143143
- bucket
144-
- credentialsSecret
145144
type: object
146145
lastTransition:
147146
format: date-time

‎config/crd/bases/psmdb.percona.com_perconaservermongodbrestores.yaml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,7 +99,6 @@ spec:
9999
type: object
100100
required:
101101
- bucket
102-
- credentialsSecret
103102
type: object
104103
lastTransition:
105104
format: date-time

‎config/crd/bases/psmdb.percona.com_perconaservermongodbs.yaml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -525,7 +525,6 @@ spec:
525525
type: object
526526
required:
527527
- bucket
528-
- credentialsSecret
529528
type: object
530529
main:
531530
type: boolean

‎deploy/bundle.yaml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,6 @@ spec:
145145
type: object
146146
required:
147147
- bucket
148-
- credentialsSecret
149148
type: object
150149
lastTransition:
151150
format: date-time
@@ -1464,7 +1463,6 @@ spec:
14641463
type: object
14651464
required:
14661465
- bucket
1467-
- credentialsSecret
14681466
type: object
14691467
lastTransition:
14701468
format: date-time
@@ -2551,7 +2549,6 @@ spec:
25512549
type: object
25522550
required:
25532551
- bucket
2554-
- credentialsSecret
25552552
type: object
25562553
main:
25572554
type: boolean

‎deploy/crd.yaml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,6 @@ spec:
145145
type: object
146146
required:
147147
- bucket
148-
- credentialsSecret
149148
type: object
150149
lastTransition:
151150
format: date-time
@@ -1464,7 +1463,6 @@ spec:
14641463
type: object
14651464
required:
14661465
- bucket
1467-
- credentialsSecret
14681466
type: object
14691467
lastTransition:
14701468
format: date-time
@@ -2551,7 +2549,6 @@ spec:
25512549
type: object
25522550
required:
25532551
- bucket
2554-
- credentialsSecret
25552552
type: object
25562553
main:
25572554
type: boolean

‎deploy/cw-bundle.yaml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,6 @@ spec:
145145
type: object
146146
required:
147147
- bucket
148-
- credentialsSecret
149148
type: object
150149
lastTransition:
151150
format: date-time
@@ -1464,7 +1463,6 @@ spec:
14641463
type: object
14651464
required:
14661465
- bucket
1467-
- credentialsSecret
14681466
type: object
14691467
lastTransition:
14701468
format: date-time
@@ -2551,7 +2549,6 @@ spec:
25512549
type: object
25522550
required:
25532551
- bucket
2554-
- credentialsSecret
25552552
type: object
25562553
main:
25572554
type: boolean

‎e2e-tests/demand-backup-gcs-workload-identity/conf/backup-gcs-wi.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: psmdb.percona.com/v1
2+
kind: PerconaServerMongoDBBackup
3+
metadata:
4+
finalizers:
5+
- percona.com/delete-backup
6+
name:
7+
spec:
8+
type: physical
9+
clusterName: some-name
10+
storageName:

‎e2e-tests/demand-backup-gcs-workload-identity/conf/restore.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
apiVersion: psmdb.percona.com/v1
2+
kind: PerconaServerMongoDBRestore
3+
metadata:
4+
name:
5+
spec:
6+
clusterName: some-name
7+
backupName:

‎e2e-tests/demand-backup-gcs-workload-identity/conf/some-name.yml‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
apiVersion: psmdb.percona.com/v1
2+
kind: PerconaServerMongoDB
3+
metadata:
4+
finalizers:
5+
- percona.com/delete-psmdb-pvc
6+
name: some-name
7+
spec:
8+
image:
9+
imagePullPolicy: Always
10+
updateStrategy: SmartUpdate
11+
backup:
12+
enabled: true
13+
image: perconalab/percona-server-mongodb-operator:1.1.0-backup
14+
serviceAccountName: percona-server-mongodb-operator
15+
storages:
16+
gcs-wi:
17+
type: gcs
18+
gcs:
19+
bucket: operator-testing
20+
prefix: psmdb-demand-backup-gcs-wi
21+
replsets:
22+
- name: rs0
23+
affinity:
24+
antiAffinityTopologyKey: none
25+
resources:
26+
limits:
27+
cpu: 500m
28+
memory: 1G
29+
requests:
30+
cpu: 100m
31+
memory: 0.1G
32+
volumeSpec:
33+
persistentVolumeClaim:
34+
resources:
35+
requests:
36+
storage: 3Gi
37+
expose:
38+
enabled: false
39+
type: ClusterIP
40+
size: 3
41+
configuration: |
42+
operationProfiling:
43+
mode: slowOp
44+
slowOpThresholdMs: 100
45+
security:
46+
enableEncryption: true
47+
redactClientLogData: false
48+
setParameter:
49+
ttlMonitorSleepSecs: 60
50+
wiredTigerConcurrentReadTransactions: 128
51+
wiredTigerConcurrentWriteTransactions: 128
52+
storage:
53+
engine: wiredTiger
54+
wiredTiger:
55+
collectionConfig:
56+
blockCompressor: snappy
57+
engineConfig:
58+
directoryForIndexes: false
59+
journalCompressor: snappy
60+
indexConfig:
61+
prefixCompression: true
62+
secrets:
63+
users: some-users

‎e2e-tests/demand-backup-gcs-workload-identity/run‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
#!/bin/bash
2+
3+
set -o errexit
4+
5+
test_dir=$(realpath "$(dirname "$0")")
6+
. "${test_dir}/../functions"
7+
set_debug
8+
9+
if [ -z "$GKE" ]; then
10+
desc 'Skip test. Set GKE=1 to run on GKE cluster.'
11+
exit 0
12+
fi
13+
14+
if [ -n "$SKIP_BACKUPS_TO_AWS_GCP_AZURE" ]; then
15+
desc 'Skip tests related to GCP Cloud Storage'
16+
exit 0
17+
fi
18+
19+
if [ -z "$GCP_PROJECT" ] || [ -z "$GCS_WI_SERVICE_ACCOUNT" ]; then
20+
desc 'Skip test. Set GCP_PROJECT and GCS_WI_SERVICE_ACCOUNT for GCS Workload Identity test.'
21+
exit 0
22+
fi
23+
24+
create_infra "${namespace}"
25+
26+
desc 'create GCP workload identity binding'
27+
kubectl_bin annotate serviceaccount \
28+
--namespace "${namespace}" \
29+
"${namespace}-psmdb-db" \
30+
"iam.gke.io/gcp-service-account=${GCS_WI_SERVICE_ACCOUNT}"
31+
32+
desc 'create PSMDB cluster without GCS credentialsSecret'
33+
cluster="some-name"
34+
kubectl_bin apply -f "${test_dir}/conf/${cluster}.yml"
35+
kubectl_bin apply -f "${conf_dir}/client_with_tls.yml"
36+
37+
desc 'check if all pods started'
38+
wait_for_running "${cluster}-rs0" 3
39+
wait_cluster_consistency "${cluster}"
40+
41+
sleep 60
42+
wait_for_pbm_operations "${cluster}"
43+
44+
desc 'write test data'
45+
run_mongo \
46+
'db.createUser({user:"myApp",pwd:"myPass",roles:[{db:"myApp",role:"readWrite"}]})' \
47+
"userAdmin:userAdmin123456@${cluster}-rs0.${namespace}"
48+
sleep 1
49+
run_mongo \
50+
'use myApp\n db.test.insert({ x: 100500 })' \
51+
"myApp:myPass@${cluster}-rs0.${namespace}"
52+
sleep 5
53+
compare_mongo_cmd "find" "myApp:myPass@${cluster}-rs0-0.${cluster}-rs0.${namespace}"
54+
compare_mongo_cmd "find" "myApp:myPass@${cluster}-rs0-1.${cluster}-rs0.${namespace}"
55+
compare_mongo_cmd "find" "myApp:myPass@${cluster}-rs0-2.${cluster}-rs0.${namespace}"
56+
57+
desc 'run GCS backup with workload identity'
58+
backup_name="backup-gcs-wi"
59+
run_backup gcs-wi "${backup_name}" 'physical'
60+
wait_backup "${backup_name}"
61+
check_backup_in_storage "${backup_name}" gcs rs0
62+
63+
desc 'drop and restore'
64+
run_mongo 'use myApp\n db.test.drop()' "myApp:myPass@${cluster}-rs0.${namespace}"
65+
run_restore "${backup_name}"
66+
run_recovery_check "${backup_name}" "${cluster}"
67+
68+
destroy "$namespace"
69+
70+
desc 'test passed'

0 commit comments

Comments
 (0)