Skip to content

Commit b58045d

Browse files
mayankshah1607gkech
mayankshah1607
and
gkech
authored
K8SPSMDB-1363: Snapshot-based backups (#2247)
* use pbm dev latest Signed-off-by: Mayank Shah <mayank.shah@percona.com> * use storage.MaskedString Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update backup apis for snapshots Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update go mods Signed-off-by: Mayank Shah <mayank.shah@percona.com> * refactor Signed-off-by: Mayank Shah <mayank.shah@percona.com> * implement backup snapshots executor Signed-off-by: Mayank Shah <mayank.shah@percona.com> * add FinishBackup Signed-off-by: Mayank Shah <mayank.shah@percona.com> * generate manifests Signed-off-by: Mayank Shah <mayank.shah@percona.com> * implement snapshot backups Signed-off-by: Mayank Shah <mayank.shah@percona.com> * restores wip Signed-off-by: Mayank Shah <mayank.shah@percona.com> * track replsetname in status Signed-off-by: Mayank Shah <mayank.shah@percona.com> * wip: implement restore logic Signed-off-by: Mayank Shah <mayank.shah@percona.com> * wip: snapshot restore improvements Signed-off-by: Mayank Shah <mayank.shah@percona.com> * improve conditions Signed-off-by: Mayank Shah <mayank.shah@percona.com> * fix status assignment Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update description comments Signed-off-by: Mayank Shah <mayank.shah@percona.com> * fix len checks Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update CheckFields Signed-off-by: Mayank Shah <mayank.shah@percona.com> * remove redundant code Signed-off-by: Mayank Shah <mayank.shah@percona.com> * imports Signed-off-by: Mayank Shah <mayank.shah@percona.com> * address copilot comments Signed-off-by: Mayank Shah <mayank.shah@percona.com> * revert CR Signed-off-by: Mayank Shah <mayank.shah@percona.com> * fix loop Signed-off-by: Mayank Shah <mayank.shah@percona.com> * typo Signed-off-by: Mayank Shah <mayank.shah@percona.com> * add unit tests for snapshot backups Signed-off-by: Mayank Shah <mayank.shah@percona.com> * add unit tests for snapshot restore Signed-off-by: Mayank Shah <mayank.shah@percona.com> * db config * combine conditions Signed-off-by: Mayank Shah <mayank.shah@percona.com> * improve compatibility Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update e2e test comparisons Signed-off-by: Mayank Shah <mayank.shah@percona.com> * address copilot Signed-off-by: Mayank Shah <mayank.shah@percona.com> * improve condition naming Signed-off-by: Mayank Shah <mayank.shah@percona.com> * add e2e test * support external backup type in scheduled backup tasks Add `external` as a valid backup type for scheduled tasks along with a `VolumeSnapshotClass` field to configure which VolumeSnapshotClass to use. External backups skip storage validation and clear finalizers since snapshots are managed externally. Signed-off-by: Mayank Shah <mayank.shah@percona.com> * fix restore from backup source Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update examples Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update e2e test for restore to new cluster Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update run-pr and run-release Signed-off-by: Mayank Shah <mayank.shah@percona.com> * test fixes Signed-off-by: Mayank Shah <mayank.shah@percona.com> * test fixes Signed-off-by: Mayank Shah <mayank.shah@percona.com> * bugfix: restored pvcs cannot be cleaned up Signed-off-by: Mayank Shah <mayank.shah@percona.com> * test fix Signed-off-by: Mayank Shah <mayank.shah@percona.com> * revert backup images Signed-off-by: Mayank Shah <mayank.shah@percona.com> * use pbm v2.13.0 Signed-off-by: Mayank Shah <mayank.shah@percona.com> * use only dbConfigEntry instead of a map with the rs.name * revert init container script Signed-off-by: Mayank Shah <mayank.shah@percona.com> * fix db-config Signed-off-by: Mayank Shah <mayank.shah@percona.com> * add unit test * fix key permission errors Signed-off-by: Mayank Shah <mayank.shah@percona.com> * fixes * update outdated checks in e2e tests Signed-off-by: Mayank Shah <mayank.shah@percona.com> * delete secrets when Snapshot restore finished * fix imports * fix imports Signed-off-by: Mayank Shah <mayank.shah@percona.com> * cleanup log Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update run-backup.csv Signed-off-by: Mayank Shah <mayank.shah@percona.com> * validate snapshot existence before restore Signed-off-by: Mayank Shah <mayank.shah@percona.com> * ensure PVCs are replaced one at a time Signed-off-by: Mayank Shah <mayank.shah@percona.com> * backup storage deletion Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update rbac Signed-off-by: Mayank Shah <mayank.shah@percona.com> * fix validation check Signed-off-by: Mayank Shah <mayank.shah@percona.com> * address copilot Signed-off-by: Mayank Shah <mayank.shah@percona.com> * logging Signed-off-by: Mayank Shah <mayank.shah@percona.com> * cleanup Signed-off-by: Mayank Shah <mayank.shah@percona.com> * update rbac Signed-off-by: Mayank Shah <mayank.shah@percona.com> * delete volume snapshots when PBM is nil during backup finalizer cleanup Signed-off-by: Mayank Shah <mayank.shah@percona.com> --------- Signed-off-by: Mayank Shah <mayank.shah@percona.com> Co-authored-by: George Kechagias <george.kechagias@percona.com>
1 parent 7489d6b commit b58045d

54 files changed

Lines changed: 4339 additions & 348 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎cmd/manager/main.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import (
1313

1414
certmgrscheme "github.com/cert-manager/cert-manager/pkg/client/clientset/versioned/scheme"
1515
"github.com/go-logr/logr"
16+
volumesnapshotv1 "github.com/kubernetes-csi/external-snapshotter/client/v8/apis/volumesnapshot/v1"
1617
uzap "go.uber.org/zap"
1718
"go.uber.org/zap/zapcore"
1819
k8sruntime "k8s.io/apimachinery/pkg/runtime"
@@ -44,6 +45,7 @@ var (
4445
func init() {
4546
utilruntime.Must(clientgoscheme.AddToScheme(scheme))
4647
utilruntime.Must(apis.AddToScheme(scheme))
48+
utilruntime.Must(volumesnapshotv1.AddToScheme(scheme))
4749
}
4850

4951
func main() {

‎config/crd/bases/psmdb.percona.com_perconaservermongodbbackups.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,9 +78,12 @@ spec:
7878
enum:
7979
- logical
8080
- physical
81+
- external
8182
- incremental
8283
- incremental-base
8384
type: string
85+
volumeSnapshotClass:
86+
type: string
8487
type: object
8588
status:
8689
properties:
@@ -255,6 +258,15 @@ spec:
255258
type: object
256259
size:
257260
type: string
261+
snapshots:
262+
items:
263+
properties:
264+
replsetName:
265+
type: string
266+
snapshotName:
267+
type: string
268+
type: object
269+
type: array
258270
start:
259271
format: date-time
260272
type: string

‎config/crd/bases/psmdb.percona.com_perconaservermongodbrestores.yaml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -216,6 +216,15 @@ spec:
216216
type: object
217217
size:
218218
type: string
219+
snapshots:
220+
items:
221+
properties:
222+
replsetName:
223+
type: string
224+
snapshotName:
225+
type: string
226+
type: object
227+
type: array
219228
start:
220229
format: date-time
221230
type: string
@@ -263,6 +272,42 @@ spec:
263272
completed:
264273
format: date-time
265274
type: string
275+
conditions:
276+
items:
277+
properties:
278+
lastTransitionTime:
279+
format: date-time
280+
type: string
281+
message:
282+
maxLength: 32768
283+
type: string
284+
observedGeneration:
285+
format: int64
286+
minimum: 0
287+
type: integer
288+
reason:
289+
maxLength: 1024
290+
minLength: 1
291+
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
292+
type: string
293+
status:
294+
enum:
295+
- "True"
296+
- "False"
297+
- Unknown
298+
type: string
299+
type:
300+
maxLength: 316
301+
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
302+
type: string
303+
required:
304+
- lastTransitionTime
305+
- message
306+
- reason
307+
- status
308+
- type
309+
type: object
310+
type: array
266311
error:
267312
type: string
268313
lastTransition:

‎config/crd/bases/psmdb.percona.com_perconaservermongodbs.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -504,6 +504,9 @@ spec:
504504
- physical
505505
- incremental
506506
- incremental-base
507+
- external
508+
type: string
509+
volumeSnapshotClass:
507510
type: string
508511
required:
509512
- enabled

‎deploy/backup/backup.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,8 @@ metadata:
77
spec:
88
clusterName: my-cluster-name
99
storageName: s3-us-west
10+
# volumeSnapshotClass: YOUR-VOLUME-SNAPSHOT-CLASS
1011
# type: physical
1112
# compressionType: gzip
1213
# compressionLevel: 6
13-
# startingDeadlineSeconds: 300
14+
# startingDeadlineSeconds: 300

‎deploy/backup/restore.yaml‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,12 @@ spec:
2929
# bucket: S3-BACKUP-BUCKET-NAME-HERE
3030
# endpointUrl: https://s3.us-west-2.amazonaws.com/
3131
# prefix: ""
32+
# snapshots:
33+
# - replsetName: rs0
34+
# snapshotName: YOUR-RS0-SNAPSHOT-NAME
35+
# - replsetName: cfg
36+
# snapshotName: YOUR-CFG-SNAPSHOT-NAME
3237
# azure:
3338
# credentialsSecret: SECRET-NAME
3439
# prefix: PREFIX-NAME
35-
# container: CONTAINER-NAME
40+
# container: CONTAINER-NAME

‎deploy/bundle.yaml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,9 +82,12 @@ spec:
8282
enum:
8383
- logical
8484
- physical
85+
- external
8586
- incremental
8687
- incremental-base
8788
type: string
89+
volumeSnapshotClass:
90+
type: string
8891
type: object
8992
status:
9093
properties:
@@ -259,6 +262,15 @@ spec:
259262
type: object
260263
size:
261264
type: string
265+
snapshots:
266+
items:
267+
properties:
268+
replsetName:
269+
type: string
270+
snapshotName:
271+
type: string
272+
type: object
273+
type: array
262274
start:
263275
format: date-time
264276
type: string
@@ -497,6 +509,15 @@ spec:
497509
type: object
498510
size:
499511
type: string
512+
snapshots:
513+
items:
514+
properties:
515+
replsetName:
516+
type: string
517+
snapshotName:
518+
type: string
519+
type: object
520+
type: array
500521
start:
501522
format: date-time
502523
type: string
@@ -544,6 +565,42 @@ spec:
544565
completed:
545566
format: date-time
546567
type: string
568+
conditions:
569+
items:
570+
properties:
571+
lastTransitionTime:
572+
format: date-time
573+
type: string
574+
message:
575+
maxLength: 32768
576+
type: string
577+
observedGeneration:
578+
format: int64
579+
minimum: 0
580+
type: integer
581+
reason:
582+
maxLength: 1024
583+
minLength: 1
584+
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
585+
type: string
586+
status:
587+
enum:
588+
- "True"
589+
- "False"
590+
- Unknown
591+
type: string
592+
type:
593+
maxLength: 316
594+
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
595+
type: string
596+
required:
597+
- lastTransitionTime
598+
- message
599+
- reason
600+
- status
601+
- type
602+
type: object
603+
type: array
547604
error:
548605
type: string
549606
lastTransition:
@@ -1391,6 +1448,9 @@ spec:
13911448
- physical
13921449
- incremental
13931450
- incremental-base
1451+
- external
1452+
type: string
1453+
volumeSnapshotClass:
13941454
type: string
13951455
required:
13961456
- enabled
@@ -26661,6 +26721,16 @@ rules:
2666126721
- patch
2666226722
- delete
2666326723
- deletecollection
26724+
- apiGroups:
26725+
- snapshot.storage.k8s.io
26726+
resources:
26727+
- volumesnapshots
26728+
verbs:
26729+
- get
26730+
- list
26731+
- watch
26732+
- create
26733+
- delete
2666426734
---
2666526735
apiVersion: v1
2666626736
kind: ServiceAccount

‎deploy/cr.yaml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -914,6 +914,15 @@ spec:
914914
# storageName: s3-us-west
915915
# compressionType: gzip
916916
# compressionLevel: 6
917+
# - name: daily-snapshot
918+
# enabled: false
919+
# schedule: "0 0 * * *"
920+
# retention:
921+
# count: 1
922+
# type: count
923+
# deleteFromStorage: true
924+
# type: external
925+
# volumeSnapshotClass: YOUR-VOLUME-SNAPSHOT-CLASS
917926
logcollector:
918927
enabled: true
919928
image: perconalab/fluentbit:main-logcollector

‎deploy/crd.yaml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,9 +82,12 @@ spec:
8282
enum:
8383
- logical
8484
- physical
85+
- external
8586
- incremental
8687
- incremental-base
8788
type: string
89+
volumeSnapshotClass:
90+
type: string
8891
type: object
8992
status:
9093
properties:
@@ -259,6 +262,15 @@ spec:
259262
type: object
260263
size:
261264
type: string
265+
snapshots:
266+
items:
267+
properties:
268+
replsetName:
269+
type: string
270+
snapshotName:
271+
type: string
272+
type: object
273+
type: array
262274
start:
263275
format: date-time
264276
type: string
@@ -497,6 +509,15 @@ spec:
497509
type: object
498510
size:
499511
type: string
512+
snapshots:
513+
items:
514+
properties:
515+
replsetName:
516+
type: string
517+
snapshotName:
518+
type: string
519+
type: object
520+
type: array
500521
start:
501522
format: date-time
502523
type: string
@@ -544,6 +565,42 @@ spec:
544565
completed:
545566
format: date-time
546567
type: string
568+
conditions:
569+
items:
570+
properties:
571+
lastTransitionTime:
572+
format: date-time
573+
type: string
574+
message:
575+
maxLength: 32768
576+
type: string
577+
observedGeneration:
578+
format: int64
579+
minimum: 0
580+
type: integer
581+
reason:
582+
maxLength: 1024
583+
minLength: 1
584+
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
585+
type: string
586+
status:
587+
enum:
588+
- "True"
589+
- "False"
590+
- Unknown
591+
type: string
592+
type:
593+
maxLength: 316
594+
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
595+
type: string
596+
required:
597+
- lastTransitionTime
598+
- message
599+
- reason
600+
- status
601+
- type
602+
type: object
603+
type: array
547604
error:
548605
type: string
549606
lastTransition:
@@ -1391,6 +1448,9 @@ spec:
13911448
- physical
13921449
- incremental
13931450
- incremental-base
1451+
- external
1452+
type: string
1453+
volumeSnapshotClass:
13941454
type: string
13951455
required:
13961456
- enabled

0 commit comments

Comments
 (0)