Skip to content
This repository was archived by the owner on Mar 28, 2020. It is now read-only.

Commit 86211e9

Browse files
stormgbshexfusion
stormgbs
authored and
hexfusion
committed
*: add Alibaba Cloud Object Storage Service (OSS) backend for etcd-backup/restore-operators (#2065)
1 parent 767c12b commit 86211e9

13 files changed

Lines changed: 650 additions & 1 deletion

File tree

‎Gopkg.lock‎

Lines changed: 9 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Gopkg.toml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,3 +73,7 @@ required = [
7373
[[prune.project]]
7474
name = "k8s.io/code-generator"
7575
non-go = false
76+
77+
[[constraint]]
78+
name = "github.com/aliyun/aliyun-oss-go-sdk"
79+
version = "=1.9.4"

‎doc/design/oss_backup.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Backups using Alibaba Cloud Object Storage Service (OSS)
2+
3+
## Cluster configured with OSS backup
4+
5+
To create a backup in OSS, set `backup.storageType` to `"OSS"`, supply the path (in the format `<oss-bucket-name>/<path-to-backup-object>`) in `oss.path` and provide the Kubernetes secret storing the Alibaba Cloud account credentials to `oss.ossSecret`. The secret must exist prior to backup creation. Etcd backup operator will create the bucket and object if not found. The field `oss.endpoint` is the target OSS service endpoint where the data is backed up. By default, `http://oss-cn-hangzhou.aliyuncs.com` will be used. If you want to back up the data to other regions, please specify another endpoint from [the list of region endpoints](https://www.alibabacloud.com/help/doc-detail/31837.htm).
6+
7+
8+
An example backup manifest would look like:
9+
10+
```yaml
11+
apiVersion: etcd.database.coreos.com/v1beta2
12+
kind: EtcdBackup
13+
metadata:
14+
name: etcd-cluster-with-oss-backup
15+
namespace: my-namespace
16+
spec:
17+
backupPolicy:
18+
...
19+
etcdEndpoints:
20+
- "http://example-etcd-cluster-client:2379"
21+
storageType: OSS
22+
oss:
23+
endpoint: http://oss-cn-hangzhou.aliyuncs.com
24+
ossSecret: my-oss-credentials
25+
path: my-etcd-backups-bucket/etcd.backup
26+
```
27+
28+
### In Detail:
29+
30+
- `"ossSecret"` represents the name of the Kubernetes secret object that stores the credentials needed for Alibaba Cloud authorization, namely an authorization token.
31+
32+
The Kubernetes secret manifest must have the following format:
33+
```yaml
34+
apiVersion: v1
35+
kind: Secret
36+
metadata:
37+
name: my-oss-credentials
38+
type: Opaque
39+
data:
40+
accessKeyID: <my-access-key-id>
41+
accessKeySecret: <my-access-key-secret>
42+
```

‎doc/user/oss_backup.md‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Backups using Alibaba Cloud Object Storage Service (OSS)
2+
3+
Etcd backup operator backs up the data of an etcd cluster running on Kubernetes to a remote storage such as Alibaba Cloud Object Storage Service (OSS). If it is not deployed yet, please follow the [instructions](walkthrough/backup-operator.md#deploy-etcd-backup-operator) to deploy it, e.g. by running
4+
5+
```sh
6+
kubectl apply -f example/etcd-backup-operator/deployment.yaml
7+
```
8+
9+
## Setup Alibaba Cloud backup account, OSS bucket, and Secret
10+
11+
1. Login [Alibaba Cloud Console](https://www.alibabacloud.com) (or [Aliyun Console](https://www.aliyun.com/) if you are in China) and create your own [AccessKey](https://www.alibabacloud.com/help/doc-detail/29009.htm) which gives you the AccessKeyID (AKID) and AccessKeySecret (AKS). You can optionally create an Object Storage Service ([OSS](https://www.alibabacloud.com/help/doc-detail/31947.htm)) bucket for backups.
12+
2. Create a secret storing your AKID and AKS in Kubernetes.
13+
14+
```yaml
15+
apiVersion: v1
16+
kind: Secret
17+
metadata:
18+
name: my-oss-credentials
19+
type: Opaque
20+
data:
21+
accessKeyID: <my-access-key-id>
22+
accessKeySecret: <my-access-key-secret>
23+
```
24+
25+
3. Create an `EtcdBackup` CR file `etcdbackup.yaml` which uses secret `my-oss-credentials` from the previous step.
26+
```yaml
27+
apiVersion: etcd.database.coreos.com/v1beta2
28+
kind: EtcdBackup
29+
metadata:
30+
name: etcd-cluster-with-oss-backup
31+
spec:
32+
backupPolicy:
33+
...
34+
etcdEndpoints:
35+
- "http://example-etcd-cluster-client:2379"
36+
storageType: OSS
37+
oss:
38+
endpoint: http://oss-cn-hangzhou.aliyuncs.com
39+
ossSecret: my-oss-credentials
40+
path: my-etcd-backups-bucket/etcd.backup
41+
```
42+
43+
4. Apply yaml file to kubernetes cluster.
44+
```sh
45+
kubectl apply -f etcdbackup.yaml
46+
```
47+
5. Check the `status` section of the `EtcdBackup` CR.
48+
```console
49+
$ kubectl get EtcdBackup etcd-cluster-with-oss-backup -o yaml
50+
apiVersion: etcd.database.coreos.com/v1beta2
51+
kind: EtcdBackup
52+
...
53+
spec:
54+
oss:
55+
ossSecret: my-oss-credentials
56+
path: my-etcd-backups-bucket/etcd.backup
57+
endpoint: http://oss-cn-hangzhou.aliyuncs.com
58+
etcdEndpoints:
59+
- http://example-etcd-cluster-client:2379
60+
storageType: OSS
61+
status:
62+
etcdRevision: 1
63+
etcdVersion: 3.2.13
64+
succeeded: true
65+
```
66+
67+
6. We should see the backup files from Alibaba Cloud OSS Console.
68+
69+
70+
## Restore etcd based on data from OSS.
71+
72+
Etcd restore operator is in charge of restoring etcd cluster from backup. If it is not deployed, please deploy by following command:
73+
74+
```sh
75+
kubectl apply -f example/etcd-restore-operator/deployment.yaml
76+
```
77+
78+
Now kill all the etcd pods to simulate a cluster failure:
79+
80+
```sh
81+
kubectl delete pod -l app=etcd,etcd_cluster=example-etcd-cluster --force --grace-period=0
82+
```
83+
84+
1. Create an EtcdRestore CR.
85+
```yaml
86+
apiVersion: "etcd.database.coreos.com/v1beta2"
87+
kind: "EtcdRestore"
88+
metadata:
89+
# The restore CR name must be the same as spec.etcdCluster.name
90+
name: example-etcd-cluster
91+
spec:
92+
etcdCluster:
93+
# The namespace is the same as this EtcdRestore CR
94+
name: example-etcd-cluster
95+
backupStorageType: OSS
96+
oss:
97+
# The format of the path must be: "<oss-bucket-name>/<path-to-backup-file>"
98+
path: my-etcd-backups-bucket/etcd.backup
99+
ossSecret: my-oss-credentials
100+
endpoint: http://oss-cn-hangzhou.aliyuncs.com
101+
```
102+
103+
2. Check the `status` section of the `EtcdRestore` CR.
104+
```sh
105+
$ kubectl get etcdrestore example-etcd-cluster -o yaml
106+
apiVersion: etcd.database.coreos.com/v1beta2
107+
kind: EtcdRestore
108+
...
109+
spec:
110+
oss:
111+
ossSecret: my-oss-credentials
112+
path: my-etcd-backups-bucket/etcd.backup
113+
endpoint: http://oss-cn-hangzhou.aliyuncs.com
114+
backupStorageType: OSS
115+
etcdCluster:
116+
name: example-etcd-cluster
117+
status:
118+
succeeded: true
119+
```
120+
121+
3. Verify the `EtcdCluster` CR and restored pods for the restored cluster.
122+
```sh
123+
$ kubectl get etcdcluster
124+
$ kubectl get pods -l app=etcd,etcd_cluster=example-etcd-cluster
125+
```

‎pkg/apis/etcd/v1beta2/backup_types.go‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,11 @@ const (
3434
BackupStorageTypeGCS BackupStorageType = "GCS"
3535
GCPAccessToken = "access-token"
3636
GCPCredentialsJson = "credentials.json"
37+
38+
// Alibaba Cloud OSS related consts
39+
BackupStorageTypeOSS BackupStorageType = "OSS"
40+
AlibabaCloudSecretCredentialsAccessKeyID = "accessKeyID"
41+
AlibabaCloudSecretCredentialsAccessKeySecret = "accessKeySecret"
3742
)
3843

3944
type BackupStorageType string
@@ -90,6 +95,8 @@ type BackupSource struct {
9095
ABS *ABSBackupSource `json:"abs,omitempty"`
9196
// GCS defines the GCS backup source spec.
9297
GCS *GCSBackupSource `json:"gcs,omitempty"`
98+
// OSS defines the OSS backup source spec.
99+
OSS *OSSBackupSource `json:"oss,omitempty"`
93100
}
94101

95102
// BackupPolicy defines backup policy.
@@ -169,3 +176,38 @@ type GCSBackupSource struct {
169176
// If omitted, client will use the default application credentials.
170177
GCPSecret string `json:"gcpSecret,omitempty"`
171178
}
179+
180+
// OSSBackupSource provides the spec how to store backups on OSS.
181+
type OSSBackupSource struct {
182+
// Path is the full abs path where the backup is saved.
183+
// The format of the path must be: "<oss-bucket-name>/<path-to-backup-file>"
184+
// e.g: "mybucket/etcd.backup"
185+
Path string `json:"path"`
186+
187+
// The name of the secret object that stores the credential which will be used
188+
// to access Alibaba Cloud OSS.
189+
//
190+
// The secret must contain the following keys/fields:
191+
// accessKeyID
192+
// accessKeySecret
193+
//
194+
// The format of secret:
195+
//
196+
// apiVersion: v1
197+
// kind: Secret
198+
// metadata:
199+
// name: <my-credential-name>
200+
// type: Opaque
201+
// data:
202+
// accessKeyID: <base64 of my-access-key-id>
203+
// accessKeySecret: <base64 of my-access-key-secret>
204+
//
205+
OSSSecret string `json:"ossSecret"`
206+
207+
// Endpoint is the OSS service endpoint on alibaba cloud, defaults to
208+
// "http://oss-cn-hangzhou.aliyuncs.com".
209+
//
210+
// Details about regions and endpoints, see:
211+
// https://www.alibabacloud.com/help/doc-detail/31837.htm
212+
Endpoint string `json:"endpoint,omitempty"`
213+
}

‎pkg/apis/etcd/v1beta2/restore_types.go‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,9 @@ type RestoreSource struct {
6969

7070
// GCS tells where on GCS the backup is saved and how to fetch the backup.
7171
GCS *GCSRestoreSource `json:"gcs,omitempty"`
72+
73+
// OSS tells where on OSS the backup is saved and how to fetch the backup.
74+
OSS *OSSRestoreSource `json:"oss,omitempty"`
7275
}
7376

7477
type S3RestoreSource struct {
@@ -120,6 +123,40 @@ type GCSRestoreSource struct {
120123
GCPSecret string `json:"gcpSecret,omitempty"`
121124
}
122125

126+
type OSSRestoreSource struct {
127+
// Path is the full abs path where the backup is saved.
128+
// The format of the path must be: "<oss-bucket-name>/<path-to-backup-file>"
129+
// e.g: "myossbucket/etcd.backup"
130+
Path string `json:"path"`
131+
132+
// The name of the secret object that stores the credential which will be used
133+
// to access Alibaba Cloud OSS.
134+
//
135+
// The secret must contain the following keys/fields:
136+
// accessKeyID
137+
// accessKeySecret
138+
//
139+
// The format of secret:
140+
//
141+
// apiVersion: v1
142+
// kind: Secret
143+
// metadata:
144+
// name: <my-credential-name>
145+
// type: Opaque
146+
// data:
147+
// accessKeyID: <base64 of my-access-key-id>
148+
// accessKeySecret: <base64 of my-access-key-secret>
149+
//
150+
OSSSecret string `json:"ossSecret"`
151+
152+
// Endpoint is the OSS service endpoint on alibaba cloud, defaults to
153+
// "http://oss-cn-hangzhou.aliyuncs.com".
154+
//
155+
// Details about regions and endpoints, see:
156+
// https://www.alibabacloud.com/help/doc-detail/31837.htm
157+
Endpoint string `json:"endpoint,omitempty"`
158+
}
159+
123160
// RestoreStatus reports the status of this restore operation.
124161
type RestoreStatus struct {
125162
// Succeeded indicates if the backup has Succeeded.

‎pkg/apis/etcd/v1beta2/zz_generated.deepcopy.go‎

Lines changed: 42 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)