Skip to content

Commit e9d26bc

Browse files
markus-hentschjosephineSeianjastrunk
authored
Add standard for volume backup functionality (#567)
Signed-off-by: Markus Hentsch <markus.hentsch@cloudandheat.com> Signed-off-by: Markus Hentsch <129268441+markus-hentsch@users.noreply.github.com> Co-authored-by: josephineSei <128813814+josephineSei@users.noreply.github.com> Co-authored-by: anjastrunk <119566837+anjastrunk@users.noreply.github.com>
1 parent 0b759af commit e9d26bc

3 files changed

Lines changed: 449 additions & 0 deletions

File tree

‎Standards/scs-0117-v1-volume-backup-service.md‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
---
2+
title: Volume Backup Functionality
3+
type: Standard
4+
status: Draft
5+
track: IaaS
6+
---
7+
8+
## Introduction
9+
10+
OpenStack offers a variety of resources where users are able to transfer and store data in the infrastructure.
11+
A prime example of these resources are volumes which are attached to virtual machines as virtual block storage devices.
12+
As such they carry potentially large amounts of user data which is constantly changing at runtime.
13+
It is important for users to have the ability to create backups of this data in a reliable and effifcient manner.
14+
15+
## Terminology
16+
17+
| Term | Meaning |
18+
|---|---|
19+
| CSP | Cloud Service Provider, provider managing the OpenStack infrastructure |
20+
| IaaS | Abbreviation for Infrastructure as a Service |
21+
| Image | IaaS resource representing a snapshot of a block storage disk, can be used to create Volumes |
22+
| Volume | IaaS resource representing a virtual block storage device that can be attached as a disk to virtual machines |
23+
24+
## Motivation
25+
26+
The [volume backup functionality of the Block Storage API](https://docs.openstack.org/cinder/latest/admin/volume-backups.html) is a feature that is not available in all clouds per default, e.g., in OpenStack.
27+
The feature requires a backend to be prepared and configured correctly before it can be used.
28+
In the Block Storage service, the backup storage backend is usually configured separately from the storage backend of the general volume service and may not be mandatory.
29+
Thus, an arbitrary cloud may or may not offer the backup feature in the Block Storage API.
30+
31+
This standard aims to make this functionality the default in SCS clouds so that customers can expect the feature to be usable.
32+
33+
## Design Considerations
34+
35+
The standard should make sure that the feature is available and usable but should not limit the exact implementation (e.g. choice of backend driver) any further than necessary.
36+
37+
### Options considered
38+
39+
#### Only recommend volume backup feature, use images as alternative
40+
41+
As an alternative to the volume backup feature of the Block Storage API, images can also be created based on volumes and act as a backup under certain circumstances.
42+
As an option, this standard could keep the actual integration of the volume backup feature optional and guide users how to use images as backup targets instead in case the feature is unavailable.
43+
44+
However, it is not guaranteed that the image backend storage is separate from the volume storage.
45+
For instance, both could be using the same Ceph cluster.
46+
In such case, the images would not count as genuine backups.
47+
48+
Although users are able to download images and transfer them to a different storage location, this approach might also prove unfeasible depending on the image size and the existence (or lack) of appropriate target storage on the user side.
49+
50+
Furthermore, incremental backups are not possible when creating images from volumes either.
51+
This results in time-consuming backup operations of fully copying a volume everytime a backup is created.
52+
53+
#### Focus on feature availability, make feature mandatory
54+
55+
This option is pretty straightforward.
56+
It would make the volume backup feature mandatory for SCS clouds.
57+
This way users can expect the feature to be available and usable.
58+
59+
With this, users can leverage functionalities like incremental backups and benefit from optimized performance of the backup process due to the tight integration with the volume service.
60+
61+
However, it does not seem feasible to also mandate having a separate storage backend for volume backups at the same time due to potential infrastructure limitations at CSP-side making it hard or even impossible to offer.
62+
As such, the actual benefit of backups in terms of reliability and security aspects would be questionable if a separate storage backend is not mandated and therefore not guaranteed.
63+
64+
This approach would focus on feature availability rather than backup reliability.
65+
66+
#### Focus on backup reliability, make separate backend mandatory
67+
68+
As an alternative, the volume backup feature availability could be made optional but in case a CSP chooses to offer it, the standard would mandate a separate storage backend to be used for volume backups.
69+
This way, failures of the volume storage backend would not directly impact the availability and safety of volume backups, making them actually live up to their name.
70+
71+
In contrast to the above, this approach would focus on backup reliability rather than feature availability.
72+
73+
## Standard
74+
75+
This standard decides to go with the second option and makes the volume backup feature mandatory in the following way:
76+
77+
In an SCS cloud, the volume backup functionality MUST be configured properly and its API as defined per `/v3/{project_id}/backups` MUST be offered to customers.
78+
If using Cinder, a suitable [backup driver](https://docs.openstack.org/cinder/latest/configuration/block-storage/backup-drivers.html) MUST be set up.
79+
80+
The volume backup target storage SHOULD be a separate storage system from the one used for volumes themselves.
81+
82+
## Related Documents
83+
84+
- [OpenStack Block Storage v3 Backup API reference](https://docs.openstack.org/api-ref/block-storage/v3/index.html#backups-backups)
85+
- [OpenStack Volume Backup Drivers](https://docs.openstack.org/cinder/latest/configuration/block-storage/backup-drivers.html)
86+
87+
## Conformance Tests
88+
89+
Conformance tests include using the `/v3/{project_id}/backups` Block Storage API endpoint to create a volume and a backup of it as a non-admin user and subsequently restore the backup on a new volume while verifying the success of each operation.
90+
These tests verify the mandatory part of the standard: providing the Volume Backup API.
91+
92+
There is a test suite in [`volume-backup-tester.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/volume-backup/volume-backup-tester.py).
93+
The test suite connects to the OpenStack API and executes basic operations using the volume backup API to verify that the functionality requested by the standard is available.
94+
Please consult the associated [README.md](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/volume-backup/README.md) for detailed setup and testing instructions.
95+
96+
Note that these tests don't verify the optional part of the standard: providing a separate storage backend for Cinder volume backups.
97+
This cannot be checked from outside of the infrastructure as it is an architectural property of the infrastructure itself and transparent to customers.

‎Tests/iaas/volume-backup/README.md‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Volume Backup API Test Suite
2+
3+
## Test Environment Setup
4+
5+
### Test Execution Environment
6+
7+
> **NOTE:** The test execution procedure does not require cloud admin rights.
8+
9+
To execute the test suite a valid cloud configuration for the OpenStack SDK in the shape of "`clouds.yaml`" is mandatory[^1].
10+
**The file is expected to be located in the current working directory where the test script is executed unless configured otherwise.**
11+
12+
[^1]: [OpenStack Documentation: Configuring OpenStack SDK Applications](https://docs.openstack.org/openstacksdk/latest/user/config/configuration.html)
13+
14+
The test execution environment can be located on any system outside of the cloud infrastructure that has OpenStack API access.
15+
Make sure that the API access is configured properly in "`clouds.yaml`".
16+
17+
It is recommended to use a Python virtual environment[^2].
18+
Next, install the OpenStack SDK required by the test suite:
19+
20+
```bash
21+
pip3 install openstacksdk
22+
```
23+
24+
Within this environment execute the test suite.
25+
26+
[^2]: [Python 3 Documentation: Virtual Environments and Packages](https://docs.python.org/3/tutorial/venv.html)
27+
28+
## Test Execution
29+
30+
The test suite is executed as follows:
31+
32+
```bash
33+
python3 volume-backup-tester.py --os-cloud mycloud
34+
```
35+
36+
As an alternative to "`--os-cloud`", the "`OS_CLOUD`" environment variable may be specified instead.
37+
The parameter is used to look up the correct cloud configuration in "`clouds.yaml`".
38+
For the example command above, this file should contain a `clouds.mycloud` section like this:
39+
40+
```yaml
41+
---
42+
clouds:
43+
mycloud:
44+
auth:
45+
auth_url: ...
46+
...
47+
...
48+
```
49+
50+
If the test suite fails and leaves test resources behind, the "`--cleanup-only`" flag may be used to delete those resources from the domains:
51+
52+
```bash
53+
python3 volume-backup-tester.py --os-cloud mycloud --cleanup-only
54+
```
55+
56+
For any further options consult the output of "`python3 volume-backup-tester.py --help`".
57+
58+
### Script Behavior & Test Results
59+
60+
> **NOTE:** Before any execution of test batches, the script will automatically perform a cleanup of volumes and volume backups matching a special prefix (see the "`--prefix`" flag).
61+
> This cleanup behavior is identical to "`--cleanup-only`".
62+
63+
The script will print all cleanup actions and passed tests to `stdout`.
64+
65+
If all tests pass, the script will return with an exit code of `0`.
66+
67+
If any test fails, the script will halt, print the exact error to `stderr` and return with a non-zero exit code.
68+
69+
In case of a failed test, cleanup is not performed automatically, allowing for manual inspection of the cloud state for debugging purposes.
70+
Although unnecessary due to automatic cleanup upon next execution, you can manually trigger a cleanup using the "`--cleanup-only`" flag of this script.

0 commit comments

Comments
 (0)