Skip to content

Commit 1665569

Browse files
stuggistuggi
committed
[b/r] Add demo slide content and asciinema recording script
Slide content covering overview, architecture, and automation. Demo script with step-by-step commands and fast-forward notes for a 10-15 minute asciinema recording. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> # Please enter the commit message for your changes. Lines starting # with '#' will be kept; you may remove them yourself if you want to. # An empty message aborts the commit. # # Date: Wed Mar 25 11:13:10 2026 +0100 # # On branch backup_restore # Changes to be committed: # new file: docs/dev/backup-restore/demo/demo-script.md # new file: docs/dev/backup-restore/demo/slides.md # # Changes not staged for commit: # modified: config/operator/deployment/kustomization.yaml # modified: docs/dev/backup-restore/demo/slides.md # # Untracked files: # 1 # 2 # IMPLEMENTATION_SUMMARY.md # TEST1_IMPROVEMENTS.md # TEST2_FIXES_TODO.md # TEST2_WEBHOOK_FIX.md # api/core/v1beta1/SERVICE_NAME_CACHING.md # building-test-catalog-for-upgrades.md # galerabackup-cell1.yaml # galerabackup.yaml # galerarestore-cell1.yaml # galerarestore.yaml # hack/pin-bundle-images.sh.bk # openstack-catalog-clean.Dockerfile # openstack-catalog-clean/ # openstack-catalog.Dockerfile # t.yaml # tt.yaml # webhook-backup.yaml # webhook-backup2.yaml #
1 parent ee259a3 commit 1665569

2 files changed

Lines changed: 234 additions & 0 deletions

File tree

‎docs/dev/backup-restore/demo/demo-script.md‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# RHOSO Backup & Restore - Demo Script
2+
3+
## Prerequisites
4+
- OpenStack control plane deployed and healthy
5+
- asciinema installed (`pip install asciinema`)
6+
7+
## Recording
8+
9+
```bash
10+
asciinema rec demo-backup-restore.cast -t "RHOSO Backup & Restore Demo"
11+
```
12+
13+
## Demo Flow
14+
15+
### 1. Show the environment (~1 min)
16+
17+
```bash
18+
# Show the healthy control plane
19+
oc get openstackcontrolplane -n openstack
20+
21+
# Show the OpenStackBackupConfig (auto-created by the operator)
22+
oc get openstackbackupconfig -n openstack
23+
24+
# Show labeled resources
25+
oc get secret -n openstack -l backup.openstack.org/restore=true --no-headers | wc -l
26+
oc get configmap -n openstack -l backup.openstack.org/restore=true --no-headers | wc -l
27+
28+
# Show labeled PVCs
29+
oc get pvc -n openstack -l backup.openstack.org/backup=true
30+
```
31+
32+
### 2. Install dependencies - MinIO & OADP (~2 min, FAST-FORWARD)
33+
34+
```bash
35+
cd docs/dev/backup-restore/role
36+
37+
# Install MinIO and OADP
38+
ansible-playbook cifmw_backup_restore_test.yaml \
39+
-e backup=false -e cleanup=false -e restore=false
40+
41+
# Show MinIO is running
42+
oc get pods -n minio
43+
oc get routes -n minio
44+
45+
# Show OADP is running
46+
oc get pods -n openshift-adp
47+
oc get backupstoragelocation -n openshift-adp
48+
```
49+
50+
### 3. Create backup (~3 min, FAST-FORWARD PVC backup wait)
51+
52+
```bash
53+
# Run backup
54+
ansible-playbook cifmw_backup_restore_test.yaml \
55+
-e install_deps=false -e cleanup=false -e restore=false
56+
57+
# Show the GaleraBackup CRs were created
58+
oc get galerabackup -n openstack
59+
60+
# Show the OADP backups
61+
oc get backup -n openshift-adp
62+
63+
# Note the backup timestamp for restore
64+
# e.g., 20260325-101234
65+
```
66+
67+
### 4. Simulate disaster - Cleanup (~1 min, FAST-FORWARD)
68+
69+
```bash
70+
# Delete the control plane (simulates disaster)
71+
ansible-playbook cifmw_backup_restore_test.yaml \
72+
-e install_deps=false -e backup=false -e restore=false
73+
74+
# Show everything is gone
75+
oc get openstackcontrolplane -n openstack
76+
oc get pods -n openstack
77+
oc get secret -n openstack | wc -l
78+
```
79+
80+
### 5. Restore (~5 min, FAST-FORWARD waits)
81+
82+
```bash
83+
# Restore from backup
84+
ansible-playbook cifmw_backup_restore_test.yaml \
85+
-e install_deps=false -e backup=false -e cleanup=false \
86+
-e backup_timestamp=<TIMESTAMP>
87+
88+
# --- FAST-FORWARD: PVC restore ---
89+
# --- FAST-FORWARD: Wait for infrastructure (infra-only annotation) ---
90+
# --- FAST-FORWARD: Wait for control plane ready ---
91+
# --- FAST-FORWARD: EDPM deployment ---
92+
```
93+
94+
### 6. Verify restored environment (~1 min)
95+
96+
```bash
97+
# Show control plane is back
98+
oc get openstackcontrolplane -n openstack
99+
100+
# Show pods are running
101+
oc get pods -n openstack | head -20
102+
103+
# Show databases are restored
104+
oc get galera -n openstack
105+
106+
# Show secrets are back
107+
oc get secret osp-secret -n openstack
108+
109+
# Verify an OpenStack endpoint
110+
oc exec -t openstackclient -- openstack endpoint list | head -10
111+
```
112+
113+
## Post-recording
114+
115+
```bash
116+
# Stop recording
117+
exit
118+
119+
# Play back (can use --speed for fast-forward sections)
120+
asciinema play demo-backup-restore.cast
121+
122+
# Upload (optional)
123+
asciinema upload demo-backup-restore.cast
124+
```
125+
126+
## Fast-forward tips
127+
128+
- Use `asciinema play --speed=10` for fast-forward sections during playback
129+
- Alternatively, edit the .cast file to compress wait times:
130+
- The .cast format is JSON lines with timestamps
131+
- Reduce large time gaps between lines to speed up boring sections
132+
- Or use `agg` to convert to GIF with speed adjustments
133+
134+
---

‎docs/dev/backup-restore/demo/slides.md‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# RHOSO Backup & Restore - Slide Content
2+
3+
## Slide 1: Overview - What & Why
4+
5+
**Title: RHOSO Control Plane Backup & Restore**
6+
7+
- Full backup and restore of the OpenStack control plane on OpenShift
8+
- Covers: databases (Galera), Kubernetes resources (CRs, Secrets, ConfigMaps), persistent volumes
9+
- Uses OADP (OpenShift API for Data Protection) / Velero as the backup engine
10+
- CSI volume snapshots for PVC data (Galera dumps, other persistent data)
11+
- Goal: recover from catastrophic control plane failure or migrate between clusters
12+
13+
**Key points:**
14+
- Non-disruptive backup (no downtime during backup)
15+
- Ordered restore (dependencies respected: secrets -> infrastructure -> control plane)
16+
- Data plane nodes are untouched during backup/restore
17+
18+
---
19+
20+
## Slide 2: Architecture - How It Works
21+
22+
**Title: Architecture**
23+
24+
```
25+
+-------------------+ +--------------------+
26+
| OpenStackBackup- | | Service Operators |
27+
| Config Controller | | (glance, mariadb, |
28+
+--------+----------+ | swift, ...) |
29+
| +---------+----------+
30+
Labels user resources |
31+
(secrets, configmaps, Label their PVCs
32+
NADs, issuers) and secrets (CA certs)
33+
| at creation time
34+
+----------+---------------+
35+
|
36+
+-------+-------+ +------------------+
37+
| OADP / Velero +---------+ S3 Storage |
38+
+-------+-------+ | (MinIO/ODF/S3) |
39+
| +------------------+
40+
Creates backups
41+
(2 Backup CRs):
42+
1. PVCs (CSI snapshots)
43+
2. Resources (CRs, etc.)
44+
|
45+
+-------------------+---+ +-------------------------+
46+
| GaleraBackup CRs | | Ordered Restore |
47+
| (mariadb-operator) | | (phases): |
48+
+-----------------------+ | 00: PVCs |
49+
Defines backup config | 10: User Cfg |
50+
(CronJobs); Jobs | 20: Infra CRs |
51+
triggered before OADP | 30: CtlPlane |
52+
| 40: IPSet, GaleraBackup |
53+
| 50: Manual (DB, RMQ) |
54+
| 60: DataPlane |
55+
+-------------------------+
56+
```
57+
58+
**Labeling responsibilities:**
59+
- **Operators** (e.g. glance-operator, mariadb-operator): label their own PVCs and secrets (e.g. CA certs) at creation time
60+
- **BackupConfig controller**: labels user-provided resources (secrets, configmaps, NADs, issuers) that have no ownerReferences
61+
62+
**Backup flow:**
63+
1. GaleraBackup CRs define backup config; ad-hoc Jobs are triggered from the resulting CronJobs
64+
2. OADP Backup #1: CSI snapshots of labeled PVCs (includes fresh DB dumps)
65+
3. OADP Backup #2: All labeled Kubernetes resources
66+
67+
**Restore flow:**
68+
1. Ordered Velero Restores (PVCs -> Secrets -> Infrastructure -> ControlPlane)
69+
2. GaleraRestore CRs restore databases from dumps
70+
3. RabbitMQ credential restore (original credentials from backup)
71+
4. DataPlane restore and EDPM re-deployment
72+
73+
---
74+
75+
## Slide 3: Automation & CI Integration
76+
77+
**Title: Ansible Automation**
78+
79+
- Three independent Ansible roles:
80+
- `deploy_minio` - S3 storage backend (dev/test)
81+
- `openshift_adp` - OADP operator installation
82+
- `cifmw_backup_restore` - Backup/restore/cleanup lifecycle
83+
84+
- Single test playbook runs full end-to-end:
85+
```
86+
ansible-playbook cifmw_backup_restore_test.yaml
87+
```
88+
89+
- Each step independently controllable:
90+
```
91+
-e install_deps=false -e backup=false -e cleanup=false -e restore=true
92+
```
93+
94+
- Dynamic discovery: Galera instances and RabbitMQ clusters read from OpenStackControlPlane CR
95+
96+
- BackupConfig controller integrated into openstack-operator:
97+
- Auto-created with OpenStackControlPlane
98+
- Labels managed automatically, user can override via annotations
99+
100+
---

0 commit comments

Comments
 (0)