Skip to content

Commit 8d0dc79

Browse files
ThirunarayananThirunarayanan
committed
MDEV-39061 mariadb-backup compatible wrappers for BACKUP SERVER
scripts/mariabackup/mariabackup.sh: a drop-in wrapper that lets existing mariabackup --backup invocations drive the server-side BACKUP SERVER command without changing user scripts. mariabackup.sh translates --backup into "BACKUP SERVER TO '<dir>'" via the mariadb client, forwarding connection options, and layers --stream/--compress as tar/gzip pipelines on the result. It validates the target directory up front and rejects the - mbstream.sh shims the mbstream CLI onto tar, dropping mbstream-only flags (-p/--parallel) so legacy pipelines keep working. - README.md maps every supported --backup option to its BACKUP SERVER equivalent and documents current limitations. Add include/have_mariabackup_wrapper.inc redirects $XTRABACKUP to the wrapper so a test opts in by sourcing one file; skips when the wrapper, bash, or the mariadb client is unavailable. wrapper_basic.test : Exercises full backup, streaming, compression, the ignored legacy options
1 parent c77278b commit 8d0dc79

6 files changed

Lines changed: 779 additions & 0 deletions

File tree

‎mysql-test/include/have_mariabackup_wrapper.inc‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# ==== Purpose ====
2+
#
3+
# Redirect `$XTRABACKUP` so existing test invocations like
4+
#
5+
# --exec $XTRABACKUP --defaults-file=$MYSQLTEST_VARDIR/my.cnf \
6+
# --backup --target-dir=$targetdir
7+
#
8+
# run through scripts/mariabackup/mariabackup.sh — the BACKUP SERVER
9+
# compatibility wrapper — without any change to the test body.
10+
#
11+
#
12+
# Skip the test if any of these are missing:
13+
# - the wrapper script
14+
# - bash
15+
# - the mariadb client (wrapper shells out to it)
16+
#
17+
# ==== Usage ====
18+
#
19+
# --source include/have_mariabackup_wrapper.inc
20+
# # ... rest of the test, using $XTRABACKUP as usual ...
21+
#
22+
# ==== Exposed variables ====
23+
#
24+
# $XTRABACKUP — now points at mariabackup.sh
25+
26+
--source include/linux.inc
27+
28+
--let MARIABACKUP_WRAPPER=$MYSQL_TEST_DIR/../scripts/mariabackup/mariabackup.sh
29+
30+
--error 0,1
31+
perl;
32+
use strict;
33+
use warnings;
34+
use File::Basename;
35+
36+
my $wrapper = $ENV{MARIABACKUP_WRAPPER};
37+
exit 1 unless $wrapper && -x $wrapper;
38+
39+
chomp(my $bash = `command -v bash 2>/dev/null`);
40+
exit 1 unless $bash && -x $bash;
41+
42+
# Prepend its directory to PATH so the bare `mariadb` invocation
43+
# inside the wrapper resolves.
44+
my ($mariadb) = split /\s+/, ($ENV{MYSQL} // '');
45+
exit 1 unless $mariadb && -x $mariadb;
46+
$ENV{PATH} = dirname($mariadb) . ":$ENV{PATH}";
47+
48+
exit 0;
49+
EOF
50+
51+
if ($errno)
52+
{
53+
--skip mariabackup.sh wrapper unavailable (script, bash, or mariadb client missing)
54+
}
55+
56+
--let XTRABACKUP=$MARIABACKUP_WRAPPER

‎mysql-test/suite/mariabackup/wrapper_basic.result‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
CREATE TABLE t1(a INT PRIMARY KEY) ENGINE=InnoDB;
2+
INSERT INTO t1 VALUES (1), (2), (3);
3+
#
4+
# Full backup succeeds and runs BACKUP SERVER
5+
#
6+
FOUND 1 /Executing: BACKUP SERVER TO/ in wrapper.log
7+
#
8+
# (--parallel/--throttle/--no-lock/--safe-slave-backup)
9+
#
10+
FOUND 1 /Executing: BACKUP SERVER TO/ in wrapper.log
11+
#
12+
# --stream=mbstream emits a valid tar archive to stdout
13+
#
14+
FOUND 1 /Creating tar stream/ in wrapper.log
15+
#
16+
# --compress produces a valid gzip stream
17+
#
18+
FOUND 1 /Compressing with gzip/ in wrapper.log
19+
#
20+
# Backup into an already-existing target directory is rejected
21+
#
22+
FOUND 1 /Target directory already exists/ in wrapper.log
23+
#
24+
# Missing --target-dir is rejected
25+
#
26+
FOUND 1 /--target-dir required/ in wrapper.log
27+
#
28+
# Non-existent parent directory is rejected
29+
#
30+
FOUND 1 /Parent directory does not exist/ in wrapper.log
31+
DROP TABLE t1;

‎mysql-test/suite/mariabackup/wrapper_basic.test‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
--source include/have_mariabackup_wrapper.inc
2+
--source include/have_innodb.inc
3+
4+
--let $defaults=--defaults-file=$MYSQLTEST_VARDIR/my.cnf
5+
--let $logfile=$MYSQLTEST_VARDIR/tmp/wrapper.log
6+
--let SEARCH_FILE=$logfile
7+
--let SEARCH_ABORT=NOT FOUND
8+
9+
CREATE TABLE t1(a INT PRIMARY KEY) ENGINE=InnoDB;
10+
INSERT INTO t1 VALUES (1), (2), (3);
11+
12+
--echo #
13+
--echo # Full backup succeeds and runs BACKUP SERVER
14+
--echo #
15+
--let $targetdir=$MYSQLTEST_VARDIR/tmp/bk_full
16+
--exec $XTRABACKUP $defaults --backup --target-dir=$targetdir > $logfile 2>&1
17+
--let SEARCH_PATTERN=Executing: BACKUP SERVER TO
18+
--source include/search_pattern_in_file.inc
19+
--rmdir $targetdir
20+
21+
--echo #
22+
--echo # (--parallel/--throttle/--no-lock/--safe-slave-backup)
23+
--echo #
24+
--let $targetdir=$MYSQLTEST_VARDIR/tmp/bk_legacy
25+
--exec $XTRABACKUP $defaults --backup --target-dir=$targetdir --parallel=4 --throttle=100 --no-lock --safe-slave-backup > $logfile 2>&1
26+
--let SEARCH_PATTERN=Executing: BACKUP SERVER TO
27+
--source include/search_pattern_in_file.inc
28+
--rmdir $targetdir
29+
30+
--echo #
31+
--echo # --stream=mbstream emits a valid tar archive to stdout
32+
--echo #
33+
--let $targetdir=$MYSQLTEST_VARDIR/tmp/bk_stream
34+
--let $streamfile=$MYSQLTEST_VARDIR/tmp/bk_stream.tar
35+
--exec $XTRABACKUP $defaults --backup --target-dir=$targetdir --stream=mbstream > $streamfile 2>$logfile
36+
--exec tar -tf $streamfile > /dev/null
37+
--let SEARCH_PATTERN=Creating tar stream
38+
--source include/search_pattern_in_file.inc
39+
--rmdir $targetdir
40+
--remove_file $streamfile
41+
42+
--echo #
43+
--echo # --compress produces a valid gzip stream
44+
--echo #
45+
--let $targetdir=$MYSQLTEST_VARDIR/tmp/bk_gz
46+
--let $gzfile=$MYSQLTEST_VARDIR/tmp/bk.tar.gz
47+
--exec $XTRABACKUP $defaults --backup --target-dir=$targetdir --compress > $gzfile 2>$logfile
48+
--exec gzip -t $gzfile
49+
--let SEARCH_PATTERN=Compressing with gzip
50+
--source include/search_pattern_in_file.inc
51+
--rmdir $targetdir
52+
--remove_file $gzfile
53+
54+
--echo #
55+
--echo # Backup into an already-existing target directory is rejected
56+
--echo #
57+
--let $targetdir=$MYSQLTEST_VARDIR/tmp/bk_exists
58+
--mkdir $targetdir
59+
--error 1
60+
--exec $XTRABACKUP $defaults --backup --target-dir=$targetdir > $logfile 2>&1
61+
--let SEARCH_PATTERN=Target directory already exists
62+
--source include/search_pattern_in_file.inc
63+
--rmdir $targetdir
64+
65+
--echo #
66+
--echo # Missing --target-dir is rejected
67+
--echo #
68+
--error 1
69+
--exec $XTRABACKUP $defaults --backup > $logfile 2>&1
70+
--let SEARCH_PATTERN=--target-dir required
71+
--source include/search_pattern_in_file.inc
72+
73+
--echo #
74+
--echo # Non-existent parent directory is rejected
75+
--echo #
76+
--error 1
77+
--exec $XTRABACKUP $defaults --backup --target-dir=$MYSQLTEST_VARDIR/tmp/no_such_parent/bk > $logfile 2>&1
78+
--let SEARCH_PATTERN=Parent directory does not exist
79+
--source include/search_pattern_in_file.inc
80+
81+
DROP TABLE t1;
82+
--remove_file $logfile

‎scripts/mariabackup/README.md‎

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
# MariaDB Backup Wrapper
2+
3+
A drop-in `mariabackup`-compatible shell wrapper that translates the
4+
familiar CLI into MariaDB's server-side `BACKUP SERVER` SQL command.
5+
Lets DBAs migrate to BACKUP SERVER without changing existing scripts.
6+
7+
## Overview
8+
9+
`mariabackup.sh` masks the traditional `mariabackup` binary. With
10+
`--backup`, it parses MariaBackup options, sets the matching globals
11+
(`backup_include`, `backup_exclude`, `innodb_log_archive_start`) via
12+
the `mariadb` client, then issues `BACKUP SERVER TO '<dir>'`.
13+
Optional streaming, compression, and encryption are shell
14+
pipelines layered on the resulting directory. All actual
15+
backup work happens server-side.
16+
17+
**Prerequisites:** MariaDB with BACKUP SERVER support, `mariadb`
18+
client in `PATH`, an account with `BACKUP SERVER` + `SET GLOBAL`
19+
privileges, and `innodb_log_archive=ON` if using `--incremental-basedir`.
20+
21+
---
22+
23+
## --backup
24+
25+
### Description
26+
27+
Creates a backup using `BACKUP SERVER`. Produces a backup directory
28+
with data files, redo logs, and `backup.cnf` carrying LSN metadata.
29+
30+
### Structure
31+
32+
```
33+
mariabackup.sh --backup --target-dir=DIRECTORY [OPTIONS]
34+
``'
35+
36+
### Options
37+
38+
| Option | Description |
39+
| ---------------------------- | ------------------------------------------------------------------------ |
40+
| `--target-dir=DIR` | **(required)** Backup destination |
41+
| `--incremental-basedir=DIR` | Incremental backup based on a prior full backup |
42+
| `--stream=mbstream` | Stream the backup to stdout as a tar archive |
43+
| `--databases=REGEX` | Include pattern (comma-separated list supported) |
44+
| `--databases-exclude=REGEX` | Exclude pattern (comma-separated list supported) |
45+
| `--tables=REGEX` | Table-level include (used only if `--databases` not set) |
46+
| `--tables-exclude=REGEX` | Table-level exclude (used only if `--databases-exclude` not set) |
47+
| `--tables-file=FILE` | File of `database.table` entries, one per line, merged into `--tables` |
48+
| `--compress` | Pipe stream through `gzip` (or `pigz` if `--compress-threads` is set) |
49+
| `--compress-threads=N` | Use `pigz -p N` instead of `gzip` |
50+
| `--encrypt=ALG` | Pipe stream through `openssl enc -ALG -salt -pbkdf2` |
51+
52+
**Connection options** (forwarded to the `mariadb` client):
53+
`--user`/`-u`, `--password`/`-p`, `--host`/`-h`, `--port`/`-P`,
54+
`--socket`/`-S`, `--defaults-file`, `--defaults-extra-file`.
55+
56+
**Silently ignored** (BACKUP SERVER handles server-side):
57+
`--parallel`, `--throttle`, `--no-lock`, `--safe-slave-backup`.
58+
59+
60+
**Precedence:**
61+
62+
- `--databases` wins over `--tables`; `--databases-exclude` wins over
63+
`--tables-exclude` (the loser is ignored with a warning).
64+
- `--tables-exclude` wins over `--tables`.
65+
- `--tables-file` is merged into `--tables`.
66+
67+
### BACKUP SERVER Mapping
68+
69+
```sql
70+
SET GLOBAL innodb_log_archive_start=<lsn>; -- only for --incremental-basedir
71+
SET GLOBAL backup_include='<pattern>'; -- only if include built
72+
SET GLOBAL backup_exclude='<pattern>'; -- only if exclude built
73+
BACKUP SERVER TO '/path/to/backup';
74+
```
75+
76+
The patterns and LSNs land in `backup.cnf` inside the target directory.
77+
78+
---
79+
80+
### --target-dir
81+
82+
#### Description
83+
84+
Backup destination directory. Required. Must not already exist; parent
85+
must exist and be writable.
86+
87+
#### BACKUP SERVER Mapping
88+
89+
```sql
90+
BACKUP SERVER TO '/path/to/backup';
91+
```
92+
---
93+
94+
### --incremental-basedir
95+
96+
#### Description
97+
98+
Creates an incremental backup containing only redo logs since the base
99+
backup. The wrapper reads `innodb_log_recovery_target` from the base
100+
`backup.cnf` and sets `innodb_log_archive_start` to that LSN.
101+
102+
Requires `innodb_log_archive=ON` on the server.
103+
104+
#### BACKUP SERVER Mapping
105+
106+
```bash
107+
BASE_LSN=$(grep ^innodb_log_recovery_target /base/backup.cnf | cut -d= -f2)
108+
mariadb -e "SET GLOBAL innodb_log_archive_start=$BASE_LSN"
109+
mariadb -e "BACKUP SERVER TO '/incremental/path'"
110+
```
111+
---
112+
113+
### --stream
114+
115+
#### Description
116+
117+
Streams the backup directory to stdout as a tar archive. Only
118+
`mbstream` is supported (mapped to `tar`). The included `mbstream.sh`
119+
wrapper drops mbstream-specific flags (`-p`/`--parallel`) so legacy
120+
pipelines keep working.
121+
122+
#### BACKUP SERVER Mapping
123+
124+
```bash
125+
mariadb -e "BACKUP SERVER TO '/tmp/backup'"
126+
tar -c -f - -C /tmp/backup .
127+
```
128+
---
129+
130+
### --compress
131+
132+
#### Description
133+
134+
Pipes the stream through `gzip` (or `pigz` if `--compress-threads` is
135+
set). Implies `--stream=mbstream`. The compression algorithm argument
136+
(e.g. `--compress=quicklz`) is accepted for CLI compatibility but
137+
ignored; output is always gzip-compatible.
138+
139+
#### BACKUP SERVER Mapping
140+
141+
```bash
142+
mariadb -e "BACKUP SERVER TO '/tmp/backup'"
143+
tar -c -f - -C /tmp/backup . | gzip
144+
```
145+
146+
---
147+
148+
### --compress-threads
149+
150+
#### Description
151+
152+
Switches compression from `gzip` to `pigz -p N`. Implies `--compress`.
153+
154+
#### BACKUP SERVER Mapping
155+
156+
```bash
157+
tar -c -f - -C /tmp/backup . | pigz -p N
158+
```
159+
160+
---
161+
162+
### --encrypt
163+
164+
#### Description
165+
166+
Pipes the stream through `openssl enc -ALG -salt -pbkdf2`. Implies
167+
`--stream=mbstream`. Combines with `--compress`: compression runs
168+
before encryption.
169+
170+
#### BACKUP SERVER Mapping
171+
172+
```bash
173+
tar -c -f - -C /tmp/backup . \
174+
| gzip \ # if --compress
175+
| openssl enc -aes-256-cbc -salt -pbkdf2
176+
```
177+
---
178+
179+
180+
## backup.cnf Format
181+
182+
Auto-generated by `BACKUP SERVER` inside the target directory.
183+
184+
```ini
185+
[mariadbd]
186+
datadir=/backup/partial
187+
innodb_log_recovery_start=12288
188+
innodb_log_recovery_target=15000
189+
backup_include=^prod\..*
190+
backup_exclude=^prod\.temp.*,^prod\.cache.*
191+
```
192+
193+
| Field | Description |
194+
| ----------------------------- | -------------------------------------------- |
195+
| `datadir` | Backup directory path |
196+
| `innodb_log_recovery_start` | Starting LSN for redo log scanning |
197+
| `innodb_log_recovery_target` | Stopping LSN for applying redo log |
198+
| `backup_include` | Include pattern, partial backups only |
199+
| `backup_exclude` | Exclude pattern, partial backups only |
200+
201+
For a full backup, `_start` equals `_target`. For an incremental,
202+
`_target` advances past `_start`. The include/exclude lines are
203+
omitted when no filter was applied.
204+
205+
---
206+
207+
## BACKUP SERVER Variables
208+
209+
| Variable | Type | Description |
210+
| --------------------------- | -------------- | ---------------------------------------------------------------------------- |
211+
| `innodb_log_archive` | Boolean | Enables redo log archiving. Must be `ON` for incremental backups. |
212+
| `innodb_log_archive_start` | Integer (LSN) | LSN from which to preserve archived redo logs. Set by `--incremental-basedir`.|
213+
| `innodb_log_archive_target` | Integer (LSN) | LSN to which redo log should be applied
214+
| `backup_include` | String (POSIX ERE) | Comma-separated patterns matched against `db.table` (literal `.`). Same syntax as original `mariabackup --tables`. |
215+
| `backup_exclude` | String (POSIX ERE) | Comma-separated patterns matched against `db.table`. Beats `backup_include` on overlap. |
216+
217+
All are global. Wrapper sets them via `SET GLOBAL`, then runs
218+
`BACKUP SERVER`. Final patterns are also written into `backup.cnf` for
219+
restore tooling.
220+
221+
---

0 commit comments

Comments
 (0)