Skip to content

Commit adae82c

Browse files
authored
feat(历史数据归档): 增加历史数据归档脚本说明 (#60)
* feat(历史数据归档): 增加历史数据归档脚本 * 改成md格式
1 parent 2b9eb1c commit adae82c

File tree

2 files changed

+62
-0
lines changed

2 files changed

+62
-0
lines changed

mysql-backup/pt-archiver.md

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# pt-archiver 数据归档脚本
2+
3+
该脚本用于将 MySQL 表中的历史数据从源数据库归档到目标数据库,基于 `percona-toolkit``pt-archiver` 工具,并以 Docker 方式运行。
4+
5+
## 脚本内容
6+
7+
```bash
8+
#!/bin/bash
9+
set -euo pipefail
10+
#set -x
11+
12+
# ====== 源数据库 ======
13+
SRC_HOST=127.0.0.1
14+
SRC_PORT=3306
15+
SRC_USER=root
16+
SRC_PASS=xxxx
17+
SRC_DB=maintain
18+
SRC_TABLE=jtt808_position
19+
20+
# ====== 归档目标数据库 ======
21+
DST_HOST=127.0.0.1
22+
DST_PORT=3306
23+
DST_USER=root
24+
DST_PASS=xxxx
25+
DST_DB=maintain
26+
DST_TABLE=jtt808_position_history
27+
28+
# ====== 归档参数:开始ID,结束ID,每次批量归档数量 ======
29+
START_ID=0
30+
END_ID=100000
31+
BATCH_SIZE=30000
32+
33+
LOG_DIR=/var/log/pt-archiver
34+
mkdir -p "$LOG_DIR"
35+
LOG_FILE="$LOG_DIR/archive_$(date +%F_%H%M%S).log"
36+
37+
# ====== 构造 where 条件 ======
38+
WHERE="id > ${START_ID} AND id <= ${END_ID}"
39+
40+
# ====== 执行归档 ======
41+
docker run --rm -t --network host \
42+
-e LANG=C.UTF-8 -e PERL_UNICODE=SDA \
43+
percona/percona-toolkit \
44+
pt-archiver \
45+
--source h=$SRC_HOST,P=$SRC_PORT,u=$SRC_USER,p=$SRC_PASS,D=$SRC_DB,t=$SRC_TABLE,L=1,A=utf8mb4 \
46+
--dest h=$DST_HOST,P=$DST_PORT,u=$DST_USER,p=$DST_PASS,D=$DST_DB,t=$DST_TABLE,L=1,A=utf8mb4 \
47+
--charset=utf8mb4 \
48+
--where "$WHERE" \
49+
--limit $BATCH_SIZE \
50+
--bulk-insert \
51+
--nosafe-auto-increment \
52+
--no-delete \
53+
--progress 10000 \
54+
--sleep 1 \
55+
--statistics \
56+
2>&1 | tee -a "$LOG_FILE"
57+
58+
echo "Archive finished at $(date)" >> "$LOG_FILE"
59+
```

mysql8/conf/my.cnf

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,9 @@
44

55
[mysqld]
66

7+
# 开启 LOAD DATA LOCAL INFILE
8+
local_infile = ON
9+
710
# 不区分大小写 注:mysql8.x创建数据库后不能更改
811
lower_case_table_names = 1
912

0 commit comments

Comments
 (0)