Skip to content

Commit 2dfec59

Browse files
authored
br: add more info about the br log structure (#20443) (#20458)
1 parent 0e90fa7 commit 2dfec59

1 file changed

Lines changed: 32 additions & 26 deletions

File tree

br/br-log-architecture.md

Lines changed: 32 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -108,18 +108,24 @@ Log backup generates the following types of files:
108108
.
109109
├── v1
110110
│   ├── backupmeta
111-
│   │   ├── {min_restored_ts}-{uuid}.meta
112-
│   │   ── {checkpoint}-{uuid}.meta
111+
│   │   ├── ...
112+
│   │   ── {resolved_ts}-{uuid}.meta
113113
│   ├── global_checkpoint
114-
│   │   ── {store_id}.ts
115-
│   ── {date}
116-
│      ── {hour}
117-
│         ── {store_id}
118-
│            ├── {min_ts}-{uuid}.log
119-
│            ── {min_ts}-{uuid}.log
120-
── v1_stream_truncate_safepoint.txt
114+
│   │   ── {store_id}.ts
115+
│   ── {date}
116+
│      ── {hour}
117+
│         ── {store_id}
118+
│            ├── ...
119+
│            ── {min_ts}-{uuid}.log
120+
── v1_stream_truncate_safepoint.txt
121121
```
122122

123+
Explanation of the backup file directory structure:
124+
125+
- `backupmeta`: stores backup metadata. The `resolved_ts` in the filename indicates the backup progress, meaning that data before this TSO has been fully backed up. However, note that this TSO only reflects the progress of certain shards.
126+
- `global_checkpoint`: represents the global backup progress. It records the latest point in time to which data can be restored using `br restore point`.
127+
- `{date}/{hour}`: stores backup data for the corresponding date and hour. When cleaning up storage, always use `br log truncate` instead of manually deleting data. This is because the metadata references the data in this directory, and manual deletion might lead to restore failures or data inconsistencies after restore.
128+
123129
The following is an example:
124130

125131
```
@@ -129,24 +135,24 @@ The following is an example:
129135
│   │   ├── ...
130136
│   │   ├── 435213818858112001-e2569bda-a75a-4411-88de-f469b49d6256.meta
131137
│   │   ├── 435214043785779202-1780f291-3b8a-455e-a31d-8a1302c43ead.meta
132-
│   │   ── 435214443785779202-224f1408-fff5-445f-8e41-ca4fcfbd2a67.meta
138+
│   │   ── 435214443785779202-224f1408-fff5-445f-8e41-ca4fcfbd2a67.meta
133139
│   ├── global_checkpoint
134140
│   │   ├── 1.ts
135141
│   │   ├── 2.ts
136-
│   │   ── 3.ts
137-
│   ── 20220811
138-
│      ── 03
139-
│         ├── 1
140-
│         │   ├── ...
141-
│         │   ├── 435213866703257604-60fcbdb6-8f55-4098-b3e7-2ce604dafe54.log
142-
│         │   ── 435214023989657606-72ce65ff-1fa8-4705-9fd9-cb4a1e803a56.log
143-
│         ├── 2
144-
│         │   ├── ...
145-
│         │   ├── 435214102632857605-11deba64-beff-4414-bc9c-7a161b6fb22c.log
146-
│         │   ── 435214417205657604-e6980303-cbaa-4629-a863-1e745d7b8aed.log
147-
│         ── 3
148-
│            ├── ...
149-
│            ├── 435214495848857605-7bf65e92-8c43-427e-b81e-f0050bd40be0.log
150-
│            ── 435214574492057604-80d3b15e-3d9f-4b0c-b133-87ed3f6b2697.log
151-
── v1_stream_truncate_safepoint.txt
142+
│   │   ── 3.ts
143+
│   ── 20220811
144+
│      ── 03
145+
│         ├── 1
146+
│         │   ├── ...
147+
│         │   ├── 435213866703257604-60fcbdb6-8f55-4098-b3e7-2ce604dafe54.log
148+
│         │   ── 435214023989657606-72ce65ff-1fa8-4705-9fd9-cb4a1e803a56.log
149+
│         ├── 2
150+
│         │   ├── ...
151+
│         │   ├── 435214102632857605-11deba64-beff-4414-bc9c-7a161b6fb22c.log
152+
│         │   ── 435214417205657604-e6980303-cbaa-4629-a863-1e745d7b8aed.log
153+
│         ── 3
154+
│            ├── ...
155+
│            ├── 435214495848857605-7bf65e92-8c43-427e-b81e-f0050bd40be0.log
156+
│            ── 435214574492057604-80d3b15e-3d9f-4b0c-b133-87ed3f6b2697.log
157+
── v1_stream_truncate_safepoint.txt
152158
```

0 commit comments

Comments
 (0)