Skip to content

Commit 16721ee

Browse files
committed
docs: [#315] add backup support implementation spec
- Add detailed implementation spec for backup support (Issue #315) - Update epic #309 to reflect completed research and consolidated tasks - Update roadmap section 7 with research completion and implementation link - Add new project words: pullable, sourceable The implementation follows a vertical slice approach: - Phase 1: Backup container image on Docker Hub (torrust/backup) - Phase 2: Backup service on first run (Docker Compose integration) - Phase 3: Scheduled backups via crontab - Phase 4: Documentation and final testing Key architectural decisions: - Pre-built container on Docker Hub (not built at deploy time) - Sourceable config file approach (backup.conf + backup-paths.txt) - DependencyDerivation trait for domain-driven depends_on rules - NetworkDerivation trait for domain-driven network assignment
1 parent a09067c commit 16721ee

4 files changed

Lines changed: 958 additions & 78 deletions

File tree

docs/issues/309-epic-add-backup-support.md

Lines changed: 52 additions & 64 deletions
Original file line numberDiff line numberDiff line change
@@ -50,68 +50,49 @@ The epic covers three types of backup targets:
5050

5151
## Goals
5252

53-
- [ ] Research and document backup strategies for each target type (Task 7.1)
54-
- [ ] Design backup feature specification with configuration schema (Task 7.2)
55-
- [ ] Implement configuration file backup functionality (Task 7.3)
56-
- [ ] Implement SQLite database backup functionality (Task 7.4)
57-
- [ ] Implement MySQL database backup functionality (Task 7.5)
53+
- [x] Research and document backup strategies for each target type (Task 7.1) ✅ Completed
54+
- [ ] Implement backup support with all features (Task 7.2 - consolidates original 7.2-7.5)
55+
- Backup container templates (Dockerfile, backup.sh)
56+
- Docker Compose integration (backup service)
57+
- Configuration schema extension
58+
- Ansible deployment playbooks
59+
- Crontab installation for scheduled backups
60+
- MySQL, SQLite, and config file backup support
5861

59-
## Implementation Approach
60-
61-
### Phase 1: Research (Task 7.1)
62-
63-
Research and document backup approaches for SQLite, MySQL, and configuration files. Focus on understanding:
64-
65-
- Available tools and techniques
66-
- Trade-offs between different approaches
67-
- Cloud redundancy strategies
68-
- Current Torrust Live Demo implementation
69-
70-
**Deliverable**: Research documentation in `docs/research/backup-strategies/`
71-
72-
### Phase 2: Specification (Task 7.2)
73-
74-
Design the backup feature specification based on research findings:
62+
## Research Findings Summary
7563

76-
- Backup configuration schema (per-service-type enablement)
77-
- User-provided destination paths
78-
- Integration with existing command structure
79-
- DDD architecture planning
64+
The research phase (Issue #310, PR #312) recommended the **maintenance-window
65+
hybrid approach**:
8066

81-
**Deliverable**: Feature specification document
67+
- **95% Container**: Backup logic in Docker container (portable, tested)
68+
- **5% Host Script**: Simple crontab + shell script for scheduling
69+
- **Architecture**: Crontab → Stop tracker → Run backup container → Start tracker
70+
- **Artifacts**: 58 unit tests, production-ready scripts and Dockerfile
8271

83-
### Phase 3: Configuration Backups (Task 7.3)
72+
Key decision: Use maintenance-window backups for all production deployments.
73+
The always-running sidecar approach is only viable for small databases (< 1GB).
8474

85-
Implement backup support for configuration files:
75+
See [Research Conclusions](../research/backup-strategies/conclusions.md) for
76+
detailed findings.
8677

87-
- Identify and document all configuration files
88-
- Create backup commands/steps
89-
- Support user-provided backup destinations
90-
- Test with E2E scenarios
91-
92-
**Deliverable**: Working configuration backup functionality
93-
94-
### Phase 4: SQLite Backups (Task 7.4)
95-
96-
Implement SQLite backup based on research and Live Demo approach:
78+
## Implementation Approach
9779

98-
- Safe file copying mechanism
99-
- Compression support
100-
- Integration with deployment workflow
101-
- E2E testing
80+
### Phase 1: Research ✅ COMPLETED (Task 7.1)
10281

103-
**Deliverable**: Working SQLite backup functionality
82+
Research documented backup approaches for SQLite, MySQL, and configuration files:
10483

105-
### Phase 5: MySQL Backups (Task 7.5)
84+
- Evaluated safe backup techniques for each database type
85+
- Tested with real 17GB production database (Torrust Demo)
86+
- Built and tested POC with 58 unit tests
87+
- Recommended maintenance-window hybrid approach
10688

107-
Implement MySQL backup based on research findings:
89+
**Deliverable**: Research documentation in `docs/research/backup-strategies/`
10890

109-
- Tool selection (mysqldump, physical backup, etc.)
110-
- Concurrency handling
111-
- Docker container integration
112-
- E2E testing with MySQL deployments
91+
### Phase 2: Implementation (Task 7.2)
11392

114-
**Deliverable**: Working MySQL backup functionality
93+
Based on research findings, implement backup support as a single integrated feature.
94+
See [implementation spec](315-implement-backup-support.md) for the detailed implementation
95+
plan with incremental steps.
11596

11697
## Context: Torrust Live Demo Implementation
11798

@@ -134,13 +115,13 @@ This provides a proven baseline for SQLite backups in production, but opportunit
134115

135116
## Tasks
136117

137-
- [ ] #310 - Research database backup strategies
138-
- [ ] #TBD - Define backup feature specification
139-
- [ ] #TBD - Implement configuration file backups
140-
- [ ] #TBD - Implement SQLite database backups
141-
- [ ] #TBD - Implement MySQL database backups
118+
- [x] #310 - Research database backup strategies ✅ Completed
119+
- [ ] #315 - Implement backup support (consolidates original tasks 7.2-7.5)
142120

143-
(Tasks will be created and linked as work progresses)
121+
**Note**: Based on research findings (PR #312), the original tasks 7.2-7.5 have been
122+
consolidated into a single implementation issue (see [spec](315-implement-backup-support.md)). The research provided
123+
production-ready POC artifacts and clear implementation guidance that makes
124+
incremental sub-issues unnecessary.
144125

145126
## Scope Boundaries
146127

@@ -150,17 +131,18 @@ This provides a proven baseline for SQLite backups in production, but opportunit
150131
- ✅ Configuration file backup
151132
- ✅ User-provided backup destination paths
152133
- ✅ Compression support
134+
- ✅ Backup retention cleanup (configurable days)
135+
- ✅ Scheduled backups via crontab (deployer installs crontab)
153136
- ✅ Integration with existing command structure
154137

155138
### Out of Scope
156139

157-
- ❌ Automated backup scheduling (users can use cron/systemd timers)
158140
- ❌ Backup encryption (future enhancement)
159141
- ❌ Automated restore functionality (future enhancement)
160142
- ❌ Backup testing/verification (future enhancement)
161143
- ❌ Multi-region backup replication (users handle with volumes/cloud services)
162-
- ❌ Backup retention policies (users manage retention)
163144
- ❌ Volume management (users provide mounted locations)
145+
- ❌ Backup notifications/alerts (future enhancement)
164146

165147
**Note**: Volume management is explicitly out of scope. Users are responsible for:
166148

@@ -174,11 +156,14 @@ The deployer will assume backup destinations are already mounted and accessible.
174156

175157
This epic is complete when:
176158

177-
- [ ] Research is documented for all backup target types
178-
- [ ] Feature specification is complete and approved
179-
- [ ] Configuration file backup is implemented and tested
180-
- [ ] SQLite backup is implemented and tested (compatible with Live Demo approach)
159+
- [x] Research is documented for all backup target types
160+
- [ ] Backup container templates are integrated into deployer
161+
- [ ] Backup configuration is part of environment creation schema
162+
- [ ] Backup artifacts are deployed via Ansible playbooks
163+
- [ ] Crontab is installed for scheduled backups
164+
- [ ] SQLite backup is implemented and tested
181165
- [ ] MySQL backup is implemented and tested
166+
- [ ] Configuration file backup is implemented and tested
182167
- [ ] All E2E tests pass
183168
- [ ] Documentation is updated (user guide, commands reference)
184169
- [ ] Pre-commit checks pass for all changes
@@ -198,9 +183,12 @@ This epic is complete when:
198183

199184
The Torrust Tracker stores critical data:
200185

201-
- **Peer information**: Active peers and swarm state
202-
- **Torrent statistics**: Upload/download counters, peer counts
186+
- **Torrent statistics**: Upload/download counters, peer counts, torrent metadata
203187
- **Authentication tokens**: User credentials and access control
188+
- **Whitelisted torrents**: List of allowed torrents (if private tracker)
189+
190+
**Note**: Peer information (active peers and swarm state) is kept in memory only
191+
and is not persisted to the database, so it cannot be backed up.
204192

205193
Losing this data would be catastrophic for tracker operators. Backups must be:
206194

0 commit comments

Comments
 (0)