Skip to content

feat: backup encryption#1400

Closed
caffeinated92 wants to merge 13 commits into
developfrom
backup-enc
Closed

feat: backup encryption#1400
caffeinated92 wants to merge 13 commits into
developfrom
backup-enc

Conversation

@caffeinated92
Copy link
Copy Markdown
Collaborator

@caffeinated92 caffeinated92 commented Mar 11, 2026

Summary

  • add AES-256-CBC backup encryption with CLI decrypt command and secret-aware passphrase resolution
  • support directory logical backups via archive encryption (tar/tar.gz) with streaming OpenSSL encrypt/decrypt and restic compatibility
  • introduce per-file directory encryption mode (opt-in) with encryptionMode/encryptionTool metadata for forward-compatible restores
  • add OpenSSL startup observability and hard-fail when encryption is enabled but OpenSSL is missing
  • update implementation docs covering phase 1 and phase 2 encryption behavior

Testing

  • go test ./cluster/... ./server/... ./config/...

@caffeinated92
feat: add backup encryption workflow and decrypt command
29d920c 
Enable optional AES-256-CBC encryption for single-file backups before restic upload and add a CLI decrypt-backup command for restore workflows. Treat backup-encryption-passphrase as a managed secret and resolve passphrase sources with env/config/admin fallback for safer operations.
@caffeinated92
Merge remote-tracking branch 'origin/develop' into backup-enc
05e2f2e
@caffeinated92
feat: encrypt logical backup directories as OpenSSL archives
f51c567 
Add tar/tar.gz packaging plus streaming OpenSSL encryption/decryption for directory-based logical backups so large artifacts avoid in-memory processing while preserving restore and restic compatibility. Record encryptionTool metadata for forward-compatible decrypt behavior and expose new directory encryption config controls.
@caffeinated92
feat: add per-file mode for directory backup encryption
c44325c 
Introduce backup-encryption-directory-mode with archive default and per-file option, plus encryptionMode metadata to preserve cross-version compatibility. Extend restore logic to decrypt per-file directories in place and clean up .enc files after successful decrypt.
@caffeinated92
Merge remote-tracking branch 'origin/develop' into backup-enc
167e098
@caffeinated92 caffeinated92 marked this pull request as draft March 11, 2026 10:03
@caffeinated92
Align backup decryption with salted passphrase format
16f9531
@caffeinated92
Secure backup encryption by passing passphrase via file descriptor
ef318cd
@caffeinated92
Add safe per-file encrypted restore with transactional rollback and u…
00704dd 
…nsafe override

- Per-file encrypted restore now requires backup-keep-until-valid=true for safe reversible staging
- Implements incremental journal persistence for crash-safe rollback (decrypt -> rename to .old -> activate plaintext)
- Journal write failures cause fatal abort with in-memory rollback
- Stale journal rollback failure aborts restore preparation
- Cleanup preserves journal if rollback fails for recovery
- Add backup-encryption-unsafe-per-file-restore flag to bypass safety for low-disk environments
- Add tests for gated behavior, transactional flow, unsafe path, and journal failure rollback
- Update docs with new safety guarantees
@caffeinated92
cluster: validate symlink targets in archive extraction to prevent pa…
0645e28 
…th traversal

- Reject empty, absolute, and escaping symlink targets in extractArchiveToDir
- Use isPathWithinBase to enforce boundary checking
- Fail-fast on first unsafe symlink to prevent write-through attacks
- Add comprehensive tests for absolute/escaping/empty/in-tree symlinks
- Add deterministic traversal-chain test with shared temp root
@caffeinated92
cluster: harden archive extraction for symlinks and hardlinks
cef7bd4 
Validate symlink and hardlink targets stay within the extraction root, normalize
archive link paths for cross-platform tar semantics, and resolve hardlinks safely
with a copy fallback. Add regression tests covering traversal protection,
hardlink ordering, ambiguity handling, and path normalization policies.
@caffeinated92
clients: stream encrypted backup files through OpenSSL
bc26dbc 
Replaces in-memory buffering with file-to-file streaming to support
decrypting large (100+ GB) backups without loading entire file into
memory. Pipes *os.File directly to OpenSSL stdin/stdout instead of
using bytes.NewReader. Maintains passphrase-first then legacy key/iv
fallback behavior with proper temp file cleanup.
@caffeinated92
cluster: add passphrase fallback warnings and strict mode for backup …
1482db9 
…encryption

- Track passphrase source (env, config, api-credentials, server DB)
- Warn when fallback rotating credentials are used for encryption
- Add backup-encryption-require-explicit-passphrase flag to enforce
  explicit passphrase (env var or config) and reject rotating credentials
- Use lazy passphrase resolution in per-file encrypt/decrypt to avoid
  warnings on no-op directory operations
- Remove duplicate warnings from preflight (now only warn on actual use)
- Add nil-safety guards to prevent panics when ClusterGroup is absent
- Add comprehensive tests for source precedence, strict mode, and
  lazy resolution behavior
@caffeinated92
cluster: add tests for backup encryption failures and archive extract…
2f18b63 
…ion hardening

- Add streaming encryption negative-path tests:
  - TestBackupEncryptionStreamDecryptWrongPasswordFails
  - TestBackupEncryptionStreamEncryptEmptyPassphraseFails
- Add per-file directory encrypt/decrypt round-trip test
- Add archive extraction path traversal tests:
  - TestExtractArchiveToDirRejectsRelativePathTraversalEntryName
  - TestExtractArchiveToDirRejectsAbsoluteEntryName (table-driven)
- Add hardlink source-type edge case tests:
  - TestExtractArchiveToDirRejectsHardlinkToSymlinkSource
  - TestExtractArchiveToDirRejectsHardlinkToDirectorySource
- Security fix: reject absolute archive entry names before switch
  (covers TypeDir, TypeReg, TypeSymlink, TypeLink uniformly)
@caffeinated92 caffeinated92 marked this pull request as ready for review March 13, 2026 08:16
@signal18 signal18 deleted a comment from claude Bot Mar 13, 2026
@signal18 signal18 deleted a comment from claude Bot Mar 13, 2026
@signal18 signal18 deleted a comment from claude Bot Mar 13, 2026
@signal18 signal18 deleted a comment from claude Bot Mar 13, 2026
@signal18 signal18 deleted a comment from claude Bot Mar 13, 2026
@caffeinated92 caffeinated92 changed the title feat: expand backup encryption for directory outputs feat: backup encryption Mar 13, 2026
@caffeinated92
Copy link
Copy Markdown
Collaborator Author

caffeinated92 commented Mar 13, 2026

@claude

@claude
Copy link
Copy Markdown

claude Bot commented Mar 13, 2026 •
edited
Loading

Claude encountered an error —— View job

PR Review: feat: backup encryption

  • Gather context and read changed files
  • Review core encryption implementation (cluster/srv_job_backup.go)
  • Review CLI decrypt command (clients/client_decrypt_backup.go)
  • Review configuration and server changes
  • Review tests
  • Summarize findings

@signal18 signal18 deleted a comment from claude Bot Mar 13, 2026
@caffeinated92 caffeinated92 marked this pull request as draft March 13, 2026 09:52
@caffeinated92
Copy link
Copy Markdown
Collaborator Author

caffeinated92 commented Mar 13, 2026

I found 3 important issues to address before merge:

  • High Encrypted single-file logical backups are not decrypted on restore
    • encryptBackupMetadataFile marks encrypted files with EncryptionMode = "archive" (cluster/srv_job_backup.go:4600).
    • But restore mode detection only treats "per-file" as encrypted metadata; otherwise it only infers archive mode from .tar.enc/.tar.gz.enc suffix (cluster/srv_job_backup.go:5294).
    • For common logical file backups like *.sql.gz.enc, prepareEncryptedDirectoryLogicalRestorePath returns the original encrypted path unchanged (cluster/srv_job_backup.go:5312), so mysqldump reseed paths get ciphertext.
    • This likely breaks restore for encrypted non-directory logical backups.
  • High Tar extraction still allows symlink write-chain traversal for regular files
    • In extractArchiveToDir, regular entries are written directly to destAbs after lexical path checks (cluster/srv_job_backup.go:5139), but there’s no symlink-resolution check on parent or final path.
    • If an earlier archive entry creates a symlink inside tree, a later regular file entry targeting that path can escape the extraction root.
    • This is exactly the threat modeled by TestExtractArchiveToDirBlocksSymlinkTraversalWriteChain (cluster/srv_job_backup_extract_test.go:118), but current extraction logic doesn’t appear to enforce it for tar.TypeReg.
  • Medium CLI decrypt still exposes passphrase via process arguments
    • decryptWithPassphraseOpenSSL builds OpenSSL args with -pass pass: (clients/client_decrypt_backup.go:110).
    • That leaks secret material to process list/audit surfaces, unlike the server path that uses fd:3.
    • Suggest aligning CLI decryption to fd-based passphrase handling for consistency and safer local ops.

@caffeinated92 caffeinated92 deleted the backup-enc branch April 9, 2026 23:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@caffeinated92