PROJECT_STRUCTURE.md

Project Structure

Complete file structure of the postgres-backup-s3 project.

📁 Directory Tree

postgres-backup-s3/
│
├── 🆕 src/
│   └── index.ts                    # Main TypeScript application (scheduler & orchestrator)
│
├── 🔧 Configuration Files
│   ├── package.json                # Node/Bun dependencies
│   ├── tsconfig.json               # TypeScript configuration
│   ├── docker-compose.example.yml  # Example Docker Compose setup
│   ├── env.example                 # Environment variables template
│   ├── Makefile                    # Development convenience commands
│   ├── .dockerignore               # Docker build exclusions
│   └── .gitignore                  # Git exclusions
│
├── 🐳 Docker Files
│   ├── Dockerfile              # Bun-based Dockerfile
│   └── run.sh                      # Container entry point
│
├── 💾 Backup/Restore Scripts
│   ├── backup.sh                   # Backup shell script (enhanced with storage types)
│   └── restore.sh                  # Restore shell script (enhanced with storage types)
│
├── 📖 Documentation
│   ├── README.md                   # Comprehensive documentation
│   ├── CONTRIBUTING.md             # Contribution guidelines
│   ├── QUICK_REFERENCE.md          # Quick command reference
│   └── PROJECT_STRUCTURE.md        # This file
│
├── 📜 License
│   ├── LICENSE                     # MIT license with attribution
│
└── 🤖 CI/CD
    └── .github/
        └── workflows/
            └── docker-build.yml    # Automated Docker image builds

📄 File Descriptions

Core Application Files

src/index.ts

Purpose: Main application entry point written in TypeScript
Responsibilities:

• Parse environment variables
• Detect operation mode (backup/restore/scheduled)
• Handle multiple database configurations
• Execute shell scripts with proper streaming
• Manage cron scheduling
• Error handling and logging

Key Functions:

• parseDatabases() - Parse comma-separated database list
• executeScript() - Execute shell scripts with streaming output
• backupDatabase() - Backup single database
• executeBackup() - Handle multiple databases (sequential/parallel)
• executeRestore() - Handle restore operations
• main() - Main entry point with mode detection

Shell Scripts

backup.sh

Purpose: PostgreSQL backup logic
Enhancements:

• ✅ Accepts database name as first argument
• ✅ Storage type detection (S3/R2/COMPATIBLE)
• ✅ Dynamic endpoint configuration
• ✅ Better error messages
• ✅ Support for custom format and compression

Key Features:

• Database validation
• Storage configuration based on type
• pg_dump or pg_dumpall execution
• Compression (gzip/pigz or custom format)
• Optional encryption (AES-256-CBC)
• S3/R2 upload with AWS CLI
• Optional old backup cleanup

restore.sh

Purpose: PostgreSQL restore logic
Enhancements:

• ✅ Storage type detection (S3/R2/COMPATIBLE)
• ✅ Consistent with backup.sh logic

Key Features:

• Backup file download from S3/R2
• Automatic decryption if needed
• Optional database drop/create
• Support for compressed and custom format
• Parallel restore with pg_restore
• Cleanup of temporary files

run.sh

Purpose: Container entry point
Responsibilities:

• Configure AWS S3v4 signature if needed
• Execute the compiled Bun binary

Configuration Files

package.json

Dependencies:

• cron - Cron job scheduling
• @types/node - Node.js type definitions
• @types/cron - Cron type definitions
• @types/bun - Bun runtime types

tsconfig.json

Configuration:

• Target: ESNext
• Module: ESNext
• Bundler module resolution
• Strict mode disabled for compatibility
• Bun types included

docker-compose.example.yml

Contents:

• PostgreSQL service with health checks
• Backup service with full configuration
• Optional Minio service
• Network and volume definitions
• Comprehensive environment variable examples

Makefile

Commands:

• make help - Show all commands
• make install - Install dependencies
• make build - Build Docker image
• make run - Start services
• make test-backup - Test backup
• make test-restore - Test restore
• make list-backups - List S3 backups
• make clean - Cleanup

Docker Files

Dockerfile

Multi-stage build:

1. Build stage: Compile TypeScript with Bun
2. Final stage: Alpine Linux with runtime dependencies

Installed packages:

• bash, coreutils
• postgresql18-client
• aws-cli
• openssl (for encryption)
• pigz (parallel compression)

Size: ~150MB (estimated)

Dockerfile (original)

Kept for reference, uses Go compilation.

Documentation Files

README.md (2000+ lines)

Sections:

• Features overview
• Quick start guide
• Storage configuration examples
• Multiple database support
• Backup scheduling
• Encryption guide
• Compression options
• Restore instructions
• Full environment variable reference
• Production examples
• Troubleshooting

QUICK_REFERENCE.md

Sections:

• Quick start commands
• Storage type configs
• Schedule examples
• Common commands
• Troubleshooting tips

CONTRIBUTING.md

Sections:

• Development setup
• Coding standards
• Testing checklist
• PR process
• Commit message format

CI/CD

.github/workflows/docker-build.yml

Triggers:

• Push to main/master
• Git tags (v*)
• Pull requests

Actions:

• Multi-platform build (amd64, arm64)
• Push to Docker Hub
• Push to GitHub Container Registry
• Tag management (latest, semver)

🔄 Data Flow

Backup Flow

User
  ↓
docker-compose.yml (configuration)
  ↓
run.sh (entry point)
  ↓
postgres-backup (compiled binary)
  ↓
src/index.ts (TypeScript orchestrator)
  ↓
  ├─ Parse env vars
  ├─ Detect mode
  ├─ Parse database list
  └─ For each database:
       ↓
     backup.sh <database_name>
       ↓
       ├─ Detect storage type
       ├─ Connect to PostgreSQL
       ├─ pg_dump/pg_dumpall
       ├─ Compress
       ├─ Encrypt (optional)
       ├─ Upload to S3/R2
       └─ Cleanup old backups (optional)

Restore Flow

User (docker compose run)
  ↓
Override BACKUP_FILE env var
  ↓
postgres-backup
  ↓
src/index.ts (detect restore mode)
  ↓
restore.sh
  ↓
  ├─ Detect storage type
  ├─ Download from S3/R2
  ├─ Decrypt (if needed)
  ├─ Drop DB (optional)
  ├─ Create DB (optional)
  ├─ Restore with psql/pg_restore
  └─ Cleanup temp files

📊 File Sizes (Approximate)

File	Lines	Size	Purpose
src/index.ts	200	6KB	Main application
backup.sh	150	4KB	Backup logic
restore.sh	180	5KB	Restore logic
README.md	600	40KB	Documentation
Dockerfile	60	2KB	Docker build
docker-compose.example.yml	100	4KB	Example setup

🎯 Key Design Decisions

Why Bun?

• ✅ Faster startup than Node.js
• ✅ Built-in TypeScript support
• ✅ Native subprocess handling
• ✅ Smaller compiled binary
• ✅ Better developer experience

Why Keep Shell Scripts?

• ✅ Battle-tested backup/restore logic
• ✅ Direct access to pg_dump/pg_restore
• ✅ Easy to debug and modify
• ✅ Portable across systems
• ✅ Community familiarity

Why TypeScript for Orchestration?

• ✅ Type safety
• ✅ Better async handling
• ✅ Cron library availability
• ✅ Easy to extend
• ✅ Modern syntax

Why Multi-stage Docker Build?

• ✅ Smaller final image
• ✅ No build dependencies in production
• ✅ Better security (fewer packages)
• ✅ Faster container startup

🔐 Security Considerations

Sensitive Files

• .env - Never commit (in .gitignore)
• env.example - Safe template without secrets

Credentials in Docker

• Use environment variables
• Consider Docker secrets for production
• Rotate credentials regularly

S3/R2 Permissions

• Use least-privilege IAM policies
• Enable bucket encryption
• Use private buckets only

🚀 Deployment Options

Docker Compose (Recommended)

docker compose up -d

Kubernetes

Use deployment YAML (user-created)

Standalone Docker

docker run -e ... postgres-backup-s3

📈 Future Enhancements (Not Implemented)

Potential features for future versions:

• Database pattern matching
• Webhook notifications
• Backup verification
• Web UI
• Metrics/monitoring
• Incremental backups

🎓 Learning Resources

For contributors and users:

• Bun Documentation
• PostgreSQL Backup Documentation
• AWS CLI S3 Commands
• Cloudflare R2 Documentation

---

Last Updated: October 6, 2025
Version: 2.0.0
Status: ✅ Production Ready