Skip to content

Commit cb5e0de

Browse files
committed
refactor(storage): trim to local + S3-compatible; add NAS exposure guide
Storage adapters trimmed from 9 to 2: keep LocalStorageAdapter and S3StorageAdapter. Remove Google Drive, OneDrive, Dropbox, FTP, GCS, Azure, and rclone-as-adapter. S3-compatible object storage covers all of these use cases (MinIO, Cloudflare R2, Garage, SeaweedFS, Ceph RGW, etc.) with a single, vendor-neutral code path that supports the multipart semantics OpenCodeHub needs for large blobs. Code: - src/lib/storage.ts: rewrite (2331 -> 762 lines). Drop 7 adapter classes and the driver-specific StorageConfig fields. Factory is now strictly { local, s3 }. - src/lib/git-storage.ts: isCloudStorage() = type === 's3' (fixes a pre-existing bug where dropbox/ftp were silently treated as local). - src/lib/validation.ts: StorageConfigSchema restricted to ['local', 's3']. - src/lib/env-validation.ts: STORAGE_TYPE validator + S3-required fields; gdrive/rclone validators removed. - src/pages/api/admin/config/storage.ts: normalizeConfig() whitelists drivers and sanitises the payload (was a passthrough spread, which silently accepted removed fields). - scripts/verify-env-config.ts: rewritten to verify the two supported drivers end-to-end (was hardcoded to STORAGE_TYPE=gdrive). - package.json: drop @google-cloud/storage, @azure/storage-blob, googleapis, basic-ftp (4 packages, ~12 MB of transitive deps). Configuration: - .env.example: STORAGE_TYPE is the canonical var; drop the gcs/azure/ ftp/rclone/gdrive sections. Document that any S3-v4 vendor works via STORAGE_ENDPOINT. Documentation: - docs/guides/storage-adapters.md + docs-site mirror: full rewrite for local + S3 with preset configs for AWS S3, MinIO, Cloudflare R2, Garage, SeaweedFS, Ceph RGW, Wasabi, Backblaze B2, DigitalOcean Spaces. - docs/administration/configuration.md (both versions): storage env table updated; removed drivers called out explicitly. - docs/administration/deployment.md, deploy-cpanel.md, deploy-cyberpanel.md, deploy-cloudflare.md, installation.md (both versions): STORAGE_DRIVER -> STORAGE_TYPE; canonical env variable names throughout. - docs/development/architecture.md: storage-layer diagram reduced to two adapters; removed driver rows replaced with explanatory note. - AGENTS.md: 8+ backends -> 2 backends; env table; section 6.4 rewritten. - src/pages/admin/_storage-disabled.txt: rclone + rclone-sub-remote options removed (file is intentionally not routed). NEW: docs/administration/expose-nas.md + docs-site mirror How to put OpenCodeHub on the public internet from a NAS or home server WITHOUT port forwarding, covering: 1. Tailscale Funnel (WireGuard mesh + edge ingress) 2. Cloudflare Tunnel (outbound cloudflared daemon) with a side-by-side comparison, NAS-specific install snippets (Synology DSM, TrueNAS SCALE, QNAP, generic Linux), the right TRUSTED_PROXIES values for each, an 8-item hardening checklist, SSH-git note, and a troubleshooting section. Validation: - bun install -> 4 packages removed cleanly. - bun run typecheck -> 0 errors. - bun run lint -> 0 errors, 0 warnings, 478 hints. - SKIP_REDIS_CHECK=1 bun run test -> 546/546 pass. - bun run security:audit -> PASSED (0 disallowed high/critical). - bun run scripts/verify-env-config.ts -> 10/10 storage checks pass.
1 parent b382597 commit cb5e0de

27 files changed

Lines changed: 1594 additions & 2290 deletions

.env.example

Lines changed: 18 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -64,31 +64,26 @@ GIT_SSH_PORT=2222
6464
GIT_SSH_HOST_KEY=./data/ssh/host_key
6565

6666
# Storage Configuration
67-
# Supported: local, s3, gcs, azure, dropbox, gdrive, ftp, rclone
68-
STORAGE_DRIVER=local
67+
# Supported backends: local, s3
68+
# local — filesystem rooted at STORAGE_PATH (default ./data/storage).
69+
# s3 — any S3-compatible object store: AWS S3, MinIO, Cloudflare R2,
70+
# Garage, SeaweedFS, Ceph RGW, Wasabi, Backblaze B2, etc.
71+
# Set STORAGE_ENDPOINT for non-AWS vendors. Path-style is
72+
# enabled automatically when an endpoint is provided.
73+
#
74+
# Previously supported: gcs, azure, dropbox, gdrive, ftp, rclone-as-adapter.
75+
# These were removed in favour of S3-compatible object storage and the
76+
# rclone backup utility (see docs/administration/storage-adapters.md).
77+
STORAGE_TYPE=local
6978
STORAGE_PATH=./data/storage
7079

71-
# S3 Configuration (if STORAGE_DRIVER=s3)
72-
S3_BUCKET=
73-
S3_REGION=
74-
S3_ACCESS_KEY=
75-
S3_SECRET_KEY=
76-
S3_ENDPOINT=
77-
78-
# Google Cloud Storage (if STORAGE_DRIVER=gcs)
79-
GCS_BUCKET=
80-
GCS_PROJECT_ID=
81-
GCS_KEY_FILE=
82-
83-
# FTP Configuration (if STORAGE_DRIVER=ftp)
84-
FTP_HOST=
85-
FTP_PORT=21
86-
FTP_USER=
87-
FTP_PASSWORD=
88-
89-
# Rclone Configuration (if STORAGE_DRIVER=rclone)
90-
RCLONE_REMOTE=
91-
RCLONE_CONFIG_PATH=
80+
# S3 Configuration (required when STORAGE_TYPE=s3)
81+
STORAGE_BUCKET=
82+
STORAGE_REGION=us-east-1
83+
STORAGE_ENDPOINT=
84+
STORAGE_ACCESS_KEY_ID=
85+
STORAGE_SECRET_ACCESS_KEY=
86+
# STORAGE_PATH= (left empty for s3; only used for local)
9287

9388
# CI/CD Runner Configuration
9489
RUNNER_ENABLED=true

AGENTS.md

Lines changed: 8 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
- Stack-first PR workflows (Graphite-style stacked PRs)
1313
- Merge queue with speculative builds and priority lanes
1414
- GitHub Actions-compatible CI/CD pipeline engine + Docker-based runner
15-
- Pluggable storage (local, S3, GCS, Azure, Google Drive, OneDrive, Dropbox, FTP, rclone)
15+
- Pluggable storage (`local` filesystem or `s3` — any S3-compatible object store: AWS S3, MinIO, Cloudflare R2, Garage, SeaweedFS, Ceph RGW, Wasabi, Backblaze B2, etc.)
1616
- Multi-database support (PostgreSQL, SQLite, Turso/LibSQL)
1717
- Built-in AI code review, automations, webhooks
1818

@@ -66,7 +66,7 @@
6666
| Git | Native `git` CLI + `simple-git` + `isomorphic-git` + `nodegit` |
6767
| SSH | `ssh2` library |
6868
| CI/CD | Dockerode (Docker API) |
69-
| Storage | Abstract adapter pattern with 8+ backends |
69+
| Storage | Abstract adapter pattern; 2 backends (`local`, `s3`-compatible) |
7070
| Realtime | Custom (see `src/lib/realtime.ts`) |
7171
| Queue | BullMQ + Redis |
7272
| CLI | Commander.js + Inquirer + Chalk + `simple-git` |
@@ -250,16 +250,12 @@ Browser/Git CLI → Astro middleware (auth, rate limit, CSRF)
250250

251251
### 6.4 Storage Abstraction
252252
All blob storage goes through `StorageAdapter` abstract class:
253-
- `LocalStorageAdapter` — filesystem
254-
- `S3StorageAdapter` — S3/MinIO/R2
255-
- `GCSStorageAdapter` — Google Cloud Storage
256-
- `AzureStorageAdapter` — Azure Blob
257-
- `GoogleDriveStorageAdapter` — Google Drive
258-
- `OneDriveStorageAdapter` — Microsoft OneDrive
259-
- `DropboxStorageAdapter` — Dropbox
260-
- `RcloneStorageAdapter` — Any rclone remote
253+
- `LocalStorageAdapter` — filesystem rooted at `STORAGE_PATH`
254+
- `S3StorageAdapter` — any S3-compatible object store (AWS S3, MinIO, Cloudflare R2, Garage, SeaweedFS, Ceph RGW, Wasabi, Backblaze B2, ...). Path-style addressing is enabled automatically when `STORAGE_ENDPOINT` is set.
261255

262-
Configured via `STORAGE_DRIVER` env var.
256+
> Previous releases also shipped GCS, Azure, Google Drive, OneDrive, Dropbox, FTP, and rclone-as-adapter backends. These were removed in favour of S3-compatible object storage (one code path, multipart semantics, vendor-neutral). rclone remains available as a separate optional backup utility (`scripts/sync-storage.ts` + `/api/admin/sync`).
257+
258+
Configured via `STORAGE_TYPE` env var (`local` or `s3`).
263259

264260
### 6.5 Database Flexibility
265261
`src/db/index.ts` provides a factory pattern:
@@ -328,7 +324,7 @@ och focus # Terminal UI for PRs/queue/reviews
328324
| `RUNNER_SECRET` | Runner-server shared secret |
329325
| `AI_CONFIG_ENCRYPTION_KEY` | AI provider key encryption |
330326
| `WORKFLOW_SECRET_ENCRYPTION_KEY` | CI secret encryption |
331-
| `STORAGE_DRIVER` | Blob storage backend |
327+
| `STORAGE_TYPE` | Blob storage backend (`local` or `s3`) |
332328
| `REDIS_URL` | Redis for sessions/queues |
333329
| `GIT_REPOS_PATH` | Local git repo storage |
334330
| `GIT_SSH_PORT` | SSH git server port |

bun.lock

Lines changed: 15 additions & 177 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs-site/src/content/docs/administration/configuration.md

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -33,13 +33,18 @@ OpenCodeHub is configured exclusively via environment variables.
3333

3434
| Variable | Description | Default |
3535
|----------|-------------|---------|
36-
| `STORAGE_TYPE` | `local`, `s3`, `gdrive`, `azure` | `local` |
37-
| `STORAGE_PATH` | Path for local storage | `./data` |
38-
| `STORAGE_BUCKET` | Bucket name (S3/GCP) | |
39-
| `STORAGE_REGION` | AWS Region | `us-east-1` |
40-
| `STORAGE_ENDPOINT` | Custom S3 endpoint | |
41-
| `S3_ACCESS_KEY` | AWS Access Key | |
42-
| `S3_SECRET_KEY` | AWS Secret Key | |
36+
| `STORAGE_TYPE` | `local` (filesystem) or `s3` (any S3-compatible object store) | `local` |
37+
| `STORAGE_PATH` | Path for local storage | `./data/storage` |
38+
| `STORAGE_BUCKET` | Bucket name (required for `s3`) | |
39+
| `STORAGE_REGION` | S3 region | `us-east-1` |
40+
| `STORAGE_ENDPOINT` | Custom S3 endpoint (MinIO, R2, Garage, SeaweedFS, Ceph RGW, etc.) | |
41+
| `STORAGE_ACCESS_KEY_ID` | S3 access key | |
42+
| `STORAGE_SECRET_ACCESS_KEY` | S3 secret key | |
43+
44+
> Only `local` and `s3` storage backends are supported. Previous releases
45+
> also supported `gdrive`, `azure`, `gcs`, `dropbox`, `onedrive`, `ftp`, and
46+
> an rclone-as-adapter option; these have been removed in favour of
47+
> S3-compatible object storage. See `docs/guides/storage-adapters.md`.
4348
4449
## AI Review
4550

docs-site/src/content/docs/administration/deploy-cloudflare.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -130,8 +130,8 @@ STORAGE_TYPE=s3
130130
STORAGE_BUCKET=opencodehub-repos
131131
STORAGE_REGION=auto
132132
STORAGE_ENDPOINT=https://<account-id>.r2.cloudflarestorage.com
133-
S3_ACCESS_KEY=<your-r2-access-key-id>
134-
S3_SECRET_KEY=<your-r2-secret-access-key>
133+
STORAGE_ACCESS_KEY_ID=<your-r2-access-key-id>
134+
STORAGE_SECRET_ACCESS_KEY=<your-r2-secret-access-key>
135135
```
136136

137137
Find your account ID in Cloudflare Dashboard → R2 → Overview.
@@ -298,8 +298,8 @@ STORAGE_TYPE=s3
298298
STORAGE_BUCKET=opencodehub-repos
299299
STORAGE_REGION=auto
300300
STORAGE_ENDPOINT=https://<account-id>.r2.cloudflarestorage.com
301-
S3_ACCESS_KEY=<r2-access-key>
302-
S3_SECRET_KEY=<r2-secret-key>
301+
STORAGE_ACCESS_KEY_ID=<r2-access-key>
302+
STORAGE_SECRET_ACCESS_KEY=<r2-secret-key>
303303
```
304304

305305
---

docs-site/src/content/docs/administration/deploy-cpanel.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -153,7 +153,7 @@ SITE_URL=https://git.yourdomain.com
153153
NODE_ENV=production
154154

155155
# Storage (local is fine for single-server cPanel)
156-
STORAGE_DRIVER=local
156+
STORAGE_TYPE=local
157157
STORAGE_PATH=./data/storage
158158
REPOS_PATH=./data/repos
159159

docs-site/src/content/docs/administration/deploy-cyberpanel.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -143,7 +143,7 @@ SITE_URL=https://git.yourdomain.com
143143
NODE_ENV=production
144144

145145
# Storage
146-
STORAGE_DRIVER=local
146+
STORAGE_TYPE=local
147147
STORAGE_PATH=/data/storage
148148
REPOS_PATH=/data/repos
149149
SSH_HOST_KEY_PATH=/data/ssh/host_key

docs-site/src/content/docs/administration/deployment.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -90,18 +90,21 @@ DATABASE_DRIVER=postgres
9090
DATABASE_URL=postgresql://user:password@host:5432/opencodehub
9191
```
9292

93-
**Storage (Use cloud storage, not local):**
93+
**Storage (S3-compatible; works with AWS S3, MinIO, Cloudflare R2, Garage, SeaweedFS, Ceph RGW, etc.):**
9494
```bash
9595
STORAGE_TYPE=s3
9696
STORAGE_BUCKET=opencodehub-production
9797
STORAGE_REGION=us-east-1
98-
S3_ACCESS_KEY=<your-access-key>
99-
S3_ACCESS_KEY=<your-access-key>
100-
S3_SECRET_KEY=<your-secret-key>
98+
STORAGE_ACCESS_KEY_ID=<your-access-key>
99+
STORAGE_SECRET_ACCESS_KEY=<your-secret-key>
100+
# STORAGE_ENDPOINT=https://<account>.r2.cloudflarestorage.com # set for non-AWS vendors
101101
```
102102

103-
**Google Drive + Turso Stack:**
104-
See [docs/GDRIVE_STACK.md](docs/GDRIVE_STACK.md) for detailed setup.
103+
> Only `local` and `s3` storage backends are supported. Google Drive,
104+
> OneDrive, Dropbox, FTP, GCS, Azure, and rclone-as-adapter have been
105+
> removed; use the `s3` driver with a vendor-specific `STORAGE_ENDPOINT`
106+
> for object storage. See `docs/guides/storage-adapters.md` for the
107+
> full list of compatible object stores.
105108
```bash
106109
STORAGE_TYPE=gdrive
107110
GOOGLE_CLIENT_ID=...

0 commit comments

Comments
 (0)