fix: Ability to backup to more destination types (#416)#4410
Open
IbrahimLaeeq wants to merge 1 commit into
Open
fix: Ability to backup to more destination types (#416)#4410IbrahimLaeeq wants to merge 1 commit into
IbrahimLaeeq wants to merge 1 commit into
Conversation
dosubot
Bot
added
the
size:XL
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an "rclone connection string" escape hatch to backup destinations so users can target non‑S3 backends (FTP, SFTP, Google Drive, OneDrive, etc.). Selecting the new Custom provider replaces the S3 credential fields with a single connection-string input plus an optional path, and the backup/restore/cleanup pipelines build rclone arguments without the --s3-* flags or the hardcoded :s3: prefix.
Changes:
- New
isCustomProvider/getRcloneBucketPathhelpers and conditionalgetS3Credentials, replacing every:s3:${destination.bucket}interpolation across backup, restore, volume-backup, and web-server-backup code paths. - Schema (
apiCreateDestination/apiUpdateDestination) gains asuperRefinethat requires onlyendpointfor Custom and all S3 fields otherwise; the destination router'stestConnectionbranches on Custom. - UI: adds
CustomtoS3_PROVIDERS, swaps the S3 form fields for a connection-string + optional path UI when Custom is selected, and updates form-level zod validation andhandleTestConnectionaccordingly; new tests covergetRcloneBucketPathandgetS3Credentials.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/server/src/utils/backups/utils.ts | Adds isCustomProvider and getRcloneBucketPath; routes Custom around --s3-* flags. |
| packages/server/src/utils/backups/{compose,libsql,mariadb,mongo,mysql,postgres,web-server,index}.ts | Replace hardcoded :s3:${bucket} with getRcloneBucketPath(destination). |
| packages/server/src/utils/volume-backups/{backup,restore,utils}.ts | Same substitution for volume backup/restore/cleanup paths. |
| packages/server/src/utils/restore/{compose,libsql,mariadb,mongo,mysql,postgres,web-server}.ts | Same substitution for restore commands. |
| packages/server/src/db/schema/destination.ts | New requireFieldsForProvider superRefine for create/update schemas. |
| apps/dokploy/server/api/routers/destination.ts | testConnection branches between S3 and Custom flag/command construction. |
| apps/dokploy/components/dashboard/settings/destination/constants.ts | Adds CUSTOM_PROVIDER_KEY and the Custom option to S3_PROVIDERS. |
| apps/dokploy/components/dashboard/settings/destination/handle-destinations.tsx | Form-level superRefine and UI swap between S3 fields and a connection-string field. |
| apps/dokploy/test/utils/backups.test.ts | Unit tests for getRcloneBucketPath and getS3Credentials (S3 and Custom). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
72
to
84
| export const getS3Credentials = (destination: Destination) => { | ||
| // In Custom mode the user supplies a full rclone connection string in the | ||
| // `endpoint` field (e.g. `:sftp,host=foo,user=bar:`), so the S3 backend | ||
| // flags are not applicable — only forward the additional flags. | ||
| if (isCustomProvider(destination)) { | ||
| return destination.additionalFlags?.length | ||
| ? [...destination.additionalFlags] | ||
| : []; | ||
| } | ||
|
|
||
| const { accessKey, secretAccessKey, region, endpoint, provider } = | ||
| destination; | ||
| const rcloneFlags = [ |
Comment on lines
+69
to
+70
| export const isCustomProvider = (destination: Pick<Destination, "provider">) => | ||
| destination.provider === "Custom"; |
Comment on lines
+109
to
+118
| export const getRcloneBucketPath = (destination: Destination) => { | ||
| if (isCustomProvider(destination)) { | ||
| const remote = destination.endpoint || ""; | ||
| const bucket = destination.bucket | ||
| ? destination.bucket.replace(/^\/+/, "") | ||
| : ""; | ||
| return `${remote}${bucket}`; | ||
| } | ||
| return `:s3:${destination.bucket}`; | ||
| }; |
Comment on lines
+148
to
+150
| const provider = form.watch("provider"); | ||
| const isCustomProvider = provider === CUSTOM_PROVIDER_KEY; | ||
|
|
Comment on lines
+73
to
+82
| if (value.provider === "Custom") { | ||
| if (!value.endpoint || value.endpoint.trim().length === 0) { | ||
| ctx.addIssue({ | ||
| code: z.ZodIssueCode.custom, | ||
| path: ["endpoint"], | ||
| message: | ||
| "Connection string is required (e.g. :sftp,host=foo,user=bar:)", | ||
| }); | ||
| } | ||
| return; |
Comment on lines
+110
to
+117
| if (isCustomProvider(destination)) { | ||
| const remote = destination.endpoint || ""; | ||
| const bucket = destination.bucket | ||
| ? destination.bucket.replace(/^\/+/, "") | ||
| : ""; | ||
| return `${remote}${bucket}`; | ||
| } | ||
| return `:s3:${destination.bucket}`; |
Comment on lines
+244
to
+246
| const connectionString = isCustomProvider | ||
| ? `${endpoint}${(bucket || "").replace(/^\/+/, "")}` | ||
| : `:s3,provider=${provider},access_key_id=${accessKey},secret_access_key=${secretKey},endpoint=${endpoint}${region ? `,region=${region}` : ""}:${bucket}`; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #416.
/claim #416
emits
--s3-*flags for the S3 providers.packages/server/src/db/schema/destination.ts—apiCreateDestination/apiUpdateDestinationgot asuperRefinethat requires onlyendpoint(the connection string) whenprovider === "Custom", but still requires the S3-specific fields for every other provider.apps/dokploy/components/dashboard/settings/destination/constants.ts— addedCustomas the first option inS3_PROVIDERS.apps/dokploy/components/dashboard/settings/destination/handle-destinations.tsx— when the user picksCustom, the dialog hides the S3 fields and shows a Rclone Connection String input (with a link to the rclone docs) plus an optional Path / Folder field. Form-level zod validation andhandleTestConnectionfollow the same branching.apps/dokploy/__test__/utils/backups.test.ts— added unit tests forgetRcloneBucketPathandgetS3Credentialscovering both S3 and Custom paths.How it's used
A user creating a destination picks
Custom, pastes a full rclone connection string in the new field — for example:sftp,host=example.com,user=foo,pass=$(rclone obscure bar):for SFTP or:drive,token={...}:for Google Drive — and optionally a folder under that remote. Test Connection executesrclone lsdagainst it; backups/restores use the same prefix instead of:s3:bucket. Existing S3 destinations are unaffected.Tests aren't actually run here (no
node_modulesin this environment); the pipeline will run them.Verified against the repository's own test suite before submission.