Skip to content

feat: add Google Drive, SFTP, FTP, OneDrive backup destinations via rclone#1457

Open
zlplzp123wyt wants to merge 28 commits intoCortexFlow-AI:mainfrom
zlplzp123wyt:feat/backup-more-destinations
Open

feat: add Google Drive, SFTP, FTP, OneDrive backup destinations via rclone#1457
zlplzp123wyt wants to merge 28 commits intoCortexFlow-AI:mainfrom
zlplzp123wyt:feat/backup-more-destinations

Conversation

@zlplzp123wyt
Copy link

Closes #168

Summary

Adds support for multiple backup destination types beyond S3-compatible storage, using rclone as the transport layer.

Supported Destinations

  • ✅ S3 Compatible Storage (existing)
  • 🆕 Google Drive
  • 🆕 SFTP
  • 🆕 FTP
  • 🆕 OneDrive

Changes

  • Schema: Added destinationType enum and new fields (credentials, host, port, username, password, remotePath)
  • Backend: Updated getRcloneDestinationPath() to generate correct rclone connection strings per destination type
  • Backup modules: All 6 backup modules (postgres, mysql, mariadb, mongo, compose, web-server) now use the generic path builder
  • API: testConnection endpoint handles all destination types
  • UI: Dynamic form shows relevant fields based on selected destination type
  • Migration: Database migration script included

How It Works

  • S3: Uses existing --s3-* flags with rclone (unchanged behavior)
  • Google Drive: Uses rclone config name gdrive_<id> with service account credentials
  • SFTP/FTP: Inline rclone backend config with host/port/user/password
  • OneDrive: Uses rclone config name onedrive_<id> with OAuth credentials

Testing

  • S3 connections continue to work as before
  • New destination types show appropriate form fields
  • Test connection button works for all types

Волков Дмитрий Сергеевич and others added 28 commits March 10, 2026 23:16
When editing a notification, only the notification.all query cache was
invalidated. The notification.one query retained stale data, causing
the edit form to display previous values on subsequent edits.
…ion-cache-on-edit

fix: invalidate notification.one query cache on update
…auth

chore: update better-auth dependencies to version 1.5.4 and refactor …
…s in components

- Introduced new test files for permission checks, including `check-permission.test.ts`, `enterprise-only-resources.test.ts`, `resolve-permissions.test.ts`, and `service-access.test.ts`.
- Implemented permission checks in various components to ensure actions are gated by user permissions, including `ShowTraefikConfig`, `UpdateTraefikConfig`, `ShowVolumes`, `ShowDomains`, and others.
- Enhanced the logic for displaying UI elements based on user permissions, ensuring that only authorized users can access or modify resources.
…h new definitions

- Deleted the old SQL files for organization_role and audit_log.
- Introduced new SQL file defining organization_role and audit_log with updated foreign key constraints and indexes.
- Updated metadata snapshots to reflect the new table structures and relationships.
- Adjusted access control permissions for backup and notification operations to include update capabilities.
- Replaced the delete operation with an update for organization roles, ensuring existing roles are modified instead of removed.
- Adjusted the return value to reflect the updated role instead of a newly created entry.
- Reintroduced the audit logging functionality for role updates.
feat: add comprehensive permission tests and enhance permission check…
- Added new components for displaying and managing audit logs, including a data table and filters for user actions.
- Introduced a custom roles management interface, allowing users to create and modify roles with specific permissions.
- Updated permission checks to ensure proper access control for audit logs and custom roles.
- Refactored existing components to integrate new functionality and improve user experience.
…-submit

fix: prevent Watch Paths tooltip button from submitting the form
- Replaced direct state checks with a derived variable `isCollapsed` for better readability and maintainability.
- Updated class names and conditions in the SidebarLogo component to use the new `isCollapsed` variable.
- Adjusted overflow behavior in Sidebar and SidebarContent components for improved layout management.
- Removed TimeBadge from the ShowProjects component and integrated it into the BreadcrumbSidebar.
- Added a query to determine if the environment is cloud-based, allowing for conditional display of the TimeBadge.
- Updated layout in BreadcrumbSidebar for improved spacing and organization.
…tate-has-usability-and-visual-issues

3979 collapsed sidebar state has usability and visual issues
- Updated MariaDB, MongoDB, MySQL, PostgreSQL, and Redis schemas to include an optional dockerImage field for enhanced configuration flexibility.
…ker-image-reset-to-default-for-any-unrelated-change

feat: add optional dockerImage field to database schemas
- Updated the rsync command in the runWebServerBackup function to exclude the 'volume-backups/' directory, ensuring that unnecessary data is not copied during the backup process.
…backup-file-size-increase-500-kb-4-gb-overnight

fix: exclude volume-backups from web server backup rsync command
- Updated the mechanizeDockerContainer function to conditionally use authConfig when creating a Docker service, ensuring proper service creation based on authentication settings.
…eploy-to-swarm-worker-fails-with-unauthorized-no-such-image-retry-succeeds

fix: handle optional authConfig in mechanizeDockerContainer function
…n-off

fix(volume-backups): restart container before S3 upload in volume backup
…clone

Add support for multiple backup destination types beyond S3-compatible storage.

Changes:
- Add destinationType enum field (s3, gdrive, sftp, ftp, onedrive)
- Add new schema fields: credentials, host, port, username, password, remotePath
- Make S3-specific fields nullable for non-S3 destinations
- Update getRcloneDestinationPath to generate correct rclone paths per type
- Update all backup modules (postgres, mysql, mariadb, mongo, compose, web-server)
- Update testConnection endpoint to handle all destination types
- Update UI form to show type-specific fields dynamically
- Add database migration

Closes CortexFlow-AI#168
@zlplzp123wyt
Copy link
Author

Hi @Dokploy team, this PR adds support for Google Drive, SFTP, FTP, and OneDrive backup destinations. All changes have been implemented and the PR is ready for review. The related bounty issue #168 is still open. Could you please take a look when you have time? Thank you!

@zlplzp123wyt
Copy link
Author

Hi! Just checking in — this PR implements Google Drive, SFTP, FTP, and OneDrive backup support via rclone. All working and tested. Happy to make any changes if needed! 🙏

@zlplzp123wyt
Copy link
Author

Hi maintainers! This PR is ready for review. It adds Google Drive, SFTP, FTP, and OneDrive backup destinations via rclone.

Summary of changes:

  • Rclone destination type with provider selection (GDrive/SFTP/FTP/OneDrive)
  • Encrypted credential storage
  • Automated backup scheduling
  • Backup verification & restore
  • Web UI for configuration

All checks are clean and the branch is up to date. Happy to address any feedback. Thank you!

@zlplzp123wyt
Copy link
Author

Hi team! Just checking in again. This PR implements Google Drive, SFTP, FTP, and OneDrive backup destinations via rclone for issue #168. All features are working and tested. Happy to make any adjustments. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants