ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Using Rclone from a server to backup multiple volumes from different SSH hosts

    Scheduled Pinned Locked Moved Solved IT Discussion
    rcloneb2backupautomationusers
    20 Posts 4 Posters 1.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      marcinozga @DustinB3403
      last edited by

      @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

      @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

      @DustinB3403 Install rclone on each workstation and setup cron jobs on each machine. Brew and Ansible make this trivial. Do you really need Fedora Server in this setup?

      This I considered, but would involve tinkering with each workstation.

      That's why I mentioned Ansible. Your tinkering is limited to workstation that runs playbooks.

      DustinB3403D 1 Reply Last reply Reply Quote 0
      • DustinB3403D
        DustinB3403 @marcinozga
        last edited by

        @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

        @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

        @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

        @DustinB3403 Install rclone on each workstation and setup cron jobs on each machine. Brew and Ansible make this trivial. Do you really need Fedora Server in this setup?

        This I considered, but would involve tinkering with each workstation.

        That's why I mentioned Ansible. Your tinkering is limited to workstation that runs playbooks.

        That's not at all accurate, I would have to install rclone on every workstation, have custom cron jobs on every workstation and then would have to validate that every workstation sync'd.

        M 1 Reply Last reply Reply Quote 0
        • DustinB3403D
          DustinB3403
          last edited by

          In addition to that, I would have to enable cron on every apple workstation, which while doable, is a pain in the rear should I ever need to make any changes to the backup operation.

          Centrally managing this makes way more sense rather than the ad hoc approach.

          M 1 Reply Last reply Reply Quote 0
          • DustinB3403D
            DustinB3403
            last edited by

            One option I considered is just installing Cyberduck on the workstations and having the user do the backup.

            Of course that just won't be done and thus is off the table.

            1 Reply Last reply Reply Quote 0
            • M
              marcinozga @DustinB3403
              last edited by marcinozga

              @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

              @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

              @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

              @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

              @DustinB3403 Install rclone on each workstation and setup cron jobs on each machine. Brew and Ansible make this trivial. Do you really need Fedora Server in this setup?

              This I considered, but would involve tinkering with each workstation.

              That's why I mentioned Ansible. Your tinkering is limited to workstation that runs playbooks.

              That's not at all accurate, I would have to install rclone on every workstation, have custom cron jobs on every workstation and then would have to validate that every workstation sync'd.

              And all that you can do with Ansible.
              Install rclone? https://docs.ansible.com/ansible/latest/modules/homebrew_module.html#homebrew-module
              Setup cron? https://docs.ansible.com/ansible/latest/modules/cron_module.html#cron-module

              1 Reply Last reply Reply Quote 0
              • M
                marcinozga @DustinB3403
                last edited by marcinozga

                @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                In addition to that, I would have to enable cron on every apple workstation, which while doable, is a pain in the rear should I ever need to make any changes to the backup operation.

                Centrally managing this makes way more sense rather than the ad hoc approach.

                And since when has cron been disabled in macOS?

                DustinB3403D 1 Reply Last reply Reply Quote 0
                • DustinB3403D
                  DustinB3403 @marcinozga
                  last edited by

                  @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                  @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                  In addition to that, I would have to enable cron on every apple workstation, which while doable, is a pain in the rear should I ever need to make any changes to the backup operation.

                  Centrally managing this makes way more sense rather than the ad hoc approach.

                  And since when has cron been disabled in macOS?

                  For quite some time, it's still there, but you have to enable it, grant it root access etc.

                  M 1 Reply Last reply Reply Quote 0
                  • M
                    marcinozga @DustinB3403
                    last edited by

                    @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                    @marcinozga said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                    @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                    In addition to that, I would have to enable cron on every apple workstation, which while doable, is a pain in the rear should I ever need to make any changes to the backup operation.

                    Centrally managing this makes way more sense rather than the ad hoc approach.

                    And since when has cron been disabled in macOS?

                    For quite some time, it's still there, but you have to enable it, grant it root access etc.

                    That's simply not true. You don't have to enable it. And you don't have to give it root access unless you're writing somewhere you shouldn't be in first place.

                    1 Reply Last reply Reply Quote 0
                    • DustinB3403D
                      DustinB3403
                      last edited by

                      Anyways I figured this out.

                      Having multiple rclone sources like
                      SKtlxrr.png

                      and running

                      b9469d45-8af9-4407-b185-7105a79a7557-image.png

                      Works

                      dafyreD 1 Reply Last reply Reply Quote 2
                      • dafyreD
                        dafyre @DustinB3403
                        last edited by

                        @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                        Anyways I figured this out.

                        Having multiple rclone sources like
                        SKtlxrr.png

                        and running

                        b9469d45-8af9-4407-b185-7105a79a7557-image.png

                        Works

                        The /Volumes/<path to external> would change for each system, wouldn't it?

                        Could you back up to b2-rclone:/<username>/external ?

                        DustinB3403D 1 Reply Last reply Reply Quote 0
                        • DustinB3403D
                          DustinB3403 @dafyre
                          last edited by

                          @dafyre said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                          @DustinB3403 said in Using Rclone from a server to backup multiple volumes from different SSH hosts:

                          Anyways I figured this out.

                          Having multiple rclone sources like
                          SKtlxrr.png

                          and running

                          b9469d45-8af9-4407-b185-7105a79a7557-image.png

                          Works

                          The /Volumes/<path to external> would change for each system, wouldn't it?

                          Could you back up to b2-rclone:/<username>/external ?

                          Possibly, but what I was thinking I would do is simply connect to the administrative user on each of these workstations, hit the volume that way. The Volume name may change (as spaces make everything more difficult). But yeah I think this'll work.

                          1 Reply Last reply Reply Quote 0
                          • DustinB3403D
                            DustinB3403
                            last edited by

                            To clarify for anyone else in the future, you need to have multiple apple-# sftp connections.

                            The b2 connections could also be custom as you can create multiple buckets and simply connect to that bucket via rclone.

                            1 Reply Last reply Reply Quote 0
                            • DustinB3403D
                              DustinB3403
                              last edited by

                              This is what the b2-rclone remote setting looks like for my needs, obviously this could be adjusted with the advanced configuration menu.

                              sj858rr.png

                              And this is the Apple workstation

                              jGytpzZ.png

                              1 Reply Last reply Reply Quote 0
                              • DustinB3403D
                                DustinB3403
                                last edited by

                                Ideally when this is all said and done, what I think would make the most sense is to create multiple buckets for each of these apple workstations.

                                So apple-bucket-1, apple-bucket-2 and so on.

                                From there, simply have cron run the sync commands for each of these and have it run hourly.

                                1 Reply Last reply Reply Quote 0
                                • DustinB3403D
                                  DustinB3403
                                  last edited by DustinB3403

                                  Worth noting on this that while sync works, copy is the much safer option.

                                  So

                                  rclone copy source:/path dest:/path is what I'll likely end up using. In any case, the resulting solution still works.

                                  Still need to tweak OSX a bit as it seems to kill the SSH session after some time for sporadic reasons - still testing but I may end up having to add ServerAliveInterval 120 to /etc/ssh/ssh_config to resolve that issue.

                                  1 Reply Last reply Reply Quote 0
                                  • 1 / 1
                                  • First post
                                    Last post