ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Anyone backing up a file server with 13 million plus files?

    IT Discussion
    11
    39
    2.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ObsolesceO
      Obsolesce @DustinB3403
      last edited by

      @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

      @tim_g said in Anyone backing up a file server with 13 million plus files?:

      @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

      What hypervisor? 7TB and 13 Million files doesn't seem that excessive for pretty much any solution today.

      It's a physical server, with an attached MD1000 for on-prem backup.

      Gotcha. . . anything that can do block level, while skipping reading every file is what you're looking for then. While also offering the ability to perform per file recovery (I'm assuming).

      Good point.

      I haven't paid much attention to that fact, but which backup softwares do it like that?

      scottalanmillerS 1 Reply Last reply Reply Quote 0
      • NashBrydgesN
        NashBrydges
        last edited by

        Veeam Windows Agent does block level backup but if it is being linked to an existing Veeam B&R Repository, there might be search indexes that are built during backup. I remember reading some best-practice documentation about their catalog but only related to VM backups. Not sure if you're poiting Veeam Windows Agent to a B&R or just to storage.

        1 Reply Last reply Reply Quote 1
        • scottalanmillerS
          scottalanmiller @Obsolesce
          last edited by

          @tim_g said in Anyone backing up a file server with 13 million plus files?:

          @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

          @tim_g said in Anyone backing up a file server with 13 million plus files?:

          @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

          What hypervisor? 7TB and 13 Million files doesn't seem that excessive for pretty much any solution today.

          It's a physical server, with an attached MD1000 for on-prem backup.

          Gotcha. . . anything that can do block level, while skipping reading every file is what you're looking for then. While also offering the ability to perform per file recovery (I'm assuming).

          Good point.

          I haven't paid much attention to that fact, but which backup softwares do it like that?

          Scale HC3's built in backup will do that without looking at the files 🙂

          DashrenderD ObsolesceO 2 Replies Last reply Reply Quote 1
          • DashrenderD
            Dashrender @scottalanmiller
            last edited by

            @scottalanmiller said in Anyone backing up a file server with 13 million plus files?:

            @tim_g said in Anyone backing up a file server with 13 million plus files?:

            @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

            @tim_g said in Anyone backing up a file server with 13 million plus files?:

            @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

            What hypervisor? 7TB and 13 Million files doesn't seem that excessive for pretty much any solution today.

            It's a physical server, with an attached MD1000 for on-prem backup.

            Gotcha. . . anything that can do block level, while skipping reading every file is what you're looking for then. While also offering the ability to perform per file recovery (I'm assuming).

            Good point.

            I haven't paid much attention to that fact, but which backup softwares do it like that?

            Scale HC3's built in backup will do that without looking at the files 🙂

            Can it do individual file restores?

            black3dynamiteB 1 Reply Last reply Reply Quote 0
            • ObsolesceO
              Obsolesce @scottalanmiller
              last edited by

              @scottalanmiller said in Anyone backing up a file server with 13 million plus files?:

              @tim_g said in Anyone backing up a file server with 13 million plus files?:

              @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

              @tim_g said in Anyone backing up a file server with 13 million plus files?:

              @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

              What hypervisor? 7TB and 13 Million files doesn't seem that excessive for pretty much any solution today.

              It's a physical server, with an attached MD1000 for on-prem backup.

              Gotcha. . . anything that can do block level, while skipping reading every file is what you're looking for then. While also offering the ability to perform per file recovery (I'm assuming).

              Good point.

              I haven't paid much attention to that fact, but which backup softwares do it like that?

              Scale HC3's built in backup will do that without looking at the files 🙂

              Sure, but a $200 Veeam Agent for Windows Server is much more likely than $30k + a migration LOL

              1 Reply Last reply Reply Quote 1
              • black3dynamiteB
                black3dynamite @Dashrender
                last edited by

                @dashrender said in Anyone backing up a file server with 13 million plus files?:

                @scottalanmiller said in Anyone backing up a file server with 13 million plus files?:

                @tim_g said in Anyone backing up a file server with 13 million plus files?:

                @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

                @tim_g said in Anyone backing up a file server with 13 million plus files?:

                @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

                What hypervisor? 7TB and 13 Million files doesn't seem that excessive for pretty much any solution today.

                It's a physical server, with an attached MD1000 for on-prem backup.

                Gotcha. . . anything that can do block level, while skipping reading every file is what you're looking for then. While also offering the ability to perform per file recovery (I'm assuming).

                Good point.

                I haven't paid much attention to that fact, but which backup softwares do it like that?

                Scale HC3's built in backup will do that without looking at the files 🙂

                Can it do individual file restores?

                https://mangolassi.it/topic/16110/restoring-files-and-folders-out-of-scale-hc3-vm-snapshots

                1 Reply Last reply Reply Quote 1
                • DustinB3403D
                  DustinB3403
                  last edited by DustinB3403

                  Veeam likely would work well here, but I'm not sure if it actually skips reading and creating a file index though. . .

                  I honestly don't know of any solution that would do this. . . agent-wise.

                  1 Reply Last reply Reply Quote 0
                  • ObsolesceO
                    Obsolesce
                    last edited by

                    I'm going to test it with the free version just to see how it handles it.

                    If it works out, we'll buy it. $200 is nothing at this point... and if it proves itself, it may help with going all Veeam later (hopefully).

                    1 Reply Last reply Reply Quote 1
                    • ObsolesceO
                      Obsolesce
                      last edited by Obsolesce

                      Veeam test is on-going.

                      I started it over an hour ago, and so far, it's over a week ahead of where the other backup software would be...

                      It's going to back it all up into a .vbk file. So after it's complete, I'll see what can be done with it tape-wise, file restore wise, etc.

                      I'm not exactly sure "how" it's doing the backup... but I did see from Resmon it read the Master File Table. Much more efficient than scanning every damn file...

                      0_1520359201511_c1297e21-967c-405c-ae6b-9c8dd5e661e1-image.png

                      matteo nunziatiM 1 Reply Last reply Reply Quote 3
                      • matteo nunziatiM
                        matteo nunziati @Obsolesce
                        last edited by

                        @tim_g if you are testing the freeware and not a trial, mind that it doesnt do CBT only the paid server edition does.

                        ObsolesceO 1 Reply Last reply Reply Quote 0
                        • ObsolesceO
                          Obsolesce @matteo nunziati
                          last edited by

                          @matteo-nunziati said in Anyone backing up a file server with 13 million plus files?:

                          @tim_g if you are testing the freeware and not a trial, mind that it doesnt do CBT only the paid server edition does.

                          Good to know.

                          It is the free version, which I chose just to do a quick test. I didn't feel like being bothered by sales (yet).

                          DustinB3403D 1 Reply Last reply Reply Quote 1
                          • ObsolesceO
                            Obsolesce
                            last edited by

                            On a slightly relavant note... I noticed they updated their Fedora agent to Fedora 26. Perhaps we'll see 27 soon!

                            1 Reply Last reply Reply Quote 1
                            • DustinB3403D
                              DustinB3403 @Obsolesce
                              last edited by

                              @tim_g said in Anyone backing up a file server with 13 million plus files?:

                              @matteo-nunziati said in Anyone backing up a file server with 13 million plus files?:

                              @tim_g if you are testing the freeware and not a trial, mind that it doesnt do CBT only the paid server edition does.

                              Good to know.

                              It is the free version, which I chose just to do a quick test. I didn't feel like being bothered by sales (yet).

                              On this note (and partly due to this topic and on-going projects) I discovered that UrBackup has CBT for only $17/system.

                              I'm not sure how well it would scale with this many files though. Something I'm still digging into.

                              ObsolesceO 1 Reply Last reply Reply Quote 0
                              • ObsolesceO
                                Obsolesce @DustinB3403
                                last edited by

                                @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

                                @tim_g said in Anyone backing up a file server with 13 million plus files?:

                                @matteo-nunziati said in Anyone backing up a file server with 13 million plus files?:

                                @tim_g if you are testing the freeware and not a trial, mind that it doesnt do CBT only the paid server edition does.

                                Good to know.

                                It is the free version, which I chose just to do a quick test. I didn't feel like being bothered by sales (yet).

                                On this note (and partly due to this topic and on-going projects) I discovered that UrBackup has CBT for only $17/system.

                                I'm not sure how well it would scale with this many files though. Something I'm still digging into.

                                Does UrBackup use VSS and is it capable of backing up live VMs?

                                DustinB3403D 1 Reply Last reply Reply Quote 0
                                • DustinB3403D
                                  DustinB3403 @Obsolesce
                                  last edited by

                                  @tim_g said in Anyone backing up a file server with 13 million plus files?:

                                  @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

                                  @tim_g said in Anyone backing up a file server with 13 million plus files?:

                                  @matteo-nunziati said in Anyone backing up a file server with 13 million plus files?:

                                  @tim_g if you are testing the freeware and not a trial, mind that it doesnt do CBT only the paid server edition does.

                                  Good to know.

                                  It is the free version, which I chose just to do a quick test. I didn't feel like being bothered by sales (yet).

                                  On this note (and partly due to this topic and on-going projects) I discovered that UrBackup has CBT for only $17/system.

                                  I'm not sure how well it would scale with this many files though. Something I'm still digging into.

                                  Does UrBackup use VSS and is it capable of backing up live VMs?

                                  I'm not sure if it actively uses VSS or what the mechanism is, I do know it can backup VM's, files and shares.

                                  I'm running a demo from v-box on my desktop, backing up just a specific folder from my desktop to UrBackup.

                                  Only about 12Gb im testing with but it seems to work well.

                                  1 Reply Last reply Reply Quote 0
                                  • DustinB3403D
                                    DustinB3403
                                    last edited by

                                    Here is a PDF version of the administration guide. This is their web version.

                                    1 Reply Last reply Reply Quote 0
                                    • ObsolesceO
                                      Obsolesce
                                      last edited by

                                      I was looking at Zmanda a while back, looks like they added Windows Server 2016 to their compatibility matrix now.

                                      travisdh1T 1 Reply Last reply Reply Quote 1
                                      • DustinB3403D
                                        DustinB3403
                                        last edited by

                                        Zmanda is just Amanda under the hood. I setup Amanda a long time ago, and it works well, but was a huge pain in the ass to get going with.

                                        I'll blame it on way to in-depth documentation and a case of keyboard-tourettes by who ever was writing the documentation. . .

                                        ObsolesceO 1 Reply Last reply Reply Quote 1
                                        • ObsolesceO
                                          Obsolesce @DustinB3403
                                          last edited by

                                          @dustinb3403 said in Anyone backing up a file server with 13 million plus files?:

                                          Zmanda is just Amanda under the hood. I setup Amanda a long time ago, and it works well, but was a huge pain in the ass to get going with.

                                          I'll blame it on way to in-depth documentation and a case of keyboard-tourettes by who ever was writing the documentation. . .

                                          That's why I didn't go with Amanda... it doesn't natively work with VSS and such on Windows and Hyper-V VSS.

                                          But with Zmanda, they have all those bells and whistles worked out, it seems. I haven't tried Zmanda though.

                                          1 Reply Last reply Reply Quote 0
                                          • DustinB3403D
                                            DustinB3403
                                            last edited by

                                            UrBackup on the other hand is so easy to get setup and going that it is borderline scary.

                                            Setup server on linux, Windows or Mac, install agents on client systems.

                                            Configure items to backup and then go drink . . . because the system is already working.

                                            dafyreD 1 Reply Last reply Reply Quote 1
                                            • 1
                                            • 2
                                            • 2 / 2
                                            • First post
                                              Last post