Anyone backing up a file server with 13 million plus files?
-
@dashrender said in Anyone backing up a file server with 13 million plus files?:
@scottalanmiller said in Anyone backing up a file server with 13 million plus files?:
@tim_g said in Anyone backing up a file server with 13 million plus files?:
@dustinb3403 said in Anyone backing up a file server with 13 million plus files?:
@tim_g said in Anyone backing up a file server with 13 million plus files?:
@dustinb3403 said in Anyone backing up a file server with 13 million plus files?:
What hypervisor? 7TB and 13 Million files doesn't seem that excessive for pretty much any solution today.
It's a physical server, with an attached MD1000 for on-prem backup.
Gotcha. . . anything that can do block level, while skipping reading every file is what you're looking for then. While also offering the ability to perform per file recovery (I'm assuming).
Good point.
I haven't paid much attention to that fact, but which backup softwares do it like that?
Scale HC3's built in backup will do that without looking at the files
Can it do individual file restores?
https://mangolassi.it/topic/16110/restoring-files-and-folders-out-of-scale-hc3-vm-snapshots
-
Veeam likely would work well here, but I'm not sure if it actually skips reading and creating a file index though. . .
I honestly don't know of any solution that would do this. . . agent-wise.
-
I'm going to test it with the free version just to see how it handles it.
If it works out, we'll buy it. $200 is nothing at this point... and if it proves itself, it may help with going all Veeam later (hopefully).
-
Veeam test is on-going.
I started it over an hour ago, and so far, it's over a week ahead of where the other backup software would be...
It's going to back it all up into a .vbk file. So after it's complete, I'll see what can be done with it tape-wise, file restore wise, etc.
I'm not exactly sure "how" it's doing the backup... but I did see from Resmon it read the Master File Table. Much more efficient than scanning every damn file...
-
@tim_g if you are testing the freeware and not a trial, mind that it doesnt do CBT only the paid server edition does.
-
@matteo-nunziati said in Anyone backing up a file server with 13 million plus files?:
@tim_g if you are testing the freeware and not a trial, mind that it doesnt do CBT only the paid server edition does.
Good to know.
It is the free version, which I chose just to do a quick test. I didn't feel like being bothered by sales (yet).
-
On a slightly relavant note... I noticed they updated their Fedora agent to Fedora 26. Perhaps we'll see 27 soon!
-
@tim_g said in Anyone backing up a file server with 13 million plus files?:
@matteo-nunziati said in Anyone backing up a file server with 13 million plus files?:
@tim_g if you are testing the freeware and not a trial, mind that it doesnt do CBT only the paid server edition does.
Good to know.
It is the free version, which I chose just to do a quick test. I didn't feel like being bothered by sales (yet).
On this note (and partly due to this topic and on-going projects) I discovered that UrBackup has CBT for only $17/system.
I'm not sure how well it would scale with this many files though. Something I'm still digging into.
-
@dustinb3403 said in Anyone backing up a file server with 13 million plus files?:
@tim_g said in Anyone backing up a file server with 13 million plus files?:
@matteo-nunziati said in Anyone backing up a file server with 13 million plus files?:
@tim_g if you are testing the freeware and not a trial, mind that it doesnt do CBT only the paid server edition does.
Good to know.
It is the free version, which I chose just to do a quick test. I didn't feel like being bothered by sales (yet).
On this note (and partly due to this topic and on-going projects) I discovered that UrBackup has CBT for only $17/system.
I'm not sure how well it would scale with this many files though. Something I'm still digging into.
Does UrBackup use VSS and is it capable of backing up live VMs?
-
@tim_g said in Anyone backing up a file server with 13 million plus files?:
@dustinb3403 said in Anyone backing up a file server with 13 million plus files?:
@tim_g said in Anyone backing up a file server with 13 million plus files?:
@matteo-nunziati said in Anyone backing up a file server with 13 million plus files?:
@tim_g if you are testing the freeware and not a trial, mind that it doesnt do CBT only the paid server edition does.
Good to know.
It is the free version, which I chose just to do a quick test. I didn't feel like being bothered by sales (yet).
On this note (and partly due to this topic and on-going projects) I discovered that UrBackup has CBT for only $17/system.
I'm not sure how well it would scale with this many files though. Something I'm still digging into.
Does UrBackup use VSS and is it capable of backing up live VMs?
I'm not sure if it actively uses VSS or what the mechanism is, I do know it can backup VM's, files and shares.
I'm running a demo from v-box on my desktop, backing up just a specific folder from my desktop to UrBackup.
Only about 12Gb im testing with but it seems to work well.
-
-
I was looking at Zmanda a while back, looks like they added Windows Server 2016 to their compatibility matrix now.
-
Zmanda is just Amanda under the hood. I setup Amanda a long time ago, and it works well, but was a huge pain in the ass to get going with.
I'll blame it on way to in-depth documentation and a case of keyboard-tourettes by who ever was writing the documentation. . .
-
@dustinb3403 said in Anyone backing up a file server with 13 million plus files?:
Zmanda is just Amanda under the hood. I setup Amanda a long time ago, and it works well, but was a huge pain in the ass to get going with.
I'll blame it on way to in-depth documentation and a case of keyboard-tourettes by who ever was writing the documentation. . .
That's why I didn't go with Amanda... it doesn't natively work with VSS and such on Windows and Hyper-V VSS.
But with Zmanda, they have all those bells and whistles worked out, it seems. I haven't tried Zmanda though.
-
UrBackup on the other hand is so easy to get setup and going that it is borderline scary.
Setup server on linux, Windows or Mac, install agents on client systems.
Configure items to backup and then go drink . . . because the system is already working.
-
@dustinb3403 said in Anyone backing up a file server with 13 million plus files?:
UrBackup on the other hand is so easy to get setup and going that it is borderline scary.
Setup server on linux, Windows or Mac, install agents on client systems.
Configure items to backup and then go drink . . . because the system is already working.
That was my experience as well.
-
@tim_g said in Anyone backing up a file server with 13 million plus files?:
I was looking at Zmanda a while back, looks like they added Windows Server 2016 to their compatibility matrix now.
Zmanda is just the "supported" version of Amanda, but their support sucked. Specifically they said "That error doesn't matter". Only that error mattered, because no backup could be restored from after getting said error. The only thing to show them is the door.
-
@travisdh1 said in Anyone backing up a file server with 13 million plus files?:
@tim_g said in Anyone backing up a file server with 13 million plus files?:
I was looking at Zmanda a while back, looks like they added Windows Server 2016 to their compatibility matrix now.
Zmanda is just the "supported" version of Amanda, but their support sucked. Specifically they said "That error doesn't matter". Only that error mattered, because no backup could be restored from after getting said error. The only thing to show them is the door.
Almost forgot, I had a ticket open with them on this issue for 6 months with no action on it at all.
-
@travisdh1 said in Anyone backing up a file server with 13 million plus files?:
@travisdh1 said in Anyone backing up a file server with 13 million plus files?:
@tim_g said in Anyone backing up a file server with 13 million plus files?:
I was looking at Zmanda a while back, looks like they added Windows Server 2016 to their compatibility matrix now.
Zmanda is just the "supported" version of Amanda, but their support sucked. Specifically they said "That error doesn't matter". Only that error mattered, because no backup could be restored from after getting said error. The only thing to show them is the door.
Almost forgot, I had a ticket open with them on this issue for 6 months with no action on it at all.
I've not know anyone that has been happy with them in forever. Amanda was a neat idea in 2001 or whatever. It really has little to no place anymore.
-
@scottalanmiller said in Anyone backing up a file server with 13 million plus files?:
@travisdh1 said in Anyone backing up a file server with 13 million plus files?:
@travisdh1 said in Anyone backing up a file server with 13 million plus files?:
@tim_g said in Anyone backing up a file server with 13 million plus files?:
I was looking at Zmanda a while back, looks like they added Windows Server 2016 to their compatibility matrix now.
Zmanda is just the "supported" version of Amanda, but their support sucked. Specifically they said "That error doesn't matter". Only that error mattered, because no backup could be restored from after getting said error. The only thing to show them is the door.
Almost forgot, I had a ticket open with them on this issue for 6 months with no action on it at all.
I've not know anyone that has been happy with them in forever. Amanda was a neat idea in 2001 or whatever. It really has little to no place anymore.
I wonder if they've updated their agent support matrix since then? At least for the Linux distros they officially supported, it was horribly old.
