BackUp device for local or colo storage
-
My two bits:
Definitely ditch the full backup process and go with a continuous incremental process. StorageCraft will do continuous incremental backups of your virtual or physical systems as frequently as every 15 minutes. These are byte/sector level files so they're small and efficient.
Just to run the math (*assuming you have 24TB of data to back up) here are three quick examples:
-
Create a weekly full backup. This produces 24TB x 4 weeks = 96TB of backup files. Even with good compression you're still looking at a lot of storage and network traffic when replicating these offsite.
-
Initial full and then weekly incrementals backups every Saturday. Let's assume a constant change rate of around 20%/month to keep this simple which means that every weekly incremental would be about 5% x 24TB in size. The first month would be 24TB (base full) + 1.2TB x 3 weeks or 27.6 TB. Every subsequent month would only be 4 x 1.2TB or 4.8TB of storage. Compression would further reduce the storage requirements.
-
Now for the really slick option... since this is a continuous rate of change we can increase the recovery points (capture incremental files more frequently) without affecting storage too much. For example, each 15 minute incremental file would be approximately 24TB x (.05/week) x (1 week / 7 days) x (1 day / 24 hours) x (4 backups / hour) = about 28.5GB every 15 minutes. The advantage here is that you have about the same amount of data your storing as in option #2 but you have granular recovery points every 15 minutes of every day in the week. So you can select a very specific point in time to recover.
Obviously, this math is over-simplified and you should benchmark your own numbers. But even with a simplified model it should be obvious that periodic full backups are much more storage intensive than incremental backups. And a continuous incremental scheme can produce a powerful granular recovery through the amount of recovery points generated.
The only reason I see people do full backups is because their backup process rolls up these continuous incremental files into a synthetic full which means that if corruption gets into just one of my recovery points my synthetic full is now corrupt. Essentially the periodic full backup is their way of re-basing a backup chain to keep out corruption.
(This became longer than I expected... maybe I should've made the value bigger than "2 bits")
Cheers!
-
-
@scottalanmiller earlier was mentioning differentials vs incrementals.
I wonder if the terms still apply, or if the industry at large has dropped the two terms and simply moved to the use of incrementals.
Most systems that I've seen, Unitrends, Veeam, AppAssure allow for the use of continuous incrementals. You take a snapshot at some point as a full backup, then create incremental backups on the desired schedule. The system is then able to create a synthetic full backup from the last full and various incrementals along the way.
In the old days you'd have to do a full restore from the full backup, then apply the most recent differential, or all of the incrementals since the last full. The new way makes it all a single step.Furthermore, some system can take a backup and export to storage somewhere as a VHD ready to be mounted as a VM where ever you need it.
-
With StorageCraft - does anyone know what the dedup levels are? Does it even support it? Will it dedup across all backups, or only within a single server's worth of backups?
-
@Dashrender said:
@scottalanmiller earlier was mentioning differentials vs incrementals.
I wonder if the terms still apply, or if the industry at large has dropped the two terms and simply moved to the use of incrementals.
No, in no way whatsoever has this happened nor could it.
-
@Dashrender said:
In the old days you'd have to do a full restore from the full backup, then apply the most recent differential, or all of the incrementals since the last full. The new way makes it all a single step.
Still do. You are comparing different things and mixing them together.
-
@Dashrender said:
Most systems that I've seen, Unitrends, Veeam, AppAssure allow for the use of continuous incrementals. You take a snapshot at some point as a full backup, then create incremental backups on the desired schedule. The system is then able to create a synthetic full backup from the last full and various incrementals along the way.
Incremental Forever, as you are picturing, is incrementals, not differentials for obvious reasons. The terms are still used very precisely as they have always been used. Differential Forever would be a disaster.
Incremental Forever is only possible on systems were there is a single storage media for everything - fulls and all incrementals in one place. When you get to large, enterprise backups no one talks about Incremental Forever because you can't consider that with tape or with offline, archival storage. Think about how disastrous Incremental Forever would be when you go to offline storage.
-
@scottalanmiller said:
@Dashrender said:
Most systems that I've seen, Unitrends, Veeam, AppAssure allow for the use of continuous incrementals. You take a snapshot at some point as a full backup, then create incremental backups on the desired schedule. The system is then able to create a synthetic full backup from the last full and various incrementals along the way.
Incremental Forever, as you are picturing, is incrementals, not differentials for obvious reasons. The terms are still used very precisely as they have always been used. Differential Forever would be a disaster.
Incremental Forever is only possible on systems were there is a single storage media for everything - fulls and all incrementals in one place. When you get to large, enterprise backups no one talks about Incremental Forever because you can't consider that with tape or with offline, archival storage. Think about how disastrous Incremental Forever would be when you go to offline storage.
lol - why does it need to be. You pick a point in time, spin off a full backup from that time to the archive/tape/whatever and you're done.
This is where synthetic fulls come into play. Unless I just don't understand the actual use.
-
@Dashrender said:
This is where synthetic fulls come into play. Unless I just don't understand the actual use.
Synthetic Fulls are still fulls, but riskier as they can clone corruption. If you do a synthetic full with tape, you still have a full, just one that isn't as reliable.
-
OK so @scottalanmiller was able to explain StorageCraft backup and restore to me in much more clear terms.
Create a new VM, boot from SC ISO media, and import the files.
That's simple.
Why didn't anyone explain that to me in such simple terms before?!
-
@DustinB3403 said:
OK so @scottalanmiller was able to explain StorageCraft backup and restore to me in much more clear terms.
Create a new VM, boot from SC ISO media, and import the files.
That's simple.
Why didn't anyone explain that to me in such simple terms before?!
I suppose my explanation was more verbose than that. But the ideas are identical.
-
sorry @Dashrender it just wasn't making sense earlier today.
-
@Dashrender said:
@DustinB3403 said:
OK so @scottalanmiller was able to explain StorageCraft backup and restore to me in much more clear terms.
Create a new VM, boot from SC ISO media, and import the files.
That's simple.
Why didn't anyone explain that to me in such simple terms before?!
I suppose my explanation was more verbose than that. But the ideas are identical.
I was more verbose. It was just offline.
-
@DustinB3403 said:
sorry @Dashrender it just wasn't making sense earlier today.
it's all good - I completely understand needing something explained 2-infinity different ways before you understand something.
-
@Steven with me being unfamiliar with the specifics of what SC version we have I have a few questions. (sorry, just not something I've dug into at work)
Is there any documentation that I can review regarding SC and XS working together?
Does every version of SC, support VM backup and restore?
Are there data limits to SC?
Is there anyway to see a demo on StorageCraft restoring a XenServer VM? -
@DustinB3403 said:
Is there any documentation that I can review regarding SC and XS working together?
Conceptually this doesn't make sense. SC does not talk to XS. There is no "working together." SC is virtualization agnostic. It is agent based from inside the OS.
-
@DustinB3403 said:
Does every version of SC, support VM backup and restore?
No version does, it does not talk to VMs or deal with them in any way. This is traditional backup from inside of the OS.
-
@DustinB3403 said:
Is there anyway to see a demo on StorageCraft restoring a XenServer VM?
There is nothing to see. All restores are identical as they happen above the platform. This would be similar to "seeing a VM running on XS." It looks just like any other OS, there is nothing to see.
-
OK to clarify...
Is there a demo that I can watch, of a VM being restored with SC ISO and the import of the SC backup files.
-
For example, you could boot a bare metal server off the cd and pull the restore to that server, the same as pulling it to a VM.
-
@DustinB3403 said:
OK to clarify...
Is there a demo that I can watch, of a VM being restored with SC ISO and the import of the SC backup files.
Why a VM? VM or physical, the OS is the same.
