Can Linux Run Incremental Backups?
-
@thanksaj said:
From what I could see, it never appears to run new masters, but is just continuous differentials. If this is incrementals, it's not as bad, although you should run new masters every now and then.
Other way around. If you take one master and then only ever take something else, that something else needs to be a differential. You want no more than two components needed for a restore. In this example, that would be the master and the latest differential.
To restore an incremental would mean that you need the master and every single incremental backup taken since the last master. If you do that over a period of a week, that is annoying and a little risk. But most companies are willing to consider that risk. If you are doing that over months or years, that's an impossible situation. You can't need to fetch hundreds of backups AND trust that every single one of them is flawless. That would be effectively impossible to restore.
-
@thanksaj said:
For example, Unitrends can't run incrementals on a Linux system. Unitrends is able to run incrementals on Windows (and VMware but not going there atm but it's for a similar reason) because of VSS tracking changes on the system. AFAIK, Linux does not have anything like VSS that tracks changes, thus differentials are the only option.
VSS is a copy of a Linux feature. LVM and BtrFS are two core means of doing system images like VSS. But you don't need either on Linux. Linux was doing differential and incremental backups a decade before VSS existed and without LVM.
Symantec BackupExec and Netbackup both do incremental on all UNIX.
-
@thanksaj said:
All backup solutions, as a rule, seem to have the same abilities and limitations, due to the OSes.
I'm not aware of any differences between them. What examples do you have?
-
@scottalanmiller said:
@thanksaj said:
From what I could see, it never appears to run new masters, but is just continuous differentials. If this is incrementals, it's not as bad, although you should run new masters every now and then.
Other way around. If you take one master and then only ever take something else, that something else needs to be a differential. You want no more than two components needed for a restore. In this example, that would be the master and the latest differential.
To restore an incremental would mean that you need the master and every single incremental backup taken since the last master. If you do that over a period of a week, that is annoying and a little risk. But most companies are willing to consider that risk. If you are doing that over months or years, that's an impossible situation. You can't need to fetch hundreds of backups AND trust that every single one of them is flawless. That would be effectively impossible to restore.
While I agree that doing it that way makes sense for the reason of data loss, for efficiency of the backups, it's INCREDIBLY inefficient. Running a new full/master at least, oh, once a month, would make sense. Then run your differentials off that.
-
@thanksaj said:
For example, Unitrends can't run incrementals on a Linux system.
It sure can. It would be pretty crazy if it didn't.
http://support.unitrends.com/ikm/questions.php?questionid=494
-
@scottalanmiller said:
@thanksaj said:
All backup solutions, as a rule, seem to have the same abilities and limitations, due to the OSes.
I'm not aware of any differences between them. What examples do you have?
I can't give brand names off the top of my head, but, perfect example, I've never seen a system that can run Incremental Forever on a Linux system. However, all backup solutions that I've seen that have that option CAN and DO run it on Windows systems, as one example. I haven't been an installer for 7 months now, but when I was, everything I was told was that incrementals were not even possible on Linux. I never worked with any Unix systems when I was doing installs, so I can't comment on that. But if you create a backup schedule on a Untirends device of a Linux system, Fulls w/ Incrementals is not even an option.
-
@scottalanmiller said:
@thanksaj said:
For example, Unitrends can't run incrementals on a Linux system.
It sure can. It would be pretty crazy if it didn't.
http://support.unitrends.com/ikm/questions.php?questionid=494
I've never seen it on any of the Linux systems I touched.
-
@thanksaj said:
While I agree that doing it that way makes sense for the reason of data loss, for efficiency of the backups, it's INCREDIBLY inefficient. Running a new full/master at least, oh, once a month, would make sense. Then run your differentials off that.
It's really not inefficient most of the time. Not sure why you think so. Doing that with incremental is only an option if all of your backups exist on a single, high reliability array (like a Unitrends appliance doing Incremental Forever.) In normal GFS tape backup rotation, doing any amount of incrementals is absolutely insane. It's no different than not having backups at all.
What about doing regular diffs makes you uncomfortable? That IS the industry standard practice, you understand. Not that "everyone does it and that makes it right." Of course some industry practices are poor. But I am not aware of there being any dispute on the differential approach being the most widely applicable concept.
-
@thanksaj said:
I've never seen it on any of the Linux systems I touched.
Why would the above statement ever lead you to believe that it didn't support it?
-
@thanksaj said:
I can't give brand names off the top of my head, but, perfect example, I've never seen a system that can run Incremental Forever on a Linux system.
You are using a single product example, that we already established is incorrect, as the only example as to why something else is true? Do you see the failed chain of reasoning here?
My car can't make it out of the garage. Therefore trucks don't drive on roads.
-
@scottalanmiller said:
@thanksaj said:
While I agree that doing it that way makes sense for the reason of data loss, for efficiency of the backups, it's INCREDIBLY inefficient. Running a new full/master at least, oh, once a month, would make sense. Then run your differentials off that.
It's really not inefficient most of the time. Not sure why you think so. Doing that with incremental is only an option if all of your backups exist on a single, high reliability array (like a Unitrends appliance doing Incremental Forever.) In normal GFS tape backup rotation, doing any amount of incrementals is absolutely insane. It's no different than not having backups at all.
What about doing regular diffs makes you uncomfortable? That IS the industry standard practice, you understand. Not that "everyone does it and that makes it right." Of course some industry practices are poor. But I am not aware of there being any dispute on the differential approach being the most widely applicable concept.
Incrementals are much smaller and faster. Differentials tend to be much larger backups. I get why you say what you do, but it all comes down to how much change happens in what span of time, and your backup method. Using tape or have few changes overall? Diffs make sense. Have a large amount of changes (like a large file server) or are using disk? Incrementals are more efficient.
-
@scottalanmiller said:
@thanksaj said:
I've never seen it on any of the Linux systems I touched.
Why would the above statement ever lead you to believe that it didn't support it?
That's also what I was always told.
-
@thanksaj said:
@scottalanmiller said:
@thanksaj said:
I've never seen it on any of the Linux systems I touched.
Why would the above statement ever lead you to believe that it didn't support it?
That's also what I was always told.
Or is it what you always believed?
-
@scottalanmiller said:
@thanksaj said:
I can't give brand names off the top of my head, but, perfect example, I've never seen a system that can run Incremental Forever on a Linux system.
You are using a single product example, that we already established is incorrect, as the only example as to why something else is true? Do you see the failed chain of reasoning here?
My car can't make it out of the garage. Therefore trucks don't drive on roads.
I can't think of the specific name but I've seen the same thing on other backup appliances I've looked at.
-
@IRJ said:
@thanksaj said:
@scottalanmiller said:
@thanksaj said:
I've never seen it on any of the Linux systems I touched.
Why would the above statement ever lead you to believe that it didn't support it?
That's also what I was always told.
Or is it what you always believed?
It's what I was told. I don't remember who though.
-
@thanksaj said:
I haven't been an installer for 7 months now, but when I was, everything I was told was that incrementals were not even possible on Linux.
Unitrends own documents from far more than 17 months ago, let alone 7, mention how Linux is the ONLY platform on which incremental forever is available. All other platforms, like Windows, only offer it on certain versions. Linux isn't just supported, it is the most supported.
-
@scottalanmiller said:
@thanksaj said:
I haven't been an installer for 7 months now, but when I was, everything I was told was that incrementals were not even possible on Linux.
Unitrends own documents from far more than 17 months ago, let alone 7, mention how Linux is the ONLY platform on which incremental forever is available. All other platforms, like Windows, only offer it on certain versions. Linux isn't just supported, it is the most supported.
I was misinformed then.
-
@thanksaj said:
I can't think of the specific name but I've seen the same thing on other backup appliances I've looked at.
I think that that is safe to assume was a misconception, as it was on Unitrends. Every enterprise backup appliance had that feature in the 1990s. Hard to believe anyone would be lacking it today.
-
@thanksaj said:
Incrementals are much smaller and faster. Differentials tend to be much larger backups. I get why you say what you do, but it all comes down to how much change happens in what span of time, and your backup method. Using tape or have few changes overall? Diffs make sense. Have a large amount of changes (like a large file server) or are using disk? Incrementals are more efficient.
Incrementals are typically trivially smaller. It is quite rare for a differential to be much larger than an incremental over an appropriate period of time (say a week.)
Incrementals are always equal or better for taking backups. But are very risky for doing restores. Remember the first rule of backups - no one cares about backups, they only care about restores. Incrementals are about making backups a little easier in exchange for a lot of restore risk. Differentials are about making backups a little harder in exchange for a lot of restore protection.
Most SMBs should just do fulls all the time and be done with it.
-
@scottalanmiller said:
Most SMBs should just do fulls all the time and be done with it.
This was always my choice assuming the backups would complete in the backup window.
