Should backup and virtualization infrastructure be decoupled?
-
I think the IT sector is going beyond the enthusiasm wave about vm-based backup, it's not a novelty anymore, in any sense.
I was a veeam-on-vmware fan and I still use it, but I have some thought to share with the community… do we really need this kind o backups for modern workloads?
- VM-based backup comes from a world of deeply stateful, snowflakes machine. The world is moving towards stateless software with even more clear separation between data and applications.
- In my experience, full-VM block-based backups takes longer to run (think about and ERP where only the DB change), are more resource intensive and takes more space.
- Every VM-based system is tied to the specific virtualization platform, so migrate between different environment becomes harder VS agent-based backup system where restore on another platform is trivial.
- VM-based software tends to be more costly and complex than the agent-based counterpart.
- Backups are usually stored in proprietary formats that cannot be read easily; many non-VM backup software just compress the extracted data into common format.
I'm trying to rebuild one of my customer backup infrastructure without leverage any VM-based backup suite, implementing both regular full snapshot export of the VM (golden master templates used to rebuild the VM) and tradition backup methods.
Also, I've the feeling that VM-based backups are more geared towards legacy workloads that cannot adopt a modern design like those of the modern cloud-based, microserver or serverless application, so I think I'll start to use more and more devops-style or virtual-agnostic backup tools just to remain in touch with the latest trends in industry.
What do you think about it?
-
I like the idea!
What are you proposing, can you give a little more detail? -
I have two server based applications at this time, Exchange and Business Works. My other servers are AD, WSUS, file/print.
I don't see myself not backing up any of these whole machines other than the WSUS.
I'm sure I could build a salt/ansible script and to rebuild the Print/file server easy enough (sounds like a good project) so I could move to a pure data only backup in my the file server.
-
Agent and Agentless backup solutions have always had a place. Agent based backups are great for active database systems, so they can stop any write process to the database, take a snap and then continue.
The issue with agent based backups is someone needs to manage every agent. This costs time (and money).
Agentless on the other hand means only the hypervisor (or backup appliance control center) needs to be managed, and reduces the time to manage the backups.
There are pluses and minuses to each which need to be considered carefully when making a decision like this.
-
@Francesco-Provino said in Should backup and virtualization infrastructure be decoupled?:
- VM-based backup comes from a world of deeply stateful, snowflakes machine. The world is moving towards stateless software with even more clear separation between data and applications.
This is the huge piece. In a world of load balanced systems that are identical and ephemeral and small, isolated bits of dynamic data, old "grab everything and sort it out later" backups are slow and inefficient. It's a great technology, but is it a great way to design your systems? In most cases, I would say no.
-
Sorry to crash in, but If i am backing up raw file and folders data, using rsycn or robocopy and mysql database, from inside the VM and throwing it to NAS each day, what is this considered Agent or agentless ?
I dont care much about the VM itself, however I do sometimes backup it (monthly encase of big update, but uts getting harder each time it is growing)using Vmware export function, or backing up the qcow2 file encase it is KVM. again what is this called Manual Backups ? I automate much of them via scripts ...
-
@DustinB3403 said in Should backup and virtualization infrastructure be decoupled?:
Agent and Agentless backup solutions have always had a place. Agent based backups are great for active database systems, so they can stop any write process to the database, take a snap and then continue.
Agentless can do that too.
-
@FATeknollogee said in Should backup and virtualization infrastructure be decoupled?:
I like the idea!
What are you proposing, can you give a little more detail?Same thing that I keep talking about.... what I call "DevOps style backups". Identify your stateful data, back only it up. It's how I handle most systems.
Like if you have a traditional web server (LAMP) you don't need to back up the server, only a monthly backup of the /var/www directory and a daily database backup. Neither requires special tools or anything. Both generally make tiny files. Your 16GB server backup is easily 50MB. Compress that before or after transfer. Transfer to a backup system and/or directly to a cloud host.
-
@scottalanmiller said in Should backup and virtualization infrastructure be decoupled?:
@DustinB3403 said in Should backup and virtualization infrastructure be decoupled?:
Agent and Agentless backup solutions have always had a place. Agent based backups are great for active database systems, so they can stop any write process to the database, take a snap and then continue.
Agentless can do that too.
True, I didn't say they couldn't though either. . .
-
@scottalanmiller said in Should backup and virtualization infrastructure be decoupled?:
@FATeknollogee said in Should backup and virtualization infrastructure be decoupled?:
I like the idea!
What are you proposing, can you give a little more detail?Same thing that I keep talking about.... what I call "DevOps style backups". Identify your stateful data, back only it up. It's how I handle most systems.
Like if you have a traditional web server (LAMP) you don't need to back up the server, only a monthly backup of the /var/www directory and a daily database backup. Neither requires special tools or anything. Both generally make tiny files. Your 16GB server backup is easily 50MB. Compress that before or after transfer. Transfer to a backup system and/or directly to a cloud host.
Ok, what if I have a Windows 20xx SQL vm with 500gb worth of data or a VOIP vm with 15gb of data, how would I handle this "DevOps" style?
-
@FATeknollogee said in Should backup and virtualization infrastructure be decoupled?:
@scottalanmiller said in Should backup and virtualization infrastructure be decoupled?:
@FATeknollogee said in Should backup and virtualization infrastructure be decoupled?:
I like the idea!
What are you proposing, can you give a little more detail?Same thing that I keep talking about.... what I call "DevOps style backups". Identify your stateful data, back only it up. It's how I handle most systems.
Like if you have a traditional web server (LAMP) you don't need to back up the server, only a monthly backup of the /var/www directory and a daily database backup. Neither requires special tools or anything. Both generally make tiny files. Your 16GB server backup is easily 50MB. Compress that before or after transfer. Transfer to a backup system and/or directly to a cloud host.
Ok, what if I have a Windows 20xx SQL vm with 500gb worth of data or a VOIP vm with 15gb of data, how would I handle this "DevOps" style?
Same, nothing really changes with size. It's still "Just one thing to back up".
-
Voicemails are easy, rsync really rules there.
-
@scottalanmiller said in Should backup and virtualization infrastructure be decoupled?:
@FATeknollogee said in Should backup and virtualization infrastructure be decoupled?:
@scottalanmiller said in Should backup and virtualization infrastructure be decoupled?:
@FATeknollogee said in Should backup and virtualization infrastructure be decoupled?:
I like the idea!
What are you proposing, can you give a little more detail?Same thing that I keep talking about.... what I call "DevOps style backups". Identify your stateful data, back only it up. It's how I handle most systems.
Like if you have a traditional web server (LAMP) you don't need to back up the server, only a monthly backup of the /var/www directory and a daily database backup. Neither requires special tools or anything. Both generally make tiny files. Your 16GB server backup is easily 50MB. Compress that before or after transfer. Transfer to a backup system and/or directly to a cloud host.
Ok, what if I have a Windows 20xx SQL vm with 500gb worth of data or a VOIP vm with 15gb of data, how would I handle this "DevOps" style?
Same, nothing really changes with size. It's still "Just one thing to back up".
I assume you just use a process to backup the DB and call it done. Then you have a script the builds a new VM as needed, retire data and done.
-
@scottalanmiller said in Should backup and virtualization infrastructure be decoupled?:
@FATeknollogee said in Should backup and virtualization infrastructure be decoupled?:
I like the idea!
What are you proposing, can you give a little more detail?Same thing that I keep talking about.... what I call "DevOps style backups". Identify your stateful data, back only it up. It's how I handle most systems.
Like if you have a traditional web server (LAMP) you don't need to back up the server, only a monthly backup of the /var/www directory and a daily database backup. Neither requires special tools or anything. Both generally make tiny files. Your 16GB server backup is easily 50MB. Compress that before or after transfer. Transfer to a backup system and/or directly to a cloud host.
Another thing that helps here is keeping data on a backing store that you can mount. Web server data can be mounted from a share and only that system needs backed up. Then the web server itself can be built/destroyed at will and has no affect on the data itself.
Treating whole VMs as containers gives a lot of flexibility. It's obviously not as efficient with resources but it really lessens the burden of administration.
-
Yup, and database clusters so that you only restore data, rather than capacity ,should the entire cluster fail, not just a node.
-
I've been backing up VMs at the host level for a while.
More recently, also testing RCT based backups through SCDPM MBS (2016):
https://mangolassi.it/topic/13232/scdpm-2016-using-mbs
It's backing up almost 8TB worth of VMs in an hour or so. Fully browseable and recoverable to a lot of options.
-
@Francesco-Provino another thing. I have vm managed by a supplier. With multi tenant a clear separation of duties must be defined.
In our case the local sys admin is in charge to keep everything updated. In this scenatio hypervisor level backup helps not entering the vm to backup it. Surely less efficient especially with db which have their own backup procedures. -
@fateknollogee said in Should backup and virtualization infrastructure be decoupled?:
I like the idea!
What are you proposing, can you give a little more detail?Use bacula/borgbackup/veeam endpoint or just plain remote rsnpshot, plus database-specific tools… and, of course, take a full backup if the VM once in a while for quick disaster recovery, but without all of the fancy incremental-dedupe stuff.
