Agent and Agentless Backups

[scottalanmiller]

@dustinb3403 said in Agent and Agentless Backups:

At the last job I was actually tempted to setup not only XOCE but also UrBackup as a means of having constantly created backups from my server because I didn't have a great way of performing Continuous Replication as quickly as I personally wanted.

And UrBackup, as an example, was trivially easy to deploy as agents, correct? Did you attempt any restores, was it super easy, too? Why look at it, if agentless has so much more to offer?

[DustinB3403]

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

@scottalanmiller said in Agent and Agentless Backups:

I don't know a single shop that I've worked with in years now that had an environment where agentless could be used reliably in that way.

But that is you, in your limited experience there, with clients that have opt'd for bad options. Either the agentless systems at the time just sucked, or literally did not have these kinds of features.

You can't go and lump in everything today as "oh it's bad because it's agentless".

Well take your environment for example. Guaranteed agentless can't do it all alone without other backup mechanisms doing the heavy lifting. guaranteed.

But agentless did, 100% no issues. So there is a case that agentless worked, without a hitch, met every requirement that the business had, and met the business needs.

[scottalanmiller]

Ridiculous anecdote, but we recently implemented an agent based system in minutes to keep us protected while waiting for support of an agentless system to get it working again. Means nothing, but a useful example that the assumptions that agentless makes it easy and agent makes it hard are just assumptions. In the real world, either can be easy or hard.

[DustinB3403]

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

At the last job I was actually tempted to setup not only XOCE but also UrBackup as a means of having constantly created backups from my server because I didn't have a great way of performing Continuous Replication as quickly as I personally wanted.

And UrBackup, as an example, was trivially easy to deploy as agents, correct? Did you attempt any restores, was it super easy, too? Why look at it, if agentless has so much more to offer?

I looked at it personally (and have already stated this in the previous post) was because the hypervisors we had (and likely network) could only produce Continous Replications every 15 minutes.

This was "good enough" from a business perspective.

Adding UrBackup on top of that, meant I would need double the backup space available, to protect for a possible 15 minute down time. Which the employees and work isn't so valuable that it justified the spend for that much more storage.

[scottalanmiller]

@dustinb3403 said in Agent and Agentless Backups:

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

@scottalanmiller said in Agent and Agentless Backups:

I don't know a single shop that I've worked with in years now that had an environment where agentless could be used reliably in that way.

But that is you, in your limited experience there, with clients that have opt'd for bad options. Either the agentless systems at the time just sucked, or literally did not have these kinds of features.

You can't go and lump in everything today as "oh it's bad because it's agentless".

Well take your environment for example. Guaranteed agentless can't do it all alone without other backup mechanisms doing the heavy lifting. guaranteed.

But agentless did, 100% no issues. So there is a case that agentless worked, without a hitch, met every requirement that the business had, and met the business needs.

Are you sure? Or did the workloads just not get evaluated? This is my point, @CCWTech and I just had a meeting with a firm that used agentless and said exactly what you said, but we were able to show them that the one thing that they cared about most wasn't properly protected and that the belief that agentless would "just cover it" had put them in a dangerous position. Unstable databases, the only thing tha tthey were bothering to pay for the backup for in the first place.

My point is, there is no reasonable way that you have all workloads that agentless can handle on its own (or agent based, I'm sure), but if you used agents, likely you'd have considered the workload needs but when doing agentless, it's become the norm to ignore the stability issues.

[momurda]

Using an agent is like using the HyperV role on top of Windows Server.

[scottalanmiller]

@dustinb3403 said in Agent and Agentless Backups:

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

At the last job I was actually tempted to setup not only XOCE but also UrBackup as a means of having constantly created backups from my server because I didn't have a great way of performing Continuous Replication as quickly as I personally wanted.

And UrBackup, as an example, was trivially easy to deploy as agents, correct? Did you attempt any restores, was it super easy, too? Why look at it, if agentless has so much more to offer?

I looked at it personally (and have already stated this in the previous post) was because the hypervisors we had (and likely network) could only produce Continous Replications every 15 minutes.

This was "good enough" from a business perspective.

Adding UrBackup on top of that, meant I would need double the backup space available, to protect for a possible 15 minute down time. Which the employees and work isn't so valuable that it justified the spend for that much more storage.

In that case, why not only use the UrBackup? Seems like the agentless system isn't doing as much as an agent based would do? I feel like you made the case that, in that situation, agents were more robust. And politics, not business need, led to agentless.

[scottalanmiller]

@momurda said in Agent and Agentless Backups:

Using an agent is like using the HyperV role on top of Windows Server.

It's really not. But this is exactly why my article above is important. It's become a solid believe that agents are bad, and agentless is good. It's stopped being an evaluation of needs and protection and just a "this is how it is done because it's the popular new thing." Agents remain very important, and very powerful. Agent based is a useful tool, but more limited and because of market pressures, tends to be very misleading.

[DustinB3403]

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

@scottalanmiller said in Agent and Agentless Backups:

I don't know a single shop that I've worked with in years now that had an environment where agentless could be used reliably in that way.

But that is you, in your limited experience there, with clients that have opt'd for bad options. Either the agentless systems at the time just sucked, or literally did not have these kinds of features.

You can't go and lump in everything today as "oh it's bad because it's agentless".

Well take your environment for example. Guaranteed agentless can't do it all alone without other backup mechanisms doing the heavy lifting. guaranteed.

But agentless did, 100% no issues. So there is a case that agentless worked, without a hitch, met every requirement that the business had, and met the business needs.

Are you sure? Or did the workloads just not get evaluated? This is my point, @CCWTech and I just had a meeting with a firm that used agentless and said exactly what you said, but we were able to show them that the one thing that they cared about most wasn't properly protected and that the belief that agentless would "just cover it" had put them in a dangerous position. Unstable databases, the only thing tha tthey were bothering to pay for the backup for in the first place.

My point is, there is no reasonable way that you have all workloads that agentless can handle on its own (or agent based, I'm sure), but if you used agents, likely you'd have considered the workload needs but when doing agentless, it's become the norm to ignore the stability issues.

Again, that is a customer who fell into a sales trap and didn't do their own job, or pay for a proper consult. You're lumping shitty customer decisions into a conversation about the merits of two different approaches and stating that anything that uses one approach is "risky" without evaluating the other options.

As much as Olivier is price breaking the XCP-NG world with his pricing models the solution and XOCE work just fine for 99% of the cases that take the time to consider it.

I know for a fact you haven't actually gone and tested XCP-NG or XOCE due to Oliviers business practices, but it is a solid solution as a whole when considered in context of this conversation.

Ignoring who makes it or what it is under the hood.

[DustinB3403]

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

At the last job I was actually tempted to setup not only XOCE but also UrBackup as a means of having constantly created backups from my server because I didn't have a great way of performing Continuous Replication as quickly as I personally wanted.

And UrBackup, as an example, was trivially easy to deploy as agents, correct? Did you attempt any restores, was it super easy, too? Why look at it, if agentless has so much more to offer?

I looked at it personally (and have already stated this in the previous post) was because the hypervisors we had (and likely network) could only produce Continous Replications every 15 minutes.

This was "good enough" from a business perspective.

Adding UrBackup on top of that, meant I would need double the backup space available, to protect for a possible 15 minute down time. Which the employees and work isn't so valuable that it justified the spend for that much more storage.

In that case, why not only use the UrBackup? Seems like the agentless system isn't doing as much as an agent based would do? I feel like you made the case that, in that situation, agents were more robust. And politics, not business need, led to agentless.

Because the agent based backup meant that to recover the entire guest, I would have to mount a special ISO, it wasn't nearly as straightforward as "restore this backup from 15 minutes ago to Host 2".

This was the approach of "we want the receptionist to be able to do this should you be on vacation" type of decision. So simple that you could screw up by simply being a moron. Granted only the IT department had the access required to make these kinds of changes, an even there it was with "least-access".

I was evaluating it only to offset the risk of "opps I deleted that file 8 minutes ago, can you restore it" to which the business decided to tell the employee, no you aren't that valuable.

The coverage of that 15 minute window, simply wasn't worth the added cost, not when considering the cost of added storage. It wasn't a question of "Agent does it better" as it didn't do it better in the big case of "we're afraid this VM might die because of sunk-cost decisions years ago".

[bnrstnr]

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

And a wonderful kicker to it is that I was even able to mount my agentless backups as a disks in my VM and restore individual files.

Or the entire VM in a matter of minutes, be it AD or the file server.

That's a HORRIBLE way to deal with file restores. But agentless is better than that. Agentless has no such limitations. If it did, that would be the big killer right there.

A little late to the party... but for the record, you definitely do NOT have to mount a snapshot and do all that manual stuff. You can do it straight from the XOCE web interface

https://i.imgur.com/FmHmIDT.png

[DustinB3403]

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

@scottalanmiller said in Agent and Agentless Backups:

I don't know a single shop that I've worked with in years now that had an environment where agentless could be used reliably in that way.

But that is you, in your limited experience there, with clients that have opt'd for bad options. Either the agentless systems at the time just sucked, or literally did not have these kinds of features.

You can't go and lump in everything today as "oh it's bad because it's agentless".

Well take your environment for example. Guaranteed agentless can't do it all alone without other backup mechanisms doing the heavy lifting. guaranteed.

Also, I have no idea what you're talking about here, agentless DID do everything that we needed, guaranteed.

[Emad R]

@scottalanmiller

Finally a clear and simple explanation of agent and agentless backup modes.

Also I do believe in the emerging devops/stateless way of backups regardless of the company size as it depends on the person, paired with golden image and your set, i dont know why people deem this way as mission impossible, and even without golden image, centos 7 installs on servers in like 5 mins ? then you can install salt-minion and apply state from the master. I know this does not apply on every server role but hay it is newer way of thinking and can work with people that document their work, and testing the backup is very easy on test VM.

[dbeato]

@scottalanmiller said in Agent and Agentless Backups:

Ah, Datto has added agentless now, but it was doing this before adding that.

Agentless backup for Datto specifically is only for VMware environments not Hyperv.

[stacksofplates]

As much as I like to argue with @scottalanmiller I have to agree that usually agents are easier.

Because the agent based backup meant that to recover the entire guest, I would have to mount a special ISO, it wasn't nearly as straightforward as "restore this backup from 15 minutes ago to Host 2".

That is in no way a requirement of agent based systems. This sounds more like there isn't a reliable way to reproduce a system and the data isn't on a separate volume. Make sure the systems can be rebuilt immediately and you can just reattach the data and be done.

Even with something as simple as ReaR, you define your skeleton volumes you want backed up and include that in your template.

Example (I'll use Ansible since that's what I know):

The template would have this in it:

BACKUP_PROG_INCLUDE=('{{ backup_dirs | join("' '") }}')

You'd have a list (in this case called backup_dirs) that gets iterated over:

backup_dirs:
  - /home/*
  - /data/*
  - /var/www/html/*

That backup_dirs list is specific to each machine when it's created.

The agent based stuff is really simple because it can very easily be specific for each machine and always be specific when the systems are built without any work after the initial set up.

[stacksofplates]

And again, this is only for systems that have data actually stored in them. Stuff like DNS servers can immediately be rebuilt with just definitions like this:

{% for key, value in records.iteritems() %}
{{ key }}   {{ value.type }}   {{ dns_subnet }}.{{ value.last }}
{% endfor %}

and the records dictionary looks like this:

dns_subnet: 192.168.0

records:
  router: { type: A, last: 1 }
  hypervisor1: { type: A, last: 2 }
  hypervisor2: { type: A, last: 3 }
  fileserver: { type: A, last: 4 }

This is specific to BIND (and you'd probably use Unbound anyway, but the logic is still the same) but it gives you awesome flexibility. So now I can run 1000 replicas if I want all with the same data and all changed at the same time, and my SVN system stores the data.

[scottalanmiller]

@dustinb3403 said in Agent and Agentless Backups:

@scottalanmiller said in Agent and Agentless Backups:

@dustinb3403 said in Agent and Agentless Backups:

@scottalanmiller said in Agent and Agentless Backups:

I don't know a single shop that I've worked with in years now that had an environment where agentless could be used reliably in that way.

But that is you, in your limited experience there, with clients that have opt'd for bad options. Either the agentless systems at the time just sucked, or literally did not have these kinds of features.

You can't go and lump in everything today as "oh it's bad because it's agentless".

Well take your environment for example. Guaranteed agentless can't do it all alone without other backup mechanisms doing the heavy lifting. guaranteed.

Also, I have no idea what you're talking about here, agentless DID do everything that we needed, guaranteed.

How do you know? You are stating the exact attitude that I'm warning about. Unless you've done a deep investigation into your workloads and their ability to be supported by this type of backup, you can't say this. Everyone who gets this wrong states it just like this, this is exactly what we went through with the client and it turned out that they had no reliable backups and had never researched what was needed for their workloads.

Are you saying you have zero workloads that aren't handled by your agentless system? How tiny is the company? Even QuickBooks isn't supported. I've never heard of any company of any size that could just use them. Things like MySQL are not supported without scripting an agent (even if the native one) to handle the reliability piece.

It's just not reasonable to think that the company has an IT staff and only workloads that agentless can handle. That's rare to the point of being a theoretical niche.

This would require things like MS SQL Server to be the only data store in the organization. Typically its' the only one supported. I know of no ERP that can be done agentless, no distributed database, etc.

[Obsolesce]

@stacksofplates said in Agent and Agentless Backups:

As much as I like to argue with @scottalanmiller I have to agree that usually agents are easier.

Because the agent based backup meant that to recover the entire guest, I would have to mount a special ISO, it wasn't nearly as straightforward as "restore this backup from 15 minutes ago to Host 2".

That is in no way a requirement of agent based systems. This sounds more like there isn't a reliable way to reproduce a system and the data isn't on a separate volume. Make sure the systems can be rebuilt immediately and you can just reattach the data and be done.

Even with something as simple as ReaR, you define your skeleton volumes you want backed up and include that in your template.

Example (I'll use Ansible since that's what I know):

The template would have this in it:

BACKUP_PROG_INCLUDE=('{{ backup_dirs | join("' '") }}')

You'd have a list (in this case called backup_dirs) that gets iterated over:

backup_dirs:
  - /home/*
  - /data/*
  - /var/www/html/*

That backup_dirs list is specific to each machine when it's created.

The agent based stuff is really simple because it can very easily be specific for each machine and always be specific when the systems are built without any work after the initial set up.

Just because a system is backed up via agentless backup doesn't mean the VMs data can't be separate. I could have a VM backed up agentlessly that has its data on a separate volume / virtual disk... then I could instantly restore the VM and OS, reattach the data, and not have to rebuild shit.

[Obsolesce]

It works both ways. Do what's best for a given environment. Not every damn system in a given environment can have the exact same backup configuration. In the one I work with for example, I use both agentless and agent based backups... because it's not one size fits all.

[Obsolesce]

But I will say it's a lot faster to restore the agentless backups that I either needed to restore or test restored.