Centrally Controlled Local Backup System Options
-
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
Basically we need to treat the backups at the customer the same way that we typically do RMM, AntiVirus, Remote Access and other functions. All of the risk already exists from those covering the same bases. They just seem to all lack the ability to kick off and monitor a backup.
This you can do for free:
Have the built-in backup software do a backup, or even the free version of Veeam. You could use a PowerShell script to run a Veeam backup, verify success, send a webhook to an Azure function or Runbook, or an email, to do whatever you want. This specific part of the how isn't the point, as there's tons of ways to do it to fit any requirement.
But after that part, you can schedule an Azure Runbook to run daily or whenever, to check for X, if not X, then it alerts. If X, then sends success message if you prefer. This way, no matter what, even if the on-prem solution totally fails and MS Azure or AWS is still up, you'll get whatever you want to happen still happen.
Example:
- Built-in backup runs via powershell or bash script.
- Backup success or fail webhook or email is sent via powershell or bash, or backup software if supported.
- Depending on method, you have a free serverless script take care of back end processing.
- Back-end processing can take care of whether or not something happened when it was supposed to, and if so, if it was a success for failure, and respond appropriately, by doing anything you can imagine... send email, write to database that your fancy web GUI can read, write to cloud storage, basically anything.
There you'll have a way to verify backups whether success, failure, or complete failure. You can even automate, via scripting, backup restore testing as well.
If Windows backup, I know you can back up to VHD(x), auto mount, attempt to restore a known file to somewhere, test for that, respond appropriately.
Yeah, that will work, for sure. Basically it's building the solution that we want, which I know we can do. I'm trying to determine if really no one has made this. Everyone makes it where this happens, but the data also goes external. But keeping the data on premises with the customer seems to make this just not exist in the real world. The tech is obviously all there, every backup vendor has all the pieces built, they just don't allow the local target.
-
@stacksofplates said in Centrally Controlled Local Backup System Options:
I mean an outside the box scenario is you could use Jenkins to schedule all of this. Then just use something like Restic or Borg to do the backups either locally or to a central location.
Then you can have Jenkins send messages/emails on a failure. You'd just need a Jenkins slave at every site (can be whatever you are running the backup on).
Not specifically looking to trigger the backups remotely. Would like them to trigger locally and just report because if the Internet goes down, you don't want things to not keep going.
Salt or Ansible should be able to do this too. And you could script the responses.
-
This just seems like such a simple thing. And I'm not the one that came up with the need, I was just the one asked to research it for another company. But I talked to companies that make this stuff as part of the process and it seems like there is just so much money in the cloud storage portion, that no one is willing to make local targets overly accessible or attractive.
-
@scottalanmiller said in Centrally Controlled Local Backup System Options:
@stacksofplates said in Centrally Controlled Local Backup System Options:
I mean an outside the box scenario is you could use Jenkins to schedule all of this. Then just use something like Restic or Borg to do the backups either locally or to a central location.
Then you can have Jenkins send messages/emails on a failure. You'd just need a Jenkins slave at every site (can be whatever you are running the backup on).
Not specifically looking to trigger the backups remotely. Would like them to trigger locally and just report because if the Internet goes down, you don't want things to not keep going.
Salt or Ansible should be able to do this too. And you could script the responses.
I assumed that Jenkins would have the Ansible plugin (or Salt if there is one, not sure) so the slave could do whatever it needed with that tool. You could always deploy script and then just have a Jenkins job to scrape the data.
Probably more of a realistic scenario is to just have the scripts at each site deploy to the location (remote or local) and then have Prometheus monitor that. You could pretty easily have an exporter report any failures on the systemd unit (or whatever service you want).
-
Its been a long while but didn't BackupExec used to have a central management console? I haven't looked at or used their product in 5-7 years now but might be worth checking into
-
@jt1001001 said in Centrally Controlled Local Backup System Options:
Its been a long while but didn't BackupExec used to have a central management console? I haven't looked at or used their product in 5-7 years now but might be worth checking into
OMG, do they still exist? LMAO
-
Yes they were re-spun off of Symatec a couple years back.
-
@jt1001001 said in Centrally Controlled Local Backup System Options:
Yes they were re-spun off of Symatec a couple years back.
You'd think that I'd remember them. I worked for them both pre- and during-Symantec (I was there during the buyout.) But I never worked on the BE side, only on the Netbackup side.
-
Metallic looks like it might do what I want here.
-
@scottalanmiller said in Centrally Controlled Local Backup System Options:
Metallic looks like it might do what I want here.
Looks great, but I don't see anything to suggest it will centrally manage backups to locally attached USB devices.
-
We use https://www.backupassist.com at one site to backup up USB RDX drive. Think they have central management.
-
@scottalanmiller said in Centrally Controlled Local Backup System Options:
Metallic looks like it might do what I want here.
Unless you're talking about using Metallic's Endpoint, the pricing structure seems... strange to me. Maybe I'm looking at it wrong.
The endpoint pricing seems to be ~$10 /user /month.
The Core pricing seems to be $200/tb/month for using your own storage? Seems a bit pricey to me on that front.
-
@dafyre said in Centrally Controlled Local Backup System Options:
The endpoint pricing seems to be ~$10 /user /month.
How does that work with multiple sites? I'm one user backing up 100 sites, so it's just $10 a month?
-
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
Metallic looks like it might do what I want here.
Looks great, but I don't see anything to suggest it will centrally manage backups to locally attached USB devices.
Well it's a SaaS product, so presumably hosted or at least hostable. I swear I read that local storage was allowed.
-
Not sure but Arcserve might fit the bill. Although it would require Windows server(s) and is probably very close to Veeam's architecture.
-
For those wondering why email doesn't cut it.... literally had an unrelated situation where they were using Veeam with email alerts and it didn't alert and no one knew it wasn't backing up for four months! The customer presumably knew that they had removed the backup target and not replaced it or communicated with anyone. But nonetheless, had there been the central console for this local backup job, we would have known instantly.
-
@scottalanmiller said in Centrally Controlled Local Backup System Options:
For those wondering why email doesn't cut it.... literally had an unrelated situation where they were using Veeam with email alerts and it didn't alert and no one knew it wasn't backing up for four months! The customer presumably knew that they had removed the backup target and not replaced it or communicated with anyone. But nonetheless, had there been the central console for this local backup job, we would have known instantly.
Yeah, that's why you can't rely on the backup software client by itself. You need another piece to handle the "nothing happening" part.
-
@Obsolesce said in Centrally Controlled Local Backup System Options:
Yeah, that's why you can't rely on the backup software client by itself. You need another piece to handle the "nothing happening" part.
Right. Something not on the server or at the customer site because it is often server, software, or network failure causing the issue.
-
@scottalanmiller said in Centrally Controlled Local Backup System Options:
For those wondering why email doesn't cut it.... literally had an unrelated situation where they were using Veeam with email alerts and it didn't alert and no one knew it wasn't backing up for four months! The customer presumably knew that they had removed the backup target and not replaced it or communicated with anyone. But nonetheless, had there been the central console for this local backup job, we would have known instantly.
Reminds me of a couple weeks ago when we had an emergency alert go out on campus. I know this is fairly unrelated but but it does describe a flaw with email. I did not get an email giving me the notice because we were also having email issues at the time. So i walked out of a classroom at 4:00 in the afternoon and there was no staff left on campus but me. So yeah, have to have something else working besides email.
-
@jmoore said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
For those wondering why email doesn't cut it.... literally had an unrelated situation where they were using Veeam with email alerts and it didn't alert and no one knew it wasn't backing up for four months! The customer presumably knew that they had removed the backup target and not replaced it or communicated with anyone. But nonetheless, had there been the central console for this local backup job, we would have known instantly.
Reminds me of a couple weeks ago when we had an emergency alert go out on campus. I know this is fairly unrelated but but it does describe a flaw with email. I did not get an email giving me the notice because we were also having email issues at the time. So i walked out of a classroom at 4:00 in the afternoon and there was no staff left on campus but me. So yeah, have to have something else working besides email.
Yeah, the problem with any email based system is you don't know when you don't receive something. Email alerting is based on the "tell me when something is wrong" concept, which is fine to do. But alone is rarely enough. We need a "tell me that everything is okay, all the time" approach. We need to have constant verification that nothing has failed.
