Posts made by BRRABill

@Jason said:

75min from 100GB? Sounds like your disk arrays are a bit slow.

That's over the LAN. Might have been a bit less time and a bit more GB.

NOTE: Anything I think I've learned here, discussed here, or needs discussed here, I'll mark with a *ML.

My day started with only a mild ominous touch. It was raining, and my youngest daugher thought she was going to throw up. My wife decided to stay home, so I went in to work. Good thing I did.

The day was going fine until around 3:00PM. I noticed our e-mail server was moving slowly. Tried Remote Desktop, was not responding. Time to go to the local box, possibly the old fashioned hard reboot.

When I got into the server room, I noticed one of the drives on our main (and ONLY) data server was blinking amber. I go from a 1 to a 5 on the 1 to 10 anxiety scale because that kind of stuff always makes me nervous. Anyway, no problem, I have spare drives on the shelf ready to go. I pull out the old drive. No problem. I put in the new drive, no problem. I go to log in to start rebuilding the array, and I notice that the server is rebooting. Hmm, that's odd. I look at the drive. Now TWO of the four are blinking amber. I've now gone to a 10, LOL.

Turns out a second drive failed after I did the hot plug. I now realize my data server array is gone. 25 years of data possibly gone forever. Let's hope our Datto device is as good as advertised.

I started up a a hybrid virtualization on our Datto. (Our Datto ALto 2 device cannot virtualize locally, only in the cloud.) Within about 15 minutes of the "event", we had a virtual server up and running on the LAN. The users could go about working as they had been. I made an announcment not to save anything to the server, and began the process of doing a BMR.

*ML1: I actually have extra non-OEM licenses of Server 2003, so this was actually a legit use of the technology.

*ML2: The reason I said not to save is because the Datto allows you to save, but then you need to do a backup on the virtual device, and then do the BMR from that. Since our device is VM in the cloud only, that would not be a great option. All other Datto devices virtual locally and in the cloud, so that would be more feasible.

The BMR is where the trouble began. We have a brand new server, but I did not want to use that, as that will be the platform for our new Hyper-V VMs. I grabbed a spare desktop we had around that also had an Intel RAID controller in it. I plugged in an SSD, and began the BMR. In my tests, I had some issues with BMR, so just in case, I only restored the boot drive. In those test issues, I was able to fix it with the StorageCraft Recovery Environment. (Datto uses ShadowProtect as its backup program.) But we were not able to fix this particular issue. It booted to a black screen. After a while on the phone with Datto support, I decided to BMR another machine while the tech did some backend work on the Datto box to try another BMR method. I got the second desktop up and running, received a STOP 7B error, and was able to fix it with the StorageCraft recovery CD. But then got another strange error, a C0000135 error. I started Googling this while the Datto tech did another BMR on the Intel RAID machine.

Google told me this error was caused by a recent Windows Update. I was able to boot into the SC recovery environment and manually "uninstall" the KB update (by copying files from the uninstall folder for the KB) that caused the 135 error. With fingers crossed I rebooted the machine, and it came up. I started to restore the data drive image.

This took about 75 minutes for 100GB. Rebooted again, and everything was exactly how it was at 2:59.

So it took about 12 hours, but I had the server back up as it was. About 9 of the hours was getting the BMR to work. I think most of the wasted time was due to a driver issue with the Intel RAID card.

*ML3: There has been a lot of discussion about BMR and why it's not always a great idea. I'm not sure how I could have made this better. I considered in the future having a machine I knew I could BMR to, but I'm not sure if the image itself (and the filesit has loaded) makes a difference in what to BMR to.

The server is now running on a DELL desktop with a single SSD, but it's up. Considering the age of the server, this is honestly probably a better solution! The data on this machine will be moved to the new server once that is up and running (whenever 2016 comes out).

The main thing I took away from this is ... working backups are so, so important. I also understand why virtualizing is so awesome ... no need to worry about these hardware issues.

All I am saying is I want a product that gives that.

I've had issue with cloud stuff, but just on a personal level. That's why I am looking for reviews/suggestions.

And I don't necessarily mean an exact mimic of what they are doing. I just want them to be able to save files in a way (such as from explorer) that makes sense to them.

BTW: my original post was some of the worst typing I've ever seen. Kudos on putting those letters together into English.

@scottalanmiller said:
OneDrive is supposed to delete in both places together because it is a mirror, it's not a lack of robustness.

Yes but something like DropBox for Business has backups to restore from. OneDrive said .. so, so sorry.

With all of the talk about SOHO/SMB using the cloud and not even owning a NAS/server, what are some recommendations/reviews of cloud storage for these companies?

I know someone at one point mentioned DropBox for Business.

I'm looking for something that will sync files locally, and integrate and emulate as close as possible what the user is used to now.

I've had some issues with some of these services (such as OneDrive), but it was on the personal level.

(My OneDrive issue was on a new installtion of Windows. It was on a tablet, and automaticaly installed itself. I delted some files and .... oops, gone from One Drive cloud as well. I'm assuming that the business products ar emore robut and have backup copies available.)

Talk to me!

No matter. I am on a new machine now with a new drive. Neither server grade, but all temporary. Probably safer.

All to be written up some day soon. I had to go into work today on my day off (with the two kids in tow who LOVED IT (for real)) for non-IT stuff.

I'm now having beer and watching the Jets/Bills game.

@scottalanmiller said:

@JaredBusch that sucks. that's very high. She can't be feeling very good

Kids can generally handle it much better than us adults.

I remember the first time our first kid had a 104 degree or higher. Our ped was like, eh, she'll be ok.

Still sucks for them, though. Sending good feelings over the net!

@MattSpeller said:

Trying to create a new laptop skin to show corporate branding. Somehow this wasn't a popular choice!?!?

Too soon.

@drewlander said:

Go right ahead. Did that drive fail after replacement while it was in a degraded state? Id say your controller is failing if that happened.

There were 4 drives.

1 2 3 4

2 was degraded/failed. I took it out, and put in a fresh one. The server then rebooted, and both 1 and 2 showed up as failed when it came back up.

@drewlander said:

@BRRABill That Chinese character means "Spring".

Maybe she gave it to me in the Spring.

THOUGH...if all my server die tonight, I am blaming you.

It was firmly plugged in. I think it just gave up the ghost.

I've seen that kind of stuff happen with a surge, but that seems unlikely in a hotplug backplane.

My drive failed almost immediately. I mean, whatever happened rebooted the server.

@scottalanmiller said:

RAID 5 induces other failures when you go to rebuild. It's extremely common and just an artifact of that RAID level. Doesn't mean that it will always do it or even normally do it, but it is very common. Once you do a drive swap it immediately increases the load on the drives and makes them more likely to fail.

Is it just RAID 5 that induces failures? I mean, theoretically couldn't a RAID 10 array do the same thing?

@Jason

Yeah that's the server I have that the RAID 5 array died on me Tuesday.

Ironic.

@Reid-Cooper said:

What the heck is that thing?

Which thing?

The paper thing?

Way back in the day when I used to assemble computers, the wife of the guy whose shop I went to made that for me and said it was a good luck charm. I hung it in our server room, and it's been with the servers ever since.

@scottalanmiller said:

Only two drives, so no parity overhead, and they sure don't brag about 512 MB

I mean how much could it cost to go to 1GB for them?

P.S. If anyone can read that, and it DOESN'T say good luck, please don't let me know.

True story. Right after I posted that last post, I went into the server room to take a picture of this paper good luck charm. On the way back down the hall, the building's power went out, and has been out the past 3 hours. This week is just AWESOME!

Anyway, here is the picture:

Note the failed DELL right below it.

It did its job for many years, though. No complaints.

@scottalanmiller said:

We once had a set of Compaq Proliant 800s that made it a decade without failing. They were all retired effectively still healthy - just old and worthless.

That's about where we are. I've hung lucky mementos in there, and am hoping for the best.

I actually have a construction paper good luck charm a vendor's wife once gave me a long time ago (before these servers even) that's actually hanging in there. It has done it's job pretty good so far.