Huge Mistake
-
@WrCombs said in Huge Mistake:
So let me start this off by saying if I hadn't listened to my boss first thing this morning, this mistake wouldn't have happened. but god forbid he say he was wrong.
Edit I took ownership of the fact that I had Moved to fast and didn't verify what I was doing, before formatting the drive- The plan changed and I made the mistake, trying to resolve the issue.
Okay: so, Went and picked up a PC from a customer: the plan as it was said to me was
- Pick up PC
*Bring back to office, put 2 new HDD's in and pull over the information after we image it.
*take it back and install it at the site again.
So, what actually happened?
Brought the PC back, boss told me to stop, he has an idea.
Reformat one of the Hard drives we have here, on that PC and then have the FakeRAID we use rebuild the information, then test the PC to run a terminal and verify it works properly.So I added the other drive to the PC. Little did I know (nor did I check) the Optical drive was set to boot first (which is where I added this Drive to the PC ). It came up as
C:and the PC I wanted to load asC:loaded asD:so when I opened cmd and typed in
format d:and pressed enter, I wiped all of the customer data from the Drives..it wasnt until I noticed a program we don't use on aloha PC's was when I realized what I had done.
My Tuesday Fuck up in a nutshell. -- Let's all take a moment to give me shit for this colossal screw up.
we already downloaded a Software to Recover lost partitions and I have that running right now .
Mistakes happen. That's just the way it is. It's not a big deal.
Only way to minimize mistakes is to have SOPs and follow them. That's how it's done everywhere where mistakes can have catastrophic consequences (aviation, military etc).
This is what we have used for recovery of deleted files, partitions and what not in the past.
https://www.r-studio.com/
They have a bunch of different options. I'm pretty sure you can recover from a format, at least I'm sure you can recover most of the files. -
@scottalanmiller said in Huge Mistake:
If we were at someplace like Bridgewater, we'd take a moment to post mortem and talk about what actually went wrong...
For example... did the boss' change of plan cause you to rush or get flustered? If so, learning to recognize when that is happening to you and take a minute to breath, focus, and compose yourself can go a long way. Or simply being able to say "I can't handle a last minute change, this is when I make mistakes." Lots of people can't plan for something then change last minute, because if you are like me, certain steps are already verified or checked off in your brain that the new plan might not account for.
In the original plan there were safety measures, did the boss tell you to skip those, too? Or did you skip them by accident?
Was the boss' plan to format the drive meant to be done with both drives in there currently?
Did you let the system do its own format that is automatic, required, and would meet the boss' plan, or was the additional step an assumption that turned out to not be needed?
There's a lot to delve into to try to figure out where and when mistake(s) were made, and how to try to hedge against them in the future.
This is actually something I did on my lunch break.
Before reading it I Took inventory of what happened,
causes and how I can make sure the mistake doesn't happen again:
here's what I found:
The change in plan caused me to shift my thinking which caused some confusion, Cause I had a gameplan with the idea of a backup already in mind.I didn't stop and think about what I was doing prior to hitting enter. I followed the simple steps i put into my head: thinking that The drives on the FakeRAID Promise Card would come up as
C:because that was the way it was before.
I rushed what I was doing, Possibly due to over confidence, of formatting a drive which I've seen done multiple times, but mostly with Removable USB drives (which I've done in the past so i thought " pshh Piece of Cake" -
@scottalanmiller said in Huge Mistake:
Was the boss' plan to format the drive meant to be done with both drives in there currently?
There is a big lesson in this statement. I think the majority of us would have shut down and removed power from the drive we didnt want to format. This doesn't even allow this mistake to ever happen.
-
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
but since we have to try and Screw over our customers as much as possible, and not do it the right way, and just replace both hard drives
While often that appears to be the logic, how did this screw over the customer here? Wasn't this going to get the customer up faster and cheaper?
It was my understanding that the customer was planning on buying new hard drives since we couldn't put a Western Digital with a Seagate.
It wasn't until I got to the office.Why can't you mix the drives? It's software RAID, it doesn't care.
When I've tried to do it in the past it wouldn't work. when I asked my boss he said " because you can't mix the drives. bring it back and we'll add 2 WD drives and pull the information back.."
you can mix drives ? in a Mirror RAID (if that's the right term for it) environment.
-
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
but since we have to try and Screw over our customers as much as possible, and not do it the right way, and just replace both hard drives
While often that appears to be the logic, how did this screw over the customer here? Wasn't this going to get the customer up faster and cheaper?
It was my understanding that the customer was planning on buying new hard drives since we couldn't put a Western Digital with a Seagate.
It wasn't until I got to the office.Why can't you mix the drives? It's software RAID, it doesn't care.
When I've tried to do it in the past it wouldn't work. when I asked my boss he said " because you can't mix the drives. bring it back and we'll add 2 WD drives and pull the information back.."
you can mix drives ? in a Mirror RAID (if that's the right term for it) environment.
You can mix but it absolutely best to have two identical drives with identical size and performance. We'd never build a new array with mixed drives.
-
@IRJ said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
Was the boss' plan to format the drive meant to be done with both drives in there currently?
There is a big lesson in this statement. I think the majority of us would have shut down and removed power from the drive we didnt want to format. This doesn't even allow this mistake to ever happen.
And I think my problem was not understanding what he meant.
So the whole issue stemmed from a Seagate hard drive crashing, but we use FakeRAID 1 which i was told was "Mirror RAID" so the PC still worked and we just needed to replace the bad drive.
Seagate Hard Drives would continuously fail thoughout the past so they moved to WD. -
@Pete-S said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
but since we have to try and Screw over our customers as much as possible, and not do it the right way, and just replace both hard drives
While often that appears to be the logic, how did this screw over the customer here? Wasn't this going to get the customer up faster and cheaper?
It was my understanding that the customer was planning on buying new hard drives since we couldn't put a Western Digital with a Seagate.
It wasn't until I got to the office.Why can't you mix the drives? It's software RAID, it doesn't care.
When I've tried to do it in the past it wouldn't work. when I asked my boss he said " because you can't mix the drives. bring it back and we'll add 2 WD drives and pull the information back.."
you can mix drives ? in a Mirror RAID (if that's the right term for it) environment.
You can mix but it absolutely best to have two identical drives with identical size and performance. We'd never build a new array with mixed drives.
ah, The way my company taught me was Never to do that because it causes more issues.
Good to know it can happen in case of emergency. -
@Dashrender said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
but since we have to try and Screw over our customers as much as possible, and not do it the right way, and just replace both hard drives
While often that appears to be the logic, how did this screw over the customer here? Wasn't this going to get the customer up faster and cheaper?
It was my understanding that the customer was planning on buying new hard drives since we couldn't put a Western Digital with a Seagate.
It wasn't until I got to the office.Why can't you mix the drives? It's software RAID, it doesn't care.
Because they have crazy people at his work who don't know this.
FYI - you could even put a larger drive in, you would just end up only using the amount of space on the smaller one.
guess that would make sense. Not sure anyone here would it see it that way.
-
@WrCombs said in Huge Mistake:
you can mix drives ? in a Mirror RAID (if that's the right term for it) environment.
You probably should take the A+ training on this site shared by @scottalanmiller . Usually A+ isn't recommended for most IT career paths, but what you are doing on a daily basis would match up well with A+.
You have asked about training here several times and then bring up not being able to move up in your career. This is basic stuff and you should really consider building a foundation on something even if its A+ and then branch out from there.
This not to pick on you, but hopefully give you a chance to move forward in your career with tough love. You need to focus on training and follow through. You aren't brand new (2-6 months) in IT anymore.
-
@IRJ said in Huge Mistake:
If you are doing PC repair often, a hardware cloner can help. Back when I was doing desktops, I could clone a drive in 5-10 mins. SSDs are a fraction of that now.
I learned your same lesson and always had spare HDs that I would use to clone information. Somebody on here will talk about how this is bad for SSDs to be written over and over, and they are right to a point. However, to reach that write level is pretty fucking difficult and drives are cheap. So cover your ass by cloning especially in a situation like this.
Yeah I still have a toaster that will do block by block cloning. You just pop the drives in and hit the button.
-
@WrCombs said in Huge Mistake:
@Pete-S said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
but since we have to try and Screw over our customers as much as possible, and not do it the right way, and just replace both hard drives
While often that appears to be the logic, how did this screw over the customer here? Wasn't this going to get the customer up faster and cheaper?
It was my understanding that the customer was planning on buying new hard drives since we couldn't put a Western Digital with a Seagate.
It wasn't until I got to the office.Why can't you mix the drives? It's software RAID, it doesn't care.
When I've tried to do it in the past it wouldn't work. when I asked my boss he said " because you can't mix the drives. bring it back and we'll add 2 WD drives and pull the information back.."
you can mix drives ? in a Mirror RAID (if that's the right term for it) environment.
You can mix but it absolutely best to have two identical drives with identical size and performance. We'd never build a new array with mixed drives.
ah, The way my company taught me was Never to do that because it causes more issues.
Good to know it can happen in case of emergency.They taught you not to mirror drives that are exactly the same?
-
@IRJ said in Huge Mistake:
@WrCombs said in Huge Mistake:
you can mix drives ? in a Mirror RAID (if that's the right term for it) environment.
You probably should take the A+ training on this site shared by @scottalanmiller . Usually A+ isn't recommended for most IT career paths, but what you are doing on a daily basis would match up well with A+.
You have asked about training here several times and then bring up not being able to move up in your career. This is basic stuff and you should really consider building a foundation on something even if its A+ and then branch out from there.
This not to pick on you, but hopefully give you a chance to move forward in your career with tough love. You need to focus on training and follow through. You aren't brand new (2-6 months) in IT anymore.
I took a course for A+ back in Middle school, And passed. Dont remember That being Covered.
But that was over 13 years ago by now, and I never used it again until after high school. Probably could use a refresher.The way it works here is 60% of calls are Software related strictly to Aloha
20% Windows OS issues
15% hardware swaps/troubleshooting .
5% computer hardware (like this one) -
@stacksofplates said in Huge Mistake:
@WrCombs said in Huge Mistake:
@Pete-S said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
but since we have to try and Screw over our customers as much as possible, and not do it the right way, and just replace both hard drives
While often that appears to be the logic, how did this screw over the customer here? Wasn't this going to get the customer up faster and cheaper?
It was my understanding that the customer was planning on buying new hard drives since we couldn't put a Western Digital with a Seagate.
It wasn't until I got to the office.Why can't you mix the drives? It's software RAID, it doesn't care.
When I've tried to do it in the past it wouldn't work. when I asked my boss he said " because you can't mix the drives. bring it back and we'll add 2 WD drives and pull the information back.."
you can mix drives ? in a Mirror RAID (if that's the right term for it) environment.
You can mix but it absolutely best to have two identical drives with identical size and performance. We'd never build a new array with mixed drives.
ah, The way my company taught me was Never to do that because it causes more issues.
Good to know it can happen in case of emergency.They taught you not to mirror drives that are exactly the same?
More or less...
-
@IRJ said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
Was the boss' plan to format the drive meant to be done with both drives in there currently?
There is a big lesson in this statement. I think the majority of us would have shut down and removed power from the drive we didnt want to format. This doesn't even allow this mistake to ever happen.
Right, typically that's what I would expect, since the drive is being removed anyway. And to format on another machine, if possible. Formatting is a terrifying process in general.
And in this case, wasn't part of the work. It was just an extra thing done for no known reason. Why the boss wanted any format done at all is unclear.
-
@WrCombs said in Huge Mistake:
@Pete-S said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
but since we have to try and Screw over our customers as much as possible, and not do it the right way, and just replace both hard drives
While often that appears to be the logic, how did this screw over the customer here? Wasn't this going to get the customer up faster and cheaper?
It was my understanding that the customer was planning on buying new hard drives since we couldn't put a Western Digital with a Seagate.
It wasn't until I got to the office.Why can't you mix the drives? It's software RAID, it doesn't care.
When I've tried to do it in the past it wouldn't work. when I asked my boss he said " because you can't mix the drives. bring it back and we'll add 2 WD drives and pull the information back.."
you can mix drives ? in a Mirror RAID (if that's the right term for it) environment.
You can mix but it absolutely best to have two identical drives with identical size and performance. We'd never build a new array with mixed drives.
ah, The way my company taught me was Never to do that because it causes more issues.
Good to know it can happen in case of emergency.It's nice to use matching drives, obviously. But there is no concept of an issue with mismatched drives in RAID. That's one of those myths that arose in the mid-2000s, but no one knows why. It's never been a thing.
You get the absolute best performance and capacity by having them be matched, but there's no "problem" with unmatched drives. Many people even promote that for safety, but I wouldn't go that far. I'd always do matched drives all things being equal.
-
@WrCombs said in Huge Mistake:
The way my company taught me
This is consistently biting you in the ass, or at least leading you astray. You know that you don't have IT people there and they have little to no understanding of IT processes, technology, etc. Not that they get everything wrong, but they clearly don't understand what they are telling you in case after case and are just repeating things that they think that they heard somewhere. Like your boss wanting you to format a drive before adding it to an array, clearly he is missing some really basic drive knowledge. It's not a big deal that he doesn't understand IT fundamentals, but you keep coming back to "they told me" in thread after thread and I know that you are not defending them, but you keep letting them confuse you.
Basically... if they tell you something you need to either ignore it out of hand, or go immediately and research it. Because they are literally moving you backwards.
-
@WrCombs said in Huge Mistake:
So the whole issue stemmed from a Seagate hard drive crashing, but we use FakeRAID 1 which i was told was "Mirror RAID" so the PC still worked and we just needed to replace the bad drive.
FakeRAID = software RAID that is sold with the pretense of being hardware, it exists for malicious purposes
"Mirror RAID" = RAID 1, it's a bizarre way that someone who doesn't work in IT might try to explain a standard term.
Yes, the PC should have continued to have worked both as it was, and once the failed drive was removed, and with the new drive put in. That it was FakeRAID or mirrored aren't factors in that, just that it was RAID that wasn't RAID 0 was all that mattered there. They weren't wrong, just not really relevant.
This is why a backup was easy to do, because the system was still working fine. It's also why there was no need to format or do all the extra steps.
-
In a case like this, the simplest process is to do what is intended by the system design...
- Take a backup before doing anything else.
- Remove the failed drive.
- Replace with a drive of equal or larger size (any brand, make, model, etc.)
- Tell the RAID system to use that drive to rebuild.
That's it. Fast, simple, safe. This is what the RAID system is designed for and how it is intended to be used.
-
@WrCombs said in Huge Mistake:
@Dashrender said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
but since we have to try and Screw over our customers as much as possible, and not do it the right way, and just replace both hard drives
While often that appears to be the logic, how did this screw over the customer here? Wasn't this going to get the customer up faster and cheaper?
It was my understanding that the customer was planning on buying new hard drives since we couldn't put a Western Digital with a Seagate.
It wasn't until I got to the office.Why can't you mix the drives? It's software RAID, it doesn't care.
Because they have crazy people at his work who don't know this.
FYI - you could even put a larger drive in, you would just end up only using the amount of space on the smaller one.
guess that would make sense. Not sure anyone here would it see it that way.
As mentioned by others, it's not advisable - you end up with likely performance issues because the drives might operate at different performance levels, but at worse really, you should mostly only be as bad as the slowest drive (but I can think of reasons why it could be much worse than that).
-
@Dashrender said in Huge Mistake:
@WrCombs said in Huge Mistake:
@Dashrender said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
@scottalanmiller said in Huge Mistake:
@WrCombs said in Huge Mistake:
but since we have to try and Screw over our customers as much as possible, and not do it the right way, and just replace both hard drives
While often that appears to be the logic, how did this screw over the customer here? Wasn't this going to get the customer up faster and cheaper?
It was my understanding that the customer was planning on buying new hard drives since we couldn't put a Western Digital with a Seagate.
It wasn't until I got to the office.Why can't you mix the drives? It's software RAID, it doesn't care.
Because they have crazy people at his work who don't know this.
FYI - you could even put a larger drive in, you would just end up only using the amount of space on the smaller one.
guess that would make sense. Not sure anyone here would it see it that way.
As mentioned by others, it's not advisable - you end up with likely performance issues because the drives might operate at different performance levels, but at worse really, you should mostly only be as bad as the slowest drive (but I can think of reasons why it could be much worse than that).
It CAN be slower than the slowest drive, but only by the tiniest bit.
