How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller
-
@JaredBusch said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
Shut the system down pull the failed drive and the hot spare turn it back on. You should be up and running on a degraded RAID array
We are terrified to turn the system off at this point.
-
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@JaredBusch said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
Shut the system down pull the failed drive and the hot spare turn it back on. You should be up and running on a degraded RAID array
We are terrified to turn the system off at this point.
I am guessing no backups? Is the system available at all?
-
@wrx7m said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@JaredBusch said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
Shut the system down pull the failed drive and the hot spare turn it back on. You should be up and running on a degraded RAID array
We are terrified to turn the system off at this point.
I am guessing no backups? Is the system available at all?
System is currently available. In theory there are backups, but no capacity to test them and they are across the country. This isn't a managed customer, so we have no documentation, no visibility, no access. We just have to trust that they say that there are backups. But they confused their UPS with their NAS calling both their "backup" so we have literally no clue what they do and do not have.
-
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@wrx7m said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@JaredBusch said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
Shut the system down pull the failed drive and the hot spare turn it back on. You should be up and running on a degraded RAID array
We are terrified to turn the system off at this point.
I am guessing no backups? Is the system available at all?
System is currently available. In theory there are backups, but no capacity to test them and they are across the country. This isn't a managed customer, so we have no documentation, no visibility, no access. We just have to trust that they say that there are backups. But they confused their UPS with their NAS calling both their "backup" so we have literally no clue what they do and do not have.
I'd burn a couple of billable hours at least doing some investigation into their things using Mesh Commander. That way you have a better idea of where they are on the we're screwed o'meter.
-
@dafyre said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@wrx7m said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@JaredBusch said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
Shut the system down pull the failed drive and the hot spare turn it back on. You should be up and running on a degraded RAID array
We are terrified to turn the system off at this point.
I am guessing no backups? Is the system available at all?
System is currently available. In theory there are backups, but no capacity to test them and they are across the country. This isn't a managed customer, so we have no documentation, no visibility, no access. We just have to trust that they say that there are backups. But they confused their UPS with their NAS calling both their "backup" so we have literally no clue what they do and do not have.
I'd burn a couple of billable hours at least doing some investigation into their things using Mesh Commander. That way you have a better idea of where they are on the we're screwed o'meter.
That'd be great. But their CIO doesn't like us doing anything.
-
@scottalanmiller I just checked this morning. Still at 6% / 100%. No progress on the rebuild. Stuck in a loop, or just simply stuck and still a de facto RAID 0 right now.
-
@CCWTech said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller I just checked this morning. Still at 6% / 100%. No progress on the rebuild. Stuck in a loop, or just simply stuck and still a de facto RAID 0 right now.
Could be either, the interface isn't clear any longer.
-
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@dafyre said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@wrx7m said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@JaredBusch said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
Shut the system down pull the failed drive and the hot spare turn it back on. You should be up and running on a degraded RAID array
We are terrified to turn the system off at this point.
I am guessing no backups? Is the system available at all?
System is currently available. In theory there are backups, but no capacity to test them and they are across the country. This isn't a managed customer, so we have no documentation, no visibility, no access. We just have to trust that they say that there are backups. But they confused their UPS with their NAS calling both their "backup" so we have literally no clue what they do and do not have.
I'd burn a couple of billable hours at least doing some investigation into their things using Mesh Commander. That way you have a better idea of where they are on the we're screwed o'meter.
That'd be great. But their CIO doesn't like us doing anything.
It's even worse than that. CIO wants us to make the current situation work for 2-3 more years without spending any money.
CIO is completely out of touch with reality, Intentionally ignoring our advice and is allowing a meltdown to happen.
-
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@dafyre said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@wrx7m said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@JaredBusch said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
Shut the system down pull the failed drive and the hot spare turn it back on. You should be up and running on a degraded RAID array
We are terrified to turn the system off at this point.
I am guessing no backups? Is the system available at all?
System is currently available. In theory there are backups, but no capacity to test them and they are across the country. This isn't a managed customer, so we have no documentation, no visibility, no access. We just have to trust that they say that there are backups. But they confused their UPS with their NAS calling both their "backup" so we have literally no clue what they do and do not have.
I'd burn a couple of billable hours at least doing some investigation into their things using Mesh Commander. That way you have a better idea of where they are on the we're screwed o'meter.
That'd be great. But their CIO doesn't like us doing anything.
Then why are you still helping them?
-
@Dashrender said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@dafyre said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@wrx7m said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@JaredBusch said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
Shut the system down pull the failed drive and the hot spare turn it back on. You should be up and running on a degraded RAID array
We are terrified to turn the system off at this point.
I am guessing no backups? Is the system available at all?
System is currently available. In theory there are backups, but no capacity to test them and they are across the country. This isn't a managed customer, so we have no documentation, no visibility, no access. We just have to trust that they say that there are backups. But they confused their UPS with their NAS calling both their "backup" so we have literally no clue what they do and do not have.
I'd burn a couple of billable hours at least doing some investigation into their things using Mesh Commander. That way you have a better idea of where they are on the we're screwed o'meter.
That'd be great. But their CIO doesn't like us doing anything.
Then why are you still helping them?
At this point we are not.
We have made our recommendations and it's up to them.
If they choose to disregard our advise, it's at their own peril and we will watch it burn. That's not of course what we want to happen, but the manager has chosen to be the CIO even though she is not competent to do so.
-
@Dashrender said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@dafyre said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@wrx7m said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@JaredBusch said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
Shut the system down pull the failed drive and the hot spare turn it back on. You should be up and running on a degraded RAID array
We are terrified to turn the system off at this point.
I am guessing no backups? Is the system available at all?
System is currently available. In theory there are backups, but no capacity to test them and they are across the country. This isn't a managed customer, so we have no documentation, no visibility, no access. We just have to trust that they say that there are backups. But they confused their UPS with their NAS calling both their "backup" so we have literally no clue what they do and do not have.
I'd burn a couple of billable hours at least doing some investigation into their things using Mesh Commander. That way you have a better idea of where they are on the we're screwed o'meter.
That'd be great. But their CIO doesn't like us doing anything.
Then why are you still helping them?
Just waiting on them to give the orders. We aren't the CIO or the decision makers. We are the well paid button pushers here.
-
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
But they confused their UPS with their NAS calling both their "backup"
WOW.
-
@wrx7m said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
But they confused their UPS with their NAS calling both their "backup"
WOW.
Yeah... their CIO did that.
-
I've never seen anything like that - however we've always used BIOS to set degrading.. could it be something along those lines?
we use Intel Rapid Storage - CTL +I on Boot gets you there? - Disclaimer I may be thinking of Software RAID rather than Hardware RAID - but It may be worth a shot to try.. -
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@Dashrender said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@dafyre said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@wrx7m said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@scottalanmiller said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
@JaredBusch said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
Shut the system down pull the failed drive and the hot spare turn it back on. You should be up and running on a degraded RAID array
We are terrified to turn the system off at this point.
I am guessing no backups? Is the system available at all?
System is currently available. In theory there are backups, but no capacity to test them and they are across the country. This isn't a managed customer, so we have no documentation, no visibility, no access. We just have to trust that they say that there are backups. But they confused their UPS with their NAS calling both their "backup" so we have literally no clue what they do and do not have.
I'd burn a couple of billable hours at least doing some investigation into their things using Mesh Commander. That way you have a better idea of where they are on the we're screwed o'meter.
That'd be great. But their CIO doesn't like us doing anything.
Then why are you still helping them?
Just waiting on them to give the orders. We aren't the CIO or the decision makers. We are the well paid button pushers here.
If they are being that "tight" (and possibly stupid) do you worry about your final bill being paid?
-
@WrCombs said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
I've never seen anything like that - however we've always used BIOS to set degrading.. could it be something along those lines?
THis is a server with hardware RAID, the BIOS can't talk to the RAID card.
-
@WrCombs said in How to Stop an Ongoing RAID Rebuild HP P420i RAID Controller:
we use Intel Rapid Storage - CTL +I on Boot gets you there? - Disclaimer I may be thinking of Software RAID rather than Hardware RAID - but It may be worth a shot to try..
Yeah, Intel Rapid Storage is the most prominent example of Fake RAID. So it is software.
