Strange Smart Array p410i problem
-
@scottalanmiller said:
@flomer said:
But, I have seen earlier on other DL380s that a drive will have an amber LED indicating that it is in "pre-failure" state, probably because of SMART-errors. That didn't happen here, and I have not seen a RAID before that has been so slow and troubled by a bad disk... I don't have that much experience, though... I will be interesting to hear what HP or our dealer will say about this. I also asked them if they think the RAID card is faulty. The ease of having a controller guiding you is sort of not present anymore, might as well buy an LSI HBA and go for ZFS. I woudl surely have been notified about this.
Any chance that you are using third party drives instead of HP drives? Non-HP drives will not fully report to the SmartArray.
I have not touched the RAID since setting it up almost three years ago, and all parts are HP parts. See below for snapshot of error message. The RAID is RAID 6, not 60, by the way...
I still haven't heard from the reseller why the RAID controller didn't kick the drive out.
-
@flomer said:
But, will this not lead to slower operation? There must be some overhead? And this should be the only VM on that server, then?
Yes but if you can measure it that would be shocking. There is effectively no overhead on the disk IO and all of your bottlenecks are from spinning disks and such. You still get all of your available threads, nearly all of your memory, etc. The CPU hit is nominal and the disk IO hit is nominal. The benefits are huge and the caveats are things you probably can't even measure.
But yes, the only VM on the server most likely. Virtualization won't cause performance issues. Consolidation might. But test it, you might get a lot of consolidation out of it too. All depends on your workload, hardware changes, etc.
-
@flomer said:
I still haven't heard from the reseller why the RAID controller didn't kick the drive out.
There is a really good chance, and I really mean VERY GOOD chance, that the threshold of "too many errors" was not hit until the reboot. Rebooting a system will cause changes in drive activity that could easily trigger the difference between "too few" and "too many" errors. I have seen this a lot.
-
@scottalanmiller said:
@flomer said:
I still haven't heard from the reseller why the RAID controller didn't kick the drive out.
There is a really good chance, and I really mean VERY GOOD chance, that the threshold of "too many errors" was not hit until the reboot. Rebooting a system will cause changes in drive activity that could easily trigger the difference between "too few" and "too many" errors. I have seen this a lot.
Well, I actually rebooted the machine thinking it might help before I knew it was a failing drive. It seemed to help a little, but I guess I must just have imagined it getting better. Or at least the situation got just as a bad after a little while.
-
@scottalanmiller said:
@flomer said:
But, will this not lead to slower operation? There must be some overhead? And this should be the only VM on that server, then?
Yes but if you can measure it that would be shocking. There is effectively no overhead on the disk IO and all of your bottlenecks are from spinning disks and such. You still get all of your available threads, nearly all of your memory, etc. The CPU hit is nominal and the disk IO hit is nominal. The benefits are huge and the caveats are things you probably can't even measure.
But yes, the only VM on the server most likely. Virtualization won't cause performance issues. Consolidation might. But test it, you might get a lot of consolidation out of it too. All depends on your workload, hardware changes, etc.
But, isn't virtualizing FreeNAS something that is generally adviced against? And how to presen the entire RAID 6 drive to FreeNAS? By using several 2 TB disks and LVM? I guess ZFS is out of the question anyway, since I don't have direct access to the individual drives.
-
@flomer said:
But, isn't virtualizing FreeNAS something that is generally adviced against?
Based on what? The rule is "virtualize everything." I know of no reason that FreeNAS should be physical.
-
@flomer said:
And how to presen the entire RAID 6 drive to FreeNAS?
Not sure what you mean. As it is, the RAID array is presented as a disk. When you virtualize you put the FreeNAS storage onto the presented disk from the hypervisor. There is nothing to know here, just set up in the default way and as long as you have hardware RAID 6 that is the only storage option that you have.
-
@flomer said:
I guess ZFS is out of the question anyway, since I don't have direct access to the individual drives.
ZFS doesn't need direct access to anything. Using hardware RAID or virtualization don't block ZFS in any way.
-
@scottalanmiller said:
@flomer said:
But, isn't virtualizing FreeNAS something that is generally adviced against?
Based on what? The rule is "virtualize everything." I know of no reason that FreeNAS should be physical.
I have read on the FreeNAS forums that they advice people not to virtualize, but it might have been specifically about ZFS.
-
@flomer said:
@scottalanmiller said:
@flomer said:
But, isn't virtualizing FreeNAS something that is generally adviced against?
Based on what? The rule is "virtualize everything." I know of no reason that FreeNAS should be physical.
I have read on the FreeNAS forums that they advice people not to virtualize, but it might have been specifically about ZFS.
I'm a bit confused at the moment... I was under the assumption that ZFS needed raw access to drives. For SMART?
-
@flomer said:
By using several 2 TB disks and LVM?
No LVM in FreeNAS. That's a Linux system. You would do this with ZFS. But yes, groups of 2TB virtual disks, until someone surpasses that limitation.
-
@flomer said:
I have read on the FreeNAS forums that they advice people not to virtualize, but it might have been specifically about ZFS.
I would avoid those forums. I've been dealing with horrible information from them for years now. You should read this article, it explains why people are saying that. It isn't because they don't want you to be virtual, it's because they are "religious zealots" about using ZFS as software RAID. They don't mention that, though, instead they give bad general advice without context. They leave out the parts that matter (that they are trying to sacrifice stability and reliability to promote the use of ZFS as a software RAID system at any cost.)
-
@flomer said:
I'm a bit confused at the moment... I was under the assumption that ZFS needed raw access to drives. For SMART?
ZFS needs access if you want it to get the SMART data or to replace your RAID controller. Neither of these are things that you want. That's the careful marketing of the FreeNAS forums. The statement sounds reasonable but the "if" is a negative. You have enterprise hardware RAID and you want that reading the SMART data, not FreeNAS. You don't want FreeNAS having access to the drives, that would be bad.
-
ZFS is great when you want or need software RAID. It's probably the best software RAID option on the market. But it is not common for SMBs to want software RAID and when they do want it, it is normally because they are cutting costs and wanting to save the hundreds of dollars that quality hardware RAID costs. But you already have enterprise hardware RAID and to get SMART data to FreeNAS you don't just need to bypass it, you need to replace it which will cost even more money. That doesn't make sense.
You have an excellent enterprise storage server, trying to use ZFS to replace the parts that you already have will just undermine you. The FreeNAS forums assume that you care about using ZFS more than you care about anything else (business goals, cost, common sense, etc.) so they write the forums with that assumption in mind and it doesn't often come out as useful advice to people looking to actually implement FreeNAS in a business.
-
OK, now I have a bit of reading to do... Thank you for all the information! I might return with a question or two after doing some reading
-
No problem.
If you are comfortable with Linux, I would also suggest strongly considering dropping FreeNAS or FreeBSD at all. FreeBSD is not ideal as a storage platform. OpenSuse would be my first choice if you are comfortable working on a Linux server.
-
Hello,
I have the same problem with 2 DL380G7-machines.
They are both exactly identical, because we use Marathon High-Availibilty VM.
Transfer-rates from/to the VMs were bad and I also noticed the blinking mentioned by OP.The first logical Drive on each server consits of 4 original HP-SAS-Drives 450GB.
Smart-Start tells me, these drives are online for 44 months, but are Ok and should NOT be replaced...but these drives are all blinking about every second.
All drives simultanously, all drives green - no amber, red or blue - just on and off and on and off...all the time.The second logical drive on each server consits of 4 original HP-SAS-Drives 300GB.
Online for 16 months, according to Smart-Start (these have been added to the servers later - so that seems correct).
All drives good...and the LEDs of these drives are all four constant green. No blinking...just straight green (all VMs are down, so there is really no activity at the moment.)
THAT´S how it´s supposed to be, I think.But what is wrong with the first logical drive?
I cannot find a hint, that any of its HDDs has a failure.
A controller-error? Only on Port 1 and in 2 different servers at the same time? Not very realistic...
Are the old drives somehow incompatible to the new ones?? But there are 2 independent RAIDs and they´re not mixed within a RAID.
How can that be?So what should I do?
Replace all 8 SAS-HDDs just to "give it a try"? Quite expensive...Any suggestions??
Please?!?! -
I would post a new thread that is dedicated to your issue as it will be extremely difficult and confusing to separate your question and detail here from the details of the person whose thread it is. It will be much cleaner and less confusing that way.
What OS are you running?
Drives blinking just means that they are being accessed, that alone is not a problem; it just indicated activity.
What issue are you seeing? Just slow access rates?
-
@aroelle said:
Are the old drives somehow incompatible to the new ones??
Drive ages do not cause incompatibilities. Drives are not compatible or not compatible with each other.
-
@aroelle said:
But what is wrong with the first logical drive?
That would be the question. What makes you feel that something is wrong?
