Deciding Between Hardware and Software RAID in My FreeNAS Deployment
-
@saniplastic said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
but our server is under load.
How much load is it under? ZFS software RAID uses some CPU and a small amount of RAM. It would be rare for any server today to be under so much load that this would be noticed for RAID functions. Possible, but rare.
To determine the load of ZFS... what RAID level are you implementing in ZFS currently? RAID 1 and 10 (mirroring) use essentially no resources. RAIDZ levels use a bit of CPU with RAIDZ being very little and RAIDZ3 being a lot and RAIDZ2 being in the middle.
-
@saniplastic said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
should i use hardware raid or continue with exist config ?
This will mostly depend on the load description, RAID level desired, and cache info. If you have zero cache, then you will not want to try to use the RAID card to do anything unnecessary. Cache-less RAID of any sort is all but useless.
If you run RAIDZ3, then you either have to change your RAID planning to use hardware, or you are stuck with ZFS RAID.
If you aren't using RAIDZ3, and you have cache on the RAID card, then almost certainly you want to use your RAID card to handle all of your RAID. Having the card and using it for the physical drive management, and using ZFS just for software RAID a level up without complete visibility down the stack is the worst of all options. It means that the holistic disk monitoring, management, and repair cannot be done by any layer of the system. So, for example, ZFS could repair things in case of a failure, but doesn't know about the failure. And the hardware RAID can know about the failure, but has no power to fix it.
The general rule is that if you have hardware RAID that requires that you do the "RAID 0 per drive" trick, it means that the decision to not use ZFS RAID was made at the time of the hardware purchase and that's all that there is to it. ZFS' software RAID is the best in the business, but is nearly useless when it can't talk directly to the drives. But the P410 is a great RAID card, also one of the best in the business. So a great option to use it and not use ZFS for RAID at all.
-
This post is deleted! -
@scottalanmiller said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
@saniplastic said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
in my freenas server, each disk is on Raid0 Hardware Raid P410i
How much cache does this have?
1G
-
@scottalanmiller said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
Cache-less RAID of any sort is all but useless
thanks for reply
before The more I read, the more I get confused :
FreeNAS
11.2- User Guide, Release 11.2 in page 322 says that:"ZFS was designed for commodity disks so no RAID controller is needed. While ZFS can also be
used with a RAID controller, it is recommended that the controller be put into JBOD mode so that ZFS has full control of the disks"it says it is not needed ( not explicitly don't use ) and While ZFS can also be used with a RAID controller.
and in page 110 says that:
To prevent problems, do not enable the S.M.A.R.T. service if the disks are controlled by a RAID controller. It is the job of the controller to monitor S.M.A.R.T. and mark drives as Predictive Failure when they trip.
it means this is possible to use RAID controller by freenas
But in freenas forum every time you ask for Hardware RAID and zfs , all strictly says NO, DO NOT DO THAT.
When I read your article, I was happy to find out.I use P410i (1G chache) on DL380 G7 and as i said raid0 each drive and present separate raid0 to freenas.
i have one mirror pool and one Raidz1 pool and one Raidz2 pool.If I understand well because raid controller have 1G cache i should use Hardware RAID and present one logical drive to freenas and not using zfs raid.
am i right?
-
This thread covers exactly what you are asking about.
https://mangolassi.it/topic/12047/zfs-is-perfectly-safe-on-hardware-raid/ -
The thing that I'm surprised hasn't happened yet is - Why hasn't @scottalanmiller asked why you're using FreeNAS instead of just using a Linux Distro and enabling sharing? (i.e. NFS or SAMBA)
-
@Dashrender said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
This thread covers exactly what you are asking about.
https://mangolassi.it/topic/12047/zfs-is-perfectly-safe-on-hardware-raid/i read that several time and find out zfs can implement on top of the raid controller.but can not get my answer related to my question.
-
@Dashrender said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
This thread covers exactly what you are asking about.
https://mangolassi.it/topic/12047/zfs-is-perfectly-safe-on-hardware-raid/that's where this was originally posted. But the question wasn't if it was safe, but which was better given the specific situation. Since both are options.
-
@Dashrender said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
The thing that I'm surprised hasn't happened yet is - Why hasn't @scottalanmiller asked why you're using FreeNAS instead of just using a Linux Distro and enabling sharing? (i.e. NFS or SAMBA)
Because it's already in use and not part of the question.
-
@saniplastic said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
@Dashrender said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
This thread covers exactly what you are asking about.
https://mangolassi.it/topic/12047/zfs-is-perfectly-safe-on-hardware-raid/i read that several time and find out zfs can implement on top of the raid controller.but can not get my answer related to my question.
I'm pretty sure that I answered your question here in this thread... unless you have weird stuff going on (all of which I outlined), the hardware RAID is going to be way better for you... basically for all of the reasons that the ZFS crowd claims it to be worse. All of the features that they point out with ZFS that requires that you not have hardware RAID, hardware RAID gives you as long as you don't use ZFS for software RAID
The one thing that you should avoid is using ZFS software RAID on top of a RAID controller that can't do pass through - that's the one situation where the error detection and tools are split and can't do their job from either side. Both protection mechanisms get bypassed. Should be one or the other.
-
@scottalanmiller said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
@Dashrender said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
The thing that I'm surprised hasn't happened yet is - Why hasn't @scottalanmiller asked why you're using FreeNAS instead of just using a Linux Distro and enabling sharing? (i.e. NFS or SAMBA)
Because it's already in use and not part of the question.
If that was true - then there would be no point in the question because the decision of RAID on HW or not would be already decided - well i suppose FreeNAS could have been installed on a separate disk, then the actual storage be on the RAID attached drives.. but that seems like over kill.
-
To follow up to an offline request for more info...
Why do you need to avoid splitting RAID duties between a controller, and ZFS?
First, this is a general rule and not specific to any of the tech here. The RAID system just need to be intact, wherever it is.
Now, why is this?
In order for the data integrity systems to be able to do their job and protect the data, there needs to be deep insight and coordination between logical levels and the hardware itself. This is so that components looking for hardware failure can verify higher up the chain if there is a way to reconstruct the data. The layer doing the scrubbing and the parity or mirroring needs to know if the hardware integrity is still there and, if not, what it needs to do to recover. Either layer can do this, hardware or software, but what it can't do is talk between the two.
So if you have all hardware RAID, physical and logical layers can communicate to verify integrity. If you have all software RAID, you can do the same. But if the hardware layer and the logical layer cannot talk to one another, and one of them detects a problem, there is no way to do anything about it. Your overall system has all this information and data protection, but no way to do anything about it - the left hand doesn't know what the right hand is doing.
Hence why it is so important to pick one or the other. Riding the fence and trying to leverage both is where problems arise.
-
@Dashrender said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
@scottalanmiller said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
@Dashrender said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
The thing that I'm surprised hasn't happened yet is - Why hasn't @scottalanmiller asked why you're using FreeNAS instead of just using a Linux Distro and enabling sharing? (i.e. NFS or SAMBA)
Because it's already in use and not part of the question.
If that was true - then there would be no point in the question because the decision of RAID on HW or not would be already decided -
No, because you skipped over the and portion. Yes, the split HW RAID and ZFS Software RAID is already in use, but it is the whole question.
-
@scottalanmiller said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
To follow up to an offline request for more info...
Why do you need to avoid splitting RAID duties between a controller, and ZFS?
First, this is a general rule and not specific to any of the tech here. The RAID system just need to be intact, wherever it is.
Now, why is this?
In order for the data integrity systems to be able to do their job and protect the data, there needs to be deep insight and coordination between logical levels and the hardware itself. This is so that components looking for hardware failure can verify higher up the chain if there is a way to reconstruct the data. The layer doing the scrubbing and the parity or mirroring needs to know if the hardware integrity is still there and, if not, what it needs to do to recover. Either layer can do this, hardware or software, but what it can't do is talk between the two.
So if you have all hardware RAID, physical and logical layers can communicate to verify integrity. If you have all software RAID, you can do the same. But if the hardware layer and the logical layer cannot talk to one another, and one of them detects a problem, there is no way to do anything about it. Your overall system has all this information and data protection, but no way to do anything about it - the left hand doesn't know what the right hand is doing.
Hence why it is so important to pick one or the other. Riding the fence and trying to leverage both is where problems arise.
thanks all for reply
so i need:
1- backup my vms and recreate raid on Hardware and just present one logical drive to ZFS.
2-disable S.M.A.R.T service.
3-just use file system and not software RAID system of zfs. -
@saniplastic said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
1- backup my vms and recreate raid on Hardware and just present one logical drive to ZFS.
2-disable S.M.A.R.T service.
3-just use file system and not software RAID system of zfs.Yes, that's what I would do here. Simple, safe, effective.
-
@saniplastic said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
1- backup my vms and recreate raid on Hardware and just present one logical drive to ZFS.
Just an FYI: it's not a backup when it's the only data. If you back up your VMs, then delete the originals, you no longer have a backup. All you have then is original data with no backup! Proceed wisely.
-
@Obsolesce said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
@saniplastic said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
1- backup my vms and recreate raid on Hardware and just present one logical drive to ZFS.
Just an FYI: it's not a backup when it's the only data. If you back up your VMs, then delete the originals, you no longer have a backup. All you have then is original data with no backup! Proceed wisely.
you right
i should have 2 series of backup.
thanks -
@saniplastic said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
@scottalanmiller said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
To follow up to an offline request for more info...
Why do you need to avoid splitting RAID duties between a controller, and ZFS?
First, this is a general rule and not specific to any of the tech here. The RAID system just need to be intact, wherever it is.
Now, why is this?
In order for the data integrity systems to be able to do their job and protect the data, there needs to be deep insight and coordination between logical levels and the hardware itself. This is so that components looking for hardware failure can verify higher up the chain if there is a way to reconstruct the data. The layer doing the scrubbing and the parity or mirroring needs to know if the hardware integrity is still there and, if not, what it needs to do to recover. Either layer can do this, hardware or software, but what it can't do is talk between the two.
So if you have all hardware RAID, physical and logical layers can communicate to verify integrity. If you have all software RAID, you can do the same. But if the hardware layer and the logical layer cannot talk to one another, and one of them detects a problem, there is no way to do anything about it. Your overall system has all this information and data protection, but no way to do anything about it - the left hand doesn't know what the right hand is doing.
Hence why it is so important to pick one or the other. Riding the fence and trying to leverage both is where problems arise.
thanks all for reply
so i need:
1- backup my vms and recreate raid on Hardware and just present one logical drive to ZFS.
2-disable S.M.A.R.T service.
3-just use file system and not software RAID system of zfs.Where are you disabling SMART?
-
@Dashrender said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
@saniplastic said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
@scottalanmiller said in Deciding Between Hardware and Software RAID in My FreeNAS Deployment:
To follow up to an offline request for more info...
Why do you need to avoid splitting RAID duties between a controller, and ZFS?
First, this is a general rule and not specific to any of the tech here. The RAID system just need to be intact, wherever it is.
Now, why is this?
In order for the data integrity systems to be able to do their job and protect the data, there needs to be deep insight and coordination between logical levels and the hardware itself. This is so that components looking for hardware failure can verify higher up the chain if there is a way to reconstruct the data. The layer doing the scrubbing and the parity or mirroring needs to know if the hardware integrity is still there and, if not, what it needs to do to recover. Either layer can do this, hardware or software, but what it can't do is talk between the two.
So if you have all hardware RAID, physical and logical layers can communicate to verify integrity. If you have all software RAID, you can do the same. But if the hardware layer and the logical layer cannot talk to one another, and one of them detects a problem, there is no way to do anything about it. Your overall system has all this information and data protection, but no way to do anything about it - the left hand doesn't know what the right hand is doing.
Hence why it is so important to pick one or the other. Riding the fence and trying to leverage both is where problems arise.
thanks all for reply
so i need:
1- backup my vms and recreate raid on Hardware and just present one logical drive to ZFS.
2-disable S.M.A.R.T service.
3-just use file system and not software RAID system of zfs.Where are you disabling SMART?
In the OS, I assume.
