SAS SSD vs SAS HDD in a RAID 10?
-
I'm planning the build on a new server. I originally intended on putting 8 x "900GB 15K RPM SAS 12Gbps 512e" drives into a RAID 10 config using an H740P adapter, but then I saw that there are quite a few options for SAS SSD. I haven't really learned too much about the differences of putting SSD in RAID and how it compares to HDD in RAID, so I wanted to see if anyone here (Scott) had any input on the matter.
Example: Would it be worth putting, say, 6 x "1.6TB SSD SAS Mix Use 12Gbps 512e" drives into a RAID 10 instead? Is there a better approach with SSD in RAID?
-
OBR5 is the standard if you are going to be using an SSD
-
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
OBR5 is the standard if you are going to be using an SSD
Are there any good sources that express that as best practice? I'm looking for myself now too and by the way....
-
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
OBR5 is the standard if you are going to be using an SSD
Are there any good sources that express that as best practice? I'm looking for myself now too and by the way....
The reason OBR5 is acceptable when using SSDs is that there is no mathematical proof that you can or would hit a URE with SSDs.
An SSD doesn't limp along like an winchester drive would if it was to fail. It would just fail and be replaced. The recovery time of an SSD array would be multiple magnitudes faster than a winchester equivalent. So your chances of a second drive failing during a recovery are greatly reduced because of the performance difference between the individual technologies.
SSDs aren't a best practice, but an SSD OBR5 is acceptable. If you wanted even more protection you would go with OBR6. Then next step if you needed even more protection and performance would be OBR10.
Since an SSD array will be way faster than its classic counterpart you start with the minimal requirements and go up. OBR10 means you're giving up half of your usable space to protect from a failed disk.
OBR5 and 6 don't require as steep of a storage sacrifice and are thus acceptable since the recovery time and risk offset is so much better than when compared to classic disks.
Plus the cost per GB is still more for an SSD than a classic drive. So saving your usable space is a premium consideration for an SSD array.
-
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
OBR5 is the standard if you are going to be using an SSD
Are there any good sources that express that as best practice? I'm looking for myself now too and by the way....
The reason OBR5 is acceptable when using SSDs is that there is no mathematical proof that you can or would hit a URE with SSDs.
An SSD doesn't limp along like an winchester drive would if it was to fail. It would just fail and be replaced. The recovery time of an SSD array would be multiple magnitudes faster than a winchester equivalent. So your chances of a second drive failing during a recovery are greatly reduced because of the performance difference between the individual technologies.
SSDs aren't a best practice, but an SSD OBR5 is acceptable. If you wanted even more protection you would go with OBR6. Then next step if you needed even more protection and performance would be OBR10.
Since an SSD array will be way faster than its classic counterpart you start with the minimal requirements and go up. OBR10 means you're giving up half of your usable space to protect from a failed disk.
OBR5 and 6 don't require as steep of a storage sacrifice and are thus acceptable since the recovery time and risk offset is so much better than when compared to classic disks.
Plus the cost per GB is still more for an SSD than a classic drive. So saving your usable space is a premium consideration for an SSD array.
Ah, good info, thanks for that.
This is for an SQL database server with about 2 TB of growing storage and while we have excellent backups, I really want to minimize any possible outage of this server, hence the RAID 10.
-
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
OBR5 is the standard if you are going to be using an SSD
Are there any good sources that express that as best practice? I'm looking for myself now too and by the way....
The reason OBR5 is acceptable when using SSDs is that there is no mathematical proof that you can or would hit a URE with SSDs.
An SSD doesn't limp along like an winchester drive would if it was to fail. It would just fail and be replaced. The recovery time of an SSD array would be multiple magnitudes faster than a winchester equivalent. So your chances of a second drive failing during a recovery are greatly reduced because of the performance difference between the individual technologies.
SSDs aren't a best practice, but an SSD OBR5 is acceptable. If you wanted even more protection you would go with OBR6. Then next step if you needed even more protection and performance would be OBR10.
Since an SSD array will be way faster than its classic counterpart you start with the minimal requirements and go up. OBR10 means you're giving up half of your usable space to protect from a failed disk.
OBR5 and 6 don't require as steep of a storage sacrifice and are thus acceptable since the recovery time and risk offset is so much better than when compared to classic disks.
Plus the cost per GB is still more for an SSD than a classic drive. So saving your usable space is a premium consideration for an SSD array.
Ah, good info, thanks for that.
This is for an SQL database server with about 2 TB of growing storage and while we have excellent backups, I really want to minimize any possible outage of this server, hence the RAID 10.
RAID 6 can always protect against two simultaneous disk failures. Because RAID 6 doubles up its parity data, it can withstand two disks failing at the same time. Whether RAID 10 can handle two disk failures simultaneously depends on where the failed disks occur.
OBR6 is likely your better option, more space for your database to grow and about as reliable as OBR10.
-
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
OBR5 is the standard if you are going to be using an SSD
Are there any good sources that express that as best practice? I'm looking for myself now too and by the way....
The reason OBR5 is acceptable when using SSDs is that there is no mathematical proof that you can or would hit a URE with SSDs.
An SSD doesn't limp along like an winchester drive would if it was to fail. It would just fail and be replaced. The recovery time of an SSD array would be multiple magnitudes faster than a winchester equivalent. So your chances of a second drive failing during a recovery are greatly reduced because of the performance difference between the individual technologies.
SSDs aren't a best practice, but an SSD OBR5 is acceptable. If you wanted even more protection you would go with OBR6. Then next step if you needed even more protection and performance would be OBR10.
Since an SSD array will be way faster than its classic counterpart you start with the minimal requirements and go up. OBR10 means you're giving up half of your usable space to protect from a failed disk.
OBR5 and 6 don't require as steep of a storage sacrifice and are thus acceptable since the recovery time and risk offset is so much better than when compared to classic disks.
Plus the cost per GB is still more for an SSD than a classic drive. So saving your usable space is a premium consideration for an SSD array.
Ah, good info, thanks for that.
This is for an SQL database server with about 2 TB of growing storage and while we have excellent backups, I really want to minimize any possible outage of this server, hence the RAID 10.
RAID 6 can always protect against two simultaneous disk failures. Because RAID 6 doubles up its parity data, it can withstand two disks failing at the same time. Whether RAID 10 can handle two disk failures simultaneously depends on where the failed disks occur.
OBR6 is likely your better option, more space for your database to grow and about as reliable as OBR10.
Great points. Thanks for helping me out!
-
OBR6 would give you approximately 10TB worth of usable space (with the SSDs you listed) where as OBR10 would give you 6.5TB of usable space.
-
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
OBR6 would give you approximately 10TB worth of usable space (with the SSDs you listed) where as OBR10 would give you 6.5TB of usable space.
If I did RAID 6, I would only get 5 drives which would give me about 4.4 TB of storage. We really only need about an additional 2 TB or so but I'm giving us about 1 extra TB for leg room.
-
I went with the assumption of 8 drives for both estimates, but sure. For OBR10 you need more disks to account for the lost capacity.
-
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
OBR5 is the standard if you are going to be using an SSD
Are there any good sources that express that as best practice? I'm looking for myself now too and by the way....
There can never be a best practice of this sort. It's standard practice to start with RAID 5 for SSD due to the risk types and levels, but not on HDs for the same reason. RAID 10 tends to saturate RAID controllers with SSD, but not with HDs.
As with all RAID, it comes down to price / risk / performance. And for most deployments, RAID 5 gives the best blend with SSDs; and RAID 10 gives the best blend for HDs. But in both cases, RAID 6 is the second most popular choice, and RAID 10 is an option with SSDs.
With SSDs, you rarely do RAID 10. If you really need the speed, you tend to do RAID 1 with giant NVMe cards instead.
-
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
OBR6 would give you approximately 10TB worth of usable space (with the SSDs you listed) where as OBR10 would give you 6.5TB of usable space.
If I did RAID 6, I would only get 5 drives which would give me about 4.4 TB of storage. We really only need about an additional 2 TB or so but I'm giving us about 1 extra TB for leg room.
RAID 6 will give you loads of protection. And without realistic URE worries, it can approach RAID 10 reliability, at a fraction of the cost.
-
@scottalanmiller said in SAS SSD vs SAS HDD in a RAID 10?:
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
OBR5 is the standard if you are going to be using an SSD
Are there any good sources that express that as best practice? I'm looking for myself now too and by the way....
There can never be a best practice of this sort. It's standard practice to start with RAID 5 for SSD due to the risk types and levels, but not on HDs for the same reason. RAID 10 tends to saturate RAID controllers with SSD, but not with HDs.
As with all RAID, it comes down to price / risk / performance. And for most deployments, RAID 5 gives the best blend with SSDs; and RAID 10 gives the best blend for HDs. But in both cases, RAID 6 is the second most popular choice, and RAID 10 is an option with SSDs.
With SSDs, you rarely do RAID 10. If you really need the speed, you tend to do RAID 1 with giant NVMe cards instead.
Yeah, sorry, I guess I shouldn't have said "best practice". I was more or less looking for some information that would help validate what Dustin said. I wanted to look into it more and educate myself as much as possible.
Well I think if I am able to go with the SSD drives, I will do a RAID 6. I am creating a few different server builds as options that display different levels of performance and cost.
Speaking of my RAID card, I am looking at the H740P which has 8GB of NV cache memory and flash backed cache. I still need to educate myself on this stuff as well because I'm not sure if this is overkill or not. My other option was the H330, which has none of that.
EDIT: Nevermind on the H330 doesn't offer RIAD 6 as an option.
-
@dave247 Really no need to buy SATA/SAS SSDs anymore for new servers when U.2 NVMe SSDs are readily available and Dell have plenty of servers that can do hotplug NVMe drives. And NVMe are very superior in every way to SATA/SAS.
U.2 is 2.5" drives but fatter than regular SSDs and have better cooling and 4 lane PCIe bus connection. Comes in sizes from about 0.5TB up to 8TB per drive and whatever DWPD you need.
-
@pete-s except the cost per GB is even more for U.2 drives than it is for the traditional formats.
-
Before the RAID array type is selected, it would be great to know the workload in terms of write operations. For RAID5/6 write performance will be significantly lower than that of RAID10 on the same disks. Just bear that in mind.
-
@darek-hamann said in SAS SSD vs SAS HDD in a RAID 10?:
Before the RAID array type is selected, it would be great to know the workload in terms of write operations. For RAID5/6 write performance will be significantly lower than that of RAID10 on the same disks. Just bear that in mind.
While true that RAID5/6 would be slower in terms of IOPS, the array if made of SSD would still be way faster that an array 5/6 of classic disks.
So we can safely guess that performance of the array will be a critical factor.
-
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
@pete-s except the cost per GB is even more for U.2 drives than it is for the traditional formats.
Depends on how you calculate. One NVMe drive, say 4TB, it will beat the performance of four 1TB SATA SSD drives in RAID0. If you take 4x1TB drives and a raid controller the 4TB NVMe will be cheaper or similar. So per IOPS the NVMe drives are cheaper. And total failure rate is lower because you need fewer drives to accomplish the same thing.
If you on the other side take a 4TB NVMe drive compared to a 4TB SATA SSD you will pay about 25% premium for the NVMe drive. Intel for instance have their DC series of drives that are available both in SATA and NVMe which makes it easier to compare apples to apples.
-
@pete-s said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
@pete-s except the cost per GB is even more for U.2 drives than it is for the traditional formats.
Depends on how you calculate. One NVMe drive, say 4TB, it will beat the performance of four 1TB SATA SSD drives in RAID0. If you take 4x1TB drives and a raid controller the 4TB NVMe will be cheaper or similar. So per IOPS the NVMe drives are cheaper. And total failure rate is lower because you need fewer drives to accomplish the same thing.
If you on the other side take a 4TB NVMe drive compared to a 4TB SATA SSD you will pay about 25% premium for the NVMe drive. Intel for instance have their DC series of drives that are available both in SATA and NVMe which makes it easier to compare apples to apples.
Except in your RAID example above you totally miss that the OP wants hardware protection. So a RAID card is required regardless (either onboard real RAID 1 for your dual drive setup, or add-in card).
-
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@scottalanmiller said in SAS SSD vs SAS HDD in a RAID 10?:
@dave247 said in SAS SSD vs SAS HDD in a RAID 10?:
@dustinb3403 said in SAS SSD vs SAS HDD in a RAID 10?:
OBR5 is the standard if you are going to be using an SSD
Are there any good sources that express that as best practice? I'm looking for myself now too and by the way....
There can never be a best practice of this sort. It's standard practice to start with RAID 5 for SSD due to the risk types and levels, but not on HDs for the same reason. RAID 10 tends to saturate RAID controllers with SSD, but not with HDs.
As with all RAID, it comes down to price / risk / performance. And for most deployments, RAID 5 gives the best blend with SSDs; and RAID 10 gives the best blend for HDs. But in both cases, RAID 6 is the second most popular choice, and RAID 10 is an option with SSDs.
With SSDs, you rarely do RAID 10. If you really need the speed, you tend to do RAID 1 with giant NVMe cards instead.
Yeah, sorry, I guess I shouldn't have said "best practice". I was more or less looking for some information that would help validate what Dustin said. I wanted to look into it more and educate myself as much as possible.
Well I think if I am able to go with the SSD drives, I will do a RAID 6. I am creating a few different server builds as options that display different levels of performance and cost.
Speaking of my RAID card, I am looking at the H740P which has 8GB of NV cache memory and flash backed cache. I still need to educate myself on this stuff as well because I'm not sure if this is overkill or not. My other option was the H330, which has none of that.
EDIT: Nevermind on the H330 doesn't offer RIAD 6 as an option.
Dave - why do you trust HDD in RAID 10, but don't trust Enterprise SSD drives in RAID 5?
This really boils down to math. The URE rate of Enterprise SSD drives starts one level higher than that of HDDs, which is the reason that Scott started preaching SSDs in RAID 5 years ago.
As you said - definitely dive into the whys and you'll see that unless your needs are very special, it's pretty likely that RAID 5 SSDs will do the job for you.
But all that is really moot until you know what your real requirements are. If you need 10,000 IOPs, can you even reasonably get that from HDDS in RAID 10? what if you need 100,000 or 1 million?
This information is way more important to know before you look at what to buy, because it basically tells you were to look to get the parts to meet the goal.
