16TB spinning rust is here
-
@Obsolesce said in 16TB spinning rust is here:
@Pete-S said in 16TB spinning rust is here:
I think you can divide 3.5" drives into three major categories:
- Desktop drives - Consumer 8/24 usage, 2y warranty, SATA - WD Desktop, Seagate Desktop etc
- NAS drives - "Pro/semi-pro" 24/7 usage, 3y warranty, SATA - WD Red, Seagate Iron Wolf etc
- Enterprise drives - Heavy 24/7 usage, 5y warranty, SATA or SAS - WD Ultrastar, Seagate Exos etc
I think large desktop drives has become a niche market since that is not what people buy. And if you look at 10TB or more, the NAS drives and the enterprise drives cost almost the same but enterprise drives always have 5 year warranty so...
Unless trends will change I think you'll see a lot more enterprise drives used by Backblaze. I think in general they will always pick the lowest cost per TB drives they can find in volumes they can buy.
Assuming everything keeps moving to the cloud, there are going to be huge volumes of hyperscale high capacity drives sold and less of everything else. Considering energy cost and density, hyperscale companies are always looking for the highest capacity drives.
WD Blue 3/year warranty
WD Black 5/year warrantyBoth are non-Enterprise drives, and labeled as Desktop drives.
The fact that Backblaze used 34,737 Seagate st4000dm000 desktop consumer level drives that had a failure rate of 2.13% is by no means surprising. Most of the drives they use are not enterprise drives.
They must do it like that because they must think it's cheaper to deal with failed hard drives for the bulk of that tier of data.
However, looking at the HGST Enterprise grade drives they used almost 10,000 of (hms5c4040ble640)... those had a significantly lower failure rate.
Perhaps it's cheaper to go with Desktop drives for certain tiers of data, dealing with the failure rate, and for other data tiers, they choose Enterprise drives with much lower failure rates.
WD Blue for HDDs only has 2y warranty and is not made in large capacities. https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/internal-drives/wd-blue-hdd/data-sheet-wd-blue-pc-hard-drives-2879-771436.pdf
Regardless of that Backblaze have in the past said they pick the cheapest drives because the lower drive replacement cost doesn't offset the higher cost for better drives. Maybe they do pick different drives for different uses but I can't see why they should really. They have enough redundancy regardless.
But as I said, just looking at the trend last 12 months or so, I think you will see more enterprise drives in their lineup because prices are basically the same. Looking at our own prices I see that we pay only 2% more for the WD Ultrastar 10TB compared to the WD Red 10TB.
-
@Pete-S right, they don’t tier anything like that. Just jack it all into pods.
-
@Pete-S said in 16TB spinning rust is here:
@Obsolesce said in 16TB spinning rust is here:
@Pete-S said in 16TB spinning rust is here:
I think you can divide 3.5" drives into three major categories:
- Desktop drives - Consumer 8/24 usage, 2y warranty, SATA - WD Desktop, Seagate Desktop etc
- NAS drives - "Pro/semi-pro" 24/7 usage, 3y warranty, SATA - WD Red, Seagate Iron Wolf etc
- Enterprise drives - Heavy 24/7 usage, 5y warranty, SATA or SAS - WD Ultrastar, Seagate Exos etc
I think large desktop drives has become a niche market since that is not what people buy. And if you look at 10TB or more, the NAS drives and the enterprise drives cost almost the same but enterprise drives always have 5 year warranty so...
Unless trends will change I think you'll see a lot more enterprise drives used by Backblaze. I think in general they will always pick the lowest cost per TB drives they can find in volumes they can buy.
Assuming everything keeps moving to the cloud, there are going to be huge volumes of hyperscale high capacity drives sold and less of everything else. Considering energy cost and density, hyperscale companies are always looking for the highest capacity drives.
WD Blue 3/year warranty
WD Black 5/year warrantyBoth are non-Enterprise drives, and labeled as Desktop drives.
The fact that Backblaze used 34,737 Seagate st4000dm000 desktop consumer level drives that had a failure rate of 2.13% is by no means surprising. Most of the drives they use are not enterprise drives.
They must do it like that because they must think it's cheaper to deal with failed hard drives for the bulk of that tier of data.
However, looking at the HGST Enterprise grade drives they used almost 10,000 of (hms5c4040ble640)... those had a significantly lower failure rate.
Perhaps it's cheaper to go with Desktop drives for certain tiers of data, dealing with the failure rate, and for other data tiers, they choose Enterprise drives with much lower failure rates.
WD Blue for HDDs only has 2y warranty and is not made in large capacities. https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/internal-drives/wd-blue-hdd/data-sheet-wd-blue-pc-hard-drives-2879-771436.pdf
Regardless of that Backblaze have in the past said they pick the cheapest drives because the lower drive replacement cost doesn't offset the higher cost for better drives. Maybe they do pick different drives for different uses but I can't see why they should really. They have enough redundancy regardless.
But as I said, just looking at the trend last 12 months or so, I think you will see more enterprise drives in their lineup because prices are basically the same. Looking at our own prices I see that we pay only 2% more for the WD Ultrastar 10TB compared to the WD Red 10TB.
Yeah, no idea why they buy so many Desktop drives....
More results broken down:
-
Because 2% more is a lot.
-
@JaredBusch said in 16TB spinning rust is here:
Because 2% more is a lot.
Especially at the scale in which they purchase drives.
-
@DustinB3403 said in 16TB spinning rust is here:
@JaredBusch said in 16TB spinning rust is here:
Because 2% more is a lot.
Especially at the scale in which they purchase drives.
Most of the desktops drives you see in their stats are small drives. They are not buying small drives anymore. Only 8 TB and up. So mostly enterprise drives.
With their volume they get a good price anyway so who knows if they pay 2% more or not. Also if you get a longer warranty that might offset the price difference a little bit.
This is how much data they have stored on each type of drive:
-
@Pete-S said in 16TB spinning rust is here:
With their volume they get a good price anyway so who knows if they pay 2% more or not.
We have to assume so, as they would get volume discounts on consumer drives, too.
-
@scottalanmiller said in 16TB spinning rust is here:
@Pete-S said in 16TB spinning rust is here:
With their volume they get a good price anyway so who knows if they pay 2% more or not.
We have to assume so, as they would get volume discounts on consumer drives, too.
Exactly - sure they get volume discounts.. but the consumer at volume is still likely 2% less than enterprise at volume.. that's cash in their pockets...
Of course as they drop out from the smaller consumer grade drives, that becomes less and less of an issue.
-
@Dashrender said in 16TB spinning rust is here:
@scottalanmiller said in 16TB spinning rust is here:
@Pete-S said in 16TB spinning rust is here:
With their volume they get a good price anyway so who knows if they pay 2% more or not.
We have to assume so, as they would get volume discounts on consumer drives, too.
Exactly - sure they get volume discounts.. but the consumer at volume is still likely 2% less than enterprise at volume.. that's cash in their pockets...
Of course as they drop out from the smaller consumer grade drives, that becomes less and less of an issue.
@Dashrender said in 16TB spinning rust is here:
@scottalanmiller said in 16TB spinning rust is here:
@Pete-S said in 16TB spinning rust is here:
With their volume they get a good price anyway so who knows if they pay 2% more or not.
We have to assume so, as they would get volume discounts on consumer drives, too.
Exactly - sure they get volume discounts.. but the consumer at volume is still likely 2% less than enterprise at volume.. that's cash in their pockets...
Of course as they drop out from the smaller consumer grade drives, that becomes less and less of an issue.
2% of 100k HDDs has got to be over $200k. The time that goes into replacing them, performance loss, etc can't be overlooked either, which is added costs on top of the drive costs.
-
@Obsolesce said in 16TB spinning rust is here:
2% of 100k HDDs has got to be over $200k. The time that goes into replacing them, performance loss, etc can't be overlooked either, which is added costs on top of the drive costs.
In theory, from old BB studies, consumer and enterprise drives fail at roughly the same rates. Speeds are the bigger differences, and sometimes capacities. Often hard to directly compare as different drive types tend to be made for the different audiences.
-
@scottalanmiller said in 16TB spinning rust is here:
@Obsolesce said in 16TB spinning rust is here:
2% of 100k HDDs has got to be over $200k. The time that goes into replacing them, performance loss, etc can't be overlooked either, which is added costs on top of the drive costs.
In theory, from old BB studies, consumer and enterprise drives fail at roughly the same rates. Speeds are the bigger differences, and sometimes capacities. Often hard to directly compare as different drive types tend to be made for the different audiences.
I haven't seen those. In practice, it seems like a different story according to their charts.
-
@Obsolesce said in 16TB spinning rust is here:
@scottalanmiller said in 16TB spinning rust is here:
@Obsolesce said in 16TB spinning rust is here:
2% of 100k HDDs has got to be over $200k. The time that goes into replacing them, performance loss, etc can't be overlooked either, which is added costs on top of the drive costs.
In theory, from old BB studies, consumer and enterprise drives fail at roughly the same rates. Speeds are the bigger differences, and sometimes capacities. Often hard to directly compare as different drive types tend to be made for the different audiences.
I haven't seen those. In practice, it seems like a different story according to their charts.
Older blog, but BB stated that consumer drives outperformed enterprise drives on failure rates...
https://www.backblaze.com/blog/enterprise-drive-reliability/
-
@scottalanmiller said in 16TB spinning rust is here:
@G-I-Jones said in 16TB spinning rust is here:
@scottalanmiller what's the general consensus here? Do we not buy Seagate because they fail more? Genuinely curious.
Generally avoid them, yes. Big usage shops like BB have had issues with them, and repair shops have advised that they see the failure rates on them in the wild being very high.
In 3.5" SAS we have plenty of 60-bay JBODs out there filled with Seagate NearLine drives. Our failure rates over the five year solution life across the board are virtually nil.
We had one drive fail in a Dell MD3060e but who the chicken knows what drive was installed in there in the first place.
We avoid Seagate SAS SSDs due to firmware problems that we've seen in the same settings.
We've been running with HGST now Western Digital drives and JBODs and have seen similar virtually nil fail rates.
As far as 2.5" 10K SAS goes, we have lots of Seagate and HGST out there with, again, very little in the way of fail rates.
Hard Drives have become quite reliable as a whole in the settings we deploy them in.
We've not deployed spinning rust in a workstation/desktop/laptop for years now. So, that negates the consumer side of things. We just don't.
-
-
@Obsolesce said in 16TB spinning rust is here:
@PhlipElder said in 16TB spinning rust is here:
JBODs
Totally fine is its some kind of RAIN. Actually i'm looking at building a CEPH cluster, Dell&RedHat recommend using single-drive RAID0s
-
@Kris_K said in 16TB spinning rust is here:
@Obsolesce said in 16TB spinning rust is here:
@PhlipElder said in 16TB spinning rust is here:
JBODs
Totally fine is its some kind of RAIN. Actually i'm looking at building a CEPH cluster, Dell&RedHat recommend using single-drive RAID0s
Our Storage Spaces Direct (S2D) hyper-converged clusters use IT Mode SAS HBAs for the SAS/SATA disk setup. NVMe are straight-through by default.
-
Many systems today can use JBOD, the system itself takes care of any reliability needs. Exchange added that back in 2013.
-
@Kris_K said in 16TB spinning rust is here:
@Obsolesce said in 16TB spinning rust is here:
@PhlipElder said in 16TB spinning rust is here:
JBODs
Totally fine is its some kind of RAIN. Actually i'm looking at building a CEPH cluster, Dell&RedHat recommend using single-drive RAID0s
No it was about the 'term', nevermind lol.
-
@PhlipElder said in 16TB spinning rust is here:
Hard Drives have become quite reliable as a whole in the settings we deploy them in.
Yes, even drives that fail "a lot" doesn't imply that you are likely to ever see one. If the best drives fail .000001% of the time, and the worst drives fail 300% as often, that's still only .000003% of the time.
-
@Kris_K said in 16TB spinning rust is here:
@Obsolesce said in 16TB spinning rust is here:
@PhlipElder said in 16TB spinning rust is here:
JBODs
Totally fine is its some kind of RAIN. Actually i'm looking at building a CEPH cluster, Dell&RedHat recommend using single-drive RAID0s
If you add RAIN, it's not a JBOD
Then it is a RAIN.
