Is this server strategy reckless and/or insane?
-
RAID 5 gives you N-1 Read and worse for write.
-
@dashrender said in Is this server strategy reckless and/or insane?:
RAID 5 gives you N-1 Read and worse for write.
No, all RAID gives you N reads.
-
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@dashrender said in Is this server strategy reckless and/or insane?:
RAID 5 gives you N-1 Read and worse for write.
No, all RAID gives you N reads.
Is the explanation for slower reads for Raid 5 because it's doing other stuff on the drives while trying to read ( like writing the parity data ), or should it be simliar to a Raid 0 of the same number of drives?
-
@creayt said in Is this server strategy reckless and/or insane?:
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@dashrender said in Is this server strategy reckless and/or insane?:
RAID 5 gives you N-1 Read and worse for write.
No, all RAID gives you N reads.
Is the explanation for slower reads for Raid 5 because it's doing other stuff on the drives while trying to read ( like writing the parity data ), or should it be simliar to a Raid 0 of the same number of drives?
Strangely enough on this hardware a Raid 0 of just 2 of these SSDs dramatically outperforms a Raid 5 of 5 of the same drives for reads in my first test ( 500 MiB ), as in 1 GB/s faster Seq Q32TI and almost 3 times faster Seq in Crystal, will post the full results in a sec.
-
@creayt said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@scottalanmiller said in Is this server strategy reckless and/or insane?:
@dashrender said in Is this server strategy reckless and/or insane?:
RAID 5 gives you N-1 Read and worse for write.
No, all RAID gives you N reads.
Is the explanation for slower reads for Raid 5 because it's doing other stuff on the drives while trying to read ( like writing the parity data ), or should it be simliar to a Raid 0 of the same number of drives?
Strangely enough on this hardware a Raid 0 of just 2 of these SSDs dramatically outperforms a Raid 5 of 5 of the same drives for reads, as in 1 GB/s faster Seq Q32TI and almost 3 times faster Seq in Crystal, will post the full results in a sec.
THat's a controller problem, not a RAID problem. That means that the controller is oversaturated.
-
RAID is RAID, the math is trivial. What gets hard is figuring out what is wrong with a controller, when a RAID implementation is bad, when a cache is kicking in and so forth.
-
Ok these aren't apples to apples, some of the numbers are from the previous config so I'm not saying the Raid 5 to Raid 0 / 10 differences are exactly what they'd be w/ the same number of drives, but the single drive and 2 drive Raid 0 are hopefully helpful in predicting the performance characteristics of 0 at each quantity.
-
We all understand that there are differences with different RAID types.
The point of the matter is you opt'd for RAID0 because you believe you have a need for all of the IOPS in the world, yet don't care about backups.
But you are missing critical pieces of this design like virtualization, ram cache etc to get better, safer results.
-
@dustinb3403 said in Is this server strategy reckless and/or insane?:
We all understand that there are differences with different RAID types.
The point of the matter is you opt'd for RAID0 because you believe you have a need for all of the IOPS in the world, yet don't care about backups.
But you are missing critical pieces of this design like virtualization, ram cache etc to get better, safer results.
Are IOPS what you want for heavy duty users are making database writes concurrently all day long? I don't know much about drive characteristics/performance other than the basic throughput stuff. Because backup is streamed out in realtime that's taken care of as far as I'm concerned, part of what makes Raid 0 a candidate at least.
-
@creayt said in Is this server strategy reckless and/or insane?:
@dustinb3403 said in Is this server strategy reckless and/or insane?:
We all understand that there are differences with different RAID types.
The point of the matter is you opt'd for RAID0 because you believe you have a need for all of the IOPS in the world, yet don't care about backups.
But you are missing critical pieces of this design like virtualization, ram cache etc to get better, safer results.
Are IOPS what you want for heavy duty users are making database writes concurrently all day long? I don't know much about drive characteristics/performance other than the basic throughput stuff. Because backup is streamed out in realtime that's taken care of as far as I'm concerned, part of what makes Raid 0 a candidate at least.
Yes IOPS are the consideration you need to be looking at. What has yet to be answered is how active is this database going to actually be?
Will you have 10,000 people/processes constantly making changes?
-
@dustinb3403 said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@dustinb3403 said in Is this server strategy reckless and/or insane?:
We all understand that there are differences with different RAID types.
The point of the matter is you opt'd for RAID0 because you believe you have a need for all of the IOPS in the world, yet don't care about backups.
But you are missing critical pieces of this design like virtualization, ram cache etc to get better, safer results.
Are IOPS what you want for heavy duty users are making database writes concurrently all day long? I don't know much about drive characteristics/performance other than the basic throughput stuff. Because backup is streamed out in realtime that's taken care of as far as I'm concerned, part of what makes Raid 0 a candidate at least.
Yes IOPS are the consideration you need to be looking at. What has yet to be answered is how active is this database going to actually be?
Will you have 10,000 people/processes constantly making changes?
Ideally more than that, but it'll be a gradual climb. Right now it's in private alpha w/ ~ 100 users and they post stuff all the time. Once I make it public I imagine the content volume will skyrocket.
-
@dustinb3403 said in Is this server strategy reckless and/or insane?:
We all understand that there are differences with different RAID types.
The point of the matter is you opt'd for RAID0 because you believe you have a need for all of the IOPS in the world, yet don't care about backups.
But you are missing critical pieces of this design like virtualization, ram cache etc to get better, safer results.
He's got backups of the data. He's doing devops style backups.
-
@creayt said in Is this server strategy reckless and/or insane?:
@dustinb3403 said in Is this server strategy reckless and/or insane?:
@creayt said in Is this server strategy reckless and/or insane?:
@dustinb3403 said in Is this server strategy reckless and/or insane?:
We all understand that there are differences with different RAID types.
The point of the matter is you opt'd for RAID0 because you believe you have a need for all of the IOPS in the world, yet don't care about backups.
But you are missing critical pieces of this design like virtualization, ram cache etc to get better, safer results.
Are IOPS what you want for heavy duty users are making database writes concurrently all day long? I don't know much about drive characteristics/performance other than the basic throughput stuff. Because backup is streamed out in realtime that's taken care of as far as I'm concerned, part of what makes Raid 0 a candidate at least.
Yes IOPS are the consideration you need to be looking at. What has yet to be answered is how active is this database going to actually be?
Will you have 10,000 people/processes constantly making changes?
Ideally more than that, but it'll be a gradual climb. Right now it's in private alpha w/ ~ 100 users and they post stuff all the time. Once I make it public I imagine the content volume will skyrocket.
MySQL is likely your performance bottleneck there.
-
How is your internet going to serve up all this RAID0 SSD awesomeness?? Do you really have the bandwidth to allow the hardware to be the bottleneck?
-
Copying to the other server is not a backup, FYI.
-
@dashrender said in Is this server strategy reckless and/or insane?:
Copying to the other server is not a backup, FYI.
Not a good one, that's for sure. As there is no way to be certain that the copy is functional.
-
@bnrstnr said in Is this server strategy reckless and/or insane?:
How is your internet going to serve up all this RAID0 SSD awesomeness?? Do you really have the bandwidth to allow the hardware to be the bottleneck?
A combination of things, I'm architecting the front-end in a way that it sends the bare minimum out to the user on each request and uses persistent libraries to construct the interfaces to decimate the amount of transfer in general, all of the media and static resources are served out by a CDN, etc. But yeah, I don't think bandwidth will be the issue, but the datacenter I use has super duper bandwidth options if it gets to that point from what I understand.
-
@dustinb3403 said in Is this server strategy reckless and/or insane?:
@dashrender said in Is this server strategy reckless and/or insane?:
Copying to the other server is not a backup, FYI.
Not a good one, that's for sure. As there is no way to be certain that the copy is functional.
What do you mean? The live sites will be serving from both copies of the database, which is the evidence/certainty that it's functional, no?
-
@dashrender said in Is this server strategy reckless and/or insane?:
Copying to the other server is not a backup, FYI.
I don't think he means that. He has an HA pair AND he's taking a backup from what I saw.
-
@dustinb3403 said in Is this server strategy reckless and/or insane?:
@dashrender said in Is this server strategy reckless and/or insane?:
Copying to the other server is not a backup, FYI.
Not a good one, that's for sure. As there is no way to be certain that the copy is functional.
What do you mean? It's a live cluster. That is definitely being testing constantly. It's mirrored, live copies. THe backup itself, that he has to test offline.
