Why the SMB Still Needs Hardware RAID
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
Many devices, like NAS and SAN, implement blind swapping in software RAID. If your software RAID handles blind swapping, then this issue does not apply. But currently that is effectively limited to appliances that use their own software RAID automation systems and not to systems where you control the RAID components.
I'm curious how Thecus, Synology and the rest manage blind swap, because I know that many of them use software RAID under the hood.
-
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
Many devices, like NAS and SAN, implement blind swapping in software RAID. If your software RAID handles blind swapping, then this issue does not apply. But currently that is effectively limited to appliances that use their own software RAID automation systems and not to systems where you control the RAID components.
I'm curious how Thecus, Synology and the rest manage blind swap, because I know that many of them use software RAID under the hood.
Automation scripts and limits on variables. General software RAID allows for tons more possibilities. They lock down the options to make it viable to automate.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
Many devices, like NAS and SAN, implement blind swapping in software RAID. If your software RAID handles blind swapping, then this issue does not apply. But currently that is effectively limited to appliances that use their own software RAID automation systems and not to systems where you control the RAID components.
I'm curious how Thecus, Synology and the rest manage blind swap, because I know that many of them use software RAID under the hood.
Automation scripts and limits on variables. General software RAID allows for tons more possibilities. They lock down the options to make it viable to automate.
So they always know that slot 1 in the chassis is connected to SATA bus 1. Makes sense.
-
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
Many devices, like NAS and SAN, implement blind swapping in software RAID. If your software RAID handles blind swapping, then this issue does not apply. But currently that is effectively limited to appliances that use their own software RAID automation systems and not to systems where you control the RAID components.
I'm curious how Thecus, Synology and the rest manage blind swap, because I know that many of them use software RAID under the hood.
Automation scripts and limits on variables. General software RAID allows for tons more possibilities. They lock down the options to make it viable to automate.
So they always know that slot 1 in the chassis is connected to SATA bus 1. Makes sense.
And they know the intention of the drives.
-
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
-
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
vSAN, from talking with some VMWare engineers, is really meant to be handled as RAIN. You don't care about the individual underlying hardware components because each individual node is replicating to all the other nodes.
-
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
RAIN I think.
-
@coliver said in Why the SMB Still Needs Hardware RAID:
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
vSAN, from talking with some VMWare engineers, is really meant to be handled as RAIN. You don't care about the individual underlying hardware components because each individual node is replicating to all the other nodes.
And the drives are independent.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
vSAN, from talking with some VMWare engineers, is really meant to be handled as RAIN. You don't care about the individual underlying hardware components because each individual node is replicating to all the other nodes.
And the drives are independent.
How are the drives configured in a RAIN setup? as JBOD? Does the RAIN setup (aka software managing the storage) handle the distribution of data over the drives? Does it work more or less like RAID 0 in the hopes of getting the best speed/performance?
-
@Dashrender said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
vSAN, from talking with some VMWare engineers, is really meant to be handled as RAIN. You don't care about the individual underlying hardware components because each individual node is replicating to all the other nodes.
And the drives are independent.
How are the drives configured in a RAIN setup? as JBOD? Does the RAIN setup (aka software managing the storage) handle the distribution of data over the drives? Does it work more or less like RAID 0 in the hopes of getting the best speed/performance?
I hate the term JBOD, it means nothing. All disks are JBOD at some level and not at others. All RAID starts as JBOD.
RAIN isn't a specific approach, so you can't answer at that level. RAID is specific and doesn't handle RAIN. So like relational databases and NoSQL, we have "this popular thing" and "everything else" categories, more or less.
-
@Dashrender said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@NetworkNerd said in Why the SMB Still Needs Hardware RAID:
Wouldn't a technology like vSAN be considered software RAID in some form? It's just not enabled in ESXi by default unless you buy the license.
vSAN, from talking with some VMWare engineers, is really meant to be handled as RAIN. You don't care about the individual underlying hardware components because each individual node is replicating to all the other nodes.
And the drives are independent.
How are the drives configured in a RAIN setup? as JBOD?
To the host system, yes.
Does the RAIN setup (aka software managing the storage) handle the distribution of data over the drives?
http://www.computerworld.com/article/2579611/data-center/rain-explained.html
And let's not forget Erasure Coding to really understand what's going on under the hood of a RAIN system.
https://www.backblaze.com/blog/reed-solomon/Does it work more or less like RAID 0 in the hopes of getting the best speed/performance?
If you really want it to, it could.
-
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Ok that makes sense.. thanks.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
-
@dafyre said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
Gluster and Swift
-
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@dafyre said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
Gluster and Swift
I think Ceph and Lustre may be two others.
-
@coliver said in Why the SMB Still Needs Hardware RAID:
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@dafyre said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
Gluster and Swift
I think Ceph and Lustre may be two others.
Lustre is RAIN, but is closed. Gluster was the open replacement for Lustre.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@dafyre said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
Gluster and Swift
I think Ceph and Lustre may be two others.
Lustre is RAIN, but is closed. Gluster was the open replacement for Lustre.
Just a quick search showed that Lustre was GPL 2.0, not sure if that is new or not.
-
@coliver said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@dafyre said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
Gluster and Swift
I think Ceph and Lustre may be two others.
Lustre is RAIN, but is closed. Gluster was the open replacement for Lustre.
Just a quick search showed that Lustre was GPL 2.0, not sure if that is new or not.
Oh wow, must be new. It was crazy expensive in 2006 when we were really investigating it. That's awesome.
-
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
@coliver said in Why the SMB Still Needs Hardware RAID:
@travisdh1 said in Why the SMB Still Needs Hardware RAID:
@dafyre said in Why the SMB Still Needs Hardware RAID:
@scottalanmiller said in Why the SMB Still Needs Hardware RAID:
The most common RAIN approach that I see is taking all disks in the pool, noting their nodal presence and using mirroring to distribute the data so that data mirrors never go to the same disk and/or the same node. So a little like a networked RAID 1E but with more flexibility and the option to add nodal separation and performance testing so that data moves to where it is used.
Are you aware of any open source RAIN systems?
Gluster and Swift
I think Ceph and Lustre may be two others.
Lustre is RAIN, but is closed. Gluster was the open replacement for Lustre.
Just a quick search showed that Lustre was GPL 2.0, not sure if that is new or not.
Oh wow, must be new. It was crazy expensive in 2006 when we were really investigating it. That's awesome.
Ah looks like it went open source in 2010.
