Why are local drives better
-
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@Dashrender said in Why are local drives better:
@Grey said in Why are local drives better:
ay max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM c
You can literally use local disk for any thing you can use remote disk. So I'm not really sure what you're digging for.
Transfer rates. A local bus will max at 6, while a SAN on a 10 GB link (or dual 10s, whatever) can go higher as the node can cache in RAM and then write back to the drives that are local to the SAN at the slower, local rate.
So, just like the cache on a local controller?
Does your local controller have a 256gb cache?
If I've got that much ram assigned to it, sure, why not?
I've worked on local storage systems that have that much cache just recently, in fact.
-
@scottalanmiller said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@Dashrender said in Why are local drives better:
@Grey said in Why are local drives better:
ay max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM c
You can literally use local disk for any thing you can use remote disk. So I'm not really sure what you're digging for.
Transfer rates. A local bus will max at 6, while a SAN on a 10 GB link (or dual 10s, whatever) can go higher as the node can cache in RAM and then write back to the drives that are local to the SAN at the slower, local rate.
So, just like the cache on a local controller?
Does your local controller have a 256gb cache?
If I've got that much ram assigned to it, sure, why not?
I've worked on local storage systems that have that much cache just recently, in fact.
Nice. Was that a hardware controller or software based?
-
@scottalanmiller said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@Dashrender said in Why are local drives better:
@Grey said in Why are local drives better:
ay max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM c
You can literally use local disk for any thing you can use remote disk. So I'm not really sure what you're digging for.
Transfer rates. A local bus will max at 6, while a SAN on a 10 GB link (or dual 10s, whatever) can go higher as the node can cache in RAM and then write back to the drives that are local to the SAN at the slower, local rate.
So, just like the cache on a local controller?
Does your local controller have a 256gb cache?
If I've got that much ram assigned to it, sure, why not?
I've worked on local storage systems that have that much cache just recently, in fact.
Cool. Many of the systems I've seen deployed and worked with are so old that a 1gb cache is considered exotic.
-
@travisdh1 said in Why are local drives better:
@scottalanmiller said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@Dashrender said in Why are local drives better:
@Grey said in Why are local drives better:
ay max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM c
You can literally use local disk for any thing you can use remote disk. So I'm not really sure what you're digging for.
Transfer rates. A local bus will max at 6, while a SAN on a 10 GB link (or dual 10s, whatever) can go higher as the node can cache in RAM and then write back to the drives that are local to the SAN at the slower, local rate.
So, just like the cache on a local controller?
Does your local controller have a 256gb cache?
If I've got that much ram assigned to it, sure, why not?
I've worked on local storage systems that have that much cache just recently, in fact.
Nice. Was that a hardware controller or software based?
Software. Which is pretty much the only thing for enterprise systems.
-
@Grey said in Why are local drives better:
@scottalanmiller said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@Dashrender said in Why are local drives better:
@Grey said in Why are local drives better:
ay max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM c
You can literally use local disk for any thing you can use remote disk. So I'm not really sure what you're digging for.
Transfer rates. A local bus will max at 6, while a SAN on a 10 GB link (or dual 10s, whatever) can go higher as the node can cache in RAM and then write back to the drives that are local to the SAN at the slower, local rate.
So, just like the cache on a local controller?
Does your local controller have a 256gb cache?
If I've got that much ram assigned to it, sure, why not?
I've worked on local storage systems that have that much cache just recently, in fact.
Cool. Many of the systems I've seen deployed and worked with are so old that a 1gb cache is considered exotic.
That's only on commodity hardware with hardware RAID. That's generally what SAN and NAS skip to get around those limitations.
-
@scottalanmiller said in Why are local drives better:
@Grey said in Why are local drives better:
@scottalanmiller said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@Dashrender said in Why are local drives better:
@Grey said in Why are local drives better:
ay max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM c
You can literally use local disk for any thing you can use remote disk. So I'm not really sure what you're digging for.
Transfer rates. A local bus will max at 6, while a SAN on a 10 GB link (or dual 10s, whatever) can go higher as the node can cache in RAM and then write back to the drives that are local to the SAN at the slower, local rate.
So, just like the cache on a local controller?
Does your local controller have a 256gb cache?
If I've got that much ram assigned to it, sure, why not?
I've worked on local storage systems that have that much cache just recently, in fact.
Cool. Many of the systems I've seen deployed and worked with are so old that a 1gb cache is considered exotic.
That's only on commodity hardware with hardware RAID. That's generally what SAN and NAS skip to get around those limitations.
-
@scottalanmiller said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@scottalanmiller said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@Dashrender said in Why are local drives better:
@Grey said in Why are local drives better:
ay max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM c
You can literally use local disk for any thing you can use remote disk. So I'm not really sure what you're digging for.
Transfer rates. A local bus will max at 6, while a SAN on a 10 GB link (or dual 10s, whatever) can go higher as the node can cache in RAM and then write back to the drives that are local to the SAN at the slower, local rate.
So, just like the cache on a local controller?
Does your local controller have a 256gb cache?
If I've got that much ram assigned to it, sure, why not?
I've worked on local storage systems that have that much cache just recently, in fact.
Nice. Was that a hardware controller or software based?
Software. Which is pretty much the only thing for enterprise systems.
I should've known. It's so easy to make and use a huge cache with md, more system memory = more cache.
-
What local storage approaches these days tend to skip the mammoth cache approach and instead go for insanely fast SSDs because of the lack of latency between the system and the disks. You can get millions of IOPS from local disks even before the cache. So often the cache is kept small only to absorb some writes rather than for read.
But having the cache is very possible, just not always practical.
-
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey While I agree, I agree for different reasoning.
The array protection, isn't something that I think needs to be thought of in the traditional sense. I do agree that the local drive needs to be excluded from anything but secure services. So ransomware etc couldn't mess it up.
Maybe we're thinking on different levels. Are you only talking about DAS or is there something else here?
Yea.... haha
sorry for being so vague, just trying to get some ideas. Ignore raid. Its not an item to consider.
So, this is a workstation?
It could be workstation, it could also be a server. Just looking for possible use cases of a locally attached disk.
The only reason is for speed, in my world. You could do an SSD DAS (array or not) and it will be faster than any NAS, until you get up to the level of a fiber network SAN that has more transfer speed than the SAS or SATA drives in question, which may max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM cache could transfer larger files faster than DAS using SATA 3 which maxes out at 6GBit. The problem, obviously, is contention so you might never see those max speeds on the SAN in the real world production environment.
I guess it all depends on how you're going to use your DAS, and if it's just a machine with a single drive, that screams workstation (that you don't care about the data or uptime is implied by the lack of an array). If you want to make a faster workstation, get a pair of SSDs and run them in RAID0.
^^ speaking of (and hopefully not a tangent)
Anyone heard of a sata upgrade/replacement coming down the line?
-
@MattSpeller said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey While I agree, I agree for different reasoning.
The array protection, isn't something that I think needs to be thought of in the traditional sense. I do agree that the local drive needs to be excluded from anything but secure services. So ransomware etc couldn't mess it up.
Maybe we're thinking on different levels. Are you only talking about DAS or is there something else here?
Yea.... haha
sorry for being so vague, just trying to get some ideas. Ignore raid. Its not an item to consider.
So, this is a workstation?
It could be workstation, it could also be a server. Just looking for possible use cases of a locally attached disk.
The only reason is for speed, in my world. You could do an SSD DAS (array or not) and it will be faster than any NAS, until you get up to the level of a fiber network SAN that has more transfer speed than the SAS or SATA drives in question, which may max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM cache could transfer larger files faster than DAS using SATA 3 which maxes out at 6GBit. The problem, obviously, is contention so you might never see those max speeds on the SAN in the real world production environment.
I guess it all depends on how you're going to use your DAS, and if it's just a machine with a single drive, that screams workstation (that you don't care about the data or uptime is implied by the lack of an array). If you want to make a faster workstation, get a pair of SSDs and run them in RAID0.
^^ speaking of (and hopefully not a tangent)
Anyone heard of a sata upgrade/replacement coming down the line?
According to the Wiki Pedia article, SAS4 will be hitting 22.5 Gbit/s this year, but SATA will still be stuck at 16 Gbit/s.
-
@MattSpeller said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey While I agree, I agree for different reasoning.
The array protection, isn't something that I think needs to be thought of in the traditional sense. I do agree that the local drive needs to be excluded from anything but secure services. So ransomware etc couldn't mess it up.
Maybe we're thinking on different levels. Are you only talking about DAS or is there something else here?
Yea.... haha
sorry for being so vague, just trying to get some ideas. Ignore raid. Its not an item to consider.
So, this is a workstation?
It could be workstation, it could also be a server. Just looking for possible use cases of a locally attached disk.
The only reason is for speed, in my world. You could do an SSD DAS (array or not) and it will be faster than any NAS, until you get up to the level of a fiber network SAN that has more transfer speed than the SAS or SATA drives in question, which may max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM cache could transfer larger files faster than DAS using SATA 3 which maxes out at 6GBit. The problem, obviously, is contention so you might never see those max speeds on the SAN in the real world production environment.
I guess it all depends on how you're going to use your DAS, and if it's just a machine with a single drive, that screams workstation (that you don't care about the data or uptime is implied by the lack of an array). If you want to make a faster workstation, get a pair of SSDs and run them in RAID0.
^^ speaking of (and hopefully not a tangent)
Anyone heard of a sata upgrade/replacement coming down the line?
I thought M.2 already replaced it.
-
@scottalanmiller said in Why are local drives better:
@MattSpeller said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey While I agree, I agree for different reasoning.
The array protection, isn't something that I think needs to be thought of in the traditional sense. I do agree that the local drive needs to be excluded from anything but secure services. So ransomware etc couldn't mess it up.
Maybe we're thinking on different levels. Are you only talking about DAS or is there something else here?
Yea.... haha
sorry for being so vague, just trying to get some ideas. Ignore raid. Its not an item to consider.
So, this is a workstation?
It could be workstation, it could also be a server. Just looking for possible use cases of a locally attached disk.
The only reason is for speed, in my world. You could do an SSD DAS (array or not) and it will be faster than any NAS, until you get up to the level of a fiber network SAN that has more transfer speed than the SAS or SATA drives in question, which may max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM cache could transfer larger files faster than DAS using SATA 3 which maxes out at 6GBit. The problem, obviously, is contention so you might never see those max speeds on the SAN in the real world production environment.
I guess it all depends on how you're going to use your DAS, and if it's just a machine with a single drive, that screams workstation (that you don't care about the data or uptime is implied by the lack of an array). If you want to make a faster workstation, get a pair of SSDs and run them in RAID0.
^^ speaking of (and hopefully not a tangent)
Anyone heard of a sata upgrade/replacement coming down the line?
I thought M.2 already replaced it.
That's the problem with the M.2 interface. Is it using SATA or PCIe? It can be either, you have to actually read documentation to figure it out.
-
@travisdh1 said in Why are local drives better:
@scottalanmiller said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@scottalanmiller said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@travisdh1 said in Why are local drives better:
@Grey said in Why are local drives better:
@Dashrender said in Why are local drives better:
@Grey said in Why are local drives better:
ay max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM c
You can literally use local disk for any thing you can use remote disk. So I'm not really sure what you're digging for.
Transfer rates. A local bus will max at 6, while a SAN on a 10 GB link (or dual 10s, whatever) can go higher as the node can cache in RAM and then write back to the drives that are local to the SAN at the slower, local rate.
So, just like the cache on a local controller?
Does your local controller have a 256gb cache?
If I've got that much ram assigned to it, sure, why not?
I've worked on local storage systems that have that much cache just recently, in fact.
Nice. Was that a hardware controller or software based?
Software. Which is pretty much the only thing for enterprise systems.
I should've known. It's so easy to make and use a huge cache with md, more system memory = more cache.
Using "md"? What do you mean? Linux automatically cache I/O with available ram AFAIK.
-
@Grey said in Why are local drives better:
@Dashrender said in Why are local drives better:
@Grey said in Why are local drives better:
ay max at 3, 6 or 3.2 16 Gbit/s (1969 MB/s). Ergo, a 10GB FCoE SAN that's running OBR10 or Raid6 with a large SSD/RAM c
You can literally use local disk for any thing you can use remote disk. So I'm not really sure what you're digging for.
Transfer rates. A local bus will max at 6, while a SAN on a 10 GB link (or dual 10s, whatever) can go higher as the node can cache in RAM and then write back to the drives that are local to the SAN at the slower, local rate.
I'm coming back to this thread after a while - so I see that Scott has already mentioned that you can get 10 Gb locally.
The thing that those of us that aren't Scott have to remember, NAS/SAN is always second to local. Anything those can do, local can do better.
Scott's already given the reasons. -
Less COST makes local drives better.
Less network traffic makes local drives better.
Less switch/jack/cable needed makes local drives better.
Less IP address/setup/maintenance/update/firmware makes local drives better. -
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey While I agree, I agree for different reasoning.
The array protection, isn't something that I think needs to be thought of in the traditional sense. I do agree that the local drive needs to be excluded from anything but secure services. So ransomware etc couldn't mess it up.
Maybe we're thinking on different levels. Are you only talking about DAS or is there something else here?
Yea.... haha
sorry for being so vague, just trying to get some ideas. Ignore raid. Its not an item to consider.
So, this is a workstation?
It could be workstation, it could also be a server. Just looking for possible use cases of a locally attached disk.
Workstation without a Hard Disk would be SSD or Dumb Terminal
-
@Texkonc said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey While I agree, I agree for different reasoning.
The array protection, isn't something that I think needs to be thought of in the traditional sense. I do agree that the local drive needs to be excluded from anything but secure services. So ransomware etc couldn't mess it up.
Maybe we're thinking on different levels. Are you only talking about DAS or is there something else here?
Yea.... haha
sorry for being so vague, just trying to get some ideas. Ignore raid. Its not an item to consider.
So, this is a workstation?
It could be workstation, it could also be a server. Just looking for possible use cases of a locally attached disk.
Workstation without a Hard Disk would be SSD or Dumb Terminal
You can have a workstation with iSCSI or similar remote disk for booting. Used to be a thing, actually. Not so much any more. I know a company that claims to do this in Toronto as they think it is highly beneficial (hint: it is not.)
-
@scottalanmiller said in Why are local drives better:
@Texkonc said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey While I agree, I agree for different reasoning.
The array protection, isn't something that I think needs to be thought of in the traditional sense. I do agree that the local drive needs to be excluded from anything but secure services. So ransomware etc couldn't mess it up.
Maybe we're thinking on different levels. Are you only talking about DAS or is there something else here?
Yea.... haha
sorry for being so vague, just trying to get some ideas. Ignore raid. Its not an item to consider.
So, this is a workstation?
It could be workstation, it could also be a server. Just looking for possible use cases of a locally attached disk.
Workstation without a Hard Disk would be SSD or Dumb Terminal
You can have a workstation with iSCSI or similar remote disk for booting. Used to be a thing, actually. Not so much any more. I know a company that claims to do this in Toronto as they think it is highly beneficial (hint: it is not.)
Thats just silly talk.
-
@scottalanmiller said in Why are local drives better:
@Texkonc said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey While I agree, I agree for different reasoning.
The array protection, isn't something that I think needs to be thought of in the traditional sense. I do agree that the local drive needs to be excluded from anything but secure services. So ransomware etc couldn't mess it up.
Maybe we're thinking on different levels. Are you only talking about DAS or is there something else here?
Yea.... haha
sorry for being so vague, just trying to get some ideas. Ignore raid. Its not an item to consider.
So, this is a workstation?
It could be workstation, it could also be a server. Just looking for possible use cases of a locally attached disk.
Workstation without a Hard Disk would be SSD or Dumb Terminal
You can have a workstation with iSCSI or similar remote disk for booting. Used to be a thing, actually. Not so much any more. I know a company that claims to do this in Toronto as they think it is highly beneficial (hint: it is not.)
Oh, I remember those days. All our OpenVMS workstations were all booted from tftp. Of course, this is also the place that thought it was an ok idea to plug a T1 directly into the network without any firewall.
-
@Texkonc said in Why are local drives better:
@scottalanmiller said in Why are local drives better:
@Texkonc said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey said in Why are local drives better:
@DustinB3403 said in Why are local drives better:
@Grey While I agree, I agree for different reasoning.
The array protection, isn't something that I think needs to be thought of in the traditional sense. I do agree that the local drive needs to be excluded from anything but secure services. So ransomware etc couldn't mess it up.
Maybe we're thinking on different levels. Are you only talking about DAS or is there something else here?
Yea.... haha
sorry for being so vague, just trying to get some ideas. Ignore raid. Its not an item to consider.
So, this is a workstation?
It could be workstation, it could also be a server. Just looking for possible use cases of a locally attached disk.
Workstation without a Hard Disk would be SSD or Dumb Terminal
You can have a workstation with iSCSI or similar remote disk for booting. Used to be a thing, actually. Not so much any more. I know a company that claims to do this in Toronto as they think it is highly beneficial (hint: it is not.)
Thats just silly talk.
Makes more sense than you might think. Workstations rarely need many IOPS and they pretty much all have hard drives storing the same data - a copy of Windows redundant on every machine in the company. That's a LOT of copies of the same files. A company of 1,000 users might have several terabytes of redundant, unimportant files. Using a SAN can mean one super high performance read only SSD-based LUN with a few million IOPS through RAM caching could power the entire company with no chance of malware infection via the workstations. There is logic to it at scale.
