ZFS Based Storage for Medium VMWare Workload
-
@donaldlandru said:
Edit -- Stepping back and thinking, the lack of drive bays are not a valid limiting factor as I could easily add SAS and do DAS storage on these nodes.
You can do a hybrid too. Local for some workloads and DAS or shared for others.
Figuring out if you need to just do local storage, which is super simple, or if you need to have replicated local storage, which is more complex, is the place to start. From the description, it sounds like straight local storage might be the way to go. Very cheap, very easy to tune for big time performance. XenCenter will happily put many independent (non-clustered) nodes into a single interface to make it super simple for the support staff wherever they are.
-
It seems I remember @donaldlandru mentioning making one big 5 host cluster. If he were to use something such as XenServer he would get the big cluster and still be able to separate the workloads out between the dev servers and the ops servers and still have "Local" storage right?
Even if the answer to the "Local" storage (I say that because XenServer can do its own shared storage now, right?) is a resounding "No", he can still leverage replicatoin to replicate the Dev hosts into the Ops environment and vice versa for maintenance and emergencies, right?
-
@dafyre said:
It seems I remember @donaldlandru mentioning making one big 5 host cluster. If he were to use something such as XenServer he would get the big cluster and still be able to separate the workloads out between the dev servers and the ops servers and still have "Local" storage right?
Even if the answer to the "Local" storage (I say that because XenServer can do its own shared storage now, right?) is a resounding "No", he can still leverage replicatoin to replicate the Dev hosts into the Ops environment and vice versa for maintenance and emergencies, right?
The answer to all your questions is yes. XenServer can deploy VMs on the same "cluster" to different storage devices. It will also do live migrations between various storage devices.
-
If that's the case, then @donaldlandru could just build one big 5-host cluster (assuming he can get the Politics taken care of and the CPUs are compatible -- if that is even an issue) on XenServer and be happy... Upgrade to 4 or 6TB drives per host (RAID 10) and also be happy.
-
@dafyre said:
It seems I remember @donaldlandru mentioning making one big 5 host cluster. If he were to use something such as XenServer he would get the big cluster and still be able to separate the workloads out between the dev servers and the ops servers and still have "Local" storage right?
Even if the answer to the "Local" storage (I say that because XenServer can do its own shared storage now, right?) is a resounding "No", he can still leverage replicatoin to replicate the Dev hosts into the Ops environment and vice versa for maintenance and emergencies, right?
Correct. This would actually make you question the term cluster as the boxes would actually not be associated with each other except that they are all managed from the same interface. Is that a cluster? Not to most people. Does it look like a single entity to someone managing it? Yes.
He could replicate things into other environments, yes.
-
@scottalanmiller I was thinking in terms of XenServer doing its own shared storage amongst the 5 servers that make up the cluster.
-
So to define cluster a little better in our environment.
For the ops servers cluster is the likely the proper term. We have two nodes, ensure that each node has available resources to run the entire workload of both servers if needed and use VMware's HA to manage to this.
For the dev servers, it is simply a single pane of glass, which really is all the essentials kit provides you and the access to the backup APIs.
The politics are likely to be harder to play as we just renewed our SnS for both Essentials and Essentials plus in January for three years.
Coupled with this, our offshore datacenter also has a 3 node development "cluster" which pushes us even further from truly having a single pane of glass (three so far if you are keeping count) which is also based on an essentials kit.
Another important piece of information with the local storage is that everything is based on 2.5" disks -- and all but two servers only have two bays each, getting any really kind of local storage without going external direct attached (non-shared) is going to be a challenge.
-
@dafyre said:
@scottalanmiller I was thinking in terms of XenServer doing its own shared storage amongst the 5 servers that make up the cluster.
I don't think XenServer has anything like VMWare's vSAN. I think you could probably do something like this inside of dom0 and make a RAIN or something.
-
@scottalanmiller said:
By dropping VMware vSphere Essentials you are looking at a roughly $1200 savings right away. Both HyperV and XenServer will do what you need absolutely free.
Did the price of Essentials double? I thought it was $600 for three nodes for Essentials? and something like $5000 for Essentials Plus.
-
@coliver That's what I was thinking. Somebody somewhere (Seems lik I remember Scott mentioning this) has said that XenServer uses DRBD under the hood for this.
-
What DAS chassis would someone recommend for this setup?
@donaldlandru mentioned that he needed at least 9 TB for the dev trio of servers, but not how much was needed for operations two servers.
-
-
@dafyre said:
If that's the case, then @donaldlandru could just build one big 5-host cluster (assuming he can get the Politics taken care of and the CPUs are compatible -- if that is even an issue) on XenServer and be happy... Upgrade to 4 or 6TB drives per host (RAID 10) and also be happy.
Yes. Getting big drives and RAID 10 are critical to getting necessary speed. Using WD RE SAS drives is probably best to get up to the kinds of IOPS that he needs. Best to get some kind of caching going on to really make sure enough performance is there.
With RAID 10 WD RE SAS we are probably looking at around 500 - 700 Read IOPS per machine, which is tons better than what was stated as needed, but without the shared IOPS you want extra overhead on a node by node basis to be "safe" in performance terms.
The additional capacity will be a huge win. With 3TB drives he would have 6TB usable PER NODE rather than 9TB usable for all five machines. That's huge. Leaping from 9TB total to 30TB total. Go to 4TB, 5TB or 6TB drives and those numbers skyrocket to as high as 60TB total available space!
-
-
@Dashrender said:
@scottalanmiller said:
By dropping VMware vSphere Essentials you are looking at a roughly $1200 savings right away. Both HyperV and XenServer will do what you need absolutely free.
Did the price of Essentials double? I thought it was $600 for three nodes for Essentials? and something like $5000 for Essentials Plus.
Those are the rough numbers. He has five nodes so that means either buying all licenses twice (so $1200 and $10,000) or being disqualified from Essentials pricing altogether and needing to move to Standard licensing options.
-
@scottalanmiller said:
Currently you have an inverted pyramid of doom, not the best design as you know.
This is true, in all scenarios we are playing out we are left with this giant SPOF. This is why I really like the alternatives shared solutions, ZFS, openindiana, etc. because it will allow me to build a second storage node and do replication for failover.
The business is also screaming for reliability and 110% uptime, but falls short when it comes time to writing the check for what they want.
Do the dev environments need to be highly available -- IMO no, but the business sees that as it's bread and butter, they are aware that we still have not fulfilled this requirements.
-
@scottalanmiller said:
@Dashrender said:
@scottalanmiller said:
By dropping VMware vSphere Essentials you are looking at a roughly $1200 savings right away. Both HyperV and XenServer will do what you need absolutely free.
Did the price of Essentials double? I thought it was $600 for three nodes for Essentials? and something like $5000 for Essentials Plus.
Those are the rough numbers. He has five nodes so that means either buying all licenses twice (so $1200 and $10,000) or being disqualified from Essentials pricing altogether and needing to move to Standard licensing options.
This is very much the case here.
To get this all into a single cluster (and hopefully using something like VSAN) would require us to upgrade to standard or higher, we would be able to use acceleration kits to get us there but is no small investment.
-
@donaldlandru said:
The politics are likely to be harder to play as we just renewed our SnS for both Essentials and Essentials plus in January for three years.
<snip>
Another important piece of information with the local storage is that everything is based on 2.5" disks -- and all but two servers only have two bays each, getting any really kind of local storage without going external direct attached (non-shared) is going to be a challenge.He brings a good point about the 2 bays and 2.5" drives... Do they even make 4 / 6 TB drives in 2.5" form yet?
If not, would it be worth getting an external DAS shelf for each of the servers?
-
@scottalanmiller said:
@Dashrender said:
@scottalanmiller said:
By dropping VMware vSphere Essentials you are looking at a roughly $1200 savings right away. Both HyperV and XenServer will do what you need absolutely free.
Did the price of Essentials double? I thought it was $600 for three nodes for Essentials? and something like $5000 for Essentials Plus.
Those are the rough numbers. He has five nodes so that means either buying all licenses twice (so $1200 and $10,000) or being disqualified from Essentials pricing altogether and needing to move to Standard licensing options.
Aww, gotcha - you were doubling them up.... His current spend was $5600.
-
@donaldlandru said:
@scottalanmiller said:
Currently you have an inverted pyramid of doom, not the best design as you know.
This is true, in all scenarios we are playing out we are left with this giant SPOF. This is why I really like the alternatives shared solutions, ZFS, openindiana, etc. because it will allow me to build a second storage node and do replication for failover.
The business is also screaming for reliability and 110% uptime, but falls short when it comes time to writing the check for what they want.
Do the dev environments need to be highly available -- IMO no, but the business sees that as it's bread and butter, they are aware that we still have not fulfilled this requirements.
The question is - do they loose more money when the operations systems are down or when the dev environment is down?
