Windows Failover Clustering... what are your views and why?
-
Hi folks,
We have a Windows Failover Cluster here using Starwind vSAN over three hosts, all local SSD storage. I'm looking for your thoughts on best practice on where VM should sit, as we have a few differences of opinions internally, and would like to get some external thoughts...
The two lines of thoughts currently are:
-
We have a Windows Failover Cluster, so we should add all VM to cluster storage and all VMs to the cluster, for management and HA for all VM
-
We have a Windows Failover Cluster, we should add VMs that need HA to the cluster storage and the cluster, and add all other VMs that do not need HA to the cluster, but with the VM storage local to one of the servers off of CSV storage (not HA)
I'm in the second group. Where are you and why?
My thoughts are (maybe wrong):
-
Data in the CSV will replicate over all three nodes because of Starwind. So, each write to a CSV is actually three fold. If all VMs are in CSV storage and writing to all three hosts, we could considerably lower the life of our SSDs.
-
CSVFS comes with a performance hit as each write has to be committed to each server which takes time. Plus, as a clustered file system that also comes with a performance hit. Adding everything to the cluster and CSV will just lower the performance over all for no reason as some applications do not need HA.
-
If all VMs are on the CSV the Starwind sync channel will have more work to do. Possibly introducing additional performance issues where we do not need HA for each service.
-
We can still add a VM to the cluster, but keep storage local to one of the three servers off of CSV. Where they dont require HA, thats fine as we can manage them all from WFC, whilst keeping performance (keeping load off the CSVs which isnt needed).
-
We save space on the CSV for future HA server requirements. If the CSV is used for everything, the vSAN space will fill up quickly for non HA VMs, and when we finally have a need for a new HA system, we wont have room.
-
Some applications naturally have HA with how they work, so as long as one VM is on hyper-v on each of the 3 hosts, the application stays up even without the VM being in cluster/CSV storage. So, why take up CSV space.
Logically, to me... this all adds up to using the CSV for only VMs that the company say need to be Highly Available, and leaving all else just on the local array outside of CSV storage.
What do you folks think?
Best,
Jim -
-
Is the local physical storage all part of the pool for the CSV? If so, could you be spiking a single server's storage with a VM on that host, which could then cause performance delay for the whole CSV?
-
@Dashrender said in Windows Failover Clustering... what are your views and why?:
Is the local physical storage all part of the pool for the CSV? If so, could you be spiking a single server's storage with a VM on that host, which could then cause performance delay for the whole CSV?
The storage currently looks like this. Each server has 21 TB of SSD as a $V. There are 5 CSVs/vSAN images on $V, each of 3 TB. That uses 15 TB of $V storage to provide the vSAN to the WFC. This leaves ~ 6TB usable on each host (18 TB total) as non CSV storage non HA local storage. The vSAN could of course be expanded in to this, rather than as is, but I dont think all should be CSV.
-
Oh I see your points for sure, I was only asking if a non CSV based VM could cause a performance bottleneck on a single host - the CSV itself could get stalled out, causing a problem for all CSV based VMs. Now you have another issue to consider when troubleshooting CSV based issues.
Now - perhaps you have so many IOPs that this isn't a real issue, it was only a thought.
-
@Dashrender said in Windows Failover Clustering... what are your views and why?:
Oh I see your points for sure, I was only asking if a non CSV based VM could cause a performance bottleneck on a single host - the CSV itself could get stalled out, causing a problem for all CSV based VMs. Now you have another issue to consider when troubleshooting CSV based issues.
Now - perhaps you have so many IOPs that this isn't a real issue, it was only a thought.
Oh I see. As another item on the list of why to not add everything to the cluster storage without that VM needing to be HA? Yeah, ill add that to my list.
So, do you agree option 2 is the way to go? Only add to CSV where needed...
-
How much data changes every day? Do you have 100gb of changes per day? 1Tb?
-
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
How much data changes every day? Do you have 100gb of changes per day? 1Tb?
Last time I ran live optics (a week or two ago), we were at around 6 TB of changes per day
-
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
@Dashrender said in Windows Failover Clustering... what are your views and why?:
Oh I see your points for sure, I was only asking if a non CSV based VM could cause a performance bottleneck on a single host - the CSV itself could get stalled out, causing a problem for all CSV based VMs. Now you have another issue to consider when troubleshooting CSV based issues.
Now - perhaps you have so many IOPs that this isn't a real issue, it was only a thought.
Oh I see. As another item on the list of why to not add everything to the cluster storage without that VM needing to be HA? Yeah, ill add that to my list.
So, do you agree option 2 is the way to go? Only add to CSV where needed...
I'm not agreeing or disagreeing - I don't know enough to have an opinion... but my question I think would lean more toward option 1 - because then whole system would be affected equally by the mentioned problem, instead of just a single node, which, which on one side might cause a fail-over and a kicking of this node from the cluster, all the way to crashing the whole cluster (but damn I would hope not).
-
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
How much data changes every day? Do you have 100gb of changes per day? 1Tb?
Last time I ran live optics (a week or two ago), we were at around 6 TB of changes per day
What are your drives warrantied at? What's the dwpd or whatever? The idea is they only need to last 5 years / X dwpd or whatever the period is they are rated for anyways.
-
@Dashrender said in Windows Failover Clustering... what are your views and why?:
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
@Dashrender said in Windows Failover Clustering... what are your views and why?:
Oh I see your points for sure, I was only asking if a non CSV based VM could cause a performance bottleneck on a single host - the CSV itself could get stalled out, causing a problem for all CSV based VMs. Now you have another issue to consider when troubleshooting CSV based issues.
Now - perhaps you have so many IOPs that this isn't a real issue, it was only a thought.
Oh I see. As another item on the list of why to not add everything to the cluster storage without that VM needing to be HA? Yeah, ill add that to my list.
So, do you agree option 2 is the way to go? Only add to CSV where needed...
I'm not agreeing or disagreeing - I don't know enough to have an opinion... but my question I think would lean more toward option 1 - because then whole system would be affected equally by the mentioned problem, instead of just a single node, which, which on one side might cause a fail-over and a kicking of this node from the cluster, all the way to crashing the whole cluster (but damn I would hope not).
If the single non CSV VM could cause a bottleneck on a host, couldnt you argue that that same VM would also cause a bottleneck on the CSV? If anything, as the CSVFS comes with the additional overheads due to being a clustered file system, its more likely having the VM on the CSV would create a performance issue, no?
-
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
How much data changes every day? Do you have 100gb of changes per day? 1Tb?
Last time I ran live optics (a week or two ago), we were at around 6 TB of changes per day
What are your drives warrantied at? What's the dwpd or whatever? The idea is they only need to last 5 years / X dwpd or whatever the period is they are rated for anyways.
1 DWPD, im not so worried about the writes. I just would like to avoid additional writes where not really needed.
-
Another thought. CSV data is replicated to all three hosts. So, 100 GB is actually 300 GB. 1 TB is actually 3 TB. Why would it make sense to add VMs (applications) the company can sustain long downtime with to an area where it takes up 3 x the space, on expensive SSDs. Why not put that application you dont care about, on one host, where it takes up one lot of space, leaving the other space for things the company does care about...
-
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
So, 100 GB is actually 300 GB.
Not for the drives themselves. I'm assuming some kind of hardware/software raid, so that 100GB gets split among the drives the data goes to according to RAID level.
Blocks that are accessed more frequently (read data) don't really count as much, as I'm sure there is caching in multiple places.
-
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
How much data changes every day? Do you have 100gb of changes per day? 1Tb?
Last time I ran live optics (a week or two ago), we were at around 6 TB of changes per day
What are your drives warrantied at? What's the dwpd or whatever? The idea is they only need to last 5 years / X dwpd or whatever the period is they are rated for anyways.
1 DWPD, im not so worried about the writes. I just would like to avoid additional writes where not really needed.
WHat drive model are they? How many drives per server? What RAID level is being used? Is it hw/sw raid? If hw, which card?
-
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
So, 100 GB is actually 300 GB.
Not for the drives themselves. I'm assuming some kind of hardware/software raid, so that 100GB gets split among the drives the data goes to according to RAID level.
Blocks that are accessed more frequently (read data) don't really count as much, as I'm sure there is caching in multiple places.
If I have a VM on the CSV using 100 GB, the whole point of having the vSAN is that every byte exists on the vSAN partners to avoid any downtime of failure. So, it really is copied entirely three times.
-
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
So, 100 GB is actually 300 GB.
Not for the drives themselves. I'm assuming some kind of hardware/software raid, so that 100GB gets split among the drives the data goes to according to RAID level.
Blocks that are accessed more frequently (read data) don't really count as much, as I'm sure there is caching in multiple places.
If I have a VM on the CSV using 100 GB, the whole point of having the vSAN is that every byte exists on the vSAN partners to avoid any downtime of failure. So, it really is copied entirely three times.
Yeah, I know that. But what you shown concern of before was the wear on drives. If you pick out one random drive, it's not getting 3x the data writes.
-
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
How much data changes every day? Do you have 100gb of changes per day? 1Tb?
Last time I ran live optics (a week or two ago), we were at around 6 TB of changes per day
What are your drives warrantied at? What's the dwpd or whatever? The idea is they only need to last 5 years / X dwpd or whatever the period is they are rated for anyways.
1 DWPD, im not so worried about the writes. I just would like to avoid additional writes where not really needed.
WHat drive model are they? How many drives per server? What RAID level is being used? Is it hw/sw raid? If hw, which card?
PERC H740P 8 GB Cache. Drive: MTFDDAK1T9TDN
14 per server Raid6 -
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
@Obsolesce said in Windows Failover Clustering... what are your views and why?:
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
So, 100 GB is actually 300 GB.
Not for the drives themselves. I'm assuming some kind of hardware/software raid, so that 100GB gets split among the drives the data goes to according to RAID level.
Blocks that are accessed more frequently (read data) don't really count as much, as I'm sure there is caching in multiple places.
If I have a VM on the CSV using 100 GB, the whole point of having the vSAN is that every byte exists on the vSAN partners to avoid any downtime of failure. So, it really is copied entirely three times.
Yeah, I know that. But what you shown concern of before was the wear on drives. If you pick out one random drive, it's not getting 3x the data use.
Yes, correct. I misunderstood what you are saying. Either way, I am sure we are fine on writes. Im not worried about them. I do however think its silly to use more writes than needed, for data that doesnt need HA, just because we can. By nature, leaving that data where we can take large downtime off of the vSAN, causes less wear on the disks... Why would we want to add wear where not needed...
-
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
I do however think its silly to use more writes than needed, for data that doesnt need HA, just because we can.
I haven't yet expressed which route to take, I was first trying to understand the data aspect.
-
@Jimmy9008 said in Windows Failover Clustering... what are your views and why?:
We have a Windows Failover Cluster here using Starwind vSAN over three hosts, all local SSD storage.
I'm sure that this was covered somewhere, but are you saying that this is hyperconverged and that there are three total nodes and that Starwind VSAN is what is clustering them? So not three compute nodes hooked to a remote VSAN, but just standard three node HC?
