I don't know how Starwind vSAN can be run but if it's on a hypervisor it's severely limited by I/O congestion through the kernel. NVMe drives is causing problems that was of no concern whatsoever with spinners. Both KVM and Xen has made a lot of work to limit their I/O latency and use polling techniques now but it's still a problem. That's why you really need SR-IOV on NVMe drives so any VM can bypass the hypervisor and just have it's own kernel to slow things down.

Anton: There are no problems with polling these days You normally spawn a SPDK-enabled VM (Linux is unbeatable here as most of the new gen I/O development happens there) and pass thru RDMA-capable network hardware (virtual function with SR-IOV or whole card with PCIe pass-thru, this is really irrelevant...) and NMVe drives and... magic starts happening This is how our NVMe-oF target works on ESXi & Hyper-V (KVM & Xen have no benefits here architecturally, this is where you're either wrong or I failed to get your arguments). It's possible to port SPDK into Windows user-mode but lack of NVMe and NIC polling drivers takes away all the fun: to move the same amount of data we normally use ~4x more CPU horsepower on "Pure Windows" Vs. "Linux-SPDK-VM-on-Windows" models. Microsoft is trying to bring SPDK to Windows kernel (so does VMware from what I know), but it needs a lot of work from NIC and NVMe engineers and... nobody wants to contribute. Really.

Just my $0.02

@pmoncho said in I think I am missing something about Hyper-V....?:

@kooler said in I think I am missing something about Hyper-V....?:

In general we shift our focus from "Hyper-V and VMware" to "VMware and KVM". Reason: Hyper-V doesn't grow anymore and KVM has very high chances to supersede it. VMware... There's just more money there

Could you expand on your statement about Hyper-V not growing? Thanks

% between VMware, ESXi and KVM for acquired number of customers doesn't look good for Microsoft.

@stacksofplates said in I think I am missing something about Hyper-V....?:

@scottalanmiller said in I think I am missing something about Hyper-V....?:

@bnrstnr said in I think I am missing something about Hyper-V....?:

@scottalanmiller said in I think I am missing something about Hyper-V....?:

@bnrstnr said in I think I am missing something about Hyper-V....?:

My opinion is Hyper-V is only an option when you need vSAN, otherwise I’m just not buying it.

What makes it special in that case? AFAIK there is no production vSAN for Hyper-V that is unique to it. Hyper-V is effectively completely dependent on Starwind for vSAN and they recommend KVM most of the time.

Right, Starwind makes it a more viable solution because it’s free and easy and well documented, right?

No, Starwind does not. Starwind just doesn't discriminate against it. Starwind is just as easy on VMware or KVM. So it's a draw, unless you consider Xen, then it is just a negative for Xen.

Eh not really. They dropped support for the virtual appliance on anything other than VMware. So you have to manually do the work on KVM/Hyper-V or just aren't able to do it at all. I can't tell.

This one will be back soon. We'll have a GA for VMware VSA around next week and KVM will follow.

In general we shift our focus from "Hyper-V and VMware" to "VMware and KVM". Reason: Hyper-V doesn't grow anymore and KVM has very high chances to supersede it. VMware... There's just more money there

@scottalanmiller said in Hyper-V replication, Starwind, or something else?:

@doyler3000 said in Hyper-V replication, Starwind, or something else?:

Oh and I've already got Veeam backup and replication (a 30 vm license) which gives me good agentless backup options for Hyper-V. 5-nine manager or something like it would likely be required as well.

5-Nine would be "extra" stuff only needed because Hyper-V doesn't have the native options that it provides. Another reason for KVM.

Veeam is great, but they have agent based for your scenario. We have another thread right now talking about this, but why do you see agentless as even something you want, let alone a driving factor in decision making? It sounds nice, but is very rarely (especially in such a large, diverse shop) viable.

Well, Microsoft released WAC (Windows Admin Center, ex- "Project Honolulu") to fill lack of management gap, but... I'm very pessimistic about WAC so far: too many compatibility issues and no single scenario is covered from Day Zero till the very end. Bottom line: You'll have to learn PowerShell and Windows Server management cmdlets.

@scottalanmiller said in Hyper-V replication, Starwind, or something else?:

@doyler3000 said in Hyper-V replication, Starwind, or something else?:

Out of roughly 30 VMs, 2 are Windows server instances. We're mostly running Centos here.
I'll look into SA licensing. I'm not very familiar with it.

Given that skill set, why look at Hyper-V instead of KVM? Both have been officially dropped from Starwind support temporarily until their new product roles out, which is KVM first, that's their key focus as they see it as the one with the greater future and potential. Hyper-V in general is more complex to manage than KVM. It's a fine product, but given your skill set and existing products, KVM seems like a more natural fit. Generally, Hyper-V makes sense only when you require a specific feature of it.

I'd tend to agree here, KVM running Linux VMs sounds like a better choice. Hyper-V makes sense if it's a "free" offering comping as part of already paid Windows Server licenses.

P.S. Not sure about VM backup with KVM.

@kuyaz said in 2 RAID 1 or 1 RAID 10 for VM Server Host:

Hi,

I have server with 2 x 1 TB SSD and 2 x 2 TB SATA.

I want to use this server as VM host.
My questions are :

1. Can I do RAID 1 for each SSD and SATA?
2. Can Raid 10 do different size and model (1TB SSD & 2TB SATA)?
3. What is the best RAID configuration for above use?
4. Should I go hardware RAID? or MDADM? I heard hardware is slower for SSD RAID?
5. What filesystem I should use on the host? ext4 or LVM? or other?
6. If I use CentOS 7, do you recommend XEN / VM for stability and user friendly system?

My aim is to get full speed with SSD for critical VM (database server VM guest, web apps, etc) and less critical VM on sata (mail server VM guest, etc).

Thank you.

Don't mix different types of the drives with different performance inside the same pool: you'll limit your resulting performance with the slowest (in your content - HDD).

@r3dpand4 said in Configuring Multi-Resilient Volume with Storage Spaces:

Should be noted that any server running 1709 cannot be added to S2D pools as it stands currently since it's not considered production ready....

Correct. S2D is gone from non-GA WS2016.

https://docs.microsoft.com/en-us/windows-server/get-started/server-1709-relnotes

Storage Spaces Direct

Storage Spaces Direct is not included in Windows Server, version 1709. If you call Enable-ClusterStorageSpacesDirect or its alias Enable-ClusterS2D, on a server running Windows Server, version 1709, you will receive an error with the message "The requested operation is not supported". It is also not supported to introduce servers running Windows Server, version 1709 into a Windows Server 2016 Storage Spaces Direct deployment.

@momurda said in Microsoft Removes Storage Spaces Direct from Windows Server 2016:

You will have to buy Windows Super Server Edition 2016 to get this functionality. At twice the price.
Seriously, that blurb says that MS is 'thrilled' about adoption of Storage Spaces Direct

"Storage Spaces Direct was introduced in Windows Server 2016 and is the foundation for our hyper-converged platform. We have been thrilled by the positive adoption of the Microsoft hyper-converged platform and we are committed to our customers."

so why take it away?

Don't listen to anything people say, watch what they DO.

@tim_g said in Microsoft Removes Storage Spaces Direct from Windows Server 2016:

You can still deploy systems with S2D by using RS1, and is still supported as in LTS.

It looks like they will be adding S2D back in, though. Just not yet (until it's production ready?).

You can't upgrade anyways, so if you don't already use S2D, it doesn't effect you. If you want to use S2D, you can still deploy systems with S2D by using Server 2016 LTS (RS1), and is still supported.

> It looks like they are moving S2D to a different product/SKU from what the comments say... It's hard to know what they are up to without being in their circle like @KOOLER is.

You have any info you're allowed to share regarding this to clear things up?

Unfortunately I won't be a valuable source of information here - I don't know about their plans

Forget about 15K spinners. For consumer-grade SSDs make sure your SDS of choice is OK with them (StarWind is OK, VMware vSAN / S2D are NOT OK f.e.)

https://fojta.wordpress.com/2014/01/12/monitor-wear-and-tear-of-your-vsan-ssds/

https://blogs.technet.microsoft.com/filecab/2016/11/18/dont-do-it-consumer-ssd/

https://www.starwindsoftware.com/starwind-virtual-san

@olivier said in Xenserver and Storage:

Having local storage is good for perfs, but you can't live migrate without moving the disks or HA on the other host.

I did a recap on local vs (non hyperconverged) shared storage in XS:

Most of the "budget" SMB customers shouldn't care about that.

Note, XOSAN is just Gluster under the hood. You do NOT WANT TO RUN GLUSTSER WITH 2 nodes. IT IS NOT SUPPORTED. (you can run a 3rd metadata only node, but you need SOMETHING out there to provide quorum).

It requires a proper stateful quorum of a 3rd node. Also for maintenance, you really likely want 4 nodes at a minimum so you can do patching and still take a failure. You'll also need to consider having enough free capacity on the cluster to maintain health slack on the Bricks, (20-30%) AND take a failure, so do that math into your overhead. Also for reasons, I'll get into in a moment you REALLY want to run local raid on Gluster nodes.

Also note, Gluster's local drive failure handling is very... binary... RedHat (who owns Gluster) refuses to issue a general support statement for JBOD mode with their HCI product, and directs you to use RAID 6 for 7.2K drives (no RAID 10). Given the unpredictable latency issues with SSD's (Garbage collection triggering failure detection etc) their deployment guide completely skips SSDs (as I would expect until they can fix the failure detection code to be more dynamic, or they can build a HCL). JBOD because of these risks is a "Contact your Red Hat representative for details." (Code for we think this is a bad idea, but might do a narrowly tested RPQ type process).

Gluster one node performance is very... encouraging You definitely need more nodes for a reasonable performance (even with some NVMe back end).

That's another story... Software Defined Storage on top of the hardware RAID isn't something many companies do for a good reason (we do, but we're sliding away from that approach for anything beyond 2 or maybe 3 node configurations). You want raw device access (better even with firmware bypassed) or... nobody will guarantee just confirmed write had actually been 100% completed.

@jrc said in Xenserver and Storage:

So currently I have 2 HP servers that are being used and XenServer hosts. The shared storage is on an HP MSA1040 SAN, connected via 8Gb/s Fiber.

The servers have worked flawlessly since I got them, not a single issue and have only been re-booted for updates and upgrades. I cannot say the same for the SAN. It has gone done about 4 or 5 times, and these outages have highlighted the fragility of my setup.

The HP servers have 24 2.5" drive bays. So I am contemplating filling them with drives and moving away from the SAN, but in order to that I would need the space to be shared between the two hosts.

How can I do that? What would that look like? What kind of cost would it be (outside of buying the drives) and is it a good idea?

Someone mentioned VSAN to me while I was talking about this, but I am not that clued up about VSANs and how they work or how they are put together.

Any advice on this would be greatly appreciated. But please don't lecture me on how bad a SAN is, and that my setup is doomed or that I am an idiot for doing it this way. I am looking for a path forward and not a beratement for things that have long since passed.

You can mount some NVMe performers and some high-capacity spindles into your Xen hosts to run VMs from and leave aging HP SAN for backup purpose only. Two hosts with a SAN make zero sense really...

@jaredbusch said in Can I get some direction on setting up Hyper-V server with a storage cluster?:

@dave247 said in Can I get some direction on setting up Hyper-V server with a storage cluster?:

@kooler said in Can I get some direction on setting up Hyper-V server with a storage cluster?:

@dave247 said in Can I get some direction on setting up Hyper-V server with a storage cluster?:

I have a few servers that are now available for whatever I want, since I've virtualized them to our vSphere 6.5 environment. We currently have a single SAN unit for our vm datastore which connects to two switches and then to three virtual hosts (SAM's Inverted Pyramid of Doom thing).

Anyway, I am trying to experiment with a different design as well as set up a new test environment. I want to install Hyper-V 2016 Server on my most powerful spare server, then I want to use my other two servers as mirrored or a distributed storage cluster.

I am not 100% on what is best practice on how exactly to set this up, so I'm hoping for some input. I mean, I'm a sysadmin at my job, so I understand how to install and configure stuff.. but I've not set up a completely new environment from scratch before.

Any advice is much appreciated!

SAM has a point (thanks for reference!)

Dave ping me anton AT starwind DOT com and I'll get you in touch with engineers who could help. You're welcomed to proceed with either commercial or a free version (no time bombs, no capacity or feature limits there).

https://www.starwindsoftware.com/starwind-virtual-san

https://www.starwindsoftware.com/starwind-virtual-san-free

Good luck

Sure, I may do that if I need help. I really only plan to use this as a lab + backup testing environment at work, so I would use the free version.. not sure what the difference is though.

Mostly support.

Support, UI and different "flavors" of support:

StarWind Pro-Active Support

https://www.starwindsoftware.com/starwind-pro-active-support

@dave247 said in Can I get some direction on setting up Hyper-V server with a storage cluster?:

I have a few servers that are now available for whatever I want, since I've virtualized them to our vSphere 6.5 environment. We currently have a single SAN unit for our vm datastore which connects to two switches and then to three virtual hosts (SAM's Inverted Pyramid of Doom thing).

Anyway, I am trying to experiment with a different design as well as set up a new test environment. I want to install Hyper-V 2016 Server on my most powerful spare server, then I want to use my other two servers as mirrored or a distributed storage cluster.

I am not 100% on what is best practice on how exactly to set this up, so I'm hoping for some input. I mean, I'm a sysadmin at my job, so I understand how to install and configure stuff.. but I've not set up a completely new environment from scratch before.

Any advice is much appreciated!

SAM has a point (thanks for reference!)

Dave ping me anton AT starwind DOT com and I'll get you in touch with engineers who could help. You're welcomed to proceed with either commercial or a free version (no time bombs, no capacity or feature limits there).

https://www.starwindsoftware.com/starwind-virtual-san

https://www.starwindsoftware.com/starwind-virtual-san-free

Good luck

@tim_g said in Port - Storage Spaces Direct (S2D): SCSI Enclosure Services (SES) needed only on HBA?:

@dashrender

RAID would be provided by SS.

He would need a controller that is pass-through.

Technically S2D is designed to be used by DAS systems (or JBODs if you like to make Scott facepalm), and not internal server storage... even though it works just fine.

There's no problem to use RAID controller instead of a basic HBA as long as you a) patch the registry (see my link below) or use special "filter" driver to report "RAID" bus (sic!) as SATA or SAS, and b) disable write-back cache on BOTH controller and disk itself, so all writes become atomic.

Storage Spaces Direct: Enabling S2D work with unsupported device types (BusType = NVMe, RAID, Fibre Channel). Part 1: Registry hack

https://www.starwindsoftware.com/blog/resolving-enable-clusters2d-bus-type-support-issue-on-some-storage-controllers

P.S. It's a BAD idea to use anything like that in production because as long as Microsoft support will discover your S2D configuration isn't supported they will pull out and walk away with a grin face.

You don't need to partition your disk: We'll put a container file somewhere where you allow us to do that and we'll basically mirror it's content between the hosts.

P.S. For lab it's OK but your design is not production ready.

@worden2 said in Stanford Removes Java in Favor of JavaScript in Intro to Computer Science Course:

@kooler said in Stanford Removes Java in Favor of JavaScript in Intro to Computer Science Course:

@mlnews said in Stanford Removes Java in Favor of JavaScript in Intro to Computer Science Course:

In a bit of a surprise move in the educational space, computer science bulwark Stanford University has chosen to remove Java and replace it with JavaScript in their Intro to Computer Science class. Java has been the language of this somewhat famous class since 2002, a run of fifteen years. They say that Java is showing its age, although to be fair JavaScript is nearly as old. Java itself is 22 years old this year. Since being purchased by Oracle, interest in Java has slowly fallen from its lofty peak during stewardship under Sun.

Anybody who's starting with anything except assembly language is WRONG!!! If somebody doesn't know how CPU works he can't make a decent software engineer: he'll use bloatware, write things in interpreted languages and bring file systems to kernel from user-land.

As a matter of purity or getting down to basics when introducing concepts, I agree. But, when you're talking about trying to introduce CS concepts to people new to the field assembly is a nightmare. I had some simple assembly as part of intro CS when in college, but it almost soured me completely on programming. Of course, what really killed my nascent interest in programming (in college) was having to learn Cobol... Perhaps it's safe to say that the closer you get to the kernel the more you need assembly?

Nope, modern kernel is developed in C and some C++ mostly.

Its really about basics: if you don't know how hardware works and how expensive f.e. thread content switch is you'll write something looking very cool and simple and it actually crinkles and mangles on a real hardware.

@storageninja said in Is Tintri Heading for Pure and Nutanix Territory Financially?:

@kooler HTML5 GUI's you say?

Very nice!!!

@fateknollogee said in Is Tintri Heading for Pure and Nutanix Territory Financially?:

@kooler said in Is Tintri Heading for Pure and Nutanix Territory Financially?:

Prism is nice

Will the Starwind version be as nice as Prism?

You can get a basic understanding of how it will look like here:

https://www.starwindsoftware.com/starwind-manager