@Tim_G said in Caching Needs and SSDs:

I WISH there was a built-in way in Windows Server to use RAM as cache. I think it's awesome that StarWind has it.

Well it opens a big can of worms if done in the wrong way. People don't understand block DRAM cache is dangerous w/out synchronous replication between nodes to keep multiple copies of the cache coherent between independent "controller" nodes and sort of a log at the back end, they install StarWind on a single controller w/out UPS, experience power outage and few GBs of transactions lost and... come blame us!

@scottalanmiller said in Caching Needs and SSDs:

One of the biggest benefits to Starwind is that it uses RAM cache in its SAN stack to give you millions of IOPS, instead of tens of thousands of IOPS, for lots of operations.

Yes we do! Will be happy to help to anybody who wants to play with it. Here:

StarWind vSAN Free

https://www.starwindsoftware.com/starwind-virtual-san-free

@Tim_G said in Caching Needs and SSDs:

@JaredBusch said in Caching Needs and SSDs:

@scottalanmiller said in Caching Needs and SSDs:

@JaredBusch said in Caching Needs and SSDs:

@scottalanmiller said in Caching Needs and SSDs:

Also, the need for the battery went away long ago. That's a 2000's problem.

Do what? This was a standard feature as recently as 2012 on the Dell T600 line.

Standard as a cheaper downgrade, right?

Standard as in it was the "Dell Recommended" option at the time.

And it looks like it was 2011.

And it looks like the R720xd I purchased from XByte to replace that server has the PERC H710P with a battery.

This page here says it uses NVCache. Why does it have a battery?

NV = battery-powered DRAM + flash for persistent storage. On power outage detected battery will drive on-board micro controller go gain access to memory bus and copy fast but volatile DRAM to slow but NV flash.

@Tim_G said in Caching Needs and SSDs:

@scottalanmiller said in Caching Needs and SSDs:

@Tim_G said in Caching Needs and SSDs:

Double caching seems dangerous or at least slow things down a little.

Not dangerous or slow. We do similar things for high speed websites. But in the opposite direction, of course.

Double caching is no more dangerous than single caching, if caching is dangerous, it's dangerous. If it is safe, it doesn't matter how many times you do it.

As for latency, it's not an issue and all hardware RAID systems like this do it under the hood because it's the only good way to add the SSD layer. You always want a RAM cache on the front end, and you need disks on the back end. The question is, do you want SSD in the middle or not.

If you think about the performance leap from spinners to SSD, it makes total sense to have SSD in front of them for cache. Even a little SSD helps the spinners a LOT. Same difference from SSD to RAM. RAM is orders of magnitude faster than SSD, just like SSD is over spinners. So the gains are similar. And RAM doesn't wear like SSD does, so it doesn't just speed things up but allows for fewer and more efficient writes and that means longer SSD life which actually adds reliability.

Also, cache allows writes to happen faster allowing more data to be protected before a failure.

I have a main Hypervisor running with a PERC H730p (2gb cache), configured with write-back (uses the 2gb cache) for all volumes, even the SSD SanDisk DAS Cache slow HDD volume.

But if someone has a H330 (without a cache), you can still use an SSD cache (such as with SanDisk DAS Cache) perfectly fine. You don't NEED to have the onboard RAID card cache. You can even write-through to your SSD cache volume.

Don't use DAS Cache. Western Digital had basically discontinued the product. If you rely on in within your infrastructure it means you have something on I/O path going wrong ;(

@Tim_G said in Erasure Coding:

I've no experience with Erasure on VMWare vSAN... but I know that it's production worthy and safe with S2D. It gives you the same resiliency but more efficient capacity. I believe it's nothing more than just an algorithm... so I can't see it being any less safe/efficient when used with a different product.

I do know that all flash = better efficiency.

It actually should do much better one. For some reason MSFT decided to cut off own balls and stop with double parity which is one linear parity sum and one global parity, while it was possible to make N => M e/c, same way as Azure and Ceph does.

@alboup said in Erasure Coding:

Hi all,

Erasure coding - is it a safe for use in production on an all-flash array? I'm specifically talking about VMware's vSAN here, however the question is fairly broad.

The alternative is one I'm most familiar and comfortable with; RAID1(/10) mirrored to another node(s) to provide simple/reliable fault tolerance. However, there are plenty of people talking up the new RAID5/RAID6 erasure coding features as it substantially reduces overheads. Apparently there is far less risk of failure due to the much lower URE rates in flash storage.

I'm curious what you guys think? Is it risk adverse? @scottalanmiller has posted up some passionate threads in the past about why R5/R6 is the devil (which I totally agree with) so where do you stand with erasure coding?

Thanks

1. VMware VSAN has software RAID5 and RAID6, these are XOR-based software parity RAID, I don't know why VMware decided to call them "erasure coding" (typically it's something like Reed-Solomon codes or whatever). Probably they decided "parity RAID" isn't cool and "erasure coding" is cool.

https://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.vsphere.virtualsan.doc%2FGUID-AD408FA8-5898-4541-9F82-FE72E6CD6227.html

2. Microsoft erasure coding is indeed one coming from Azure and while it's OK it's a speed freak because it uses GLOBAL parity, means some regions will be updated more frequently compared to other ones. FTL will take care o that but their E/C was never designed to run with flash for sure!

https://www.usenix.org/conference/atc12/technical-sessions/presentation/huang

Verdict: you can use whatever you want in production, both solutions have many-many adopters but none of them wasn't;t designed to run on flash (think about Pure engine) just because in such a case erasure coding should be done within FTL (flash translation layer) on so-called OpenSSDs (or their equivalent, whatever Pure is calling them).

http://openssd-project.org

Hope this helped

@msff-amman-Itofficer said in Should I build it myself (iSCSI Storage) or use AS5008T ?:

Hello,

I have this product in my country, selling for 1000$ which due to my country have high taxes.

It supports the following:

Windows + Mac + Linux cross-platform file sharing
A multimedia hub for your photos, music and movies
Your complete backup solution
Support for RAID 0,1,5,6,10 and hard drive hot swapping
Equipped with 4 Ethernet ports that support failover and link aggregation
> SMB 2/3 support increases Windows networking performance by 30%-50%

And while i am looking at its features and and even Vmware certifications for NFS I am pleasantly encouraged to by this product for my company. bu I cant help but wonder what stuff like SAM-SD does, I mean why cant I use a full fledged OS and good motherboard and fill it up with the same WD RED NAS HDD and create RAID 10 , and have the benefit of full fledged OS. I am certain Centos 7 can do this, Wondering what are your thoughts cause I reckon it would be alot cheaper for 1000$to get motherboard + CPU + case + Ram and it should do more and beyond than this device with its crammed linux, and my device can be updated faster and easier. Wondering what are the risks, or it all depends on how competent you're in doing this, if your competent enough going your own path and making your own server is better, right ?

Sorry for any typos writing this with the monitor turned off, due to recent Epi-LASIK

The only reliable SMB3 stack comes from Microsoft, so be ready to spend $800 on Windows Server Standard (unless you're ready to abuse MSFT licensing and use free Hyper-V Server for that purpose). Samba doesn't go anywhere anytime soon IMHO

Yeah, this is exactly what you want most probably. No hardware vendor will give you any flexibility SAM-SD has.

@NerdyDad said in ExaGrid:

So I asked the guy "What would happen if a controller in one of their ExaGrid node had died?" He said that the only things that are user replaceable are the drives, the PSUs, and the chassis. If there were multiple ExaGrid nodes clustered together, and one dies, the data on that one would not be available to you until it was restored.

I wasn't too keen on that idea. I was thinking more like striping across different nodes and restoring the missing data from checksums of the data that it had, but he was saying that that data would just be missing.

Not sounding like a viable solution to me. I'll keep my Synology's.

This means you don't want what they sell.

@NerdyDad said in ExaGrid:

Anybody heard of it? Thoughts?

At a Veeam User Group and ExaGrid is sponsoring it. The salesman gives a nice presentation of course, but I want to hear from experiences people on this one.

This is what you really want/need:

Restoronix

Decent hardware platform + software you know how to use (Windows Server + Veeam) + support in truly fire-n-forget mode.

In the worst case you'll re-provision non-proprietary hardware after EOL.

@Mike-Davis said in Mirror spinning disk to SSD?:

I had an HPE drive fail in a server under warranty. I don't have the drive in hand yet, but the field tech told me that the replacement drive is a 1TB SSD. The old drive was a 1TB mechanical. The drives were mirrored.

Can I swap out a HDD for a SSD, let it mirror and then swap out the other HDD? I know in general you want the drives to be identical, and everything will slow down to the HDD, but since it's for a one time mirror, it seems like it could work.

You'd better run your workload on SSD and use HDD as a backup destination. Mirroring SSD + HDD isn't going to make anybody any good: speed of a fleet is limited with a slowest ship in it (read - HDD in our context).

@dafyre said in Is HyperConvergence Even a Thing:

@scottalanmiller said in Is HyperConvergence Even a Thing:

Merging components into a single node doesn't stop you from scaling horizontally to other nodes. But EACH node must have the stack collapsed onto it. That's the key.

Then what stops a VMware setup with vSAN from being called hyperconverged?

VMware vSphere with tightly coupled vSAN and VMware vCenter with vSAN management seamlessly integrated IS a hyperconverged solution. Lots of HCI vendors will give you FUD telling their own GUI is "single plane of glass management hiding all burdens behind" but reality is - most often their GUI is just a bunch of an open-source junk compiled by some script kiddie and what they try to sell you is actually "single pain in the ass".

P.S. No, I won't give any names. This is tiny industry so you know whom I mean.

@coliver said in Is HyperConvergence Even a Thing:

Hyperconverged to me is bringing disparate interfaces and technologies into one box. Storage, Hypervisor, Management, and Network on a single box managed from a single interface.

This is exactly correct and it's what a lot of vendors (including Microsoft) don't understand.

@dafyre said in Kooler on DFS-R Issues:

With Starwind's coming Linux release (or has it already been released?)... Would this not be done in a Linux VM? That would eliminate concerns about licensing and such.

1. StarWind Linux VSA is released

2. There's no problem to install anything like us into parent partition, question was is it OK to use it as a file server with a free version of Windows

@JaredBusch said in Kooler on DFS-R Issues:

All I want is a clarification on what licenses are supposedly required here.

That you can do something on a Microsoft OS (Hyper-V in this case) means nothing. Microsoft has never been about locked down compliance.

If it requires a Server 2012 R2 license and then CALS, it is simply Server 2012R2 + Hyper-V roles, even if you only installed Hyper-V Server 2012 R2.

If it somehow only requires user CALS, then great.

Nah, you don't need anything except CALs.

@JaredBusch said in Kooler on DFS-R Issues:

@KOOLER Alright, I read it all again, and I see it is on Hyper-V Server 2012 R2.

So what licensing would be involved. Server 2012 User CALS only?
That is cheap and simple.

Yes. Or 2016 for the same price.

P.S. I've changed wording for my post - too rude IMHO, you might want to edit your quoted one.

@Tim_G said in Kooler on DFS-R Issues:

You can't install a 3rd party software to do storage or file server roles in place of the built in, thinking that is a way around it. It's not.

You absolutely can. You just have to license this usage in the proper way. Let's get to Hyper-V Server EULA here:

https://www.microsoft.com/en-us/UseTerms/Retail/HyperVServer2016/All/UseTerms_Retail_HyperVServer2016_All_English.htm

Running Instances of the Server Software. For each software license you assign, you may run one instance of the server software in the physical operating system environment on the licensed server. The instance of the server software running in the physical operating system environment may be used only to:

· provide hardware virtualization services, and/or

· run software to manage and service operating system environments on the licensed server.

Key point here is - LICENSED server. Hyper-V Server isn't licensed, so you have to buy CAL for every single instance (physical or virtual) accessing it from "outside". That's it

@JaredBusch said in Kooler on DFS-R Issues:

@Tim_G said in Kooler on DFS-R Issues:

So... Keeping in mind you can use free Hyper-V server and free StarWind virtual SAN to build a two-node shared nothing SMB3 clustered file server free of charge... I think it's time to retire DFS-R See Step-by-Step guide:

Hyper-V: Free “Shared Nothing” SMB3 Failover File Server

https://www.starwindsoftware.com/blog/part-2-smb-3-0-file-server-on-free-microsoft-hyper-v-server-20...

Except this violates the Hyper-V Server 20xx license and is illegal. Do it on Windows Server and all is well. You'll need two Windows Server licenses, but StarWind vSAN is free. Or use Linux with StarWind vSAN.

I'm not too sure about SMB 3.x on Linux, but there may be ways.

I only skimmed things, was this enabling a role on the hyper-v server itself? If so, this is completely against the license agreement and not something anyone related to this forum should be supporting or posting.

1. Sourced page has a disclaimer right on top about people violating licensing agreement by just doing things AS IS.

2. You absolutely can do what article is suggesting as long as you buy CALs. This is a blessed and hugged way. By Microsoft.

Scott, thank you for bring in this thread! I've actually forgot about performance. Both source and destination updated

4. Performance issues

DFS isn't in-line, it writes file first to read it and replicate later. This means there's 100% IOPS (read) overhead on everything you write to DFS-R enabled share.

DFS-R is reading from one replica always so there's no performance "boost" on reading data from the second copy as well (this is something what active-active clustered guys will do).

@scottalanmiller said in Kooler on DFS-R Issues:

@Tim_G said in Kooler on DFS-R Issues:

The physical server running Hyper-V Server (the hypervisor) cannot act as a file server, serving files to users or clients. It can only be used for supporting Hyper-V... including clustering, monitoring, etc.

The problem is, those are conflicting statements. Supporting Hyper-V Clustering is specifically what it is used for. Otherwise, you consume a license using Starwind always or even not using Starwind, just using local disks. But we know that local disks are okay. So using Starwind for Hyper-V clustering is logically okay as well. It just makes sense. It follows by the wording and the intent of the license.

Now maybe there is an argument that Hyper-V cannot provide its own storage via SMB3 and only iSCSI, in which case, I could see that being convoluted and weird, but could make sense.

You can do that but you need to buy CALs for that purpose.

@Tim_G said in Kooler on DFS-R Issues:

So... Keeping in mind you can use free Hyper-V server and free StarWind virtual SAN to build a two-node shared nothing SMB3 clustered file server free of charge... I think it's time to retire DFS-R See Step-by-Step guide:

Hyper-V: Free “Shared Nothing” SMB3 Failover File Server

https://www.starwindsoftware.com/blog/part-2-smb-3-0-file-server-on-free-microsoft-hyper-v-server-20...

Except this violates the Hyper-V Server 20xx license and is illegal. Do it on Windows Server and all is well. You'll need two Windows Server licenses, but StarWind vSAN is free. Or use Linux with StarWind vSAN.

I'm not too sure about SMB 3.x on Linux, but there may be ways.

1. Yup. That's why there's a disclaimer on the page I reference

Disclaimer: Please, do not violate license agreements for financial benefit. If you can do something, it doesn’t mean you should. This post is dedicated for one-time sole use of the mentioned setup – non-commercial, home lab or experiment. If you plan to earn money, please refrain from proceeding repeating the test described in this post.

2. There are third-party stacks like Visuality Systems NQ or MoSMB. I don't think Samba is going anywhere