Revisiting ZFS and FreeNAS in 2019
-
@xrobau said in Changes at Sangoma:
@travisdh1 You still seem to be hung up on the RAID concept of parity. Truly, honestly, that's not how ZFS works. I cross my heart. It works on copies of data. Literal, exact, copies of the data.
Unfortunately, THIS is where the 'ZFS Can't expand a volume' problem comes into play. Because the location of those copies (on the physical spindles) is made by hashing the size and number of volumes in a zraid (I'm kinda simplifying here, but work with me!)
So if a ZRAID2 has 5 spindles, that means that a copy of block 1 of the zpool will be placed on spindle 1 at sector 10, 3 at sector 100 and spindle 5 at sector 500. The location of that doesn't need to be looked up, because it can be calculated.
The 'expansion' problem comes about because if you add another spindle to that ZRAID2, suddenly block 1 would be on spindle 1 at sector 10, spindle 3 at sector 500 and spindle 6 at sector 1000 (or whatever). So the location of the data would be wrong, and everything falls apart.
(Now, this is actually wrong, and there IS an index, but, that's advanced stuff, and this is good enough for you to get the idea).
Does that make more sense?
Not at all, because you are describing RAID 5/6. There is nothing special or different here.
-
@travisdh1 Ahh, yeah, there is a difference though.
RAID5 and 6 recover data based on parity.
ZFS recovers data based on reading a copy of the data from another spindle.
So, if a disk fails in a RAID5 (of, say 5 disks), the machine has to read the data from the remaining 4 disks, and then calculate what is missing by figuring it out from the parity data.
If a disk fails in a ZRAID2 (of, again, 5 disks), it just reads the data from one of the other 2 copies (which it knows the location of already).
This is where the zraid stuff scales. You can have a ZRAID8 for example (requiring 9 or more disks), where there are 8 copies of the data. Literal copies. The data is replicated onto those disks. There are no parity calculations, as there are with RAID5 and 6.
So, as long as you can get the idea of parity out of your mind, suddenly the zraid stuff makes sense.
Think of it like a traditional RAID10, but instead of the stripe and mirror being fixed in place, the data is load balanced across the physical spindles, and there is more than one mirror.
-
@xrobau said in Changes at Sangoma:
RAID5 and 6 recover data based on parity.
ZFS recovers data based on reading a copy of the data from another spindle.That's literally the same thing stated two different ways. The data "on the other spindle" is read via parity. ZFS is literally RAID 5 and 6 (and 7), it's not an alternative to them. If ZFS does it, those RAID levels do it.
-
@xrobau said in Changes at Sangoma:
So, if a disk fails in a RAID5 (of, say 5 disks), the machine has to read the data from the remaining 4 disks, and then calculate what is missing by figuring it out from the parity data.
If a disk fails in a ZRAID2 (of, again, 5 disks), it just reads the data from one of the other 2 copies (which it knows the location of already).Yes, that's the same thing in both cases. The data on the disks in both examples is identical, what it can read from them is identical.
-
@xrobau said in Changes at Sangoma:
This is where the zraid stuff scales. You can have a ZRAID8 for example (requiring 9 or more disks), where there are 8 copies of the data. Literal copies. The data is replicated onto those disks. There are no parity calculations, as there are with RAID5 and 6.
This is REALLY off. First, ZRAID8 is not a thing. Where did you see that? That's not something that exists.
Second, what you are talking about is RAID 1, we've had that forever. Every RAID system has that. ZFS isn't doing something special, nor does it scale. It's just the same RAID 1 as everywhere else.
I've been teaching this exact thing since before ZFS was created. This isn't new, or ZFS, or RAID8. RAID 1 has always, since day one, been able to have has many copies of the data as you want. But since they are literally copies as you say, the one thing it can't possibly do is scale - no matter how many drives you add, the storage never gets bigger.
-
@xrobau said in Changes at Sangoma:
So, as long as you can get the idea of parity out of your mind, suddenly the zraid stuff makes sense.
No, because everything that people rave about with ZFS is its parity capabilities. Yes, ZFS can do RAID 0, 1, and 10, but so can everything else.
What ZFS does special is that it is the sole implementation of triple parity, aka RAIDZ3, aka RAID 7. So if you keep thinking about parity, then ZFS offers that one, really isolated, unique case that is pretty cool. Without that, it's RAID is just run of the mill RAID like everyone else. And why we use ZFS' RAIDZ system as the reference standard for all RAID discussions - to specifically make sure no one can ever point to ZFS and say "see its better", because every time we say how RAID works, it's ZFS' implementation we look to.
-
@xrobau said in Changes at Sangoma:
Think of it like a traditional RAID10, but instead of the stripe and mirror being fixed in place, the data is load balanced across the physical spindles, and there is more than one mirror.
What you are describing here is just RAID 10 as it has always been. That it can be more than two disks in a mirror is standard and assumed when you say RAID 10.
You aren't describing something special about ZFS, nor something that people use in the real world because it doesn't scale. It's too costly. So even though RAID 10 has done that for decades, it's unheard of to implement due to cost.
ZFS does nothing to fix the limitations of RAID, RAIN continues to march forward for scalability. ZFS hasn't changed its RAID since 2005, and none of it (other than triple parity) was new or special then. It was all old hat.
But these beliefs that ZFS is magic, that it breaks all the rules, that it isn't RAID, that it isn't the RAID we talk about... read the articles I linked. I covered every one of these things in the Cult of ZFS and Myths of ZFS articles already. This is well trodden territory. These are exactly the kinds of beliefs that are repeated about ZFS and cause things like FreeNAS and ZFS to be so risky - because the people implementing them are mostly not IT pros and don't understand the things that are being said.
So you get a pool of "generally interested non-IT people" who hear about super basic RAID features, think that they are new and exciting, and repeat them as if they come from ZFS, then assign "magic" to ZFS for being so much cooler than they thought stock tech already was, and then treating it as magic rather than evaluating it logically or just asking experienced storage engineers why if this is magic and so amazing, it's not widely used and why FreeNAS is considered to be a flag for dangerously deployed storage.
The problem with things like FreeNAS and ZFS is that they require a good amount of storage knowledge or else they can make it seem like they are special. So unfortunately, the only people who can safely use them are storage experts who really know this stuff. And those are exactly the group that would never use FreeNAS because it makes no sense.
-
@xrobau dont' get us wrong, we totally know that you are a developer, not IT. We know that these scam firms work hard to push this stuff because that's their bread and butter. They prey on exactly people like you, people without the time to be IT pros who get this stuff thrown on their plate because they are in an adjacent technical field and it seems like IT must be something that can be done casually on the side. So we totally understand why these marketing tricks that their community is playing work and why they seem like they must be legit. But keep in mind that that community has long been identified for its cult mentality and that their user base is self filtering - the nature of FreeNAS means that basically only non-IT pros even end up looking at it (full time IT people have no need for it) so there is no IT presence in their community to expose the false statements.
This is a great example, though, of where bringing in IT folks, even for just fifteen minutes of info, can save hours or days of non-IT research. IT advisers would be able to tell you to avoid products like this, and give you quick lists of viable options off the cuff in minutes. IT advisory is cheap up front, and hella expensive to skip.
One of the most important roles that IT plays in any organization is protecting the business from predatory vendors.
-
@xrobau said in Changes at Sangoma:
So let's take it from MY perspective - Basically, FreeNAS is a really snazzy front end to ZFS. That's it.
Right, a really dangerous front end that adds risk. ZFS is super simple to manage on its own, it's a file system. You don't need a fancy front end to any other file system, why would you need it for ZFS? Filesystems aren't things that should have GUIs.
FreeNAS has been known to both...
- Cause huge data loss on its own.
- Set people up who used the GUI to not know how to manage ZFS when it is critical to do so - something FreeNAS doesn't keep a GUi for!
So your point is exactly why it is bad. Having a "fancy GUI for ZFS" is exactly the problem. It doesn't add value, but it does add risk.
-
@xrobau said in Changes at Sangoma:
It ALREADY comes with a bunch of scripts to monitor SMART, resilver, validate data integrity, and send me a warm-and-fuzzy email every week saying that everything is wonderful. That's pretty much it.
Standard business class servers already do this in hardware. It's not bad that FreeNAS does that, it's just not as valuable as it sounds.
-
@xrobau said in Changes at Sangoma:
So, Why is ZFS so cool? Because it has MULTIPLE checks on data integrity. In fact, the whole design of ZFS was based around integrity, not speed.
This is what the marketing says. But basically every feature that ZFS touts is something we already have.
-
@xrobau said in Changes at Sangoma:
ZFS Is an Alternative to RAID - Yes. It's a DIFFERENT TYPE of what normal people think of as 'RAID' - or specifically, RAID5/RAID6. They use Parity, and when a disk is broken/missing, it does calculations to figure out the missing data. ZFS uses copies of the data. Striping and Mirroring is obviously the same.
RAID 5/6 are not what people think of as RAID. Not sure where you got that idea. Nor is ZFS an alternative in any way. It's standard RAID in every possible way. It does not vary from it in the least.
This is CompTIA A+ stuff these days, not even IT knowledge. This is stuff required even for Best Buy techs. Our interns are required to go through this training before they even hit the IT classes. It's just "computing basics."
But not only is ZFS standard RAID, but everything that people rave about with its RAID is specifically its parity.
-
@xrobau said in Changes at Sangoma:
I'm not going to get into SSDs, because I've had terrible results with a couple that wildly skew the stats - but they've burned me so badly, that the only way I'm trusting SSDs these days is in a ZRAID2 (eg, 3 copies of the data on 3 devices).
This is an emotional approach. Basically you are saying that you are doing really weird and reckless things because you've had some isolated, anecdotal problems from SSDs (or attributed to them.) Storage, or all things, is not a place to act emotionally. It's about data integrity. You've been stating how important data integrity is, but are then approaching this without a production mindset.
ZRAID is not part of ZFS.
Z-RAID is a Napp It product that no one has used or seen in the real world. http://napp-it.org/doc/downloads/z-raid.pdf
ZFS has the RAIDZ system
RAIDZ is RAID 5
RAIDZ2 is RAID 6
RAIDZ3 is RAID 7The RAID levels you are describing don't even have RAID names under ZFS, they are RAID 0, 1, and 10, but they are just called mirrored, striped and stripped mirrored VDEVs.
Both the name "ZRAID" and the numbering system you are using "ZRAID8" do not exist within ZFS, nor would they have a purpose, nor do they make sense, nor do they have something that does exist to map to. I don't know what source information you are working from, but it is so far from the real world that it's not reasonable that someone is confused, it has to be someone trolling you.
-
@xrobau said in Changes at Sangoma:
Scrubbing is special - Yes it is! If you have a disk that is faulty, scrubbing will pick it up and repair the damage. That can only happen with ZFS, because it has COPIES of the data, and every copy has a checksum. If one of the copies doesn't match the checksum, it'll repair it by writing over it with one of the good copies.
You are again, showing the same basic "missing everything" points. ZFS is NOT special in having copies, and ZFS RAID 1 is not what ZFS means. Every RAID system out there has copies. Every. Single. One. In their RAID 1 implementations.
They all scrub. Everybody. We have other threads literally right now today about people setting up their scrubbing on other systems that have everything you just described. This isn't some theory, this is how we all use our systems, every day.
Edit: Also, here is the Linux Kernel MD documentation that shows how MD RAID is able to scrub and repair broken files using either parity or mirroring. So this shows that ZFS isn't special, that the copy mechanism can't be what does it, and parity can't be a limitation.
-
@xrobau said in Changes at Sangoma:
"What?" I hear you say, "why would I want to compress and decompress my data, surely that will add an immense CPU load to my NAS!"
Um, no, that's not something we would say. Because compression has been a standard technique for speeding up slow spindle drive access since the advent of the Pentium III processor.
In the IT space, this is totally normal. Even in the Windows world this is considered "standard knowledge."
-
@xrobau I'm an intern and nothing you have listed is a special feature. I honestly thought you were a salesman or vendor on this thread.
-
@xrobau said in Changes at Sangoma:
No. Most people have no idea how much time their NAS's CPU sits around waiting for data to be returned from the disk (hint: a lot). Modern CPUs are blisteringly fast. So fast that compressing 16kb of data and writing 2kb to disk is often 10x faster than just writing 16kb of data to disk in the first place.
Actually, they do. Again, considered standard knowledge since ~2000. Now, most people who use NAS and definitely most people using FreeNAS, aren't aware of standard IT knowledge so we expect them, as non-technical people, to have no idea about such things. So from the viewpoint of people using NAS devices and FreeNAS, yes, most are clueless.
This is also why hardware RAID is dead - modern CPUs do this in their spare cycles, while they're waiting for other things to happen.
Except it isn't dead, that's literally crazy. And, more importantly, it's not at the RAID level where this happens. So this makes absolutely no sense. You are confused about how this works at the basic level. You need to step back and look at the big picture. Once again, this is how the ZFS marketing confuses non-IT people.
ZFS has three layers... RAID, FS, and LVM. In other systems, they are separate. Because of this, you are thinking that it is the RAID, rather than the FS or LVM, that is doing the compression. But it is not. So pointing out that the hardware RAID doesn't do this only shows that you are missing the basics, it is not making hardware RAID look bad. Hardware RAID supports compression exactly the same. In face, you can trivially prove this because ZFS on hardware RAID maintains the compression capabilities exact the same.
You can compress using any layer that you want. Commonly it is in the filesystem layer. And yes, when using hardware RAID with NTFS or ZFS or any number of things, the CPU does the compression in the background using spare cycles.
-
@xrobau said in Changes at Sangoma:
ZFS does not use parity. That seems to be where a bunch of this confusion is coming from.
Traditional RAID uses parity. ZFS uses copies.This is just false. All "complete" RAID systems offer stripes, mirrors (what you are confusingly calling copies) and parity. All of them.
Using terms like "copies" instead of the standard term "mirrors" makes it seem like maybe it's something unique. But it is not.
You are both confused that you think other RAID systems don't use mirrors (the very first RAID ever was a mirror), and that you think that ZFS is not parity (nearly all deployments of ZFS are specifically for its parity.) Everything you are basing your position on is as wrong as could possibly be.
-
@xrobau said in Changes at Sangoma:
ZRAIDx means there are X ADDITIONAL copies of the data. ZRAID2 has 3 copies of each chunk, spread across physical devices. The "X" number is how many HDDs the zraid can tolerate failing.
No, it does not. Not to be repetative, but I'm addressing each non-factual statement as they were posted.
First, ZRAID isn't a thing. But we know that you mean RAIDZ (just as @travisdh1 means levels, not levers.) We assume you are just making a typo over and over again.
But RAIDZ is parity, always, no exceptions. The term RAIDZ is a reference to parity RAID within ZFS. If you want non-parity RAID, it can't have the word RAID in it at all, it's literally just called a mirror. Not a copy.
And the numbers means nothing of the sort. The number in RAIDZ (with blank implying a "1") refers to the numbers of parity, nothing to do with copies or mirrors. And there is only RAIDZ, RAIDZ2, and RAIDZ3. There is no 4+, no one has ever implemented fourth order parity yet. And no one is expected to, RAIDZ3 has proven to be effectively worthless in the real world (the tech is great, it just has almost no viable use case) so making RAIDZ4 would be an exercise in futility.
-
There, I am caught up. Sorry that that was so long, but it was so many of the most wildly wrong posts that I've seen since being just in this community. I can only imagine that we have a downstream troll (or salesman?) trying a play a trick and @xrobau got caught by them. But it is really important that no one stumble on this thread and think that any of that information is somehow in any way correct.
But good news, Google now ranks us the very top hit for ZRAID8
Literally, since this is the first place it has ever been mentioned.
