Revisiting ZFS and FreeNAS in 2019
-
@xrobau are the official docs enough?
https://docs.oracle.com/cd/E19253-01/819-5461/gamtu/index.html
ZFS provides a RAID-Z configuration with either single-, double-, or triple-parity fault tolerance. Single-parity RAID-Z (raidz or raidz1) is similar to RAID-5. Double-parity RAID-Z (raidz2) is similar to RAID-6.
-
@scottalanmiller said in Changes at Sangoma:
RAID 2 (no existing implementation)
RAID 3 (deprecated)
RAID 4 (rare)
RAID 5 (aka RAIDZ)
RAID 6 (aka RAIDZ2)
RAID 7 (aka RAIDZ3)No. RAIDZ does not use parity. Just because people REFER to it as parity does not mean it is such. That is the 'simplification confusion'.
So yeah, sorry. There's no use continuing this discussion because you're adamant that you're correct, and you refuse to even think you could be wrong. So, being that there is no possible way for me to change you mind, there's no use me continuing, is there?
-
@xrobau said in Changes at Sangoma:
No. RAIDZ does not use parity. Just because people REFER to it as parity does not mean it is such. That is the 'simplification confusion'.
It uses parity. Oracle says so, the code says so, the entire industry says so. Just making things up doesn't change that. It's not "like parity", it's standard RAID 5 parity with a variable stripe width.
And parity acts nothing like mirroring (copies), so if your claims were correct, the amount of available storage wouldn't be correct. So it's trivial for anyone who has used RAIDZ to show that it has to be parity and cannot be copying.
-
@xrobau said in Changes at Sangoma:
So, being that there is no possible way for me to change you mind, there's no use me continuing, is there?
Well, start trying to change my mind by...
- Saying something rational.
- Saying something that seems crazy but at least has a REASON for your belief in it.
Right now, you are stating things we know are absolutely false, are common sense provably not true, go against ALL available documentation... and flatly refusing to provide so much as a casual reference. Basically you just called the sky red and instead of providing any reason to support why you believe it not to be blue, you are just jumping up and down screaming it is red.
So no, you've only convinced us that you made this all up and have no basis for what you are saying. The louder you yell and refuse to provide evidence (even just logic based evidence) the more it shows how likely you are to be wrong. People who are telling the truth can normally say why they believe something to be the truth. But you've not stated even where you might have gotten these ideas, let alone shown a solid foundation for them.
-
@xrobau said in Changes at Sangoma:
If you don't want to work with me here, then I'm just going to not bother.
In what way have I not worked with you? I've gone point by point and shown in every single case...
- Why it is historically believed to be wrong.
- Why it goes against previously documented understanding.
- Why it doesn't make sense.
- What seems to be (where possible) the source of your confusion.
- Asked you to provide the same level of documentation that I am to support your beliefs.
Working with you is exactly what I'm doing. Everyone else seems to have written you off. I actually bothered to respond as if you were being serious about this. But you are not giving me the common courtesy of providing a single reference to any of this. Just claiming to be an expert while missing basics at an A+ level (literally) doesn't suggest that you are an expert. I knew a guy who once installed a video game from a box he bought and claimed it made him a developer because that's what he thought programming was. Doesn't make him a developer.
-
-
Even FreeNAS doesn't say things like this. FreeNAS states very clearly that ZFS' RAIDZ is parity:
"ZFS is designed for data integrity from top to bottom. RAID-Z, the software RAID that is part of ZFS, offers single parity protection like RAID 5, but without the “write hole” vulnerability thanks to the copy-on-write architecture of ZFS. The additional levels RAID-Z2 and RAID-Z3 offer double and triple parity protection, respectively."
-
-
@xrobau said in Changes at Sangoma:
Just because people REFER to it as parity does not mean it is such. That is the 'simplification confusion'.
So your hypothesis is that people, who all know how RAID and mirroring (copying) works, and who all know what parity is, and all know their RAID levels and know ZFS.... "simplify" by getting the term backwards? And that that makes it "simpler", why?
I think your desire to justify the belief that there is no parity is a well known parity system is leading you to concoct some truly outrageous beliefs. Simplification is the farthest thing from an explanation for the entire industry stating something complex and backwards, when what you claim to be true would be the simplest, easiest thing ever.
This just doesn't make any sense, you must see how crazy this sounds.
-
Nexenta is a pretty big player in the ZFS space, they document it as parity as well...
-
Here is SUN's original 2007 documentation stating that it did mirroring and parity. SUN used RAID to refer solely to parity. But notice that SUN themselves call it RAID. Statements that parity and RAID don't mean parity and RAID go a bit far. Every document that we can find anywhere all agree. And instead of providing any documentation for your position, you are just resorting to saying that all of the source documents are wrong and that none of the words, terms, or functionality means what is stated and that all of it means something other than what is said?
http://web.archive.org/web/20071015014209/http://www.sun.com/2004-0914/feature/
Also, you stated that the self healing was made possible by the copies, but even in 2007 SUN stated that the parity made self healing possible as well. So even your position that the copies are necessary for that feature are incorrect (which we knew, it's just that I now provided a specific reference for that.)
I think you should read Predictably Irrational, Revised and Expanded Edition: The Hidden Forces That Shape Our Decisions , it's a great volume for understanding how humans tend to get misinformation or a random belief and then in an attempt to justify something that we believed without foundation, will begin to formulate false reasons for it after the fact to make it seem like what we believed makes sense. If are you familiar with this process, we can see that happening in this thread.
You started with some mildly incorrect beliefs based on common marketing. Then to defend those beliefs you started adding in technical details that were incorrect, but could be chalked up to misreading some docs or overhearing something and misunderstanding. But then to defend those you just get getting crazier and crazier. As we go down through the thread, we can see the conversation devolving from simple misunderstandings to bizarre claims. By the end, you are basically calling every IT whose ever worked with ZFS a liar, like there is some secret conspiracy to make everyone confused about a really simple technology that we use every day.
-
https://blogs.oracle.com/ahl/double-parity-raid-z
"Double-Parity RAID-Z
As of build 42 of OpenSolaris, RAID-Z comes in a double-parity version to complement the existing single-parity version -- and it only took about 400 additional lines of code. "
-
Sigh.
@scottalanmiller said in Changes at Sangoma:
You started with some mildly incorrect beliefs based on common marketing.
I'm a sysadmin. I've done courses. This is not marketing, this is training. Now, maybe I've been trained wrong, or maybe I've forgotten my training - I was actually looking through the ZoL source code to see if I was wrong (and, it looks like I may be!), but I'm just not interested any more.
You're so hung up about HOW BLOCKS ARE STORED ON THE DISK that you've ignored everything else I've said.
So. Whatever. I'm REALLY done this time.
Oh, and here's the link to the code where they ARE doing math, which is when I was about to come in and go 'Whoops, looks like I was wrong', but turns out you're just interested in being an arsehole, rather than actually engaging in discussion.
-
@xrobau said in Changes at Sangoma:
You're so hung up about HOW BLOCKS ARE STORED ON THE DISK that you've ignored everything else I've said.
But everything you've said was good, based on your beliefs of how the data was stored on the disk. That was your foundation for refuting things that we had researched thoroughly in the past.
You are acting like I'm ignoring you, that's the absolute last thing that I've done. I've paid so much attention, from the very first post.
-
@xrobau said in Changes at Sangoma:
Oh, and here's the link to the code where they ARE doing math, which is when I was about to come in and go 'Whoops, looks like I was wrong', but turns out you're just interested in being an arsehole, rather than actually engaging in discussion.
How exactly am I being an ass here? I have engaged well since the beginning and worked hard, even to the point of documenting this years ago, to help educate and train. You put me in the position of having to defend the truth. You didn't believe me, or anyone, in the industry. No amount of me providing documentation helped. Then you call me an ass?
I am not the one who ignored what was written. And I'm not the one making wild claims. All I did was defend the truth and try to educate you. You had misinformation and instead of providing references, attempted to brow beat me into accepting them no matter how much they were not based on fact. You are still trying to attack me personally because I wasn't wrong and didn't let you promote a product based on a misunderstanding of it.
-
@xrobau said in Changes at Sangoma:
I'm a sysadmin. I've done courses. This is not marketing, this is training. Now, maybe I've been trained wrong, or maybe I've forgotten my training
In a situation like this, we all will forget things that we don't use regularly. Even an "every day full time Solaris admin on ZFS" will potentially forget details if they aren't making RAID decisions all the time (you can implement a pre-chosen standard all the time and not have to think about exactly what it is doing or what the alternatives are.) Forgetting isn't a big deal. You gotta use very specific stuff often to have any live memory of it, normally. And the number of people implementing RAID levels or deciding between them is very small, and of that number, the number that do it often is much smaller still.
-
@xrobau said in Revisiting ZFS and FreeNAS in 2019:
- ZFS Is an Alternative to RAID - Yes. It's a DIFFERENT TYPE of what normal people think of as 'RAID' - or specifically, RAID5/RAID6. They use Parity, and when a disk is broken/missing, it does calculations to figure out the missing data. ZFS uses copies of the data. Striping and Mirroring is obviously the same.
- "ZFS Raid Levers aren't standard" - I have no idea what that means.
- Scrubbing is special - Yes it is! If you have a disk that is faulty, scrubbing will pick it up and repair the damage. That can only happen with ZFS, because it has COPIES of the data, and every copy has a checksum. If one of the copies doesn't match the checksum, it'll repair it by writing over it with one of the good copies.
So now that we got to this point, you can see why the myth's document has those items and why I was defending them. ZFS is not an alternative, but just standard RAID levels, and yes I went back and corrected the typo. And the scrubbing isn't special because it's the same as anywhere else.
My point has never been that ZFS is bad, it's amazing. It was the first filesystem of its type, it broke new ground, it remains the most stable and advanced implementation of its family (which includes BtrFS and ReFS), but that there are so many myths around it that people tend to feel that it is absolutely astounding how safe and special it is, but mostly the things that it does are just normal features of filesystems, LVMs, and RAID. That it is stuff lumped into one makes it easy to promote as doing something special because people assign the features to the filesystem layer and then say "can your filesystem do that" when they should be saying "can your storage layer do that?"
The danger is ZFS is getting overwhelmed and feeling that it does more than it does, or believing that other systems do less than they do.
I compare it to Texas. Hold on, this actually makes sense. In Texas they make a huge deal of teaching students that "Texas is a republic" and that that makes it special compared to the other states. But in the other 49 states we are taught how US law requires all individual states to be republics before joining the union as a state. So while the statement that Texas is a republic is true, it is stated in the misleading way to imply that the other 49 states are not. So what people feel makes Texas special, actually just makes them run of the mill.
I've been fortunate enough to get to work with the ZFS team themselves from nearly the very beginning as they stepped in assist with the SAM-SD project in 2007. ZFS was young and mostly unknown then. Great people doing brilliant work. I got to see Thumper before it was released on the market and it was the inspiration for my SAM-SD work in many ways (but with different goals, we couldn't use ZFS for SAM-SD for performance reasons.) I was one of the original cheerleaders trying to get the word out about ZFS and software RAID and open storage practices back when almost everyone was sure that software RAID was a bad thing and that ZFS was crazy.
But around 2012 things flipped and suddenly the Cult of ZFS stuff started, seemingly out of the FreeNAS community, and tons of weird misinformation started coming out that took the hesitance to deploy ZFS and started acting like it was crazy to use anything else and basically stating that ZFS was magic and doing all kinds of things that it wasn't doing, wasn't meant to do, etc. So for the same reasons that I was a big ZFS cheerleader for most of a decade, I found myself having to constantly temper peoples' emotions about ZFS and downplay it rather than promote it because we went from people emotionally avoiding it because they tied it to software RAID mentally (which was only an option) to people believing that it wasn't RAID and was doing all kinds of magic and never asking what that magic was.
Soon we found people who would never have implemented RAID 5 on spinning drives because they knew the risks well, would implement it under the RAIDZ brand name because they were convinced that RAIDZ wasn't RAID 5, and didn't carry the risks that they knew about, and never asked how or why it could do that. That's why people like John Nicholson and I did all of our risk analysis for RAID 5 using RAIDZ to make sure that people understood that all the publications about parity RAID risks and problems were based on ZFS and if people felt that ZFS was better than any other RAID option (and it basically is) then all others would be equal or worse, never better.
-
@xrobau said in Revisiting ZFS and FreeNAS in 2019:
"XXX is special" - Don't know why you think someone would say that. So let's just skip over that.
I know, right? But if you go on places like the Spiceworks community or the FreeNAS forums, people would state that stuff constantly about caching, snapshots, etc. Basic stuff that we had had for a long time. The biggest thing seemed to be anything that was held in the LVM layer was unknown to many people and when ZFS offered it, they felt like they were discovering something new. So there were countless weird conversations of people acting like they had just discovered the snapshot, lol.
Here is an example from as late as 2016: "The ZFS snapshots and ZFS send/receive features make this a very enterprise and mature file system. I find it hard not to choose ZFS when given an option."
Basically he thought that because it could do snaps, that that alone made it mature and he was picking it based on that. The send/receive is unique, and neat, but ultimately not a big deal. Handy, but that's about it. If he knew that other filesystems were being snapped many years earlier (making them more "mature") would he then have ruled out ZFS based on the same logic?
-
Here is another one, this time back to 2011: "I am testing ZFS is FreeNAS. ...My main reason for wanting ZFS is the awesome snapshot capablity."
Already at that point you can tell from the responses that having things like snaps listed as key differentiation features were a big thing that had already been discussed a bit. It's an odd thing, but people were really pushing it.
-
@xrobau said in Revisiting ZFS and FreeNAS in 2019:
If you have a HDD with bad cache ram on it, ZFS (and, to a lesser extent, btrfs) is the only thing that's going to warn you that something is wrong with that device (and it's not like this is uncommon - I've discovered two drives in the last year that were writing corrupt data). This happens.
This is increasingly uncommon today as this is mostly a risk with Winchester (spinning) drives. As the world moves to more reliable SSD, this has dropped off. It doesn't go away, but most corruption is believed to come from the network not the storage media, which nothing protects against (yet). But for the media level issues, newer firmware and tech, better components, and the biggest thing - moving from magnetic media to solid state all reduce this risk a bit. It remains, of course, but most companies go years or decades without experiencing it and probably the average person will never see it in a life time. It happens, but so often to things of no consequence (like empty areas of a disk) that it doesn't matter. When it does strike, typically it is a file so trivial to replace that it's not considered consequential. A real risk and certain scenarios definitely require watching for it carefully, but by and large between how rarely it impacts people and that most shops have moved to safer tech it's all but vanished as a talking point. In the 2000s, it was a bit concern.
