@Dashrender said in I have to change cloud drive service yet again:

@Donahue said in I have to change cloud drive service yet again:

@Dashrender said in I have to change cloud drive service yet again:

@scottalanmiller said in I have to change cloud drive service yet again:

@Dashrender said in I have to change cloud drive service yet again:

simply searching titles are often useless, because we don't always remember the actual title we give something. etc etc.

If titles (and presumable folder titles) don't let you find something, how does moving files around solve any problems?

folders do solve it, because it narrows you from 1000's of files to maybe 100. then you just read through the list.

Again - tags would really solve this much better, because you can often have way more than just one tag, making searching significantly better.

you still have the problem of having meaningful tags. where do these come from and who is responsible for maintaining them? most people are lazy and wouldn't use tags to the level of their capability. plus, whats to say that the tag I give a file is what you would use for searching for that file?

Well - that would be up to the departments. You could pre create dozens of tags, etc... if the system is like ML here- as you type win - it will show you a list of all existing tags that have win in it.

Yes - getting people to use TAGs is the hardest part. Sharepoint for example, so I'm told, can force this on users.

maybe it could force someone to use A tag, but nothing that can be programmed can force a user to use the correct tag, because it is a matter of intent and actual intelligence. Computers can only be as good as the programmer programming them.

I've seen other forums that required things like higher post counts before being able to start a new thread. I am not sure of that solves the problem or just moves it to somewhere else. I think I have also seen where new users must post to the "newbie" section before a moderator would unlock their access to post in the other sections, that way they didn't dilute the main content. I am not sure if that is a good idea or not, but it is something that is sometimes done.

@scottalanmiller said in Large or small Raid 5 with SSD:

@Donahue said in Large or small Raid 5 with SSD:

@scottalanmiller said in Large or small Raid 5 with SSD:

@Donahue said in Large or small Raid 5 with SSD:

I know you said earlier that with raid 5, you may as well add that 5th drive to the array and make it a raid 6 as opposed to sitting on the shelf.

Not "might as well", but "had better make sure you do." Difference in risk is astronomic. If you are even thinking hot spare is an option, we've not explain adequately how it works.

I was thinking cold spare, not hot spare. I don't want the array rebuilding automatically before I have time to make a conscience decision to do it. But the different is similar, I still would have a spare and is not helping the array at all just sitting on the shelf.

This isn't a good idea. You should have an array stable enough that you want it rebuilt. If you have this fear, you need a safer array.

Having never personally used a raid 5, all I have to go on is information that is presented online through mediums like ML. Some, perhaps even most, of the information I find is either out of date or pertains to the use of raid 5 with spinners. I know that in the last 4 years I have had two or three spinners fail in raid 10 arrays, and a few single drives fail in desktops, both spinners and SSD's. So in my mind, a drive failure is a reasonable assumption to occur in the next 5 years. But, we have also never had drives with warranties, so that changes the cost equation too.

I am not sure that my fear is rational, because my understanding of the actual risk is limited.

I feel like this is a 100% human problem, and no matter what we do with the OS, its still human judgment that is the cause of the problems.

@Dashrender said in I have to change cloud drive service yet again:

@scottalanmiller said in I have to change cloud drive service yet again:

@Dashrender said in I have to change cloud drive service yet again:

simply searching titles are often useless, because we don't always remember the actual title we give something. etc etc.

If titles (and presumable folder titles) don't let you find something, how does moving files around solve any problems?

folders do solve it, because it narrows you from 1000's of files to maybe 100. then you just read through the list.

Again - tags would really solve this much better, because you can often have way more than just one tag, making searching significantly better.

you still have the problem of having meaningful tags. where do these come from and who is responsible for maintaining them? most people are lazy and wouldn't use tags to the level of their capability. plus, whats to say that the tag I give a file is what you would use for searching for that file?

@RojoLoco said in What Are You Watching Now:

Awesome production from WGBH in Boston... Lennon Claypool Delirium, live at the House of Blues.

Youtube Video

(That's right, John Lennon managed to make 1 talented kid. Who woulda thought it would be with Yoko?)

does anyone else think this looks like Adam Savage?

one post wonder?

what's 1PW?

@Dashrender said in Large or small Raid 5 with SSD:

@Donahue said in Large or small Raid 5 with SSD:

I know the analogy is not perfect, but in my head I am thinking of the spare disk as a spare tire on a car. having a cold spare on the shelf to me is like having the spare tire mounted to the back or underneath the car, not being actively used to help the car stay on the road. So my instinct is to make sure I've got a spare. In the case of a 4 drive raid 5, that means a 5th disk. But as you say, IF I have that disk anyways, it is better, and as you say, emphatically so, to actually use that disk in the array from the beginning and have a 5 disk raid 6 and no spare. But that leads me back to my original position of not having a spare which my animal brain intuitively thinks of as bad and that I should get a spare. I know that my assumptions and instincts are wrong here, because I do not fully understand the scope of the difference in risks between the 4 drive raid 5 and the 5 drive raid 6. That is why I am asking all these questions, so that I can more fully understand my options and evaluate my choices based on empirical data or good logic, and not on instinct or intuition.

In the case of the cold spare with RAID 5, if you loose one drive, you're now at risk of a second drive failing, that second drive is doing you zero benefit until the rebuild process is 100% complete - AFTER you start that process.

with RAID 6, you are protected from a second drive failure situation entirely. Now you order a second drive, and assuming no more failures, you stayed as safe as possible during the entire endeaver, BUT, if you loose a second drive during the process, you saved yourself the hassle of restoring because of RAID 6.

This all mostly only matters because you've 'decided' the expense of having the 'spare/extra' drive onsite already was worth it. If you determined that the spare wasn't worth having onsite, then back to RAID 5 you go.

I agree, and that is where I am at now. I need to decide if it is worth having that 5th disk.

is there a file system that will track all this meta data, even when the file is deleted? To me it feels like general ledger type transactions where the past history is part of the permanent record. I know its apples and oranges, but is there anything on the horizon that looks at this from the point of view of historical chronology? I would probably lead to meta data bloat, which would be my guess as to why it doesn't exist now.

I am still thinking of the problem as being one of linear risk and safety, not logarithmic, and that is my fundamental flaw I think.

I know the analogy is not perfect, but in my head I am thinking of the spare disk as a spare tire on a car. having a cold spare on the shelf to me is like having the spare tire mounted to the back or underneath the car, not being actively used to help the car stay on the road. So my instinct is to make sure I've got a spare. In the case of a 4 drive raid 5, that means a 5th disk. But as you say, IF I have that disk anyways, it is better, and as you say, emphatically so, to actually use that disk in the array from the beginning and have a 5 disk raid 6 and no spare. But that leads me back to my original position of not having a spare which my animal brain intuitively thinks of as bad and that I should get a spare. I know that my assumptions and instincts are wrong here, because I do not fully understand the scope of the difference in risks between the 4 drive raid 5 and the 5 drive raid 6. That is why I am asking all these questions, so that I can more fully understand my options and evaluate my choices based on empirical data or good logic, and not on instinct or intuition.

@scottalanmiller said in Large or small Raid 5 with SSD:

@Donahue said in Large or small Raid 5 with SSD:

I know you said earlier that with raid 5, you may as well add that 5th drive to the array and make it a raid 6 as opposed to sitting on the shelf.

Not "might as well", but "had better make sure you do." Difference in risk is astronomic. If you are even thinking hot spare is an option, we've not explain adequately how it works.

I was thinking cold spare, not hot spare. I don't want the array rebuilding automatically before I have time to make a conscience decision to do it. But the different is similar, I still would have a spare and is not helping the array at all just sitting on the shelf.

@scottalanmiller While I don't disagree with what you are saying, The exposure of the file system has a very practical purpose in the real world in that it allows users, especially those that are not tech people, to understand that hard concepts of what a file is and where it exists. By presenting it the way that windows does, individual files can be treated in the users mind like their analog counterpart. People move physical files and folders around in real filing cabinets all the time and they expect computers to facilitate this as an extension of the old paper workflow. Windows calls these actions "move", but as you say, it's really a copy and delete action, and the resulting file is a new creation as far as the file system is concerned. But most people don't have that level of understanding of the file system and therefore it is very reasonable for people to expect them to behave like their physical counterpart. I also think using files moving between folders is a very legitimate workflow mechanism, maybe not the most efficient, and that people that use this method might have a folder for work in progress, a folder for submitted, or what ever. Yes, I know that in the end it is all meta data and we are just substituting the file location meta data for proper tags like "in progress" or "approved". But since our OS does not natively have these meta data tags (do any?), we can either invest in something like a document management system that adds and keeps track of those tags, or we can use the tools built into the OS.

One problem that I see from the iOS argument is when you have more than one program that is compatible with the same file. To give you an example, once a quarter I take an sql query and dump the results to a csv file. I then change the file extension on it to txt and upload that to the state. This is for quarterly taxes. If all of this is abstracted away by the OS, then trying to do something similar would be tedious at best, perhaps even impossible. You could argue that the application can be changed to output the desired file format the first time instead of using the csv middle man, but that is well beyond the scope of most people that can only use tools given to them, not create new ones.

just to clarify, we are talking about two different risks, with two different triggers, correct? The risk of a second disk failure while degraded, which is triggered the moment the first disk dies. The second risk (and less so for SSD) is URE, but my question is does this risk only trigger once you initiate a rebuild? Because it is the rebuild itself that is trying to read the unreadable block during its parity calculation?

Well, there was nothing in the 24 hour test. I have started another one for 7 days, which is the longest option in the current version.

@scottalanmiller said in Large or small Raid 5 with SSD:

@Donahue said in Large or small Raid 5 with SSD:

also, am I right to assume that network contention can influence IOPS?

Resulting IOPS to a third party service, but not IOPS themselves.

It will certainly improve latency. That synology is averaging 14.6ms reads, with spikes over 280. writes are averaging 4.5ms with spikes over 200.

I am probably looking at more like next day replacement

@scottalanmiller said in Large or small Raid 5 with SSD:

@Donahue said in Large or small Raid 5 with SSD:

Is the URE risk primarily during rebuild, or anytime it is in a degraded state?

URE is quite nominal on SSDs typically. Not zero, but not like you are used to, either.

but is the risk only present one I initiate a rebuild? As in, if a primary failure occurs, do I have time to assess my options before starting? I am basically trying to figure out if I should buy 4 or 5 drives. I know you said earlier that with raid 5, you may as well add that 5th drive to the array and make it a raid 6 as opposed to sitting on the shelf.

also, am I right to assume that network contention can influence IOPS?