Raid 6 Amateur File Server Setup Questions
-
Thanks; I'm beginning to understand.
On my desktop change of OS would depend on where data is stored. With JBODs and a seperate OS disk or ssd I could run linux on a stick or install it as a second OS and access the data.
If the desktop OS is on the same disk as the data, it would be hard to get rid of the old one. But otherwise changing OS would be quite easy, or am I wrong? -
That sounds reasonable. But even if the OS is on the disk with the data, as long as you aren't formating the disk, you could still install a new OS on that disk, normally it won't overwrite data on the drive.
-
Firmware update was succesful. The drives are still not recognized at 3GB/s, but this seems to change arbitrarily. At some point I had one or two of the 5 WD drives recognized as 3GB/s, now I'm down to 0 again (all show 1.5GB/s); the only stable one is the HGST. A few other drives I tested got 3GB/s too.
-
I wonder if you have a bad cable between the drives and the RAID card?
Also, are there any firmware updates for the drives?
-
Although I rebooted thrice after the firmware update, the issue seems to be gone now. I didn't change anything though. I coulnd't find any firmware uppdate for the drives, except the lcc update, but none of the drives were affected by this.
I've built the array now with a stripe size of 512KB, hopefully not too much (I consulted the web; let's see whether that actually holds) It was the second highest, 1024 would have also been available (down to 16).
Just ran a quick test and got transfer rates of about 11MB/s write and 115MB/s read. The write penality is probably within range?!
-
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
Virtual really has no downsides, though. The "single point of risk" is misleading because that's an emotional way of looking at it and empirical. One of the most important reasons that we use virtualization always is because it makes those workloads safer, it doesn't increase risk. Sure, it "adds a point of risk" but it vastly offsets this by making tons of other points safer. This is why terms like "Single Point of Failure" are dangerous - they make people panic over something that might be safe and make them see having two of something as safe even when its dangerous. It's an emotional way to view risks and very dangerous.
Virtualization is not a pro/con situation. It's just pro.. pro... pros. It's always free, so there is no cost. It improves a lot of things with only nominal negatives. There are very important reasons why you never talk about other options... because no one should be led to believe that there is a use case where not having virtualization is acceptable. That's not to say that at home you can't do whatever you want, of course you can. But even at home you should recognize that doing something without virtualization is risky just for the sake of.. whatever it is making you want to do that.
I absolutely agree. I just wanted to present that caveat. It is absolutely outweighed by all of the benefits.
Q: Have any of you ever attempted to cluster two VM host machines together? Is that a thing?
-
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
Virtual really has no downsides, though. The "single point of risk" is misleading because that's an emotional way of looking at it and empirical. One of the most important reasons that we use virtualization always is because it makes those workloads safer, it doesn't increase risk. Sure, it "adds a point of risk" but it vastly offsets this by making tons of other points safer. This is why terms like "Single Point of Failure" are dangerous - they make people panic over something that might be safe and make them see having two of something as safe even when its dangerous. It's an emotional way to view risks and very dangerous.
Virtualization is not a pro/con situation. It's just pro.. pro... pros. It's always free, so there is no cost. It improves a lot of things with only nominal negatives. There are very important reasons why you never talk about other options... because no one should be led to believe that there is a use case where not having virtualization is acceptable. That's not to say that at home you can't do whatever you want, of course you can. But even at home you should recognize that doing something without virtualization is risky just for the sake of.. whatever it is making you want to do that.
I absolutely agree. I just wanted to present that caveat. It is absolutely outweighed by all of the benefits.
Q: Have any of you ever attempted to cluster two VM host machines together? Is that a thing?
What do you mean by cluster?
-
@coliver said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
Virtual really has no downsides, though. The "single point of risk" is misleading because that's an emotional way of looking at it and empirical. One of the most important reasons that we use virtualization always is because it makes those workloads safer, it doesn't increase risk. Sure, it "adds a point of risk" but it vastly offsets this by making tons of other points safer. This is why terms like "Single Point of Failure" are dangerous - they make people panic over something that might be safe and make them see having two of something as safe even when its dangerous. It's an emotional way to view risks and very dangerous.
Virtualization is not a pro/con situation. It's just pro.. pro... pros. It's always free, so there is no cost. It improves a lot of things with only nominal negatives. There are very important reasons why you never talk about other options... because no one should be led to believe that there is a use case where not having virtualization is acceptable. That's not to say that at home you can't do whatever you want, of course you can. But even at home you should recognize that doing something without virtualization is risky just for the sake of.. whatever it is making you want to do that.
I absolutely agree. I just wanted to present that caveat. It is absolutely outweighed by all of the benefits.
Q: Have any of you ever attempted to cluster two VM host machines together? Is that a thing?
What do you mean by cluster?
Server clustering as in something akin to data centers where multiple servers run as a single server. That may be old terminology
-
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@coliver said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
Virtual really has no downsides, though. The "single point of risk" is misleading because that's an emotional way of looking at it and empirical. One of the most important reasons that we use virtualization always is because it makes those workloads safer, it doesn't increase risk. Sure, it "adds a point of risk" but it vastly offsets this by making tons of other points safer. This is why terms like "Single Point of Failure" are dangerous - they make people panic over something that might be safe and make them see having two of something as safe even when its dangerous. It's an emotional way to view risks and very dangerous.
Virtualization is not a pro/con situation. It's just pro.. pro... pros. It's always free, so there is no cost. It improves a lot of things with only nominal negatives. There are very important reasons why you never talk about other options... because no one should be led to believe that there is a use case where not having virtualization is acceptable. That's not to say that at home you can't do whatever you want, of course you can. But even at home you should recognize that doing something without virtualization is risky just for the sake of.. whatever it is making you want to do that.
I absolutely agree. I just wanted to present that caveat. It is absolutely outweighed by all of the benefits.
Q: Have any of you ever attempted to cluster two VM host machines together? Is that a thing?
What do you mean by cluster?
Server clustering as in something akin to data centers where multiple servers run as a single server. That may be old terminology
Nope, not really old technology. We do clustering here, to an extent, we have a software that monitors our systems and automatically live migrates VM loads between hosts. So if one host becomes overloaded it migrates some VMs off of it to another one to free up resources.
-
Really the hyperconverged systems, like Scale, are an example of clustering. You take the entire infrastructure as a whole and don't worry too much about the individual nodes. If you need more compute or storage you purchase another node and put it into the RAIN infrastructure and you don't really think about it again.
-
@coliver said in Raid 6 Amateur File Server Setup Questions:
Really the hyperconverged systems, like Scale, are an example of clustering. You take the entire infrastructure as a whole and don't worry too much about the individual nodes. If you need more compute or storage you purchase another node and put it into the RAIN infrastructure and you don't really think about it again.
That was basically the assumption I was making in my head as to how it would be done. Sounds great.
Terminology not technology
-
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
Virtual really has no downsides, though. The "single point of risk" is misleading because that's an emotional way of looking at it and empirical. One of the most important reasons that we use virtualization always is because it makes those workloads safer, it doesn't increase risk. Sure, it "adds a point of risk" but it vastly offsets this by making tons of other points safer. This is why terms like "Single Point of Failure" are dangerous - they make people panic over something that might be safe and make them see having two of something as safe even when its dangerous. It's an emotional way to view risks and very dangerous.
Virtualization is not a pro/con situation. It's just pro.. pro... pros. It's always free, so there is no cost. It improves a lot of things with only nominal negatives. There are very important reasons why you never talk about other options... because no one should be led to believe that there is a use case where not having virtualization is acceptable. That's not to say that at home you can't do whatever you want, of course you can. But even at home you should recognize that doing something without virtualization is risky just for the sake of.. whatever it is making you want to do that.
I absolutely agree. I just wanted to present that caveat. It is absolutely outweighed by all of the benefits.
Q: Have any of you ever attempted to cluster two VM host machines together? Is that a thing?
Misuse of the term cluster here I think. WikiPedia: "A computer cluster consists of a set of loosely or tightly connected computers that work together so that, in many respects, they can be viewed as a single system."
So, technically, Scale setups could be called a cluster. Traditionally it's a term reserved for referencing many computers working on the same task. Things like weather analysis/prediction or nuclear modeling, where you need supercomputer class equipment to get results in a timely manner.
So @wirestyle22, you're thought was correct. You just used a term not really associated with virualization.
-
@travisdh1 said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
Virtual really has no downsides, though. The "single point of risk" is misleading because that's an emotional way of looking at it and empirical. One of the most important reasons that we use virtualization always is because it makes those workloads safer, it doesn't increase risk. Sure, it "adds a point of risk" but it vastly offsets this by making tons of other points safer. This is why terms like "Single Point of Failure" are dangerous - they make people panic over something that might be safe and make them see having two of something as safe even when its dangerous. It's an emotional way to view risks and very dangerous.
Virtualization is not a pro/con situation. It's just pro.. pro... pros. It's always free, so there is no cost. It improves a lot of things with only nominal negatives. There are very important reasons why you never talk about other options... because no one should be led to believe that there is a use case where not having virtualization is acceptable. That's not to say that at home you can't do whatever you want, of course you can. But even at home you should recognize that doing something without virtualization is risky just for the sake of.. whatever it is making you want to do that.
I absolutely agree. I just wanted to present that caveat. It is absolutely outweighed by all of the benefits.
Q: Have any of you ever attempted to cluster two VM host machines together? Is that a thing?
Misuse of the term cluster here I think. WikiPedia: "A computer cluster consists of a set of loosely or tightly connected computers that work together so that, in many respects, they can be viewed as a single system."
So, technically, Scale setups could be called a cluster. Traditionally it's a term reserved for referencing many computers working on the same task. Things like weather analysis/prediction or nuclear modeling, where you need supercomputer class equipment to get results in a timely manner.
So @wirestyle22, you're thought was correct. You just used a term not really associated with virualization.
My thought was working around the downside of VM's being a single point of failure in regards to hardware failure. If we had two VM host machines clustered together, as the wiki defines, setup in the same way wouldn't that provide redundancy and strengthen that weakness? If you have a VM domain and the hardware it's hosted on dies the other VM host machine picks up the job, etc.
-
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@travisdh1 said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
Virtual really has no downsides, though. The "single point of risk" is misleading because that's an emotional way of looking at it and empirical. One of the most important reasons that we use virtualization always is because it makes those workloads safer, it doesn't increase risk. Sure, it "adds a point of risk" but it vastly offsets this by making tons of other points safer. This is why terms like "Single Point of Failure" are dangerous - they make people panic over something that might be safe and make them see having two of something as safe even when its dangerous. It's an emotional way to view risks and very dangerous.
Virtualization is not a pro/con situation. It's just pro.. pro... pros. It's always free, so there is no cost. It improves a lot of things with only nominal negatives. There are very important reasons why you never talk about other options... because no one should be led to believe that there is a use case where not having virtualization is acceptable. That's not to say that at home you can't do whatever you want, of course you can. But even at home you should recognize that doing something without virtualization is risky just for the sake of.. whatever it is making you want to do that.
I absolutely agree. I just wanted to present that caveat. It is absolutely outweighed by all of the benefits.
Q: Have any of you ever attempted to cluster two VM host machines together? Is that a thing?
Misuse of the term cluster here I think. WikiPedia: "A computer cluster consists of a set of loosely or tightly connected computers that work together so that, in many respects, they can be viewed as a single system."
So, technically, Scale setups could be called a cluster. Traditionally it's a term reserved for referencing many computers working on the same task. Things like weather analysis/prediction or nuclear modeling, where you need supercomputer class equipment to get results in a timely manner.
So @wirestyle22, you're thought was correct. You just used a term not really associated with virualization.
My thought was working around the downside of VM's being a single point of failure in regards to hardware failure. If we had two VM host machines clustered together, as the wiki defines, setup in the same way wouldn't that provide redundancy and strengthen that weakness? If you have a VM domain and the hardware it's hosted on dies the other VM host machine picks up the job
Providing you have an HA style system in place.
-
@coliver said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@travisdh1 said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
@wirestyle22 said in Raid 6 Amateur File Server Setup Questions:
@Dashrender said in Raid 6 Amateur File Server Setup Questions:
@scottalanmiller said in Raid 6 Amateur File Server Setup Questions:
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
I'll at least start an installation with one of your recommendations, either on the server now are as a second boot option on my work pc. CentOS would also be good as a server system, or was this a general recommendation to be paired with XenServer?
I'd like to start small with only what's necessary, if that makes sense. If i find it easier than anticipated, I can still go further.Skipping XenServer will often make for "more work". I realize that this is for home and not business so the strong guidance of "always virtualize" still has to be tempered with "do what you are happy with doing." But some of the reasons that we say that everything should be virtual is that it makes things easier, not harder. Technically it is "doing more", but only technically.
Exactly. For example, when you reboot an OS that's installed on physical hardware, you have to reboot the hardware as well as the software. My IBM server takes over 7 mins to initialize the hardware. But, if I'm virtualized that time is zero or near zero. When I reboot a VM, I see a BIOS screen presented by the hypervisor and pop - my OS is booting. Also through the hypervisor I can mount ISOs for booting and troubleshooting. The hypervisor allows me to create virtual hardware to the VM so I don't have to worry about creating real DVDs or USB flash drives with installers on them. Another awesome advantage, backups. The VMs are just giant files to the hypervisor. If a VM is shutdown, you can just copy it anywhere you want, just like any other file.
Yes all of this. @geertcourmacher This also creates a single point of failure for multiple computers instead of one in the case of hardware failure. Most things aren't absolute. There are always advantages and disadvantages which is why you have multiple options. Sometimes the disadvantage is literally only increased cost but it's superior in every other way.
Virtual really has no downsides, though. The "single point of risk" is misleading because that's an emotional way of looking at it and empirical. One of the most important reasons that we use virtualization always is because it makes those workloads safer, it doesn't increase risk. Sure, it "adds a point of risk" but it vastly offsets this by making tons of other points safer. This is why terms like "Single Point of Failure" are dangerous - they make people panic over something that might be safe and make them see having two of something as safe even when its dangerous. It's an emotional way to view risks and very dangerous.
Virtualization is not a pro/con situation. It's just pro.. pro... pros. It's always free, so there is no cost. It improves a lot of things with only nominal negatives. There are very important reasons why you never talk about other options... because no one should be led to believe that there is a use case where not having virtualization is acceptable. That's not to say that at home you can't do whatever you want, of course you can. But even at home you should recognize that doing something without virtualization is risky just for the sake of.. whatever it is making you want to do that.
I absolutely agree. I just wanted to present that caveat. It is absolutely outweighed by all of the benefits.
Q: Have any of you ever attempted to cluster two VM host machines together? Is that a thing?
Misuse of the term cluster here I think. WikiPedia: "A computer cluster consists of a set of loosely or tightly connected computers that work together so that, in many respects, they can be viewed as a single system."
So, technically, Scale setups could be called a cluster. Traditionally it's a term reserved for referencing many computers working on the same task. Things like weather analysis/prediction or nuclear modeling, where you need supercomputer class equipment to get results in a timely manner.
So @wirestyle22, you're thought was correct. You just used a term not really associated with virualization.
My thought was working around the downside of VM's being a single point of failure in regards to hardware failure. If we had two VM host machines clustered together, as the wiki defines, setup in the same way wouldn't that provide redundancy and strengthen that weakness? If you have a VM domain and the hardware it's hosted on dies the other VM host machine picks up the job
Providing you have an HA style system in place.
Probably more than double the cost to do but still neat
-
Mind if I ask you what speaks against the approach of having the OS not on the array? You said performance, number of ports, etc...
Just as a trial I threw in a small SSD connected to the motherboard rather than the controller. It produced the results above; write is not great, but for my purpose OK (and could you really expect much more with RAID6?; Read could have actually been even better; I think 115MB/s should also be close to the maximum of what my Gigabit ethernet can do, right?).
I think with the way I write, that is I copy it over from my main pc, I could even risk enabling write-cache. That would mean that in the event of power-failure the data currently written is lost, right? The likelihood of that happening during I copy something over is almost zero, but even if it happened, I would still have the data on my pc. (Write performance is not so critical, so I won't be doing it, just out of interest).
I have also booted the entire thing with Ubuntu on a stick and I could access the array with no issues.
So it almost seems like a panacea for me; I could switch OS later on, while taking full advantage of the energy conservation options of the controller (switching drives off etc...). The OS would still run on the SSD, instead of having 6 disks spinning up and down for access to the OS alone.
I'm probably missing something though.The issue with drives only being recognized at 1.5GB/s has returned. I think I can rule out a bad cable since the drives are attached to a 1-4 splitter cable (SFF 8087 to 4 SATA) and there is no pattern of, say, one cable producing bad results. I haven't changed any of the cables, and yet the speed seems to change every reboot. Write-performance seems not be impacted (still got around 10MB/s when 5 out of 6 where recognized as SATA I; next time I get 4/6 recognized as SATA II).
-
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
Mind if I ask you what speaks against the approach of having the OS not on the array? You said performance, number of ports, etc...
Just as a trial I threw in a small SSD connected to the motherboard rather than the controller. It produced the results above; write is not great, but for my purpose OK (and could you really expect much more with RAID6?; Read could have actually been even better; I think 115MB/s should also be close to the maximum of what my Gigabit ethernet can do, right?).
I think with the way I write, that is I copy it over from my main pc, I could even risk enabling write-cache. That would mean that in the event of power-failure the data currently written is lost, right? The likelihood of that happening during I copy something over is almost zero, but even if it happened, I would still have the data on my pc. (Write performance is not so critical, so I won't be doing it, just out of interest).
I have also booted the entire thing with Ubuntu on a stick and I could access the array with no issues.
So it almost seems like a panacea for me; I could switch OS later on, while taking full advantage of the energy conservation options of the controller (switching drives off etc...). The OS would still run on the SSD, instead of having 6 disks spinning up and down for access to the OS alone.
I'm probably missing something though.The issue with drives only being recognized at 1.5GB/s has returned. I think I can rule out a bad cable since the drives are attached to a 1-4 splitter cable (SFF 8087 to 4 SATA) and there is no pattern of, say, one cable producing bad results. I haven't changed any of the cables, and yet the speed seems to change every reboot. Write-performance seems not be impacted (still got around 10MB/s when 5 out of 6 where recognized as SATA I; next time I get 4/6 recognized as SATA II).
I assume you are letting your raid controller automatically negotiate the speed. There should be a way to manually set it to 3.0. I would wait until someone more knowledgeable than I am responds as well because I don't know how risky this is to do in your situation. I have read of Adaptec and WD Red's not getting along in this way though.
-
As I have seen sometimes RAID controllers and drives do not play nicely.
Did you Google your drives and the adapter? Maybe there is some info out there.
-
@geertcourmacher said in Raid 6 Amateur File Server Setup Questions:
Mind if I ask you what speaks against the approach of having the OS not on the array? You said performance, number of ports, etc...
Waste of Performance, waste of capacity, extra effort...
Ask the question the opposite way, given that splitting arrays causes huge losses in capacity and performance, what factor would make it a consideration? Try to answer the "pro" of going against standard best practices instead of only looking for the cons of following them.
-
