Reconsidering ProxMox
-
@biggen said in Reconsidering ProxMox:
@scottalanmiller What’s your storage configuration like?
I’ve been playing with it on ZFS Raid 1 mirror. Proxmox OS and VMs all on same mirror. Performance is “OK”. Not as good as MD with same setup though.
Wonder if it’s better to create separate Raid 1 ZFS pools. One for the Proxmox OS and one for the VMs.
We don't use ZFS - slow and we don't want its features (few actually do.) LVM is what we use. What is making you want to look at ZFS? It's not meant for speed and has little generally purpose these days. It's not bad, but mostly it's deployed by accident when people aren't sure what it is. Then people swear by "features" that everything has thinking they are unique to ZFS.
ZFS is a great system, with niche applicability.
-
@biggen said in Reconsidering ProxMox:
Box only has 32GB of RAM so that would mean that ZFS on Proxmox would be using at most 16GB of RAM for the ARC by default. Seems like ZFS needs a ton of RAM.
ZFS can use a lot of RAM, but has no actual requirement for it. And it's important to recognize that ZFS is three products under one name, using it as your filesystem is very different from using it as your RAID, which is very different from using it as your LVM. Each component has different capabilities and caveats and all are separate, just like they always are.
Most people intend to use ZFS in a way that would have to use a lot of RAM. But nothing forces you to do that.
-
@biggen said in Reconsidering ProxMox:
@VoIP_n00b I'll read over your link. I admit I haven't messed with it a ton. Kinda assumed it would work "out of the box" but looks like I need to tinker.
It does, ZFS is tuned by default for the purposes where it is most useful. Which is NOT how 99% of people would want to use it and why it is almost never the right platform for any normal business.
If you tune ZFS to try to mimic MD / LVM / XFS... why not use them as they are faster, better known, more mature and easier? Unless there is something specific to ZFS that you need. Which is normally RAID 7. Using it for RAID 1 puts you automatically in the "never seen someone want to do this before" category.
-
Good read from long ago talking about how ZFS became a big buzzword long after its heyday was over and why it suddenly became promoted so oddly.
-
-
@VoIP_n00b said in Reconsidering ProxMox:
@scottalanmiller said in Reconsidering ProxMox:
LVM is what we use.
So you don't take snapshots?
? You can take snapshots without ZFS?
-
@stacksofplates You can't take snapshots on LVM. https://pve.proxmox.com/wiki/Storage
-
@VoIP_n00b said in Reconsidering ProxMox:
@stacksofplates You can't take snapshots on LVM. https://pve.proxmox.com/wiki/Storage
You snapshot it through the hypervisor? Why would you do it that way?
-
@VoIP_n00b said in Reconsidering ProxMox:
@stacksofplates You can't take snapshots on LVM. https://pve.proxmox.com/wiki/Storage
that says you can to me. And since we do, I'm sure it's correct.
-
@VoIP_n00b said in Reconsidering ProxMox:
@scottalanmiller said in Reconsidering ProxMox:
LVM is what we use.
So you don't take snapshots?
We do, shouldn't, but we do because customers don't want to pay for better backups.
But if it was my own servers, of course not.
-
@stacksofplates said in Reconsidering ProxMox:
@VoIP_n00b said in Reconsidering ProxMox:
@scottalanmiller said in Reconsidering ProxMox:
LVM is what we use.
So you don't take snapshots?
? You can take snapshots without ZFS?
You definitely can. We use that as the backing to our main backup tools on LVM.
-
@scottalanmiller said in Reconsidering ProxMox:
You definitely can. We use that as the backing to our main backup tools on LVM.
Then your using LVM-thin. Maybe that's just a subset of LVM?
-
@VoIP_n00b said in Reconsidering ProxMox:
@scottalanmiller said in Reconsidering ProxMox:
You definitely can. We use that as the backing to our main backup tools on LVM.
Then your using LVM-thin. Maybe that's just a subset of LVM?
LVM-thin is just LVM with the -t flag set when created. They are both equally LVM. It's general practice to use thin provisioning by default and only vary when you have a specific workload that specifically benefits from being thick. Which for us is few and far between for sure. Those workloads exist, but they basically require special underlying hardware as well, like dedicated NVMe. So nothing exotic, but not general purpose, either.
-
@VoIP_n00b said in Reconsidering ProxMox:
@scottalanmiller said in Reconsidering ProxMox:
You definitely can. We use that as the backing to our main backup tools on LVM.
Then your using LVM-thin. Maybe that's just a subset of LVM?
You can snapshot thick provisioned volumes also.
-
@scottalanmiller said in Reconsidering ProxMox:
We do, shouldn't, but we do because customers don't want to pay for better backups.
I am surprised NTG would take on a client like this. We sure wouldn't. The risk is too great. We just tell the client to take there business elsewhere. Maybe we are sending clients to you guys! =P
-
@stacksofplates said in Reconsidering ProxMox:
@VoIP_n00b said in Reconsidering ProxMox:
@scottalanmiller said in Reconsidering ProxMox:
You definitely can. We use that as the backing to our main backup tools on LVM.
Then your using LVM-thin. Maybe that's just a subset of LVM?
You can snapshot thick provisioned volumes also.
Yeah, just for some reason in that table it shows that you can't. My guess is that it is a typo, but I've not tested it so I can't confirm.
-
@scottalanmiller said in Reconsidering ProxMox:
@stacksofplates said in Reconsidering ProxMox:
@VoIP_n00b said in Reconsidering ProxMox:
@scottalanmiller said in Reconsidering ProxMox:
You definitely can. We use that as the backing to our main backup tools on LVM.
Then your using LVM-thin. Maybe that's just a subset of LVM?
You can snapshot thick provisioned volumes also.
Yeah, just for some reason in that table it shows that you can't. My guess is that it is a typo, but I've not tested it so I can't confirm.
It's been a few years since I've done that. But I'm pretty sure you can (on normal systems I have no idea with ProxMox).
-
@VoIP_n00b said in Reconsidering ProxMox:
@scottalanmiller said in Reconsidering ProxMox:
We do, shouldn't, but we do because customers don't want to pay for better backups.
I am surprised NTG would take on a client like this. We sure wouldn't. The risk is too great. We just tell the client to take there business elsewhere. Maybe we am sending clients to you guys! =P
So your reaction to thinking I didn't do snapshots was meant to mean that you were impressed? I feel like you are flipping back and forth.
At first you acted like I was crazy for not doing snapshots. Then when you found out I do use them, you act like MSPs shouldn't take on any client that uses traditional backup software - which all use snapshots.
Are you truly telling me you don't have a single customer that demands you use regular backup software? And that you don't have them because you turn them down? Absolutely send them my way.
My job is to take care of the customer and give good advice. In no way is it my concern if they listen to that advice. I work in IT, I'm not an investor. Only an investor would care about that. I absolutely want those customers that demand we use Veeam or whatever.
-
@stacksofplates said in Reconsidering ProxMox:
@scottalanmiller said in Reconsidering ProxMox:
@stacksofplates said in Reconsidering ProxMox:
@VoIP_n00b said in Reconsidering ProxMox:
@scottalanmiller said in Reconsidering ProxMox:
You definitely can. We use that as the backing to our main backup tools on LVM.
Then your using LVM-thin. Maybe that's just a subset of LVM?
You can snapshot thick provisioned volumes also.
Yeah, just for some reason in that table it shows that you can't. My guess is that it is a typo, but I've not tested it so I can't confirm.
It's been a few years since I've done that. But I'm pretty sure you can (on normal systems I have no idea with ProxMox).
Oh I KNOW it works outside of Proxmox. The question here is if Proxmox has a button to push to make it happen in their interface.
-
@scottalanmiller said in Reconsidering ProxMox:
Yeah, just for some reason in that table it shows that you can't. My guess is that it is a typo, but I've not tested it so I can't confirm.
It's defiantly not a typo. I am the member of a proxmox user group, and someone just did a install with LVM and everything related to snapshots was grayed out. Then we figured out it was LVM and not LVM-thin. So regardless if it's "all the same" it matters to proxmox.
