Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance
-
The SPECTRE / MELTDOWN patches absolutely killed XS performance for some of our workloads. Ensure that your BIOS and other FW is up to date and consider trying XCP-ng.
-
@notverypunny said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
The SPECTRE / MELTDOWN patches absolutely killed XS performance for some of our workloads. Ensure that your BIOS and other FW is up to date and consider trying XCP-ng.
Technically those patches killed performance for everyone, not just XenServer.
-
@DustinB3403 said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@notverypunny said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
The SPECTRE / MELTDOWN patches absolutely killed XS performance for some of our workloads. Ensure that your BIOS and other FW is up to date and consider trying XCP-ng.
Technically those patches killed performance for everyone, not just XenServer.
Fair point. If he's limited to 7.1 for whatever reason it'll be worse, they managed to claw back some of the performance in later releases, but the last patches on 7.1 were deadly. Seems to be worse depending on the specific CPUs from what I was testing on our gear a couple of months back. Our R720s weren't quite rendered useless but it's been a huge factor for new gear for our XenDesktop VDI hosts. The newer procs in the R730s at another site didn't seem to be hit as hard, but it was still significant.
-
@scottalanmiller so what should I do?
-
@scottalanmiller I also recall you telling me years ago, that there was a limit to how many hard drives (spinning rust) I could add to a raid 10 depending on if I was looking for a boost in either read or write. If I recall, read was the only thing that would speed up.
-
@notverypunny See I want to get on the latest and greatest release. I just want to avoid having to nuke everything.
-
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@scottalanmiller I also recall you telling me years ago, that there was a limit to how many hard drives (spinning rust) I could add to a raid 10 depending on if I was looking for a boost in either read or write. If I recall, read was the only thing that would speed up.
That's RAID 1, not RAID 10.
-
@notverypunny I mean after looking around on the net I see what Scott is saying the better performance might actually be better with thick provisioning, but it's negligible when people run analytics on it vs thin. Like only a couple of %.
But I think I see what Scott is also saying that maybe one of the HDD's if possibly throwing errors or failing. I've already replaced 2 drives in 2 years.
-
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@scottalanmiller so what should I do?
Start by looking at the RAID controller and see if the space has been set up yet.
-
@scottalanmiller on one of our servers it's properly done. One the other not so much. I'm going to get the dell engineer I originally worked with to fix the one that isn't working right. So when we use Dell's tool to look at the raid it shows the proper capacity expansion on one server. The other server doesn't show the increase.
When I go into Xen Center, it doesn't "dynamically" see the adjust space. So I see what you're saying I was hoping it would automagically be there, not the case. So how do I get it to add it?
-
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@scottalanmiller on one of our servers it's properly done. One the other not so much. I'm going to get the dell engineer I originally worked with to fix the one that isn't working right. So when we use Dell's tool to look at the raid it shows the proper capacity expansion on one server. The other server doesn't show the increase.
When I go into Xen Center, it doesn't "dynamically" see the adjust space. So I see what you're saying I was hoping it would automagically be there, not the case. So how do I get it to add it?
Use XAPI interface to create a new SR or go through the same process using XO and a gui.
-
Create a new SR? Wouldn't that break something? I'm confident when adding a single or more hard drive to the SR. I'm not very confident when we talking about the RAID
it doesn't seem to make sense. -
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
Create a new SR? Wouldn't that break something? I'm confident when adding a single or more hard drive to the SR. I'm not very confident when we talking about the RAID
it doesn't seem to make sense.You need to create a new SR or expand the existing one. It's a logical volume on top of the block device.
Since expanding a production LV is likely more risky, it's easier to just create a new storage repository on your block device.
-
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
Create a new SR? Wouldn't that break something? I'm confident when adding a single or more hard drive to the SR. I'm not very confident when we talking about the RAID
it doesn't seem to make sense.It's all making perfect sense. Thinking of RAID as a special case is confusing you.
-
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@scottalanmiller on one of our servers it's properly done. One the other not so much. I'm going to get the dell engineer I originally worked with to fix the one that isn't working right. So when we use Dell's tool to look at the raid it shows the proper capacity expansion on one server. The other server doesn't show the increase.
Oh, that would cause an issue if it didn't really expand.
-
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
When I go into Xen Center, it doesn't "dynamically" see the adjust space. So I see what you're saying I was hoping it would automagically be there, not the case. So how do I get it to add it?
First you have to add it in the LVM on the RAID controller after the RAID is expanded. THEN you can look at growing what XC sees.
-
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@notverypunny See I want to get on the latest and greatest release. I just want to avoid having to nuke everything.
I feel your pain, but in the interests of being able to sleep at night you might be better off moving your stuff and starting from scratch.
In your position I would have created a new virtual disk on the raid controller and added it as a new SR, mainly because I've never considered the option of expanding an existing volume, seems risky without any real benefit that I can see right off the bat.
-
@notverypunny said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@notverypunny See I want to get on the latest and greatest release. I just want to avoid having to nuke everything.
I feel your pain, but in the interests of being able to sleep at night you might be better off moving your stuff and starting from scratch.
In your position I would have created a new virtual disk on the raid controller and added it as a new SR, mainly because I've never considered the option of expanding an existing volume, seems risky without any real benefit that I can see right off the bat.
This assumes you can just add more mountpoints to whatever VMs he's using. In some situations you simply want to grow the disk you're using, not add a whole new disk to the VM.
-
@scottalanmiller said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
When I go into Xen Center, it doesn't "dynamically" see the adjust space. So I see what you're saying I was hoping it would automagically be there, not the case. So how do I get it to add it?
First you have to add it in the LVM on the RAID controller after the RAID is expanded. THEN you can look at growing what XC sees.
RAID controller - grow RAID 10 from 4 to 6 disks (or whatever new number you have)
RAID controller - Use LVM to grow the virtual disk the RAID controller presents to Xen Server
Xen Server - Use LVM to grow the SR to include the new space it sees now.
Xen Server - do whatever you need for the VMs to use the new space. -
@Dashrender said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@notverypunny said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@krisleslie said in Having sluggish performance on my Xen Server VM's, looking for suggestions to boost performance:
@notverypunny See I want to get on the latest and greatest release. I just want to avoid having to nuke everything.
I feel your pain, but in the interests of being able to sleep at night you might be better off moving your stuff and starting from scratch.
In your position I would have created a new virtual disk on the raid controller and added it as a new SR, mainly because I've never considered the option of expanding an existing volume, seems risky without any real benefit that I can see right off the bat.
This assumes you can just add more mountpoints to whatever VMs he's using. In some situations you simply want to grow the disk you're using, not add a whole new disk to the VM.
You wouldn't need to add a new disk to the VM to add a vPBD to the hypervisor. You can have the VDI on any SR and "expand" another disks on the VM.
But that's besides the point.
