XenServer Export Performance Seems Poor

scottalanmiller

@olivier said:

@Dashrender The current title is perfect (didn't see the change before)

Maybe Scott edited it before, I haven't changed it since I posted it.

Yeah, that was me

BRRABill

@Dashrender said

Yeah, I'm already seeing that be the case. 2 hours in, 108 GB transferred.

So around 15MBps is the max we can hope for?

Is that what you ended up determining?

Dashrender

I never tested again after this situation. Perhaps when I get back in the office I will try one.

BRRABill

I'm trying to figure out the quickest way to get a VM from one XS to another.

I'll ping @olivier again here, as I am sure XO is part of that discussion.

At least if you export it to your desktop you can follow the progress. But it seems like it does compression on the export, right? Is that was was ascertained from this thread?

Dashrender

@BRRABill said in XenServer Export Performance Seems Poor:

I'm trying to figure out the quickest way to get a VM from one XS to another.

I'll ping @olivier again here, as I am sure XO is part of that discussion.

At least if you export it to your desktop you can follow the progress. But it seems like it does compression on the export, right? Is that was was ascertained from this thread?

The is a way to do an export without compression. This site is not the easiest to look through on my kindle fire. I thought Oliver posted about it in this thread?

BRRABill

@Dashrender said

The is a way to do an export without compression. This site is not the easiest to look through on my kindle fire. I thought Oliver posted about it in this thread?

No, I looked through the whole thread and did not see it.

He only mentions turning it off for backups.

BRRABill

After some research today, I have come to the conclusion (could be wrong) that EXPORT does not compress. I came to this conclusion because a lot of the articles I saw spoke of compressing the export.

But it seems like it was said in the thread it IS enabled.

So confusing...

Take a look at this:
https://www.citrix.com/blogs/2009/02/10/xenserver-tips-and-tricks-minimize-the-size-of-your-exported-virtual-machines/

"In the interest of not overloading the system that is exporting the virtual machine we don’t compress the data that is exported. So to really save on space after you do an export you can then use your favorite compression program to shrink down the exported file even more. "

BRRABill

I also found today the copy through XO is really slow. I wonder why...

Dashrender

XO might be doing the compression, and XC might not be doing compression.

BRRABill

Next week I am going to test everything.

All I know if=s last night when I gave up copying through XO, I was getting out 4-8MBps. A straight export today got me in the 50-60MBps range. Not super fast, but a hell of a lot better. It did 12 GB in around 5 minutes.

I think copy is slow for some reason, but I've looked through so much XS/XO stuff today I can't think anymore.

olivier

XO is activating the compression by default for a VM copy. And because of GZIP and probably a not so fast CPU, the bottleneck is there.

Sadly, we had a choice to make, between faster copy but larger bandwidth usage and the opposite. Changing it is easy, but I have a bad feeling about this (a lot of people not agree with the new choice).

The best would be to have the choice, but it's tricky in term of UI.

Dashrender

@olivier said in XenServer Export Performance Seems Poor:

XO is activating the compression by default for a VM copy. And because of GZIP and probably a not so fast CPU, the bottleneck is there.

Sadly, we had a choice to make, between faster copy but larger bandwidth usage and the opposite. Changing it is easy, but I have a bad feeling about this (a lot of people not agree with the new choice).

The best would be to have the choice, but it's tricky in term of UI.

Didn't there used to be a check box to enable or disable GZIP?

olivier

@Dashrender For backup yes, not for VM copy.

BRRABill

@olivier

Is there a way to export without compression directly from XS, and bypass XC and XO?

It's not something that happens a lot, but boy those slow speeds are a killer. It adds many hours to a job you;d like to happen as quick as possible.

BRRABill

Also...

When I did my test export, I was seeing around 5--60 MBps. Not super fast considering it is GB, but not terrible.

When I did the actual export of my shutdown VM, it slowed down to 20MSps or less. Same setup (aka where the export was going).

Is this because of compression?

I also realized it must read every block in the provisioned disk, because it got to a certain point pretty quickly (90GB) and then just sat and sat for over an hour. (The disk is provisioned at 200GB.)

Seems like there are still a lof of unanswered questions here.

olivier

You can create a basic backup schedule and de-activate the compression, then run the job manually. You'll have an export without compression.

BRRABill

@olivier said in XenServer Export Performance Seems Poor:

You can create a basic backup schedule and de-activate the compression, then run the job manually. You'll have an export without compression.

When I was running my test, it seemed the exporting directly from XC was quicker than running a one time backup from XO of the same VM to the same location. The export of a small VM (I set it up just for testing this) was about 6 minutes, and the backup was 13.

But it sounds like what you are saying is that it should have been the opposite.