Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s
-
it barely gets better on any host. I have roughly 10-20 vm's on each host. 3 at work are in production, 1 server at my home is being setup. Out of the 4 servers, 2 are brand new. The others aren't super old just 2.5 years old. I have my own lab and 2 servers also. Only 1 of my home servers is in use, but its running a Raid 0 for 2 10K RPM drives. All my servers have crappy speed when it pertains to windows file transfer. I do use linux also but I'm not as proficient with it so I focus right now only on Windows. My new servers are R540's one with full SSD's the other with 4 SAS 10K RPM hdd's. Only the full ssd host is in production. XCP-NG is installed on a BOSS ssd.
Just for reference, my home server is a Dell R220 and a R710. I've installed XCP-NG on everything you can imagine, still same issue. Bottleneck of Windows and I did confirm on my R710 it does use a broadcom nic.
-
Also I gave up using thin provisioning in production unless its a damn good reason to be honest. Just seem to be better off with thick and keep it moving.
-
Iām also using r710 for my home lab but Iām using Fedora KVM. I like the flexibility of being able to use LVM, LVM Thin, or file-based for my VMs on the same host.
-
@krisleslie said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
Nothing has raised a flag to that being an issue. Speeds like that have been present for literally every server I've used over the years with virtualization. I have a fresh new server sitting behind me that when I finish installing XCP to it tonight I bet barely yields much of anything better ;(
No issues on any of our hosts. We get great speeds all of the time.
-
@krisleslie said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
Honestly, every system I've virtualized ever has had the same issue with windows slow as hell file transfers.
What's the common factor?
-
@krisleslie said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
I use to believe heh maybe there is a driver issue or OEMU but I've not seen anything like that.
You are definitely using PV drivers?
-
-
@black3dynamite Scott has been begging me for a while to switch over to KVM
I have enough servers at home to give it a try. What are you using for host management? Last I saw, Kimichi(sp?) was hot. -
@krisleslie said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
@black3dynamite Scott has been begging me for a while to switch over to KVM
I have enough servers at home to give it a try. What are you using for host management? Last I saw, Kimichi(sp?) was hot.You can also use Cockpit.
-
@krisleslie said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
it barely gets better on any host. I have roughly 10-20 vm's on each host. 3 at work are in production, 1 server at my home is being setup. Out of the 4 servers, 2 are brand new. The others aren't super old just 2.5 years old. I have my own lab and 2 servers also. Only 1 of my home servers is in use, but its running a Raid 0 for 2 10K RPM drives. All my servers have crappy speed when it pertains to windows file transfer. I do use linux also but I'm not as proficient with it so I focus right now only on Windows. My new servers are R540's one with full SSD's the other with 4 SAS 10K RPM hdd's. Only the full ssd host is in production. XCP-NG is installed on a BOSS ssd.
Just for reference, my home server is a Dell R220 and a R710. I've installed XCP-NG on everything you can imagine, still same issue. Bottleneck of Windows and I did confirm on my R710 it does use a broadcom nic.
I have a 10G and a 12G Dell in production at work, and an older HP server at home, all with XCP-ng. None of them have storage speed issues. Mix of thin and thick, too. I definitely prefer thin.
I hate to say this, because it seems obvious, but are you transferring millions of tiny files? Because that is always slow AF.
-
I would poke around https://xcp-ng.org/forum/category/2/storage a little and maybe there are other suggestions for tweaking your storage performance. I know it comes up a lot over there.
-
@krisleslie said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
@black3dynamite Scott has been begging me for a while to switch over to KVM
I have enough servers at home to give it a try. What are you using for host management? Last I saw, Kimichi(sp?) was hot.For host management, it will be Cockpit. Just make sure
cockpit-machinesplugin is installed.
sudo dnf install cockpit cockpit-storaged cockpit-machines -
@bnrstnr well hate to tell you but you hit the nail on the head. It's literally 1000's of small files between 100-500k.
-
@black3dynamite thats why you the man, I sure will.
-
@krisleslie said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
@bnrstnr well hate to tell you but you hit the nail on the head. It's literally 1000's of small files between 100-500k.
Try zipping them into a single, or multiple large files, then transfer.
I have found that transferring via xcopy is much faster for small files than just drag and drop in windows.
-
@Dashrender said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
I have found that transferring via xcopy is much faster for small files than just drag and drop in windows.
I use Robocopy, I believe it's more powerful, but I think it's a little more complex too. Either will be way better at handling all of those small files than copying them in explorer.
-
@bnrstnr said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
@Dashrender said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
I have found that transferring via xcopy is much faster for small files than just drag and drop in windows.
I use Robocopy, I believe it's more powerful, but I think it's a little more complex too. Either will be way better at handling all of those small files than copying them in explorer.
yeah, xcopy when it's a simple copy.. Robo when you need more power.
-
@Dashrender I actually started the job off with Robocopy. :0) 500 GB got transferred after about 12 hours or so. Thats SLOW.
-
@krisleslie said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
@Dashrender I actually started the job off with Robocopy. :0) 500 GB got transferred after about 12 hours or so. Thats SLOW.
well, small files.
Now, make 500 1 GB files and see how fast it goes.
-
@Dashrender said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
@krisleslie said in Using XCP-NG latest release, Windows 2019 Server VM file copies are pegged at 10 MB/s:
@Dashrender I actually started the job off with Robocopy. :0) 500 GB got transferred after about 12 hours or so. Thats SLOW.
well, small files.
Now, make 500 1 GB files and see how fast it goes.
Or 1 500GB file. Even better.
