Large file sharing to, from and within mainland China
-
@scottalanmiller NM - https://mangolassi.it/topic/12856/nextcloud-and-php-7
I guess it was more related to CentOS 7 and you said you were using Fedora 25 because of that. My go to these days is CentOS 7. Not that I have tons of Linux experience, but I have done most of my tinkering and work with it.
-
Fedora fixes that stuff
-
@scottalanmiller said in Large file sharing to, from and within mainland China:
Fedora fixes that stuff
I am trying to build a VM in my on-prem VMware (version 6) environment to play around with Fedora and NextCloud while I wait for Vultr to build out its inventory. I can't find Fedora listed on the Guest OS section and when I check the guest OS compatibility guide from VMware ( http://partnerweb.vmware.com/GOSIG/home.html ), it only shows 24 and only in the desktop version. Should I just pick RHEL in the New VM configuration?
-
@wrx7m I used to choose Other Linux when i used vmware and specific distro i was using wasnt listed.
-
Picking RHEL 7 is probably fine.
-
@scottalanmiller said in Large file sharing to, from and within mainland China:
Picking RHEL 7 is probably fine.
I tried Other Linux but it only had the E1000 NIC so I tried RHEL 7 and it had the VMXNET 3 NIC so I went with that and am doing the updates on the Fedora installation now.
-
This post is deleted! -
@scottalanmiller I am looking at your post on installing via saltstack and I am not sure where to begin. I posted on that thread to keep it related.
-
@wrx7m said in Large file sharing to, from and within mainland China:
@scottalanmiller I am looking at your post on installing via saltstack and I am not sure where to begin. I posted on that thread to keep it related.
I just responded. I'm on my phone in bed. But I have Salt guides on here separate from this that show how to set up both pieces. All pieces can be on one box. Although that makes it a bit silly.
-
@scottalanmiller Thanks for the reply.
-
So it looks like there are essentially 2 speeds when dealing with China- The inside China speed and the International speed. Their inside speed is 100Mbps but their average international speed (they refer to it as the International Corridor) is 1Mbps. Yeah... 1!
I have yet to find an optimal way of doing this.
I have asked them to find out how to increase their international bandwidth. I was doing some Googling over the weekend and found that larger companies have this capability but I don't know how they are doing it and how much it costs (probably more than we can spend).
Also, if I can work with a Chinese host for a nextcloud server, setting up some public-facing websites, which nextcloud may or may not be classified as, may or may not require some sort of license from the Chinese government, which could take a month or more or get denied. If I can get that setup, then we are still dealing with an issue where we would need to replicate the data across to a second nextcloud server here. From what I was reading on the nextcloud forums, there is no native way of doing this yet and even if we could, we are still constrained by the 1Mbps average international speed.
The user proposed syncing two Synology NAS devices (1 in China and 1 here), but that introduces all sorts of other headaches on how to get the data shared out to our users here and backed up consistently. As we all know, syncing does not equal a backup.
-
@wrx7m said in Large file sharing to, from and within mainland China:
So it looks like there are essentially 2 speeds when dealing with China- The inside China speed and the International speed. Their inside speed is 100Mbps but their average international speed (they refer to it as the International Corridor) is 1Mbps. Yeah... 1!
I have yet to find an optimal way of doing this.
I have asked them to find out how to increase their international bandwidth. I was doing some Googling over the weekend and found that larger companies have this capability but I don't know how they are doing it and how much it costs (probably more than we can spend).
Also, if I can work with a Chinese host for a nextcloud server, setting up some public-facing websites, which nextcloud may or may not be classified as, may or may not require some sort of license from the Chinese government, which could take a month or more or get denied. If I can get that setup, then we are still dealing with an issue where we would need to replicate the data across to a second nextcloud server here. From what I was reading on the nextcloud forums, there is no native way of doing this yet and even if we could, we are still constrained by the 1Mbps average international speed.
The user proposed syncing two Synology NAS devices (1 in China and 1 here), but that introduces all sorts of other headaches on how to get the data shared out to our users here and backed up consistently. As we all know, syncing does not equal a backup.
Would something like RDS or VDI work in this instance?
-
@coliver said in Large file sharing to, from and within mainland China:
@wrx7m said in Large file sharing to, from and within mainland China:
So it looks like there are essentially 2 speeds when dealing with China- The inside China speed and the International speed. Their inside speed is 100Mbps but their average international speed (they refer to it as the International Corridor) is 1Mbps. Yeah... 1!
I have yet to find an optimal way of doing this.
I have asked them to find out how to increase their international bandwidth. I was doing some Googling over the weekend and found that larger companies have this capability but I don't know how they are doing it and how much it costs (probably more than we can spend).
Also, if I can work with a Chinese host for a nextcloud server, setting up some public-facing websites, which nextcloud may or may not be classified as, may or may not require some sort of license from the Chinese government, which could take a month or more or get denied. If I can get that setup, then we are still dealing with an issue where we would need to replicate the data across to a second nextcloud server here. From what I was reading on the nextcloud forums, there is no native way of doing this yet and even if we could, we are still constrained by the 1Mbps average international speed.
The user proposed syncing two Synology NAS devices (1 in China and 1 here), but that introduces all sorts of other headaches on how to get the data shared out to our users here and backed up consistently. As we all know, syncing does not equal a backup.
Would something like RDS or VDI work in this instance?
I have a server, here, running RDS via RDGateway but they are still needing to transfer the large files to, from and within China.
-
@wrx7m said in Large file sharing to, from and within mainland China:
The user proposed syncing two Synology NAS devices (1 in China and 1 here), but that introduces all sorts of other headaches on how to get the data shared out to our users here and backed up consistently. As we all know, syncing does not equal a backup.
That basically never works.
-
@coliver said in Large file sharing to, from and within mainland China:
Would something like RDS or VDI work in this instance?
1Mb/s with high latency... I'd guess no.
-
@scottalanmiller said in Large file sharing to, from and within mainland China:
@coliver said in Large file sharing to, from and within mainland China:
Would something like RDS or VDI work in this instance?
1Mb/s with high latency... I'd guess no.
You're probably right. There doesn't really seem to be a good solution in this event.
-
@scottalanmiller I figured as much. I have a couple synology NAS devices for some scratch space and even then, it is a datastore for VMware. The other is my Veeam backup repo.
-
@wrx7m said in Large file sharing to, from and within mainland China:
@coliver said in Large file sharing to, from and within mainland China:
@wrx7m said in Large file sharing to, from and within mainland China:
So it looks like there are essentially 2 speeds when dealing with China- The inside China speed and the International speed. Their inside speed is 100Mbps but their average international speed (they refer to it as the International Corridor) is 1Mbps. Yeah... 1!
I have yet to find an optimal way of doing this.
I have asked them to find out how to increase their international bandwidth. I was doing some Googling over the weekend and found that larger companies have this capability but I don't know how they are doing it and how much it costs (probably more than we can spend).
Also, if I can work with a Chinese host for a nextcloud server, setting up some public-facing websites, which nextcloud may or may not be classified as, may or may not require some sort of license from the Chinese government, which could take a month or more or get denied. If I can get that setup, then we are still dealing with an issue where we would need to replicate the data across to a second nextcloud server here. From what I was reading on the nextcloud forums, there is no native way of doing this yet and even if we could, we are still constrained by the 1Mbps average international speed.
The user proposed syncing two Synology NAS devices (1 in China and 1 here), but that introduces all sorts of other headaches on how to get the data shared out to our users here and backed up consistently. As we all know, syncing does not equal a backup.
Would something like RDS or VDI work in this instance?
I have a server, here, running RDS via RDGateway but they are still needing to transfer the large files to, from and within China.
What about a more manual process? Like a dedicated machine inside China that has a NextCloud client? Then they manually manage files inside of China, rather than having them be automatically handled elsewhere? I don't know the end user use case, but if there aren't a lot of users, this is likely pretty simple.
-
@scottalanmiller The problems with that, is that we still need to give and receive the large files from them and also get them backed up.
-
WAN Optimization can help, but with users not in a central location in China, it doesn't help much. There are some serious WAN optimization products out there, but this isn't a great use case for them.
Is there anything that you can do to compress these large files? How big are they, how many and what is the change rate on them? And how much is changed in China vs, in the US?
