Download and Delete from Source every 5 minutes (Ubuntu)
-
@Sparkum said in Download and Delete from Source every 5 minutes (Ubuntu):
No problem using rsync with a ftp I assume?
Rsync and FTP are different protocols. The term "using rsync with FTP" is meaningless. Imagine saying "using email with a phone call." What does that even mean?
Rsync and FTP are competing options. You can only use the technologies that the site you are downloading from provides. If they only offer FTP, you can only use FTP, it's that simple.
-
Let's start with the source, what is the actual source of this data. And why do you need it every five minutes? This is a weird task and I think we need to understand the real goals to really know what is and isn't a reasonable approach.
-
@Sparkum said in Download and Delete from Source every 5 minutes (Ubuntu):
Alright think I've got it
So mount my drives with fstab and then just a simple
rsync --remove-source-files -options /path/to/src/ /path/to/dest
Seems that this would be a good option.
-
I'm trying to mount the drive under /etc/fstab but I cant seem to get it to work.
I have my local windows drive mounted, just cant get the remote Ubuntu drive mounted.
I'm assuming my problem child will either be firewall or port, does anyone know what port mounting the drive takes on Ubuntu?
-
@Sparkum said in Download and Delete from Source every 5 minutes (Ubuntu):
I'm trying to mount the drive under /etc/fstab but I cant seem to get it to work.
I have my local windows drive mounted, just cant get the remote Ubuntu drive mounted.
I'm assuming my problem child will either be firewall or port, does anyone know what port mounting the drive takes on Ubuntu?
What is the mounting protocol in question?
-
Trying to mount using /etc/fstab
So I'm doing...
//XXX.XXX.XXX.XXX/var/www/completed /mnt/ftp cifs username=root,password=PASSWORD,iocharset=utf8,sec=ntlm,dir_mode=0777,file_mode=0777 0 0
but I'm getting error code 115
So I'm wondering if there's a certain port that needs to be opened up, or a typical thing that has to be done when connecting to remote computers (both ubuntu 14.04)
-
@Sparkum said in Download and Delete from Source every 5 minutes (Ubuntu):
Trying to mount using /etc/fstab
So I'm doing...
//XXX.XXX.XXX.XXX/var/www/completed /mnt/ftp cifs username=root,password=PASSWORD,iocharset=utf8,sec=ntlm,dir_mode=0777,file_mode=0777 0 0
but I'm getting error code 115
So I'm wondering if there's a certain port that needs to be opened up, or a typical thing that has to be done when connecting to remote computers (both ubuntu 14.04)
Can you run the mount commands by hand and have it work?
-
@Sparkum said in Download and Delete from Source every 5 minutes (Ubuntu):
Trying to mount using /etc/fstab
So I'm doing...
//XXX.XXX.XXX.XXX/var/www/completed /mnt/ftp cifs username=root,password=PASSWORD,iocharset=utf8,sec=ntlm,dir_mode=0777,file_mode=0777 0 0
but I'm getting error code 115
So I'm wondering if there's a certain port that needs to be opened up, or a typical thing that has to be done when connecting to remote computers (both ubuntu 14.04)
Why are you using CIFS between Linux machines? This seems like it is being treated in a very complicated way for something that should be really simple.
-
Why are you not just using Rsync?
-
Because thats what was suggested below, thats about it.
And what should I use to mount between linux machines?
-
NFS tends to work better between Linux boxes.
-
@Sparkum said in Download and Delete from Source every 5 minutes (Ubuntu):
Because thats what was suggested below, thats about it.
And what should I use to mount between linux machines?
Who suggested that? CIFS would be the worst option. Rsync the best. NFS in the middle.
Only NFS between LInux machines. But we need to back up. Why are you mounting anything? Mounting sounds like the wrong approach here. Hence why I keep saying Rsync instead of mounting.
-
@dafyre said in Download and Delete from Source every 5 minutes (Ubuntu):
NFS tends to work better between Linux boxes.
A LOT better...
- Native libraries.
- Faster
- More stable
- Must less to configure
- No weird permissions mismatches
-
@Sparkum said in Download and Delete from Source every 5 minutes (Ubuntu):
So mount my drives with fstab and then just a simple
rsync --remove-source-files -options /path/to/src/ /path/to/dest
This is where the error crept in. You added "mount my drives" as a step. Remove that. You ONLY Rsync, no mounting of drives at all.
-
I don't see a post where anyone suggested mounting anything and especially no mention of CIFS. Maybe I am missing it.
For future reference, CIFS or SMB as it is properly known, is the Windows Network File protocol. It has one and only one useful purpose - talking to Windows. This is because it is native to Windows and is the only protocol that Windows handles well, ever. Windows "can" do NFS, but you need an add on package and it is totally garbage.
You only use SMB (CIFS) when you are trying to talk to Windows. If you are not talking to Windows, you avoid it entirely. NFS is generally the better protocol.
-
Ooh ok gotcha,
I assumed I mounted and then simply went from mnt/server to local kinda thing.Looking into NFS right now as well.
Thanks
-
I've mounted to Windows a few times, so what I knew was CIFS, didnt know it was only meant for Windows, thanks for clearing that one up.
-
@Sparkum said in Download and Delete from Source every 5 minutes (Ubuntu):
Ooh ok gotcha,
I assumed I mounted and then simply went from mnt/server to local kinda thing.Looking into NFS right now as well.
Thanks
No, don't look at NFS either. Use Rsync!!
Look at NFS for a future project where you need to mount things.
-
@Sparkum said in Download and Delete from Source every 5 minutes (Ubuntu):
I've mounted to Windows a few times, so what I knew was CIFS, didnt know it was only meant for Windows, thanks for clearing that one up.
SMB (CIFS) is recently the official protocol for Mac OSX too, replacing AFS which is what they used to use.
NFS is the official protocol for "all" UNIX systems (by convention) including all major Linux, FreeBSD, OpenBSD, NetBSD, Dragonfly, Solaris, HP-UX and AIX. People say "all UNIX" but Mac OSX is UNIX and uses SMB officially (but supports NFS just fine.) And there could be other exceptions. But all the UNIX Server OSes use NFS natively.
-
Hey look at that Rsync is working!
Thanks guys,
I'll let this sync the files then I'll will start looking into Cron jobs in the meantime
Thanks all!
