Backup plan using Robocopy + 7-zip command line (7za.exe)

[Emad R]

Hi,

I know the community outgrew the Windows stuff, and favors Linux (Centos especially), but after being a reader in this community I wanted to share something, hopefully it might help someone and it is very Keep It Simple approach for backups for Windows environments and requires a Windows machine, preferably virtual machine, so here goes:

1)Download 7za.exe from:
http://www.7-zip.org/download.html

Either (old stable) Download 7-Zip 9.20 (2010-11-18) for Windows:
32-bit 7-Zip Command Line Version

Or (latest stable) Download 7-Zip 16.04 (2016-10-04) for Windows:
7-Zip Extra: standalone console version, 7z DLL, Plugin for Far Manager

2)Extract 7za.exe from those archives to the root of your drive (preferably the 32-bit version for broader compatibility).

3)Now to create backup files you will need to use the following plan at the backup location (NAS) create 2 folders:

· Archives
· Mirror

Archives will have multiple versions of archives of the files you select to backup, while Mirror will always have latest sync of the backup, this is intended so we don’t archive the real source files, and instead we archive the Mirror folder.

4)Now on the server that have access to the source files and backup location, create the following .cmd file (Supply_Unit_Backup.cmd) on the Desktop for example, in a folder called Scripts (optional).

robocopy.exe /MIR "source folder" "backup Mirror folder"

C:\7za.exe a –ttar "backup Archives folder\date and time in a windows batch script.tar" "backup Mirror folder"

Real life example of the above:

robocopy.exe /MIR "\192.168.1.200\Public\Supply Unit" "Z:\Supply_Unit_Backup\Mirror"

C:\7za.exe a -ttar "Z:\Supply_Unit_Backup\Archives%DATE:~10,4%%DATE:~4,2%%DATE:~7,2%__%TIME:~-11,2%hour.tar"
"Z:\Supply_Unit_Backup\Mirror"

Do note Robocopy command will do the sync or mirroring of the files, from the source folder to the Mirror folder, and it can support many useful argument, for example if you wish to grab/backup only excel files you can do the following:

robocopy.exe /MIR "source folder" "backup Mirror folder" .xls .xlsx

Also if you want to backup only specific files using robocopy, you will need to run this command instead:

robocopy.exe /MIR "source folder" "backup Mirror folder" uCalendar.ini

And as seen above basically anything that will be placed in the Mirror we will take archive snapshot of it on regular intervals, and robocopy is good with network shares and incremental backup runs.
The 7za a –ttar is recommended (TAR format), but you can choose any other archiving/compression format, I think it makes sense to use 7z format, but the added bonus with .tar is that you can open it in Linux natively, and when you compare archiving file types with each other, there is no real performance winner. However if you want to compress files I highly recommend using 7z format, which can be done by this command:

C:\7za.exe a "backup Archives folder\date and time in a windows batch script.7z" "backup Mirror folder"

And this way you can compress and reduce your backup files sizes, However using tar archive (or 7z with zero compression) a.k.a archiving have its benefits against compression, it is much faster performed and you can test the archives easier, and theoretically reduces corrupt the chance of file corruption; cause it takes less time dealing with the file than with compression.

Now create task with Windows task scheduler to execute the cmd by pointing Windows Task Scheduler to run the script inside the scripts folder at different intervals, depending on your work nature and selected files this can vary from weekly backups to 3 times a day backup.

And you will have similar output like the below:

The fact that we used 24 hour naming scheme will make your life a lot easier recognizing those files, I guess the only limitations we have is with this script that it you can’t run it each 1-59 minutes otherwise it will keep overwriting the same file over and over, for example if you ran the script 2 times at 11:01 and 11:20, you will get only 1 backup file, however if you ran it at 11:01 and 12:01 (hourly based) you will get 2 backup files.
For me this is not issue, cause I never handled situation where backups needed to be done on minutes bases, and I reckon that will be very interfering with the work, notice in the above screenshot I backup every 8PM + 1PM + 8AM, which translates in this company as after working hours when everybody leaves + break time + before working hours.

4)Retention policy:

Now that the backups are getting created you will notice that they can be populated quite easily, so you can create another cmd file (Purge_Backup.cmd) with the following command:

forfiles -p "backup Archives folder" -s -m . -d -8 -c "cmd /c del @file"

And have it run ok weekly basis, starting from the first working day of the week of the company, and what this will do is keep a week worth of backups always.
A real life example of this will be:

forfiles -p "Z:\Supply_Unit_Backup\Archives" -s -m . -d -8 -c "cmd /c del @path"

You can list files without deletion by:

forfiles -p "backup Archives folder" -s -m . -d -8 -c "cmd /c echo @file"

*note: UNC paths (\machine\share) are not supported, you will need to MAP them to a drive letter.

5)You can then test the backups manually (tar files) by selecting them all and right clicking them on the backup destination -> 7-zip -> test archive (you will need to have 7zip installed for the context menu to appear), or test from command line and schedule it, but I didn’t need to do this cause once you use archiving with zero compression format, everything just works. Especially since we move the files to another location first then we backup. This seems like a good method to backup but may not be appealing if you many large files.

You can test backups using

7za.exe t "backup Archives folder *.tar" >backup_log.txt

Then open the txt file and ensure the backup test result is = "Everything is Ok".

Extra Stuff:

You can use 7za to capture many folders and generate 1 file using windows system variables and produce log and have notepad open it for end users, for example:

7za.exe a -ttar "Backup%DATE:~10,4%%DATE:~4,2%%DATE:~7,2%__%TIME:~-11,2%hour.tar" "%USERPROFILE%\Desktop" "%USERPROFILE%\Documents" "%USERPROFILE%\Contacts" "%USERPROFILE%\Music" "%USERPROFILE%\Pictures" "%USERPROFILE%\Videos"

Will create backup folder, wherever you run this script and will take the following folders:

Desktop Music Documents Pictures Contacts Videos

And create singe tar archive out of them. Then using this command:

7za.exe t "Backup*.tar" >backup_log.txt

msg * Notepad will open shortly. Please ensure the backup test result is "Everything is Ok".

notepad.exe backup_log.txt

You can instruct users on how to verify that their backups are complete. This can work as no software approach for users to backup manually, you will be surprised with 7za engine and how fast it can capture data and create the archive, especially since there is no GUI involved. Also with a bit of tinkering and closing programs like Outlook automatically by the script you can actually make it work. I also use with the above script that is more targeted for users with a program called everything which search using indexing of NTFS volumes and I can pass it:
Everything.exe -admin -s "*.pst | *.ost" which will show the user the location of their Outlook files, so they can add it in the backup folder, and tell them to only launch the script from inside of their N.A.S secure share on weekly/monthly basis.

[iroal]

I use a very similar method with robocopy.

I like to receive an email with the log so I use blat.exe to send and email with the robocopy Log

blat.exe -to [email protected] -server smpt.server -f [email protected] -subject "Backup Diario" -body "Backup diario a Nas" -attach c:\Robocopy\Logs\Robo_logs.7z

[Emad R]

@iroal
Great minds think alike !

Thanks for the addition, I'm researching right now and blat.exe seems very worthy looking and simple with good reviews on its Sourceforge page:
https://sourceforge.net/projects/blat/