Robocopy/7-zip bat file with high compression ratio

Clem

Re: Backup plan using Robocopy + 7-zip command line (7za.exe)
@@@@@@@@@@@@ Thought I should share @@@@@@@@@@@@

This script was adapted from another sourcecode but modified script to accommodate Robocopy.
does the following:
Initial setup provides whether to have "full", "weekly incremental" or "alternate day" backups.
1.) copies the folders using robocopy with multi-thread,
2.) 7z the files with highest compression,
3.) adds the checksum at the end of the archive as a separate text file.


@echo off
::**** start debug syntax ***
::call :main >debug_Logfile_BackupScript_revC.txt 2>&1
::pause
::exit/b
::**** end debug syntax ***
::**** start INI file  ***
IF EXIST "TS-Backup_revD.ini" DEL /f /q "TS-Backup_revD.ini"
call :build-config >TS-Backup_revD.ini 2>&1 
:build-config
echo # Enter file and folder names, one per line. > TS-Backup_revD.ini
echo C:\21EC2020 >> TS-Backup_revD.ini
::echo C:\21EC2020\writeup\folder to back-up >> TS-Backup_revD.ini
::**** end of INI file ***
REM BackupScript
REM Version 1.01, Updated: 2008-05-21
REM By Jason Faulkner (articles[-at-]132solutions.com)
:::::http://www.pcmech.com/article/windows-batch-script-to-backup-data/
:::::::::http://jasonfaulkner.com/RandomChars.aspx

REM Performs full or incremental backups of folders and files configured by the user.

REM Usage---
REM   > BackupScript

::: -- Version History GÇô-
:::(Major version).(Minor version).(Revision number).(Build number)
:::          XX.XXX         YYYYMMDD Author      Description
SET "version=01.1-beta  &:20080521 J.Faulkner  initial version, providing the framework
SET "version=01.0.1"    &:20121031 C.Reyes     framework ready
SET "version=01.0.1"    &:20121031 C.Reyes     Set default settings
SET "version=01.0.2"    &:20141208 C.Reyes     Added windows 7 parameters
SET "version=01.1.0"    &:20160828 C.Reyes     modded xcopy to robocopy windows 7 parameters
SET "version=01.1.1"    &:20160828 C.Reyes     changed date format to accomomdate robocopy windows 7 parameters
SET "version=01.1.2"    &:20160830 C.Reyes     Added Log parameter from robocopy
SET "version=01.2.0"    &:20191030 C.Reyes     increased compression size x10
SET "version=01.2.1"    &:20191101 C.Reyes     Added timestamp
SET "version=01.2.2"    &:20191101 C.Reyes     Added HASH (SHA256) file for user checksum validation
SET "version=01.2.2"    &:20191114 C.Reyes     updated 7z to highest compression ratio syntax to all compatible 7z versions
SET "title=%~nx0 - version %version% #####  Compressing Backup. DO NOT CLOSE   #####"
title %title%
SET SUBDIR=%~dp0
SETLOCAL ENABLEEXTENSIONS ENABLEDELAYEDEXPANSION
:main
::mode 50,3
color 0e
echo.
echo.   This will take awhile. Please Wait....
echo.
REM ---Configuration Options---

REM Folder location where you want to store the resulting backup archive.
REM This folder must exist. Do not put a '\' on the end, this will be added automatically.
REM You can enter a local path, an external drive letter (ex. F:) or a network location (ex. \\server\backups)
:: windows XP OS
::SET BackupStorage=C:\Bkup_assembly_logfile
:: Windows 7 OS
SET BackupStorage=%subdir%

REM Which day of the week do you want to perform a full backup on?
REM Enter one of the following: Sun, Mon, Tue, Wed, Thu, Fri, Sat, *
REM Any day of the week other than the one specified below will run an incremental backup.
REM If you enter '*', a full backup will be run every time.
SET FullBackupDay=*
rem only used for major large backups over 1 terabyte 
::SET FullBackupDay= Tue

REM Location where 7-Zip is installed on your computer.
REM The default is in a folder, '7-Zip' in your Program Files directory.
SET InstallLocationOf7Zip=%ProgramFiles%\7-Zip

set hh=%time:~0,2%
if "%hh:~0,1%" == " " @set hh=0%hh:~1,1%
set min=%time:~3,2%
set ss=%time:~6,2%
set timeStamp=-%hh%.%min%.%ss%

REM +-----------------------------------------------------------------------+
REM | Do not change anything below here unless you know what you are doing. |
REM +-----------------------------------------------------------------------+

REM Usage variables.
Set logs=%subdir%
SET exe7Zip=%InstallLocationOf7Zip%\7z.exe
SET dirTempBackup=%TEMP%\backup
SET filBackupConfig=%subdir%\TS-Backup_revD.ini
IF EXIST "%logs%%nowdate%%timeStamp%_Userdrive_log.txt" DEL /f /q "%logs%%nowdate%%timeStamp%_Userdrive_log.txt"
REM Validation.
IF NOT EXIST %filBackupConfig% (
  ECHO No configuration file found, missing: %filBackupConfig%
  GOTO End
)
IF NOT EXIST "%exe7Zip%" (
  ECHO 7-Zip is not installed in the location: %dir7Zip%
  ECHO Please update the directory where 7-Zip is installed.
  GOTO End
)

REM Backup variables.
FOR /f "tokens=1,2,3,4 delims=/ " %%a IN ('date /t') DO (
  SET DayOfWeek=%%a
  SET NowDate=%%d-%%b-%%c
  SET FileDate=%%d%%b%%c
  rem SET FileDate=%%b-%%c-%%d
)

IF {%FullBackupDay%}=={*} SET FullBackupDay=%DayOfWeek%
IF /i {%FullBackupDay%}=={%DayOfWeek%} (
  SET txtBackup=Full
  SET swXCopy=/e 
) ELSE (
  SET txtBackup=Incremental
  SET swXCopy=/s /maxage:%FileDate%
)

ECHO Starting to copy files.
IF NOT EXIST "%dirTempBackup%" MKDIR "%dirTempBackup%"
FOR /f "skip=1 tokens=*" %%A IN (%filBackupConfig%) DO (
  SET Current=%%~A
  IF NOT EXIST "!Current!" (
    ECHO ERROR! Not found: !Current!
  ) ELSE (
    ECHO Copying: !Current!
    SET Destination=%dirTempBackup%\!Current:~0,1!%%~pnxA
    REM Determine if the entry is a file or directory.
    IF "%%~xA"=="" (
      REM Directory. Ensure the /L is removed before final archive otherwise the zipfile is empty.
      roboCOPY "!Current!" "!Destination!" *.ini *.rcj *.wsf *.ttl *.cmd *.bat *.HTA *.vbs *_CSCcode_*.txt /NDL /NFL /xo /nc /np /ns /ia:RASHCE /r:4 /w:5 /xx /is /MT:128 %swXCopy% /log+:"%logs%%nowdate%%timeStamp%_Userdrive_log.txt"
    )
  )
)
ECHO. Done copying files.
ECHO.

SET BackupFileDestination=%BackupStorage%\TS-scripts_Backup_%FileDate%%timeStamp%_%txtBackup%.7z
set BackupFileDestination-hash=%BackupStorage%\TS-scripts_Backup_%FileDate%%timeStamp%_%txtBackup%.txt

REM If the backup file exists, remove it in favor of the new file.
IF EXIST "%BackupFileDestination%" DEL /f /q "%BackupFileDestination%"
IF EXIST "%BackupFileDestination-hash%" DEL /f /q "%BackupFileDestination-hash%"

ECHO Compressing backed up files. (New window)
REM Compress files using 7-Zip in a lower priority process.

Rem This method may have a High "CPU usage"
:: 7z switch definitions https://superuser.com/questions/281573/what-are-the-best-options-to-use-when-compressing-files-using-7-zip/1449735
START "Compressing Backup. DO NOT CLOSE" /low /W /B "%exe7Zip%" a -mx=9 -mfb=273 -ms=on -r "%BackupFileDestination%" "%dirTempBackup%\"
ECHO. Done compressing backed up files.
ECHO.

:: added hash file
:: Pass the file to hash as the only arg
Set "MD5="
For /f "skip=1 Delims=" %%# in (
  'certutil -hashfile "%BackupFileDestination%" MD5'
) Do If not defined MD5 Set MD5=%%#
Set MD5=%MD5: =%
Echo %MD5% > %BackupFileDestination-hash%

ECHO Cleaning up.
IF EXIST "%dirTempBackup%" RMDIR /s /q "%dirTempBackup%"
ECHO.
echo. xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
echo. File was Archived on Computer: >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
echo.                                %computername% >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
echo. File used for backup is: >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
echo.                                %title% >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
echo. xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
::mode 30,15
color 0e
cls
echo   ??????????????????????????
echo   ?                        ?
echo   ?       DONE...          ?
echo   ?                        ?
echo   ?   Thank You for        ?
echo   ?    your patience.      ?
echo   ??????????????????????????
:Pause
:End
ECHO Finished.
ECHO.
ENDLOCAL

Emad R

try to use those

simple put ``` at the start and end to make block

And that looks very complex, robocopy is meant to be easy I used it back in the day

simply

robocopy /mir /mt source target

anytime you want to make full copy make 7zip archive of it, like snapshot.

bbigford

@Emad-R said in Robocopy/7-zip bat file with high compression ratio:

try to use those
simple put ``` at the start and end to make block
And that looks very complex, robocopy is meant to be easy I used it back in the day

simply
robocopy /mir /mt source target
anytime you want to make full copy make 7zip archive of it, like snapshot.

I use /mir /mt: most often. Works fine. I sometimes forget /mt: and immediately see slowness with large jobs.

Clem

@Emad-R
Thanks for the tip. updated.