@black3dynamite said in What Are You Doing Right Now:
So much work to find out about pending reboots in Windows 10.
https://adamtheautomator.com/pending-reboot-registry-windows/
Why check? At this point in time you can just assume there is a pending reboot.
@scottalanmiller said in QoS on Edgerouter Lite:
The problem with limiting him is that a whole department waits on that download that he does. So while he cripples the network, slowing him down would cripple the team.
That wasn't mentioned in any post.
But if it's a recurring thing, why are the files not downloaded during the night?
Or if it has to be during the day, isn't it possible in the Edgerouter do de-prioritize based on IP for the server, desktop or wherever the download is happening?
Snappy web apps is a question of primarily latency, while download speed is a question of bandwidth over long time.
I don't know if it's possible on the Edgerouter to set a bandwidth limit per user (IP address), for instance during working hours. But if it's possible it will solve the problem because one user can't hog all the capacity.
@scottalanmiller said in Converting VMware VMs to KVM:
and they require remote GUI management
Moot I know but to get that you just install Windows 2019 Server with Desktop Experience. It's the LTSC release.
I'd like to put it behind a firewall. It could be a virtual one. One connection to the outside, one to the ERP server and one to the RRP server(s). That would allow you to only have minimal traffic without the problem of doing it on the W2008 servers. You could also limit clients access to the RDP servers. If you need to change anything it would be done in the firewall.
I don't know if RDP on W2008 is a security problem. I don't think so but if it was you could put in a VPN or something.
Next and perhaps most importantly, I would take one master snapshot. And revert back to that same snapshot every night automatically. Since it's a read-only system for archival purposes there is no point in having it save anything. If Windows could be run on a read-only filesystem that have been even better but I don't think it can. So this system will start fresh every morning. It doesn't matter what someone did or didn't do.
If you want to check the integrity of a bunch of files you can do it with md5deep, which can be thought of as a recursive version of md5sum. It was initially designed for forensic work.
If a file has the same hash as another file they are identical. If you save the md5 hash of a file and later recheck it, you can be sure the file hasn't been changed, corrupted or tampered with.
You'll find it in the package md5deep.
apt install md5deep
Inside the package you'll also find sha256deep and some other good stuff. Use sha256deep instead if you want to use sha256 hash. It's better and actually more secure than md5 but might be slower. You use it in the exact the same way though.
Besides linux it's also available on other OSs such as Windows, MacOS. You can build it from source too. https://github.com/jessek/hashdeep
md5deep -rl /check_this_dir/* > files.md5
This will create a text file (files.md5) with the md5 hash of all files (*) in the "/check_this_dir" directory.
md5deep -rlX files.md5 /check_this_dir/*
It will return the files that don't match. So if any file has been changed, it will show up.
-r is to go into subdirectories as well
-l is to use local paths instead of absolute paths
-X is to do check the signatures
-e is if you want to see the progress while it's working.
Find more info on basic usage with examples here:
http://md5deep.sourceforge.net/start-md5deep.html#basic
Let's check that our files in /boot and it's sub-directories stays intact.
First let's create an md5 file that we will compare with.
md5deep -r /boot/ > boot.md5
Let's verify the files have not been tampered with.
md5deep -rX boot.md5 /boot/
If a file or several files has been changed it will return the file and the new hash (exit code 1).
If all is good it will not return anything (exit code 0).
@DustinB3403 said in Tar gzip file compression calculation without decompressing the file:
@Pete-S So the simplest way I can think to explain this would be like this.
You have a network share which is relatively organized
You create a compressed tarball of any folder on that share and then move that tarball to offsite storage.
How would I realistically get a hash of that folder pre and post tar and compression and have it make sense? They aren't the same thing, even if they contain the same things.
@Pete-S said in Tar gzip file compression calculation without decompressing the file:
Is it safe to assume that the gzip file is correct when it is created?
This is what I'm looking to verify
I'm assuming that files are static during backup.
If you first of all run md5deep on all files in the folder, you'll create a textfile that contains md5 (or sha256 or what you want) signatures on every file in the folder. Place it into the folder so it ends up inside the backup and you'll always have the ability to verify any uncompressed individual file.
If you really want to verify your tar.gz file after it's created I think you have to decompress the files to a temporary folder, run md5deep on the files to compare them with the original file. What you really are testing is that the backup-compress-decompress-restore operation is lossless on every file. It should be by design, but if there is an unlikely bug somewhere it's technically possible that it might not be.
If you use the gzip compression with tar, gzip has a CRC-32 checksum inside that can be used to verify the integrity of the gzip file.
Or to be even more certain you can create an md5 signature of the entire gzip archive with md5sum or md5deep. Then you can always verify that the archive has not been corrupted.
If you ever need to restore the files you can verify the integrity of the restored files with the md5 you created on the original files, before you did the backup.
@DustinB3403 said in Tar gzip file compression calculation without decompressing the file:
@Pete-S said in Tar gzip file compression calculation without decompressing the file:
@DustinB3403 said in Tar gzip file compression calculation without decompressing the file:
I guess a more relevant way to have expressed my question would have been to ask:
How do you verify what is in your tarball before you offload it in a quick and efficient manner?
I want to trust, but verify (as this is a backup).
Generally done with a hash of the files - sha-256, md5 or similar.
And how would that work from my source of unpacked files?
What point in the chain from original file -> backup -> tar ball -> gzip -> offload to archive do you want to verify?
Is it safe to assume that the gzip file is correct when it is created?
@DustinB3403 said in Tar gzip file compression calculation without decompressing the file:
I guess a more relevant way to have expressed my question would have been to ask:
How do you verify what is in your tarball before you offload it in a quick and efficient manner?
I want to trust, but verify (as this is a backup).
Generally done with a hash of the files - sha-256, md5 or similar.
@DustinB3403 said in Tar gzip file compression calculation without decompressing the file:
So I have a tar.gz file that I've built, it's about 95GB compressed from what the system shows. I would like to determine the uncompressed size of the tar file without actually decompressing the file.
Using
gzip -l file.tar.gzshould work, but reports an incorrect record (total size in bytes is just over 1GB).How else should I do this?
I'm not sure exactly what you're after. I think the gzip file format header doesn't include the original file size. So the only way to calculate how big the uncompressed file becomes, is to look through the entire compressed file.
The problem with decompression is that you have to make a big file and it takes time. If you however decompress on the fly and calculate the number of bytes you will avoid that problem.
gzip -c -d file.gz | wc -c