I need to see if there is a tool out there that can help me with a project I am working on. I have a remote set of files that we have imported a few TBs, the remote site has sent me a text file that contains a very long list of each filename and its md5 hash. I need a tool or script that will allow for an automated approach to make sure each file has a match. I need to take each file name and hash one by one from the list and compare with the imported files for the matching file and hash to validate the data is the same. If a match is not found I would like to create a report of the file name and hash from the list that are missing a matching imported file. Any information on the best approach to take either for an existing software tool or script creation is appreciated.
I was dealing with this issue today as well. It turns out that the standard
*sum utilities in Linux take care of this with their
-c option for checking with a filename argument. Here's an example of going through the whole process with
Downloads !104 $ ls *.rpm rescuetime_current_amd64.rpm slack-3.3.7-0.1.fc21.x86_64.rpm zoom_x86_64.rpm Downloads !105 $ md5sum *.rpm > md5sum.txt Downloads !105 $ cat md5sum.txt 4abe6d4a8a279d0b0bb45535bb48d566 rescuetime_current_amd64.rpm 68dff7bb9eaafa0df5637f576c54641d slack-3.3.7-0.1.fc21.x86_64.rpm da0540c22c46e2264d62eac9deddf8b0 zoom_x86_64.rpm Downloads !106 $ md5sum -c md5sum.txt rescuetime_current_amd64.rpm: OK slack-3.3.7-0.1.fc21.x86_64.rpm: OK zoom_x86_64.rpm: OK
It might also be worth noting that md5 is deprecated and newer tools are available. Even if you're not worried about a nefarious attacker it is nice to reduce the remote chance of having a false validation.