I have a large amount of files that should never change, including RAW photographs and video files. I'm worried about silent bit rot.
I do have backups to restore lost/corrupted files, but comparing current files against backups is not practical (for example, video files are on digital tapes). Also, my backup software does not provide functionality for this.
Is there a software that scans list of folders, stores reliable checksums and can validate that selection for added/removed/modified (corrupted) files?
There's about 3TB and 21 million files (a large portion of that is really small files, obviously), so memory consumption is important. It should run on Linux, and preferably on OS X too.
Note: on Linux, I'm already running ZFS, which has robust checksums, and
scrub for detecting bitflips. However, it's not possible or practical to use either that or btrfs on OS X / optical disks / USB disks that should be portable (i.e FAT). I highly prefer filesystem agnostic solution.