I have a scenario where we frequently pull in duplicated files that take up a lot of space. There's no reasonable way to stop the intake of these duplicated files due to their irregularity and the method of input.
I've found some graphical apps that do a good job of deduplication / removal of the appropriate files, but I really need this to run as a script without a UI routinely.
Is anyone familiar with a comprehensive script / shell binary that will deduplicate by deleting the most recent version (leaving the oldest version) of files based on an md5 sum + date comparison?
