check if folder is completely copied

Question

I have a number of folders that may contain duplicated content one of which will be the destination folder. I would like a commandline app to tell me which folders are safe to delete as there is a duplicated file in the destination folder and there are no conflicts.

so it should give a conflict warning for

• files with same name and size but diff paths.
• if there are files with the same size in diff paths, a matching file must be in dest.
• files should not use the same path + name in different folders if they are diff sizes

it would be nice if it was cross platform windows, linux

Answer 1

For all these you could perform a simple shell scripting task. Here it is only sketched.

• For each file, calculate the md5sum or the sha1 hash, write this out:
• create a textfile with a 2-column table: pathname, hash.
• Perform a self-join on the hash column, perhaps with the /usr/bin/join command line tool. This will find files that are exactly identical but are stored in different directories.

If you join on the basename of the path column you might find files that are in different directories but have the same filename (but differ in their hashes)

I don't fully understand your third bullet point, but you might figure it out by modifiying the join(s) yourself.