30

I need recommendations for a software for de-duplication.

Requirements:

  • Runs on Windows

  • Ability to find duplicate archives: files that have exactly the same content (and, so, the same size). Names, paths or dates are irrelevant.

  • Options to perform automatic actions (like to keep the newest, the oldest, or files in some path) and also the manual action "do nothing, prompt for action". This last case (manual action) could be performed on more than one file

  • Option to create symlinks: instead of simply removing all-but-one ocurrence, create symlink to one version.

  • Graphical interface (and not only command prompt)

12

Duplicate Files Finder : http://doubles.sourceforge.net/

For Windows and Linux. Lets user find dup files (by their content, not only their names). The search can include and exclude files (ex.: exclude all *.tmp) and restrict which file sizes are searched.

Can remove dup files by deleting them or by creating links.

Instead of using hashes, it uses the following algorithm:

The algorithm is as follows: First, all files are sorted by their size, because files can be only equal, if they have the same size (logically). Then the files are compared with each other, and thus the equal files are determined. If two files are not equal from a given point on, reading is interrupted; no more has to be read for determining that these files are not equal. Because of this the results are determined much faster than in programs which use hashing algorithms, for which all files have to be read completely. Additional caching of the contents of the files additionally improves performance.

It's easy to use and fast. Can store the results (list of file names) in a .txt file, so that you can process it further with some other technique.

License: GPL

  • The program still works on Windows 7 x64, although not mentioned. However, you can't tell it to delete all files in folder "backup2011" if you identified another directory "backup2012". Needs a click for each file, which is not doable with 45000 files. Or you have to parse the file, which is a proprietary format. Other than that it works fine, even with 2.000.000 files. – Thomas Weller Jan 23 '16 at 9:45
4

Well, for years now, I have used Duplicate Cleaner Free. It has an intuitive, but powerful, GUI, and lots of features, including "Turn duplicate file into hard links".

 -    Find duplicate files, fast!  
 -    Cleaner hard drives means faster performance.  
 -    Fast Scanning and accurate duplicate comparison.  
 -    Locate duplicate music (includes MP3, iTunes (M4A) and more.)  
 -    Find Similar images (resized, rotated, edited and more.)  
 -    Find duplicate folders.  
 -    Flexible search settings - search the way you want to.  
 -    Intelligent Selection Assistant.  
 -    Search home or networked drives.  
 -    Search multiple locations.  
 -    Export / Import your results.  
 -    View all file information in a detailed list.  
 -    Large, resizable windows for easy browsing.  
 -    Handy image preview window with side-by-side comparison.  
 -    Delete duplicate files to the Recycle bin.  
 -    Move dupe files to a new location.  
 -    ***Turn duplicate file into hard links***.  
 -    Protection for Windows and system files.  
 -    Supports multiple languages.  
 -    Comprehensive user manual.  
 -  For Microsoft Windows XP, Windows Vista, Windows 7 and 8.  

enter image description here

0

Total Commander: Alt+F7, advanced, search for duplicates: [x] size, [x] content. It uses similar algorithm like Duplicate Files Finder.

Licence: shareware, but you can use it forever, it only asks you to press a button on start. Full text:

Total Commander for Windows is a Shareware (demo version) program. This means that you can test it for a period of 30 days. After testing the program, you must either order the full version, or delete the program from your harddisk.

Far manager can probably make it too.

protected by Community Mar 2 '15 at 3:00

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.