21

Can you please recommend me a lossless file compressor for general use, which is the more space-efficient than standard ZIP compression? Of course, the better it compresses, the better…

It should satisfy the following additional criteria:

  • Compresses files and directories (can contain an internal directory structure).
  • Should be designed to be efficient with input file sizes from around 1 to 200 MB.
  • Password protection is nice, but optional.
  • Compresses text as well as media files (MP3, EXE, movies, photos) without making them bigger in most cases.
  • Free to use.
  • The amount of time that it takes to compress the files is not a very important criterion, as long as it completes in a "reasonable" time (and by that I mean, a few hours on a typical desktop computer might be OK for several hundred megs of input data, but a few hundred years is probably not acceptable).
  • Cross platform (Windows, Linux and OS X) is preferred, but not required.

Since I am looking for something that is more space-efficient, it would be cool if you would compare your recommendation to standard ZIP compression.

Note: I am not asking for "the best" tool. What I am looking for are "tools more efficient than ZIP" which match the mentioned criteria. In the end, I will decide for myself which recommendation(s) I like "best".

|improve this question
16
+50

The PAQ (official website here) compressors are very efficient (as in compressed file size). It does so at the expense of speed and memory usage (note: this is only true for some of the experimental PAQ versions).

PAQ has won the Hutter Prize and the Calgary Challenge.

NOTE: The ZPAQ Open Standard Journaling Archive is a culmination of all of the experimental PAQ Libraries.

ZPaQ uses deduplication, multi-threading, and a range of compression algorithms (LZ77, BWT, and context mixing) for high performance.

UPDATE: There is a GUI wrapper built (using .Net) for ZPAQ called winzpaq. You can read release notes here and download the latest version directly from this site. As of this edit, it works on Windows, but from the release notes it sounds like it may work on Linux as well (because of Mono).

UPDATE: I will address the criteria list:

  • Compresses files and directories (can contain an internal directory structure).
  • Yes, most versions of PAQ will do this (paq8hp12any is a specialized version of PAQ8hp that specifically states that it does not compress or create directories).
  • Should be designed to be efficient with input file sizes from around 1 to 200 MB.
  • This benchmark shows that the different PAQ utilities were able to compress the 100 MB enwik8 text file (used in the Large Text Compression Benchmark) down to less than 20 MB (whereas the same benchmark show ZIP at level 9 compression at approx 36 MB).
  • Password protection is nice, but optional.
  • ZPAQ (which is really the culmination of all of the experimental PAQ libraries, put into a general purpose compression utility) supports password encrypting (there is mention of it in the History section as of version 6.44).
  • Compresses text as well as media files (mp3, exe, movies, photos) without making them bigger in most cases.
  • While I couldn't find mention of mp3/exe/movies directly, the Silesia Open Source Compression Benchmark tested against various file types (Windows DLL, binary, DICOM images, PDF, ASCII text) and the various PAQ libraries outperformed the various zip utilities. At a glance (there is a lot of data), the files were all smaller than the original (original file sizes found at the bottom of the list). There is also mention of being able to compress JPG files in this benchmark.
  • Free to use.
  • According to the official ZPAQ website, zpaq is a free and open source (GPL v3).
  • The amount of time that it takes to compress the files is not a very important criterion, as long as it completes in a "reasonable" time (and by that I mean, a few hours on a typical desktop computer might be OK for several hundred megs of input data, but a few hundred years is probably not acceptable).
  • The speed will depend on the version of PAQ you use. As I've been exploring these to improve this answer, I have noticed that the ZPAQ library has made great speed enhancements and still has nearly the compression ratio of some of the experimental libraries that are much slower. Based on the ZPAQ library, there is a good benchmark here:

The following comparison is from the 10 GB benchmark (10 GB in 83,437 files and directories) using default settings on a Core i7 M620, 2.66 GHz, 4 GB backup to an external USB hard drive in Ubuntu Linux.

  Archiver    Size     Compress  Extract  License
  --------    -------  --------  -------  ------- 
  zip         4844 MB  1206 sec  229 sec  Free, open source with specification
  rar         4493 MB  1423 sec  239 sec  Commercial, unrar is free and open source
  7zip        3892 MB  6593 sec  472 sec  Free, open source
  freearc     3711 MB  1448 sec  422 sec  Free, open source
  zpaq        3825 MB   379 sec  249 sec  Free, open source with specification
|improve this answer
  • I'd say that's now a great answer - in fact possibly better than my answer. I'm also interested in that comparison; might have to try ZPAQ out myself and consider using it instead if it performs well and that benchmark is correct and it will usually take 1/20th the time of 7z for slightly better compression. – Nick Wilde Feb 12 '14 at 20:20
  • 3
    @Nick-BriarMoonDesign: I just tried out ZPAQ for the first time (I had mostly used PAQ8f back about 5 years ago). I did a comparison of 7zip (default options from right-click menu into a .7z file) and zpaq (default options from command line). I tried compressing my programming books directory (221 files, 8 folders, 690 MB... mostly PDF and some epubs). 7zip - 1m 23s - 419 MB. ZPAQ - 14s - 423 MB. So for about the same compression rate, ZPAQ was much faster. – Jason Down Feb 12 '14 at 21:50
  • +1. Actually, I am intrigued by this compressor and will certainly try it myself, it even has an API in C++. Thanks for recommending it. – Ivaylo Slavov Feb 18 '14 at 21:51
  • 1
    @IvayloSlavov: You're welcome. The author of the tool has certainly put a lot of research into compression. It has come a long way since I originally tried it out back around 2006/7. – Jason Down Feb 19 '14 at 2:43
26

I personally recommend 7-zip using the 7z archive format and LZMA2 compression algorithm.

  • Maintains a relative directory structure.
  • It is designed to be fully functional with filesizes up to "16,000,000,000 GB" (you can read that as 16 exabytes for simplicity)
  • fairly high quality encryption - AES-256 - so security is mainly dependent on password quality.
  • Even with very er non-compressable file types bloat is VERY minimal.
  • Free and mostly open source
  • Quite fast - like ~350MB of music (what I just did right now to give a good number) with LZMA2 at 2nd highest compression takes ~4minutes on a reasonable laptop (2gb ram, Win7, Intel i7 processor).

GUI only supports Windows but there are linux builds available for the command line (though I haven't personally used them - unlike the cmd line for windows version which I've used quite significantly(1))

(1): read: tested hundreds of commands for extracting and compressing from python program (Wrye Bash)

|improve this answer
  • POSIX cli tools are adequate for most purposes. I would definitly recommend this as most tools can now extract 7z files so most people will be able to extract them. – dtech Feb 12 '14 at 12:46
  • Keep in mind that 7z doesn't keep file permissions on linux, unless they fixed that recently. – OneOfOne Aug 30 '14 at 4:56
5

I'd like to coin gzip and bzip2. These two formats are very common on POSIX (e.g. Linux) systems and available on nearly all of them, therefore they also seeped into the surrounding ecosystem (e.g. MySQL).

They both only compress files and are thus usually combined with tar (which concatenates files) yielding .tar.gz and .tar.bz files

advantages:

  • (Both) Very common in POSIX ecosystem
  • gzip is very fast both in compressing and decompressing, on par with zip
  • (Both) open-source, patent-free and free

disadvantages:

  • (Both) Windows tool support is suboptimal at best (e.g. 7zip extracts .tar.gz first into a .tar file which needs to be extracted again)
  • gzip doesn't have a very high compression ratio compared to most other tools, but it is better than zip
  • bzip2 is quite slow, on par with 7zip
  • bzip2 has an okay compression ratio, on par with rar, but is worse than 7zip and other modern methods
|improve this answer
  • 1
    +1 for your conclusive advantage / disadvantage list! I'd like to see that in more answers. – Angelo Fuchs Feb 12 '14 at 20:49
4

PeaZip is a free, open source file and archive manager for 32 and 64 bit Windows and Linux. It is available either as installable and portable application (not needing installation).

Compress, convert, split, encrypt, secure delete,Open RAR TAR 7Z ZIP ZIPX files, 150+ formats supported.

Supported systems: Windows 9x, Windows NT/2K/XP,Windows Vista/7/8 (32 and 64 bit), ReactOS, Wine.

More info is available here on SourceForge.

|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.