I'm dealing with a large archive of satellite images of the Earth, each one taken 15 minutes apart over the same area, therefore they are quite similar to each other. Two contiguous ones look like this:
Video algorithms do very well compressing multiple similar images. However, this images are too large for video (10848x10848) and using video encoders would delete the metadata of the images, so extracting them and restoring the metadata would be cumbersome even if I get a video encoder to work with such large images.
To make some tests I've reduced the 96 images of one day to 1080x1080 pixels, totaling 40.1MB and try different compression with the folowing results:
- zip: 39.8 MB
- rar: 39.8 MB
- 7z : 39.6 MB
- tar.bz2: 39.7 MB
- zpaq v7.14: 38.3 MB
- fp8 v2: 32.5 MB
- paq8pxd v45: 30.9 MB
The last three, are supposed to take much better advantage of the context and indeed work better than traditional compression, but the compression ratio is still pretty poor compared with mp4 video that can take it to 15 MB or even less preserving the image quality.
In addition, using packJPG, that packs each image separately. The whole set get down to 32.9 MB, quite close to fp8 and paq8pxd but without taking at all advantage of the similarities between images (because each image is compressed individually).
In another experiment, I calculated in Matlab the difference of the two images above, and it looks like this:
Compressing both original images (219.5 + 217.0 = 436.5 kB total) with fp8 get them down to 350.0 kB (80%), but compressing one of them and the difference image (as a jpg of the same quality and using 122.5 kB), result in a file of 270.8 kB (62%), so again (as revealed by the mp4 and packJPG comparison), fp8 doesn't seem to take much advantage of the similarities. Even compressed with rar, one image plus the difference do better than fp8 on the original images. In that case, rar get it down to 333.6 kB (76%).
I guess there must be a good compression solution for this problem, as I can envision many applications. Beside my particular case, I guess many professional photographers have many similar shots due to sequential shooting, or time-lapse images, etc. All cases that would benefit from such compression.
Also, I don't require loseless compression, at least not for the image data (metadata must be preserved).
So... Is there a compression utility that take advantage of the similarities between images better than zpaq and paq8pxd?
I envision that the solution could be some kind of wrapper for a video encoder, that would allows to easily compress the images as video but extract them as individual frames, keeping filenames and metadata (like EXIF info) for each.
NOTE: The two images of the above test can be downloaded here, and the 96 images of the first test here.
