Command line tool to check whether two images are exactly the same graphically

Question

I need a command line tool that compares 2 images and says if their contents are the same, regardless of encoding - i.e. one might be a *.bmp and the other might be a *.png, so long as all their width, height and all the corresponding pixels are the same.

• Exact graphical sameness is needed
• Compression loss, even if nearly invisible, makes a different image
• Same alpha-transparency is also important
• EXIF/etc irrelevant

Answer 1

With ImageMagick (apt-get install imagemagick), you can compare images independent of encoding and metadata like this:

identify -quiet -format "%#" images...

Note that images that have been encoded with lossy compression like JPEG (*.jpg) have subtle, often invisible changes.

See also ImageMagick Examples: Image Signatures.

Strictly speaking, you need to compare the color model, and scaling of pixel values, too. They may not be part of the metadata in the image.

Answer 2

ImageMagick compare -metric AE exit status is non-zero if at least one pixel color differs.

Let’s say you have a folder named before which contains original images, and a folder after which should contain visually identical images with the same file name. Using ImageMagick’s compare, you can do this:

for file in before/*.png; do
    result=$(compare -metric AE "${file}" "${file/before/after}" /tmp/diff.png 2>&1);
    if [ "${result}" != '0' ]; then
        echo "${result} incorrect pixels in ${file}";
    fi;
done;

Answer 3

If you have MATLAB, you can use:

% Reading images as array to variable 'a' & 'b'. 
a = imread('MIMICDatacollection.bmp'); 
b = imread('MIMICDatacollection.png'); 

% Flatten multidimensional arrays to 1D
c=a(:);
d=b(:);

% Perform comparison
if length(c) ~= length(d)
    disp('The images do not have the same size') 
else
    e = corr2(c,d);           
    if e==1 
        disp('The images are same')
    else 
        disp('The images are not same') 
    end; 
end

Personally, I use it with PNG and BMP, but it should work for any format supported by imread.

If you need to run it on a machine that doesn't have Matlab, you can turn it into a function and compile it to make it CLI.

If you don't have Matlab that should be easy to port in any high-level language with a decent imaging library, such as Python Imaging Library (PIL).

Related: How can I tell if I am downloading/saving duplicate images?

Answer 4

findimagedupes - Finds visually similar or duplicate images

findimagedupes is a commandline utility which performs a rough "visual diff" to two images. This allows you to compare two images or a whole tree of images and determine if any are similar or identical. On common image types, findimagedupes seems to be around 98% accurate.

Answer 5

I eventually created a small Qt program that I called imgdiff, which takes 2 filenames and performs a pixel-by-pixel comparison. It will print out an error message if they differ and exit with 1, or silently exit with 0 if they are the same.

Example usage would be:

imgdiff img1.png img2.bmp

Link to the Google Code project - https://code.google.com/p/imgdiff/.

Answer 6

Try dupeguru from here: https://dupeguru.voltaicideas.net/. In Picture mode it checks for similar images, even though they have different sizes, it has a threshold setting, creates groups of similar files, choosing the biggest file as main, but can be set another file for reference too. Is able to move files to some path or to the recycle bin or delete them completely. It is cross-platform.

Another interesting tool, that can be used for various merge operations too, is WinMerge: https://winmerge.org/. This has an interesting feature that highlights the difference areas in the images, but is able to compare up to three files at once, or three paths that contain files, but it compares the files having the same name only, if found on those paths. This is good for folder syncing. This is Windows only, runs fine in Wine, but there is a QT version available that seems to be in an early stage: https://github.com/OzzieIsaacs/winmerge-qt.

Answer 7

`I have a python script that works for me... in Ubuntu Linux. Compares all pngs in a directory (same size) 4 different ways... its a little crazy but it works for me identifying looping in my 4k uhd art videos I make. I explode the mp4s into pngs and then run this to look for matches.

# Function to compare images using VGG16
# Function to compare images using SSIM
# Function to compare images bit-by-bit
# Function to compare images using Histogram
# Check for matches within the + or - 5% tolerance

#!/usr/bin/env python3

import os
import logging
import numpy as np
import cv2
from tensorflow.keras.applications.vgg16 import VGG16, preprocess_input
from tensorflow.keras.preprocessing import image
from tensorflow.keras.models import Model
from sklearn.metrics.pairwise import cosine_similarity
from skimage.metrics import structural_similarity as ssim

# Configure logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')

# Function to extract features using VGG16
def extract_features(img_path):
    logging.debug(f"Extracting features from image: {img_path}")
    img = image.load_img(img_path, target_size=(224, 224))
    img_data = image.img_to_array(img)
    img_data = np.expand_dims(img_data, axis=0)
    img_data = preprocess_input(img_data)

    features = model.predict(img_data)
    logging.debug(f"Features extracted from {img_path}")
    return features.flatten()

# Function to compare images using VGG16
def compare_vgg16(img1, img2):
    logging.debug(f"Comparing images using VGG16: {img1} vs {img2}")
    features1 = extract_features(img1)
    features2 = extract_features(img2)

    similarity = cosine_similarity([features1], [features2])
    logging.debug(f"VGG16 similarity score: {similarity[0][0]}")
    return similarity[0][0]

# Function to compare images using SSIM
def compare_ssim(img1, img2):
    logging.debug(f"Comparing images using SSIM: {img1} vs {img2}")
    image1 = cv2.imread(img1, cv2.IMREAD_GRAYSCALE)
    image2 = cv2.imread(img2, cv2.IMREAD_GRAYSCALE)

    score, _ = ssim(image1, image2, full=True)
    logging.debug(f"SSIM score: {score}")
    return score

# Function to compare images using Histogram
def compare_histogram(img1, img2):
    logging.debug(f"Comparing images using Histogram: {img1} vs {img2}")
    image1 = cv2.imread(img1)
    image2 = cv2.imread(img2)

    hist1 = cv2.calcHist([image1], [0, 1, 2], None, [8, 8, 8], [0, 256, 0, 256, 0, 256])
    hist2 = cv2.calcHist([image2], [0, 1, 2], None, [8, 8, 8], [0, 256, 0, 256, 0, 256])

    hist1 = cv2.normalize(hist1, hist1).flatten()
    hist2 = cv2.normalize(hist2, hist2).flatten()

    similarity = cv2.compareHist(hist1, hist2, cv2.HISTCMP_CORREL)
    logging.debug(f"Histogram similarity score: {similarity}")
    return similarity

# Function to compare images bit-by-bit
def bit_by_bit_comparison(img1, img2, tolerance=0.05):
    logging.debug(f"Performing bit-by-bit comparison: {img1} vs {img2}")
    image1 = cv2.imread(img1, cv2.IMREAD_GRAYSCALE)
    image2 = cv2.imread(img2, cv2.IMREAD_GRAYSCALE)

    if image1.shape != image2.shape:
        logging.error("Images have different dimensions and cannot be compared bit-by-bit.")
        return None

    difference = np.abs(image1.astype(np.int16) - image2.astype(np.int16))
    threshold = 255 * tolerance
    num_diff_pixels = np.sum(difference > threshold)
    total_pixels = image1.size

    percentage_difference = (num_diff_pixels / total_pixels) * 100
    logging.debug(f"Bit-by-bit difference percentage: {percentage_difference:.2f}%")

    return percentage_difference

# Function to get the input directory from the user
def get_input_directory():
    logging.debug("Requesting input directory path from the user.")
    input_dir = input("Please enter the input directory path: ")
    while not os.path.isdir(input_dir):
        logging.error("The provided directory path is invalid. Please try again.")
        input_dir = input("Please enter the input directory path: ")
    logging.info(f"Input directory selected: {input_dir}")
    return input_dir

# Main Script Execution
if __name__ == "__main__":
    # Ask the user to input the directory containing the frames
    frames_dir = get_input_directory()

    # Get the list of frame files
    logging.debug("Retrieving list of frame files...")
    frames = sorted([os.path.join(frames_dir, f) for f in os.listdir(frames_dir) if f.endswith('.png')])

    # Variables to track matches
    ssim_matches = 0
    histogram_matches = 0
    vgg16_matches = 0
    bit_by_bit_matches = 0

    # Tolerance value
    tolerance = 0.05

    # Load the VGG16 model pre-trained on ImageNet
    logging.info("Loading VGG16 model pre-trained on ImageNet...")
    base_model = VGG16(weights='imagenet')
    model = Model(inputs=base_model.input, outputs=base_model.get_layer('fc1').output)
    logging.info("VGG16 model loaded successfully.")

    # Compare every frame with every other frame in the dataset
    for i in range(len(frames)):
        for j in range(i + 1, len(frames)):
            logging.debug(f"Comparing {frames[i]} with {frames[j]}...")
            ssim_score = compare_ssim(frames[i], frames[j])
            hist_similarity = compare_histogram(frames[i], frames[j])
            vgg16_similarity = compare_vgg16(frames[i], frames[j])
            bit_diff = bit_by_bit_comparison(frames[i], frames[j], tolerance=tolerance)

            logging.info(f"Comparing {frames[i]} with {frames[j]}:")
            logging.info(f"  SSIM: {ssim_score:.4f}")
            logging.info(f"  Histogram Similarity: {hist_similarity:.4f}")
            logging.info(f"  VGG16 Cosine Similarity: {vgg16_similarity:.4f}")
            logging.info(f"  Bit-by-Bit Difference: {bit_diff:.2f}%")

            # Check for matches within the + or - 5% tolerance
            if ssim_score >= 0.95:
                logging.info("  * SSIM match within 5% tolerance")
                ssim_matches += 1
            if hist_similarity >= 0.95:
                logging.info("  * Histogram similarity match within 5% tolerance")
                histogram_matches += 1
            if vgg16_similarity >= 0.95:
                logging.info("  * VGG16 cosine similarity match within 5% tolerance")
                vgg16_matches += 1
            if bit_diff is not None and bit_diff <= 5:
                logging.info("  * Bit-by-Bit comparison match within 5% tolerance")
                bit_by_bit_matches += 1

    # Summary of matches
    logging.info("\nSummary of Matches:")
    logging.info(f"  SSIM Matches within 5% tolerance: {ssim_matches}")
    logging.info(f"  Histogram Matches within 5% tolerance: {histogram_matches}")
    logging.info(f"  VGG16 Matches within 5% tolerance: {vgg16_matches}")
    logging.info(f"  Bit-by-Bit Matches within 5% tolerance: {bit_by_bit_matches}")

Answer 8

Might need tweaking to handle alpha channel appropriately but converting to ppm and checksumming seems to work:

#!/bin/bash

find "${@:-.}" -type f -print |\
while IFS= read -r file; do
    hash=$(convert 2>&- "${file}" -strip ppm:- | md5sum) 
    # we get this hash if convert fails and produces no output
    [ "$hash" = 'd41d8cd98f00b204e9800998ecf8427e  -' ] \
    || echo "$hash ${file}"
done |\
sort | uniq -w32 --all-repeated=separate | sed 's/^.\{36\}//'