Automatically find shot-transitions in a movie

Question

I would like to have a piece of software which automatically finds the shot-transitions in a video. A search on Google shows it's an active research-area of interest, however I can't find any software which does this.

My ideal application would:

• be a console application which takes as input the path of a video-file.

• gives as output a list of timestamps where in the video the shot-transitions occur.

• work on Windows.

• work with all common video-formats.

• is gratis or costs no more than around €50.

However I am open to other types of programs running on different operating systems which achieve the same goal, automatically find shot-transitions in a video-file.

UPDATE

I recently found out that what I call a shot-transition is in general more referred to as a cut.

For example in the following movie

https://www.youtube.com/watch?v=5fr8ML7r3Xg

the first two cuts are on 9 seconds and 26 seconds in the movie.

I hope it's for everybody now clear what I mean.

Answer 1

As far as a computer is concerned the distinguishing feature of a shot transition is that the content of the screen differs by more than some given threshold. For MPEG files this will always occur at a key frame.

I would suggest trying to implement this in python with MoviePy which uses FFMPEG & numpy.

For your purposes I would suggest that a 1 second resolution is probably good enough so you could take the frame at your start point (it may be worth having an input parameter to skip the first n seconds to get past titles). This will give you a numpy array for the image at that timestamp. I would suggest applying some blurring and reducing the resolution. Then take the frame at 1 second on, apply the same processing, and compare to see if the differences are greater than your threshold. If they are then report that time index as a possible cut. Then make your +1 second image your reference and move on to the next. It will probably take a little playing with the image comparison methods and comparison thresholds but you should be able to get good results.

Once you have your python script you have your command line tool.

• Free, gratis & open source.
• Just about any platform
• Will work with any format supported by FFMPEG for read.

Proof of Concept

As a really rough proof of concept I tried, using iPython, and simply taking the mean of each frame as an indication of changes:

from moviepy.editor import *
import itertools

mov = VideoFileClip("SIR David Attenborough () Secrets of Wild India Ep1 () Elephant Kingdom - HD-5fr8ML7r3Xg.mkv")
means = [mov.get_frame(n).mean() for n in range(int(mov.duration))]
deltas = [y-x for x,y in itertools.zip_longest(means, means[1:])]
cuts = [i for i,x in enumerate(deltas) if abs(x) > 20] # 20 arrived on by looking at the deltas

This successful gave me the timestamps for cuts at 320 times, these started with: [0, 9, 26, 41, 46, 53, 56, 59, 66, 67, 69, 74, 75, 84, 93,...

I have to say that a simple mean of all of the pixels for all 3 channels is a very crude estimation of a scene change but even such a crude check seems to be working to some degree.