You could check out the LEADTOOLS Multimedia SDK to implement this type of task. It supports C, C++, .NET and will work with other languages that support COM objects. This SDK supports decompressing video files and provides a video callback filter to get the individual frames to do custom image processing on. This would allow you to use the same algorithm for processing without needing special code to handle different file formats. If the files you're processing use MPEG-2 or H.264 compression, you would also be able to utilize (natively) hardware decompression to offload some of the work to the GPU.
Here is a forum post on the LEADTOOLS Support Forums that has a sample project illustrating how to use the Video Callback:
HOW TO: OCR Video using the callback filter
A key part of this forum post is in adding the video callback. Here is the relevant code on how to do that:
Processor videoCallback = playctrl.VideoProcessors.Callback;
lmvCallback = (LMVCallback)playctrl.GetSubObject(PlayObject.SelVideoProcessor);
lmvMyUserCallback = new LMVMyUserCallback();
lmvCallback.ReceiveProcObj = lmvMyUserCallback;
The other key part of this demo is the implementation of the ReceivePro() method, as this is the invoked callback function. The code to pay attention to is this:
public void ReceiveProc(int pData, int lWidth, int lHeight, int lBitCount, int lSize, int bTopDown)
// Get the background image
if (bTopDown == 1)
viewPerspective = RasterViewPerspective.TopLeft;
viewPerspective = RasterViewPerspective.BottomLeft;
RasterImage img = new RasterImage(
RasterMemoryFlags.User, //A combination of the RasterMemoryFlags enumeration members indicating the type of memory to allocate for the image data.
lWidth, // Width of the image in pixels.
lHeight, //Height of the image in pixels.
lBitCount, //The number of bits per pixel.
RasterByteOrder.Bgr, //Color order for 16-, 24-, 32-, 48- and 64-bit images.
viewPerspective, //Specifies where the beginning of the image is stored.
null, //The palette that the image will use. You can specify your own palette, or use null (Nothing in Visual Basic) for LEAD's fixed palette. The palette member is used only when bitsPerPixel is less than or equal to 8.
IntPtr.Zero, //Unmanaged data pointer that will contain the image data when flags is RasterMemoryFlags.User.
0); //Length in bytes of the data passed to userData. Only when used when userData is not IntPtr.Zero and flags is RasterMemoryFlags.User.
img.SetUserData(new IntPtr(pData), lSize);
As this project also illustrates, you can use features from the LEADTOOLS Recognition Imaging SDK if the common frames you are looking for contain text. This SDK also contains the image processing class CorrelationCommand, which is designed to find specific image data within another image. Here is some sample code illustrating how you would set up and run this method:
RasterCodecs codecs = new RasterCodecs();
// Prepare the command
RasterImage DstImage = codecs.Load("source.png");
CorrelationCommand command = new CorrelationCommand();
command.CorrelationImage = DstImage;
command.Threshold = 70;
command.XStep = 1;
command.YStep = 1;
command.Points = new LeadPoint;
//Apply the correlation filter.
With access to each frame in the video stream, you would be able to do any type of image processing you wanted to do on the frame.
Disclaimer: I am an employee of the company that wrote this library.