Program that can extract subtitles from hard-subbed videos?

Question

I am looking for a program that can extract/create subtitles from a video that has been hard-subbed rather than a video that has been soft-subbed where it is rather quite easy to extract these subtitles.

Ideally a program that would suite my needs …

• is hopefully free (gratis)
• runs on Windows 7
• is lightweight
• is simple, easy to use
• can be used by anyone, not too hard to use/configure
• has a good user interface
• can extract subtitles from hard-subbed videos or at least create subtitles from a hard-subbed video which is pretty accurate

Hardsubbed: Actually infused into the actual video itself.

Softsubbed: Softsub are basically like captions in other words and you can easily turn them off. Also it is quite easy to extract the subtitles from a video that has been softsubbed.

Any suggestions/ideas on how I could do this? Is it possible?

Answer 1

What you are looking for is OCR software.

Hardsubbed text is merged into video stream, so OCR'ing is only way to extract it.

However most of OCR software you can find for free is able to process only images. You can overcome it by feeding it frames from video file. (Or you can use OpenCV and create your own soft, it is quite easy for such complex task)

It is possible to greatly shorten amount of time needed to process video by limiting size and amount of pictures (data).

• Subtitles stay (usually) in same spot on screen, so take "screenshot" only of part containing text (lower ~20%?).
• Subtitles are big, single color with sharp edges, with character border/shadow and sometimes solid background (unless video is of bad quality) - you can safely lower resolution of pictures by half or so.
• Subtitles stay on screen for some time, you can skip a lot of frames. Even one frame per second will be more than enough so you can skip a lot of data by using frameskip (over 90% less data)

After collecting text entries you only need to remove repeating ones, leaving some margin of error as OCR is never perfect ("ą" can be detected as "a", "O" as "0" etc).

There are few guides on how to do it, for example this one. (although it uses Linux/Unix software)

Answer 2

The classical software to do this is SubRip, after which the popular SRT subtitle format is named. It doesn't seem to be under active development; the latest version is from 2015. It can only extract hard subtitles from files that AviSynth can open. So you would have to convert the video to AVI first.

I tried this a while ago by converting a video to AVI(xvid/mp3), but SubRip refused to load the video. However I used it successfully years ago, when AVIs were still in use. I remember that the OCR needs to be trained letter by letter, which is quite tedious, unless you happen to find a definition file for the exact font of your subtitles. Afterwards you need to correct common OCR errors such as confusing "m" and "rn", or "I" (uppercase i) and "l" (ell). Subtitle editors often have that function.

It's strange that there doesn't seem to be a current replacement, given the great advancements in OCR in the last years.