Looking for a library that extracts text from images

Question

The basic description for what I'm looking for is something that can extract text (written by hand) from an image and return it as a plain character stream, without any page processing.

More specifications:

• I am using Java for the software I'm developing, so having a Java library would be nice, although if the software/library has a binary it's fine

• The images it should be able to process are pictures of paper that has text on it written by hand, though what the software has to recognize is between 6 and 66 characters that will be written by hand, but more "computer recognizable", kinda like this:

• The text must be extracted line by line; in the case of a table, it should just regard each row and column as another line, no special processing required

My own research pointed me to the following software/libraries:

• Asprise Java OCR

• Tesseract

• ABBYY Cloud OCR SDK

I've tested both Tesseract and ABBYY, and the result they provide is VERY FAR from what I expect: Tesseract gave me an output of 200+ characters for the image I attached to this post and ABBYY did not provide any characters, just some lines.

Answer 1

OCR generally works by trying to recognize black text characters on white background, so most OCR approaches try to “binarize” the image and convert it to black and white.

Considering the image you are using, there are a number of issues here that make this difficult:

1. The grey background has shades close to the pencil writing and the line.
2. The penciled characters are not made of solid shapes have many shades in them.
3. The dotted line and the characters have close shades of color.

It is a combination of these problems that leads to either no text being recognized, as the text does not stand out enough, or large amount of false text being detected as the OCR tries to interpret the dotted line and the background.

This means that, for this type of image, some image processing will be needed to make the text more detectable for OCR.

For this particular image, for example, I was able to use image processing classes from the LEADTOOLS toolkit to achieve the result below. (Disclaimer: I am an employee of the vendor of this toolkit)

The image processing used was as follows:

1. Histogram Equalization to bring out the details in the dark characters of the text.
2. Intensity detection and binary segmentation along a wide range of pixel values to filter out the noise in the image.
3. Noise reduction through along a median of pixel color values to remove the dotted line but keeping the text as clear as possible.

The code for this looks like the following:

HistogramEqualizeCommand histCommand = new HistogramEqualizeCommand(); 
histCommand.Run(image);

IntensityDetectCommand intensityDetectCommand = new IntensityDetectCommand(); 
intensityDetectCommand.LowThreshold = 5; 
intensityDetectCommand.HighThreshold = 255; 
intensityDetectCommand.Run(image); 

MedianCommand medCommand = new MedianCommand(); 
medCommand.Dimension = 9;
medCommand.Run(image); 

Then by using OCR/ICR classes from the toolkit I was able to recognize the text:

ICR was required to recognize the text. Please note that this is not available on the Java version of the toolkit.

Alternatively, if you do not want to perform image processing, consider changing the base image:

1. Use black text on a white background.
2. Use ink to write solid text character instead of pencil.
3. Hand-print the characters to make them stand out more.
4. Use solid faint lines instead of dotted lines.