Question
Is there an OCR solution that would run natively on a recent iPhone with better accuracy than Tesseract? It need only recognize enough to recover dollar amounts.
Objective
To scrape a table of dollar amounts from a screen image taken on a mobile device. We’ve examined all the frequently-recommended solutions, but they don’t satisfy our constraints.
I'm starting here in case there’s an off-the-shelf solution; if not, I’ll try Stack Overflow.
Many of the objectives and priorities come from our client. It’s fair to wonder whether these are wise, but I have to live with them.
That’s the TL;DR summary. Below are the details, which are not of interest to everybody.
Priorities
- The solution must run on a fairly-recent model of the Apple iPhone (let's say iPhone 6s minimum).
- The accuracy of the interpreted characters is paramount.
- There are ~30 numbers to retrieve. They are arranged in a table of three columns.
4.The format of the numbers may include currency symbols, grouping delimiters, and decimal separators. ('
$', ',', and '.' in the US). Or any reasonable combination of these. - Expect the bitmap for each numeral to be ~25w x ~35h (10-pt Helvetica at 2x or 3x resolution). The font would be proportional, so separators will be significantly narrower.
- The application needs some way of recovering the placement of each number in the table.
- Run time must be under 10 seconds, ideally under 5. Usage will probably be rare enough that power consumption won’t be a consideration.
Constraints
This is a proof-of-concept application. I am allowed sharply limited time and labor. I am strongly discouraged from resorting to services with ongoing or large initial costs, even though those may make sense for the final product.
What I’ve tried/considered
Tesseract
Tesseract is about 85% accurate in this application.
I’ve tried the native image preprocessor and my own I experimented with: Monochrome, various brightness and contrast, various blur/sharpen; the processed image looks plausibly distinct to me, with no empty spaces filled-in.
I can get the accuracy above 90% by correcting the output so it makes sense; I know the greatest magnitude, for instance, so too large a number with a leading 8 probably begins with a $.
The client isn't satisfied.
Tesseract in small pieces
The Vision framework in iOS 11 can identify the location of words and characters in an image, though it does not interpret them. Feeding the sub--images to Tesseract yields much worse results. I’m guessing Tesseract depends on context to resolve ambiguities.
Source scraping
There is no textual native format such as HTML or text PDF. I'm stuck with the pixmap.
Remote services
A remote service is less appealing than native processing, even though “free-trial” experiments yield much better results:
- The client does not want to be out-of-pocket for the proof-of-concept phase of the product.
- We can’t rely on the client’s managing his demos to fall within free-trial date or quantity limits.
- As a matter of principle, we’d like the app not to depend on network availability or turnaround.
- As a matter of principle, we’d like to avoid any user data going over the net and probably retained for third-party inspection.
ABBYY looks like a good option, except for these restrictions.
Machine learning
The problem is fairly-neatly contained:
- Predictable type style.
- No geometric distortion.
- Sharp images to work from.
- 15 symbols to classify (0-9.,${space}{other})
A machine-learning model promises to be highly accurate, very fast, and easy to train.
The calendar and budget we've been allowed doesn’t cover my learning how to train a model. However, we have access to a pool of computer-science students, one of whom might be able to do the training quickly and relatively cheap.
At this point, we’re beyond the scope of Software Recommendations and into Stack Overflow or local talent. I’m including this option for completeness.
