13

Starting with Acrobat 9, Adobe engineers added a new flavor of OCR called ClearScan.

The advantages of the ClearScan over Searchable Image OCR are both in size and clarity. The biggest innovation of Clearscan is that:

ClearScan does not replace the font with your system fonts. Rather, a custom font it is created to match the visual appearance of the pixels.1

Are there other products than Adobe's that support Clearscan or similar products that use a custom font that is created to match the visual appearance of the pixels?

3
  1. Smoothscan is a tool to convert scanned text into a vectorized output form. Because printed text is assembled from fonts, each particular letter (like ‘o’) will have the same shape as every other ‘o’ in the document. We can take advantage of this, by building a table of such symbols, and represent each occurrence of a symbol with a reference to that symbol’s table entry. This will save a lot of space, and a similar idea is used in djvu’s jb2 mode and JBIG2 for PDF.

    smoothscan builds up this table, but instead of filling the table with the original raster images, it vectorizes each symbol. Vector images will look smoother than their raster equivalents, and can be scaled without introducing pixelation. These properties result in a smaller output file size, as well as making the scanned text images more readable.

    smoothscan saves the vectorized images into a custom TrueType font and embeds the font into the output pdf file. Currently each symbol is mapped to an arbitrary letter in the font, but in future versions you could run OCR on each symbol, and ensure that the ‘o’ image is associate with the ‘o’ character encoding in the generated font.

    A search of pkgs.org returns no smoothscan package for any Linux distro. So it needs to be compiled from the source code, its README notes these dependencies:


    On a Ubuntu box, most of these can be had like so:

    apt install libleptonica-dev libhpdf-dev potrace python-fontforge
    

    A test with a sample picture. Load the README in abiword, print it to a .ps file, and then...

    • Use ghostscript to convert that .ps file to .tif,
    • convert that to monochrome .tif, (the only input smoothscan accepts)
    • run smoothscan
    • compare the various file sizes with wc -c:

      gs -sDEVICE=tiffg4 -o README.tif README.ps
      convert -monochrome README.tif README.mono.tif
      smoothscan -o README.pdf README.mono.tif
      wc -c README README.ps README.*tif README.pdf | head -n -1
      

    Output:

      2432 README
     83516 README.ps
     33707 README.tif
     33618 README.mono.tif
     20394 README.pdf
    

    First paragraph of resulting README.pdf, showing vectorized fonts:

    image of first smoothscan paragraph, showing vectorized fonts.

  2. The djvu file format includes a JB2 option, which is much the same thing as ClearScan.

    Using the above README.ps input, the any2djvu util returns excellent results with:

    any2djvu README.ps
    

    ...the resulting README.djvu is only 7K, (1/3 the size of smoothscan's output), and looks sharper:

    any2djvu image of first two lines of README.djvu

    That sharpness is probably helped by not using a monochrome .tiff as input. That test is an unfair comparison. Let's try it with the mono rasterized .tiff:

    # "-f 6" sets input format to "Scanned Document - B&W - >400 dpi"
    any2djvu -f 6 README.mono.tif
    

    Now the output is less than 6k, but looks a bit jaggier:

    enter image description here

    Unlike smoothscan's .pdf output, the .djvu output allows text selection, even with rasterized input.

    One bad thing however, any2djvu needs an online server to do the conversion work -- it's not a suitable tool for confidential documents.

| improve this answer | |
  • Thanks for the nice post, I thought that there were no alternatives but it seems there are rather good ones +1/accepted – Eduard Florinescu May 14 '16 at 11:44
  • Very nice answer, but... Did you say you converted the PS file to DJVU? Wasn't the Postscript file already vectorized and with selectable text? – Andrea Lazzarotto May 15 '16 at 0:08
  • Whether the .ps file was vectorized or rasterized, I dunno, it's whatever abiword prints out to (since the .ps file is 85K, which is even bigger than the .tif, I'd guess it's rasterized). I'm unhappy with the present PS>DJVU any2djvu test though, and am still looking for ways to test djvu against rasterized text. – agc May 15 '16 at 17:59
  • Is smoothscan still supported? The latest commit on the git is dated in 2013, 5 years ago – raffaem Oct 12 '18 at 12:59
  • In hindsight, using sample text that included the name of the first program makes the djvu entry more confusing... – agc Oct 12 '18 at 13:17
2

Having ImageMagick, potrace and pdftools (disclaimer: I am the author of pdftools). Assuming you have an input PDF file called input.pdf:

  1. Convert every page in a BMP file using ImageMagick:
mkdir bmp
magick convert -density 300 input.pdf -quality 100 ./bmp/input.bmp
  1. (optional) pre-process image files:
mkdir bmpproc
for /r %%i in (./bmp/*.bmp) do mkbitmap ./bmp/%%~ni.bmp --output ./bmpproc/%%~ni.pbm
  1. Rasterize every BMP images and save the output in a PDF:
mkdir pdf
for /r %%i in (./bmpproc/*.pbm) do potrace ./bmpproc/%%~ni.pbm -b pdf --output ./pdf/%%~ni.pdf
  1. Merge the output PDF files together:

pdftools --input-dir .\pdf --output merged.pdf --fitpaper

EDIT: You can obtain a version of your PDF vectorized and OCRed. Thus the only difference with Adobe ClearScan is that the fonts are not embedded (but the text is saved as a "Path" object nevertheless

  1. Using ImageMagick, convert your pdf to bmp images (one per page)

magick convert -monochrome -density 300 -alpha off input.pdf ./bmp/input.bmp

  1. Using potrace, postprocess your bmp, save to pbm files, and generate a vectorized version of your pdf
for /r %%i in (./bmp/*.bmp) do mkbitmap ./bmp/%%~ni.bmp --output ./bmpproc/%%~ni.pbm
for /r %%i in (./bmpproc/*.pbm) do potrace ./bmpproc/%%~ni.pbm -b pdf --output ./vectorized/%%~ni.pdf
  1. Using TesserAct, OCR your pdf file

for /r %%i in (./bmpproc/*.pbm) do tesseract ./bmpproc/%%~ni.pbm ./ocred/%%~ni pdf

  1. Using pdftools, merge together the pages of the single files for the ocr-ed and the vectorized version:
python -m pdftools --input-dir ocred  --overwrite --fitpaper --output ocred.pdf --natural-sort
python -m pdftools --input-dir vectorized  --overwrite --fitpaper --output vectorized.pdf --natural-sort
  1. Now we need to remove the image layer from the ocr-ed file (because we will use the vectorized image). Using qPDF, generate an uncompressed version of your ocr-ed file:

qpdf --qdf --object-streams=disable ocr.pdf ocr-uncompressed.pdf

  1. Remove whatever is not an image from your ocr-uncompressed.pdf file with the following python 3 script
with open("ocr-uncompressed.pdf", "rb") as fileh:
    filedata = fileh.read()

streams = filedata.split("endstream".encode("utf8"))
outdata = bytearray()

for rawstream in streams[:-1]:
    before, stream = rawstream.split("stream".encode("utf8"))
    outdata += before + "stream".encode("utf8") 
    if not stream.startswith(bytearray.fromhex("0aff")):
        outdata += stream         
    outdata += "endstream".encode("utf8")

outdata += streams[-1]        

with open("ocr-no-img.pdf", "wb") as fileh:
    fileh.write(outdata)
  1. Now using pdftk, overlay the ocr-ed version (with no images) to the vectorized version:

pdftk vectorized.pdf multibackground ocr-no-image.pdf output mergedlayer.pdf

and you are done!

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.