Are there other products than Adobe's that support ClearScan or similiar?

Question

Starting with Acrobat 9, Adobe engineers added a new flavor of OCR called ClearScan.

The advantages of the ClearScan over Searchable Image OCR are both in size and clarity. The biggest innovation of Clearscan is that:

ClearScan does not replace the font with your system fonts. Rather, a custom font it is created to match the visual appearance of the pixels.¹

Are there other products than Adobe's that support Clearscan or similar products that use a custom font that is created to match the visual appearance of the pixels?

Answer 1

1. Smoothscan is a tool to convert scanned text into a vectorized output form. Because printed text is assembled from fonts, each particular letter (like ‘o’) will have the same shape as every other ‘o’ in the document. We can take advantage of this, by building a table of such symbols, and represent each occurrence of a symbol with a reference to that symbol’s table entry. This will save a lot of space, and a similar idea is used in djvu’s jb2 mode and JBIG2 for PDF.

   smoothscan builds up this table, but instead of filling the table with the original raster images, it vectorizes each symbol. Vector images will look smoother than their raster equivalents, and can be scaled without introducing pixelation. These properties result in a smaller output file size, as well as making the scanned text images more readable.

   smoothscan saves the vectorized images into a custom TrueType font and embeds the font into the output pdf file. Currently each symbol is mapped to an arbitrary letter in the font, but in future versions you could run OCR on each symbol, and ensure that the ‘o’ image is associate with the ‘o’ character encoding in the generated font.

   A search of pkgs.org returns no smoothscan package for any Linux distro. So it needs to be compiled from the source code, its README notes these dependencies:

   • leptonica
   • libharu
   • potrace
   • fontforge (compiled w/ python support):
   • python

   
   On a Ubuntu box, most of these can be had like so:

   apt install libleptonica-dev libhpdf-dev potrace python-fontforge
   

   A test with a sample picture. Load the README in abiword, print it to a .ps file, and then...

   • Use ghostscript to convert that .ps file to .tif,
   • convert that to monochrome .tif, (the only input smoothscan accepts)
   • run smoothscan

   • compare the various file sizes with wc -c:

     gs -sDEVICE=tiffg4 -o README.tif README.ps
     convert -monochrome README.tif README.mono.tif
     smoothscan -o README.pdf README.mono.tif
     wc -c README README.ps README.*tif README.pdf | head -n -1
     

   Output:

     2432 README
    83516 README.ps
    33707 README.tif
    33618 README.mono.tif
    20394 README.pdf
   

   First paragraph of resulting README.pdf, showing vectorized fonts:

   

2. The djvu file format includes a JB2 option, which is much the same thing as ClearScan.

   Using the above README.ps input, the any2djvu util returns excellent results with:

   any2djvu README.ps
   

   ...the resulting README.djvu is only 7K, (1/3 the size of smoothscan's output), and looks sharper:

   

   That sharpness is probably helped by not using a monochrome .tiff as input. That test is an unfair comparison. Let's try it with the mono rasterized .tiff:

   # "-f 6" sets input format to "Scanned Document - B&W - >400 dpi"
   any2djvu -f 6 README.mono.tif
   

   Now the output is less than 6k, but looks a bit jaggier:

   

   Unlike smoothscan's .pdf output, the .djvu output allows text selection, even with rasterized input.

   One bad thing however, any2djvu needs an online server to do the conversion work -- it's not a suitable tool for confidential documents.

Answer 2

NEW ANSWER

My PDFsak can now simulate Adobe Clearscan (using potrace).

OLD ANSWER

Having ImageMagick, potrace and pdftools (disclaimer: I am the author of pdftools). Assuming you have an input PDF file called input.pdf:

1. Convert every page in a BMP file using ImageMagick:

mkdir bmp
magick convert -density 300 input.pdf -quality 100 ./bmp/input.bmp

2. (optional) pre-process image files:

mkdir bmpproc
for /r %%i in (./bmp/*.bmp) do mkbitmap ./bmp/%%~ni.bmp --output ./bmpproc/%%~ni.pbm

3. Rasterize every BMP images and save the output in a PDF:

mkdir pdf
for /r %%i in (./bmpproc/*.pbm) do potrace ./bmpproc/%%~ni.pbm -b pdf --output ./pdf/%%~ni.pdf

4. Merge the output PDF files together:

pdftools --input-dir .\pdf --output merged.pdf --fitpaper

EDIT: You can obtain a version of your PDF vectorized and OCRed. Thus the only difference with Adobe ClearScan is that the fonts are not embedded (but the text is saved as a "Path" object nevertheless

1. Using ImageMagick, convert your pdf to bmp images (one per page)

magick convert -monochrome -density 300 -alpha off input.pdf ./bmp/input.bmp

2. Using potrace, postprocess your bmp, save to pbm files, and generate a vectorized version of your pdf

for /r %%i in (./bmp/*.bmp) do mkbitmap ./bmp/%%~ni.bmp --output ./bmpproc/%%~ni.pbm
for /r %%i in (./bmpproc/*.pbm) do potrace ./bmpproc/%%~ni.pbm -b pdf --output ./vectorized/%%~ni.pdf

3. Using TesserAct, OCR your pdf file

for /r %%i in (./bmpproc/*.pbm) do tesseract ./bmpproc/%%~ni.pbm ./ocred/%%~ni pdf

4. Using pdftools, merge together the pages of the single files for the ocr-ed and the vectorized version:

python -m pdftools --input-dir ocred  --overwrite --fitpaper --output ocred.pdf --natural-sort
python -m pdftools --input-dir vectorized  --overwrite --fitpaper --output vectorized.pdf --natural-sort

5. Now we need to remove the image layer from the ocr-ed file (because we will use the vectorized image). Using qPDF, generate an uncompressed version of your ocr-ed file:

qpdf --qdf --object-streams=disable ocr.pdf ocr-uncompressed.pdf

6. Remove whatever is not an image from your ocr-uncompressed.pdf file with the following python 3 script

with open("ocr-uncompressed.pdf", "rb") as fileh:
    filedata = fileh.read()
    
streams = filedata.split("endstream".encode("utf8"))
outdata = bytearray()

for rawstream in streams[:-1]:
    before, stream = rawstream.split("stream".encode("utf8"))
    outdata += before + "stream".encode("utf8") 
    if not stream.startswith(bytearray.fromhex("0aff")):
        outdata += stream         
    outdata += "endstream".encode("utf8")
    
outdata += streams[-1]        
        
with open("ocr-no-img.pdf", "wb") as fileh:
    fileh.write(outdata)

7. Now using pdftk, overlay the ocr-ed version (with no images) to the vectorized version:

pdftk vectorized.pdf multibackground ocr-no-image.pdf output mergedlayer.pdf

and you are done!