8

Can anyone recommend an open source library/API for extracting the texts and images from a PDF? Below are the requirements. I have used PDF Clown, C# .NET version, but having issues in creating an Excel table using it.

  1. It should be able to convert PDF tables to Excel tables/CSV/XML/HTML.
  2. It should be open source and non GPL.
  3. It should be able to extract images from PDF.
  4. If it can't convert PDF table to some format, it should at least give me the coordinates of texts in a table so that I can create a table out of it.
  5. It should be able to tell me the source information of PDF contents after extraction (which page, where in that page etc.).
1
  • Start by reading ISO-32000-2, then make an assessment of the amount of time it would require to write such a library. You will discover that (1) you are asking for something that is in many cases impossible (unless your PDF is a properly Tagged PDF), and (2) such a library would be so valuable that nobody would offer it under a permissive license. Apr 15, 2015 at 6:05

1 Answer 1

5

Extracting Tables from PDF

Your requirement to extract tables from PDFs is not easy to meet.

Standard PDFs do not provide any hints about the semantics of what they draw on a page: the only distinction that the syntax provides is the distinctions between vector elements (lines, fills,...), images and text.

Whether any character is part of a table or part of a line or just a lonely, single character within an otherwise empty area is not easy to recognize programmatically by parsing the PDF source code.

For a background about why the PDF file format should never, ever be thought of as suitable for hosting extractable, structured data, see this article:

Why Updating Dollars for Docs Was So Difficult (ProPublica-Website)

Having said the above now let me add this:

Tabula is written in Ruby. Its source code is open on GitHub. Its license is Free and non-GPL.

Extracting Images from a PDF

Here I'd recommend to look into Poppler. Poppler is a PDF rendering library forked from the xpdf-3.0 code base.

It ships with a command line utility, pdfimages which can list and extract images embedded in PDF files. Here is an example run which lists all images contained in page range 3-5:

$ pdfimages -list -f 3 -l 5 some.pdf

  page num  type width height color comp bpc  enc interp objectID x-ppi y-ppi  size  ratio
  ----------------------------------------------------------------------------------------
   3     0 image   577   645  rgb     3   8  image  yes     593 0   152   152   106K  9.7%
   3     1 smask   577   645  gray    1   8  image  no      593 0   152   152   383B  0.1%
   3     2 image   382   419  rgb     3   8  image  yes     594 0   110   110  56.7K   12%
   3     3 smask   382   419  gray    1   8  image  no      594 0   110   110   178B  0.1%
   4     4 image  1569   344  rgb     3   8  image  yes     534 0   162   162  36.4K  2.3%
   5     5 image   719   930  index   1   8  image  yes     535 0   146   146  42.6K  6.5%

As you can see, the output is a table containing all kinds of useful information about the embedded images:

  • the page number they appear on,
  • the image type,
  • their width/height dimensions,
  • their color space,
  • the number of base colors,
  • color depth,
  • their encoding,
  • their PDF object ID,
  • their respective resolutions in X or Y directions and
  • their sizes and their compression ratio.

To extract all images from page 9 as PNG files and naming them with a images-from-page9--- prefix, use this command:

$ pdfimages -png -f 9 -l 9 some.pdf images-from-page9---

To see more options, run pdfimages -h.

Of course, if you do not want the command line tool, but instead use the Poppler API/library with your own code, you can achieve the same functions...

Extracting Text from PDF

The same Poppler library also ships with a command line program to extract text: pdftotext.

Here is its short help output:

$ pdftotext -h

 pdftotext version 0.32.0
 Copyright 2005-2015 The Poppler Developers - http://poppler.freedesktop.org
 Copyright 1996-2011 Glyph & Cog, LLC
 Usage: pdftotext [options] <PDF-file> [<text-file>]
   -f <int>          : first page to convert
   -l <int>          : last page to convert
   -r <fp>           : resolution, in DPI (default is 72)
   -x <int>          : x-coordinate of the crop area top left corner
   -y <int>          : y-coordinate of the crop area top left corner
   -W <int>          : width of crop area in pixels (default is 0)
   -H <int>          : height of crop area in pixels (default is 0)
   -layout           : maintain original physical layout
   -fixed <fp>       : assume fixed-pitch (or tabular) text
   -raw              : keep strings in content stream order
   -htmlmeta         : generate a simple HTML file, including the meta information
   -enc <string>     : output text encoding name
   -listenc          : list available encodings
   -eol <string>     : output end-of-line convention (unix, dos, or mac)
   -nopgbrk          : don't insert page breaks between pages
   -bbox             : output bounding box for each word and page size to html.  Sets -htmlmeta
   -opw <string>     : owner password (for encrypted files)
   -upw <string>     : user password (for encrypted files)
   [....]

As you can see, there are various command line switches to address or retrieve coordinates for text locations.

Caveats

It is not always possible to extract text via pdftotext from PDFs. The PDF files may look perfectly good, but their internal structure may be faulty when it comes to text extraction.

See for example these recent StackOverflow answers:


Poppler's license is GPL however....

7
  • Hi, I have installed JRuby and tried to install Tabula using command jruby -S gem install tabula-extractor but for some reason I get syntax error, any thoughts? Apr 15, 2015 at 13:16
  • 2
    @SharkiChacko: "...any thoughts?" -- Yes, two: (1) First one: 'what exactly might that syntax error be, how the hell could I guess it, how could I answer this?!?' (2) Second one: 'why don't you describe your problems in the official tabula-extractor issue tracker?' Apr 15, 2015 at 13:21
  • Wow. Very good summary (+1), congrats! So reading between the lines: there's no "swiss-army" tool covering all 3 components, and then outputs a well-formatted Markdown page to fit an SE post (kidding; HTML with images linked and text well-formatted would be fine as well)?
    – Izzy
    Apr 17, 2015 at 14:17
  • @Izzy: No, no "swiss-army" tool meeting the conditions of the OP: open source but not GPL. (There are commercial tools, though... but they are not as good as tabula when it comes to table extraction.) Apr 18, 2015 at 9:15
  • Thanks Kurt! I completely forgot about that license constellation. Poppler is what I mostly use now, btw. And I'm fine with GPL: all my own software is GPL as well. What I was targeting, basically, was one tool to match it all: what if there's a PDF with tables, pictures, and a lot of text? Open Source preferred, of course – but GPL being fine with me (I don't think that would justify a separate question ;)
    – Izzy
    Apr 18, 2015 at 9:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.