0

I'm looking for a tool that can identify a certain pattern within the text of a Microsoft Word document (.doc) or a PDF document.

By pattern I mean, for example, "all paragraphs starting with xxx and ending with zzz".

I need to be able to collect these data from two documents and compare them together and generate an excel sheet with the differences

Are there any tools that can help me with that? preferably open source. I will use it on windows and it has to be FOSS and if it is not available a library (with a C/C++ interface) would be nice.

Improve this question
2
  • So you're not actually after a parser for DOC or PDF files, you're looking for a text traverser/search tool. Also, you marked this question with C and C++, but it seems you're not looking for source code in C or C++... are you?
    – einpoklum
    Commented Aug 11, 2017 at 21:32
  • if there is a ready software that can do that please tell me about it if not then a library that i can use with some C/C++ code is my only option if you have any other ideas please tell me
    – Eslam Gamal
    Commented Aug 11, 2017 at 21:55

2 Answers 2

Reset to default
2

For MS-Word documents, use LibreOffice on the command-line. It will work with .doc and .docx files as well as other formats (maybe even PDFs, not sure). Extract the text like so:

libreoffice --headless --cat my_file.doc

And now you can just feed that into a text search utility or your own code. On a Unix-ish system, or using Cygwin on Windows, you would do it this way:

libreoffice --headless --cat my_file.doc | grep "some_search_term"

For PDF documents, you can use the pdftotext utility:

pdftotext my_document.pdf - | grep "my_search_term"

Here's a download page for binaries or source; and many Linux distributions package it (although the name might differ).

Improve this answer
3
  • 1
    You could combine the text-output from Libre Office as input to the Apache Lucene engine. This will automatically index the text and give you a wide range of search capabilities. It is open source.
    – Paul Jowett
    Commented Nov 29, 2017 at 8:38
  • 1
    @PaulJowett: +1 on the comment, but remember OP did ask for just a text pattern search, so Lucene sounds like overkill.
    – einpoklum
    Commented Nov 29, 2017 at 14:09
  • agreed. I interpreted "all paragraphs starting with xxx and ending with zzz" as a reasonable indicator that the search requirements would become non-trivial and perhaps hard to grep.
    – Paul Jowett
    Commented Nov 30, 2017 at 1:08
1

With quite a shallow a learning curve you can do this all in python with a couple of libraries:

  • For extracting the text from recent MS-Word formats, (.docx), you will need to install python-docx
  • For extracting the text from the older .doc files you will need to have LibreOffice or MS-Word installed to convert formats, (which can be automated).
  • For extracting the text from pdf files, not a guaranteed endeavour, you can install one of: pdfminer or PyPDF2
  • For creating a file for Excel the simplest is to use the built in csv library but there are also .xlsx writers such as xlwt and numerous others.

The process will be:

  1. Read an input file and split into paragraphs.
  2. Use either starswith and endswith string methods or the standard re regular expression library to get a list of the paragraphs matching your criteria.
  3. Do the same with the other document
  4. Generate a list consisting of those that are different between the two lists.
  5. Write it to the output file.

Features of this solution:

  • Completely FOSS (apart from Word & Excel but you don't have to have them)
  • Windows (and most other platforms)
  • You will have to do a little work
  • No C/C++ required on your part
Improve this answer
2
  • +1, but as OP did not indicate he knows Python, it will be more than a little work. I wonder if you shouldn't make this an answer to a separate question though.
    – einpoklum
    Commented Aug 12, 2017 at 8:12
  • i will try it and update you with the result thank you
    – Eslam Gamal
    Commented Aug 14, 2017 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.