0

I have 80,000 PDFs (whitepaper, e-books) that will be used for Marketing. I'm in dire need to look for a tool (hopefully open source) for text mining, classification, categorization of the 80,000 PDFs according to it's Industry, sub-industry. Instead of going thru each PDF to get the correct Industry classification and content matter classification, I hope the text mining tool can do that for me.

Example.

FileName/AssetName                        Industry           Content Matter

Oracle_Security_Cloud_Platform.pdf        Cloud              Technology Overview
Oracle_Security_Cloud_Platform.pdf        Security           Technology Overview
AI_Machine_Learning_framework.pdf         A.I                Comparison Guide
AI_Machine_Learning_framework.pdf         Machine Learning   Comparison Guide

Once I have gathered all data, I hope the tool can help export the results into excel then I will insert those records in a postgre database and Google data studio for analytics or maybe powerBI/tableau to help Marketing team track their campaigns.

Do you have any suggestions that can do the following:

  1. Can provide multiple Content classification of a whitepaper in terms of industry. Eg. Artificial Intelligence, cloud, Security, Big Data, Blockchain, Business Process & Integrations etc.

  2. Can highlight the Top Keywords

  3. Export to Excel

3
  • Could you link to the similar question, please? It might be of help to others who read your question in future. Also, could you please tell us which parts of its solution are no longer acceptable to you, and why? – Mawg says reinstate Monica Dec 26 '20 at 15:37
  • Are you asking for image recognition software (which might be inaccurate), or will you look at each of the 80,000 files and categorize them (which will take a very long time)? – Mawg says reinstate Monica Dec 26 '20 at 15:40
  • And with 2020 almost over, will you also accept software supposed to work 2021? And could software released in 2019 also fit your bill? Or maybe instead of specifying a year you could specify what OS the software should run on? ;) – Izzy Dec 27 '20 at 1:43
0

For R users:

For data analyis I like the tidytext package for R.

The procedures described in the package-documentation (and the O'Reilly book) are relatively simple and generic (can be applied on any text). So you won't need neither a Computer Science PhD nor extensive training in Machine Learning to apply these techniques.

However for preprocessing Text Extraction from the PDFs will need to be done with something else.

Also Text-Mining 1000s of long documents in memory will be very CPU- and memory intensive, so you need either a very big machine from a Cloud Provider, or be familiar with modern parallelization techniques (use a cluster, use a GPU, use a multi-CPU machine etc). Or simply split up the work smartly (group documents by type, domain etc and analyses them independently)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.