Which text mining software is suited for an introduction lecture?

Question

I've to give an introduction lecture to text mining. I'm looking for a text mining platform that should be:

• open source,
• easy to use,
• well documented,
• provided with examples and tutorials,
• ideally accessible in a scripting language such as Python.
• It should, also, be easy to load commonly used corpus such as WordNet.

Does such a software exist?

Answer 1

The python Natural Language Toolkit, NTLK and optionally SciKit-Learn libraries fit your bill admirably:

• Gratis & Open Source
• easy to use - With a little background reading
• Well documnented - Yes but always open to suggestions/improvements there is even an excellent book
• Examples and Tutorials - Yes lots online
• Accessible in python - they are python libraries
• Easily import corpus such as WordNet - nltk has a list over 50 corpus downloads, including WordNet - detailed instructions are here.
• to install one, or more, copus,
  • install nltk with pip install -U nltk
  • enter the python shell with 'python'
  • import nltk
  • nltk.download() # Starts the downloader below:

• Select what yo need and click Download.
• Once you have installed, (or updated), the corpus that you need you will not need to repeat the process unless you move to a new machine.
  • An installed corpus can be used with from nltk.corpus import wordnet or whichever you wish to use.

If you decide to also use SciKit-Learn the only issue is that if you are working on windows you will probably be better off downloading the following components from the unofficial windows binaries maintained by Christopher Gohlke, in every case you need to download the file that matches the version of python you are using:

1. Download Numpy+MKL from here and install with pip install -U downloaded file
2. Download SciPy from here and install as above
3. Download SciKit-Learn from here and install as above