Is there a tool to get the body of EML files?

Question

I'm working with a corpus of emails (in the order of tens of thousands) which are in the EML format, and I need to extract the text content of these emails.

While removing the message headers is not that hard and easily handled by various scripts floating around the internet, none of them handle some of the quirks of the format, like unescaping escape sequences such as =\n and =3D.

Is there a tool which can get the body of EML files?

The tool should:

• perform the operation in batch
• be gratis
• optionally, runs on Linux

Answer 1

emaildata is a python package for extracting content from email messages.

From the examples:

import email
from emaildata.text import Text

message = email.message_from_file(open('message.eml'))
text = Text.text(message)

This combided with the standard python glob.glob() or os.walk() functions should make this a breeze to perform this as a batch operation.

• Gratis Yes
• Cross Platform (inc Linux)
• Batch/Script Yes

Testing it out

As an example, I exported an email from Quora, (Using Mozilla Thunderbird), as a file and fired up iPython:

Looking through it there were no =3D, =20 entries, (other than in a couple of the urls), but there were a number of sequences like \xe2\x80\x9cused car salesman\xe2\x80\x9d that needed sorting out, these are unicode sequence.

Running: unicode(text, 'utf-8') to decode and printing, (with a unicode compatible font), gives:

Where we find that \xe2\x80\x9cused car salesman\xe2\x80\x9d is “used car salesman” note the up & down quotes.

If you need to decode the parts of a URL with query then you can use urllib.unquote() to deal with them but for your usage you are probably not interested in URLs be they queries or not.

Answer 2

The mailparser module for nodejs handles this well.

The project readme has an example of parsing emails with the module.