2

I'm working with a corpus of emails (in the order of tens of thousands) which are in the EML format, and I need to extract the text content of these emails.

While removing the message headers is not that hard and easily handled by various scripts floating around the internet, none of them handle some of the quirks of the format, like unescaping escape sequences such as =\n and =3D.

Is there a tool which can get the body of EML files?

The tool should:

  • perform the operation in batch
  • be gratis
  • optionally, runs on Linux
2

emaildata is a python package for extracting content from email messages.

From the examples:

import email
from emaildata.text import Text

message = email.message_from_file(open('message.eml'))
text = Text.text(message)

This combided with the standard python glob.glob() or os.walk() functions should make this a breeze to perform this as a batch operation.

  • Gratis Yes
  • Cross Platform (inc Linux)
  • Batch/Script Yes

Testing it out

As an example, I exported an email from Quora, (Using Mozilla Thunderbird), as a file and fired up iPython:

enter image description here

Looking through it there were no =3D, =20 entries, (other than in a couple of the urls), but there were a number of sequences like \xe2\x80\x9cused car salesman\xe2\x80\x9d that needed sorting out, these are unicode sequence.

Running: unicode(text, 'utf-8') to decode and printing, (with a unicode compatible font), gives:

enter image description here

Where we find that \xe2\x80\x9cused car salesman\xe2\x80\x9d is “used car salesman” note the up & down quotes.

If you need to decode the parts of a URL with query then you can use urllib.unquote() to deal with them but for your usage you are probably not interested in URLs be they queries or not.

  • 1
    Unfortunately, this doesn't seem to be capable of handling conversions like =3D => =, =20 => {space}, ={newline char} => empty string that I mentioned in my question :'( – user2064000 Mar 3 '17 at 16:21
  • 1
    url encoding takes care of that nicely – Steve Barnes Mar 4 '17 at 16:27
  • Steve, could you expand on that? – Mawg says reinstate Monica Apr 3 '17 at 7:15
0

The mailparser module for nodejs handles this well.

The project readme has an example of parsing emails with the module.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.