3

I am looking for server side software to parse a very large number of emails and extract the email signature. Very specifically, I want to input the body of a mail message, identify the signature, and extract the data.

The most important feature is well written pattern recognition. In fact, I would be happy with a text parser as I am comfortable supplying only the body as a large string.

I don't care about the language.

1

You can do this very simply, either client or server side, by using some python scripting.

Python, using just the built in libraries, can:

  • Connect as a email client to download the emails,
  • Search each email body to get any text signature, (valid signatures all start with a line with just two dashes followed by a line "-- \n" but there as some other formats to look for), the python Regular Expression library is especially good for this.
  • Python can also handle image based or html based signatures with ease using only the standard libraries.
  • vCard signatures, (RFC 2426), can be handled with the vcard plugin library,
  • Many servers, especially Linux bases ones, already have python installed, so if you have access to the server you can also do this on it with python but how to access the email content on the server depends on which mail server you are running.

Unfortunately there are a number of, inconsistent, ways of denoting an signature: My signature as produced by Thunderbird, Outlook, etc:

-- 
Steve (Gadget) Barnes
Any opinions in this message are my personal opinions and do not reflect those of my employer.

Note the line containing just hyphen, hyphen, space - this is the "standard" but some other clients attach files, pictures, piles of html, etc., Likewise, the conventions regarding top, bottom or interleaved posting of new content in an email exchange are largely a matter of convention and the marking of previously received text by indentation or by prefixing each line from the previous message with > or some other characters is again dependent on both the email client and often on the users settings.

I would suggest using an incremental search method where, possibly, you try looking for:

  1. -- \n followed by one or more non-blank lines. f1=re.compile(r'^-- \n(.+)', flags=re.MULTILINE+re.DOTALL) should do nicely.
  2. if that is not found you could look for lines starting with any of Tel ephone :, Web site :, Mob ile :, etc. and look for the contiguous, i.e. without blank lines, lines above them.
  3. You could also look for sections of text starting with, Yours sincerely, Yours faithfully, Kind Regards, and other valedictions, (there is a list in the link).

While working on developing your set of regular expressions I can strongly recommend using a python regex checker such as pythex.

Of course this is complicated by there being the possibility of attaching a vCard, a scan of a signature, etc. and by the fact that any given message can be plain text, html, or both and most annoyingly some people just send a word, or other, document as an email.

As far as I know there is no current library that covers all of these possibilities, possibly you could contribute one in time.

|improve this answer|||||
  • Thanks for this. Really clear. Can you please just elaborate a little on what you mean by "-- \n" maybe if you have an example. Also, when you say that there are other formats to look for, is there a comprehensive list somewhere or a python library that has already implemented many of those formats. – Mika Sep 21 '15 at 10:49
  • @Mika I have expanded the answer above to give some examples. – Steve Barnes Sep 21 '15 at 18:09
  • Thanks a million!! Lots to digest – Mika Sep 21 '15 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.