Open source tool to manage large number of e-mail alerts

Question

We have a lot of processes which send e-mails to our whole team on completion. The e-mails include the completion status (error, success, warning), runtime, and any relevant information. The format of the e-mails isn't standardized and this is flooding our team with tons of emails, which most don't have the expertise to deal with and people mostly filter out of their inbox, so the system is not very efficient.

I'm considering building a tool which would receive these e-mail alerts, and:

• send an hourly digest with all the alerts which were received this hour and their status, plus any important information
• on error, immediately send the e-mail out to the whole team
• give the contact info of the person responsible for that specific process in case of issues
• warn if an expected process didn't report in, e.g. no email from an hourly process this hour

While we could write such a system from scratch, I'm wondering if there are any existing open source monitoring tools which may be well suited to such a task?

Answer 1

This would be a 2 stage process:

1. Modify your various process notifiers to all send their notifications to a single mailbox, (preferably with SMTP access).
2. There are a number of email digesters that you could use to collate the mails and periodically send them out - see most mailing lists but given the disparity of formats I suspect that you would have problems getting the more urgent ones sent out. So have a couple of machines, (for redundancy), running a python program that polls the mailbox at a fixed interval, say 5 minutes, and digests the mails - possibly using rules based on the sender ID - then act based on whatever rules you set up, e.g. If the parsing suggests an error then send out an message to everybody, if it suggests a more minor problem look-up the maintainer, project manager and alternate maintainer and mail/cc them, summarise all the mails for a regular update, possibly using the same mailbox. There are a number of ways your main and backup could inform each other that a given mail or mails has been acted upon.

Python has built in modules for retrieving & sending emails, has a lot of text processing capabilities, can easily be structured to apply different rules to different situations, (using dictionaries), is Free and Cross Platform.