22

We operate a small Windows shop with Exchange. Spam is a constant problem; we get thousands of spam messages a day and manage to remove most of the spam by using a filter on our incoming SMTP port. Still, some trash gets through. 1% of 10K spam messages per day is still really a pain.

We currently use Xwall as a spam filter. It has two really good features.

  1. Graylisting (simply ignoring a first email from sites previously unseen; real mailers at such sites resend after a bit so such mail gets through, but spammers almost never resend). This catches 90% or more.
  2. The ability to eliminate (spam) email by checking for certain string patterns using some simple wild string matches (text with asterisks for wildcards).

However, it isn't consistent in allowing pattern checks on any email field (notably we can't apply any filters to "From:" which the spammers seem to fill with lots of useful "I'm spam" indicators e.g., "From: Discounted Cars (Hilda.Pate@leaderRefreshing.com)". Nor does it allow powerful pattern matching (e.g., decent regexps). Absence of regexps means false positives and false negatives. We've complained to the Xwall guys about this for years but no response.

Is there a spam filter application that has:

  • Graylisting
  • Whitelisting with regexps on email addresses
  • Real regular expression filtering on all email fields, by field. Since regexp filters are invented to stop a particular wave of spam ("What has Martha Stewart done to her face?"), a really nice property would be termination date (default 90 days?) to let such filters age out with the capability to easily re-enable if the spam wave continues.
  • The ability to filter out email based on misspelled words (How many ways can you mis-spell "cialas" using funny characters for i and l?)
  • Recording of all incoming email in a log, with filtering reason (including "none")
  • Ability to designate a filtered-out email entry as "OK" to resubmit, to handle the occasional false positive

Other features such as Bayesian learning are nice to haves but not necessary.

Since such a filter accepts SMTP messages on a publicly visible SMTP port, and forwards (filtered) messages to an internal SMTP port that we provide for Exchange, it doesn't matter where it runs to us. In particular, it is acceptable for the filter to run on either Windows or Linux.

Our Exchange server is rather old (2005) and we're getting ready to upgrade. I'd be delighted to hear all this stuff is built into modern versions of Exchange. Pessimism abounds.

  • "Other features such as Bayesian learning are nice to haves but not necessary." I'm surprised that you don't just switch to other filtering software. Bayesian filtering has been proven to be far more effective than pattern matching. – Jan Doggen Apr 15 '14 at 6:35
  • In what sense? The pattern matching stuff works pretty darn well, and often catches new forms of the same old spam. Bayesian stuff has to learn new spam. I'd be please to have you recommend a solution that had both. I can live without the Bayesian part. I don't want to live without the regex part. – Ira Baxter Apr 15 '14 at 6:46
  • I had offered +100 bounty to get more action on this. It apparantly did not attract more attention. Is the state of spam management tools really this barren? – Ira Baxter Apr 22 '14 at 22:20
2

Bayesian filtering is so millenium. ;) Honestly though, over the last five years the email industry has changed so much that it's nothing like it once was. Bayesian filters used to be the norm for filtering, and now they are very lightly used to only tag the most obvious offenders that are very clearly unsolicited messages.

IP Reputation is where you need to look, as most unsolicited messages come from bot networks filled with IP's that only ever send out a couple hundred spam messages and have no built up reputation so it's an easy spam flag.

Barracuda offers such a service, along with some others.

Honestly though - for what it's worth - move your email to a dedicated email provider. Unless it is critical for your email services to stay in house, it's not worth trying to keep it in house anymore. It's getting to the point where companies need to hire a team of email experts just to keep something like that in house as the maintenance required is getting further and further out of control.

Google would be my top choice, or Amazon. Google's antispam is quite frankly, amazing and false positives are relatively rare and the worst I usually miss out on is a newsletter that I legitimately signed up for.

  • 2
    We don't want our email available to a provider, no matter what assurances they give us (NSA, anyone?) so 3rd party is out of the question. Frankly, we manage fairly well with just Xwall and Exchange, we get about 15,000 emails a day of which most are spam or virus vectors, but only 20-30 spam messages get through our filters. On inspection of these, decent regexps would finish the job pretty well, as we see it. – Ira Baxter Apr 17 '14 at 14:13
  • ... Can you say something more about IP reputation? How is the judgement made that a machine "has only sent a few hundred spam messages"? I would think that one spam message would be enough to toss a sender out. Who keeps track of IP reputation? – Ira Baxter Apr 22 '14 at 11:02
  • ... if this is SpamHaus class stuff, yes we already know about that (and our current solution uses it. Maybe it needs to be in my requirements list, because I don't know how effective it is.) I can see how it is helpful after the fact that a machine is spamming is detected, but it won't stop that machine's 1st thousand messages, whereas a filter on "Viagra" does just fine regardless. – Ira Baxter Apr 22 '14 at 22:34
  • Well the usefulness of IP reputation comes from the fact that an IP starts with 0 reputation. So combine that with the fact that yes, it sees the word viagra in there and it's pretty much a guarantee that it would get flagged. Most mail server IP's would naturally scale up their outgoing volume - they generally wouldn't go from no messages to a million in a day. IP reputation tracks these factors along with what percentage is bounced (most spam bounces a crap tonne of messages in between legitimate addresses), and using all that data will, not unlike bayesian, give it a score. – Ryan Apr 23 '14 at 17:36
  • 1
    @IraBaxter If you're really worried about the NSA, then don't ever send or receive email outside your own internal network, then. Email is not secure and so it generally doesn't matter if it's in the cloud. – Michael Hampton Sep 2 '14 at 9:00
1

GFI MailEssentials supports Outlook with

user-based filtering rules enable flexible and granular filtering of any part of the email message – including message headers, subject, body, attachment name and attachment content -- using different types of pattern-matching methods, including regular expressions.

Also

The POP2Exchange feature gives companies using external POP3 email accounts a practical way to receive emails without the need for a dedicated email server. http://www.gfi.com/products-and-solutions/email-and-messaging-solutions/gfi-mailessentials/specifications

Multiple anti-spam filters http://www.gfi.com/products-and-solutions/email-and-messaging-solutions/gfi-mailessentials/specifications/an-arsenal-of-anti-spam-filters

IP Blocklist Anti-spam Plugin - Lets email admins update GFI MailEssentials with the IP Addresses of ‘spammy’, compromised or mailicious smtp servers so that emails originating from these servers are deleted or quarantined.

  • I've looked at the site. Their online docs actually uses the word "regex" as a filter option on "headers", so this actually looks like a serious candidate. +1 and I may dive into this deeper. – Ira Baxter Jul 24 '14 at 9:36
  • @irabaxter Did you find a solution that met your needs? – WaltHouser Apr 8 '15 at 17:40
0

MailRoute Inc. is an email security provider that ‘washes’ email for viruses and spam. The service works by taking over the role as a middleman in the DNS MX record between the open Internet and the email server. The service also protects against email server outages by temporarily hosting incoming emails until the email server is ready to receive messages again.

MailRoute's features are listed at https://www.mailroute.net/

  • GreyListing to block spam 'bots and zombies
  • Blacklists of known spammers.
  • Spam fingerprint databases.
  • Lexical analysis to identify common spam words or phrases.
  • Bayesian filtering.
  • Distributed traffic pattern analysis identifies new spam.
  • White and Blacklists let you adjust scores, depending on the sender.
  • Quarantine: spam is stored in the MailRoute SpamStore® for seven days, and is accessible via a web-based interface.
  • Subject line rewriting: prepending of a word or phrase and the SpamScore® into the message's Subject line.
  • Custom header lines are added to each message so you can select your own criteria for filtering.
  • Customizable on a per-user or per-domain basis. Each user may have their own individual settings for sensitivity and processing.
  • Domain-wide or individual white and blacklists let you block or allow specific addresses.
  • Compatible with all email systems - regardless of platform or underlying operating system.
  • Implementation takes only minutes and is risk-free.
  • 99.999% uptime guarantee.

See detailed user review at http://community.spiceworks.com/topic/341240-so-far-so-good-on-mailroute-spam-filter-free-trial

  • Sounds sort of reasonable. How does it address the regular expression aspect of my question? – Ira Baxter Jun 6 '14 at 18:50
  • Per MailRoute Advanced Support "We do use a lot of regex pattern matching in our own rules, of course, but none of our user-configurable systems have regex support. I'm not sure, outside of white/blacklisting where our users would use that. Can you explain what you're looking for in more detail?" IMHO it is obvoius that user regular expressions would be used for user whitelist and blacklist. However I wanted to get your thoughts before I answer. – WaltHouser Jun 12 '14 at 21:34
  • From my query above: "Real regular expression filtering on all email fields, by field" Why is this unclear? From a real spam, an obvious example: the from field contains: "Lunar Sleep Sample <LunarSleep@oaferide.com>" I want to say, "if <Lunar Sleep> is in the From: field, mark this as spam". Or "Dr Oz". Its amazing how much cr-- you can eliminate that way. Why is it that mail tool vendors are so afraid of providing regexes to users? Those who don't know what it is, won't use it. Those who do, will use it effectively. And it isn't like regex libraries aren't easily available. – Ira Baxter Jun 12 '14 at 22:10
  • ... Regardless, I don't want a middleman (most of us already want to avoid the NSA, let alone people whose motives we can't control). Mailroute is not a solution for my problem. – Ira Baxter Jun 12 '14 at 22:12
  • End-to-end encryption of the message body would thwart listening-in while using a third party service to filter spam using the message header fields. Middlemen in the form of SMTP relay services is unavoidable. – WaltHouser Jun 17 '14 at 18:02

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.