New Antispam Defenses in MPP
I usually don’t write about my own product, MPP, in this blog, but I am very excited about some of our new features. Our goal is to build as much spam detection capabilities as possible into smtp protocol level analysis and to defer as little email as possible to content scanning engines. With this in mind, we have introduced a few new features in MPP that help towards this goal.
One features is ‘thresholds’, which allows providers to define characteristics of email streams in terms of message rate/time period and spam rate/message volume. Senders that either send too much mail or too much spam will be automatically blocked at the smtp level for a specified period of time. This is very helpful to stop drone attacks, but is also a great tool to stop outbound spam. Since both clean and spam mail can be measured this is a very useful tool to find abusive senders.
The other feature of interest is spam-traps. With this feature admins can set-up address templates using regular expressions. Hosts that send to these addresses will be blocked at the SMTP level from sending. This is a great tool to stop dictionary attacks and in large settngs can be quite an effective and simple tool.
Technorati Tags: antispam, spam, outbound spam, email, email security
Incoming Address Verification - Critical Antispam Defense
At Message Partners I work with many customers on antispam defense systems. Many of our customers use our software (MPP), on SMTP filtering proxies and I am surprised how many common it is to have no strategy for verifying incoming email addresses. The cost of accepting email for non-existant addresses is high.
In the SMTP transaction there is a greeting that is followed by the actual data or email. It is desirable to stop as much spam as possible after the smtp greeting, before the actual email data is sent from the remote smtp client. If a remote smtp client tells you in the greeting that it is sending mail to asdf@yourdomain.com and you have no asdf at your site you are better off to reject the greeting than to accept it and then accept the email data. Having a list of valid email addresses in your organization will allow you reject email more efficiently.
SMTP proxies or spam appliances can verify email addresses by checking lists, ldap directories, databases or using smtp verify transactions. SMTP verify is the simplest way to verify messages, however, there are scaling issues for large sites. Having a centralized LDAP directory of all valid email addresses will scale, however, this is difficult for many service providers. Active Directory or other user directories can be queried directly, but for non-msft shops there are good ldap directories to consider.
OpenLDAP is the standard open source directory, but the GUI interfaces tend to me confusing and if you are not an LDAP pro it can be intimidating. Redhat has a directory server, http://www.redhat.com/software/rha/directory/, and a free version called the Fedora Directory Server, that are worth checking out.
Needless Processing - Email to non-existant users should be dropped before the STMP Data transaction, i.e. before the message is accepted. If you process email for non-existant users you are wasting bandwidth, processor and storage resources if you quarantine spam.
In summary, if you don’t validate incoming email addresses you are asking for trouble. If you are using spam quarantine you will fill up your quarantine with bogus emails and pollute your user tables. You are wasting bandwidth by processing junk email to fake accounts and you are wasting storage and processing resources. Centralized directories are a powerful antispam defense and they are worth the effort that they take to establish.
Technorati Tags: spam, antispam, email, ldap, directories, mpp, message partners, isp
IronPort Investors: Merry Late Christmas
Wow! Cisco paid about 10x more than any antispam player that I am aware of for Ironport. Ironport certainly seems to have a great eroding customer base and sales infrastructure, but it is nothing compared to what CSCO already has. Not sure if there is a technology asset that I am not aware of that distinguishes them so much, but I am not aware of anything groudbreaking.
I guess it is good to have good banking buddies. Congratulations Ironport investors.
Technorati Tags: Cisco, IronPort, Spam, Antispam
Lessons of Greylisting
MPP v3.2 has introduced Greylisting via our implementation of a Postfix Policy Server. MPP is the only integrated pre and post-queue filtering solution available for Postfix, either open source or commercial. Since we have introduced greylisting we have learned some lessons that are worth sharing.
First of all, a brief description of greylisting. The idea is to send a temporary failure message to every new smtp client that an smtp server sees. A real email server will retry their message after a temp fail message, but most spam drones go away and never retry. Supposedly spam drones are wising up to this, but so far we don’t see a lot of evidence of this. See http://greylisting.org has more details.
In our very small environment we have found some astounding stats in 3 weeks; 47615 remote smtp clients have tried to send our email server. Of this number, 45149 clients never tried to resend again, in other words 94% of the remote clients were spam drones and just went away. Only 274 remote clients were legit, or less than 1%! That is an astounding number if you ask me, which you didn’t 
Not everyone likes greylisting because it delays email. Since 94% of clients simply give up after the first failure, even a retry delay of 1 second can still be very effective. The problem is that how long it takes for a remote email server to resend after a tempfail message is entirely up to the remote server. If you allow hosts to resend 1second after their first attempt and the remote host will retry in 20 minutes the recipient of the email sees a 20 minute delay and yells at the admin when they need something fast. The good thing for MPP users is that greylisting can be enabled on a per-policy basis so that it can be easily turned off for domains or users that don’t want it. This is a pain to do in most policy-server implementations and is a huge plus for MPP.
Greylisting can work on a gateway or on an email server, but care must be taken when deployed on an email server since email clients use smtp to send email. It is a pretty simple workaround to not perform greylist checks if all hosts are on known networks, but it some environments this is not known information and problems can arise. Specifically, clients will see failure messages when sending email and will get angry. So if you are implementing greylisting on your email server and not on your smtp gateway be careful to whitelist local networks from greylist checks.
Our solution uses MySQL as a backend to store remote host information and this database can become quite huge in large environments. We have seen about 1 million remote hosts in a few days in large environments and 95% were drones. This becomes a very large database, especially with the innodb storage engine so if you plant to use greylisting be very careful to properly size your database server in a large environment.
The spam and phishes that passes greylist checks are sophisticated and are caught at about a 50% rate by content scanners. So while a content scanner may be 95% accurate on all traffic, it becomes about 50% accurate on spam that passes greylist checks. Mostly phishes and some image spam will get through the greylist checks.
That’s all for now, please check out http://messagepartners.com for more on MPP and our greylisting capabilities.
Technorati Tags: MPP, antispam, postfix, greylisting, spam, email security, email