MPP and Beyond

Finally, after about 4 months we have realeased MPP v3.4. It seemed like everytime we were about to release some other requirement came up or structural developments became deeper and took longer than anticipated. The result is our best release in my opinion and makes MPP even cooler, more scalable and more useful. I briefly spoke about the features on our mailing list, but here I will go into more detail about what they are and why we did them.

Pleased to announce the we have released MPP 3.4. There has been a lot of work on this release and we have added many new features and structural enhancements.

1) Dynamic threading for Postfix Policy Server - Automatically create and destroy threads based on load.

MPP is multi-threaded for scalability, but we had been tied to a static threading model, where admins had to pre-set the amount of threads that they anticipated using. Creating too few threads can result in resource problems under high load, and too many can lead to extra RAM usage or in extreme cases, hurt performance. The problem is that when MPP is involved in the SMTP transaction we needed a more dynamic threading model. When an SMTP attack occurs, it is important to be able to handle the flood of connections as both good and bad traffic will suffer. When MPP is a Postfix Policy Server, we are involved in the SMTP process, so it was critical that introduced a dynamic threading model to keep up with traffic bursts.

MPP admins now can define a min and max amount of threads along with some information about tear down periods. The dynamic threading model only applies to SMTP traffic and only when we are working with Postfix as a policy server. In order to activate the new queueing model, set to yes.

2) Internal disk queue for quarantine and archival - Remove direct writing to DB under congestion.

In previous versions of MPP the daemon process was writing email directly to database for quarantine or archive. This can effect email traffic if there is a problem writing to the database for some reason. To remove this direct relationship we now write to a disk queue when the database does not respond in time.

3) Hierarchical quarantine/archival - MPP can use esmtp to transport quarantine/archive traffic to a centralized instance of MPP that then writes to DB.

This feature was added because we have many customers with multiple instances of MPP writing to a single database. This creates prioritization issues that can affect traffic flow. To alleviate this we created a new model whereby we have a centralized MPP instance that handles writing to the database while remote instances of MPP transport traffic destined for quarantine/archival traffic via ESMTP to the central instance of MPP. As a byproduct of this feature we can now run 2 instances of MPP on the same server, for testing purposes, and run MPP with no scanners configured.

4) International language support for signatures.

Many MPP customers are outside of the US and want to use our disclaimers/signatures option. MPP could only support US-ASCII charachters in signatures, but now, we can add signatures with other charachter sets. The administrator defines a signature for different charachter sets found in their country and we add our signature only when there is a match.

5) Loading of mppd.conf.xml from stdout, we now provide a script to load mppd.conf.xml from a tftp server and send it to stdout for mppd to use.

Many of our ISP customers do now want to have a web server on their email server, this is currently required to run our GUI module. Now, we provide a script to load our conf file from a remote TFTP server. Our GUI will support editing and saving of config files in multiple locations so now you can have one central config server that all remote MPP’s use.

6) Custom Reject Notices and API - Write custom reject notices or use MPP as standalone scanning service that can be implemented in scripts.

This feature was created because we can not easily support every mailserver in the universe. Plus, some ISP’s wanted to give more verbose reasons why they are rejecting email. With this feature rejection message templates can be created for different reject conditions that can be sent to a remote smtp server or to a script that called MPP. A script can now pass a message to MPP via SMTP and we will apply our policy engine and scan the message and give results back in a template format. The script can parse our results and take appropriate action.

7) 2 new macros for maildir storage and small maildir fixes. Macros are for first/second letter for user part/domain part of mail address: %U1%, %U2%, %D1%, %D2%.

Someone asked for this as they keep their maildirs seperated by alpahabet. It was easy so we did it.

Whitelists by IP for specific features of greylists, spam traps and tresholds/auto-blacklists

This feature was done because when MPP sits behind a relay server we don’t want to some auto-blacklists to the relay host. Our auto-blacklists will kick in if a threshold is exceeded, such as 10 spams in 30 seconds out of 10 emails. A relay server can easily do this and we wouldn’t want to backlist this IP. Once we added one feature we added a few others like grey listing, since local email should not have greylist tests applied, and spam traps, for the same reason as for auto-blacklists.

9) MPP Custom spam scoring - Assign scores to RBL sites and spam engines as an alternative to first-match actions

We added this feature to make MPP more flexible. Since some customers wanted to add scores from RBL sites and spam scanners together we give the flexiblity to do this now. Some RBL sites are unreliable, such as spamcop, and should not be used for rejection criteria, now they aren’t with MPP.

10) McAffee UVScan command line scanner is now supported

Added this for a customer that needed it, but we did it in a cool way. By supporting a daemon like interface we can use McAffee VirusScan utility much faster than other implementations that we are aware of.

11) Added new SQL format
We did a lot of testing and research on a new format for our MySQL quarantine and we came up with a very optimized format for our database and indecies that will scale very well. We took into account domain admin logins and other typical tasks that required hoggish MySQL joins and designed the DB to avoid these issues.

Technorati Tags: antispam, mpp, message partners, msyql, spam, emai security, email compliance, postfix, policy server