"> Qmail-Scanner - An Content Scanner for Qmail Last Updated:
SourceForge Logo
URL: http://qmail-scanner.sourceforge.net/

Qmail-Scanner: Content Scanner for Qmail

Copyright 2000/2004 Jason Haar. This software is distributed under the terms of the GNU General Public License. See COPYING for additional information.

Description

Qmail-Scanner is an add-on that enables a Qmail e-mail server to scan all gateway-ed e-mail for certain characteristics (i.e. a content scanner). It is typically used for its anti-virus protection functions, in which case it is used in conjunction with external virus scanners. but also enables a site (at a server/site level) to react to e-mail that contains specific strings in particular headers, or particular attachment filenames or types (e.g. *.VBS attachments). It also can be used as an archiving tool for auditing or backup purposes. Qmail-Scanner is integrated into the mail server at a lower level than some other Unix-based virus scanners, resulting in better performance. It is capable of scanning not only locally sent/received e-mail, but also e-mail that crosses the server in a relay capacity.

Features

Download

The latest release is 1.25 (via http), and is kindly housed by SourceForge. GnuPG signature of qmail-scanner-1.25.tgz.asc is also available. Of course, you'll be needing my GPG Public Key to verify that.

Requirements

Patches

Bruce Guenter's QMAILQUEUE patch is required to enable Qmail to call a different qmail-queue program than the one compiled in by default. Qmail-scanner's qmail-scanner-queue.pl perl script is used instead of Qmail's qmail-queue binary. After qmail-scanner-queue.pl has run, it calls the original qmail-queue binary to resubmit the message back into the system.

Supported Virus Scanners

The following virus scanners are known to work with qmail-scanner. Other Unix-based scanners should be simple to add support for.

CHANGES

There is a separate page listing changes that have been made between releases

TODO

There is a separate TODO page.

FAQ

There is a separate FAQ page.

Performance/Resource Usage

Adding content/virus scanning to an e-mail server will considerably add to the resource usage of that server. As this "wrapper" is written in perl instead of low-level C, quite a lot of memory and file opens/stats occurs just to get it going. Adding to this the actual scanners memory and CPU usage and it becomes quite complicated (certainly the debugging info shows that the scanner harness spends more time running the external scanners than it does doing things itself [that is to be expected as they do quite a lot of thinking...]).

As a "rule of thumb" I'd suggest you look at how many simultaneous SMTP sessions you are willing your box to have going at any one point in time. Each SMTP session can invoke up to 'n' different virus scanners (although they run one after the other - not simultaneously) and I'd estimate that leads to around 5-6Mb of memory usage per SMTP session. Thus if your dedicated SMTP host has 256Mb RAM + 256Mb swap - that should mean you can handle - well heaps ;-) The scanners cause the CPU to be thrashed while they're running, so I'm making sure for our site that our Qmail server will only accept up to 30 incoming SMTP sessions at any one time - that way I know the box will handle it. As this leads to an increased memory usage, don't forget Qmail's memory limits will need to be increased to deal with it (set via ulimit or softlimit calls with Qmail system startup scripts).

One thing you should test for is what happens if connectivity between this server and another local SMTP server is down for any length of time (due to failure/power outage). When the link is restored, can your server handle the other trying to dump 1,000's of e-mail msgs onto it at once? You need to use softlimit and tcpserver's limit options to ensure your box doesn't get killed. Note that this resource issue isn't caused by Qmail-Scanner. The same thing will happen with a pure, untouched Qmail (or any other) system - it will just happen sooner...

After that scare-mongering I should say that I have tested Qmail-Scanner under ridiculously low resource conditions - and it reacts as it should - so at worst your system should start deferring e-mail. Thankfully DJB's layering of programs is such that this is easy to accomplish :-)

Installation

At this stage qmail-smtpd will need to be "told" that Qmail knows to use qmail-scanner-queue.pl instead of qmail-queue. This is done via the tcpserver control files for smtp. Look to see where tcpserver for qmail-smtpd gets its rules from - it's the file after the "-x" option (well, that's the CDB version actually - find the text file yourself! ;-). Edit that file and tell qmail-smtpd which IP address ranges (corresponds to SMTP client IP addresses) you want Qmail-Scanner to be invoked on - typically all of them.

#/etc/tcpserver/smtp.rules
#
# No Qmail-Scanner at all for mail from 127.0.0.1
127.:allow,RELAYCLIENT="",RBLSMTPD="",QMAILQUEUE="/var/qmail/bin/qmail-queue"
# Use Qmail-Scanner without SpamAssassin on any mail from the local network
# [it triggers SpamAssassin via the presence of the RELAYCLIENT var]
10.:allow,RELAYCLIENT="",RBLSMTPD="",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"
#
# Use Qmail-Scanner with SpamAssassin on any mail from the rest of the world
:allow,QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"

The above example means from now on all SMTP mail will be scanned, but with different characteristics. Mail from the LAN (10. network) will be scanned by the supported virus scanners, whereas mail from the Internet will be scanned for virii AND tagged by SpamAssassin. This finer control allows you a lot of versatility, e.g. virus scanning only performed on mail coming from your Exchange server, and not from your Unix servers.

You must increase the amount of memory your system allows qmail-smtpd to run with, as it it now running the entire perl interpreter PLUS virus scanners. Typical installs of Qmail have system rc/startup scripts (e.g. /etc/rc.d/init.d/qmail or /service/smtp/run) that limit the amount of RAM qmail-smtpd can use via ulimit or softlimit. You must increase that to around 5-11Mb (totally dependent on your OS and choice of anti-virus scanner). If you don't qmail-smtpd will crash with a "qq" error on the receipt of the very first message... The actual amount is dependent on the OS in question as well as the virus scanners being used, so be prepared to experiment a little. Whatever you do, don't just set it to something stupid like 100M "just to be sure". The whole point about limiting RAM usage is so that "unusual" mail messages (e.g. from spammers or hackers) can't cause your system to become unusable by making it run out of RAM.

To scan all mail sent by local shell users, the QMAILQUEUE will also need to be defined within /etc/profile or the like so that when they send mail, it will be affected as well.
Although as they are obviously Unix users, you may want to save your system the effort and explicitly NOT do that! :-)

Also, think twice before running Qmail-Scanner in front of any mailing-list servers. Do you really think it's a good idea to have 10,000 messages banging away at your anti-virus system at the same time? Either put your mailing-list servers beyond the reach of your Qmail-Scanner servers, or put the mailing-list on the Qmail-Scanner servers themselves - that way each message is only scanned once and the load issues disappear.

If "$DEBUG=1" (the default) is set within qmail-scanner-queue.pl, then every transaction will be logged to /var/spool/qmailscan/qmail-queue.log - so you'll see how it goes. Regardless of debugging, errors (and attachment info if enabled) should also be recorded in the qmail logs (probably via syslog) - just look for entries containing the string "X-Qmail-Scanner".

Any SMTP sessions that are dropped (due to network outages/etc) may lead to files lying around in /var/spool/qmailscan . Running /var/qmail/bin/qmail-scanner-queue.pl -z at least once daily will ensure such files are deleted when they're over 30 hours old - make a cronjob to do that. Also realize that /var/spool/qmailscan/qmail-queue.log will grow without bounds. At some stage turn debugging off ($DEBUG=0) and delete the logfile. Personally, I like the logfile, so I run a cronjob that just does "mv -f qmail-queue.log qmail-queue.log.1" at 3am every morning. That way logs don't grow without bound, but you still end up with the logs from the past two days. The file can be safely deleted at any time if it becomes a disk-hog, but unless "$DEBUG=0" is set, it'll just get re-created the next time a message comes through.

Qmail-Scanner contains an internal scanner which allows you to reject e-mail based on attachment filenames and/or e-mail headers. Read the minimal document on it for details.

Philosophy behind Quarantining...

When Qmail-Scanner decided to quarantine a message, it moves it into a local mail folder (maildir format) - by default /var/spool/qmailscan/quarantine/. This means the message can be read in its pure "adulterated" state (e.g. still containing virii) by maildir clients like mutt - or via IMAP (if maildir format supported - you'll have to work that out for yourself). At worse you can just read it with an editor - it's just a MIME file...

If you want a good IMAP server that supports maildir natively - try Courier-IMAP.

I made the decision to write it into maildir format for performance and reliability reasons - and it expressly makes it difficult for any Windows admin to click on it with their vulnerable Windows mailer and read it :-) Qmail actually comes with a program called /var/qmail/bin/maildir2mbox which can do just that... (you could run it from cron to automatically suck all the new mail messages from /var/spool/qmailscan/quarantine/new/ into a mbox.)

Also note that Qmail-Scanner only quarantines. It doesn't drop or "clean" messages. Cleaning a message is complicated and is becoming less of an issue. Personally I feel that if someone sends a virus-laden message through Qmail-Scanner, then it should be blocked - not cleaned. Why should Qmail-Scanner "fix" someone else's problem? People need to take responsibility for their own problems. They get an alert telling them they're infected, so they will need to disinfect their system before attempting to send mail through your site again. Virus scanners that "clean" just make these people think they don't have to worry about their problem - someone else is fixing it.

Qmail-Scanner also doesn't distinguish between "blocking" attachments (via the perlscanner module) and virii. i.e. if a zip file contains a MP3 file, and you are blocking MP3s - then the message will be quarantined. Some people think that Qmail-Scanner should only block files if they are "direct" attachments - not contained within archive attachments. Again, if you have a policy saying "no MP3" - then it should apply no matter how it's sent...

Finally, I am finding the majority of virii are now trojans - which means that there is no actual human-generated message to allow through anyway. There is nothing more annoying than being behind some other vendors anti-virus scanner when 1,000 copies of the latest virus comes through. Receiving 1,000 copies of a "cleaned" trojan is only marginally better than receiving the trojan! (in fact it's worse for Unix users like me! ;-) In fact, but the 10'th copy, all the users receiving mail are now paranoid to open any mail as they don't feel secure about what's going on. The better response is that the ORIGINATOR receive 1,000 alerts telling them they're infected - maybe they'll do something about it.

Also this event is logged in /var/spool/qmailscan/quarantine.log in a tab-delimited format (for post-processing). A good script is needed to convert this file into some nice graphs for management :-). See QSS for an example of one way of generating stats.

If Qmail-Scanner was configured with the "--log-details" option, then a one-line summary of every message processed is recorded either in mailstats.csv or via syslog. e.g:

Aug 14 16:22:41 srvname qmail-scanner[30802]: Clear:RC:1(1.2.3.4): 0.030769 11569 root@x.y jdoe@y.z More_Power! <20020814042234.27902.qmail@x.y> 1029298961.30804-0.srvname:10649 
Aug 14 16:23:17 srvname qmail-scanner[30820]: Clear:RC:0(1.2.3.4): 0.033618 2021  root@x.y jdoe@y.z Cron__run-parts_/etc/cron.daily <20020814042243.28092.qmail@x.y> 1029298997.30822-0.srvname:895 

The format is as follows:

Note: fields are space-delimited when syslog used (with spaces within fields replaced by underscores), and tab-delimited in mailstats.cvs format

Support

This software is released under the GPL as found in the COPYING file enclosed.

This package is housed on SourceForge.

Any questions, suggestions, etc to the mailing-list set up to discuss this, subscribe via http://lists.sourceforge.net/mailman/listinfo/qmail-scanner-general , or subscribe to the announcements-only list via http://lists.sourceforge.net/mailman/listinfo/qmail-scanner-announce.

Last Updated: