Introduction

Email is an important part of any Web site you create. In a home environment, a free web based email service may be sufficient, but if you are running a business, then a dedicated mail server will probably be required.

This chapter will show you how to use sendmail to create a mail server that will relay your mail to a remote user's mailbox or incoming mail to a local mail box. You'll also learn how to retrieve and send mail via your mail server using a with mail client such as Outlook Express or Evolution.

Configuring Sendmail

One of the tasks in setting up DNS for your domain (my-site.com) is to use the MX record in the configuration zone file to state the hostname of the server that will handle the mail for the domain. The most popular Unix mail transport agent is sendmail, but others, such as postfix and qmail, are also gaining popularity with Linux. The steps used to convert a Linux box into a sendmail mail server will be explained here.

How Sendmail Works

As stated before, sendmail can handle both incoming and outgoing mail for your domain. Take a closer look.

Incoming Mail

Usually each user in your home has a regular Linux account on your mail server. Mail sent to each of these users (username@my-site.com) eventually arrives at your mail server and sendmail then processes it and deposits it in the mailbox file of the user's Linux account.

Mail isn't actually sent directly to the user's PC. Users retrieve their mail from the mail server using client software, such as Microsoft's Outlook or Outlook Express, that supports either the POP or IMAP mail retrieval protocols.

Linux users logged into the mail server can read their mail directly using a text-based client, such as mail, or a GUI client, such as Evolution. Linux workstation users can use the same programs to access their mail remotely.

Outgoing Mail

The process is different when sending mail via the mail server. PC and Linux workstation users configure their e-mail software to make the mail server their outbound SMTP mail server.

If the mail is destined for a local user in the mysite.com domain, then sendmail places the message in that person's mailbox so that they can retrieve it using one of the methods above.

If the mail is being sent to another domain, sendmail first uses DNS to get the MX record for the other domain. It then attempts to relay the mail to the appropriate destination mail server using the Simple Mail Transport Protocol (SMTP). One of the main advantages of mail relaying is that when a PC user A sends mail to user B on the Internet, the PC of user A can delegate the SMTP processing to the mail server.

Note: If mail relaying is not configured properly, then your mail server could be commandeered to relay spam. Simple sendmail security will be covered later.

Sendmail Macros

When mail passes through a sendmail server the mail routing information in its header is analyzed, and sometimes modified, according to the desires of the systems administrator. Using a series of highly complicated regular expressions listed in the /etc/mail/sendmail.cf file, sendmail inspects this header and then acts accordingly.

In recognition of the complexity of the /etc/mail/sendmail.cf file, a much simpler file named /etc/sendmail.mc was created, and it contains more understandable instructions for systems administrators to use. These are then interpreted by a number of macro routines to create the sendmail.cf file. After editing sendmail.mc, you must always run the macros and restart sendmail for the changes to take effect.

Each sendmail.mc directive starts with a keyword, such as DOMAIN, FEATURE, or OSTYPE, followed by a subdirective and in some cases arguments. A typical example is.

As stated before, sendmail can handle both incoming and outgoing mail for your domain. Take a closer look.

FEATURE(`virtusertable',`hash -o /etc/mail/virtusertable.db')dnl

The keywords usually define a subdirectory of /usr/share/sendmail-cf in which the macro may be found and the subdirective is usually the name of the macro file itself. So in the example, the macro name is /usr/share/sendmail-cf/feature/virtusertable.m4, and the instruction `\ hash -o /etc/mail/virtusertable.db' is being passed to it.

Notice that sendmail is sensitive to the quotation marks used in the m4 macro directives. They open with a grave mark and end with a single quote.

FEATURE(`masquerade_envelope')dnl

Some keywords, such as define for the definition of certain sendmail variables and MASQUERADE_DOMAIN, have no corresponding directories with matching macro files. The macros in the /usr/share/sendmail-cf/m4 directory deal with these.

Once you finish editing the sendmail.mc file, you can then execute the make command while in the /etc/mail directory to regenerate the new sendmail.cf file.

[root@bigboy tmp]# cd /etc/mail
[root@bigboy mail]# make

If there have been no changes to the files in /etc/mail since the last time make was run, then you'll get an error like this:

[root@bigboy mail]# make
make: Nothing to be done for `all'.
[root@bigboy mail]#

The make command actually generates the sendmail.cf file using the m4 command. The m4 usage is simple, you just specify the name of the macro file as the argument, in this case sendmail.mc, and redirect the output, which would normally go to the screen, to the sendmail.cf file with the ">" redirector symbol.

 [root@bigboy tmp]# m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

I'll discuss many of the features of the sendmail.mc file later in the chapter.

Installing Sendmail

Most RedHat and Fedora Linux software products are available in the RPM format. You will need to make sure that the sendmail, sendmail-cf, and m4 software RPMs are installed. When searching for the RPMs, remember that the filename usually starts with the software package name by a version number, as in sendmail-8.12.10-1.1.1.i386.rpm.

Starting Sendmail

You can use the chkconfig command to get sendmail configured to start at boot:

[root@bigboy tmp]# chkconfig sendmail on

To start, stop, and restart sendmail after booting, use

[root@bigboy tmp]# service sendmail start
[root@bigboy tmp]# service sendmail stop
[root@bigboy tmp]# service sendmail restart

Remember to restart the sendmail process every time you make a change to the configuration files for the changes to take effect on the running process. You can also test whether the sendmail process is running with the pgrep command:

[root@bigboy tmp]# pgrep sendmail

You should get a response of plain old process ID numbers.

How To Restart Sendmail After Editing Your Configuration Files

In this chapter, you'll see that sendmail uses a variety of configuration files that require different treatments for their commands to take effect. This little script encapsulates all the required post configuration steps.

#!/bin/bash
cd /etc/mail
make
newaliases
/etc/init.d/sendmail restart

It first runs the make command, which creates a new sendmail.cf file from the sendmail.mc file and compiles supporting configuration files in the /etc/mail directory according to the instructions in the file /etc/mail/Makefile. It then generates new e-mail aliases with the newaliases command, (this will be covered later), and then restarts sendmail.

Use this command to make the script executable.

chmod 700 filename

You'll need to run the script each time you change any of the sendmail configuration files described in the sections to follow.

The line in the script that restarts sendmail is only needed if you have made changes to the /etc/mail/sendmail.mc file, but I included it so that you don't forget. This may not be a good idea in a production system.

Note: When sendmail starts, it reads the file sendmail.cf for its configuration. sendmail.mc is a more user friendly configuration file and really is much easier to fool around with without getting burned. The sendmail.cf file is located in different directories depending on the version of RedHat you use. The /etc/sendmail.cf file is used for versions up to 7.3, and /etc/mail/sendmail.cf is used for versions 8.0 and higher and Fedora Core.

The /etc/mail/sendmail.mc File

You can define most of sendmail's configuration parameters in the /etc/mail/sendmail.mc file, which is then used by the m4 macros to create the /etc/mail/sendmail.cf file. Configuration of the sendmail.mc file is much simpler than configuration of sendmail.cf, but it is still often viewed as an intimidating task with its series of structured directive statements that get the job done. Fortunately, in most cases you won't have to edit this file very often.

How to Put Comments in sendmal.mc

In most Linux configuration files a # symbol is used at the beginning of a line convert it into a comment line or to deactivate any commands that may reside on that line.

The sendmail.mc file doesn't use this character for commenting, but instead uses the string "dnl". Here are some valid examples of comments used with the sendmail.mc configuration file:

These statements are disabled by dnl commenting.

dnl DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')
dnl # DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')

This statement is incorrectly disabled:

# DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')

This statement is active:

DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')

Configuring DNS for sendmail

Remember that you will never receive mail unless you have configured DNS for your domain to make your new Linux box mail server the target of the DNS domain's MX record.

Configure Your Mail Server's Name In DNS

You first need to make sure that your mail server's name resolves in DNS correctly. For example, if your mail server's name is bigboy and it you intend for it to mostly handle mail for the domain my-site.com, then bigboy.my-site.com must correctly resolve to the IP address of one of the mail server's interfaces. You can test this using the host command:

[root@smallfry tmp]# host bigboy.my-site.com
bigboy.my-site.com has address 192.168.1.100
[root@smallfry tmp]#

You will need to fix your DNS server's entries if the resolution isn't correct.

Configure The /etc/resolv.conf File

The sendmail program expects DNS to be configured correctly on the DNS server. The MX record for your domain must point to the IP address of the mail server.

The program also expects the files used by the mail server's DNS client to be configured correctly. The first one is the /etc/resolv.conf file in which there must be a domain directive that matches one of the domains the mail server is expected to handle mail for.

Finally, sendmail expects a nameserver directive that points to the IP address of the DNS server the mail server should use to get its DNS information.

For example, if the mail server is handling mail for my-site.com and the IP address of the DNS server is 192.168.1.100, there must be directives that look like this:

domain my-site.com
nameserver 192.168.1.100

An incorrectly configured resolv.conf file can lead to errors when running the m4 command to process the information in your sendmail.mc file.

WARNING: local host name (smallfry) is not qualified; fix $j in config file

The /etc/hosts File

The /etc/hosts file also is used by DNS clients and also needs to be correctly configured. Here is a brief example of the first line you should expect to see in it:

127.0.0.1 bigboy.my-site.com localhost.localdomain localhost bigboy

The entry for 127.0.0.1 must always be followed by the fully qualified domain name (FQDN) of the server. In the case above it would be bigboy.my-site.com. Then you must have an entry for localhost and localhost.localdomain. Linux does not function properly if the 127.0.0.1 entry in /etc/hosts doesn't also include localhost and localhost.localdomain. Finally you can add any other aliases your host may have to the end of the line.

How To Configure Linux Sendmail Clients

All Linux mail clients in your home or company need to know which server is the mail server. This is configured in the sendmail.mc file by setting the SMART_HOST statement to include the mail server. In the example below, the mail server has been set to mail.my-site.com, the mail server for the my-site.com domain.

define(`SMART_HOST',`mail.my-site.com')

If you don't have a mail server on your network, you can either create one, or use the one offered by your ISP.

Once this is done, you need to process the sendmail.mc file and restart sendmail. To do this, run the restarting script we from earlier in the chapter.

If the sendmail server is a Linux server, then the /etc/hosts file will also have to be correctly configured too.

Converting From a Mail Client to a Mail Server

All Linux systems have a virtual loopback interface that lives only in memory with an IP address of 127.0.0.1. As mail must be sent to a target IP address even when there is no NIC in the box, sendmail therefore uses the loopback address to send mail between users on the same Linux server. To become a mail server, and not a mail client, sendmail needs to be configured to listen for messages on NIC interfaces as well.

1) Determine which NICs sendmail is running on. You can see the interfaces on which sendmail is listening with the netstat command. Because sendmail listens on TCP port 25, you use netstat and grep for 25 to see a default configuration listening only on IP address 127.0.0.1 (loopback):

[root@bigboy tmp]# netstat -an | grep :25 | grep tcp
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
[root@bigboy tmp]#

2) Edit sendmail.mc to make sendmail listen on all interfaces. If sendmail is listening on the loopback interface only, you should comment out the daemon_options line in the /etc/mail/sendmail.mc file with dnl statements. It is also good practice to take precautions against spam by not accepting mail from domains that don't exist by commenting out the accept_unresolvable_domains feature too. See the fourth and next to last lines in the example.

dnl
dnl This changes sendmail to only listen on the loopback
dnl device 127.0.0.1 and not on any other network 
dnl devices. Comment this out if you want
dnl to accept email over the network.
dnl DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')
dnl
...
...
...
dnl
dnl We strongly recommend to comment this one out if you want
dnl to protect yourself from spam. However, the laptop and 
dnl users on computers that do
dnl not have 24x7 DNS do need this.
dnl FEATURE(`accept_unresolvable_domains')dnl
dnl FEATURE(`relay_based_on_MX')dnl
dnl

Note: You need to be careful with the accept_unresolvable_names feature. In the sample network, bigboy the mail server does not accept e-mail relayed from any of the other PCs on your network if they are not in DNS. shows how to create your own internal domain just for this purpose.

Note: If your server has multiple NICs and you want it to listen to one of them, then you can uncomment the localhost DAEMON_OPTIONS entry and add another one for the IP address of the NIC on which to wish to accept SMTP traffic.

3) Comment out the SMART_HOST Entry in sendmal.mc. The mail server doesn't need a SMART_HOST entry in its sendmail.mc file. Comment this out with a dnl at the beginning.

dnl define(`SMART_HOST',`mail.my-site.com')

4) Regenerate the sendmail.cf file, and restart sendmail. Again, you can do this with the restart script from the beginning of the chapter.

5) Make sure sendmail is listening on all interfaces (0.0.0.0).

[root@bigboy tmp]# netstat -an | grep :25 | grep tcp
tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN
[root@bigboy tmp]#

You have now completed the first phase of converting your Linux server into a sendmail server by enabling it to listen to SMTP traffic on its interfaces. The following sections will show you how to define what type of mail it should handle and the various ways this mail can be processed.

A General Guide To Using The sendmail.mc File

The sendmail.mc file can seem jumbled. To make it less cluttered I usually create two easily identifiable sections in it with all the custom commands I've ever added.

The first section is near the top where the FEATURE statements usually are, and the second section is at the very bottom.

Sometimes sendmail will archive this file when you do a version upgrade. Having easily identifiable modifications in the file will make post upgrade reconfiguration much easier. Here is a sample:

dnl ***** Customised section 1 start *****
dnl
dnl
FEATURE(delay_checks)dnl
FEATURE(masquerade_envelope)dnl
FEATURE(allmasquerade)dnl
FEATURE(masquerade_entire_domain)dnl
dnl
dnl
dnl ***** Customised section 1 end *****

The /etc/mail/relay-domains File

The /etc/mail/relay-domains file is used to determine domains from which it will relay mail. The contents of the relay-domains file should be limited to those domains that can be trusted not to originate spam. By default, this file does not exist in a standard RedHat / Fedora install. In this case, all mail sent from my-super-duper-site.com and not destined for this mail server will be forwarded:

my-super-duper-site.com

One disadvantage of this file is that controls mail based on the source domain only, and source domains can be spoofed by spam e-mail servers. The /etc/mail/access file has more capabilities, such as restricting relaying by IP address or network range and is more commonly used. If you delete /etc/mail/relay-domains, then relay access is fully determined by the /etc/mail/access file.

Be sure to run the restart sendmail script from the beginning of the chapter for these changes to take effect.

The /etc/mail/access File

You can make sure that only trusted PCs on your network have the ability to relay mail via your mail server by using the /etc/mail/access file. That is to say, the mail server will relay mail only for those PCs on your network that have their e-mail clients configured to use the mail server as their outgoing SMTP mail server. (In Outlook Express, you set this using: Tools>Accounts>Properties>Servers)

If you don't take the precaution of using this feature, you may find your server being used to relay mail for spam e-mail sites. Configuring the /etc/mail/access file will not stop spam coming to you, only spam flowing through you.

The /etc/mail/access file has two columns. The first lists IP addresses and domains from which the mail is coming or going. The second lists the type of action to be taken when mail from these sources or destinations is received. Keywords include RELAY, REJECT, OK (not ACCEPT), and DISCARD. There is no third column to state whether the IP address or domain is the source or destination of the mail, sendmail assumes it could be either and tries to match both. All other attempted relayed mail that doesn't match any of the entries in the /etc/mail/access file, sendmail will reject. Despite this, my experience has been that control on a per e-mail address basis is much more intuitive via the /etc/mail/virtusertable file.

The sample file that follows allows relaying for only the server itself (127.0.0.1, localhost), two client PCs on your home 192.168.1.X network, everyone on your 192.168.2.X network, and everyone passing e-mail through the mail server from servers belonging to my-site.com. Remember that a server will be considered a part of my-site.com only if its IP address can be found in a DNS reverse zone file:

localhost.localdomain           RELAY
localhost                       RELAY
127.0.0.1                       RELAY
192.168.1.16                    RELAY
192.168.1.17                    RELAY
192.168.2                       RELAY
my-site.com                     RELAY

You'll then have to convert this text file into a sendmail readable database file named /etc/mail/access.db. Here are the commands you need:

[root@bigboy tmp]# cd /etc/mail
[root@bigboy mail]# make

The sendmail restart script we configured at the beginning of the chapter does this for you too.

Remember that the relay security features of this file may not work if you don't have a correctly configured /etc/hosts file.

The /etc/mail/local-host-names File

When sendmail receives mail, it needs a way of determining whether it is responsible for the mail it receives. It uses the /etc/mail/local-host-names file to do this. This file has a list of hostnames and domains for which sendmail accepts responsibility. For example, if this mail server was to accept mail for the domains my-site.com and another-site then the file would look like this:

my-site.com
another-site.com

In this case, remember to modify the MX record of the another-site.com DNS zonefile point to my-site.com. Here is an example (Remember each "." is important):

; Primary Mail Exchanger for another-site.com

another-site.com. MX 10 mail.my-site.com.

Which User Should Really Receive The Mail?

After checking the contents of the virtusertable, sendmail checks the aliases files to determine the ultimate recipient of mail.

The /etc/mail/virtusertable file

The /etc/mail/virtusertable file contains a set of simple instructions on what to do with received mail. The first column lists the target email address and the second column lists the local user's mail box, a remote email address, or a mailing list entry in the /etc/aliases file to which the email should be forwarded.

If there is no match in the virtusertable file, sendmail checks for the full email address in the /etc/aliases file.

webmaster@another-site.com   webmasters
@another-site.com            marc
sales@my-site.com             sales@another-site.com
paul@my-site.com              paul
finance@my-site.com           paul
@my-site.com                  error:nouser User unknown

In this example, mail sent to:

webmaster@another-site.com will go to local user (or mailing list) webmasters, all other mail to another-site.com will go to local user marc.
sales at my-site.com will go to the sales department at my-othersite.com.
paul and finance at my-site.com goes to local user (or mailing list) paul

All other users at my-site.com receive a bounce back message stating "User unknown".

After editing the /etc/mail/virtusertable file, you have to convert it into a sendmail-readable database file named /etc/mail/virtusertable.db with two commands:

[root@bigboy tmp]# cd /etc/mail
[root@bigboy mail]# make

If these lines look like you've seen them before, you have: They're in your all-purpose sendmail restart script.

The /etc/aliases File

You can think of the /etc/aliases file as a mailing list file. The first column has the mailing list name (sometimes called a virtual mailbox), and the second column has the members of the mailing list separated by commas.

To start, sendmail searches the first column of the file for a match. If there is no match, then sendmail assumes the recipient is a regular user on the local server and deposits the mail in their mailbox.

If it finds a match in the first column, sendmail notes the nickname entry in the second column. It then searches for the nickname again in the first column to see if the recipient isn't on yet another mailing list.

If sendmail doesn't find a duplicate, it assumes the recipient is a regular user on the local server and deposits the mail in their mailbox.

If the recipient is a mailing list, then sendmail goes through the process all over again to determine if any of the members is on yet another list, and when it is all finished, they all get a copy of the e-mail message.

In the example that follows, you can see that mail sent to users bin, daemon, lp, shutdown, apache, named, and so on by system processes will all be sent to user (or mailing list) root. In this case, root is actually an alias for a mailing list consisting of user marc and webmaster@my-site.com.

# Basic system aliases -- these MUST be present.
mailer-daemon:        postmaster
postmaster:           root

# General redirections for pseudo accounts.
bin:                  root
daemon:               root
...
 ...
abuse:                root
# trap decode to catch security attacks
decode:               root

# Person who should get root's mail
root:                 marc,webmaster@my-site.com

Notice that there are no spaces between the mailing list entries for root: You will get errors if you add spaces.

Note: The default /etc/aliases file installed with RedHat / Fedora has the last line of this sample commented out with a #, you may want to delete the comment and change user marc to another user. Also after editing this file, you'll have to convert it into a sendmail readable database file named /etc/aliases.db. Here is the command to do that:

[root@bigboy tmp]# newaliases

In this simple mailing list example, mail sent to root actually goes to user account marc and webmaster@my-site.com. Because aliases can be very useful, here are a few more list examples for your /etc/aliases file.

Mail to "directors@my-site.com" goes to users "peter", "paul" and "mary".

# Directors of my SOHO company
directors:      peter,paul,mary

Mail sent to "family@my-site.com" goes to users "grandma", "brother" and "sister"

# My family
family:        grandma,brother,sister

Mail sent to admin-list gets sent to all the users listed in the file /home/mailings/admin-list.

# My mailing list file
admin-list:     ":include:/home/mailings/admin-list"

The advantage of using mailing list files is that the admin-list file can be a file that trusted users can edit, user root is only needed to update the aliases file. Despite this, there are some problems with mail reflectors. One is that bounce messages from failed attempts to broadcast go to all users. Another is that all subscriptions and unsubscriptions have to be done manually by the mailing list administrator. If either of these are a problem for you, then consider using a mailing list manager, such as majordomo.

One important note about the /etc/aliases file: By default your system uses sendmail to mail system messages to local user root. When sendmail sends e-mail to a local user, the mail has no To: in the e-mail header. If you then use a mail client with a spam mail filtering rule to reject mail with no To: in the header, such as Outlook Express or Evolution, you may find yourself dumping legitimate mail.

To get around this, try making root have an alias for a user with a fully qualified domain name, this forces sendmail to insert the correct fields in the header; for example:

# Person who should get root's mail
root:                 webmaster@my-site.com

Sendmail Masquerading Explained

If you want your mail to appear to come from user@mysite.com and not user@bigboy.mysite.com, then you have two choices:

Configure your email client, such as Outlook Express, to set your email address to user@mysite.com. (I'll explain this in the "Configuring Your POP Mail Server" section.).
Set up masquerading to modify the domain name of all traffic originating from and passing trough your mail server.

Configuring masquerading

In the DNS configuration, you made bigboy the mail server for the domain my-site.com. You now have to tell bigboy in the sendmail configuration file sendmail.mc that all outgoing mail originating on bigboy should appear to be coming from my-site.com; if not, based on our settings in the /etc/hosts file, mail will appear to come from mail.my-site.com. This isn't terrible, but you may not want your Web site to be remembered with the word "mail" in front of it. In other words you may want your mail server to handle all email by assigning a consistent return address to all outgoing mail, no matter which server originated the email.

You can solve this by editing your sendmail.mc configuration file and adding some masquerading commands and directives:

FEATURE(always_add_domain)dnl
FEATURE(`masquerade_entire_domain')dnl
FEATURE(`masquerade_envelope')dnl
FEATURE(`allmasquerade')dnl
MASQUERADE_AS(`my-site.com')dnl
MASQUERADE_DOMAIN(`my-site.com.')dnl
MASQUERADE_DOMAIN(localhost)dnl
MASQUERADE_DOMAIN(localhost.localdomain)dnl

The result is that:

The MASQUERADE_AS directive makes all mail originating on bigboy appear to come from a server within the domain my-site.com by rewriting the email header.
The MASQUERADE_DOMAIN directive makes mail relayed via bigboy from all machines in the another-site.com and localdomain domains appear to come from the MASQUERADE_AS domain of my-site.com. Using DNS, sendmail checks the domain name associated with the IP address of the mail relay client sending the mail to help it determine whether it should do masquerading or not.
FEATURE masquerade_entire_domain makes sendmail masquerade servers named *my-site.com, and *another-site.com as my-site.com. In other words, mail from sales.my-site.com would be masqueraded as my-site.com. If this wasn't selected, then only servers named my-site.com and my-othersite.com would be masqueraded. Use this with caution when you are sure you have the necessary authority to do this.
FEATURE allmasquerade makes sendmail rewrite both recipient addresses and sender addresses relative to the local machine. If you cc: yourself on an outgoing mail, the other recipient sees a cc: to an address he knows instead of one on localhost.localdomain.

Note: Use FEATURE allmasquerade with caution if your mail server handles email for many different domains and the mailboxes for the users in these domains reside on the mail server. The allmasquerade statement causes all mail destined for these mailboxes to appear to be destined for users in the domain defined in the MASQUERADE_AS statement. In other words, if MASQUERADE_AS is my-site.com and you use allmasquerade, then mail for peter@another-site.com enters the correct mailbox but sendmail rewrites the To:, making the e-mail appear to be sent to peter@my-ste.com originally.

FEATURE always_add_domain always masquerades email addresses, even if the mail is sent from a user on the mail server to another user on the same mail server.
FEATURE masquerade_envelope rewrites the email envelope just as MASQUERADE_AS rewrote the header.

Masquerading is an important part of any mail server configuration as it enables systems administrators to use multiple outbound mail servers, each providing only the global domain name for a company and not the fully qualified domain name of the server itself. All email correspondence then has a uniform email address format that complies with the company's brand marketing policies.

Note: E-mail clients, such as Outlook Express, consider the To: and From: statements as the e-mail header. When you choose Reply or Reply All in Outlook Express, the program automatically uses the To: and From: in the header. It is easy to fake the header, as spammers often do; it is detrimental to e-mail delivery, however, to fake the envelope.

The e-mail envelope contains the To: and From: used by mailservers for protocol negotiation. It is the envelope's From: that is used when e-mail rejection messages are sent between mail servers.

Testing Masquerading

The best way of testing masquerading from the Linux command line is to use the "mail -v username" command. I have noticed that "sendmail -v username" ignores masquerading altogether. You should also tail the /var/log/maillog file to verify that the masquerading is operating correctly and check the envelope and header of test email received by test email accounts.

Other Masquerading Notes

By default, user "root" will not be masqueraded. To remove this restriction use:

EXPOSED_USER(`root')dnl

command in /etc/mail/sendmail.mc. You can comment this out if you like with a "dnl" at the beginning of the line and running the sendmail start script.

Using Sendmail to Change the Sender's Email Address

Sometimes masquerading isn't enough. At times you may need to change not only the domain of the sender but also the username portion of the sender's e-mail address. For example, perhaps you bought a program for your SOHO office that sends out notifications to your staff, but the program inserts its own address as sender's address, not that of the IT person.

Web-based CGI scripts tend to run as user apache and, therefore, send mail as user apache too. Often you won't want this, not only because apache's e-mail address may not be a suitable, but also because some anti-spam programs check to ensure that the From:, or source e-mail address, actually exists as a real user. If your virtusertable file allows e-mail to only predefined users, then queries about the apache user will fail, and your valid e-mail may be classified as being spam.

With sendmail, you can change both the domain and username on a case-by-case basis using the genericstable feature:

1) Add these statements to your /etc/mail/sendmail.mc file to activate the feature:

FEATURE(`genericstable',`hash -o /etc/mail/genericstable.db')dnl
GENERICS_DOMAIN_FILE(`/etc/mail/generics-domains')dnl

2) Create a /etc/mail/generics-domains file that is just a list of all the domains that should be inspected. Make sure the file includes your server's canonical domain name, which you can obtain using the command:

sendmail -bt -d0.1 </dev/null

Here is a sample /etc/mail/generics-domains file:

my-site.com
another-site.com
bigboy.my-site.com

3) Create your /etc/mail/genericstable file. First sendmail searches the /etc/mail/generics-domains file for a list of domains to reverse map. It then looks at the /etc/mail/genericstable file for an individual email address from a matching domain. The format of the file is

linux-username       username@new-domain.com

Here is an example:

alert          security-alert@my-site.com
peter          urgent-message@my-site.com
apache         mailer@my-site.com

4) Run the sendmail restart script from the beginning of the chapter and then test.

Your e-mails from linux-username should now appear to come from username@new-domain.com.

Troubleshooting Sendmail

There are a number of ways to test sendmail when it doesn't appear to work correctly. Here are a few methods you can use to fix some of the most common problems.

Testing TCP connectivity

The very first step is to determine whether your mail server is accessible on the sendmail SMTP TCP port 25. Lack of connectivity could be caused by a firewall with incorrect permit, NAT, or port forwarding rules to your mail server. Failure could also be caused by the sendmail process being stopped. It is best to test this from both inside your network and from the Internet.

Chapter 4, "Simple Network Troubleshooting", covers troubleshooting with TELNET.

Further Testing of TCP connectivity

You can also mimic a full mail session using TELNET to make sure everything is working correctly. If you get a "500 Command not recognized" error message along the way, the cause is probably a typographical error. Follow these steps carefully.

1) Telnet to the mail server on port 25. You should get a response with a 220 status code.

[root@bigboy tmp]# telnet mail.my-site.com 25
Trying mail.my-site.com...
Connected to mail.my-site.com.
Escape character is '^]'.
220 mail.my-site.com ESMTP server ready

2) Use the hello command to tell the mail server the domain you belong to. You should receive a message with a successful status 250 code at the beginning of the response.

helo another-web-site.org
250 mail.my-site.com Hello c-24-4-97-110.client.comcast.net [24.4.97.110], pleased to meet you.

3) Inform the mail server from which the test message is coming with the MAIL FROM: statement.

MAIL FROM:sender@another-web-site.org
250 2.1.0 sender@another-web-site.org... Sender ok

4) Tell the mail server to whom the test message is going with the " RCPT TO:" statement.

RCPT TO: user@my-site.com
250 2.1.5 user@my-site.com... Recipient ok

5) Prepare the mail server to receive data with the DATA statement

DATA
354 Enter mail, end with "." on a line by itself

6) Type the string "subject:" then type a subject. Type in your text message, ending it with a single period on the last line. For example.

Subject: Test Message
Testing sendmail interactively
.
250 2.0.0 iA75r9si017840 Message accepted for delivery

7) Use the QUIT command to end the session.

QUIT
221 2.0.0 mail.my-site.com closing connection
Connection closed by foreign host.
[root@bigboy tmp]#

Now verify that the intended recipient received the message, and check the system logs for any mail application errors.

The /var/log/maillog File

Because sendmail writes all its status messages in the /var/log/maillog file, always monitor this file whenever you are doing changes. Open two TELNET, SSH, or console windows. Work in one of them and monitor the sendmail status output in the other using the command

[root@bigboy tmp]# tail -f /var/log/maillog

Common Errors Due To Incomplete RPM Installation

Both the newaliases and m4 commands require the sendmail-cf and m4 RPM packages. These must be installed. If they are not, you'll get errors when running various sendmail related commands.

Sample Errors when running newaliases

[root@bigboy mail]# newaliases
Warning: .cf file is out of date: sendmail 8.12.5 supports version 10, .cf file is version 0
No local mailer defined
QueueDirectory (Q) option must be set
[root@bigboy mail]#

Sample errors when processing the sendmail.mc file

[root@bigboy mail]# m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf
/etc/mail/sendmail.mc:8: m4: Cannot open /usr/share/sendmail-cf/m4/cf.m4: No such file or directory
[root@bigboy mail]#

Sample errors when restarting sendmail

[root@bigboy mail]# service sendmail restart
Shutting down sendmail: [ OK ]
Shutting down sm-client: [FAILED]
Starting sendmail: 554 5.0.0 No local mailer defined
554 5.0.0 QueueDirectory (Q) option must be set
[FAILED]
Starting sm-client: [ OK ]
[root@bigboy mail]#

If these errors occur, make sure your m4, sendmail and senmail-cf RPM packages are installed correctly.

Incorrectly Configured /etc/hosts Files

By default, Fedora inserts the hostname of the server between the 127.0.0.1 and the localhost entries in /etc/hosts like this:

127.0.0.1     bigboy    localhost.localdomain    localhost

Unfortunately in this configuration, sendmail will think that the server's FQDN is bigboy, which it will identify as being invalid because there is no extension at the end, such as .com or .net. It will then default to sending e-mails in which the domain is localhost.localdomain.

The /etc/hosts file is also important for configuring mail relay. You can create problems if you fail to place the server name in the FDQN for 127.0.0.1 entry. Here sendmail thinks that the server's FDQN was my-site and that the domain was all of .com.

127.0.0.1   my-site.com  localhost.localdomain   localhost  (Wrong!!!)

The server would therefore be open to relay all mail from any .com domain and would ignore the security features of the access and relay-domains files I'll describe later.

As mentioned, a poorly configured /etc/hosts file can make mail sent from your server to the outside world appear as if it came from users at localhost.localdomain and not bigboy.my-site.com.

Use the sendmail program to send a sample e-mail to someone in verbose mode. Enter some text after issuing the command and end your message with a single period all by itself on the last line, for example:

[root@bigboy tmp]# sendmail -v example@another-site.com
test text
test text
.
example@another-site.com... Connecting to mail.another-site.com. via esmtp... 
220 ltmail.another-site.com LiteMail v3.02(BFLITEMAIL4A); Sat, 05 Oct 2002 06:48:44 -0400
>>> EHLO localhost.localdomain
250-mx.another-site.com Hello [67.120.221.106], pleased to meet you
250 HELP
>>> MAIL From:<root@localhost.localdomain>
250 <root@localhost.localdomain>... Sender Ok
>>> RCPT To:<example@another-site.com>
250 <example@another-site.com>... Recipient Ok
>>> DATA
354 Enter mail, end with "." on a line by itself
>>> .
250 Message accepted for delivery
example@another-site.com... Sent (Message accepted for delivery)
Closing connection to mail.another-site.com.
>>> QUIT
[root@bigboy tmp]#

localhost.localdomain is the domain that all computers use to refer to themselves, it is therefore an illegal Internet domain. Consider an example: Mail sent from computer PC1 to PC2 appears to come from a user at localhost.localdomain on PC1 and is rejected. The rejected e-mail is returned to localhost.localdomain. PC2 sees that the mail originated from localhost.localdomain and thinks that the rejected e-mail should be sent to a user on PC2 that may not exist. You end up with an error in /var/log/maillog:

Oct 16 10:20:04 bigboy sendmail[2500]: g9GHK3iQ002500: SYSERR(root): savemail: cannot save rejected email anywhere
Oct 16 10:20:04 bigboy sendmail[2500]: g9GHK3iQ002500: Losing ./qfg9GHK3iQ002500: savemail panic

You may also get this error if you are using a spam prevention program, such as a script based on the PERL module Mail::Audit. An error in the script could cause this type of message too.

Another set of tell tale errors caused by the same problem can be generated when trying to send mail to a user (the example uses root) or creating a new alias database file. (I'll explain the newaliases command later.)

[root@bigboy tmp]# sendmail -v  root
WARNING: local host name (bigboy) is not qualified; fix $j in config file
[root@bigboy tmp]# newaliases
WARNING: local host name (bigboy) is not qualified; fix $j in config file
[root@bigboy tmp]#

An accompanying error in /var/log/maillog log file looks like this:

Oct 16 10:23:58 bigboy sendmail[2582]: My unqualified host name (bigboy) unknown; sleeping for retry

Fighting SPAM

Unsolicited Commercial Email (UCE or SPAM) can be annoying, time consuming to delete and in some cases dangerous when they contain viruses and worms. Fortunately there are ways you can use your mail server to combat SPAM.

Using Public SPAM Blacklists With Sendmail

There are many publicly available lists of known open mail relay servers and spam generating mail servers on the Internet. Some are maintained by volunteers, others are managed by public companies, but in all cases they rely heavily on complaints from spam victims. Some spam blacklists simply try to determine whether the e-mail is coming from a legitimate IP address.

The IP addresses of offenders usually remain on the list for six months to two years. In some cases, to provide additional pressure on the spammers, the blacklists include not only the offending IP address but also the entire subnet or network block to which it belongs. This prevents the spammers from easily switching their servers' IP addresses to the next available ones on their networks. Also, if the spammer uses a public data center, it is possible that their activities could also cause the IP addresses of legitimate e-mailers to be black listed too. It is hoped that these legitimate users will pressure the data center's management to evict the spamming customer.

You can configure sendmail to use its dnsbl feature to both query these lists and reject the mail if a match is found. Here are some sample entries you can add to your /etc/sendmail.mc file; they should all be on one line.

RFC-Ignorant: A valid IP address checker.

FEATURE(`dnsbl', `ipwhois.rfc-ignorant.org',`"550 Mail from " $&{client_addr} " refused. Rejected for bad WHOIS info on IP of your SMTP server - see http://www.rfc-ignorant.org/"')

Easynet: An open proxy list.

FEATURE(`dnsbl', `proxies.blackholes.easynet.nl', `"550 5.7.1 ACCESS DENIED to OPEN PROXY SERVER "$&{client_name}" by easynet.nl DNSBL  (http://proxies.blackholes.easynet.nl/errors.html)"', `')dnl

The Open Relay Database: An open mail relay list.

FEATURE(`dnsbl', `relays.ordb.org', `"550 Email rejected due to sending server misconfiguration - see http://www.ordb.org/faq/\#why_rejected"')dnl

Spamcop: A spammer blacklist.

FEATURE(`dnsbl', `bl.spamcop.net', `"450 Mail from " $`'&{client_addr} " refused - see http://spamcop.net/bl.shtml"')

Spamhaus: A spammer blacklist.

FEATURE(`dnsbl',`sbl.spamhaus.org',`Rejected - see http://spamhaus.org/')dnl

Be sure to visit the URLs listed to learn more about the individual services.

Spamassassin

Once sendmail receives an e-mail message, it hands the message over to procmail, which is the application that actually places the e-mail in user mailboxes on the mail server. You can make procmail temporarily hand over control to another program, such as a spam filter. The most commonly used filter is spamassassin.

spamassassin doesn't delete spam, it merely adds the word "spam" to the beginning of the subject line of suspected spam e-mails. You can then configure the e-mail filter rules in Outlook Express or any other mail client to either delete the suspect message or store it in a special Spam folder.

Downloading And Installing Spamassassin

Most RedHat and Fedora Linux software products are available in the RPM format. When searching for the RPMs, remember that the filename usually starts with the software package name and is followed by a version number, as in spamassassin-2.60-2.i386.rpm.

Starting Spamassassin

You can use the chkconfig command to get spamassassin configured to start at boot:

[root@bigboy tmp]# chkconfig --level 35 spamassassin on

To start, stop, and restart spamassassin after booting:

[root@bigboy tmp]# service spamassassin start
[root@bigboy tmp]# service spamassassin stop
[root@bigboy tmp]# service spamassassin restart

Configuring procmail for spamassassin

The /etc/procmailrc file is used by procmail to determine the procmail helper programs that should be used to filter mail. This file isn't created by default.

spamassassin has a template you can use called /etc/mail/spamassassin/spamassassin-spamc.rc. Copy the template to the /etc directory.

[root@bigboy tmp]# cp /etc/mail/spamassassin/spamassassin-spamc.rc /etc/procmailrc

Configuring Spamassassin

The spamassassin configuration file is named /etc/mail/spamassassin/local.cf. A full listing of all the options available in the local.cf file can be found in the Linux man pages using the following command:

[root@bigboy tmp]# man Mail::SpamAssassin::Conf

You can customize this fully commented sample configuration file to meet your needs.

###################################################################
# See 'perldoc Mail::SpamAssassin::Conf' for
# details of what can be adjusted.
###################################################################
 
#
# These values can be overridden by editing
# ~/.spamassassin/user_prefs.cf (see spamassassin(1) for details)
#
 
# How many hits before a message is considered spam. The lower the
# number the more sensitive it is.
 
required_hits           5.0
 
 
# Whether to change the subject of suspected spam (1=Yes, 0=No)
rewrite_subject         1
 
 
# Text to prepend to subject if rewrite_subject is used
subject_tag             *****SPAM*****
 
 
# Encapsulate spam in an attachment (1=Yes, 0=No)
report_safe             1
 
 
# Use terse version of the spam report (1=Yes, 0=No)
use_terse_report        0
 
 
# Enable the Bayes system (1=Yes, 0=No)
use_bayes               1
 
 
# Enable Bayes auto-learning (1=Yes, 0=No)
auto_learn              1
 
 
# Enable or disable network checks (1=Yes, 0=No)
skip_rbl_checks         0
use_razor2              1
use_dcc                 1
use_pyzor               1
 
 
# Mail using languages used in these country codes will not be marked
# as being possibly spam in a foreign language.
# - english
 
ok_languages            en
 
 
# Mail using locales used in these country codes will not be marked
# as being possibly spam in a foreign language.
 
ok_locales              en

Be sure to restart spamassassin for your changes to take effect.

Testing spamassassin

You can test the validity of your local.cf file by using the spamassassin command with the --lint option. This will list any syntax problems that may exist. In this example two errors were found and corrected before the command was run again.

[root@bigboy tmp]# spamassassin -d --lint
Created user preferences file: /root/.spamassassin/user_prefs
config: SpamAssassin failed to parse line, skipping: use_terse_report        0
config: SpamAssassin failed to parse line, skipping: auto_learn              1
lint: 2 issues detected.  please rerun with debug enabled for more information.
[root@bigboy tmp]# vi /etc/mail/spamassassin/local.cf
...
...
...
[root@bigboy tmp]# spamassassin -d --lint
[root@bigboy tmp]

Startup spamassassin

The final steps are to configure spamassassin to start on booting and then to start it.

[root@bigboy tmp]# chkconfig spamassassin on
[root@bigboy tmp]# service spamassassin start
Starting spamd: [  OK  ]
[root@bigboy tmp]#

Tuning spamassassin

You can tune the sensitivity of spamassassin to the type of spam you receive by adjusting the required_hits value in the local.cf file. This can be made easier by viewing the score spamassassin assigns a message in its header. In most GUI based email clients this can be done by looking at the email's properties. In this case, a Nigerian email scam spam was detected and given a score of 20.1 and marked as spam.

X-Spam-Status: Yes, score=20.1 required=2.1 tests=DEAR_FRIEND,
	DNS_FROM_RFC_POST,FROM_ENDS_IN_NUMS,MSGID_FROM_MTA_HEADER,NA_DOLLARS,
	NIGERIAN_BODY1,NIGERIAN_BODY2,NIGERIAN_BODY3,NIGERIAN_BODY4,
	RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_SBL,RISK_FREE,SARE_FRAUD_X3,
	SARE_FRAUD_X4,SARE_FRAUD_X5,US_DOLLARS_3 autolearn=failed 
	version=3.0.4
X-Spam-Report: 
	*  0.5 FROM_ENDS_IN_NUMS From: ends in numbers
	*  0.2 RISK_FREE BODY: Risk free.  Suuurreeee....
	*  0.4 US_DOLLARS_3 BODY: Mentions millions of $ ($NN,NNN,NNN.NN)
	*  0.8 DEAR_FRIEND BODY: Dear Friend? That's not very dear!
	*  2.2 NA_DOLLARS BODY: Talks about a million North American dollars
	*  1.8 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net
	*      [Blocked - see <http://www.spamcop.net/bl.shtml?213.185.106.3>]
	*  1.1 RCVD_IN_SBL RBL: Received via a relay in Spamhaus SBL
	*      [213.185.106.3 listed in sbl-xbl.spamhaus.org]
	*  1.4 DNS_FROM_RFC_POST RBL: Envelope sender in postmaster.rfc-ignorant.org
	*  1.9 NIGERIAN_BODY3 Message body looks like a Nigerian spam message 3+
	*  2.9 NIGERIAN_BODY1 Message body looks like a Nigerian spam message 1+
	*  1.4 NIGERIAN_BODY4 Message body looks like a Nigerian spam message 4+
	*  1.7 SARE_FRAUD_X5 Matches 5+ phrases commonly used in fraud spam
	*  0.5 NIGERIAN_BODY2 Message body looks like a Nigerian spam message 2+
	*  1.7 SARE_FRAUD_X3 Matches 3+ phrases commonly used in fraud spam
	*  1.7 SARE_FRAUD_X4 Matches 4+ phrases commonly used in fraud spam
	*  0.0 MSGID_FROM_MTA_HEADER Message-Id was added by a relay

Limiting your spam fighting efforts to the required_hits value isn't usually adequate. You will probably need additional spamassassin tools to be more selective and accurate in your tests. This will be covered next.

The Rules du Jour Spamassassin Tool

There are groups of open source developers that create and update customized spamassassin configuration files that make the tool even more effective. They have even made life easier for the harried systems administrator by creating a script named rules_du_jour that, on a daily basis, will automatically download the rules you select.

The rules_du_jour script can be downloaded from its website at http://www.exit0.us/index.php?pagename=RulesDuJour which has easy to understand installation instructions, but there are some caveats which need to be explained.

The /etc/rulesdujour/config Configuration File

Rules du Jour's configuration file located at /etc/rulesdujour/config has four variables that need to be defined. Each must be enclosed in quotation marks.

The first is SA_DIR, which defines the directory in which you have installed spamassassin. The second is MAIL_ADDRESS which defines the address to which Rules du Jour sends its status messages. The third, SA_RESTART, is the command to be used to restart spamassassin each time the rules_du_jour script is run.

The final parameter, TRUSTED_RULESETS, is the most complicated. It is a space delimited list of all the rules you wish to use. A full list can be found on the Rules du Jour website but there isn't much explanation about what they do and how sensitive each one is to marking email as being spam. Fortunately, you can get this information from the "Rules" section of the Rules Emporium site (http://www.rulesemporium.com/).

It is important to read the notes for each rule they sometimes have sub-groupings of rules that may more suitable for your needs. For example, the SARE_HTML rule includes all the rules in SARE_HTML0, SARE_HTML1, SARE_HTML2, SARE_HTML3 and SARE_HTML_ENG, but according to Rules Emporium, only SARE_HTML0 has a low degree of false positives.

Here is a sample of a /etc/rulesdujour/config configuration file that has taken advantage of some of the more popular and reliable rules.

#
# File: /etc/rulesdujour/config
#

#
# Script information can be found at:
#
# http://www.exit0.us/index.php?pagename=RulesDuJour
#

SA_DIR="/etc/mail/spamassassin"
MAIL_ADDRESS="rulesdujour_update@my-web-site.org"
SA_RESTART="service spamd restart"
TRUSTED_RULESETS="TRIPWIRE SARE_ADULT SARE_OBFU SARE_URI0 SARE_URI1 ANTIDRUG SARE_SPOOF SARE_BAYES_POISON_NXM SARE_OEM SARE_RANDOM SARE_FRAUD SARE_HEADER0 SARE_HEADER2 SARE_HTML0 SARE_SPECIFIC SARE_BML SARE_GENLSUBJ0 SARE_GENLSUBJ2 SARE_WHITELIST"

Note: The Rules du Jour and Rules Emporium sites use the terms "spam" and "ham" frequently. Spam is unwanted email, while ham is the opposite.

Installing Rules du Jour

Installation is fairly simple, here is how it is done:

1) Download the rules_du_jour script with the wget command, make it executable and place it in the /usr/local/bin directory.

[root@bigboy tmp]# wget http://sandgnat.com/rdj/rules_du_jour
--10:58:27--  http://sandgnat.com/rdj/rules_du_jour
           => `rules_du_jour'
Resolving sandgnat.com... 208.42.148.125
Connecting to sandgnat.com|208.42.148.125|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 63,448 (62K) [application/octet-stream]
10:58:28 (79.59 KB/s) - `rules_du_jour' saved [63448/63448]

[root@bigboy tmp]# chmod 700 rules_du_jour 
[root@bigboy tmp]# mv rules_du_jour /usr/local/bin

2) Create and edit your /etc/rulesdujour/config configuration file.

[root@bigboy tmp]# mkdir -p /etc/rulesdujour
[root@bigboy tmp]# vi /etc/rulesdujour/config

3) Run the rules_du_jour script, and then run spamassassin in lint mode to test for errors. There should be none.

[root@bigboy tmp]# /usr/local/bin/rules_du_jour 
exec: curl -w %{http_code} --compressed -O -R -s -S -z /etc/mail/spamassassin/RulesDuJour/rules_du_jour http://sandgnat.com/rdj/rules_du_jour 2>&1
curl_output: 304

------ TRIPWIRE ------
RULESET_NAME=TRIPWIRE
INDEX=0
CF_URL=http://www.rulesemporium.com/rules/99_FVGT_Tripwire.cf
...
...
...

/^\s*#.*(vers?|version|rev|revision)[:\.\s]*[0-9]/i;' | sort | tail -n 1
CF_MUNGE_SCRIPT=
[root@bigboy tmp]# spamassassin -d --lint
[root@bigboy tmp]#

4) The final step is to add /usr/local/bin/rules_du_jour to your cron table. In this case, I have just created the file /etc/cron.d/rulesdujour with the following entry:

#
# Get latest SpamAssassin rules. Runs at 12:23AM every day.
#
0 23 * * *      root    /usr/local/bin/rules_du_jour

5) You may have to restart crond to get this configuration to become active

[root@bigboy tmp]# service crond restart
Stopping crond: [  OK  ]
Starting crond: [  OK  ]
[root@bigboy tmp]#

My experience with Rules du Jour has been very good. Without it, I could only set the required_hits value in the /etc/mail/spamassassin/local.cf file to a relatively insensitive value of 4.5. Anything lower would create too many false positives and valuable mail would be marked as being suspicious. Rules du Jour selectively raises the score of specific types of spam such that I can make the required_hits setting have a very sensitive value of 2.1 with very few false positives.

Using Greylisting

To maximize the effect of their efforts, spammers try to send email as quickly as possible. They take note of the emails that bounce, so that they know which addresses to remove from their lists to make their next mailing more efficient.

When mail servers receive mail too rapidly for them to handle, they can ask the sender to try again later. Spammers often view resending emails to valid addresses as a waste of computing time that could be used to send mail to brand new addresses that belong to faster mail servers. Emails that need to be resent are usually abandoned.

Some emails need reliable delivery to be effective and the senders of these types of messages are willing to resend. These include bank statement notifications, ecommerce purchase confirmations, and subscription newsletters.

In a previous section we saw where spamassassin always rejects emails from blacklisted sources. With greylisting, sources are just asked to resend. One of the most popular greylist mail filter (milter) products is the milter-greylist package which also works seamlessly with spamassassin. It is easy to use and I’ll discuss how can be configured on your mail server.

Downloading and Installing milter-greylist

Installing milter-greylist is relatively simple, but there are a lot of steps. Here’s how to do it:

1. You will have to first install the sendmail-devel software package. Most RedHat and Fedora Linux software products are available in the RPM format. When searching for the RPMs, remember that the filename usually starts with the software package name and is followed by a version number, as in sendmail-devel-8.13.1-2.rpm.

2. The next step is to download the milter-greylist tar file which you can download from the milter’s website at http://projects.puremagic.com/greylisting/. In this case we download the version 2.0.2 file using the wget command.

[root@bigboy tmp]# wget ftp://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.0.2.tgz
--10:48:06--  ftp://ftp.espci.fr/pub/milter-greylist/milter-greylist-2.0.2.tgz
           => `milter-greylist-2.0.2.tgz'
Resolving ftp.espci.fr... 193.54.82.47
Connecting to ftp.espci.fr|193.54.82.47|:21... connected.
Logging in as anonymous ... Logged in!
=> SYST ... done.    => PWD ... done.
=> TYPE I ... done.  => CWD /pub/milter-greylist ... done.
=> PASV ... done.    => RETR milter-greylist-2.0.2.tgz ... done.
Length: 116,459 (114K) (unauthoritative)
10:48:20 (70.79 KB/s) - `milter-greylist-2.0.2.tgz' saved [116459]
[root@bigboy tmp]#

3. Now that you have the file, you’ll have to extract the contents using the tar command.

[root@bigboy tmp]# tar -xzvf milter-greylist-2.0.2.tgz 
milter-greylist-2.0.2
milter-greylist-2.0.2/ChangeLog
milter-greylist-2.0.2/Makefile
milter-greylist-2.0.2/Makefile.in
milter-greylist-2.0.2/README
…
…
…
milter-greylist-2.0.2/spf.h
milter-greylist-2.0.2/sync.c
milter-greylist-2.0.2/sync.h
[root@bigboy tmp]#

4. Now enter the newly created milter-greylist directory and compile the package with the configure command. Take a look at the README file before doing this as there may be some additional options you require, but for most cases the defaults are sufficient.

[root@bigboy milter-greylist-2.0.2]# ./configure && make && make install
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
...
...
...
/usr/bin/install -c -d -m 755 /etc/mail
test -f /etc/mail/greylist.conf -o -f /etc/mail/greylist.except || \
     /usr/bin/install -c -m 644 greylist.conf /etc/mail
/usr/bin/install -c -d -m 755 -o root /var/milter-greylist
[root@bigboy milter-greylist-2.0.2]#

The next step is to configure the package, which will be covered next.

Configuring milter-greylist

Configuring milter-greylist requires these four quick steps:

1. Add the milter-greylist statements listed in the README file to your /etc/mail/sendmail.mc file:

INPUT_MAIL_FILTER(`greylist',`S=local:/var/milter-greylist/milter-greylist.sock')
define(`confMILTER_MACROS_CONNECT', `j, {if_addr}')
define(`confMILTER_MACROS_HELO', `{verify}, {cert_subject}')
define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}')
define(`confMILTER_MACROS_ENVRCPT', `{greylist}')

2. There will be a number of shell scripts in the milter-greylist tar directory that correspond to various versions of Linux. Copy the correct version to your /etc/init.d directory and use the chkconfig command to make sure the milter starts on the next reboot.

[root@bigboy milter-greylist-2.0.2]# cp rc-redhat.sh /etc/init.d/milter-greylist
[root@bigboy milter-greylist-2.0.2]# chmod 755 /etc/init.d/milter-greylist 
[root@bigboy milter-greylist-2.0.2]# chkconfig --add milter-greylist
[root@bigboy milter-greylist-2.0.2]# chkconfig milter-greylist on
[root@bigboy milter-greylist-2.0.2]# chkconfig --list | grep milter
milter-greylist 0:off   1:off   2:on    3:on    4:on    5:on    6:off
[root@bigboy milter-greylist-2.0.2]#

3. Edit the /etc/mail/greylist.conf configuration file. Here we set the “try again later” to five minutes and use the whitelist command to deactivate the timer for trusted networks so that mail is delivered immediately.

#
# File: /etc/mail/greylist.conf
#

# How long a client has to wait before we accept
# the messages it retries to send. Here, 1 hour.
#
greylist 5m

#
# Whitelist addresses within my own home/office network
#
acl whitelist addr 192.168.0.0/16

4. Start the milter with the service command.

[root@bigboy milter-greylist-2.0.2]# service milter-greylist start 
Starting Milter-Greylist: [  OK  ]
[root@bigboy milter-greylist-2.0.2]#

Your new spam mitigation tool should now be fully functional. Delete the mister-greylist directory in /tmp and you should be ready to go!

Configuring milter-greylist

Now that we have milter-greylist installed, we need to be able to do some basic troubleshooting. The /var/log/maillog file should be used to determine what is happening to your mail. Here are two samples of what to expect:

Dec 24 00:32:31 bigboy sendmail[28847]: jBO8WVnG028847: Milter: to=<spamvictim@my-web-site.org>, 
reject=451 4.7.1 Greylisting in action, please come back in 00:05:00

Dec 23 20:40:21 bigboy milter-greylist: jBO4eF2m027418: addr 211.115.216.225 from 
<slashdot@slashdot.org> rcpt <spamvictim@my-web-site.org>: autowhitelisted for 24:00:00

In the first entry, the email received is given a tag (jBO8WVnG028847) based on key characteristics in the mail header and a request is sent to the sender to resend the email in five minutes. Any email that is received with the same calculated key within the autowhite period configured in the greylist.conf file will then be automatically accepted without delay. In the second entry, the email has been resent and immediately accepted. Any other email from that source within the next 24 hours will be accepted without delay.

Note: Greylisting is very effective, but you will have to tne its operation to make sure critical emails are not delayed at all. One soluton is to set the autowhite period in /etc/mail/greylist.conf to slightly more than 24 hours especially if you get mail from certain recipients, such as newsletters, on a daily basis. This makes them arrive without interruption.

A Simple PERL Script To Help Stop SPAM

Blacklists won't stop everything, but you can limit the amount of unsolicited spam you receive by writing a small script to intercept your mail before it is written to your mailbox.

This is fairly simple to do, because sendmail always checks the .forward file in your home directory for the name of this script. The sendmail program then looks for the filename in the directory /etc/smrsh and executes it.

By default, PERL doesn't come with modules that are able to check e-mail headers and envelopes so you have to download them from CPAN (www.cpan.org). The most important modules are:

MailTools
IO-Stringy
MIME-tools
Mail-Audit

I have written a script called mail-filter.pl that effectively filters out spam e-mail for my home system. A few steps are required to make the script work:

Install PERL and the PERL modules you downloaded from CPAN.
Place an executable version of the script in your home directory and modify the script's $FILEPATH variable point to your home directory.
Update file mail-filter.accept, which specifies the subjects and e-mail addresses to accept, and file mail-filter.reject, which specifies those to reject.
Update your .forward file and place an entry in /etc/smrsh.

Mail-filter first rejects all e-mail based on the reject file and then accepts all mail found in the accept file. It then denies everything else.

For a simple script with instructions on how to install the PERL modules, see Appendix II.

Configuring Your POP Mail Server

Each user on your Linux box will get mail sent to their account's mail folder, but sendmail just handles mail sent to your my-site.com domain. If you want to retrieve the mail from your Linux box's user account using a mail client such as Evolution, Microsoft Outlook or Outlook Express, then you have a few more steps. You'll also have to make your Linux box a POP mail server.

Fedora Linux comes with the easy to use dovecot IMAP/POP server RPM package which requires very little configuration after installation.

Installing Your POP Mail Server

Most RedHat and Fedora Linux software products are available in the RPM format. You will need to make sure that the dovecot software RPM is installed. (Chapter 6, will tell you how.) When searching for the RPMs, remember that the filename usually starts with the software package name by a version number, as in dovecot-0.99.11-1.FC3.4.i386.rpm.

Starting Your POP Mail Server

You can use the chkconfig command to get dovecot configured to start at boot:

[root@bigboy tmp]# chkconfig dovecot on

To start, stop, and restart dovecot after booting, use

[root@bigboy tmp]# service dovecot start
[root@bigboy tmp]# service dovecot stop
[root@bigboy tmp]# service dovecot restart

Remember to restart the dovecot process every time you make a change to the configuration files for the changes to take effect on the running process. You can also test whether the dovecot process is running with the pgrep command:

[root@bigboy tmp]# pgrep dovecot

You should get a response of plain old process ID numbers:

The /etc/dovecot.conf File

You can define most of dovecot's configuration parameters in the /etc/dovecot.conf file. By default dovecot will act as a server for IMAP, secure IMAP (IMAPS), POP and secure POP (POPS). You can limit this list by editing the protocols line in the /etc/dovecot.conf file and then restarting dovecot for the change to take effect. In the example below dovecot is configured to serve only POP3.

#
# File /etc/dovecot.conf sample
#
 
# Protocols we want to be serving imap imaps pop3 pop3s
#protocols = imap imaps pop3 pop3s
protocols = pop3

You can then use the netstat command to do a simple preliminary test to make sure dovecot is serving POP3 only.

[root@bigboy tmp]# netstat -a | egrep -i 'pop|imap'
tcp        0      0 *:pop3                     *:*                         LISTEN      
[root@bigboy tmp]#

How To Configure Your Windows Mail Programs

All your POP e-mail accounts are really only regular Linux user accounts in which sendmail has deposited mail. You can now configure your e-mail client such as Outlook Express to use your use your new POP/SMTP mail server quite easily. To configure POP Mail, set your POP mail server to be the IP address of your Linux mail server. Use your Linux user username and password when prompted.

Next, set your SMTP mail server to be the IP address/domain name of your Linux mail server.

Configuring Secure POP Mail

If you need to access your e-mail from the mail server via the Internet or some other insecure location, you may want to configure POP to work over an encrypted data channel. For this, modify the protocols section of the /etc/dovecot.conf file to include pop3s and then restart dovecot. Encrypted POP runs on TCP port 995, so firewall rules may need to be altered as well.

Most POP clients support secure POP. For example, Windows configures it in the Advanced menu of the Outlook Express Account Configuration window.

How to handle overlapping email addresses.

If you have user overlap, such as John Smith (john@my-site.com) and John Brown (john@another-site.com), both users will get sent to the Linux user account john by default. You have two options for a solution:

Make the user part of the email address different, john1@my-site.com and

john2@another-site.com for example, and create Linux accounts john1 and john2. If the users insist on overlapping names, then you may need to modify your virtusertable file.

Create the user accounts john1 and john2 and point virtusertable entries for john@my-site.com to account john1 and point john@another-site.com entries to account john2. The POP configuration in Outlook Express for each user should retrieve their mail via POP using john1 and john2, respectively.

With this trick you'll be able to handle many users belonging to multiple domains without many address overlap problems.

Troubleshooting POP Mail

The very first troubleshooting step is to determine whether your POP server is accessible on the POP TCP port 110 or the secure POP port of 995. Lack of connectivity could be caused by a firewall with incorrect permit, NAT, or port forwarding rules to your server. Failure could also be caused by the xinetd process being stopped or the configuration files being disabled. Test this from both inside your network and from the Internet. (Troubleshooting TCP with TELNET is covered in Chapter 4,

Linux status messages are logged to the file /var/log/messages. Use it to make sure all your files are loaded when you start xinetd. Check your configuration files if it fails to do so. This example starts xinetd and makes a successful secure POP query from a remote POP client:

Aug 11 23:20:33 bigboy xinetd[18690]: START: pop3s pid=18693 from=172.16.1.103
Aug 11 23:20:33 bigboy ipop3d[18693]: pop3s SSL service init from 172.16.1.103
Aug 11 23:20:40 bigboy ipop3d[18693]: Login user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0/0
Aug 11 23:20:40 bigboy ipop3d[18693]: Logout user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0 ndele=0
Aug 11 23:20:40 bigboy xinetd[18690]: EXIT: pop3s pid=18693 duration=7(sec)
Aug 11 23:20:52 bigboy xinetd[18690]: START: pop3s pid=18694 from=172.16.1.103
Aug 11 23:20:52 bigboy ipop3d[18694]: pop3s SSL service init from 172.16.1.103
Aug 11 23:20:52 bigboy ipop3d[18694]: Login user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0/0
Aug 11 23:20:52 bigboy ipop3d[18694]: Logout user=labmanager host=172-16-1-103.my-site.com [172.16.1.103] nmsgs=0 ndele=0
Aug 11 23:20:52 bigboy xinetd[18690]: EXIT: pop3s pid=18694 duration=0(sec)

Conclusion

E-mail is an important part of any Web site, and you need to plan its configuration carefully to make it a seamless part of the Web experience of your visitors. Without it, your Web site won't seem complete.

A fully functioning Web site is just the beginning. It needs to be maintained to reduce the risk of failure and monitored to help detect potential problems. Chapter 22, discusses many Linux-based tools that you can be use to track the health of your Linux server.