Overview of Email Delivery

Discussion

An Introduction to Email Delivery

Email service is amongst the oldest network applications commonly used on the
internet, and still remains one of the most important. People (rightfully) expect to
effortlessly compose email, and send it on its way at the click of a mouse. Behind the
scenes, however, email depends on many applications for email queuing, transport
(including retries on transient errors), and more recently, content filtering.

Traditional Configuration

Traditionally, on Unix systems, email is handled by a collection of independent

applications whose roles are identified by TLA's (Three Letter Acronyms). The diagram
below attempts to present the participants together in context with one another.

Figure 1. Overview of Email Delivery

The roles of the various participants, the names of some of the applications that fill the
particular roles, and a summary of the jobs they perform, are found in the following
table.

Table 1. Email Applications

TLA Name Analogy RHEL Role

Like a post office, the MTA receives mail
Mail from a variety of sources, and attempts to
Post sendmail,
MTA Transport deliver the mail to the appropriate
Office postfix, exim
Agent recipient, based on the destination address.
Remote mail is received over over TCP
TLA Name Analogy RHEL Role
port 25, while local mail is received from
local MUA's. Remote mail is delivered
based on a DNS MX lookup on the
destination address, by connecting to port
25 of the remote machine. Local mail is
delivered to a MDA on the local machine.
Like a postal carrier, an MTA receives mail
that is destined for local delivery, and
Mail Local performs the last step of delivering the mail
MDA Delivery Postal procmail to a mailbox (usually
Agent Carrier /var/spool/mail/$USER). Unlike
traditional postal carriers, an MTA might
filter the mail along the way.
mutt,
A MUA retrieves mail for the local mail
evolution
Mail User box, and presents it to the user. MUA's also
MUA Mailbox (Desktop),
Agent transfer mail composed by the user to a
thunderbird
MTA for delivery.
(Desktop)

Mailbox Servers

We have presented the various participants in email delivery as they evolved

traditionally. For this scheme to work, however, the MTA must run as a persistent
network service, always ready to receive new email over TCP port 25. In many
situations, users would like to manage email on a machine that does not have a
permanent connection to the internet, at least not one that is permanently routable.
Therefore, a new participant has grown in importance: the Mail Access Agent, or MAA.

Figure 1. Overview of Email Delivery with Mailbox Server

Machines that for whatever reason do not have a permanent connection to the internet
often rely on a remote server for email reception. Often informally called a mailbox
server, or POP server, or IMAP server, the remote server consistently runs an MTA,
ready to receive incoming mail. The received mail is then stored in a mailbox on the
remote server, and as far as the mail transport protocols are concerned, the email is
considered delivered.

Eventually, however, the user on the local machine will want to access their mail
queued on the mailbox server. For this purpose, the remote server also runs a MAA,
such as a POP server or an IMAP server, which allows MUA's on client machines to
retrieve their mail, at the client's convenience.
This relationship is common to home computer users. An Internet Service Provider
(ISP) provides home users with an email address which routes to the ISP's Mailbox
Server, on which the ISP runs both an MTA and a MAA. The MTA receives users' mail,
and stores it in the appropriate mail spool on the server. At their convenience, home
users can then access their mailbox through the MAA, commonly known as "popping"
their mail. Mailbox servers provided by ISP's often have names such as
<hostname>pop.isp.net</hostname>.

SMTP Servers

Often, for a variety of reasons, organizations and ISPs prefer to centralize email
delivery as well. Rather than having individual client machines run their own MTA, an
outgoing mail server is provided. Client MUA's are then configured to deliver outgoing
mail to the remote MTA directly.

Figure 1. Overview of Email Delivery with Mailbox Server and SMTP Host

ISP's often like to use outgoing mail servers, because it allows them to monitor and
possibly filter outgoing mail. To help control abusers of email service (SPAMers), ISPs
often block outgoing port 25 from their customers' machines, so that their customers
must use the outgoing mail server provided by the ISP. Often, outgoing mail servers
provided by ISPs are given names such as <hostname>smtp.isp.net</hostname>.

Other organizations might prefer centralized outgoing mail servers so as not to expose
the configuration of their internal network, as well as providing a central point to
monitor outgoing mail.

As the diagram above illustrates, the combination of a mailbox server and an outgoing
mail server greatly simplifies the role of the client machine. Often, configuration of the
client MUA consists merely of the hostname of the mailbox server and the user's
account information, and the hostname of the outgoing mail server.

Email in Red Hat Enterprise Linux

As the table above illustrates, there are often several different applications that can fill a
particular role. For example, Red Hat Enterprise Linux includes sendmail and postfix,
and exim, all of which are full featured MTAs. The procmail utility tends to be the only
MDA, but in some configurations you can choose to use it or not. Two products act as
MAA's: dovecot, for smaller installations, and cyrus-imapd, for larger installations
running dedicated mailbox servers (such as ISP's). Many applications provide MUA's,
such as the graphically based evolution and thuderbird, both found in the Desktop
version of Red Hat Enterprise Linux, and the terminal based (but every bit as
sophisticated) mutt, which is found in both the Desktop and Server editions.

For our purposes, we will focus on installing and configuring the sendmail MTA,
dovecot MAA, and use the mutt MUA in our examples.

Installing and Configuring Sendmail

The sendmail daemon is provided by the sendmail package, which is usually installed
by default. (If the postfix or exim packages are also installed, they should be removed
so as not to complicate the issue). As we will see, configuration of sendmail often relies
on an outside utility known as m4. The sendmail-cf package contains m4 templates, and
is usually not installed by default. It should be installed as well.

Once both thesendmail and sendmail-cf packages have been installed, the
<service>sendmail</service> service can be managed using the traditional service and
chkconfig commands, although <service>sendmail</service> is one of the few
networking applications which is installed enabled by default, so the service is probably
already started.

[root@station ~]# rpm -q sendmail sendmail-cf

sendmail-8.13.8-2.el5
package sendmail-cf is not installed
[root@station ~]# yum install sendmail-cf
...
======================================================================
=======
Package Arch Version Repository
Size
======================================================================
=======
Installing:
sendmail-cf i386 8.13.8-2.el5 rha-rhel
311 k
...
Installed: sendmail-cf.i386 0:8.13.8-2.el5
Complete!
[root@station ~]# service sendmail status
sendmail (pid 2451 2445) is running...
[root@station ~]# chkconfig --list sendmail
sendmail 0:off 1:off 2:on 3:on 4:on 5:on 6:off

Configuring Sendmail to Receive External Connections

While the <service>sendmail</service> service is started by default, the sendmail

daemon is configured to bind only to the loopback address, 127.0.0.1. The following
netstat command confirms this.

[root@station ~]# netstat -tuna | grep LISTEN | grep 25

tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
Recall that the first address is the "local" address, the only relevant address for a
LISTENing socket.
If sendmail is to receive mail from remote MTA's, it must bind to external interfaces as
well. While this common configuration sounds simple enough, the steps are more
involved than they should be. Because it's important, and because it's not obvious, we're
going to mention how to configure sendmail to bind to external interfaces several
times. Here's the first time.

1. Ensure that the sendmail-cf package is installed.

2. With a text editor, open the file /etc/mail/sendmail.mc, and search for the
text "127". You should soon discover the following line.

Figure 1. /etc/mail/sendmail.mc: <directive>Binding

Address</directive>

111 dnl #
dnl # The following causes sendmail to only listen on the
IPv4 loopback address
dnl # 127.0.0.1 and not on any other network devices.
Remove the loopback
dnl # address restriction to accept email from the
internet or intranet.
dnl #
116 DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl

Comment out the line starting DAEMON_OPTIONS, by prepending the text dnl, as
in the previous lines. (dnl? Don't ask. Just do it. We'll cover the syntax of this
file a little later.)

3. Restart the <service>sendmail</service> service.

If all went well, you should be able to confirm that sendmail has bound to port 25,
using all interfaces.

[root@station mail]# netstat -tuna | grep LISTEN | grep 25

tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN
The sendmail daemon is now bound to all IP interfaces.

The SMTP Protocol (Optional, but Interesting)

One of the more common protocols for delivering email between clients and MTA's, or
MTA's and MTA's, is the SMTP protocol. In order to illustrate the SMTP protocol, try
starting the wireshark network analyzer (provided by the wireshark-gnome package),
and capture packets over any interface, filtering on the port 25.

Figure 1. Using wireshark to capture a SMTP session

Next, as a user on the local machine, try sending mail to some remote recipient.
(Unfortunately, it could be the case that your local network configuration does not allow
remote email delivery.)

[elvis@academy ~]$ cal 10 1066 | mail -s "norman invasion"

spam@redhat.com

In wireshark, you should witness the TCP conversation between the client and the
server. In fact, you should witness two conversations. Right click on any packet in the
second TCP conversation, and choose Follow TCP Stream.... You should be able to
assemble a conversation between the SMTP client and the SMTP server similar to the
following.

220 mx1.redhat.com ESMTP Sendmail 8.11.6/8.11.6; Tue, 10 Jan 2006

14:55:04 -0500
EHLO academy.redhat.com
250-mx1.redhat.com Hello academy.redhat.com [172.16.62.55], pleased to
meet you
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-SIZE
250-DSN
250-ONEX
250-ETRN
250-XUSR
250 HELP
MAIL From:<elvis@academy.redhat.com> SIZE=696
250 2.1.0 <elvis@academy.redhat.com>... Sender ok
RCPT To:<spam@redhat.com>
250 2.1.5 <spam@redhat.com>... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
Received: from academy.redhat.com (localhost.localdomain [127.0.0.1])
by academy.redhat.com (8.13.1/8.13.1) with ESMTP id k2AJt3BQ030673
for <spam@redhat.com>; Tue, 10 Jan 2006 14:55:03 -0500
Received: (from elvis@localhost)
by academy.redhat.com (8.13.1/8.13.1/Submit) id k2AJt3ls030672
for spam@redhat.com; Tue, 10 Jan 2006 14:55:03 -0500
Date: Tue, 10 Jan 2006 14:55:03 -0500
From: elvis@academy.redhat.com
Message-Id: <l200601101955.k2AJt3ls030672@academy.redhat.com>
To: spam@redhat.com
Subject: norman invasion
October 1066
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31

.
250 2.0.0 k2AJt4100363 Message accepted for delivery
QUIT
221 2.0.0 mx1.redhat.com closing connection
The server starts the conversation by introducing itself, including it's version and
what protocol it speaks (ESMTP, or extended SMTP).
The client starts out with a EHLO, or extended hello, and includes its hostname.
Characteristic of many wizened elders, sendmail is very polite.
As sendmail lists its of supported extensions, note the convention for multi-line
responses: following the status code (which starts every line), a hyphen ("-") marks
an expected continuation, with the last line containing a space instead of a hyphen.
The client now identifies the sender. with the FROM command. The following SIZE
specification is optional. Although, based on the identity of the sender, the SMTP
server could choose to refuse service, in this case it doesn't.
The client now identifies the recipient with the RCPT command. Again, the SMTP
server could refuse service, but in this case, doesn't. Multiple recipients can be
defined with multiple RCPT lines.

Notice that, as far as the SMTP server is concerned, the <directive>To:</directive>

(and <directive>Cc:</directive> and <directive>Bcc:</directive>) lines in the
message body are irrelevant. The sender and recipients are instead defined in what
is referred to as the envelope, the FROM and RECV lines of the SMTP protocol. (In
theory, of course, the client deduces the envelope information from the relevant
email headers).
The client specifies the start of the message body with the DATA command. The
client the provides the email headers and body.
The end of the message is marked by a line containing a single dot.
 Finally, with QUIT, the client requests to close the connection.

The preceding dialog exemplifies what happens when things go right. The following
exemplifies when something goes wrong.

220 mx1.redhat.com ESMTP Sendmail 8.11.6/8.11.6; Tue, 10 Jan 2006

14:53:28 -0500
EHLO academy.redhat.com
250-mx1.redhat.com Hello academy.redhat.com [172.16.62.55], pleased to
meet you
250-ENHANCEDSTATUSCODES
...
250-XUSR
250 HELP
MAIL From:<elvis@academy.redhat.com> SIZE=693
250 2.1.0 <elvis@academy.redhat.com>... Sender ok
RCPT To:<devnull@redhat.com>
550 5.1.1 <devnull@redhat.com>... User unknown
RSET
250 2.0.0 Reset state
QUIT
221 2.0.0 mx1.redhat.com closing connection
In this case, the receiving SMTP server decided that the mail was destined for local
delivery, but did not recognize the recipient, so the connection was refused.

To conclude, we list the required client commands, in the order in which they should
appear in a successful SMTP conversation.

Table 1. Summary of Selected SMTP Protocol Client Commands

Command Explanation
EHLO
client_hostname Used to initiate a connection.
MAIL From:
sender_address Request a new email to be sent
RCPT To:
recipient_address Identify recipient (the "envelope")
Marks start of email message. Though not part of the SMTP
protocol, the message should consist of a series of "headers",
DATA
followed by a blank line, followed by the "body". (The internal
anatomy of an email message is detailed in RFC 822, amongst
others.). The SMTP protocol does mandate that the message
terminate with a line containing a single dot (".").
QUIT Request a close to the connection.

Exercises

Lab Exercise
 Objective: Initial setup of Sendmail

Estimated Time: 10 mins.

Specification

This lab will have you perform a fairly simple but important procedure-- configuring the
sendmail MTA to listen on all interfaces. Since sendmail's default behavior is to listen
only on the localhost interface it can be used to send mail, but not recieve mail from the
outside world.

1. First, you will need to ensure that the sendmail and sendmail-cf packages are
installed. If they are not, install them. This will install the sendmail daemon and
provide the extra tools needed to configure it.
2. You can now begin configuring sendmail by opening the
/etc/mail/sendmail.mc file in your text editor.
3. Find the line that tells sendmail to listen only on the localhost interface. This can
be made easier by searching for localhost's ip: 127.0.0.1.
4. Once you have found the appropriate line, comment it out by prepending the
string dnl.
5. Sendmail's configuration has now been updated, but it doesn't yet know about
the change. Reload sendmail's configuration by running service sendmail
reload. Note that the advantage of this over service sendmail restart is that it
does not interrupt service by taking down the daemon, even momentarily. If this
command fails, it may be because you have just installed sendmail and it is not
currently running (after all, you can't reload the configuration of a service that
isn't running. If this is the case, start the daemon with service sendmail start
6. You can test whether or not sendmail has been correctly configured by running
'netstat -tlpn' as root.
7. [root@station]# netstat -tlpn
8. tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN
22475/sendmail: acc

Deliverables

1. 1. An instance of the sendmail service listening on all network interfaces.

Managing Sendmail

While the main configuration for sendmail is /etc/mail/sendmail.cf, the daemon

also uses several databases for configuration information. In this workbook, we look at
aspects of the sendmail daemon which can be configured by modifying these database
files, defering the main configuration file until the subsequent workbook.

The /etc/aliases File

One of the most common and simplest configurations is creating email aliases. Once
sendmail has determined that email is destined for the local machine, it examines a list
of email aliases, which can be used to route the mail to different recipients on the local
machine, on remote machines, or pass the mail off to an arbitrary program.

Email aliases are maintained in the file /etc/aliases, which is backed up by a hashed
database file /etc/aliases.db.

[root@localhost ~]# file /etc/aliases*

/etc/aliases: ASCII English text
/etc/aliases.db: Berkeley DB (Hash, version 8, native byte-order)

The /etc/aliases file is a simple text file which maps a recipient email address to one
or more aliased recipients. For example, the RFC's which dictate email delivery on the
internet mandate that every MTA must support the two recipient addresses
<username>postmaster</username> and <username>mailer-daemon</username>. As
seen at the top of the /etc/aliases file, this requirement is easily accommodated
through aliases.

Figure 1. /etc/alises

#
# Aliases in this file will NOT be expanded in the header from
# Mail, but WILL be visible over networks or from /bin/mail.
#
5 # >>>>>>>>>> The program "newaliases" must be run
after
# >> NOTE >> this file is updated for any changes to
# >>>>>>>>>> show through to sendmail.
#

10 # Basic system aliases -- these MUST be present.

mailer-daemon: postmaster
postmaster: root

The recipient <username>mailer-daemon</username> gets aliased to

<username>postmaster</username>, which in turn gets aliased to
<username>root</username>. As a result, <username>root</username> receives the
email addressed to either of them. In the following, a simple mail message is sent to
<username>postmaster</username>, but the mail ends up spooled in
<username>root</username>'s inbox (/var/spool/mail/root).

[root@station mail]# cal 10 1067 | mail -s "Norman Invasion Cleanup"

postmaster@localhost
You have mail in /var/spool/mail/root
[root@station mail]# tail /var/spool/mail/root
Subject: Norman Invasion Cleanup

October 1067
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6
7 8 9 10 11 12 13
...
Notice that aliases go through multiple rounds of resolution. The alias
<username>postmaster</username> was again aliased to <username>root</username>.
This also implies that if you are not careful, alias loops can occur. (What would happen
if <username>postmaster</username> had instead been aliased to <username>mailer-
daemon</username>?)

Returning our attention to the /etc/aliases file, we find that

<username>root</username> is a popular recipient. Almost all of the system user
accounts are aliased to root.

Figure 2. /etc/alises

# General redirections for pseudo accounts.

15 bin: root
daemon: root
adm: root
lp: root
...
toor: root
70 manager: root
dumper: root
abuse: root

A few commonly supported email recipients are aliased instead to postmaster.

Figure 3. /etc/alises

90 info: postmaster
marketing: postmaster
sales: postmaster
support: postmaster

And lastly, another local user can be used to monitor root's email (without having to log
in as root), although this is commented out by default.

Figure 4. /etc/alises

# Person who should get root's mail

100 #root: marc

What types of aliases can be added? Here's some examples.

debra: blondie
wizards: elvis, hogan, linus@kernel.org
memo-list: |/usr/local/bin/mailing_list_manager.sh
log: /var/local/local_log
A simple alias, which aliases one name to another.
An alias that expands to a comma separated list of recipients. Notice that an alias
can resolve to a remote email address, as well.
Aliases can be used to deliver the email to standard in of some program, which is
meant to somehow filter the mail. [1]
An alias can refer directly to a file (which must be specified as an absolute
 reference).

As the note at the top of /etc/aliases implies, after editing, the hashed database file
must be updated by running the command newaliases, which reports how many aliases
were hashed.

[root@localhost ~]# newaliases

/etc/aliases: 76 aliases, longest 10 bytes, 765 bytes total

Why the complexity of a hashed database? Speed. It is not unheard of for an

organization to have thousands of aliases in the /etc/aliases file of the organization's
main mail server. The sendmail daemon can resolve aliases much more quickly using
the hashed database than by performing a linear search though the text file.

The Pending Mail Queue: mailq

When the sendmail MTA has trouble delivering mail to a recipient MTA, it classifies
the problem as either a temporary or a permanent problem. Permanent problems, such a
recipient MTA that replies that the intended user does not exist, are reported directly to
the sender via a email message from "mailer-daemon".

Problems considered temporary, such as not being able to contact the recipient MTA,
are handled differently: the email is kept in a queue of email pending delivery, and
every once in a while, sendmail will attempt to redeliver the mail. (The recipient
machine could have just been rebooting, or an intermediate router could have been
temporarily misconfigured).

For example, consider the following email sent to the host

<hostname>figment.example.com</hostname>, which the local nameserver has mapped
to the IP address 10.3.3.3. However, the local (private) network 10.0.0.0/24 does not
exist!

[root@station ~]# host figment.example.com

figment.example.com has address 10.3.3.3
[root@station ~]# ping figment.example.com
connect: Network is unreachable

The simple non-existence of a host does not deter us from sending mail to the host's
administrator, however.

[root@station ~]# date | mail -s "example" root@figment.example.com

Because sendmail is not able to contact the recipient MTA, it queues the mail in the
directory /var/spool/mqueue. The contents of the pending mail queue can always be
examined directly with ls, or more conveniently with the command mailq.

[root@station ~]# ls /var/spool/mqueue/

dfj85Cl4lv016992 qfj85Cl4lv016992
[root@station ~]# mailq
/var/spool/mqueue (1 request)
-----Q-ID----- --Size-- -----Q-Time----- ------------Sender/Recipient-
----------
j85Cl4lv016992 29 Mon Jul 4 08:47 <root@localhost.localdomain>
(Deferred: figment.example.com.: Network is
unreachable)
<root@figment.example.com>
Total requests: 1

Because sendmail doesn't know any better (the recipient MTA could just be suffering
from a temporary power outage), it will continue retrying to send the mail once per
hour. After four hours, sendmail will send a warning back to the sender, stating
essentially "I've been trying for four hours to send this mail, and have been having
trouble. You don't have to do anything, I'll keep trying, but I wanted to let you know".
After five days, sendmail will give up, and return the email to the sender, with the
message "I've been trying for 5 days, and I give up. You should consider this mail
undeliverable." Of course, the actual times (4 hours and 5 days) are configurable, and
will vary from site to site.

Sometimes, the problem is local, where due to a networking misconfiguration, email

can be received but not delivered. In this case, pending mail will begin collecting in the
/var/spool/mqueue directory. If the problem has been resolved, an administrator can
ask sendmail to retry delivery of pending mail by invoking sendmail -q, or simply
restarting the service.

In the following, we use the magic of IP aliases to miraculously make the host
<hostname>figment.example.com</hostname> appear, and ask sendmail to attempt to
resend pending mail. This time, our MTA is able to contact the recipient MTA, and the
mail is delivered. [1]

[root@station ~]# ifconfig eth0:1 10.3.3.3

[root@station ~]# sendmail -q
[root@station ~]# mailq
/var/spool/mqueue is empty
Total requests: 0

The observant might notice that this section introduced nothing to configure. If you feel
you must reconfigure something, you might change how often sendmail attempts to
resend pending mail, by editing the QUEUE variable in /etc/sysconfig/sendmail, and
restarting the <service>sendmail</service> service.

[root@station ~]# cat /etc/sysconfig/sendmail

DAEMON=yes
QUEUE=1h

The other two paramters are configured in /etc/sendmail/sendmail.mc, the topic of

our next lesson.

Email Logging

Hopefully, by this point in the course, administrators know the importance of keeping
an eye on log files such as /var/log/messages. For sendmail, this holds true as well,
with mail related logging dispatched to the /var/log/maillog log file. For example,
the following log message documents the delivery of mail to the user
<username>elvis</username>.
[root@station ~]# date | mail -s "another example" elvis@localhost
[root@station ~]# tail /var/log/maillog
...
Jul 4 09:30:18 localhost sendmail[17962]: j85DUIb5017962: from=root,
size=75, c
lass=0, nrcpts=1,
msgid=<200509051330.j85DUIb5017962@localhost.localdomain>, rel
ay=root@localhost
Jul 4 09:30:19 localhost sendmail[17963]: j85DUI3L017963:
from=<root@localhost.
localdomain>, size=380, class=0, nrcpts=1,
msgid=<200509051330.j85DUIb5017962@lo
calhost.localdomain>, proto=ESMTP, daemon=Daemon0,
relay=localhost.localdomain [
127.0.0.1]
Jul 4 09:30:19 localhost sendmail[17962]: j85DUIb5017962:
to=elvis@localhost, c
tladdr=root (0/0), delay=00:00:01, xdelay=00:00:01, mailer=relay,
pri=30075, rel
ay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (j85DUI3L017963
Message accepte
d for delivery)
Jul 4 09:30:19 localhost sendmail[17964]: j85DUI3L017963:
to=<elvis@localhost.l
ocaldomain>, ctladdr=<root@localhost.localdomain> (0/0),
delay=00:00:01, xdelay=
00:00:00, mailer=local, pri=30617, dsn=2.0.0, stat=Sent

Within a graphical environment, a command akin to the following can go a long way
towards knowing what sendmail is up to.

[root@station ~]# xterm -e tail -f /var/log/maillog &

Relaying Mail

Recall that the role of the MTA is akin to the role of the post office. The MTA receives
incoming mail, either from the network over port tcp/25, or from local users, and
attempts to deliver the email to the appropriate destination, either by connecting to a
remote MTA over port tcp/25, or by delivering the mail to a local user.

Whenever sendmail receives email from the network, and decides that the mail is not
destined to a local user, but should instead be routed to a remote MTA, it is said to be
relaying the mail. Generally, sendmail can take one of the following approaches
regarding email relaying.

1. Open Relay: An open relay will receive mail from any source, and forward it to
any destination (much like your local postoffice). When sendmail was being
developed, and throughout the early 1990's, an open relay was considered robust
design, and was commonly used.
2. Closed Relay: As the internet has gained in popularity, people have used open
relays to obscure the origins of mass mailings (SPAM). As a result, much
attention has been focused on curtailing who can relay mail from a given SMTP
server. A closed relay refuses to relay mail from anyone, and will only accept
mail destined to local users. This is the default Red Hat Enterprise Linux
configuration.
 3. Mostly Closed Relay: In some situations, a SMTP server needs to be willing to
relay for select clients. Consider an outgoing SMTP server, which recieves mail
from local machines, and forwards the mail out to the internet. This machine
must be willing to relay mail originating from local machines. Relaying can be
selectively enabled by modifying the /etc/mail/access database.

By default in Red Hat Enterprise Linux, sendmail acts as a closed relay. Configuring
sendmail to relay for select clients, however, is as easy as adding entries to the access
database file /etc/mail/access. The access database serves many purposes, and is
discussed in detail in the file /usr/share/sendmail-cf/README. For our purposes, we
will focus only on the task at hand: enabling relaying for specified clients.

The access database is a line based configuration file. The first token should specify a
client's IP address, either directly or using a network address, and the second token
should simply be the word <directive>RELAY</directive> (as opposed to
<directive>OK</directive>, <directive>REJECT</directive>, or other qualifiers related
to access control. See the README file for more details.) The following line would cause
sendmail to relay mail for all hosts in the 192.168.0.0/255.255.255.0 subnet.

192.168.0 RELAY

Notice that the access database (and several other databases within the /etc/mail
directory) is backed up by a hashed database file access.db (much like the
/etc/aliases is backed up by /etc/aliases.db).

[root@station ~]# ls /etc/mail

access local-host-names sendmail.mc virtusertable
access.db mailertable spamassassin virtusertable.db
domaintable mailertable.db submit.cf
domaintable.db Makefile submit.mc
helpfile sendmail.cf trusted-users

While there is not a custom command to rebuild the hashed database access.db, there
is a Makefile in the /etc/mail directory, which contains the proper commands for
rebuilding the database. All databases in this directory can be rebuilt by cd'ing into the
/etc/mail directory, and running the command make.

[root@station1 ~]# cd /etc/mail

[root@station1 mail]# make

Although make runs silently, you should be able to convince yourself that the
access.db database file was updated by comparing the modify times on the files
access and access.db using ls -l.

Dynamic Databases
Because sendmail examines the hashed databse files with each new request, once
a database file has been updated, the <service>sendmail</service> service does
not need to be restarted. However, restarting the service will automatically update
any database files which require it, and never hurts.
The make Command
 The make command is used to solve a very general problem: "This file depends
on that one. Whenever that file is changed, you need to update this one, and here's
how to do it." The make command was desgined with programming in mind,
where object files (such as foo.o) need to be recompiled whenever source files
(such as foo.c) are changed. Because of the generalness of its design, however, it
finds many uses, such as the case at hand: hashed database files depend on "flat"
text files.

Although a full discussion of the make command is beyond our scope, it's often
enough to know that the knowledge required to update the dependent file (the
"here's how to do it") is configured using recipies defined in a local Makefile.
Whenever run, the make command examines the local Makefile to learn which
files are of interest, and then compares the modify times of these files. If the
independent file (in our case, access) has a more recent modify time than the
dependent file (for us, access.db), then make follows the recipe to update the
dependent file.

The upshot: make is context dependent, and must be invoked from within the
appropriate directory (i.e., a directory containing a Makefile). With a well written
Makefile, however, it's as simple as that.

Virtual Users: /etc/mail/virtusertable

Suppose that you have a mail server configured to serve more than one domain. This is
as simple as designating that server as the Mail eXchanger for multiple domains in DNS
and adding both domains to the server's local-host-names file. However, once you
start dealing with multiple domains on the same mail server, aliases as defined in
/etc/aliases can become problematic. For example, suppose your mail server handles
incoming mail for the example.com and example.org domains. Suppose that it also has
an alias defined that forwards all of root's email to the user joe. Notice that the alias
does not differentiate between root@example.com and root@example.org. Any locally-
destined message where the destination address begins with 'root@' will be sent to joe.
This can be problematic if example.com and example.org are run by different
administrators who want different aliases for root. To accomplish this we will need to
introduce a new type of alias called a virtual user. The function of a virtual user is the
same as that of a regular alias, except that virtual users take the destination domain into
account as well as the user name.

Virtual users are defined in the /etc/mail/virtusertable file. Each line represents a
virtual user and the syntax of each line defines the specific behavior of that entry. Below
are some examples:

root@example.com joe
root@example.org jane@janesdomain.com
@example.net %1@example.com
@example.biz sales@example.com
Send messages destined for root@example.com to a local mailbox
Send messages destined for root@example.org to an external address.
Send emails destined for anyone at example.net to the corresponding example.com
 user. "%1" represents whatever is to the left of the '@' in an email address. So an
email destined for foo@example.net would be delivered to foo@example.com
instead.
Similar to the previous rule, but here we are forwarding all example.biz addresses
to a single example.com address.

As usual, sendmail does not read virtusertable directly. Whenever the daemon is
reloaded or restarted a virtusertable.db file is generated, so be sure to run
service sendmail reload after making changes, or run make from within the
/etc/mail directory. Exercises

Lab Exercise
Objective: Configure sendmail aliases

Estimated Time: 10 mins.

Specification

There is a lot of important information that gets emailed to the root user. For example, a
summary of all of the previous day's log entries is sent early every morning. But since
you've been following good security practices and doing most of your Linux work as a
non-root user (right?) it would be easier to have these emails sent to one or more
personal accounts instead.

In this lab you will create an alias that sends any message destined to your system's root
account to the mailboxes of two non-root users instead.

1. Open the /etc/alisases file in your text editor

2. Find the line at the bottom that forwards root's mail. It should be commented out
by default.
3. # Person who should get root's mail
4. #root: marc
5. Uncomment this line and alter it to forward root's mail to your primary and
secondary accounts.

root: username, username_a

6. Save the file and exit. Run ls -lt /etc/aliases* and compare the
timestamps on aliases and aliases.db. Note that just altering aliases does
not update aliases.db, which is what the MTA actually reads.
7. In order to make the changes effective, tell sendmail to refresh and reread its
configuration files by running service sendmail reload
8. Run ls -lt /etc/aliases* again. You should see that the timestamp on
aliases.db is now newer than (or equal to if you're fast enough) that of
aliases. This indicates that the file has been regenerated.
9. Test your configuration by sending an email to root. A command like echo
"foo" | mail -s "bar" root would accomplish this. You can verify that the
emails have reached their intended recipients by running a mail client (such as
 mutt) as each user or by looking directly at each user's mail spool in
/var/spool/mail/.

Deliverables

1. 1. An email alias that forwards all mail destined for the root user to your primary
and secondary accounts.

Configuring Sendmail

Sendmail is one of the oldest network services, and though its design has withstood the
test of time, it has the reputation of being crufty, cantankerous, and difficult to
configure. As we begin to examine its primary configuration file,
/etc/mail/sendmail.cf, we will discover why.

The Venerable /etc/mail/sendmail.cf

Upon startup, sendmail reads /etc/mail/sendmail.cf for configuration. We'll

examine the file as well, not with the intention of learning how to configure it, but
instead with the intention of learning why we don't configure it directly.

Figure 1. /etc/mail/sendmail.cf

######################################################################
#####
25 ##### DO NOT EDIT THIS FILE! Only edit the source .mc file.
#####

######################################################################

######################################################################

30 ##### $Id: 020_m4.dbk,v 1.5 2006/03/12 11:10:40 bowe Exp $

#####
##### $Id: 020_m4.dbk,v 1.5 2006/03/12 11:10:40 bowe Exp $
#####
##### setup for Red Hat Linux #####
##### $Id: 020_m4.dbk,v 1.5 2006/03/12 11:10:40 bowe Exp $
#####

35

##### $Id: 020_m4.dbk,v 1.5 2006/03/12 11:10:40 bowe Exp $

#####

To begin with, the configuration file doesn't seem too bad. It actually contains words.
Of course, all of these words are comments. And some of those comments are telling us
that the file should not be edited directly.

Figure 2. /etc/mail/sendmail.cf: Variables

##################
 # local info #
##################

85 # my LDAP cluster
# need to set this before any LDAP lookups are done (including
classes)
#D{sendmailMTACluster}$m

Cwlocalhost
90 # file containing names of hosts for which we receive email
Fw/etc/mail/local-host-names

# my official domain name

# ... define this only if sendmail cannot automatically
determine your domain
95 #Dj$w.Foo.COM

As we look further, we start go get an uneasy feeling as we realize that lines such as
Dj$w.Foo.COM are single letter commands ("D") operating on single letter variables
("j") using single letter substitutions ($w). But, of course, there's a man page, right?

[root@station ~]# man sendmail.cf

No manual entry for sendmail.cf

And we haven't even mentioned the rewriting rules, which have been compared to
terminal noise.

Figure 3. /etc/mail/sendmail.cf: Rewriting Rules

############################################
### Ruleset 3 -- Name Canonicalization ###
625 ############################################
Scanonify=3

# handle null input (translate to <@> special case)

R$@ $@ <@>
630
# strip group: syntax (not inside angle brackets!) and trailing
semicolon
R$* $: $1 <@> mark
addresses
R$* < $* > $* <@> $: $1 < $2 > $3 unmark <addr>
R@ $* <@> $: @ $1 unmark
@host:...
635 R$* [ IPv6 : $+ ] <@> $: $1 [ IPv6 : $2 ] unmark IPv6
addr
R$* :: $* <@> $: $1 :: $2 unmark
node::addr
R:include: $* <@> $: :include: $1 unmark
:include:...
R$* : $* [ $* ] $: $1 : $2 [ $3 ] <@> remark if
leading colon
R$* : $* <@> $: $2 strip colon
if marked
640 R$* <@> $: $1 unmark
R$* ; $1 strip
trailing semi
 R$* < $+ :; > $* $@ $2 :; <@> catch
<list:;>
R$* < $* ; > $1 < $2 > bogus
bracketed semi

The sendmail.mc Configuration File Template

There are two bits of good news with regard to configuring sendmail.

The first piece of good news is that sendmail's default configuration sends mail in most
contexts with almost no configuration. Even configuring sendmail to serve as the
receiving mail server for an entire domain (also known as the domain's Mail eXchanger
or MX) does not require much configuration.

The second piece of good news: when you do need to configure sendmail, you don't
actually have to deal with sendmail.cf directly. That's where sendmail.mc comes in.
Though sendmail.mc is still not the most user-friendly configuration file you're likely
to see on a Red Hat Enterprise Linux system, it is more concise and less cryptic than
sendmail.cf can be. When an instruction in sendmail.mc is run through the m4 macro
interpreter, it is expanded into an equivalent, but generally much longer, set of
sendmail.cf directives. To illustrate this, one only has to compare the number of lines
in each file:

[root@station]# wc -l sendmail.*
1821 sendmail.cf
173 sendmail.mc

The default sendmail.mc file was used to generate the default sendmail.cf file, as can
be observed with the following (assuming neither file has been edited).

[root@station ~]# cd /etc/mail

[root@station mail]# m4 sendmail.mc > sendmail.cf.test
[root@station mail]# diff sendmail.cf sendmail.cf.test
19,21c19,21
< ##### built by bhcompile@porky.build.redhat.com on Wed Sep 1
06:16:23 EDT 2004
< ##### in /usr/src/build/446503-i386/BUILD/sendmail-8.13.1/cf/cf
< ##### using ../ as configuration include directory
---
> ##### built by root@station.example.com on Mon Nov 14 16:42:38 EST
2005
> ##### in /etc/mail
> ##### using /usr/share/sendmail-cf/ as configuration include
directory

Out of over 1800 lines, only three lines differ, containing comments about when the file
was built.

Every time a change is made to sendmail.mc, it has to be interpreted into a new

sendmail.cf before the changes will be seen by sendmail. Though this can be done by
calling m4 directly and sending its output into sendmail.cf, the safest and easiest way
to regenerate sendmail's configuration is to run service sendmail reload. This not
only refreshes sendmail.cf, but any other sendmail configuration files that have been
altered and need to be transformed (such as /etc/aliases). It also creates a backup of
the previous configuration in sendmail.cf-bak.

The varous macros used in the sendmail.mc file, and a little introduction to m4 syntax,
are documented in the lengthy /usr/share/sendmail-cf/README text file, provided
by the sendmail-cf package.

[root@station ~]# wc /usr/share/sendmail-cf/README

4590 25708 183289 /usr/share/sendmail-cf/README

Glancing through the documentation, we find a wealth of capabilities, including TLS

(SSL) encryption and authentication, LDAP based databases, real-time blacklists for
vigilante SPAM prevention, and more.

To illustrate the process of using m4 to configure sendmail, we will examine a fairly

common scenerio: configuring an organization so that all local versions of sendmail
forward mail to an outgoing SMTP server.

Configuring sendmail to Use an Outgoing SMTP Server

Many times, organizations would like all mail to be routed through a single SMTP
server. To understand why, first consider the following scenerio, using the organization
<hostname>example.com</hostname>. Each employee has an individual workstation,
named something bland such as <hostname>stationx.example.com</hostname>. In the
simplest setup (considering the default Red Hat Enterprise Linux configuration), each
workstation would run independently configured sendmail daemons, each of which
would initiate connections to remote MTA's in order to deliver mail to, say, shadowman
at Red Hat, Inc.

Figure 1. "Simple" Outgoing Email Configuration for

<hostname>example.com</hostname>
In the diagram, we are assuming all hosts in <hostname>example.com</hostname> are
using sendmail as the MTA, running as a daemon bound to port tcp/25. When actually
delivering mail, however, a separate instance of sendmail is started, openning a random
TCP client port.

The diagram makes no assumptions about the destination's MTA, instead referring to it
as smtpd, implying only a daemon that speaks the SMTP protocol. For Red Hat, the
actual MTA is, of course, sendmail, though other sites could easily be running other
MTA's, such as Microsoft Exchange.

A popular alternative is to have the <hostname>example.com</hostname> organization

deploy a centralized outgoing mailserver, named, for example,
<hostname>smtp.example.com</hostname>. Individual workstations would forward
their mail to the local SMTP server, which would then attempt to deliver mail to its final
destination.

Figure 2. Centralized Outgoing Email Configuration for

<hostname>example.com</hostname>

This scenerio has the following advantages.

1. The only host which is required to contact external MTA's is

<hostname>smtp.example.com</hostname>. As a result, the topology of the
internal network is not exposed to the outside world.
2. The organziation has a single point of control, from which outgoing mail can be
audited, monitored, and filtered.
3. The individual workstations can genarally be configured with a simple, "one
time" operation. Any changes in policy can be implemented at the outgoing
SMTP server.
We now discuss how to configure sendmail for this scenerio, looking first at the SMTP
server (<hostname>stmp.example.com</hostname>), and then at the individual
workstations.

Configuring sendmail on the SMTP server

Configuring sendmail to act as a SMTP server is relatively simple, and only involves
skills we have alredy learned.

1. The sendmail daemon must be willing to accept connections from external

interfaces. First, ensure the sendmail-cf RPM is installed, then comment out the
following line in /etc/mail/sendmail.mc (remembering that the comment
token in m4 is <directive>dnl</directive>), and restart the
<service>sendmail</service> service.

DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl

2. The sendmail daemon must be willing to relay mail received from internal
workstations. This involves adding the right rule to /etc/mail/access, and
running make in the /etc/mail directory (or simply restarting the
<service>sendmail</service> service). The following line, when added to
/etc/mail/access, would enable relaying for the 192.168.0.0/255.255.255.0
subnet.

192.168.0. RELAY

Configuring sendmail on User Workstations

Configuring the individual worktstations is not as straightforward, and requires that new
m4 macros to be added to /etc/mail/sendmail.mc. In general terms, the macros will
accomplish the following goals.

1. When delivering outgoing mail, the sendmail daemon will contact the MTA on
<hostname>smtp.example.com</hostname>, rather than trying to contact the
destination MTA directly. This involves appropriately setting the
<directive>SMART_HOST</directive> and
<directive>MAIL_HUB</directive> macros.
2. Optionally, but commonly, the MTA on the individual workstation will
masquerade as another, more generally named, host, so that mail originating
from <username>elvis@station1.example.com</username> will instead appear
to be originating from <username>elvis@example.com</username>. This
involves setting the <directive>MASQUERADE_AS</directive> macro.

The macros relevant for our scenerio are discussed about a third of the way into
/usr/share/sendmail-cf/README, in a section called MASQUERADING AND RELAYING.
The section can be found by searching for about the 20th (!) instance of the text
<directive>MASQ</directive>.

...
+---------------------------+
| MASQUERADING AND RELAYING |
+---------------------------+

You can have your host masquerade as another using

MASQUERADE_AS(`host.domain')
...

To mercifully summarize the README documentation, we begin by providing the answer.

The user workstations in our scenerio can be appropriately configured by adding the
following lines to /etc/mail/sendmail.mc, and restarting the
<service>sendmail</service> service.

define(`SMART_HOST',`smtp.example.com')
define(`MAIL_HUB',`smtp.example.com')
MASQUERADE_AS(`example.com')

We now discuss these macros in more detail.

The SMART_HOST defines the SMTP server which should handle all outgoing mail
which is destined to remote addresses, such as mail generated on
<hostname>station1.example.com</hostname> which is destined to
shadowman@redhat.com.
The MAIL_HUB defines the SMTP server which should handle all local mail, a more
subtle problem. For example, what if <hostname>example.com</hostname> were
using NIS, so that the users blondie, elvis, and prince were all defined on each
machine. The user elvis, on <hostname>station1.example.com</hostname>, might
compose mail to the user blondie, without even specifying a domain.

[elvis@station1 ~]$ mail -s "here is the data" blondie <

sample_1.txt

This works, because NIS defines blondie as a local user. Unless the
<directive>MAIL_HUB</directive> is defined, however, it probably doesn't work
as intended, because sendmail does what it thinks it's supposed to, and delivers the
mail to blondie's mail spool (/var/spool/mail/blondie) on the local machine,
<hostname>station1.example.com</hostname>. Just because blondie is defined as
a user on <hostname>station1</hostname> doesn't mean she regularly uses it. If
<hostname>station1</hostname> is on elvis's desk, some other machine, such as
<hostname>station18</hostname>, is on blondie's desk. She would never discover
mail waiting for her on station1!
Lastly, the MASQUERADE_AS macro causes all mail originating from the user
workstation to appear as if it were written from the specified domain. Without a
masquerade, when elvis composes mail on <hostname>station1</hostname>, his
<directive>From</directive> address will read elvis@station1.example.com.
The company would rather all of its employs generate mail with more general
<directive>From</directive> lines, such as elvis@example.com.

Configuring a SMTP Server

Whether you're using a central mail server or not, somewhere on your network you will
need to have at least one mail server capable of accepting outbound messages, relaying
them to the outside world, and recieving the responses. Configuring a mail server for
this task (be it Sendmail or Postfix) requires three changes to the default configuration.

1. The daemon must be configured to listen on a public interface.

2. The daemon must be willing to relay mail from your internal network to external
hosts.
3. The daemon must able to differentiate between messages that are destined for a
local email account and those that are destined for the outside world.

Configuring the daemon to listen on a public interface was discussed in the previous
chapter. To review, you will need to comment out the following line from sendmail.mc
by prepending "dnl" to it:

DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl

The next step is to configure Sendmail to relay messages to the outside world on behalf
of your internal workstations. This process is called "relaying" and a system that will
relay for any client is known as an "open relay". Because open relays are often used by
spammers to send unwanted messages, running a server like this is discouraged. Red
Hat Enterprise Linux ships both Sendmail and Postfix configured to behave as "closed
relays", meaning they will only relay messages for the system the daemon is actually
running on unless configured to do otherwise.

Our task is to configure the daemon to open up just a little and allow relaying only for
clients on our internal network. Otherwise, when internal clients attempt to send
messages through our mail server they will recieve "Relaying Denied" errors instead.
Relaying controls are stored in a file called /etc/mail/access. As the name implies,
this file controls who may do what to our mail server, including basic host-based access
controls as well as relaying. However, for the purposes of this discussion we will focus
on using it to control relaying only. The default access file contains lines like the
following:

localhost.localdomain RELAY
localhost RELAY
127.0.0.1 RELAY

In the example above, we can see three rules, each of which allows relaying for
localhost under a different name. What we need to do is include rules that allows
relaying for an entire subnet instead of a single host. Assuming that all of our systems
are within the 192.168.0.X network, we could accomplish this with the following rule:

192.168.0. RELAY

Note that the variable portion of the IP address is simply omitted. To do the same thing
by domain name instead of IP you could also add:

.example.com RELAY

When service sendmail reload is executed, the access file will be converted to a
binary database file called access.db, which is what Sendmail will actually read.
The final step in making our mail server a full-fledged domain MTA is to instruct the
daemon on how to identify locally-destined messages. Since by default Sendmail only
recognizes its own hostname as a local destination, a message destined for
joe@example.com instead of joe@mail.example.com will be assumed to be destined
elswhere (in other words, relayed). If the sender is not from our internal network, they
will recieve a "Relaying Denied" error for their trouble.

The solution lies in the /etc/mail/local-host-names file. This file simply contains a
list of hostnames. If one of the hostnames in local-host-names appears to the right of
the the '@' in an email's destination address, it will be delivered locally to the account
(or alias) specified at the beginning of that address. So by adding example.com to
local-host-names and reloading Sendmail, we are left with a server that accepts email
for our domain.

...and that's it! To recap, three changes are neccessary to Sendmail's default
configuration in Red Hat Enterprise Linux to make it act as the mail server for a
domain:

1. Comment out the DAEMON_OPTIONS line in sendmail.mc so that Sendmail

listens on all interfaces.
2. Add your internal network to /etc/mail/access to allow relaying for internal
systems.
3. Add your domain name to /etc/mail/local-host-names so that Sendmail
knows to deliver mail destined for that domain locally.

Be sure to perform a service sendmail reload when you're finished!

Exercises

Lab Exercise
Objective: Sendmail Configuration

Estimated Time: 10 mins.

Specification

For this lab you will configure your system to accept email for a virtual domain called
rha-student.net and cause all emails sent to that domain to be delivered to your mail
spool. Bear in mind that without configuring dns so that other systems know that your
server is the MX for rha-student.net, this will only work for emails sent from the server.

1. Open /etc/mail/local-host-names in your text editor and add the name of

the virtual domain, rha-student.net.
2. Open /etc/mail/virtusertable and add a line configuring Sendmail to send
all email messages addressed to users @rha-student.net to your primary account.
A line like the following would suffice:

@rha-student.net your_username
 3. That should be it! Reload Sendmail's configuration file and test by sending
emails to random @rha-student.net addresses.

Deliverables

1. 1. Emails sent to anyone @rha-student.net should end up in the mail spool of

your primary account