Set up a transactional mail server

From Wiki
Jump to: navigation, search

A transactional email is generated by many popular packages, typically for account registration confirmation, topic reply notifications, password retrieval, and other automated uses such as sending system notification messages to administrators. The emails will commonly be sent from an email address that is a non-functioning email account used exclusively for sending, such as

The PHP mail() function is typically the default method that PHP packages will use for sending transactional mail. This function will use the server's sendmail-compliant MTA to send the mail, thus necessitating a sendmail-compliant MTA to be installed. An MTA can be used to send mail directly from the server or can be configured to send mail through a smart host to reduce server and admin overhead.

Most of the popular PHP packages will include an option or have a plugin available to send transactional email through an SMTP server, eliminating the requirement to install any type of MTA for the purpose of supporting email generated by the package. However, without an MTA installed, system messages cannot be sent to the administrators.

Transactional email via MTA

Log in as root and install the necessary packages:

username@servername:~$ sudo -i
root@servername:~# aptitude install postfix opendkim opendkim-tools

Select 'Internet site' during the postfix install.

Configure OpenDKIM

The opendkim package will add a DKIM signature to configured outbound emails. Receiving servers can compare this signature with the corresponding DNS DKIM record and verify messages as coming from an approved server and unaltered.

root@servername:~# mkdir -p /etc/opendkim/keys/
root@servername:~# opendkim-genkey -r -b 2048 -h sha256 -d -s selector -D /etc/opendkim/keys/

Replace selector as desired, as it is only used to identify the key the server will call. Many admins will simply use the date the key was created (e.g. '20140510') or, when multiple servers are being used, the server name (e.g. 'mta05'), though none of this is standardized or required - some simply use 'mail', or don't specify anything when running the command, leaving the selector of 'default'. The command creates two files, selector.private and selector.txt. The selector.private file contains the private key while selector.txt contains the basis for the DNS TXT record that will be created in a later step.

Note that the DKIM standard currently recommends a maximum key size of 2048 bits, so using a larger key size will likely cause the DKIM test to fail on many servers, while using a key smaller 1024 bits is not only not recommended, but has been demonstrated to be insecure.


Create the tables that opendkim will use:

root@servername:~# nano /etc/opendkim/KeyTable

Add: %:selector:/etc/opendkim/keys/

Since this server is being configured to have one key serve mail for multiple domains, only one private key is being used. The key could be named anything and does not have to be a domain. The % instructs opendkim to add the domain being sent into the DKIM header d= value.


root@servername:~# nano /etc/opendkim/SigningTable



The second and third entries demonstrate how additional domains served by the mail server would be added to the signing table.


root@servername:~# nano /etc/opendkim/TrustedHosts

Configure opendkim.conf

Set permissions on the directory, archive the default opendkim.conf before creating a new one:

root@servername:~# chown -R opendkim:opendkim /etc/opendkim
root@servername:~# mv /etc/opendkim.conf /etc/original.opendkim.conf
root@servername:~# nano /etc/opendkim.conf

Add to the new file:

# This is a basic configuration that can easily be adapted to suit a standard
# installation. For more advanced options, see opendkim.conf(5) and/or
# /usr/share/doc/opendkim/examples/opendkim.conf.sample.

# Log to syslog
Syslog                  yes
# Required to use local socket with MTAs that access the socket as a non-
# privileged user (e.g. Postfix)
UMask                   002

# Commonly-used options
SubDomains              yes
AutoRestart             yes
Background              yes
Canonicalization        relaxed/relaxed
DNSTimeout              5
Mode                    sv
SignatureAlgorithm      rsa-sha256

# Additional OpenDKIM options

ExternalIgnoreList      refile:/etc/opendkim/TrustedHosts
InternalHosts           refile:/etc/opendkim/TrustedHosts
KeyTable                refile:/etc/opendkim/KeyTable
SigningTable            refile:/etc/opendkim/SigningTable
LogWhy                  Yes
PidFile                 /var/run/opendkim/
Socket                  local:/var/spool/postfix/opendkim/opendkim.sock
SyslogSuccess           Yes
TemporaryDirectory      /var/tmp
UserID                  opendkim:opendkim

# Always oversign From (sign using actual From and a null From to prevent
# malicious signatures header fields (From and/or others) between the signer
# and the verifier.  From is oversigned by default in the Debian package
# because it is often the identity key used by reputation systems and thus
# somewhat security sensitive.
OversignHeaders         From

Though most of the settings are fairly self-explanatory, it is a good idea to become familiar with the various settings to reduce the time spent troubleshooting why other mail servers are failing DKIM checks on mail sent from the server. One particular setting to note is SubDomains (everything after the @ symbol in an email address) being set to no. The above configuration has SubDomains set to yes as many popular PHP packages will default send email from the domain for which they are configured (e.g.,

Create the directory for the domain socket specified in opendkim.conf, make the postfix user a member of the opendkim group so it can edit opendkim.sock, and restart the service:

root@servername:~# mkdir /var/spool/postfix/opendkim
root@servername:~# chown opendkim:root /var/spool/postfix/opendkim
root@servername:~# service opendkim restart

Configure postfix

Following the below steps will configure postfix exclusively for sending and will reject email not coming from one of the loopback addresses. Additionally, ufw has not been configured to permit inbound connections commonly monitored by postfix, so attempts to send mail to the server will fail to at least these two considerations.


The file is the primary configuration file for Postfix, and it is enormous. Archive the original version, then create a new one:

root@servername:~# mv /etc/postfix/ /etc/postfix/
root@servername:~# nano /etc/postfix/

Add the following:

# See /usr/share/postfix/ for a commented, more complete version

# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# See -- default to 2 on
# fresh installs.
compatibility_level = 2

# TLS parameters
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache

smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_security_level = may
smtp_tls_note_starttls_offer = yes

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname =
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination =
relayhost =
mynetworks = [::ffff:]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces =, [::1]
inet_protocols = all

non_smtpd_milters = unix:opendkim/opendkim.sock
smtpd_milters = unix:opendkim/opendkim.sock


Another important and huge configuration file is Make an archive of the original version and make a new one:

root@servername:~# mv /etc/postfix/ /etc/postfix/
root@servername:~# nano /etc/postfix/

Add to the file:

# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master" or
# on-line:
# Do not forget to execute "postfix reload" after editing this file.
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (no)    (never) (100)
# ==========================================================================
smtp      inet  n       -       y       -       -       smtpd
#smtp      inet  n       -       y       -       1       postscreen
#smtpd     pass  -       -       y       -       -       smtpd
#dnsblog   unix  -       -       y       -       0       dnsblog
#tlsproxy  unix  -       -       y       -       0       tlsproxy
#submission inet n       -       y       -       -       smtpd
#  -o syslog_name=postfix/submission
#  -o smtpd_tls_security_level=encrypt
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_tls_auth_only=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING
#smtps     inet  n       -       y       -       -       smtpd
#  -o syslog_name=postfix/smtps
#  -o smtpd_tls_wrappermode=yes
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING
#628       inet  n       -       y       -       -       qmqpd
pickup    unix  n       -       y       60      1       pickup
cleanup   unix  n       -       y       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
#qmgr     unix  n       -       n       300     1       oqmgr
tlsmgr    unix  -       -       y       1000?   1       tlsmgr
rewrite   unix  -       -       y       -       -       trivial-rewrite
bounce    unix  -       -       y       -       0       bounce
defer     unix  -       -       y       -       0       bounce
trace     unix  -       -       y       -       0       bounce
verify    unix  -       -       y       -       1       verify
flush     unix  n       -       y       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       y       -       -       smtp
relay     unix  -       -       y       -       -       smtp
        -o syslog_name=postfix/$service_name
#       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq     unix  n       -       y       -       -       showq
error     unix  -       -       y       -       -       error
retry     unix  -       -       y       -       -       error
discard   unix  -       -       y       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       y       -       -       lmtp
anvil     unix  -       -       y       -       1       anvil
scache    unix  -       -       y       -       1       scache
# ====================================================================
# Interfaces to non-Postfix software. Be sure to examine the manual
# pages of the non-Postfix software to find out what options it wants.
# Many of the following services use the Postfix pipe(8) delivery
# agent.  See the pipe(8) man page for information about ${recipient}
# and other message envelope options.
# ====================================================================
# maildrop. See the Postfix MAILDROP_README file for details.
# Also specify in maildrop_destination_recipient_limit=1
#maildrop  unix  -       n       n       -       -       pipe
#  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
# ====================================================================
# Recent Cyrus versions can use the existing "lmtp" entry.
# Specify in cyrus.conf:
#   lmtp    cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
# Specify in one or more of the following:
#  mailbox_transport = lmtp:inet:localhost
#  virtual_transport = lmtp:inet:localhost
# ====================================================================
# Cyrus 2.1.5 (Amos Gouaux)
# Also specify in cyrus_destination_recipient_limit=1
#cyrus     unix  -       n       n       -       -       pipe
#  user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
# ====================================================================
# Old example of delivery via Cyrus.
#old-cyrus unix  -       n       n       -       -       pipe
#  flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user}
# ====================================================================
# See the Postfix UUCP_README file for configuration details.
#uucp      unix  -       n       n       -       -       pipe
#  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
# Other external delivery methods.
#ifmail    unix  -       n       n       -       -       pipe
#  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
#bsmtp     unix  -       n       n       -       -       pipe
#  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
#scalemail-backend unix  -       n       n       -       2       pipe
#  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
#mailman   unix  -       n       n       -       -       pipe
#  flags=FR user=list argv=/usr/lib/mailman/bin/
#  ${nexthop} ${user}

Postfix user groups

The postfix user will need to be added to the opendkim group to support domain sockets:

root@servername:~# usermod -G opendkim postfix
root@servername:~# service postfix restart

Configure DNS records

There are three different DNS record entries that can be made to reduce the chances of mail being declared as spam, the SPF record, DKIM record, and DMARC record.

Configure SPF records

SPF records are used to verify that emails originating from a domain are permitted to be sent on behalf of the domain. This means there is no actual local server configuration on a transactional mail server.

Log into your DNS server and create a text record with at least the following for the tld (e.g.,

v=spf1 a -all

The above record will inform receiving servers that only the servers listed in the 'a' record (including 'aaaa' record for IPv6 addresses) are approved to send mail. The -all mechanism means nothing else is approved to send mail. Note that the softfail mechanism (~) used instead of the hardfail (-) means the record will be ignored by most popular mail servers resulting in an increased chance of mail from the sending server arriving in spam folders.

If it is desired to send mail from a subdomain from the same server, it is necessary to create an SPF record in addition to the main domain. A simple solution is to use the include mechanism. This mechanism means that the SPF record of the target domain will be included in the current domains SPF record.

Example for

v=spf1 -all

Configure DKIM record

DKIM can only function with a valid DNS TXT record. In the DNS manager for your DNS server, make the following new TXT record:

Enter into the optional subdomain field:


Enter the contents of /etc/opendkim/keys/ into the body of the record using the format below (i.e. only the contents between " and "):

v=DKIM1; h=sha256; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuX4vPnLopTAIMFnnuP4CCEfE/FtQO0mi77voGsWSvHQfvFMIkQ3W3VmeAEiSJd6SVkL/Ojr30ag2i6wA3NTU+1ndfgL371zKx4gDAnewoRA4N2P05HPUNe10DE+m4xnwB6zsQnnPJ3EgKIW6W/v+fN/EzTfeJo5UmxiAoFRIq5hgyeHHCI8aKMQLCmWhb/Pz22MiqRHxV91xmTMLx/e3BIsplcOmQjlOyGagoIZJxpcTlf9OiSWks2a5kHXEN40eh99zkPGInqTrbhDog+cn/mvPgY0uIznx1i/ubRQFtYaH5t6vCu5uSMEQjcTQnWRLI9Qt7Mp15hOMrpkKv4SPzwIDAQAB

Keys can be verified using tools such as the one at DKIM Core. Note that some DNS servers may not like the length of the key string, so it may be necessary to use one of the record formatting methods recommended in the OpenDKIM README.

Configure DMARC record

DMARC records are used to inform receiving servers of what the domain owner's policies are for handling DKIM and SPF record checks. Similar to SPF records, there is no configuration for the local server.

Generating the DMARC record is easiest with an online tool, such as the Kitterman DMARC Record Assistant. The record generated there can be added to a DNS text record for the domain using the subdomain _dmarc.

Note that the quarantine as a policy is much less substantial in preventing email from arriving in spam folders when compared to the reject policy. However, with the reject policy enforced, the email will not arrive when there is some sort of server configuration issue (e.g., opendkim stops signing messages or a PHP package or plugin upgrade adds a previously unused subdomain to the email address).

The RUA and FUA reports must be sent to either an email address for the domain the record applies to (e.g., or if a different domain is used, then the DMARC record for that domain must state it is permitted to receive reports for the domain, else the reports will not be sent.

A configuration for testing:

   v=DMARC1; p=none; adkim=s; aspf=s

It may be informative to change from strict (=s) to relaxed (=r).

A simple, strict and brutal record that would not generate reports:

   v=DMARC1; p=reject; adkim=s; aspf=s

Configure system messages

Receiving notifications from the system is a requirement for maintaining a server. For the purposes of this confirmation, the system message will be emailed by alias@mailname.


The system mailname should be set to the domain of the server.

   root@servername:~# nano /etc/mailname

Confirm the mailname matches the domain of the server. If the server is, then use If the server is, then use

For the purposes of this article, server would have the following in /etc/mailname:


The alias can be used to define the email address that messages for root are sent to, using the /etc/aliases file (see man 5 aliases for more information).

   root@servername:~# nano /etc/aliases

A basic server configuration for server where the email address to receive system emails is would look like:

   # See man 5 aliases for format
   postmaster:    root

Some services, such as monit, may require an additional entry:

   # See man 5 aliases for format
   postmaster:    root
   monit:    root

Additional measures to stay out of spam folders

Staying out of spam folders is relatively easy from a technical perspective. It is important to note that the very largest providers have made it nearly impossible to stay out of spam folders when not using one of the very largest providers to send mail from and resolving this issue is beyond the scope of this article. For the technical proof of work that the server is sending legitimate mail, perform the following in addition to the previous steps in this article:

Forward confirmed rDNS (FCrDNS)

Forward confirmed reverse DNS (FCrDNS) is simply a PTR record that in turn has an A record pointing to the IP address originally queried.

username@servername:~$ dig a +short
username@servername:~$ dig -x +short

An online tool to confirm FCrDNS has been configured properly can be found at Forward Confirmed Reverse DNS Testing Form.

DNS Blacklists

DNS Blacklists come in many different flavors and may even have nothing to do with spam ever originating from an IP address, domain, or server. For the small enthusiast, the only action that can be taken is to verify nothing managed is on any blacklists and perform all steps necessary to be removed from blacklists, though this may occasionally require migrating to a new ISP.

It is more important to pay attention to blacklists employed by email server administrators than simply to remain off of all blacklists in general, as some of the blacklists exist purely to extort money from administrators.

Testing tools

Several tools exist for checking an IP address for blacklisting, including:

DNSBL Information - Spam Database and Blacklist Check - Thorough checking of lists for IP addresses.

MX Lookup Tool - Check your DNS MX Records online - MxToolbox - includes multiple different tools for checking different aspects of mail services.