Postfix PKI Security

This article will use the traditional Alice seeking to send a message to Bob with Eve as the Eavesdropper, and how the PKI system can help mitigate attacks Eve may try.

There are two different mechanisms that can currently be used, MTA-STS (a standard) and ‘STARTTLS Everywhere’, a non-standard solution for MTA clients that are not yet MTA-STS capable.

Introduction to PKI and SMTP

PKI is the standardized system used to verify that the public key associated with a TLS connection is valid and belongs to the host distributing the public key. This is performed through a chain of trust established by a root certificate that the client (or server in the case of client authentication via PKI) implicitly trusts.

A Certificate Authority will publish an x.509 root certificate containing a public key that you either implicitly trust or do not trust. The associated private root key usually does not sign the certificates used on public servers. It signs an intermediary certificate that is flagged as being allowed to sign other certificates, which itself often signs another intermediary certificate that does the actual signing of the x.509 certificates used on public servers. This is called (at least by me) the chain of trust.

The server (or client in the case of client authentication via PKI) is expected to send the intermediary x.509 certificates in the chain of trust when they send their issued x.509 certificate to the client (or server). These are often in a separate file referred to as the ‘Certificate Authority Bundle’, but many system administrators simply append them to the file containing their server specific x.509 certificate.

If the client implicitly trusts the root certificate, none of the certificates in the chain of trust have expired, none of the certificates in the chain of trust have been revoked, none of the certificates in the chain of trust violate usage restrictions, and if the domain name of the certificate issued to the server matches the server, then the client accepts the certificate as valid.

Historically MTA clients rarely bothered with validating the x.509 certificate chain of trust presented by the MX server they connect to. With Opportunistic TLS, messages are sent in plain text if a secure connection can not be established to the MX server, so there really was no point in validating the chain of trust. This made MITM attacks relatively easy to pull off. That is changing.

MTA-STS (defined in RFC 8461) provides a standardized mechanism by which an MX server can publish that it uses a proper certificate and quality encryption, allowing MTA clients to refuse to connect if things are not as they are suppose to be. Additionally, the STARTTLS Everywhere project provides a hard-coded list of e-mail mailbox domains that are expected to have properly validating certificates and support a minimum of TLS 1.2.

Mailbox Domain MTA-STS Security

alice@deviant.email wants to send an e-mail to bob@example.org. This makes example.org the so-called ‘mailbox domain’. Before the MTA client that relays the message can avoid the pitfalls of Opportunistic TLS by enforcing MTA-STS, the domain example.org must properly support and advertise it.

Valid TLS Certificates

Before the administrator of example.org can advertises MTA-STS compliance support, they must ensure that every MX server listed in the DNS MX record not only supports STARTTLS with properly PKI validating certificates, but is committed to do so into the future.

That means no self-signed certs, no expired certs, no certs that are issued to a different domain name.

The certificate should be issued by a well-known Certificate Authority (such as Comodo, Let’s Encrypt, or DigiCert) that is widely trusted. The certificate needs to provide Certificate Transparency (often known as CT), Certificate Transparency is defined in RFC 6962 and is one of the good things to come from Google.

I did not see Certificate Transparency required in the MTA-STS RFC but it should have been. As of May 2018 all major Certificate Authorities are suppose to provide it, which may be why MTA-STS does not explicitly require it. I still wish the RFC had specified clients should check for it.

Modern TLS Stack

Before the administrator of example.org can advertises MTA-STS compliance support, they must ensure the MX servers support TLS 1.2. The servers should also support the new TLS 1.3 standard but it is not strictly required. The MX server may continue to support TLS 1.0/1.1 but they should be phased out. TLS 1.2 was standardized in 2008, that is over a decade ago, there is no justifiable reason to continue supporting the earlier standards. SSL 2.0/3.0 are broken and simply should not be allowed.

The MTA-STS RFC does not specifically require cipher suites that support Forward Secrecy, but the MX server should only allow cipher suites that use Forward Secrecy (and that in fact is a requirement for TLS 1.3). The server at least should enforce the server preferred order for cipher suite selection with ciphers that do support Forward Secrecy listed first.

Forward secrecy can be achieved with either ECDHE or DHE ciphers, but the latter is an outdated key exchange that should be phased out. It is already too weak with DH parameters smaller than 2048-bit.

MX Record Security via HTTPS: The MTA-STS Policy

When alice@deviant.email wants to send their message to bob@example.org, the MTA client that relays the message needs to know where on the Internet to route the message. It probably should not be sent to example.org itself, and very often it actually needs to go to a completely different domain that provides e-mail services for example.org.

Where the e-mail should be routed is stored in the MX record for example.org. While DNS provides a convenient distributed database for that kind of information, it is possible for the attacker Eve (in keeping with the Alice and Bob paradigm) to modify the MX response in transit so that Alice’s MTA client sends the message to Eve, who logs and reads the message, before then sending it to Bob.

DNSSEC helps thwart this kind of attack, but only if Bob’s DNS zone signs their records and Alice’s MTA client checks. Unfortunately DNSSEC adoption has been slow, so another method must be used to secure the MX record from tampering.

With MTA-STS this problem is at least partially mitigated with the use of HTTPS. The administrator of example.org would run a secure web server at the sub-domain mta-sts.example.org. The purpose of the web server being secure is to prevent the response from being modified.

On that web server at the precise path of /.well-known/mta-sts.txt is a plain text file containing the key: value pairs that define the version of MTA-STS (currently STSv1 is only meaningful version), the mode of MTA-STS, a list of MX hosts, and the max_age setting. These settings are referred to as the MTA-STS policy:

version
This is used to define the version of MTA-STS. Presently there is only one version, this should be set to STSv1 for MTA-STS as defined in RFC 8641.
mode
There are three possible modes: none, testing, and enforce.
A mode of none tells the client MTA not to enforce MTA-STS policy. It is only useful to set that mode if you previously were set up for MTA-STS but have decided to no longer implement it.
A mode of testing tells the client MTA not to enforce MTA-STS but does ask that it send it notification if enforcement would have prevented message delivery.
A mode of enforce tells the client MTA it should not deliver the message if it can not do so in a secure manner in accordance with MTA-STS policy. This is the only mode that offers real-time protection.
mx
Each of the MX servers used to accept mail for the mailbox domain should be listed with a mx key. Wilcard * is allowed at the beginning of an mx value but only at the beginning, left most side.
max_age
Number of seconds the policy should be considered valid for. This provides some protection against Eve pulling off an attack by interfering with the DNS responses for the mailbox domain. This can be set to a few hours in testing mode, but it should be set to about 30 days in enforce mode.

An example of what the policy text file might look like:

version: STSv1
mode: enforce
mx: mx.example.org
mx: *.mx.example.org
mx: backupmx.someplace.net
max_age: 2592000

Advertisement of MTA-STS in DNS

Now the admin of Bob’s mailbox domain — example.org — has everything needed to advertise MTA-STS support so the MTA clients connecting to it can enforce it. This is done with a TXT record in DNS with an owner subdomain of _mta-sts followed by a . followed by the mailbox domain. So with a mailbox domain of example.org the TXT record would be associated with _mta-sts.example.org.

This TXT record has two key=value pairs, with each pair followed by a ; indicating the end of the pair. The first specifies the version of MTA-STS (with v=STSv1 for the only version now) so that the MTA client knows https://mta-sts.example.org/.well-known/mta-sts.txt is the right path to find the policy at, and the second specifies a policy id number that should change every time the policy at the policy URL has changed.

Here is an example from the zone file for my mailbox domain, librelamp.com:

_mta-sts.librelamp.com.    IN  TXT "v=STSv1; id=5bdf460e;"

What you use for the id value does not matter too much as long as you change the value whenever you change the policy. I personally convert the base 10 value for ‘seconds since UNIX epoch’ to hex. Google seems to use a YYYYMMDDTHHMMSS timestamp. Looking around at a few policies, I've also seen it use they the same scheme as a zone file serial number, YYYYMMDDnn where nn starts as a 00 and is only incremented if a policy change happens more than once in the same day. Some I have seen appear to just be a hash of a random number and do not use an orderly sequence scheme at all.

SMTP TLS Reporting

Optionally, the mailbox domain may publish an e-mail address or URI for reporting TLS errors related to sending mail.

If the MTA-STS mode is set to testing the client will connect even if the conditions of MTA-STS are not met but may send an e-mail to the published e-mail address or URI detailing the failed conditions so they can be dealt with. If the MTA-STS mode is set to enforce the client is suppose to delay connection until the situation is resolved, but still may send a report.

TLS-RPT may also be used to report failures in DANE validation.

For details on TLS-RPT please see RFC 8460. It consists of a single TXT record in DNS.

Test Configuration

You can test your MTA-STS configuration by entering your mailbox domain at: https://aykevl.nl/apps/mta-sts/

Nutshell Client Behavior

An MTA-STS enforcing client with a message to be sent to bob@example.org without a policy cached for that mailbox domain will initially request the TXT record associated with _mta-sts.example.org. If it exists, and the v parameter is a version of MTA-STS the client supports, it will then request the A and/or AAAA records associated with mta-sts.example.org so it can fetch the current policy. Assuming successful retrieval of the policy, the client will cache the policy using the id assigned in the initial TXT record until either the policy has been replaced with a different policy, or the max_time specified in the policy has expired.

The client will then only attempt to deliver the message to a host in the DNS MX record if the host matches a host in the policy. If no hosts in the MX record match what is in the policy, delivery will be delayed and tried again later. The client will insist upon a TLS 1.2 connection or better, with an x.509 certificate that validates. Otherwise delivery will be delayed and tried again later.

If an MTA-STS enforcing client already has a policy cached, it works a little differently. It still requests the TXT record associated with _mta-sts.example.org but if it either fails to retrieve that record or if successful but the id key has the same value as what is already cached, then the MTA client will just apply the policy it already has cached since it has not changed or can not be retrieved. Presumably if request of the TXT record is successful but has not changed, the life of the cached policy will be extended as if it was just freshly fetched.

If retrieval of the TXT record is successful and the policy id has changed, the MTA will then attempt to retrieve the updated policy from the web server. Upon success it will apply the new policy. Upon failure, it will continue to apply the old policy until it expires.

The max_life should thus be set to about 30 days when the policy mode is set to enforce so that if Eve attacks the DNS responses but the client MTA already has the policy cached, it will be a rather significant length of time the attack has to be maintained before the policy is no longer applied.

Mailbox Domain STARTTLS Everywhere

This is an initiative started by Electronic Frontier Foundation to maintain a centralized list of mailbox domains that use MX servers with proper validating x.509 certificates and modern TLS.

If your mailbox domain is MTA-STS compliant, you can be listed in STARTTLS Everywhere. If your mailbox domain is not MTA-STS compliant, MTA-STS is an IETF standard, make that a priority.

For many of us, we are only implementing STARTTLS Everywhere on our client MTA systems while waiting for Postfix to get a complete proper MTA-STS implementation. MTA-STS scales better since it allows discovery of policy when needed and is not centralized. STARTTLS Everywhere should be seen as a stop-gap temporary measure.

List your mailbox domain with them, but please make sure your server is properly set up for MTA-STS first.

To have your mailbox domain listed in STARTTLS Everywhere, please visit https://starttls-everywhere.org/.


MTA Clients and MTA-STS Security

Traditionally when acting as a client, Postfix is not configured to verify the x.509 certificates of the MX servers it connects to. With Opportunistic TLS, validating the certificate does not make sense since the message is sent in plain text if a secure connection can not be established. TLS with an invalid certificate is still more secure than plain text.

MTA-STS provides a mechanism by which the Opportunistic is taken out of the equation. If the connection is not secure with a validating certificate, transmission of the message is delayed until it is secure with a validating certificate. So Postfix needs to be configured to validate the x.509 certificate of the remote server.

The LibreLAMP packaging of Postfix 3.3.4 requires the mozilla-ca-certificates-postfix package which includes all of the Certificate Authority root certificates that are trusted by Mozilla. That collection is largely considered to be the ‘De Facto Standard’ collection of Certificate Authorities to trust. However LibreLAMP Postfix is not configured to use them by default. The following line must exist un-commented in the Postfix /etc/postfix/main.cf configuration file:

smtp_tls_CApath = /etc/postfix/certs

In the event you need to add additional root certificates you want to trust to the /etc/postfix/certs directory, you can, just make sure to run:

[root@host ~]# /usr/bin/c_rehash /etc/postfix/certs

after doing so. Most people should never need to, root certificates that are not in the Mozilla bundle are usually either brand new or are excluded for a very good reason.

You also should install the Unbound caching resolver in DNSSEC enforcing mode and configure your operating system to use it. When the mailbox domain uses DNSSEC, Unbound will give you addition protection against fraudulent MX records from DNS.

Postfix and MTA-STS

At this time, Postfix itself does not directly support MTA-STS. There is however a bolt-on implementation that is largely complete: https://github.com/Snawoot/postfix-mta-sts-resolver

That implementation requires Python 3.5.3 or newer. CentOS 7 ships with Python 2.7 and EPEL offers Python 3.4.9, neither of which are new enough. You can install a newer Python via the CentOS Software Collections (SCLs) if you want to play with that implementation, but I have decided to wait with trying to package it, I am hopeful that Postfix itself will develop a complete implementation independent of Python interpreter version.

Still, that solution can probably be made to work. It does not yet implement Proactive Policy Fetch, I believe changes in how Postfix itself works would be needed to properly implement that.

For the present I am implementing the solution that follows.

Postfix and STARTTLS Everywhere

With STARTTLS Everywhere, we can generate a TLS policy map file that restricts connections to specific hosts to TLS 1.2 with valid x.509 certificates.

The STARTTLS Everywhere project distributes a JSON encoded database of mailbox domains you should only send messages to if the certificate validates and the connection is using TLS 1.2 or newer. You can view the JSON database file at https://dl.eff.org/starttls-everywhere/policy.json.

To use that policy database, it needs to be updated frequently and it needs to be converted into the format that Postfix works with. The EFF has a Python script to do that at https://github.com/EFForg/starttls-policy-cli. It probably works but I do not use it, I use a PHP script. There is a reason.

Currently the policy database has every single mailbox domain use testing mode, which their script translates to a Postfix TLS policy of may. That results in Opportunistic TLS which provides no additional security. They also do not have support at this time for creating DANE based policies.

To make their list, your mailbox MX hosts MUST have validating x.509 certificates and offer TLS 1.2 so there is no reason for those hosts to use a Postfix security level of may. Furthermore when DANE is an option, that is actually preferable to PKI based validation of the MX server.

What I have created is a set of scripts that run from the system crontab that create secure channel policies for the mailbox domains in the STARTTLS Everywhere list, taking DANE validation into consideration for mailbox domains I know have had proper DANE support for at least a year.

On average of three times in a twenty-four hour period, it will attempt to retrieve an updated policy.json file from the STARTTLS Everywhere project, and merge that with the list of DANE validating mailbox domains.

Make sure you have unbound installed and properly running. Make sure Postfix is set up to use make use of both DNSSEC and x.509 certificate validation. In your /etc/postfix/main.cf file you need to have the following directives:

smtp_tls_CApath = /etc/postfix/certs
...
smtp_dns_support_level = dnssec
smtp_host_lookup = dns

Then install the starttls-everywhere package:

[root@host ~]# yum install starttls-everywhere
[root@host ~]# sh /usr/libexec/update-postfix-tls-policy-list.sh
[root@host ~]# postconf -e 'smtp_tls_policy_maps = hash:/etc/postfix/starttls_everywhere_policy'
[root@host ~]# postfix reload

Once an hour, there are 1 in 8 odds it will decide to fetch an updated policy.json file from the STARTTLS Everywhere project. In the unlikely but mathematically possible scenario it goes 35 hours without triggering a fetch of a fresh policy, the odds do not matter, it tries to fetch a new policy.

The random nature of the policy fetching as well as the random delay when it does trigger was done to prevent lots of systems from hitting the policy server at the same time.

It is my opinion this should be used as a temporary solution until proper MTA-STS support exists in Postfix. The MTA-STS standard provides a scalable distributed mechanism for discovery of mailbox domains that should only be connected to with modern security, that needs to be the goal.

The proper long term solution is probably a policy socket server that applies the secure-channel dane-only policy to mailbox domains that are well-known to be committed to DANE, the opportunistic dane policy to any other mailbox domain properly set up for DANE, and the secure-channel secure policy to any non-DANE mailbox domain that has MTA-STS properly configured.

I do not know if anyone is working on such a policy server for Postfix but that should probably be the goal.


Inherent MTA-STS Weaknesses

MTA-STS does have some inherent weaknesses that can still be attacked.

When a client MTA does not have a cached policy for the mailbox domain it wants to send a message to, Eve can simply modify the DNS response for the MTA-STS triggering TXT record in addition to modifying the MX record for the mailbox zone.

This will prevent the client MTA from even knowing an MTA-STS policy exists, and it will happily send the messages to the hosts listed in the fraudulent MX record.

While caching of the policy (and STARTTLS everywhere) can be very effective at thwarting this attack, the only reliable defense is DNSSEC. If the mailbox zone implements DNSSEC and the client MTA uses a recursive resolver that validates DNSSEC, it becomes much more difficult to feed a fraudulent MX record response to the requesting client.

If one or more of the mx definitions in the policy uses a wildcard * at the beginning for pattern matching, it may be possible for Eve to set up a trojan MX server and use DNS hijacking to resolve a fake domain to their trojan. Then Eve could add that trojan to a fraudulent MX record response to have mail directed to it. The defense against this attack is again for the mailbox domain to use DNSSEC to protect the integrity of their MX record from being compromised, and it probably is prudent to avoid wildcards in the MTA-STS policy.

The zone (sometimes more than one for high availability) for the actual MX domains is also a potential point of weakness. They frequently differ from the zone for the mailbox domain. Even if the mailbox domain uses DNSSEC, if the zone(s) for the MX servers do not use DNSSEC, the IP addresses they resolve to can be forged. PKI validation of the x.509 certificate reduces this risk.

The MTA client should require a certificate with Certificate Transparency. CT makes it easier to detect fraudulently issued certificates so the Certificate Authority can revoke them. The MTA client should require a valid OCSP response before accepting a certificate as valid. Postfix, as well as many other MTA clients, currently lacks support for OCSP validation. Hopefully that changes.

The MTA server should staple a valid OCSP response to the certificate it sends so that the client does not need to connect to the OCSP server to verify the certificate has not been revoked. Postfix as an MTA server also currently does not have this capability. Hopefully that changes.

The best defense against fraudulently signed x.509 of course is DANE, which also requires DNSSEC. Postfix has excellent support for DANE validation, that has worked for some time. Unfortunately DANE adoption has been slow.

Despite the inherent weaknesses, mailbox domains should implement MTA-STS. Many of those inherent weaknesses can be thwarted if both the mailbox domain zone implements DNSSEC and the client MTA enforces DNSSEC. Additional weaknesses can be thwarted if the zone the MX servers are on implement DNSSEC. Security can be further maximized if the zone the MX servers are on implement DANE.

It is very important for Internet servers in general, whether they are acting as an MTA client or not, to run a DNSSEC enforcing caching resolver that listens on the localhost, and configure the operating system to use it for name resolution. That protects all services from fraudulent DNS answers, at least when the authoritative zone for the answer implements DNSSEC.