DNS records in e-mail system

blog

Recently, I have to build a personal email system, and I am systematically learning about it. Today, I am going to share the content related to DNS.

I have divided the contents into three parts according to the functions or scenarios: sending mail, receiving mail and reading mail. Sending mail is to use the mail system to send mail to an external server; receiving mail is to receive email from an external server; and watching mail is for using a client to view or download mail from a server. Let’s start with sending mail.

Sending Mail

PTR

Each e-mail address has its own server. When sending a message, the sender’s server connects to port 25 of the receiver’s server to communicate. Theoretically any device on the Internet could act as a sending server. However, a massive spam problem soon arose. The spammers used supplied dynamic IP addresses that were impossible to defend against. This eventually led to two results:

  1. Prohibit user devices from accessing port 25 of the extranet server. This policy is still implemented by many cloud service vendors.
  2. The receiving server reverse queries the domain name of the IP address through the PTR record of DNS and rejects it if it does not match the domain name of the sender.

With the above two strategies, the spam problem has been alleviated. Therefore, when setting up mail servers in the early days, it is necessary to add PTR records to the corresponding IP addresses.

For example, Google’s IP address 8.8.4.4 has a PTR domain name of 4.4.8.8.in-addr.arpa, which corresponds to the following PTR values:

drill PTR 4.4.8.8.in-addr.arpa
# or
drill -x 8.8.4.4
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 3489
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; 4.4.8.8.in-addr.arpa.        IN      PTR

;; ANSWER SECTION:
4.4.8.8.in-addr.arpa.   60908   IN      PTR     dns.google.

This points to the domain dns.google.

PTR records use a dedicated domain name space in-addr.arpa/ip6.arpa, querying PTR records will put more pressure on the DNS system and slow down the speed of mail collection. Therefore, IETF has designed a new SPF record to take care of PTR.

SPF

The so-called SPF, Sender Policy Framework, is simply a special TXT record that keeps a record of which servers are allowed to send email under a domain.

Take QQ mailbox as an example, we query its TXT records:

drill txt qq.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 13765
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; qq.com.      IN      TXT

;; ANSWER SECTION:
qq.com. 450     IN      TXT     "v=spf1 include:spf.mail.qq.com -all"

Here you first see the SPF prefix v=spf1, followed by include:spf.mail.qq.com, which indicates that you need to query the TXT record for spf.mail.qq.com again to get the real list of servers, and that only servers on that list are allowed to send mail in the @qq.com domain. The -all at the end means that if any other server tries to send @qq.com domain mail, the recipient must reject it.

For the final -all, you will sometimes see something like ~all, which means that if the message comes from a server outside of the SPF list, it needs to be received and put into spam. This strategy is more robust than reject, and is mainly used to avoid situations where the SPF is not up to date and the message is completely rejected.

We continue to query TXT records for spf.mail.qq.com:

drill txt spf.mail.qq.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 27877
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; spf.mail.qq.com.     IN      TXT

;; ANSWER SECTION:
spf.mail.qq.com.        600     IN      TXT     "v=spf1 include:qq-a.mail.qq.com include:qq-b.mail.qq.com include:qq-c.mail.qq.com include:biz-a.mail.qq.com include:biz-b.mail.qq.com include:biz-c.mail.qq.com include:biz-d.mail.qq.com -all"

Obviously, this is a much longer SPF list, so we take the first qq-a.mail.qq.com and continue the query:

drill txt qq-a.mail.qq.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 51025
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; qq-a.mail.qq.com.    IN      TXT

;; ANSWER SECTION:
qq-a.mail.qq.com.       600     IN      TXT     "v=spf1 ip4:101.226.139.0/25 ip4:101.91.43.0/25 ip4:101.91.44.128/25 ip4:112.64.237.128/25 ip4:116.128.173.0/25 ip4:121.51.40.128/25 ip4:121.51.6.0/25 ip4:162.62.52.214 ip4:162.62.55.67 ip4:162.62.57.0/24 ip4:162.62.58.211 ip4:162.62.58.216 -all"

Finally, instead of the word include, there is a list of IP segments starting with ip4, which can be either segments, such as ip4:101.226.139.0/25, or individual addresses, such as ip4:162.62.58.216. If you need to allow IPv6 network addresses, you need to add the ip6: prefix.

In addition to include/ip4/ip6, SPF records support the following types of data:

  • a means to query the A/AAAA records of a domain name pair. For example, a:example.com means you need to query the A or AAAA records corresponding to the domain name example.com according to the network type of the sender, and verify whether it corresponds to the IP address of the sender.
  • mx is similar to a, but you need to find out the corresponding MX record of the domain name first, and then query the corresponding A/AAAA record for the domain name of the MX record.
  • ptr Indicates that the PTR domain name needs to be queried against the sender’s IP address and compared to the value specified in this field. This type is no longer recommended.

From the above analysis, we can see that the recipient needs to recursively query the SPF record in DNS to confirm whether the sender is allowed to send the message before receiving the message. This process may trigger many DNS resolutions. In order to avoid DDos attacks, SPF stipulates that all DNS queries should not exceed 10 times, and if they exceed 10 times, the message will be rejected!

For personal mail services, which generally don’t have a lot of servers, it’s entirely possible to write them all in a single SPF record without recursive queries.

Suppose our mail domain is example.org, the corresponding mail server is ex1.example.com, and its IP address is A:10.2.3.4/AAAA:2001:beef::1.

The most recommended way to write this is:

mx1.example.org. TXT   "v=spf1 ip4:10.2.3.4 ip6:2001:beef::1 ~all"

If your IP address is not fixed and sometimes changes, you can use the a type:

mx1.example.org. TXT   "v=spf1 a:ex1.example.com ~all"

If the domain name of example.com’s mail server changes frequently, that is, the MX will change, then it can only be written in the following form:

mx1.example.org. TXT   "v=spf1 mx ~all"

However, either mx or a will cause the recipient to generate additional DNS queries. So it is recommended to use the first one.

No matter which one you use, you must pay attention to the format and value of SPF records, if you make a mistake, you may not be able to send emails. You can check it through MailHardener after you set it up, which is very convenient.

SPF records essentially limit the range of sending servers through IP whitelisting.The IETF has also devised a scheme for limiting the sending range through digital signatures, which is known as DKIM.

DKIM

DKIM is known as DomainKeys Identified Mail Signatures and is defined by RFC6376.

The essence of DKIM is to generate a pair of asymmetric key pairs. The public key is published via a DKIM record in DNS. The server then uses its own private key to sign the content before sending the email. After receiving the message, the recipient queries the corresponding DKIM record, finds the public key and verifies the signature. The recipient can receive the message if it passes the verification, otherwise it will be rejected.

The key pairs here are usually generated automatically by the sending server, and a common one is the RSA-SHA256 signature. An example of a DKIM is shown below:

drill txt default._domainkey.example.org
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 50686
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; default._domainkey.lvht.net. IN      TXT

;; ANSWER SECTION:
default._domainkey.example.org.    300     IN      TXT     "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAyM+Hi+2mk4cvxtTiEdFtXHuRiU0h40yEkPF1hJMQcoPp+SGM5Ikor52X7lNqgXE8zFkjMUMlykltwqdOGBa5ZEG1fRZx2sPRi3q5dXrL6kfUsNSn5V8ZBZFm4FtcIV9c7bPAvfp2ZRnLrkPSQzJ1f6R7XgfxP5JSFzNfpBFanToRXVGlftWyrw6Uro3C366mE" "Bf/n1100JpAecAsiiL1GnLtHAilbo/QRhyI3nxx5Jt0rSFf69S3mrydDWFsmxUTrw9/YGQT7xEL2tk08FTdNj9r9TRXmNpaQnSJxJJWmZ33V+VLOsEyBmiFbjuzD6U6wtSOW2PexKEOPmV0pHwHhQIDAQAB"

The default subdomain is specified under the _domainkey subdomain. The default can be anything you want, as long as it matches the value in the email signature.

The following is an example of a DKIM signature:

DKIM-Signature: a=rsa-sha256; bh=iQ/2DHA4xYuzNkuty4mmtgXpGyW0lOv+tDU9bZ5y9zA=; c=relaxed/relaxed; d=lvht.net; h=Subject:Subject:Sender:To:To:Cc:From:From:Date:Date:MIME-Version:Content-Type:Content-Transfer-Encoding:Reply-To:In-Reply-To:Message-Id:Message-Id:References:Autocrypt:Openpgp; [email protected]; s=default; t=1698495966; v=1; x=1698927966; b=fsRwrjAXpqNbPhBLPLL/cQKe1RNnAcGyBm4QSzs3NNixS1MxWwza0rUs4BAvXfRlDvSoDi6i akpFOMqeZ0PzO8TP0VSJW62BallAAwUO4iEapRsDqh7hj22KY5K2OHvHum75B0UyQTCGEJM4mzr waliehLExSCUQ8dh0C2tNzcwnuNZ54nH5thIF2yKR6M9Ty9lEIn3xnNnUFAjnZ8yn4s+M3iXQid G/R9PZ6k5/QDm9aTMvfTjIJxs8ZM0hwz/uyHsUEU7Z0eYlWd0DjVOaSr/NP/dbNKuCSTut3Vqhr B9PnVKESBY0mSw1QaMKkONpZnvE6xo9JsbUHdf9G7ZMvw==

The format and calculation of signatures will not be discussed in this article. Just pay attention to the s=default in the signature. The recipient will query the DKIM record corresponding to default._domainkey.example.org to read the public key. :

Whether it’s SPF or DKIM, if all of them fail to authenticate, the recipient will reject the mail. There are two possibilities, one is that someone tries to impersonate you to send emails, the other may be that your mailbox system is configured incorrectly. But either way, it’s best that you should notify the system administrator of the domain in a timely manner. And so DMARC was born.

DMARC

DMARC(Domain-based Message Authentication, Reporting, and Conformance).

Example:

_dmarc.example.org.   TXT    "v=DMARC1; p=quarantine; ruf=mailto:[email protected]"

The p here is for policy. administrators can use DMARC to influence the authentication behavior of recipients for SPF or DKIM. In the above example, quarantine means quarantine, which usually means that the recipient will move unverified messages to the trash folder. You can also specify p=reject for reject. If it is changed to p=none, it means not to intervene and leave it to the recipient. The ruf=mailto:[email protected] at the end is a request for the recipient to send regular exception reports so that the administrator of the sending domain can be alerted in a timely manner.

Below is a report email from GMail:

This is the mail delivery system at mx1.lehu.in.

Unfortunately, your message could not be delivered to one or more
recipients. The usual cause of this problem is invalid
recipient address or maintenance at the recipient side.

Contact the postmaster for further assistance, provide the Message ID (below):

Message ID: XXXXXXXX
Arrival: 2023-10-28 10:54:44 +0000 UTC
Last delivery attempt: 2023-10-28 10:54:44 +0000 UTC

Delivery to [email protected] failed with error: gmail-smtp-in.l.google.com.
said: This mail has been blocked because the sender is unauthenticated.
Gmail requires all senders to authenticate with either SPF or DKIM.

Authentication results:
DKIM = did not pass
SPF [example.org] with ip: [XXXX:XXXX:XXXX:XXXX::] = did not pass

...

Receiving Mail

MX

When a user from a foreign domain needs to send mail to a certain mailbox, it will first query the MX record through DNS. Suppose the mailbox address is [email protected]. The sender will first query the MX record of gmail.org:

drill mx gmail.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 393
;; flags: qr rd ra ; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; gmail.com.   IN      MX

;; ANSWER SECTION:
gmail.com.      3467    IN      MX      5 gmail-smtp-in.l.google.com.
gmail.com.      3467    IN      MX      30 alt3.gmail-smtp-in.l.google.com.
gmail.com.      3467    IN      MX      10 alt1.gmail-smtp-in.l.google.com.
gmail.com.      3467    IN      MX      20 alt2.gmail-smtp-in.l.google.com.
gmail.com.      3467    IN      MX      40 alt4.gmail-smtp-in.l.google.com.

A domain name can have multiple MX records, each corresponding to a domain name, and each domain name corresponds to a server or a group of servers, which have different weights, the smaller the value the higher the weight. In the gmail.org example above, the server with the highest weight is gmail-smtp-in.l.google.com, which has a weight of 5. Outside servers that want to send mail to the gmail.com domain will connect to this server first. If it fails to connect, it will connect to other servers in order of weight.

In fact, the most core DNS records needed to receive mail are finished. But for the sake of completeness and systematization of the content, I will talk about MTA-STS additionally here.

MTA-STS

MTA-STS is called SMTP MTA Strict Transport Security. it is called Strict because the previous security measures were very loose. The e-mail system uses the SMTP protocol and communicates on TCP port 25. In the beginning it was all plaintext, not encrypted. Later, when encryption was needed, but to maintain forward compatibility, the protocol was plaintext by default, and then upgraded to TLS encrypted communication via STARTTLS. However, this upgrade may be subject to a man-in-the-middle downgrade attack, which makes both the sender and receiver think that the other side does not support encryption, so they can only use plaintext communication.

To solve this problem, MTA-STS was developed.

First of all, add the following TXT record for the email domain:

_mta-sts.example.org.   TXT    "v=STSv1; id=1"

Here _mta-sts is the fixed prefix, v=STSv1 indicates the version, followed by id=1 for the content version. Before the domain server sends a message to the example.org domain, it queries the corresponding MTA-STS record. If there is one, it will further request its corresponding MTA-STS policy data, which is realized via HTTPS to avoid data tampering by intermediaries.

The HTTPS links corresponding to MTA-STS are also standardized as follows:

https://mta-sts.example.org/.well-known/mta-sts.txt

The format of the return content of this URL is as follows:

version: STSv1
mode: enforce
max_age: 604800
mx: mx1.example.org
mx: mx2.example.org

The id=1 in the preceding TXT record indicates this mta-sts.txt content version. If a new policy is published, the id value in DNS should be updated. As soon as the DNS is flushed, the sender will update its own local cache.

The mode: enforce in the policy requires the sender to upgrade to TLS encrypted transmission. This prevents man-in-the-middle downgrade attacks.

Careful readers may ask why both DNS + HTTPS are used here to publish mtamta-sts.txt. Actually, there is a pure DNS scheme, defined by RFC7672, based on DNSSEC and DANE technologies. However, DNSSEC technology is not yet popular, so we can only use HTTPS to transition to it.

TLSRPT

Similar to the DMARC mechanism, the IETF also defines SMTP TLS Reporting, which is used to report TLS encryption upgrades to senders and receivers. Its DNS record format is as follows:

_smtp._tls.example.org. TXT    "v=TLSRPTv1;rua=mailto:[email protected]"

Here _smtp._tls is the fixed subdomain. The rua=mailto:[email protected] at the end specifies the e-mail address to receive the report. The following is a sample report:

{
  "report-id":"2020-01-01T00:00:00Z_mailhardener.com",  
  "date-range":{
    "start-datetime":"2020-01-01T00:00:00Z",
    "end-datetime":"2020-01-07T23:59:59Z"
  },
  "organization-name":"Google Inc.",
  "contact-info":"[email protected]",
  "policies":[
    {
      "policy": {
        "policy-type":"sts",
        "policy-string":[
           "version: STSv1",
           "mode: enforce",
           "mx: demo.mailhardener.com",
           "max_age: 604800"
        ],
        "policy-domain":"mailhardener.com"
      },
      "summary":{
        "total-successful-session-count":23,
        "total-failure-session-count": 1
      },
      "failure-details": [
        {
          "result-type": "certificate-host-mismatch",
          "sending-mta-ip": "123.123.123.123",
          "receiving-ip": "234.234.234.234",
          "receiving-mx-hostname": "demo.mailhardener.com",
          "failed-session-count": 1
        }
      ]
    }
  ]
}

This contains only overall statistics and does not include private email content. Again, the administrator can determine if the configuration of this domain system is working properly based on the TLSRPT report.

However, MTA-STS is an enhanced configuration, and not adding it will not affect the functionality of the mail system or the receipt of mail. The only pitfall is that the content of users’ communications may be monitored by others.

In the previous two sections, we have talked about the DNS records needed to communicate between different mail servers (MTAs). In fact, the client (MUA) and the server can also set up some DNS records for communication. We will introduce them below.

Reading Mail

There are only two functions for a client to connect to a server, sending and retrieving mail. Sending mail is called SMTP Submission and uses SMTP port 587. There are two types of pickup mail, POP3 and IMAP, POP3 ports are 110 and 995, and IMAP ports are 143 and 993.

These ports (if any) can all be published by adding SRV records for example.org:

_submission._tcp    SRV 0 1 587 mail.example.com.
_imap._tcp          SRV 0 1 143 imap.example.com.
_imaps._tcp         SRV 0 1 993 imap.example.com.
_pop3._tcp          SRV 0 1 110 pop3.example.com.
_pop3s._tcp         SRV 0 1 995 pop3.example.com.

When we configure an email, we just need to enter an email address and the client will try to look up the corresponding SRV record and automatically complete the configuration. If a service does not exist, you can change its corresponding server domain name to ..

For example, there is no POP3 service:

_pop3._tcp          SRV 0 1 110 .
_pop3s._tcp         SRV 0 1 995 .

Summary

Obviously, the mail system is a dizzying array of DNS records alone. Why is it so complicated? It’s because the mail protocol was not designed to be so versatile from the beginning, but rather it grew bit by bit as the Internet evolved. It had to be patched over and over again to ensure compatibility with older systems. Despite this, the email protocol is still the most successful federated communication protocol, and it is the last bastion of the decentralized Internet. If it ever replaces it, then the dream of a decentralized Internet is truly dashed. In addition, the SMTP protocol, as an anonymous delivery protocol, constitutes the most basic trust model of the Internet, that is, first assuming that the other party is a good person, and then making further judgments based on behavior. I think this is the premise of the development of the open Internet.