* This page is published by the Internet Association Japan (IAJapan) for information sharing about unsolicited commercial Email measure.
Internet Initiative Japan Inc.
Kazu Yamamoto
Jan. 2006
1. Abstract
2. Problems deploying SPF
3. Background Information
4. Proposal
5. Consideration
1. Abstract
In order to deploy SPF (Sender Policy Framework), this document proposes the followings:
- The semantics of “~” is that messages must be accepted by the recipient but error messages do not have to be generated.
- “~” is used as the qualifier for “all” in an SPF Resource Record (RR) in the DNS (if “-” cannot be selected). “?” should not be used.
- Do not send an error message if the recipient user was not known and the SPF check of the original message resulted in “softfail” or “fail”.
2. Problems deploying SPF
SPF is a method for an email SMTP receiver to authenticatate the SMTP sender based on pre-agreed special data in the DNS domain of the sender.
When using SPF, the IP address of each authorized SMTP sender for a domain is on the right hand side of an SPF RR in the DNS for the sending domain.
A receiving server extracts the sender’s domain from the email address specified in the SMTP MAIL FROM (or HELO/EHLO), and can then verify IP addresses of the sending server by looking up the SPF RR in the DNS. The receiving server compares the IP address of the SMTP sender with the IP addresses in the SPF RR. If it matches, the authentication results in success. Otherwise, the authentication fails.
The SPF mechanism consists of an SPF RR in the DNS zone of the sending side and verification by the receiver. A site (or domain) can introduce them independently. Roughly speaking, the ratio of messages which are able to be authenticated does not increase unless the number of sending domains which have SPF RRs increases. For SPF deployment, we need to increase this; however, the following obstacles are pointed out:
- There are few merits for declaration of an SPF RR while SPF is not widely deployed. Also, there is a risk that messages from a site are not accepted by other sites if the site mis-configures its SPF RR. Comparing the merits and the risk, sites are prone to hesitate to declare SPF RRs.
- Even a site declares an SPF RR, the site is prone to select “?” as the qualifier of “all”. This means that the SPF RR does not exist if authentication matches “all”. Thus, this loses utility for receiving side.
This document proposes a modification to SPF where the number of error messages generated decreases if a sending site declares an SPF RR. Currently, most error messages are generated due to faked email addresses and these error messages cause receiving servers to over-load. Reducing this burden would be a big win. The author hopes that the number of domains which declare SPF RRs will increase because of this protocol change.
3. Background Information
For clarity of this document, we first explain “harvesting”.
Harvesting is the collection of email addresses with the intent to use them in SPAM messages. A first method, which we can easily hit upon, is to collect email addresses on open web pages. There is another way to see whether or not users exist by sending messages to the email addresses. The list of messages can be generated randomly or in a brute force manner, made use of biographic dictionary, or collected from web pages.
The second one typically carries out the following procedures. First a faked email address is specified through SMTP MAIL FROM. Then, one email address out of the list is specified in SMTP RCPT TO. If the mail server returns a 250 OK reply, the email address can be treated as active. After then, SMTP RCPT TO is repeated as many times as possible to evaluate all the email addresses in a list.
To deter the harvesting attacks, some receiving servers are configured to return a 250 OK always even if an email address specified in SMTP RCPT TO is unknown. This kind of receiving servers generate error messages by themselves. Since the vast majority of email address specified in SMTP MAIL FROM are faked, returning an error message to the faked address annoys the receiving domain. In fact, massive error messages due to harvesting are considered DoS against receiving servers.
Also, unnecessary messages are generated due to viruses. Most messages sent by viruses have faked email addresses. If a receiver for a message sent by a virus does not exist, an error message returns to the faked sender. If a receiving server uses an anti-virus product which reports detection of viruses, a report is sent to the faked sender.
4. Proposal
To modify SPF to reduce error messages, the author proposes the followings:
- The semantics of “~”: Messages must be accepted (must not be rejected). Harvesting-proof receiving servers need not to return error messages.
- Behavior of harvesting-proof receiving server: When it receives a message, it stores the result of the SPF check. When an error message is being generated due to user unknown or other reasons, it refers to the result of the SPF check. If the result is either fail or softfail, it does not generate an error message.
- Declaration of SPF RR: For the qualifier of “all”, either “~” or “-” is used. Initially “~” is probably preferred. Domains which already use “?” should change it to “~” as soon as possible.
With this proposal, if a domain declares an SPF RR in the DNS, the number of error messages will decrease.
5. Consideration
Most error messages are generated due to harvesting and viruses and thus are meaningless. There would be benefit, as opposed to harm, in reducing these meaningless error messages.
If the number of useless error messages can be minimized, the number of domains which declare SPF RRs would increase. If the number of domains, which do not generate error messages in response to faked messages, increases, error messages will be reduced even more. With this positive feedback cycle, the author hopes that SPF will deploy both on receiving side and sending side.
SPF is not compatible to forwarding of SMTP level. If a message is delivered with forwarding, the SPF check results in failure. But if “~”, defined in this document, is declared, the message itself is certainly accepted by the recipient. If the recipient of forwarding address does not exist, a legitimate error message does not return. In this case, however, it is certain that the configuration of forwarding has an error and thus the mis-configuration should be fixed.