* This page is published by the Internet Association Japan (IAJapan) for information sharing about unsolicited commercial Email measure.
Internet Initiative Japan Inc.
Kazu Yamamoto
Mar. 2006
1. The “SPF vs Forwarding” Problem
2. Forwarding and Error Messages
3. Requirements to Solutions
4. Overwriting SMTP MAIL FROM and Loops
5. A Proposed Solution to Prevent Loops (1)
6. A Proposed Solution to Prevent Loops (2)
7. Consideration
8. Supplemental Information
9. Supplemental Information 2
10. Deployment Scenario of SPF
11. More Considerations on Forwarding
1. The “SPF vs Forwarding” Problem
Since SPF depends on IP addresses, SPF authentication results in “fraud” if the relationship between a domain name and its addresses changes because of forwarding. (‘softfail’ if the SPF RR is “~all”, “fail” if it is “-all”.)
In the figure below, alice@example.jp sends a message to bob@example.net and it is forwarded to bob@example.com. Suppose the sending server of example.jp is 192.0.2.1, and that of example.net is 192.0.2.2.
Example.jp is specified in SMTP MAIL FROM and the peer IP address of the SMTP connection is 192.0.2.1. Since they are consistent, SPF authentication on example.net resutls in “real one” (pass). (The green domain name and the green IP address.)
On the other hand, authentication on example.com results in “fraud”. This is because the domain in SMTP MAIL FROM does not change but the peer IP address of the SMTP connection is changed to 192.0.2.2. (The green domain name and the orange IP address.)
2. Forwarding and Error Messages
As background information, we explain the relationship between forwarding and error messages.
If a forwarding receiver is a server that returns an error code, an error message is generated by the forwarder. The error message is returned to the sender. (The red line in the figure below.)
If a forwarding receiver is a server that send an error message by itself, the forwarding receiver receives the message and generates an error message. The error message is returned to the sender. (The red line in the figure below.)
3. Requirements to Solutions
To fix the “SPF vs forwarding” problem, “the SUBMITTER parameter of SMTP MAIL FROM” was proposed.
It proposes to change the format of SMTP. This means that not only forwarders but also forwarding receivers need to adopt the mechanisms. Since it is necessary both for forwarders and forwarding receivers to tackle the deployment in a cooperative way, the deployment of the mechanisms is difficult and takes a time. In the deployment point of view, the format of SMTP should not be changed so that forwarders can independently adopt a solution.
As we will explain below, some kinds of solution introduce loops of error messages. The loops continue until the limit of relay hops and exhausts network resources. Thus, such loops should be prevent.
Here is a summry of requirements for a solution:
- It does not change the format of SMTP so that forwarders can accept the solution independently
- It can prevent loops
4. Overwriting SMTP MAIL FROM and Loops
Under the restriction that the solution does not change the format of SMTP, people must hit upon a solution that a forwarder overwrites the e-mail address specified in SMTP MAIL FROM. Let’s consider this. If the forwarder (example.net) overwirte the e-mail address specified in SMTP MAIL FROM to bob@example.nt, it is obvious that the domain matches to 192.0.2.2. (The orange domain name and the orange IP address.) With this solution, the authentication on the forwarding receiver (example.com) results in “real one”.
Unfortunately, this introduces loops of error messages.
Supporse that an error occurs in a forwarding receiver for some reasons. If the forwarding receiver a server that send an error message by itself, an error message is sent to the forwarder. This error message is forwarded to the forwarding receiver with SMTP MAIL FROM overwritten. This forwarded error message cannot be distinguished from a forwarded message, a loop is generated. The loop continues the limit of relay hops which is defined either by the forwarder or by the forwarding receiver. The consideration in the section 9.3 of the SPF specification stops here.
Note that if a forwarding receiver a server that returns an error code, this loop is not generated.
5. A Proposed Solution to Prevent Loops (1)
Error messages can be distinguished from messages because the value of their SMTP MAIL FROM is <>. As a solution to prevent loops (1) the author proposes to store error messages in the spools of forwarders. It is trivial that loops are not generated.
It is ideal if we can pick up the error messages returned by the forwarding receiver and store them in the spools. But actually it is hard to tell an error message is returned from a forwarding receiver. Thus it is practical to store all error message in the spools of the forwarder.
Supporse,bob uses bob@example.net and send a message, then error message is returned. The error message is the pink line in the figure below, Since this error message is not returned from the forwarding receiver, it would be nice if the forwarder forwards it. If any errors does not occur on the forwarding receiver, bob can read it on the forwarding receiver. Unfortunately, this proposed solution cannot implement it.
Note that if a forwarding receiver a server that returns an error code, the forwarder store messages (not error messages!) to its spools.
6. A Proposed Solution to Prevent Loops (2)
As another solution to prevent loops, the author proposes to forward error messages without overwriting SMTP MAIL FROM. That is we retain SMTP MAIL FROM as <>.
If a forwarder receives an error messages from the forwarding receiver, the forwarder forwards the error message to the forwarding receiver. In SMTP, an error message againt another error message is not generated. Thus, the loop is broken in the forwarding receiver. However, the forwarded error message is silently discarded.
Error messages from other than a forwarding receiver are also forwarded to the forwarding receiver. If no error occurs in the forwarding receiver, the error messages are stored in the forwarding receiver. If any errors occur, the error messages are silently discarded.
Note that if a forwarding receiver a server that returns an error code, the forwarder relays error messages to their sender.
7. Consideration
A merit of the proposed solution (1) is that error messages are certainly retained in the spools of the forwarder. Its demerit is that utility is lost in the sense that a user cannot read error messages on the forwarding receiver.
A merit of the proposed solution (2) is that a user can read error messages on the forwarding receivers if no error occurs. Its demerit is that neither a sender nor a receiver cannot notice that an error occurs since the error message is silently discarded.
Regarding errors occurred in the forwarding receivers, “user unknown” means that the configuration of forwarding is wrong. The mis-configuration should be fixed and it is not an essential problem. A possible fatal error is “spool is full”. If a user chooses the proposal solution (2) and if his spool in the forwarding receiver is full, all further messages are to be lost.
It is said that if we gain something, we lose other things. The proposed solutions is based on that the SPF authentication is more important than error messages. Thus they take the SPF authentication and scarify a part of error messages.
Note that the proposed solution (1) and (2) can be selective in the user level.
8. Supplemental Information
The SPF authentication need to be carried out againt error messages. Otherwise, spammers would send spam messages by using error messages.
The value of SMTP MAIL FROM of an error message is “<>” and no domain name is specified. Thus we need to carry out the SPF authentication against an error mssage using its domain name in SMTP HELO/EHLO .
This means that all sending servers should specify their domain names (not host names) in SMTP HELO/EHLO correctly.
9. Supplemental Information 2
“SRS” is a mechanism to fix the “SPF vs forwarding” problem while maintaining that an error messages returns to its sender. To introduce SRS, only a forwarder needs to support it. Note that SRS is complicated to avoid a security hole of multiple forwarding.
10. Deployment Scenario of SPF
With the proposed solutions to the “SPF vs forwarding” problem and “A Proposal for Deployment of SPF“, the author expects that SPF can be deployed in the world wide. This section descrives Deployment Scenario of SPF.
The Early Stage of the Deployment
- A receiving side always receives messages independent on results of the SPF authentication. The results are recorded in message headers.
- A sending side declare “~” (softfail) as a qualifier of “all” in its SPF RR.
- If a receiving side is a server that returns an error code and if “user unknown” occurs against to a message whose the SPF authentication results in “softmail”, the receiving side does not genearte an error message. This method gives a merit that if the sender side declare the SPF RR the number of error messages to be received drastically decreases.
- A forwarding side adopts the proposed solutions described in this document. They can solve the “SPF vs forwarding” problem. A user on a forwarding receiver can tell the SPF authentication results in “fraud” due to non-adaption of the proposed solutions by investigating message headers.
The Late Stage of the Deployment
- A receiving side decides whether or not it receives messages according to results of the SPF authentication. If “fail”, it rejects messages. If “softfail”, it receives messages (but not generates error messages even if “user unknown”).
- A sending side declare “-” (fail) as a qualifier of “all” in its SPF RR.
11. More Considerations on Forwarding
Suppose that a sending side declares “~all” in the SPF RR in the early stage of the deployment. Let’s consider that a message sent from the domain is forwarded. The forwarding receiver is a server that send an error message by itself and adopts the proposal to deploy SPF. If the forwarder does not adopt the proposed solutions, the SPF authentication in the forwarding receiver results in “softfail”. If his spool is full, an error message is generated and sent to the sender. If the “user unknown” error occurs, no error message is generated. So, neither the sender nor the receiver can tell the error happens. However, the “user unknown” error is caused by mis-configuration of forwarding. When the receiver configures forwarding, he should check the configuration works well.
If a forwarder adopts the proposed solutions, the SPF authentication in the forwarding receiver results in “pass”. If the “user unknown” error occurs, an error message is generated and sent to the forwarder. If his spool is full, an error message is also sent to the forwarder. Operation in this case is described in the “Consideration” section.