One of the things Drupal does is generate emails for a variety of reasons. For example new account registrations include a step to validate the account, by sending an email containing a validation link. On the large online community website I run, a long-running problem has been that the validation email does not always get delivered. This shows up as a user registration that never gets validated. Sometimes they complain and we try to help them but it's kind of frustrating, and it would be better if the email would just go through, but I didn't have time to work out the details until recently.
In the effort to fight spamming various protocols have been invented to add validation to the email system. While I used to be an electric email expert (was on the RFC committee that defined the MIME standard) that was a long long time ago, and I haven't kept up with developments in the email world. It wasn't clear just how many validation protocol bits had to be arranged for email to successfully transit the various ISP's.
I just managed to take care of two things, while my hosting provider has taken care of a third. I hope this is all that's required, will have to see.
First thing is that my hosting provider (Dream Host) already provides DKIM support in all email sent from Dreamhost servers. That's one checkbox ticked off.
The second bit came after I decided to take a look at the headers only to find this:-
Authentication-Results: mx.google.com; spf=neutral (google.com:
208.97.132.5 is neither permitted nor denied by best guess record
for domain of <user>@ps#####.dreamhostps.com)
smtp.mail=<user>@ps#####.dreamhostps.com
Basically this is saying that there's no SPF record. SPF, or Sender Policy Framework, "helps stop spammers from masquerading as you!" Basically what I gather from reading the SPF website is that it is data you put into the DNS to declare which email servers are valid sources of email for your domain.
Unfortunately Dreamhost doesn't directly support adding SPF records into the DNS of domains managed through the Dreamhost control panel. They do support adding TXT records, and it's possible to add SPF information using a TXT record, but there are vague warnings that at some point the universe will transition the TXT record fallback away and only support SPF. Hopefully by then Dreamhost will get off their behind and support SPF records. Or maybe instead I'll get off my behind and transition to another hosting provider.
In any case, adding an SPF record is pretty straightforward. For Dreamhost here are the instructions:- http://wiki.dreamhost.com/SPF
One variant they suggest is this:
v=spf1 ip4:69.163.201.225 include:dreamhost.com include:shaw.ca mx -all
You are to replace the "69.163.201.225" with the IP address of your server.
It was easy enough but it didn't fix the issue by itself. It appeared that the email address in the SMTP envelope was not getting set appropriately. I looked at the SMTP module and rather than install that, I took a look at the implementation of drupal_mail (this is in Drupal 6.x) and realized from careful reading of the documentation for the PHP mail function that something was missing. I changed it to this:
return mail(
$message['to'],
mime_header_encode($message['subject']),
// Note: e-mail uses CRLF for line-endings, but PHP's API requires LF.
// They will appear correctly in the actual e-mail that is sent.
str_replace("\r", '', $message['body']),
// For headers, PHP's API suggests that we use CRLF normally,
// but some MTAs incorrecly replace LF with CRLF. See #234403.
join("\n", $mimeheaders),
'-f<user>@<domain>.org'
);
I added the fifth argument that starts with '-f...' ...
The '-f...' as the last argument gets passed to sendmail to set the MAIL FROM in the SMTP envelope. What'd been happening is the MAIL FROM was defaulting to an email address derived from the user name at the canonical host name of the server. The domain in this case is not related to the domain name of the website, but the domain name of the server.
Adding the -f option forced it to the correct envelope address, which got me this result in the email headers:-
Received-SPF: pass (google.com: domain of
<user>@<domain>.org designates 208.97.132.177 as permitted sender)
client-ip=208.97.132.177;
Authentication-Results: mx.google.com; spf=pass
(google.com: domain of <user>@<domain>.org designates
208.97.132.177 as permitted sender) smtp.mail=<user>@<domain>.org
With that out of the way I had one other item that came up. Namely the rDNS value for the server did not correspond to the domain name hosted on the server. Instead the rDNS value corresponded to the canonical hostname of the server.
This meant modifying the SPF entry to add a ptr: field as so:
v=spf1 ptr:ps#####.dreamhostps.com ip4:69.163.201.225 include:dreamhost.com include:shaw.ca mx -all