Lately, I’ve been working my way through the Hurricane Electric IPv6 certification tests in my spare time. They are relatively straightforward, requiring basic system administration and setting up some IPv6-enabled servers, and can be quite enlightening if you haven’t done much networking administration before. However, the Administrator test requires a working IPv6 mail server, and the last thing I wanted to do was deal with the installation and configuration of a full blown mail transfer agent like postfix.
As usual, this turned into a rabbit hole of its own…
The first problem was simply starting the SMTP DebuggingServer with an IPv6 local address.
Here is testsmtpd.py:
import smtpd import asyncore import sys smtpd.DEBUGSTREAM = sys.stderr server = smtpd.DebuggingServer(("::", 25), None) asyncore.loop()
And its first run:
$ sudo python testsmtpd.py Traceback (most recent call last): File "testsmtpd.py", line 6, in server = smtpd.DebuggingServer(("::", 25), None) File "/usr/lib/python2.7/smtpd.py", line 285, in __init__ File "/usr/lib/python2.7/asyncore.py", line 339, in bind File "/usr/lib/python2.7/socket.py", line 224, in meth socket.gaierror: [Errno -6] ai_family not supported $
Digging into into the smtpd module, Lib/smtpd.py of the Python 2.7.3 codebase, I found (http://hg.python.org/cpython/file/5319a4bf72e7/Lib/smtpd.py#l282) that SMTPServer’s server socket was being created with the IPv4 socket.AF_INET address family hardcoded into the socket() call:
class SMTPServer(asyncore.dispatcher): def __init__(self, localaddr, remoteaddr): self._localaddr = localaddr self._remoteaddr = remoteaddr asyncore.dispatcher.__init__(self) try: self.create_socket(socket.AF_INET, socket.SOCK_STREAM) # try to re-use a server port if possible self.set_reuse_addr() self.bind(localaddr) self.listen(5) ...
which prevents binding that socket later to an IPv6 local address, since the socket needs to be of the IPv6 socket.AF_INET6 address family. It turns out that this is an issue in Python 3 as well, so next came a half-hour detour for filing a bug report documenting this, http://bugs.python.org/issue14758, with a suggestion of using getaddrinfo() with the given local address to provide the appropriate address family, socket type, and address tuple for socket() and bind().
I copied the stock smtpd.py into mysmtpd.py, where I could make these fixes locally and import them in my test SMTP server.
The fixed SMTPServer inside mysmtpd.py, using getaddrinfo():
class SMTPServer(asyncore.dispatcher): def __init__(self, localaddr, remoteaddr): self._localaddr = localaddr self._remoteaddr = remoteaddr asyncore.dispatcher.__init__(self) try: gai_results = socket.getaddrinfo(localaddr, localaddr) self.create_socket(gai_results, gai_results) # try to re-use a server port if possible self.set_reuse_addr() self.bind(gai_results) self.listen(5)
The updated testsmtpd.py:
import mysmtpd import asyncore import sys mysmtpd.DEBUGSTREAM = sys.stderr server = mysmtpd.DebuggingServer(("::", 25), None) asyncore.loop()
At this point, testsmtpd.py could run and listen, but just to make sure it was doing what it was supposed to be doing (that is, receiving mail), I used sendmail() of the underlying smtplib module to write a quick SMTP test client, testsmtpclient.py:
import smtplib server = smtplib.SMTP('eskimo.mooo.com') server.set_debuglevel(1) server.sendmail("email@example.com", "firstname.lastname@example.org", "From: email@example.com\r\nTo: firstname.lastname@example.org\r\n\r\nTest message!\r\n") server.quit()
The test run of the client:
$ python testsmtpclient.py send: 'ehlo localhost.localdomain\r\n' reply: '502 Error: command "EHLO" not implemented\r\n' reply: retcode (502); Msg: Error: command "EHLO" not implemented send: 'helo localhost.localdomain\r\n' reply: '250 beaglebone\r\n' reply: retcode (250); Msg: beaglebone send: 'mail FROM:\r\n' reply: '250 Ok\r\n' reply: retcode (250); Msg: Ok send: 'rcpt TO:\r\n' reply: '250 Ok\r\n' reply: retcode (250); Msg: Ok send: 'data\r\n' reply: '354 End data with.\r\n' reply: retcode (354); Msg: End data with. data: (354, 'End data with.') send: 'From: email@example.com\r\nTo: firstname.lastname@example.org\r\n\r\nTest message!\r\n.\r\n' reply: '250 Ok\r\n' reply: retcode (250); Msg: Ok data: (250, 'Ok') send: 'quit\r\n' reply: '221 Bye\r\n' reply: retcode (221); Msg: Bye $
On the server side, the message comes through:
$ sudo python testsmtpd.py ... Incoming connection from ('2001:masked::2', 35237, 0, 0) Peer: ('2001:masked::2', 35237, 0, 0) Data: 'ehlo localhost.localdomain' Data: 'helo localhost.localdomain' Data: 'mail FROM:' ===> MAIL FROM: sender: email@example.com Data: 'rcpt TO:' ===> RCPT TO: recips: ['firstname.lastname@example.org'] Data: 'data' Data: 'From: email@example.com\r\nTo: firstname.lastname@example.org\r\n\r\nTest message!' ---------- MESSAGE FOLLOWS ---------- From: email@example.com To: firstname.lastname@example.org X-Peer: 2001:masked::2 Test message! ------------ END MESSAGE ------------ Data: 'quit' $
Phew. Now that the SMTP server was working, the next step was to add an MX record for eskimo.mooo.com, so that Hurricane Electric can lookup the mail server when emailing to testemail @ eskimo.mooo.com. This was a matter of adding the MX entry for eskimo.mooo.com at my FreeDNS account, with destination “10:eskimo.mooo.com” (that’s an arbitrary MX priority of 10). I already had the IPv6 AAAA entry for eskimo.mooo.com in the records.
After twiddling my thumbs for a bit, waiting for the new DNS record to propagate, I finally hit the “Send It!” button at Hurricane Electric to give the test email a try.
A spinner icon replaced the button and started spinning, and then it kept spinning, and spinning, and spinning, until it finally gave up with “Failed!”. I checked over at the server console to see:
$ sudo python testsmtpd.py ... Incoming connection from ('2001:470:0:64::2', 42607, 0, 0) Peer: ('2001:470:0:64::2', 42607, 0, 0) Data: 'EHLO ipv6.he.net'
The exchange was simply stopping at the EHLO command. I had noticed before in the SMTP test client output above that the server would respond to EHLO with an error, but the SMTP test client was still completing the transaction:
... send: 'ehlo localhost.localdomain\r\n' reply: '502 Error: command "EHLO" not implemented\r\n' reply: retcode (502); Msg: Error: command "EHLO" not implemented send: 'helo localhost.localdomain\r\n' reply: '250 beaglebone\r\n' reply: retcode (250); Msg: beaglebone send: 'mail FROM:\r\n' reply: '250 Ok\r\n' ... and so on
It seems that Hurricane Electric’s mail delivery service expects the mail server to support Extended SMTP. So, I delved into mysmtpd.py again to implement an EHLO handler in some minimal way that would appease Hurricane Electric. EHLO is a somewhat involved command that returns a list of supported server extensions in addition to a standard 250 <domain> success response, but simply sending back an affirmative seemed like a good start.
The modified SMTPChannel class in mysmtpd.py to reply with a “250 <domain>” success response for EHLO commands:
class SMTPChannel(asynchat.async_chat): ... # SMTP and ESMTP commands ... def smtp_EHLO(self, arg): self.push('250 %s' % self.__fqdn)
I reran the server, hit the “Send It!” button on Hurricane Electric’s page again, and finally, the end to this rabbit hole came into plain sight:
$ sudo python testsmtpdserver.py ... Incoming connection from ('2001:470:0:64::2', 42731, 0, 0) Peer: ('2001:470:0:64::2', 42731, 0, 0) Data: 'EHLO ipv6.he.net' Data: 'MAIL FROM:' ===> MAIL FROM: sender: email@example.com Data: 'RCPT TO:' ===> RCPT TO: recips: ['firstname.lastname@example.org'] Data: 'DATA' Data: 'To: email@example.com\r\nFrom: firstname.lastname@example.org\r\nMessage-ID: \r\nSubject: IPv6 Certification Mail Test\r\nDate: Tue, 08 May 2012 16:12:46 -0700\r\n\r\nPlease insert the following code into the website at http://ipv6.he.net/certification: qvyqxmit8b' ---------- MESSAGE FOLLOWS ---------- To: email@example.com From: firstname.lastname@example.org Message-ID: Subject: IPv6 Certification Mail Test Date: Tue, 08 May 2012 16:12:46 -0700 X-Peer: 2001:470:0:64::2 Please insert the following code into the website at http://ipv6.he.net/certification: qvyqxmit8b ------------ END MESSAGE ------------ Data: 'QUIT'
And that puts me at the HE Certified Administrator level.
The one detail that I omitted above was my delayed discovery of the “mysmtpd.DEBUGSTREAM = sys.stderr” debug file stream setting in the STMP server, which made the whole process take substantially longer than this post makes it seem. For a while, I wasn’t getting any indication of the mail transaction failing at the EHLO command. It became clear that that was the issue once I enabled this debug stream output.
I like wunderground a lot for weather. They’re data-centric and not interested in peddling you with news or advertisements. While I was customizing my shell aliases a few days ago, I realized that it would be nifty to have a very simple command-line weather fetcher. But after spending some twenty minutes looking at a sample RSS feed and parsers, I couldn’t find any off-the-shelf tools to pretty-print the XML and I didn’t want to bother with the parsing myself.
I pretty much left it at that. Except a few days later, a friend suggested wunderground’s API. I figured it wouldn’t hurt to sign up (it’s free). After a short visit to the documentation and admiring their simple yet rich JSON API, I was totally nerd-sniped into writing a command-line weather fetcher in python.
But generic parsing of the relevant response keys wasn’t enough. I decided that the fetcher must also render the icon representing the current weather in the console (which is easy with fabulous).
A screenshot of the result (for two different locations):
wunder.py source: https://gist.github.com/3243785
Unfortunately, I can’t distribute my wunderground API key, as I’m capped to a certain number of requests per day (500 — plenty for my uses, not so much if random people start using it). But as I said, it’s free to sign up for an account to get one.
There are many pages of useful SSH tricks out there, but this one seems to be one of the more comprehensive ones.
Handy for remembering bash syntax when you want to write a quick script with some control structures.