Machine Problem 2: Charlie on the MTA

Out: Wednesday, January 30, 2008

Due: Thursday, February 14, 2008

Preliminaries

The reason you're writing an MTA is not to build a replacement for sendmail, but rather to learn about how complex Internet servers are constructed. Initially, we'll start off simple with a single-threaded server that can only process one mail message at a time. It should be able to accept a mail message from a MUA, forward the message on to its destination, and then prepare to accept the next mail message. We'll fix this limitation in the next machine problem.

General Specifications

Requirements

Your MTA must perform two general functions:

1. It will accept mail addressed to any user on its own machine. It will do this by dumping the mail to a log file.
2. It will forward email messages on to another MTA for delivery. The other MTA may be the one at amber.ccs.neu.edu or the MTA at the destination address (see Tasks, below)

You are responsible for the following requirements:

1. Your MTA must support the following SMTP commands: HELO, MAIL, DATA, RCPT, QUIT, and RSET. You neet not support any other commands, but you MTA should respond to them with a legal SMTP error code.
2. Your MTA must be able to properly receive a message from a well-behaved MUA like Outlook Express, Thunderbird, or Eudora.
3. If the received message lacks required headers, as defined by RFC 2822, your MTA must add appropriate headers to make it well-formed. If this is not possible, then your MTA should reject the message with an appropriate reply code.
4. You must support the use of dotted-quad IP addresses (RFC 2822 domain literals) instead of domain names in both the HELO, MAIL, and RCPT commands. In particular, you should be able to send mail to name@[129.10.112.xxx] or the like.
5. You may not use the fork() system call, or its equivalent in whatever programming language you use.
6. You do not need to handle messages with more than 255 recipients. (And don't try this, either! Amber has enough work to do without all you guys flooding it with messages.)
7. You do not need to handle mail messages larger than 65535 bytes.

Non-Requirements

1. If the argument to the RCPT TO: command is not of the form user@domain.name (where domain.name could also be something more complicated, like foo.bar.baz.edu or a dotted-quad IP address like [129.10.112.xxx]), you should refuse to accept the message.
2. You don't need to queue messages that cannot be delivered. Any destination address that is unable to accept the message immediately (after trying all the MX options--see below) should be treated as a delivery failure, and an "undeliverable mail" notification should be returned to the sender immediately. Test this by sending mail to some nonexistent address at ccs.neu.edu.
3. You don't need to handle any other SMTP commands like EXPN, VRFY, HELP, ETRN, NOOP, or VERB. You may simply return an error message if your MTA receives one of these commands.
4. You do not need to handle ESMTP extensions. You do not need to recognize EHLO or any of the additional parameters to standard SMTP commands.
5. You do not need to implement the data transparency procedure described in Section 4.5.2 of the RFC. Basically, you can assume your MTA will never receive a message with a period as the first character of a line.
6. You do not need to implement any sort of loop detection.

Tasks

1. (5 points) Test and debug your MTA by first getting it to accept mail, and then getting it to send mail to yourself at ccs.neu.edu by transferring it to amber.ccs.neu.edu . The easiest way to test this is by interacting with it using telnet, as shown below. Make sure you can send mail to multiple recipients by sending mail to a few friends. For this part of the exercise, you should just contact the MTA at amber.ccs.neu.edu, regardless of where the message is going. Also, for this part of the exercise, you should do minimal header processing: you should make sure that your message contains the headers required by RFC 2822, Section 3.6, but that's all.
2. (10 points). Extend your MTA to do more extensive header processing. Make sure that the messages sent on by your MTA have headers consistent with Section 3.6 of RFC 2822, and make sure that your MTA adds all the headers that an MTA typically adds. Look at the headers of some messages that you've received from outside CCS to see what "typical" means. This part probably requires the most complicated coding in this machine problem.
3. (5 points) Then test your MTA by doing the same thing, but with a well-behaved MUA. Just use your normal mail reader (Eudora, Thunderbird, Outlook Express, etc.) on some handy machine (say, the one you use ssh on) and configure it to use your MTA rather than your default one. To do this, go to the Mail Account Settings and select the Outgoing Server (SMTP) tab in your MUA. Enter the name of the machine your MTA is running on (e.g., 129.10.112.*) and port 2225. Make sure you turn off SSL (Never) and uncheck the user name and password option. Be sure you write down the old settings so you can restore them later.
4. (5 points) Now add to your MTA the ability to do primitive local delivery. You'll need to recognize when a message is destined for your local machine.

   You don't need to anything fancy here, just copy the message to a log file and maybe print it to standard output so you can watch it work. You don't need to worry about multiple users on your machine, etc.

   For testing, you can just just copy the message to a log file and maybe print it to standard output so you can watch it work.

   Now go to your MUA and send some mail to yourself at your VM, using the 129.10.112.* address. Then mail to a friend at his or her MTA.

5. (10 points) Note: The tasks above involve transferring mail only to hosts within ccs. The final task turns you loose on the internet. This task may or may not be required, depending on my negotiations with the systems group. This is an instance where we expose the fact that different systems have different administrators!

   For the last task, add MX processing to your system. Your final MTA should look at each recipient, contact that recipient's MTA (using the MX information for the recipient's domain), and transfer the message to it for delivery. Try this first with an address like denali.ccs.neu.edu (for which the MX agent is once again amber), and then for a few non-ccs addresses. Please only send a few messages outside of CCS; otherwise our systems folks will get some nasty phone calls from the NU network-security folks.

6. (10 points) Now let's get serious about mail delivery. Create three users on your machine. Modify your MTA so that only mail to real users is accepted. Deliver the mail by putting it in each user's mail queue. Each of your users should be able to read his or her mail by using a standard Unix MUA on your machine.

Background: A sample mail transaction

% telnet smtp.ccs.neu.edu 25

Trying 129.10.116.58...
Connected to amber-alias.ccs.neu.edu.
Escape character is '^]'.
220 amber.ccs.neu.edu ESMTP Exim 4.50 Tue, 22 Jan 2008 15:26:59 -0500

HELO foobar.com

250 amber.ccs.neu.edu Hello denali.ccs.neu.edu [129.10.116.200]

MAIL FROM:

250 OK

RCPT TO: 

250 Accepted

550 5.7.1 ... Relaying denied

DATA

354 Enter message, ending with "." on a line by itself

This is a sample message.
It has two lines, followed by a "." on a line by itself
.

250 OK id=1JHPp7-0000II-4Z

QUIT
221 amber.ccs.neu.edu closing connection
Connection to amber-alias.ccs.neu.edu closed by foreign host.

Of course, there weren't any header lines in your message, so the message arrives from no recipient with no subject. This happened to work because amber is very tolerant about mail from denali. It won't be so tolerant about mail from your VM.

Task 4: Mail exchangers

Up until now, you've always contacted a fixed MTA: either amber.ccs.neu.edu or one of the MTAs on the VMs at 129.10.112.xxx.

A real mail transfer agent would take an arbitrary recipient and contact the recipient's destination more or less directly. Unfortunately, you can't simply contact the destination directly, since not every destination runs an MTA service. In that case, you need to contact the destination's mail exchanger, which is a machine (or a set of machines) that receive mail for that destination. This information is stored in so-called MX records in the DNS.

You can see for yourself using the dig mx command:

wand@denali: ~> dig ccs.neu.edu mx

; <<>> DiG 9.2.1 <<>> ccs.neu.edu mx
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50993
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 6, ADDITIONAL: 7

;; QUESTION SECTION:
;ccs.neu.edu.                   IN      MX

;; ANSWER SECTION:
ccs.neu.edu.            3600    IN      MX      10 amber.ccs.neu.edu.
ccs.neu.edu.            3600    IN      MX      50 atlantis.ccs.neu.edu.

;; AUTHORITY SECTION:
ccs.neu.edu.            3600    IN      NS      rivendell.ccs.neu.edu.
ccs.neu.edu.            3600    IN      NS      mcs.anl.gov.
ccs.neu.edu.            3600    IN      NS      amber.ccs.neu.edu.
ccs.neu.edu.            3600    IN      NS      joppa.ccs.neu.edu.
ccs.neu.edu.            3600    IN      NS      tigana.ccs.neu.edu.
ccs.neu.edu.            3600    IN      NS      alderaan.ccs.neu.edu.

;; ADDITIONAL SECTION:
amber.ccs.neu.edu.      3600    IN      A       129.10.116.51
atlantis.ccs.neu.edu.   3600    IN      A       129.10.116.41
mcs.anl.gov.            6180    IN      A       140.221.9.6
joppa.ccs.neu.edu.      3600    IN      A       129.10.116.53
tigana.ccs.neu.edu.     3600    IN      A       129.10.116.83
alderaan.ccs.neu.edu.   3600    IN      A       129.10.116.80
rivendell.ccs.neu.edu.  3600    IN      A       129.10.116.52

;; Query time: 1 msec
;; SERVER: 129.10.116.80#53(129.10.116.80)
;; WHEN: Thu Jan 24 07:18:51 2008
;; MSG SIZE  rcvd: 315

As you can see, MX processing is a complicated business, and we don't intend for you to build one yourself. There are programs available for MX processing in several languages-- go find one and use it. (And tell us where you found it, as you did in MP1.) If you really, really want to use C, Alex Snoeren, from whom I got this problem, has provided some files for doing this.

Turnin procedure

Your deliverables should include a description of your program and its parts, and a description of what header processing (and other processing) you do.

Attribution: If you got portions of your program from external libraries (on the internet or wherever), tell us what they are and where you got them. Otherwise we will assume that you wrote every line yourself, and nasty things will happen if that turns out to be false.

Submit your package as a gzipped tar file. You can build this by doing something like

mkdir mp2
cp {all files} mp2
tar cvf mp2.tar mp2
gzip mp2.tar
mv mp2.tar.gz yourname-mp2.tar.gz