If I send an email addressed to user@mail.domain.com, should the
outbound SMTP server try to connect based on an A record for
"mail" in the domain "domain.com", or should it do a lookup for the MX
record in the domain "mail.domain.com"?

Hi, 

It is a common mistake to believe a MX record is nessessary for mail
delivery. In the absensce of a MX record, mail to
mymail@mail.domain.com will be sent (and handled by) the
mail.domain.com server. So if you only have one server, say 
domain.com and your address is myaddress@domain.com, you don't really
need a MX record setup at all.

The reason for MX records is to be able to specify that your mail is
going to be handled by another server, and not domain.com. This comes
in very handy for larger websites and servers.


Reference links

support.DynDNS.org
http://support.dyndns.org/whitepapers/mail.php

Preparing for Installations Setting up MX records
https://support.openwave.com/post_office/docs/PO/SOSv35/po35-02D.html

MX-Records
http://www.jhsoft.com/help/rec_MX.htm


Useful Search Query on Google

MX Records

Thanks, 

webadept-ga

Hello there bigjosh.

I'm fairly sure there is no special case here, and mail.domain.com
should be treated like any other name to which email is being sent
(that is, you should do an MX record lookup and iterate through the
list of servers in order of MX preference).

Cheers,

--seizer-ga