Setting up your own email server with hMailServer

Submitted by Dominic Ryan on Wed, 2007-04-11 15:55. Categories:

There are a many email servers for Windows out there that range in price depending on functionality and the software vendor. However, it is not often you find an email server for Windows that is open source (free to use and modify via the GPL license), feature rich, stable and very well supported. There are some notable free offerings such as MailEnable Standard and Windows 2003 Server even comes with a basic POP3 email server, but it seems that they are always just crippled enough to make you want something a bit more. This is where hMailServer comes in as it offers just about everything you could want in an email server. SMTP with advanced routing abilities, POP3, IMAP4, distribution lists, powerful anti-spam tools, real time anti virus protection, PHP/COM based web administration, log file analysis, configurable server messages and rules, advanced performance options. The list goes on, and all this in a RFC compliant package with an easy to use GUI interface. In this guide we'll cover the basics of what you need to do to configure your email environment properly using hMailServer 4.3.1 as your email server. This includes; Configuring you DNS records A (Address) record PTR (Pointer Record) record MX (Mail eXchange) record Considerations for hosting from home Select installation path Select installation type Select built-in or external database Select program group for start menu access Create domain Create postmaster account Create abuse alias for postmaster account Set SMTP host name Configure RFC compliance Configure SMTP relay options to prevent open relay Test against dnsreport Test for open relay

Installing hMailServer

Configure hMailServer

Test your email environment

Configuring your DNS records Before we install your mail server it is a good idea to ensure you DNS records are correctly setup for email. DNS is vital to the operation of the Internet, and in a nutshell what it does is create a link between a human friendly domain name (e.g. example.com) and an IP addresses. When a user enters a web address into their browser, that domain is then resolved to an IP address which is then used to communicate. When hosting your own email server there are three very important types of DNS records to consider, and these are A, PTR and MX records. An A record, or Address record is one of the most common types of DNS records you'll come across. Its purpose is to simply create a link between an IP address and a domain name. For example for the domain name of example.com you would have an A record that contained the IP address of the server that was hosting example.com. With A records the domain must be unique, but the IP address does not. This means you can have as many domain names pointing to the same IP address as you want, and is used extensively these days for both Email and Web hosting. In regards to your Email environment, it is best to create a separate A record that identifies your email server. This can be something like mail.example.com or similar. You'll need to create an A record for each server you plan on using as a mail server. A PTR, or Pointer Record is unique in DNS in that you can only have one PTR address per IP address. This is because the function of a PTR record is to resolve a human friendly name from an IP address, instead of the other way around. Because you are resolving a domain name from an IP address, there can only be one authoritative record. If possible it is best to set your PTR record of the IP your email will be hosted from to be the same as your A record (e.g. mail.example.com), and you may need to contact your hosting company to do this for you. Finally we have the MX, or Mail eXchange record which is used by other mail servers to direct email to the right place. Unlike an A or PTR record an MX record is not associated with an IP address in anyway, but instead contains the human friendly name of the A record you wish to use for your mail server. This might sound a little redundant as all it is doing is pointing from one record to another, but MX records also have one other important function. This is to establish the pecking order of your email servers by using a preference field in which you can enter a numerical value (the lower the number, the more important that server is) to define in what order other email servers should contact your email servers. If the email server with the highest preference (lowest number in preference field) is not contactable, then incoming email servers will simply use the server identified by the MX record with the next highest preference.

Considerations for hosting from home Before we go on it is important to note that if you are wanting to host your own email server from home over a standard ADSL or Cable connection, then there are a few things that may make your environment a little more complex. These are; You'll most likely have a dynamically allocated IP address, meaning that the A record for your mail server will need to be updated everytime your IP address changes. You most likely will not be able to set a PTR record for your IP address, especially if you are using a dynamically allocated IP address. A lot of email servers block email originating from dynamic IP addresses. A lot of ISP's block the standard TCP/IP port numbers used by email servers on home connections. You ISP may well have prohibited hosting your own email (or web) server over your home broadband connection in their terms of service or acceptable usage policy. With exception to the last few points, these issues can usually be pretty easily be overcome. To update your mail server A record every time your IP changes you'll need to use a Dynamic DNS service. These services are all over the web, and will require you to install a Dynamic DNS client on your server which updates your DNS records everytime your local IP address changes. You'll probably won't be so lucky in regards to the PTR record as ISP's usually have their own and will almost never change these for you. A lot of email servers will check for the existence of a PTR record for an IP before accepting any email from it, so as long as you do have a PTR you should be ok. If you are hosting your email server on a dynamic IP address then you'll need to look at using an SMTP relay server (sometime called smart relay) which is usually your ISP SMTP server. With this cause all outgoing email from your server is sent out through your relay server which should be set up properly and will allow you to send email to servers that don't allow incoming email from dynamic IP addresses. Among the trickier problems to solve is that a lot of the time ISP's will block commonly attacked ports on home broadband services to try and stop (or slow at least) the spread of viruses and prevent the saturation of their network. Some ISP's will let you remove these blocks if you ask, or via a web interface. If your ISP has specified in the TOS or AUP that it does not allow the hosting of services over your connection, then your are bang out of luck. My suggestion is to look elsewhere as there are plenty of ISP's that will give you unrestricted use of your connection (as it should be).

Installing hMailServer That is most of the heavy stuff out of the way. From here on in it is all installing and configuring hMail, which is all done via a GUI interface meaning I can use screen shots to do the talking and is hopefully easier to follow. Click any of the images to get a pop-up window displaying a full sized image. After downloading the latest stable hMail server installer package (currently 4.3.1) from the hmailserver website, double click on it to initiate the install process as shown below in figure 1.

Figure 1

Select the installation path for hMail as shown below in figure 2.

Figure 2

Select a full or custom install of hmail (full recommended) as shown below in figure 3.

Figure 3

Select wether to use the MySQL server built into the hMail distribution, or use an external database. If this is a dedicated email server then it is recommended to use the built in database server as shown below in figure 4. However, if you already have MySQL installed (or are planning to) or would like to use Microsoft SQL server then select the external database server option.

Figure 4

Set the start menu program group for hMail as shown below in figure 5.

Figure 5

Confirm your settings as shown below in figure 6.

Figure 6

Click install and hMailServer will be installed as shown below in figure 7.

Figure 7

Once installation is complete, make sure the "Run hMailServer administrator" option is checked as shown below in figure 8 and click finish.

Figure 8

Configuring hMailServer With the installtion of hMailServer successfully completed, the next step is configuration. The configuration steps below show you how to add a domain, add an account, create an alias, setting the server host name, configure RFC settings and configure SMTP relay options to prevent open relay. Start by clicking the Add domain button as shown below in figure 9.

Figure 9

Enter the domain name as shown below in figure 10, and then set the catch-all address. If a mail is sent to an address on your domain that does not have a POP account or alias, then it is redirected to the catch-all address. In this example we have set the catch-all address to postmaster@example.com. From here you can also set the global maximum mailbox size as well as the maximum message size for your domain. Once the domain is created you are also able to access several other tabs to set global settings, but they are not covered in this guide.

Figure 10

With the domain setup, it is now time to create accounts. To be RFC compliant all domains should accept email to a postmaster account, and as we have set postmaster to be the catchall we will now set up an account for it. Place the name of the account in the Account address feild, and then enter a password as shown below in figure 11. You can also set individual mailbox and message size, as well as many other options not covered in this guide such as Active Directory intergration, auto-reply, forwarding, signature and fetching of email from external accounts.

Figure 11

Sometimes it is not practical or desirable to setup an account for every email address you want, and in this case it can be handy to uses an email alias that points to an existing account. In this example we will create an alias of abuse@example.com that points to the

10

postmaster@example.com. Simply enter the alias you'd like to use (in this case abuse) in the Redirect from feild and enter the account and domain ( in this case postmaster and example.com repsectively) in the To feilds. Like the postmaster address, domains are also required to accept email to the abuse email address to be RFC compliant.

Figure 12

Now that we have a domain, account and alias setup lets look at selecting what email services we want to use. Using the navigation window in the left, select and expand the Settings item and then select the Protocols option. Make sure you have at least the SMTP and POP3 servers ticked, as otherwise you will not be able to send or recieve mail. You may not wish to use the IMAP server, but you will need it if you wish to provide webmail functionality to your users.

11

Figure 13

Next step is to set the server host name. In the navigation window on the left, expand the Protocols item and select SMTP. This can be very important as some email servers will not accept email or mark it as spam if the host name does not match the hostname specified in the MX record we set earlier. In the Host Name field enter the full host name you specified in your MX records. In this example we'll use mail.example.com. Note: If you are looking to host your own email server over your home broadband connection, then you will want to enter the name of your ISP's SMTP server in the Relayer field. If your ISP requires authentication, then you'll also need to provide those details in the fields below.

Figure 14

With the host name set we will now set some extra RFC compliance settings. From the page you are on, simply click on the RFC Compliance tabe at the top. It is important that your email server be RFC compliant as otherwise it is likely that many domains will mark your email as spam, and that is if they accept it at all. One of the RFC requirements for email servers is that they accpet a null sender address. You can enable this ticking the Allow empty sender address option. It is also a good idea to enable the Allow incorrectly formatted line endings option as several popular email server packages out their vary slightly in the way they terminate email messages, and without this option set you email server may not be able to recieve emails from them

12

Figure 15 The final step in the basic configuration of your email server is to enure it is not an open relay. An open relay is when a server enables mails to be sent through it to other domains on behalf of domains that do not exist on the local server. Being an open relay is a very quick way in which to get yourself blacklisted, and it can be near impossible to get off these blacklists once you're on it. Luckily hMailServer makes in very easy to prevent this, and in fact by default you should not have to change a thing. Just to be sure though it is always best to check the settings. In the navigation window to the left, select the select and expand the Advanced menu item from under the Settings tree. From here select the IP Ranges option and then select the Internet option. All you have to do here is ensure that the Local to local, Local to external, and External to local options are ticked from under the Allow deliveries from section. Finally also ensure that the To remote accounts option is ticked from under the Require authentication for deliveries section. All these options are shown below in figure 16.

Figure 16

13

Testing the configuration Almost done now, the only thing left to do is test your email environment to make sure everything is configured correctly and ensure it is not an open relay. First stop is to plug your domain in at DNSReport.com which will give you a good overview of how well your email system is setup. If DNSReport finds any issues of concern it will notify you and offer advice on what needs to be fixed. There are several tests available for free on the internet for testing your email server for open relay, and the ones I'd suggest using are the tests at abuse.net and aupads.org. Once you have passed these tests then you should be all clear to go ahead and start creating other account and finally sending and recieving email from your very own email server.

14