S 5.116 Integration of an email server into a security gateway

Initiation responsibility: IT Security Officer, Head of IT

Implementation responsibility: Administrator, IT Security Officer

Two scenarios are considered regarding the question of integrating an email server into a security gateway: In the first case, it is only about making the email service available for an individual trustworthy network; in the second case, email is to be made available for several trustworthy networks.

Internal email servers are operated in the trustworthy networks for both scenarios. An email server within a trustworthy network is used for administering the alias database that is used to compile the user addresses to a uniform format, possibly for a POP or IMAP daemon, or also as gateway for transition into another email system (e.g. X.400). All internal emails are sent to this server and forwarded to an external email server on the outside, if required.

Using an internal email server is recommendable for different reasons:

• Emails between computers within the trustworthy networks do not leave the networks, since they are processed by the respective internal email servers.
• If the internal email server is simultaneously also used as groupware server, this may result in an unnecessarily high load of the ALG.
• In this way, a groupware server is provided with enhanced protection against attacks from the outside, since the distance to the untrustworthy network is greater.

However, it is recommendable to also additionally protect the email and/or groupware servers in the internal network against unauthorised access from the internal network with packet filter rules as a minimum. This corresponds to installing the server in a separate DMZ of the internal packet filter. This should absolutely be performed in the event of particular security requirements in the internal network.

As opposed to the web server, where the recommended installation is "as decentralised" in the security gateway as possible, the installation recommended herein requires email servers to be installed "as centrally as possible". The reason for this is that internal emails can still be sent even if the internet connection fails.

Connection of an individual trustworthy network

If an email server is only to be used for a single trustworthy network, the internal email server is sufficient. In this case, the ALG acts as "smart host" for the internal email server.

A smart host is a computer all emails of a network are passed over. If the ALG is configured as smart host for the internal email server, the internal email server does not have to determine the email server of the receiving domain for outgoing emails, but simply forwards the emails to the smart host assuming the task of determining the proper receiving email server. This may also be a smart host (for example with the internet service provider); if the ALG is used as a smart host for the internal network, this will normally be the case. Smart hosts are occasionally also referred to as email relays.

For incoming emails, the ALG either acts as email exchanger receiving all incoming emails from the sending email servers and forwarding these to the internal email server or as smart host for an external email exchanger.

Connection of several trustworthy networks

If one security gateway is used jointly for several trustworthy networks, for example as joint internet access for several locations of an organisation, the above-described design is often no longer feasible.

For this scenario, emails should be sent and received in two stages: Separate email servers should still be used in the trustworthy networks that can be used to directly send internal emails. Additionally, it makes sense to use a central email server in the DMZ acting as central email exchanger for the trustworthy networks and that can be used to process external emails. Depending on the product, such an email server in the DMZ may already be integrated in the ALG.

The following figure illustrates such a design with two trustworthy networks with one internal email server each that are connected to an untrustworthy network (for example the internet). The two internal email servers are responsible for different (sub-) domains, i.e. the email server in the DMZ decides which internal email server incoming emails are forwarded to.

Figure: Placement of internal MTAs and the email server for connecting two trustworthy networks (at the interface of the ALG with the DMZ, two SMTP proxies must be configured, e.g. using virtual IP addresses).

Incoming external emails pass the MTAs as follows:

• MAT in the untrustworthy network (with the sender or the internet service provider)
• MTA in the DMZ: The latter makes the decision as to which of the two trustworthy networks (and/or which MTA) the email must be forwarded to.
• Email servers in the respective trustworthy network

Outgoing emails pass the MTAs in reverse order.

Email servers for simple security gateways

If only a simple security gateway consisting of one packet filter is used, it is recommendable to locate the email server in a DMZ of the packet filter. Due to the missing ALG, the protection of the email server against being compromised from the outside is lower. However, locating the email server in the DMZ provides for a somewhat higher protection of the internal network if the email server is compromised compared with the server being placed directly in the internal network.

If it must still be possible to send internal emails even if the external (internet) connection fails, the email server may be relocated to the internal network and an additional email server (MTA) may be placed in a DMZ of the packet filter that will act as external email exchanger. To some extent, this solution is a combination of the more complex solutions described above.

Review questions:

• Is the email server protected against unauthorised accesses from the internal network by at least a packet filter?
• Are emails to external addresses routed via a separate email relay?
• Do the (external and internal) communication links to the email server correspond to the organisation's security policies?