S 5.76 Use of suitable tunnel protocols for VPN communication

Initiation responsibility: Head of IT, IT Security Officer

Implementation responsibility: Administrator

Securing the data connection

When a LAN is accessed using a virtual private network (VPN), access is typically obtained via an external data connection. For example, the network of a telecommunication provider is used when a user dials in directly to the VPN (direct dial-in). If the connection is established using the internet, the data passes through the network of the corresponding internet service provider (and possibly the networks of the provider's cooperation partners). Since direct connections to a LAN are established using a VPN connection, the network path used for data transmission must be secured in such a way that the confidentiality, integrity, and authenticity of the data is guaranteed. The route is secured by encrypting and, if necessary, signing the data packets exchanged after the communication partners have been authenticated (see also S 4.34 Using encryption, checksums or digital signatures). Various methods and mechanisms for securing the communication connection have been developed for the VPN environment, for example tunnelling.

Amongst other things, the following factors may have an impact on the selection of the procedure to be used to secure a VPN connection:

The following applies in general:

The security mechanisms are based on different cryptographic procedures. Safeguard S3.23 Introduction to basic cryptographic terms contains a short introduction to basic cryptographic terminology.

Encrypting protocol connections: Tunnelling

When an encrypted data connection is established between two communication partners, the resulting connection is a "secure channel". Any data can be transmitted securely via this channel using the underlying communication protocol (IP, for example). If the transmitted data itself represents the data packets of a communication protocol, the channel is referred to as a "tunnel".

The protocol used to encrypt the data, to transmit the encrypted data through the tunnel, and to manage the connection is also referred to as a tunnel protocol. Tunnel protocols may differ in terms of the following properties:

The tunnel protocol is primarily responsible for the following:

When selecting the VPN hardware and software to be used, it should be ensured that it supports several different and established encryption procedures at a minimum. This increases the probability that the client and the server will be able to negotiate a suitable method.

Overview of common tunnel protocols

The following tunnel protocols have become established protocols for VPN environments:

The protocols possess the characteristics shown in the following table:

Tunnel protocol Layer Transported protocols Underlying protocol needed Number of tunnels supported Tunnel authentication
PPTP 2 IP, IPX, NetBEUI IP 1 No
L2TP 2 IP, IPX, NetBEUI IP, X.25, Frame Relay, ATM several Yes
IPSec 3 IP IP 1 Yes
TLS/SSL 4 IP, HTTP, SMTP, etc. IP several Yes

Table: Protocols

One important aspect in actual applications is that the tunnel protocols selected and the cryptographic procedure specified must be supported by all endpoints of the tunnel. The following briefly describes the most common tunnel protocols.

PPTP (Point to Point Tunneling Protocol)

PPTP is a tunnel protocol in Layer 2. It is used to establish Point-to-Point Protocol (PPP) connections using an IP network. PPP connections established this way can be used to transport (tunnel) IP packets, for example. The security functions for authentication, key management, and encryption are provided by PPP, often through the use of the Microsoft Point-to-Point Encryption protocol (MPPE). The term PPP is often used, though, to refer to the actual PPTP or to the PPTP/PPP/MPPE combination.

Security gaps were discovered in common implementations of this tunnel method, especially in connection with weak passwords. For this reason, PPTP should not be used as a VPN solution without additional security mechanisms.

L2TP (Layer 2 Tunnelling Protocol)

Like PPTP, L2TP Version 2 (L2TPv2) is used to establish PPP connections using packet-based networks. In contrast to PPTP, though, L2TP can use technologies other than IP such as ATM for the carrier network. In this case, L2TP uses mechanisms in the L2F (Layer 2 Forwarding) protocol developed by the Cisco corporation to implement the tunnel functionality.

L2TP itself does not offer any functions for the encryption of the data packets. Such encryption must be performed either by the carrier network or by the protocols transported. L2TP is therefore often used in combination with IPSec (see below).

IPSec (Internet Protocol Security)

IPSec is a protocol in Layer 3 offering functions for the encryption and for securing data integrity in IP communications. When used in combination with the Internet Key Exchange procedure (IKE, formerly referred to as ISAKMP/Oakley), it is also possible to automate the key exchange and authenticate the endpoints of the tunnel. The Simple Key Management for Internet Protocol (SKIP) procedure from Sun Microsystems can also be used to exchange keys for IPSec communication. Manual key exchange is also supported by IPSec. However, the users must be authenticated using different methods in this case.

IPSec is a complex protocol that offers several different options and operating modes. In addition, the cryptographic procedures used are not conclusively defined in the specification and it only lists minimum requirements for the procedures to be used instead. It is therefore necessary when using IPSec to ensure during the configuration phase that the security requirements for the application at hand are fulfilled and suitable cryptographic procedures are used. Additional recommendations on this topic can be found in safeguards S 5.149 Secure connection of an external network with IPSec and S 2.164 Selection of a suitable cryptographic procedure.

TLS/SSL (Transport Layer Security, Secure Sockets Layer)

TLS/SSL is a widely used method for providing transport security for web applications or email transmissions. On the one hand, different application protocols such as HTTP, SMTP, POP, or IMAP can be transported via TLS/SSL, for example. On the other hand, it is also possible with the help of special software components to establish IP tunnels over TLS/SSL. Due to its principle of operation, TLS/SSL cannot be associated with a specific protocol layer even though it is often referred to as a Layer 4 protocol.

TLS/SSL offers security functions for authentication and encryption as well as for exchanging keys and protecting integrity. Like with IPSec, the cryptographic procedures necessary are not conclusively defined within the specification. Instead, the communication partners involved negotiate the procedures to be used while establishing the connection. It must therefore be ensured during the configuration phase that the security requirements are fulfilled and that suitable cryptographic procedures are used. Additional information on this topic can be found in safeguards S 5.148 Secure connection of an external network with OpenVPN, S 5.66 Use of TLS/SSL, and S 2.164 Selection of a suitable cryptographic procedure.

Other tunnel protocols

It is not only possible to develop VPN solutions using the tunnel protocols mentioned above, but also using other methods. One example includes the use of OpenSSH for the purpose of establishing a VPN. OpenSSH was primary developed as an encrypted replacement for telnet, ftp, and the r-services, but it can also be used to secure VPN connections.

Furthermore, there are products offered on the market that use proprietary tunnel and encryption procedures. The use of proprietary procedures should be avoided, though, because it is often very difficult to evaluate their security properties. Procedures based on common standards and publicly available specifications should be used instead.

Review questions:

© Federal Office for Information Security. All rights reserved.