IPSec Overview


Some Background.  IPSec - IP Security is partially described by the IETF - Internet Engineering Task Force as

a framework of open standards for ensuring private,
secure communications over IP networks,
through the use of cryptographic security services.

The framework is described in RFCs 2401 - 2412.

IPSec is not bound by any particular authentication or encryption method or algorithm.  Older security measures were usually implemented at the application layer of the OSI model, which made much of their use application dependent.  IPSec has been implemented at the network layer of the OSI model so that it is not application dependent.  So IPSec does not need each application to be configured to IPSec standards.

Modes.  IPSec can be implemented in two different modes.

  • Transport Mode
    • only the data is encrypted
    • since IP headers are not encrypted the packets are smaller than packets in tunnel mode
    • typically used in host-to-host VPNs
  • Tunnel Mode
    • data and IP headers are encrypted
    • due to additional encryption packets are larger than packets in transport mode
    • somewhat slower than transport mode since packets are larger
    • more secure
    • typically used in host-to-gateway or gateway-to-gateway VPNs

Security Protocols.  IPSec relies on two different security protocols.

  • AH - Authentication Headers
    • authenticates packets by signing them
      • signature is specific to each packet to help ensure validity of the data source
    • ensures the integrity of the data
  • ESP - Encapsulating Security Payload
    • also assists authentication and integrity of payloads
    • helps ensure confidentiality through encryption

IKE and ISAKMP.  All transmissions across the Internet can be sniffed.  There need to be ways to ensure that communications can be hidden.  Encryption is one of the main approaches in use today.  The most well known approaches to encryption rely on some shared knowledge and/or keys to make encryption work.  There must be some sort of system to authenticate users and manage secret keys.

IPSec makes use of IKE - Internet Key Exchange to authenticate both ends of a secured tunnel before an IPSec transmission begins.   IKE makes sure that there is a secure exchange of a shared key to authenticate a secured tunnel.

IKE relies on both users having a pre-shared password or key.  During initial negotiations to authenticate the users, both users swap a hashed version of this pre-shared key.  Exchanging a hashed version makes it much more difficult for an outside user to discover this key.  For example, this hash may be based on something like the time the exchange was first initiated.  If both ends of the tunnel successfully recreate the hash then the exchange can begin.

IKEv2 increases reliability of this process by making use of sequence numbers and ACKs.  But making use of these enhancements implies each end has certain additional processing capabilities such as error processing and shared state management.  Due to these additional criteria IKEv2 can sometimes end up in a dead state where both ends wait for the other to initiate an exchange that never happens.  Dead-Peer-Detection capabilities have been added to ensure this sort of eventuality doesn't occur.

IPSec can also rely on digital signatures.  Digital signatures are provided by reliable third parties called CA - certificate authorities.  A CA provides authentication and nonrepudiation where a sender cannot deny that a message came from them.

One strong advantage to using IKE over some sort of manual method is that an SA - secure association can be established when needed.  An SA can also be set to automatically expire at some time.  RFC 2408 develops the ISAKMP - Internet Security Association and Key Management Protocol framework for establishing, negotiating, modifying and deleting SAs.  ISAKMP centralizes the authority for establishing such exchanges which helps reduce duplication of functionality.  It also reduces the amount of time to setup SAs by negotiating all of the services at the same time.