5.1Secure protocols

For a network security architect, "just encrypt it" is not enough. What matters is the ability to work backward from requirements and the threat model to decide which protocol, at which layer, with which options (mode, key-exchange method), fits a given communication requirement (site-to-site VPN, email, web access, remote administration, wireless LAN). This section organizes the representative secure protocols along three axes: the layer they apply to, the protection they provide (confidentiality/integrity/non-repudiation), and their configuration choices.

5.1.1IPsec (transport/tunnel mode, ESP/AH, IKE)

• IPsec is a protocol suite that provides packet-level confidentiality, integrity, and authentication at the IP layer. Transport mode protects only the IP payload and suits end-to-end communication (host-to-host), while tunnel mode encapsulates the entire packet, including the IP header, inside a new IP header, suiting site-to-site VPNs (gateway-to-gateway).
• ESP (Encapsulating Security Payload) provides confidentiality (encryption), integrity, and source authentication. AH (Authentication Header) provides integrity and source authentication only and does not encrypt (AH is also weak against NAT traversal, since NAT rewriting the IP header tends to break authentication). Whether the payload must stay confidential is the deciding factor between ESP and AH.
• IKE (Internet Key Exchange) securely negotiates and agrees on the keys used by IPsec. Built on Diffie-Hellman key exchange, it mutually authenticates both parties and establishes a Security Association (SA). It is also used for periodic key rotation (rekeying), reducing both operational burden and compromise risk compared to manual key management.