Master in Cybersecurity: Secure Communications

Lesson 4: IPsec

Course 2022-2023

Antonio Pastor [email protected]

 Lesson outline

1. IPsec introduction:
v Objectives & benefits.
v IPsec Virtual Private Networks (VPNs).
v IPsec Transport and Tunnel Modes.
v IPsec Security Associations (SAs):
v Security Association Database (SAD).
v Security Policy Database (SPD).
v IPsec framework.
2. IPsec Security Protocols:
v Authentication Header (AH).
v Encapsulating Security Payload (ESP).
3. Internet Key Exchange (IKE):
v Phase I.
v Phase II.

2
 IPsec objectives (I)

u Why do we need IPsec?

v IPv4 has no confidentiality:
ü Eavesdropping.
v IPv4 has no integrity protection:
ü Payload could be changed without detection:
ØHeader checksum can be recomputed (i.e. not a HMAC).
v IPv4 has no authentication:
ü IP address spoofing.
v Denial of service (DoS) attacks:
ü Attacker cannot hold accountable due to the lack of authentication.

• IPsec works at the network layer, protecting and authenticating IP

packets:
- It is a framework of open standards, which is algorithm-independent.
- It provides data confidentiality, data integrity, and origin authentication.
3
 IPsec objectives (&II)

u Most security solutions are application-specific:

v TLS for web, S/MIME for email, SSH for remote login.
v But not all applications need to be security-aware.

u IPsec aims to provide a framework of open standards

for secure communications over IP:
v Originally designed for IPv6, then backported to IPv4.

u IP layer security mechanism for IPv6 and IPv4:

v Provides authentication, integrity and confidentiality.
v Protects any protocol running on top of IP. E.g.:
ü Is this Router Advertisement or ICMP Redirect authentic?
ü Is this OSPF LSA or BGP update valid?
v Can be transparent to users (but not to admins):
ü Secure Virtual Private Network for tele-workers or between sites.

4
 Benefits of IPsec

• Offers Confidentiality (encrypting data), Integrity and Authentication.

v Supports different cryptographic algorithms (a.k.a. crypto-agility).

• Data integrity and source authentication*

v Data “signed” by sender, and “signature” is verified by the recipient.
v Data modification can be detected by signature “verification”.
v Since “signature” is based on a shared secret, it also authenticates source
(but does not provide non-repudiation).

• Anti-replay protection:
v Acceptable sliding window of W packets, since IP is best effort (packets can
be lost, duplicated or reordered).

• Key management:
v Session negotiation and establishment.
v Remote peer is authenticated through varying options.
v Secret keys are securely exchanged.
v Sessions are re-keyed or deleted automatically.
*network to network authentication. You aren’t authenticating a particular user but anyone that could be using that device 5
IPsec in the TCP/IP stack

6
http://www.tcpipguide.com/free/t_IPsecModesTransportandTunnel.htm
 IPsec VPNs

Remote site Central site

Router with
VPN Client
Internet
Corporate

Intranet

Remote User
Extranet
Business-to-Business VPN
software
Client

• Secure Virtual Private Network (VPN):

– For connecting remote offices and users using public Internet.
• Low-cost remote access:
– e.g. teleworker gains secure access to company network via cheap Internet access.
• Extranet connectivity:
– Secure communication with partners, suppliers, etc.
7
 IPsec VPN models

Internet
Untrusted Network

Host-to-Host (no VPN)

IPsec Gateway
Internet
Untrusted Network

Trusted Network
“Remote access” VPN model: Host-to-Gateway

IPsec Gateway IPsec Gateway

Internet
Untrusted Network

Trusted Network Trusted Network

“Branch-to-branch” VPN model: Gateway-to-Gateway
8
 IPsec modes

Two IPsec modes: Transport and Tunnel modes

Transport Mode
IP header Data

Original IP IPsec ESP

Data
header header Tunnel Mode

Optional Encryption

New IP IPsec ESP Original IP

Data
header header header

Optional Encryption
Outer IP header

Inner IP header 9
 Application of IPsec modes (I)
Host-to-Host (uncommon):
Host Host
Internet
Untrusted Network

Can use Transport (or Tunnel) mode between hosts

Gateway-to-Gateway (e.g. branch-HQ):

IPsec Gateway IPsec Gateway
Internet
Untrusted Network

Trusted Network Trusted Network

Can ONLY use Tunnel mode between IPsec gateways

Hides IP addresses of trusted networks (e.g. private addressing) 10
 Application of the IPsec modes (&II)

Host-to-Gateway (Remote access):

IPsec Gateway
Internet
Untrusted Network

Trusted Network

SHOULD use Tunnel mode between host and gateway:

- Hides IP addresses of trusted networks
- Allow remote host to join trusted network

11
 Security Association (SA)

Security Association (SA):

• All the information shared between two IPsec
systems to establish an unidirectional secure
communication:
– Selection of the security mechanisms:
• ESP or AH protection.
• Cipher algorithm.
• Hash function.
• Authentication method.
– Authentication of the two parties.
– Negotiation of ciphersuite and authentication keys.

12
 IPsec Security Databases

• A model to ensure a minimum of interoperability.

• RFC 2401 - “Security Architecture for IP”

• Two Security Databases maintained by IPsec system:

– Security Association Database (SAD)

– Security Policy Database (SPD)

13
 Security Association Database (SAD)

Security Association Database (SAD):

• All active Security Associations (SAs)
• For each SA entry, includes:
– Identifier:
• Outer destination IP address.
• Security Protocol (AH or ESP).
• Security Parameter Index (SPI).

– Parameters:
• Authentication algorithm and key.
• Encryption algorithm and key.
• Lifetime.
SAD • Security protocol mode: Tunnel or Transport.
• Anti-replay service.
• Link with an associated policy in the SPD.
14
 Security Policy Database (SPD)

Security Policy Database (SPD):

• Applies to every inbound or outbound packet.
– Differs from firewall packet filter source/destination
• Multi IPSec demands one SPD per interface
• For each policy entry, includes:
– Selectors (5-tuple):
• Name (assigned by OS)
• Remote IP address/subnet.
• Local IP address/subnet.
• Transport Layer Protocol (protocol number)
• Source and Destination Ports
– Policy:
• Discard the packet, bypass or protect with IPsec.
• Link to an active SA in the SAD (if it already exists)
for IPsec Processing:
- Security Protocol and Mode.
SPD - Enabled Services (anti-replay, authentication,
encryption).
- Algorithms (for authentication and/or encryption). 15
 Example of GW Security Policy Database

• SPD of Gateway A in interface 2

Protocol Local IP Port Remote IP Port Action Comment
UDP 2.2.2.2 500 3.3.3.3 500 BYPASS IKE
* 1.1.1.0/24 * 1.2.1.0/24 * PROTECT Protect
ESP tunnel to (VPN traffic)
3.3.3.3
* * * * * BYPASS Internet

IPSec Gateway A IPSec Gateway B

Internet
Interface 1 Interface 2 Interface 1 Interface 2
1.1.1.1 2.2.2.2 3.3.3.3 1.2.1.1

Intranet Intranet
1.1.1.0/24 1.2.1.0/24

16
 Example of host Security Policy Database

• Corporate network: 1.2.3.0/24

• DMZ network (secure LAN): 1.2.4.0/24
• Host 1.2.3.101 is authorized to connect securely to server 1.2.4.10
and internet
17
 Inbound IPsec packet processing
IPsec System

IP IP
IPsec Header
Header

Destination
IP address
Security Protocol (AH / ESP)
SPI

SAD SPD

1. Identifies the SA 3. Performs the enabled 4. Identifies the security

in the SAD upon IPsec services: policy according to
the identifiers. - Authentication. the selector.
- Decryption.
2. Read the SA - Anti-replay service. 5. Check the policy
parameters. and apply if needed.
18
 Outbound IPsec packet processing
IPsec System

IP IP
IPsec Header
Header

Policy
Selectors

SAD SPD

4. Read the SA 1. Identifies the policy in the SPD

parameters specified according to the selectors.
by the link.
2. Read the policy parameters.
5. Computes the
IPsec processing. 3. Initiate new SA if necessary.
19
 IPsec framework

• IPsec is not a single protocol. Instead, IPsec provides a set of security algorithms
plus a general framework that allows a pair of communicating entities to use
whichever algorithms to provide security appropriate for the communication.

20
 Confidentiality

• IPsec ensures confidentiality by using encryption, which prevents third parties from
reading the data.

Least secure Most secure

Key length:
- 56-bits

Key length:
- 56-bits (3 times)

Key lengths:
-128-bits
DH7
-192 bits
-256-bits

Key length: Key length:

- 160-bits - 256-bits

21
 Integrity

• IPsec ensures data ingrity, avoids tampering, by using HMACs (Hashed Message
Authentication Code).

Least secure Most secure

Digest length:
- 128-bits

Digest length:
- 160-bits

Digest length:
- 256 bits
- 512 bits
22
HMAC for AH

23
 Authentication

• IPsec ensures that the connection is made with the desired peer by using IKE,
based on Pre-Shared Keys (PSKs) or digital certificates/public cryptography.

24
 Pre-Shared Key (PSK)

• Authentication key and the identity information are hashed by local device to form
hash_I. One-way authentication is established by sending hash_I to the remote
device. If remote device can independently create the same hash, the local device is
authenticated.
• The authentication process is also applied in the opposite direction (hash_R).

25
 Public Key Signatures

• Authentication key and identity information are hashed at local device forming hash_I, which
is encrypted using the local device's private encryption key creating a digital signature. The
digital signature and digital certificate with associated public encryption key are forwarded to
the remote device. The remote device verifies the digital signature by decrypting it using the
public encryption key, so the result is hash_I.
• The remote device independently creates hash_I’ from stored information. If the calculated
hash_I’ equals the decrypted hash_I, the local device is authenticated. Then, the
authentication process begins in the opposite direction. 26
 Secure Key Exchange

• Diffie-Hellman (DH) groups determine the strength of the key exchange process.
• Higher group # are more secure, but require more resources to compute the key.

Least secure Most secure

768-bit 1024-bit 1536-bit 163-bit EC 2048-bit 256-bit EC

modulus modulus modulus field size modulus field size
27
https://weakdh.org/
 IPsec Security Protocols

Authentication Header (AH):

R1 R2
All data is in plaintext.

AH provides the following:

§ Authentication
§ Integrity
§ Anti-replay

Encapsulating Security Payload (ESP):

R1 R2
Data payload is encrypted.

ESP provides the following:

§ Confidentiality
§ Authentication
§ Integrity Like AH à ESP has phased out AH
§ Anti-replay
28
 IPsec AH format
• AH: IPv4.Protocol = 51
• AH: IPv6.Next Header =51

Next Hdr is IP in Tunnel Mode, upper protocol (e.g. TCP, UDP, ICMP) in Transport Mode
Notice: No IP value in the AH header ( compatible for IPv4 & IPv6.) 29
 Authentication Header (AH)

1. The IP Header and data payload are hashed

IP Header*⁺ + Data + Key R2

Hash (HMAC)
AH
IP Hdr. AH Data
Authentication Data IP Header + Data + Key
(00ABCDEF)
3. The new packet is
Internet
transmitted to the Hash (HMAC)
IPsec peer router
IP Hdr. AH Data
Recomputed Received
2. The sender builds a new AH Hash Hash
header that is inserted into (00ABCDEF)
= (00ABCDEF)
R1
the original packet 4. The peer router hashes the IP
header and data payload, and
⁺ Include AH except Authentication data field (zeroed) compares with the transmitted hash
30
* Except mutable fields: ToS, Flags, Offset, TTL, Checksum. This fields are zeroed. Details in RFC4302
 IPsec Anti-Replay Window
Fixed window size Sequence Number
(default is W=64)
employed by receiver

N: Highest seq.
number for a
valid paket
recevied so far

• If received packet falls in the window:

– if authenticated and unmarked, mark it
– if marked, then duplicated (or replay!)
• If a received packet is > N:
– if authenticated, advance the window so this packet is at the rightmost edge and mark it.
• If a received packet is ≤ N - W
– old packet is discarded; this is an auditable event.
• If sequence number overflows (after 2^32 IPsec packets), then renegotiate SA.
31
Encapsulating Security Payload (ESP)

32
 IPsec ESP format
• ESP: IPv4.Protocol = 50 or IPv6.Next Header =50
ESP Hdr.

ESP Trailer
ESP Auth.

ICV applies to whole ESP header and encrypted payload (i.e., encrypt-then-MAC).

ESP Auth.
33
 ESP Mode Types

IP Hdr. Data
Original data prior to selection of IPsec protocol mode

Transport Mode:
Encrypted
ESP ESP
IP Hdr. ESP Hdr. Data Trailer Auth

Authenticated
Security is only provided for transport layer and above, leaving original IP header in clear-text
and not authenticated.

Tunnel Mode: Encrypted

ESP ESP
New IP Hdr. ESP Hdr. IP Hdr. Data Trailer Auth

Authenticated
ESP tunnel mode provides confidentiality and integrity protection for the complete original IP
packet (inner IP packet). Outer IP header is in clear-text and not authenticated as in ESP
Transport Mode. 34
 ESP operation in Tunnel Mode

Internet
Router Router
IP Hdr. Data IP Hdr. Data

ESP ESP
Outer IP Hdr. ESP Hdr. IP Hdr. Data Trailer Auth

Encrypted
Authenticated
• Provides confidentiality with encryption
• Provides integrity and authentication through hashing

35
 Internet Key Exchange (IKE)

• Bidirectional IPsec communications requires two SAs (one per direction) and usually four keys in total:
• 2 SAs x (1 Encryption key + 1 Integrity key)/SA = 2 Encryption keys + 2 Integrity keys.
• Plus the PSK or Public-private key pairs/digital certificates for endpoint authentication.
• IPsec encryption/integrity key management could be either Manual (i.e. OOB) or Automated (i.e. IKE).

36
IPsec parameters configured using IKE

IPsec parameters are configured using IKE

37
 Internet Key Exchange (IKE)

u IKE populate SAD

u Based on other protocols
v Internet Security Association and Key Management
Protocol (ISAKMP)
ü Generic protocol for multiples key exchange methods
ü Allow exchange keys and SA information
ü Define negotiation based on phases.
v OAKLEY & SKEME
ü Contribute with exchange keys based modes, such public
key and fast re-keying
u IKE has 2 version
v Version 1 (1988)
v Version 2 (2005) (RFC7296)
ü Optimizes messages exchange process ( only 2 round trips)
ü Support NAT-T (UDP 4500) negotiation
38
 IKE Phases
• IKE: UDP port 500.

R1 R2
Host A Host B

10.0.1.3 10.0.2.3
IKE Phase 1 Exchange – Main Mode

1. Negotiate IKE policy sets Policy 10

DES
IKE Message 1 (SA proposal) Policy 15
DES 1. Negotiate IKE policy sets
MD5 MD5
pre-share pre-share
DH1 IKE Message 2 (accepted SA) DH1 To prevent replay attacks
lifetime lifetime

IKE Message 3 (DH public value, nonce)

2. DH key exchange 2. DH key exchange
IKE Message 4 (DH public value, nonce)

IKE Message 5 (Authentication material, ID)

3. Verify the peer identity 3. Verify the peer identity
IKE Message 6 (Authentication material, ID)
(encrypted)

IKE Phase 2 Exchange

Negotiate IPsec policy Negotiate IPsec policy

39
 IKE Phase 1.1 – Exchange of IKE policies

R1 R2
Host A Host B
Negotiate IKE Proposals 10.0.2.3
10.0.1.3

Policy 10 Policy 15
DES DES
MD5 MD5
pre-share IKE Policy Sets pre-share
DH1 DH1
lifetime lifetime

Policy 20
3DES
SHA
pre-share
DH1
lifetime

• Negotiates matching IKE policies (sets) to protect IKE exchange

• Encryption alg. for IKE tunnel, hashing algorithm for integrity check, authentication
method, DH group and duration of ISAKMP tunnel.
40
 IKE Phase 1.2 – DH key exchange
u A.k.a. “Diffie–Hellman–Merkle” key exchange:
v Ralph C. Merkle. "Secure Communications Over Insecure Channels". Communications
of the ACM, vol. 21, no. 4, pp. 294–299. April 1978. "Received August, 1975; revised
September 1977“. doi:10.1145/359460.359473.
v Whitfield Diffie and Martin Hellman. "New directions in cryptography". IEEE Transactions
on Information Theory, vol. 22, no. 6, pp. 644–654. November 1976.
doi:10.1109/TIT.1976.1055638.

u First published in 1976

u Allows two parties, without prior knowledge, to jointly establish a shared secret
key over an insecure communications channel:
v Key can then be used to encrypt subsequent communications using a symmetric key
cipher.
v Does not authenticate peers (i.e. vulnerable to man-in-the-middle) by itself, but other
mechanisms can be used to address this.

u Also used in Transport Layer Security (TLS) ephemeral modes (DHE) to

provide Perfect Forward Secrecy (PFS):
v With PFS each session generates random public keys without using a deterministic
algorithm. Thus, if a session key is compromised, newer & older session keys are not.

41
 Diffie-Hellman illustrated

Eve
u Alice and Bob exchange their
secret colors (representing
numbers) via public transport
only as a mix.
u Eve (attacker) is able to snoop
all exchanged messages.
u Original version of D-H uses
multiplicative group of integers
modulo p
Khan Academy - Diffie-Hellman Key Exchange
https://www.youtube.com/watch?v=YEBfamv-_do

Source: https://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange
42
 Diffie-Hellman (DH) key exchange
How to establish a Diffie-Hellman Key?
Secret value: A (large random number) Secret value: B
Alice
Bob
A mod p B
YA = g YB = g mod p
YA
YB
A AB B
K= (Y A )
B mod p = gB mod p = g mod p = g A mod p = (YAB ) mod p = K
• Each peer deduces same K shared secret, after exchanging some clear-text values.
• Based on the discrete logarithm problem: Y = gX mod p ; Y, g, p à X?
• g (generator) and p (prime-modulus) are non-secret numbers pre-agreed upon.
• Some restrictions needed: g<p, A<p, B<p. In practive p should be huge!!!
• No one listening on the unsecure channel can compute K (a secret value is needed).
• B = dlogg,p(YB) and from here K = (YAB ) mod p
• Through DH exchange, the master key K is set and is used to generate other session keys.43
 Diffie-Hellman (DH) Exercise

u Alice and Bob want to exchange a key K, using

the Diffie-Hellman algorithm with a generator
g = 3 and a prime p = 17.
u What is the value of key K if Alice choses a
secret exponent A = 2 and Bob choses B = 4?

44
 Diffie-Hellman (DH) Exercise

u Alice and Bob want to exchange a key K, using

the Diffie-Hellman algorithm with a generator
g = 3 and a prime p = 17.
u What is the value of key K if Alice choses a
secret exponent A = 2 and Bob choses B = 4?
Solution:
1. Alice computes: YA = gA mod p = 32 mod 17 = 9 mod 17 = 9 ;
and sends this value to Bob.
2. Bob computes: YB = gB mod p = 34 mod 17 = 81 mod 17 = 13 ;
and sends this value to Alice.
3. Then, both Alice and Bob compute:
K = gAB mod p = 32*4 mod 17 = 38 mod 17 = 6561 mod 17 = 16

Actually Alice computes: K = YBA mod p = 132 mod 17 = 169 mod 17 = 16

and Bob computes: K = YAB mod p = 94 mod 17 = 6561 mod 17 = 16 45
 IKE Phase 1.3 – Peer authentication

• The device on the other end of the VPN tunnel (remote peer) must be
authenticated before communications path is considered secure.

Authenticate Peer
Remote Office Corporate Office

Internet
Server
Peer
Authentication

Peer authentication methods

• PSKs (pre-shared keys)
• RSA signatures

A bidirectional IKE SA is now established.

46
 IKE Phase 1 – Agressive Mode

Typical for remote VPN access

R1 R2
Host A Host B

10.0.1.3 10.0.2.3
No DoS protection -> FW/GW
IKE Phase 1 - Agressive Mode Exchange Does not have identity protection -> Cert.
Policy 10 Policy 15
1.Send IKE policy set, DES DES
MD5 MD5
identity and R1’s DH pre-share pre-share 2. Confirm IKE policy
DH1 DH1
key lifetime lifetime set, calculate
shared secret and
send R2’s DH key,
identity and auth.
3.Calculate shared material
secret, verify peer
identify, and confirm 4. Authenticate peer
with peer and begin Phase 2.
IKE Phase 2 Exchange

Negotiate IPsec policy Negotiate IPsec policy

47
 IKE Phase 2

R1 R2
Host A Host B

10.0.1.3 Negotiate IPsec 10.0.2.3

Security Parameters

• IKE negotiates matching IPsec policies.

• Upon completion, unidirectional IPsec Security
Associations (SAs) are established for each protocol and
algorithm combination; therefore a separate key
exchange is required for each data flow.

48
 IPsec VPN Negotiation

10.0.1.3 R1 R2 10.0.2.3

1. Host A wants to send interesting traffic to

Host B (which needs to be protected).
2. R1 and R2 negotiate an IKE Phase 1 session:
SECURE
COMMUNICATION IKE SA IKE Phase 1 IKE SA
CHANNEL

3. R1 and R2 negotiate an IKE Phase 2 session:

IPsec
TUNNEL IPsec SA IKE Phase 2 IPsec SA
PRF PRF

SECURED 4. Information is exchanged via IPsec tunnel:

TRAFFIC
EXCHANGE IPsec Tunnel

5. The IPsec tunnel is terminated.

49