Research

An Introduction to MPLS
Timothy G. Griffin
[email protected] http://www.research.att.com/~griffin November 21, 2002
1

Whats all this talk about MPLS?

MPLS MPLS MPLS MPLS MPLS MPLS MPLS MPLS MPLS MPLS MPLS is going to solve all of our problems is a solution in search of a problem is all about traffic engineering is what I wish on all of my competitors is all about virtual private networks solves network operations problems creates network operations problems is all about lowering operational costs is going to cost more than its worth is the natural next step in Internet evolution is too complicated to survive in the Internet

But what is MPLS anyway?

2

Goals of this Tutorial

To understand MPLS from a purely technical point of view
avoid the hype avoid the cynicism

To understand the broad technical issues without getting lost in the vast number of details
the gains the costs the tradeoffs
3

Keep in Mind
MPLS is an emerging technology Many technical issues have not yet been resolved Interest and enthusiasm is not universal, but primarily found in large providers (and their vendors) Standards are rapidly evolving Implementations are rapidly evolving Operational experience and expertise still very scarce

Expect interoperability problems and feature availability problems for the next few years
4

Outline
Why MPLS?
Problems with current IP routing and forwarding Complexity of overlay model

What is MPLS?
Label swapping Label distribution Constraint based routing

What applications could exploit MPLS?

Traffic Engineering Virtual Private Networks
Both Layer 2 and Layer 3 VPNs
5

IP forwarding paths are implemented with destination-based next hop tables

Dest.
Default to upstream router

Nxt Hop R1 Direct R3 R1 R3 R1

B
R R2 R R1 R4

A B C D E default

Dest. A B C D E default

Nxt Hop R2 R2 Direct R5 R5 R2

R3 R
R5

C
Dest.

Nxt Hop R4 R3 R3 R4 Direct R4

A B C D E default

IP Forwarding Process
1. Remove a packet from an input queue 2. Check for sanity, decrement TTL field 4. Place packet on correct output queue

Forwarding Process
If queues get full, just drop packets! 3. Match packets destination to a table entry If queues get full, just drop packets!

IP Forwarding Table
Router
7

IP routing protocols assume all forwarding is destination-based

B
R
The Fish

C
The next-hop forwarding paradigm does not allow router R to choose a route to A based on who originated the traffic, B or C.
8

IP forwarding tables are maintained by dynamic routing protocols

RIP Process
RIP Routing tables

BGP Process
BGP Routing tables

BGP

OSPF Process
OSPF Routing tables

RIP Domain

OS kernel

OSPF Domain

IP Forwarding Table

Shortest Path Routing: Link weights tend to attract or repel all traffic

A
B

C
1 2 1 1

A B

2
1

1
1

10

Overlay Networks
B

Layer 2
(virtual circuits)

C B
C

Layer 3

11

Advantages of Overlay Networks

ATM and Frame Relay switches offer high reliability and low cost Virtual circuits can be reengineered without changing the layer 3 network Large degree of control over traffic Detailed per-circuit statistics Isolates layer 2 network management from the details of higher layer services

12

Problems with Overlay Networks

Often use proprietary protocols and management tools Often requires full meshing of statically provisioned virtual circuits ATM cell tax ---- about 20% of bandwidth If layer 3 is all IP, then the overlay model seems overly complicated and costly Advances in optical networking cast some doubt on the entire approach
Overlay model is just fine when layer 2 network provides diverse non IP services.
13

Blur Layer 2 and 3?

router switch (ATM, Frame)

this is not a router this is not a switch

what is it?
14

Sanity Check?
The problems with IP forwarding and routing do not require technologies like MPLS
Many can be addressed with simple solutions. Like the design of simple networks! The problems are not show stoppers The MPLS cure will have side effects For many applications, TCP/IP handles congestion very well

Technologies like MPLS may be very valuable if they can enable new services and generate new revenue

15

Sign of the Times: New Sub-IP area in the IETF

Multiprotocol Label Switching (mpls) Common Control and Management Protocols (ccamp) Internet Traffic Engineering (tewg) Provider Provisioned Virtual Private Networks (ppvpn) IP over Optics (ipo) IP over Resilient Packet Rings (iporpr) General Switch Management Protocol (gsmp)

See http://www.ietf.org/html.charters/foo-charter.html, where foo is the working group acronym.

16

MPLS = MultiProtocol Label Switching

Network
MPLS Data Link

A Layer 2.5 tunneling protocol Based on ATM-like notion of label swapping A simple way of labeling each network layer packet Independent of Link Layer Independent of Network Layer Used to set up Label-switched paths (LSP), similar to ATM PVCs

Physical
RFC 3031 : Multiprotocol Label Switching Architecture
17

Generic MPLS Encapsulation

Layer 2 Header MPLS Label 1 MPLS Label 2

MPLS Label n

Layer 3 Packet

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Label | Exp |S| TTL | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Often called a shim (or sham) header

RFC 3032. MPLS Label Stack Encoding

Label: Exp: S: TTL:

Label Value, 20 bits Experimental, 3 bits Bottom of Stack, 1 bit Time to Live, 8 bits
18

Forwarding via Label Swapping

417

data

288

data

Labels are short, fixed-length values.

19

Popping Labels

data

288

data

577

data

288

577

data

20

Pushing Labels

288

data

data

288

577

data

577

data

21

A Label Switched Path (LSP)

POP! SWAP! SWAP! PUSH!

data

417 data

666 data

233 data

data

A label switched path

tail end head end Often called an MPLS tunnel: payload headers are not Inspected inside of an LSP. Payload could be MPLS
22

Label Switched Routers

IP

IP out

IP in
IP Forwarding Table IP

77

data

Label Swapping Table

23
MPLS in

data

MPLS out

The data plane

represents IP Lookup + label push represents label pop + IP lookup

23

Forwarding Equivalence Class (FEC)

IP2

417 IP2

666 IP2

233 IP2

IP2

IP1

417 IP1

666 IP1

233 IP1

IP1

Packets IP1 and IP2 are forwarded in the same way --- they are in the same FEC. Network layer headers are not inspected inside an MPLS LSP. This means that inside of the tunnel the LSRs do not need full IP forwarding table.
24

LSP Merge
IP2

417 IP2

823 IP2

912 IP2

IP2

IP1

111 IP1

666 IP1

233 IP1

IP1

IP2

417 IP2

823 IP2

912 IP2

IP2

IP1

417 IP1

666 IP1

233 IP1

IP1
25

LSP merge

Penultimate Hop Popping

POP + IP Lookup SWAP SWAP PUSH

IP

417 IP

666 IP

233 IP

IP

IP Lookup

POP

SWAP

PUSH

IP

IP

666 IP

233 IP

IP
26

LSP Hierarchy via Label Stacking

IP2 IP2

POP

PUSH

66

IP2

44 66

IP2

88 66

IP2

17 66

IP2

66

IP2

23

IP1

44 23

IP1

88 23

IP1

17 23

IP1

23

IP1

POP

PUSH

IP1 Did I get carried away on this slide?

IP1 27

MPLS Tunnels come at a cost

ICMP messages may be generated in the middle of a tunnel, but the source address of the bad packet may not be in the IP forwarding table of the LSR!
TTL expired: traceroute depends on this! MTU exceeded: Path MTU Discovery (RFC1191) depends on this!

None of the proposed solutions are without their own problems

28

MPLS also supports native encapsulation Layer 2 MPLS

PPP

Ethernet

ATM VPI/VCI

Frame DLCI

generic encapsulation

rest of label stack

. . .

generic encapsulation

. . .

. . .

generic encapsulation IP datagram

29

But Native Labels May Cause Big Headaches

No TTL!
Loop detection? Loop prevention?

LSP merge may not be supported

Label bindings cannot flow from destination to source, but must be requested at source
MPLS was initially designed to exploit the existence of ATM hardware and reduce the complexity of overlay networks. But IP/MPLS with native ATM labels results in a large number of problems and complications.
30

Basic MPLS Control Plane

MPLS control plane = IP control plane +

label distribution

Label distribution protocols are needed to (1) create label FEC bindings (2) distribute bindings to neighbors, (3) maintain consistent label swapping tables
31

Label Distribution: Option I

Piggyback label information on top of existing IP routing protocol
Good Guarantees consistency of IP forwarding tables and MPLS label swapping tables Points

No new protocol required Allows only traditional destination-based, hop-byBad hop forwarding paths Points Some IP routing protocols are not suitable Need explicit binding of label to FEC Link state protocols (OSPF, ISIS) are implicit, and so are not good piggyback candidates Distance vector (RIP) and path vector (BGP) are 32 good candidates. Example: BGP+

Label Distribution: Option II

Create new label distribution protocol(s)
Good Compatible with Link State routing protocols Points Not limited to destination-based, hop-by-hop

forwarding paths

Additional complexity of new protocol and Bad interactions with existing protocols Points Transient inconsistencies between IP forwarding tables and MPLS label swapping tables Examples: LDP (IETF) and TDP (Cisco proprietary)
33

The Control Plane

IP Routing Protocols + IP Routing Tables Label distribution protocols + Label Binding Tables

Routing messages

Label distribution messages

IP

IP out

IP in
IP Forwarding Table IP

77

data

Label Swapping Table

23
MPLS in
34

data

MPLS out

Label Distribution with BGP

Carrying Label Information in BGP-4 draft-ietf-mpls-bgp4-05.txt (1/2001) Associates a label (or label stack) with the BGP next hop. Uses multiprotocol features of BGP: RFC 2283. Multiprotocol Extensions for BGP-4 So routes with labels are in a different address space than a vanilla routes (no labels)
35

BGP piggyback not required for simple iBGP optimization

Map traffic to the LSP that terminates at the egress router chosen by BGP
BGP route Internal BGP

IP AS 444

417 IP

666 IP

233 IP

IP

AS 888
Routers A and B do not need full routing tables. They only need IGP routes (and label bindings).
36

BGP piggyback allows Interdomain LSPs

Use top of stack to get to egress router, bottom of stack for LSP in AS 444.
BGP route With label 99 Internal BGP with label 99

99

IP

417 99

IP

666 99

IP

233 99

IP

IP

AS 444

AS 888

37

MPLS tunnels can decrease size of core routing state

Core routers need only IGP routes and LSPs for IGP routes Implies less route oscillation Are these really Implies less memory problems? Implies less CPU usage BUT: still need route reflectors to avoid full mesh and/or to reduce BGP table size at border routers BUT: since your core routers do not have full tables you now have all of the MPLS problems associated with ICMP source unknown (TTL, MTU, traceroute )
38

Label Distribution Protocol (LDP)

RFC 3036. LDP Specification. (1/2001)
Dynamic distribution of label binding information Supports only vanilla IP hop-by-hop paths LSR discovery Reliable transport with TCP Incremental maintenance of label swapping tables (only deltas are exchanged) Designed to be extensible with Type-LengthValue (TLV) coding of messages Modes of behavior that are negotiated during session initialization Label retention (liberal or conservative) LSP control (ordered or independent) Label assignment (unsolicited or on-demand)

39

LDP Message Categories

Discovery messages: used to announce and maintain the presence of an LSR in a network. Session messages: used to establish, maintain, and terminate sessions between LDP peers. Advertisement messages: used to create, change, and delete label mappings for FECs. Notification messages: used to provide advisory information and to signal error information.
40

LDP and Hop-by-Hop routing

network next-hop 10.11.12.0/24 direct
A

network next-hop
10.11.12.0/24

network next-hop
10.11.12.0/24

network next-hop
10.11.12.0/24

LSP
10.11.12.0/24

LDP

417
10.11.12.0/24

LDP

666
10.11.12.0/24

LDP

233
10.11.12.0/24

Generate new label And bind to destination

pop
A

swap
B

swap

push

IP

417 IP

666 IP

233 IP

IP
41

MPLS Traffic Engineering

The optimization goals of traffic engineering are To enhance the performance of IP traffic while utilizing Network resources economically and reliably.

Intra-Domain
A Framework for Internet Traffic Engineering Draft-ietf-tewg-framework-02.txt A major goal of Internet Traffic Engineering is to facilitate efficient and reliable network operations while simultaneously optimizing network resource utilization and performance.

RFC 2702 Requirements for Traffic Engineering over MPLS

42

Intra-Domain

TE May Require Going Beyond Hop-by-Hop Routing

Explicit routes
Allow traffic sources to set up paths

Constraint based routing

Chose only best paths that do no violate some constraints Needs explicit routing May need resource reservation

Traffic classification
Map traffic to appropriate LSPs
43

Hop-by-Hop vs. Explicit Routes

Distributed control Trees rooted at destination Destination based forwarding Originates at source Paths from sources to destinations Traffic to path mapping based on what configuration commands your vendor(s) provide

44

Explicit path Setup

REQUEST LSPID 17 REQUEST LSPID 17 REQUEST LSPID 17 Request path D->C->B->A with LSPID 17

LSP
reply

417
LSPID 17

reply

666
LSPID 17

reply

233
LSPID 17

pop

swap

swap

push

IP

417 IP

666 IP

233 IP

IP
45

Constraint Based Routing

Basic components
1. 2. Specify path constraints Extend topology database to include resource and constraint information Find paths that do not violate constraints and optimize some metric Signal to reserve resources along path Set up LSP along path (with explicit route) Map ingress traffic to the appropriate LSPs
Problem here: OSPF areas hide information for scalability. So these extensions work best only within an area

Extend Link State Protocols (IS-IS, OSPF)

3.
4. 5.

Extend RSVP or LDP or both!

6.

Note: (3) could be offline, or online (perhaps an extension to OSPF)

Problem here: what is the correct resource model for IP services? 46

Resource Reservation + Label Distribution

Two emerging/competing/dueling approaches:
Add label distribution and explicit routes to a resource reservation protocol Add explicit routes and resource reservation to a label distribution protocol

RSVP-TE

CR-LDP

+
RSVP

+
LDP
Constraint-Based LSP Setup using LDP draft-ietf-mpls-cr-lpd-05.txt

RSVP-TE: Extensions to RSVP for LSP Tunnels draft-ietf-mpls-rsvp-lsp-tunnel-08.txt

47

RSVP-TE vs. CR-LPD

RSVP-TE CR-LDP

Soft state periodically State maintained refreshed incrementally IntServe QoS model New QoS model derived from ATM and Frame Relay
And the QoS model determines the additional information attached to links and nodes and distributed with extended link state protocols

And what about that other Internet QoS model, diffserve?

48

A closer look at CR-LDP

Defines new TLV encodings and procedures for
Explicit routing (strict and loose) Route pinning (nail down some segments of a loosely routed path) Traffic parameter specification
Peak rate Committed rate Weight Resource class or color

LSP preemption (reroute existing paths to accommodate a new path) LSP Identifiers (LSPIDs)

49

The Fish Revisited

LPS2

A
LPS1

C
Need at least one explicit route to A
50

Use Shortest Paths to get beyond Shortest Paths!

The IP routing protocol at LSR A is configured to (privately) see A -> C LSP as one link with weight 1.
1

LPS

A
1

2
1

B
Vanilla IP forwarding
51

MPLS TE: Is it worth the cost?

Much of the traffic across a (transit) ISPs network is interdomain traffic
Congestion is most common on peering links The current work on MPLS TE does not apply to interdomain links! (Actually, it does not even work well across OSPF areas)

MPLS TE is probably most valuable when IP services require more than best effort
VPNs with SLAs? Supporting differentiated services?

52

VPNs with MPLS

Traditional VPN overlay model:
MPLS-based Layer 2 VPNs draft-kompella-mpls-l2vpn-02.txt Whither Layer 2 VPNs? draft-kb-ppvpn-l2vpn-motiv-00.txt New VPN peering model:

RFC 2547. BGP/MPLS VPNs

53

Traditional Overlay VPNs

B

Providers Layer 2 Network (ATM, Frame Relay, X.25)

C
C

B A
Customers Layer 3 VPN

54

Why Not Use MPLS Tunnels?

B

Providers MPLS enabled network

C
C

MPLS LSP

B
MPLS LSP

A
Customers Layer 3 VPN
MPLS LSP

55

Potential Advantages of MPLS Layer 2 VPNs

Provider needs only a single network infrastructure to support public IP, and VPN services, traffic engineered services, and differentiated services Additional routing burden on provider is bounded Clean separation of administrative responsibilities. Service provider does MPLS connectivity, customer does layer 3 connectivity Easy transition for customers currently using traditional Layer 2 VPNs

56

BGP/MPLS VPNs
RFC 2547 Is Peer Model of VPN (not Overlay) Also draft-rosen-rfc2547bis-02.txt Cisco configuration info :
http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/ 120newft/120t/120t5/vpn.htm

AT&Ts IPFR service is based on this RFC.

Note that the Model of this RFC could be implemented without MPLS. See BGP/IPsec VPN <draft-declercq-bgp-ipsec-vpn-00.txt>

57

RFC 2547 Model

VPN 2 VPN 1

VPN Provider Network

VPN 1

VPN 2
58

CEs and PEs

Customer Site

CE = customer edge

PE = provider edge
Provider Network
59

VPN Address Overlap Means Vanilla Forwarding Tables Cant Work

Site 2 p1

Site 1 p1

VPRN 2

Provider

VPRN 1

Site 3 p2

Dest.
p1 p2

Nxt Hop
?? ??

Site 4 p2
60

VPN Overlap Means Vanilla Forwarding VanillaTables Cant Work out forwarding tables are

Site 2 p2

VPRN 1

Site 1 p1

VPRN 2

Provider Violates isolation Guarantee of A VPN: site 1 can Exchange traffic with Site 2!
61

Site 3 p3

Dest.
p1 p2 p3

Nxt Hop
s1 s2 s3

RFC 2547 : Per site forwarding tables

Called VRFs, for "VPN Routing and Forwarding" tables.

Site 2 p2

VPRN 1

Site 1 p1

VPRN 2

Provider Site 2 FT

Site 3 p3

Site 1 FT Dest. p2 Nxt Hop s2

Site 3 FT Dest. p2 Nxt Hop s2

Dest.
p1 p3

Nxt Hop
s1 s3
62

Tunnels required across backbone

Site 2 p2

VPRN 1

Site 1 p1

VPRN 2

VPRN 3

Site 3 p3

Site 4 p3
63

RFC 2547 Summary

Piggyback VPN information on BGP New address family New attributes for membership New Per-site forwarding tables (VRFs) Use MPLS Tunnels between PEs No need for VPN routes on backbone LSRs, only on PEs

64

Layer 2 VPNs vs. BGP/MPLS VPNs

Customer routing stays with customer May allow an easier transition for customers currently using Frame/ATM circuits Familiar paradigm Easier to extend to multiple providers Customer routing is outsourced to provider Transition may be complicated if customer has many extranets or multiple providers New peering paradigm Not clear how multiple provider will work (IMHO)

65

Summary
MPLS is an interesting and potentially valuable technology because it

provides an efficient and scalable tunneling mechanism provides an efficient and scalable mechanism for extending IP routing with explicit routes

66

More info on MPLS

MPLS working group
http://www.ietf.org/html.charters/mpls-charter.html

MPLS email list archive

http://cell.onecall.net/cell-relay/archives/mpls/mpls.index.html

MPLS Resource Center

http://www.mplsrc.com

Peter Ashwood-Smiths NANOG Tutorial

http://www.nanog.org/mtg-9910/mpls.html

MPLS: Technology and Applications. By Bruce Davie and Yakov Rekhter. Morgan Kaufmann. 2000. MPLS: Is it all it's cracked up to be? Talk by Pravin K. Johri
http://buckaroo.mt.att.com/~pravin/docs/mpls.pdf
67

More info on MPLS TE

tewg working group
http://www.ietf.org/html.charters/tewg-charter.html

NANOG Tutorial by Jeff Doyle and Chris Summers

http://www.nanog.org/mtg-0006/mpls.html

NANOG Tutorial by Robert Raszuk

http://www.nanog.org/mtg-0002/robert.html

68

More info on MPLS VPNs

PPVPN working group
http://www.ietf.org/html.charters/ppvpn-charter.html

PPVPN Archive
http://nbvpn.francetelecom.com

NANOG Panel:Provider-Provisioned VPNs

http://www.nanog.org/mtg-0102/jessica.html

MPLS and VPN Architectures. By Ivan Pepelnjak and Jim Guichard. Cisco Press. 2001

69