Network Architectures and Services, Georg Carle

Faculty of Informatics
Technische Universität München, Germany

Software Defined Networking

(SDN) for ISP Networks

Syed Naveed Abbas Rizvi

Supervisors:
Prof. Georg Carle
M.Sc. Florian Wohlfart
M.Sc. Daniel Raumer
 Agenda

 Introduction

 ISP Networks

 Software Defined Networking

 Introduction to RouteFlow

 Behavior of RouteFlow

 Conclusion
Software Defined Networking (SDN) for ISP Networks 2
 Introduction

 Evaluation of SDN for ISPs

 Focus is on datacenter networks

 Fewer efforts for ISP networks

 RouteFlow used as a SDN solution

 Test cases for characterization

Software Defined Networking (SDN) for ISP Networks 3

 ISP Networks
 ISPs provide network based services.
 Internet access, VPN, VPLS etc.

 ISPs implement internal functions.

 Routing, traffic engineering, monitoring, failure
recovery, billing etc.

 Interacts with customers and other ISPs

 Core-edge arrangement

 Geographically distributed network

 Vertically integrated devices

Software Defined Networking (SDN) for ISP Networks 4
 Challenges for ISPs

 Demand for swift service deployment

 Dependence on vendor development lifecycle

 Slow and manual device configurations

 Difficulty in customization for diverse customers

Software Defined Networking (SDN) for ISP Networks 5

 Software Defined Networking (SDN)

 A new concept proposed by Open Networking

Foundation (ONF)
• Separation of the control and the data planes
• Centralized controller with global network view
• Programmable control of network

Feature Feature

Feature Feature Feature

Network Operating System Feature

Operating System Operating System

Specialized Packet Specialized Packet

Forwarding Hardware Forwarding Hardware

Packet Forwarding Packet Forwarding

Hardware Hardware

Software Defined Networking (SDN) for ISP Networks 6

 SDN Components
 Physical or virtual switches/datapaths

 OpenFlow protocol
 Messages, flow tables, match fields, actions,
counters

 Network Operating System(NOS)

 Control processes & network view

 Network Applications
 VPN, VPLS, BWoD, managed routers, MPLS
tunnel creation, traffic engineering etc.

Software Defined Networking (SDN) for ISP Networks 7

 RouteFlow
 Enables legacy routing in OpenFlow networks.

 Utilizes distributed virtual control plane.

 Enables transparent operation with non OpenFlow networks.

Software Defined Networking (SDN) for ISP Networks 8
 RouteFlow Configuration

 Static and manual configuration

 Runtime changes in network are not supported

 Two step configuration process

 Router virtual machine (VM) creation
 Router virtual machines to OpenFlow switch
mappings

Software Defined Networking (SDN) for ISP Networks 9

 RouteFlow Operation

 Multiple mapping modes

 Logical split (1:1)
 Router multiplexation (1:n)
 Router aggregation (m:1)

Software Defined Networking (SDN) for ISP Networks 10

 RouteFlow Flow Table Installation
 Types of flows
 Fixed Flows (FF)
 Variable Flows (VF)

 Total Flows (TF) = FF + VF

 VF depends on
 Number of ports (NP)
 Directly (DC) & indirectly connected (IC) network destinations

 VF = (NP - 1) x (DC + IC)

Flow Type Fixed Flows (FF) Variable Flows (VF)

Installation Proactive Proactive & reactive
Priority 32800 Depends on prefix length
Number 10 variable
Packet Types BGP, OSPF, ICMP, ARP etc. IPv4

Software Defined Networking (SDN) for ISP Networks 11

 Relation between Routing & Flow Tables
 One to many relation between routing and flow table
entries

 Each routing table entry cause (NP-1) flow table

entries

 Total BGP prefixes = 0.5 million

 OpenFlow table size required ~ 5 million for a 10 port

switch

 Wildcard Input port match field or use FIBIUM like

approach

Software Defined Networking (SDN) for ISP Networks 12

 RouteFlow Behavior

 Topology independent test cases

 Control traffic handling
 Data traffic handling
 Failure handling
 RouteFlow specific

Software Defined Networking (SDN) for ISP Networks 13

 Control Traffic Handling in RouteFlow

 ARP packet flow

Mongo DB ICMP Request
lxcbr0
a0 b0 ICMP Reply
OF Packet In
Veth Tunnels
OF Packet Out
e0 RFClient RFClient e0
RFServer Starting Point
Router A Router B
e1 e2 e1 e2
Veth Tunnels
a1 a2 b1 b2
RFVS (dp0)

tcp: controller ip:port=6633

RFProxy
POX

tcp: controller ip:port=6633

Mininet

H1 e1 e1 A e2 e2 B e1 e1 H2

Software Defined Networking (SDN) for ISP Networks 14

 ARP Packet Flow

 No ARP processing in OF switches

 Reactive flow installation for the host

 Unicast ARP queries during running IP flows

Software Defined Networking (SDN) for ISP Networks 15

 Control Traffic Handling in RouteFlow

 High priority ICMP, BGP etc. flow table entries

lxcbr0 Mongo DB ICMP Request
a0 b0 OF Packet In
Veth Tunnels OF Packet Out
Starting Point
e0 RFClient RFClient e0
Router A Router B RFServer
e1 e2 e1 e2
Veth Tunnels
a1 a2 Requires
b1 Modification
b2 in RouteFlow
RFVS (dp0)

tcp: controller ip:port=6633

RFProxy
POX

tcp: controller ip:port=6633

Mininet

H1 e1 e1 A e2 e2 B e1 e1 H2

Software Defined Networking (SDN) for ISP Networks 16

 Analysis

 Control packets sent to routers at each hop

 Fine for OSPF, RIP, ARP packets but adds extra

delay for ICMP, BGP packets

 Adds extra traffic on RFVS & OF Controller link

 Decrease the priority for ICMP, BGP control flows

 Increase flow table lookup delays inside switch

Software Defined Networking (SDN) for ISP Networks 17

 Analysis…(2)

 Avoid unicast ARP packets between transit

routers

 ICMP packets for specific router sent to controller

 Similar modification is useful for BGP packets

Software Defined Networking (SDN) for ISP Networks 18

 Traceroute in RouteFlow
 Tool for route and transit delay discovery

Switch Switch Switch

Host 1 Host 2
A C B

 ICMP & UDP probe packets

Requires newer OpenFlow version
 ICMP probe yields two different results
 High priority flow >>> 4 hops
 Low priority flow >>> 1 hop
 No TTL decrement in OpenFlow 1.0

 UDP probe detects only 1 hop

 Legacy networks connected via RouteFlow network

 Number of hops remain unknown

Software Defined Networking (SDN) for ISP Networks 19
 Port & Link Failure in RouteFlow

 Port failure cause port status modification

messages Switch
C

Switch Switch
Host 1 Host 2
A B

 Link failure is simulated to avoid status change

messages
Switch
C

Switch Switch
Host 1 Hub Host 2
A B

Software Defined Networking (SDN) for ISP Networks 20

 Port & Link Failure...(2)

 No response to status modification messages

 Relies on OSPF dead interval for recovery

Requires Modification in RouteFlow

 Slow recovery process approx. 10sec for 4 sec
OSPF dead interval

 MPLS FRR ~ 50 ms recovery time

Software Defined Networking (SDN) for ISP Networks 21

 UDP/TCP Packet Flow
 Similar forwarding response

Switch Switch Switch

Host 1 Host 2
A C B

 Both UDP/TCP RouteFlow

Packet drops at egress switch B
Problem

 No flow entry for Host 2

 No ICMP error report to source Host 1

 Require a generic IPv4 flow entry in edge

switches
Software Defined Networking (SDN) for ISP Networks 22
 Link Failure between Controller & Switches
 SDN/OF specific failure

Switch Switch
Host 1 Host 2
A B
Complete
Failure

Partial Failure

Switch Switch
Host 3 C D Host 4

 Two scenarios tested

 Link failure with complete network
 Link failure with few switches

 UDP, TCP and ICMP flows used

 Switches retained flow tables for complete failure

 Switches with link to controller received flow modifications

Software Defined Networking (SDN) for ISP Networks 23

 Link Failure between Controller & Switches…(2)

 Running flow in both scenarios stopped after some

random time

 No backup controller in OF 1.0

SDN & OpenFlow
 Main cause of Problem
flow disruption is unsuccessful ARP
queries

 Impossible to start new flows using affected switches

 Flow recovery in second scenario if alternative path

available
Software Defined Networking (SDN) for ISP Networks 24
 Router VM State Modification
 Specific to RouteFlow
Switch Switch
Host 1 Host 2
A B
VM freezes
VM fails

Switch Switch
Requires Modification
Host 3 C in RouteFlow
D Host 4

 Unexpected failure or planned state modification

 Recover from planned state modification

 Unable to recover from VM failures

 Manual effort required to recover from VM failure

Software Defined Networking (SDN) for ISP Networks 25
 Router Multiplexation Mode
 Logically isolated network segments
Host 1
Host 2

Switch Switch
A B

Host 3 Host 4
Requires Modification in RouteFlow

 Isolation not achieved in OF network

 Flows for all ingress ports

 Require modification in RouteFlow

 Somewhat similar to virtual routing & forwarding (VRF)

Software Defined Networking (SDN) for ISP Networks 26

 Router Aggregation
 Single router for multiple switches

 Linear and full mesh topologies

 Requires configuration of inter switch links

Requires Modification in RouteFlow
Switch Switch Switch
Host 1 Host 2
A C B

 No flow entries for switch C except fixed flows

 Switch A and B have flow entries for directly connected

host

 Linear topology do not utilize ISL for data packets

Software Defined Networking (SDN) for ISP Networks 27
 Router Aggregation…(2)
Host 3

Switch
C

Switch Switch
Host 1 Host 2
A B

 All ISLs are used for data packets

 All switches have host specific flow entries for all hosts

 Requires full mesh topology for proper aggregation

 Useful for isolation between intra and inter domain routing

Software Defined Networking (SDN) for ISP Networks 28
 BGP Router Aggregation
 EBGP speakers aggregated for a domain

Switch
102

Host 2
Switch AS 100 Switch
Host 1 201
101
Host 3

Switch AS 200
103

 Routers in AS100 for switch 102 & 103

 No IBGP session required between switch 102 &103

 Easy to manage single router

Software Defined Networking (SDN) for ISP Networks 29
 Miscellaneous Test cases

 DHCP server in RouteFlow routers

 Interfacing with customer routers

 Some interesting future scenarios

 Multipath routing
 Multicast routing
 MPLS switching
 QoS related tests

Software Defined Networking (SDN) for ISP Networks 30

 Conclusion

 RouteFlow as a SDN solution

 Separation of the control and data plane
 Centralized controller
 Distributed routing
 No abstraction for network applications

 Other SDN solutions

 OPEN : MPLS TE, VPN
 Mutilflow : Multicast routing
 Inter AS routing component
 ONIX : Distributed controller paltform

Software Defined Networking (SDN) for ISP Networks 31

 Conclusion…(2)

 RouteFlow is not a very good SDN solution

 Require changes in RouteFlow implementation

 Integration with other SDN solutions

Software Defined Networking (SDN) for ISP Networks 32

 Future Work

 Isolation between Inter & intra domain routing

 Centralized routing for intra domain destinations

 Integration with other SDN solutions

 Provision of APIs for network applications

Software Defined Networking (SDN) for ISP Networks 33

 Questions

Software Defined Networking (SDN) for ISP Networks 34

 RouteFlow Components
 RFServer

 RFClient Management Network

RFServer
IPC DB
Virtual Control
Plane
 RFProxy IPC
Router 1 VM R
Management
F
Switch Veth Tunnels
 RFProtocol Router N VM
V
S
IPC

 RouteFlow virtual switch RFProxy

OpenFlow Controller
(RFVS)

 Management Switch
Switch 1 Switch N

 Inter-process communication OpenFlow Physical Network

(IPC) Database

Software Defined Networking (SDN) for ISP Networks 35

 RouteFlow Operation

 Management switch & IPC database is initialized

 Router VMs & OpenFlow controller is started

 RFServer with configured mappings is started

 RFVS is connected to OF controller and router

VMs

 OpenFlow switches are added to the network

Software Defined Networking (SDN) for ISP Networks 36

 RouteFlow Operation

 Routing tables are built

 Proactive and reactive flow table entries are

installed in switches

Software Defined Networking (SDN) for ISP Networks 37

 RouteFlow Flow Table Installation…(2)
 Directly and indirectly connected network destinations for A.

Host 1 Switch
B

Switch
Host 2 L2 Switch
A

Switch
Host N
C

 Directly connected (DC) destinations for A = N + 2

 Indirectly connected (IC) subnets for A = 1

 VF = (NP - 1) x (DC + IC)

 IC flows are subnet specific & DC flows are network address specific
Software Defined Networking (SDN) for ISP Networks 38
 Control Traffic Handling in RouteFlow

 High priority flow table entries

lxcbr0 Mongo DB ICMP Request
a0 b0 OF Packet In
Veth Tunnels OF Packet Out
Starting Point
e0 RFClient RFClient e0
Router A Router B RFServer
e1 e2 e1 e2
Veth Tunnels
a1 a2 b1 b2
RFVS (dp0)

tcp: controller ip:port=6633

RFProxy
POX

tcp: controller ip:port=6633

Mininet

H1 e1 e1 A e2 e2 B e1 e1 H2

Software Defined Networking (SDN) for ISP Networks 39

 Control Traffic Handling in RouteFlow

 High priority flow table entries

lxcbr0 Mongo DB ICMP Reply
a0 b0 OF Packet In
Veth Tunnels OF Packet Out
Starting Point
e0 RFClient RFClient e0
Router A Router B RFServer
e1 e2 e1 e2
Veth Tunnels
a1 a2 b1 b2
RFVS (dp0)

tcp: controller ip:port=6633

RFProxy
POX

tcp: controller ip:port=6633

Mininet

H1 e1 e1 A e2 e2 B e1 e1 H2

Software Defined Networking (SDN) for ISP Networks 40

 Network Architecture
 The design and framework of a network, including the characteristics of individual
hardware, software, and transmission system components and how they interact in
order to ensure the reliable transfer of information. Prior to the development of such
architectures, interoperability between the various systems of a single manufacturer
was unusual, and it certainly did not exist between the products of multiple
manufacturers. IBM's Systems Network Architecture (SNA) and the Digital Equipment
Corporation's (DEC's) Digital Network Architecture (DNA), aka DECnet, corrected these
shortcomings within the IBM and DEC domains, but they still did not interoperate. Truly
open systems architectures still remain in the distant future, although great strides have
been made in this regard through the Open Systems Interconnection (OSI) model
fostered by the International Organization for Standardization (ISO). Network
architectures tend to be layered, which serves to enhance their development and
management. While they primarily address issues of data communications, they also
include some data processing activities at the upper layers. These upper layers
address application software processes, presentation format, and the establishment of
user sessions. Each independent layer, or level, of a network architecture addresses
different functions and responsibilities. The layers work together, as a whole, to
maximize the performance of the process. See also ISO, OSI Reference Model, and
SNA

 Webster's New World Telecom Dictionary Copyright © 2010 by Wiley Publishing, Inc.,
Indianapolis, Indiana.

Software Defined Networking (SDN) for ISP Networks 41