Software-Defined Networks

A Systems Approach

Bruce Davie
Systems Approach, LLC
 What is SDN?

• There’s a simple answer:

• SDN (software-defined networking) is the separation of control and data
planes
• The separation allows control topology to be independent of physical network
topology
• The more interesting question is:
• Why would anyone want to do this? Logically centralized control plane
• That question has a lot of answers…
e.g.
OpenFlow

Data Plane
 Outline

• History of SDN
• Challenges faced by IP networks
• SDN architecture
• Case Studies:
• Network Virtualization
• Traffic Engineering
• SD-WAN
• Bare metal switching
A Revolution in Networking
 Foundations of SDN

• 4D, Greenberg et al. – part of a broader set of “Clean Slate” initiatives

• Ipsilon General Switch Management Protocol – RFC 2297 (1996)
• IETF Forces WG (2001-2015!!)
• Ethane (2007)
 Challenges with IP networks

• Lack of abstractions
• Inability to express intent
• Unpredictable outcome from complex distributed algorithms
• Interactions among protocols (e.g. IGP & EGP)
• Can’t manage a device unless it’s properly configured
• bootstrap issue – control & management plane dependent on correct data
plane
• Fragility, risk of change
• Glacial pace of innovation
Evolution of network provisioning: 1996-2016

1996 2016

Terminal Protocol: Telnet Terminal Protocol: SSH

 Key SDN Insights

• Centralizing the control plane enables more powerful abstractions

• E.g. X and Y should be able to communicate
• Express intent network-wide
• Distributed systems techniques to make central control scalable and fault
tolerant
• Central control means a single API for the network, rather than an API per
box
• Networks provisioned by software, not humans
• Disaggregation → innovation
• Network-wide intent → better security
 Disaggregation of computing Industry

App App App App App App

Specialized
Applications Open Interface
Mac
Windows or Linux or
Specialized OS
OS
Open Interface
Specialized
Hardware Microprocessors
 Disaggregation of networking Industry

App App App App App App

Specialized
Applications Open Interface
Network Network Network
or or
Specialized OS OS OS
OS
Open Interface
Specialized Merchant Silicon
Hardware Switching Chips
 Random side observation

• Just because an idea has been tried before without success doesn’t
mean it’s a bad idea
SDN Architecture
Traditional Control and Data Planes

Control Plane
• Protocols: BGP, OSPF, RIP
• RIB: Collection of Link/Path Attributes
Routing Table • Northbound Configuration Interface
Control Plane
(RIB) − e.g., Cisco CLI

Data Plane
• Protocols: IP
Forwarding Table • FIB: Optimized for Fast Lookup
Data Plane
(FIB) • Northbound Control Interface
− Historically Private/Internal
 SDN Control and Data Planes

Control Control Control Control

App App App ... App

Global
Network
Network OS Map
Control Plane
Data Plane
Flow Rules
 OpenFlow-style data plane

Packet Packet + Packet

In Metadata Packet Execute Out
Table Table ... Table
Action
Action 0 Action 1 n Action
(MAC) (VLAN) (IP) Set
Set = {} Set Set

OpenFlow Switch
MAC IP TCP/UDP
Header Header Header

Dst Src Src Dst Src Dst

Addr Addr Type … Proto … Addr Addr … Port Port … … Payload …

Type Ctl VLAN ID

Optional 802.1Q
VLAN Tag
 PISA: Protocol Independent Switching Architecture

Programmable Programmable Match-Action Pipeline Programmable

Parser Deparser
Memory ALU Memory ALU Memory ALU Memory ALU

Memory ALU Memory ALU Memory ALU Memory ALU

Memory ALU Memory ALU Memory ALU Memory ALU

Memory ALU Memory ALU Memory ALU Memory ALU

Memory ALU Memory ALU Memory ALU Memory ALU

Memory ALU Memory ALU Memory ALU Memory ALU

 SDN Software Stack

Control Control Control

Trellis
App App App

gRPC
API gNMI + gNOI + FlowObjectives
forward.p4 Network Operating System
arch.p4 ONOS

P4 gRPC
Compiler
API gNMI + gNOI + P4Runtime/OpenFlow

Switch OS Stratum + ONL

Merchant Silicon Tofino (Barefoot),

Tomahawk (Broadcom)
Programmable Switch
Scaling the Central Control Plane

Transport WebService Persistent Logical

Network API Storage Network

Controller Controller Controller Controller Controller

Controller
Cluster

Node Node Node Node Node

1 2 3 4 5
 Summary

Definition of SDN
A network in which the control plane is physically separate from the forwarding plane,
and a single control plane controls several forwarding devices. – Nick McKeown (2013)
Dimensions
• Disaggregated Control and Data planes
• Centralized vs Decentralized Control Plane
• Fixed-Function vs Programmable Data Plane
Phases of SDN
• Phase 1: Network operators took ownership of the control plane.
• Phase 1a: Non-traditional entrants to the networking business (via disaggregation)
• Phase 2: Network operators are taking ownership of the data plane.
 Use Cases

• Network Virtualization
• SD-WAN
• Traffic Engineering
• Bare Metal Switching
• Inband Network Telemetry
 Network Virtualization – An Analogy

Application Application Application Workload Workload Workload

x86 Environment L2, L3, L4-7 Network Services

Virtual Virtual Virtual Virtual Virtual Virtual
Machine Machine Machine Network Network Network
Hypervisor Decoupled Network Virtualization Platform
Requirement: x86 Requirement: IP Transport

Physical Compute & Memory Physical Network

2009

22
2012

23
 r
vSw pe
Hy
h r
itc rviso
vSw ype
H
h r
itc rviso
vSw ype
H
h r
itc rviso
vSw ype
H r
h
itc rviso
vSw ype
H
Virtual Machines to Virtual Networks

h r
itc rviso
vSw ype
H
h r
itc rviso
vSw ype
H
h r
itc rviso
vSw ype
H
Network, storage, compute
Virtualization layer
 r
vSw Hype
h r
itc r viso
vSw Hype
h r
itc rviso
vSw Hype
h r
itc rviso
vSw Hype
h r
itc rviso
vSw ype
H
Virtual Machines to Virtual Networks

h r
itc rviso
vSw ype
H
h r
itc rviso
vSw ype
H
h r
itc rviso
vSw ype
H
Network, storage, compute
“Network hypervisor”
Virtual Data Centers

Virtualization layer
 Network Virtualization Components

Cloud Consumption • Self Service Portal

• OpenStack, Kubernetes, etc

• Single configuration portal

Manager • REST API entry-point

• Manages Logical networks

• Run-time state
Controller • Scale out, HA
• Separation of Control and Data Plane

• High–Performance Data Plane

Data Plane • Scale-out Distributed Forwarding Model

Distributed Services Virtual Edge

• Logical Switch
vSw Hype

• Distributed Logical Router

itc rviso
h

• Firewall
• Load Balancer
r

26
 Management, Control and Data Planes
Network topology
request

Request stored
and acknowledged
MANAGEMENT
PLANE

Desired State
Calculate data
CONTROL plane state
PLANE

Discovered State Translated State

DATA
PLANE Identify data plane
resources

Realized State
 Problem: Data Center Network Security
Perimeter-centric network security has proven insufficient

Internet

VM

VM
VM

VM

IT Spend Security Spend Security Breaches

Today’s security model focuses on perimeter But continued security breaches show this model is
defense not enough
 Microsegmentation and Zero Trust

Perimeter
firewall
DMZ VLAN

VM VM VM VM VM VM

Inside App VLAN Finance HR IT

firewall

VM VM VM VM VM VM

Finance HR IT
DB VLAN

VM VM VM VM VM VM

Finance HR IT

Services VLAN

VM VM VM VM VM

AD NTP DHCP DNS CERT

 Visibility: changing the laws of physics

 Historically challenging to troubleshoot connectivity between VMs

• Is the problem in vswitch or physical network?
• What’s the path through the physical network?
• Is there a (misconfigured) middlebox in the path?
 Network virtualization gives us tools to handle this:
• Decomposition: separate the physical from the virtual
• Global view: see all the logical network state (port stats, drops, etc.) and tunnel
health from the controller API
• Synthetic traffic: insert packets at vswitch as if the VM generated them
 Network Virtualization – Discussion

• 90% of Fortune 100 have deployed network virtualization

• Foundational to hyperscale data centers
• Network configuration no longer the “long pole”
• A key step towards better network security (but much work remains)
• Increasingly important for microservices, kubernetes etc.
• Commodifying effect on physical networking
• Service Mesh can be viewed as a form of Network Virtualization
 SD-WAN
Network Policies

SD-WAN Controller

Cloud Services

Corporate
Branch Datacenter

Overlay Tunnel

SD-WAN Edge Main Office

 Traffic Engineering
Network Policies

Controller

Datacenter Datacenter

Datacenter
 Datacenter Switching Fabric
Internet

Leaf-Spine Topology
• Leaf Switches = Top-of-Rack (ToR)
• Optimized for East-West Traffic
Spine Spine Spine
• Built-in Redundancy (not shown)
• Scale with additional layers

Well-Established in Commodity Clouds

• Bare-Metal Switches
Leaf Leaf Leaf Leaf • Control Plane running in the cloud
 Leaf-Spine Switching Fabric

Spine Spine Spine Trellis Design

• Intra-Rack: L2 Domain within L3 Subnet
• Inter-Rack: L3 Routing between Subnets
• Segment Routing across Fabric
Trellis Features
Leaf Leaf Leaf • VLANs / QinQ
Leaf
• End-to-End L2 Tunnels
• IPv4 / IPv6 Routing
• Multicast (with IGMP)
• ARP (IPv4) / NDP (IPv6)
• DHCPv4 / DHCPv6
• High Availability
 Inband Network Telemetry (INT)
Add Switch ID, arrival time, Generate report with
departure, queue delay, etc. switch metadata
Header
Header Metadata S1
Metadata S1 Metadata S2

Header Payload S2 Payload Header

Payload Payload

S1 S3 S5
Metadata S1
Metadata S2
Fine-Grain Telemetry
• Metadata S5
Flow Rule(s) that matched S4
• Queuing delays of individual packets
• Other flows being buffered
• … Log, analyze,
replay, visualize
Uses
• Verify correct behavior
• Identify micro-bursts
• …
 SDN Challenges

• Scale
• Stability & Correctness
• Timeliness
• Inter-domain
Discussion