See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/323925422

Performance Evaluation of Openflow SDN Controllers

Chapter · March 2018

DOI: 10.1007/978-3-319-76348-4_87

CITATIONS READS

6 5,134

1 author:

Sangeeta Mittal
Jaypee Institute of Information Technology
59 PUBLICATIONS 354 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Human Activity recognition View project

All content following this page was uploaded by Sangeeta Mittal on 26 November 2019.

The user has requested enhancement of the downloaded file.

 Performance Evaluation of OpenFlow SDN Controllers

Sangeeta Mittal[0000-0002-8323-7681]

Jaypee Institute of Information Technology, Noida

[email protected]

Abstract. Software Defined Network (SDN) is a recent networking paradigm

being adopted by stakeholders in a big way. The concept works towards dra-
matically reducing network deployment and management costs. Controllers, al-
so known as Network Operating System of SDN, are critical to the success of
SDNs. Many open source controllers are freely available for use. In this paper,
the performance of four popular OpenFlow based controllers has been evaluat-
ed on various metrics. Latency, Bandwidth utilization, Jitter, and Packet loss
has been calculated for TCP and UDP traffic on varying network sizes, topolo-
gies, and controllers. Floodlight has been found to be overall best performer as
compared to other used controllers.

Keywords: Software Defined Networks, Mininet, Floodlight, POX,

OpenDaylight, Bandwidth Utilization, Latency.

1 Introduction

With the advent of technology, cost of storage and processing power is decreasing
sharply with every passing year. The cost of networks is not in tandem with this
downfall and has thus become the bottleneck of overall cost reduction problem. Soft-
ware Defined Network (SDN) is recent solution to the problem of cost effective
scalability of traditional networking. The paradigm is about centralized control of the
network implemented by decoupling control logic from forwarding devices. A sim-
plistic architecture of SDN is shown in Figure 1. Apart from usual networking devic-
es, some additional components are SDN Controller, SDN applications, Northbound
Interfaces and Southbound Interfaces [1].
SDN Controller is the centralized entity to control the network. All network or-
chestration functions like how switches should handle network traffic, topology man-
agement among others are implemented in it. Northbound Interface is set of APIs
defined to communicate with user domain network administration applications to help
network engineers in programmatically deploying network services, implementing
business logic and keeping track of network health. Southbound Interface deals with
communication between controller and forwarding devices such as hubs, switches and
routers. There are a number of southbound APIs like OpenFlow, OVSDB, Forward
and Control Element Separation and Protocol Oblivious Forwarding. However,
OpenFlow is most popular and has become the de facto Southbound API [2].
2

Fig.1.Simplified SDN Architecture [1]

Along with reduced time to launch and expand a network, SDN concept of net-
working offers numerous advantages in terms of programmable, agile and flexible
network leading to ease of innovations in network design [2-4]. All these advantages
are possible due to centralized control implemented by SDN Controllers. In this pa-
per, it is intended to analyze the network performance on using different OpenFlow,
the most common Southbound API, based SDN controllers.
Rest of the paper is organized as follows. Some work already done towards this prob-
lem has been cited in Section II. In Section III, various SDN controllers selected for
evaluation and network emulation software have been described. Performance met-
rics, results obtained and their analysis has been discussed in Section IV. The paper is
concluded in Section V.

2 Related Work

There have been several attempts at evaluating the performance of SDN Controllers.
Authors in [5] have evaluated five open-source SDN controllers. Experiments have
been done on benchmark tests as well as other system settings. They have claimed
that Beacon controller is most scalable on increasing number of switches. In [6], three
existing controllers have been compared against a custom designed controller defined
in the paper. However, in both of these papers, controllers have not been evaluated on
various common topologies of network. Authors in [7] present an insight into detailed
evaluation of OpenDaylight controller system messages and protocol setup. This work
gave a detailed knowledge of working of ODL and highlights possibility of degraded
performance of the controller on increasing scale. Quantitative impact of ODL on
increasing network scale however, has been omitted in this work too. Authors in [8]
report their experiences in doing performance comparison of POX and Floodlight
Controller with different inbuilt and custom defined topologies. The work done in this
 3

paper is similar to it, but with increased number of controllers and network settings.
Authors in [9] provide a feature wise comparison of various SDN Controllers, their
Mininet compatibility and visualization capabilities. Mininet is one of the most popu-
lar lightweight network emulator. The tool has been introduced with examples and
some basic performance studies in [10].
Taking cues from these research articles, a comprehensive evaluation of four SDN
controllers using Mininet as network emulation software has been done on varying
topologies and network sizes. An empirical analysis of performance of each controller
in terms of various relevant metrics has been presented in results section.

3 Implementation of Software Defined Network

Implementation of SDN requires a network emulation tool compatible to the SDN

controllers. In this section, Mininet tool used for network emulation along with cho-
sen SDN controllers has been described.

3.1 SDN Emulation in Mininet

Mininet is a network emulation tool [10]. It provides functionality for easy creation of
various network components like:
(a) End Hosts: - Servers or end user machines implemented to run user applications
that generate data packets. Hosts are assigned easy to use MAC and IP addresses.
(b) Open Virtual Switches: - End hosts are connected to one or more switches. These
are implemented in Mininet as Open Virtual Switches (OVSwitch) that don’t have
any logic of its own and it depends on controller for forwarding any traffic.
(c) Links between Hosts and Switches: - Easy configuration of links with desired
characteristics of delay and bandwidth is a main feature of Mininet. Three predefined
topologies available in Mininet, that is, star, linear and tree have been tested in this
paper. Number of hosts and switches in each are listed in Table 1.

Table 1.Topologies Tested with Each Controller.

Name #of Switches #of Hosts

Single 1 n
Linear n n
Tree 2depth -1 fanout * fanout (for
depth =2)

3.2 SDN Controllers

Various OpenFlow based controllers evaluated in this work are listed in Table II. First
one is the OpenFlow reference controller implemented as bridging switch. It is also
called ovs-controller or test-controller and is currently the default controller for
Mininet. This controller has been used as a baseline controller to compare with others.
POX controller is an open source controller available for download on github [11]. It
4

is written in python and comes bundled with Mininet virtual machine. POX has been
tested on localhost by running SSH login of Mininet VM.

Table 2. Evaluated SDN Controllers.

Name Developed and Maintained by Development Location

Language
OVS- Control- Default Controller Python Reference
ler Controller
POX Open Source Development Plat- Python Internal
form
Floodlight Big Switch Networks Java External
OpenDaylight Cisco and OpenDaylight Java External

The next controller chosen for testing is Floodlight of big switch networks. It is writ-
ten in Java and has been designed for actual heavy traffic of real data centers. The
project is available for free download from its project site at [12]. Apart from support-
ing several switches, Floodlight also has a collection of applications for traffic engi-
neering. For testing Floodlight, Floodlight VM was downloaded from the project site
and implemented in virtual box as an Ubuntu OS based system.
The last controller used here is the OpenDaylight controller. It is a popular controller
and is supported by bigwigs like Cisco. It is a Java based project initiated in 2013 and
continuing to evolve till now. In this work, the most recent Carbon distribution of the
controller has been used. OpenDaylight has certain prerequisites of exclusive java
version. Distribution package is unzipped in an Ubuntu machine and OpenDaylight is
initiated by running ./bin/karaf file in it. OpenDaylight has GUI frontend called Dlux.
It enables us to view the topologies in a browser based window. Figures 2 (a) – (c)
show tested topologies with 16 hosts.

(a)
 5

(b)

(c)

Fig.2. Arrangement of 16 nodes in (a) Star (b) Linear (c) Tree (depth=2, fan-out=4) Topolo-
gies

4 Experiments and Results

The topologies described in last section were implemented in Mininet on varying

network sizes by connecting to different controller every time. In this section, perfor-
mance metrics used to draw comparisons and results of evaluation has been discussed.

4.1 Performance Parameters

6

1) Bandwidth Utilization: Iperf tool has been used to measure bi-directional transfer
of data. Iperf also gives bandwidth utilization in terms of speed with which data is
transmitted between client and server.
For all experiments, Mininet default TCP window size of 85.3 Kbytes has been used.

2) Latency: Round Trip Transmission time (RTT) has been used to calculate delay or
latency of packet transmission between two nodes. In SDN, the first pings always take
more time than the subsequent ones due to route queries to controller in first attempt.
Therefore, to get fair idea of delay, two RTTs has been considered namely RTT_1
and RTT_2. These are computed by taking RTT of two back to back execution of
following command.
Mininet> h1 ping –c5 h16

3) Jitter: - It is the variation in response time between a pair of hosts. Jitter has been
calculated using iperf’s UDP performance testing options. For example following
command will test the jitter between host h1 and host h16. UDP packets were sent for
10 seconds and report of jitter after every 1 second has been generated. However, in
evaluation average jitter of 10 seconds has been used.

h1 iperf –s h16 –u –i 1&

h16 iperf –c h1 –u –d

4) Datagram loss: UDP is non-reliable protocol. The number of datagrams that were
lost during UDP based transmission is a measure of reliability of the link in routing
UDP traffic. It can also be measured with an Iperf UDP test described in jitter.

4.2 Results

Experiments were conducted on an Intel CORE i3 Laptop with 6 GB RAM. Each

controller was tested on supporting all three types of inbuilt topologies with number
of end hosts starting from 16 and doubling subsequently upto 1024. Beyond that
number, Mininet wasn’t able to create the network due to configuration limitations of
host computer. All controllers and Mininet were implemented as Virtual Machines in
Oracle’s Virtual Box Virtualization software.
Figure 3 compares RTT_1 and RTT_2 of different topologies. This study was done on
reference controller with 16 hosts in each topology. Both RTTs have been computed
as average of 5 ping packets. RTTs stabilized after 5 packets, thus in later experi-
ments, this has been considered as ideal for measuring first and second RTTs. As
expected by the number of links, RTT is maximum in linear topology followed by
tree.
Effect of size of network on latency was also studied for all cases. Linear topology
offers maximum load to the controller, therefore, its results have been shown as worst
case performance. Figure 4 and 5, demonstrate latency experienced for RTT_1 and
RTT_2, which is time when route should have been pre-established for linear topolo-
 7

gy. The numbers in the figure 5 represent significantly reduced delay, except in case
of internal reference controller which showed unusual high latency even on se-
cond_RTT round. OpenDaylight controller couldn’t support more than 16 switches
and thus results have not been shown for it.
As shown in Figure 6, interestingly, in linear topology each controller was dropping
some UDP datagrams and hence loss percentage was non-zero. This number is very
small for all controllers except the POX controller that dropped high number of pack-
ets for larger networks with 64- 1024 nodes. For Floodlight and reference controller,
this number remained less than 10 for all configurations.

Fig. 3. RTT_1 and RTT_2 for a network of 16 Nodes Arranged instar, Linear and Tree To-
pologies.

Fig.4. Latency (in ms) in Tree Topology in terms of RTT_1

8

Fig.5. Latency in Linear Topology in terms of RTT_2

Fig.6. Controller Wise number of UDP Datagram lost in Linear Topology

The next set of experiments was done on tree topologies. There are several possi-
bilities for generating a particular number of end hosts in tree topology by varying
depth and Fanout of the tree. However, here all tree topologies are generated with
depth 2 and doubling the Fanout in each run. This resulted in 4 equivalent sized net-
works as compared to earlier studies. These sizes are 16, 64, 256 and 1024 nodes.
Interestingly, in tree topology OpenDaylight Controller performed worst. Its band-
width utilization was multiple times lesser than other controllers. However, in jitter
and latency this controller performed one of the best. Floodlight showed optimized
performance in terms of all parameters. Hence, for overall performance it may be
considered best.
 9

Table 3. Performance of Various SDN Controllers in Tree Topology

TCP Stats TCP Stats of

of Client Server
Band Band
#of widt widt
swi Tran h Trans h
tch #of #of RTT_ RTT sfer (Gbp fer (Gbp Jit-
es links nodes Controller 1 _2 (GB) s) (GB) s) ter
OpenDayli
ght 0.87 0.53 0.08 0.07 0.03 0.02 0.05
Pox 16.40 0.34 9.91 8.51 9.56 8.21 0.03
5 20 16
Floodlight 19.50 0.54 11.00 9.46 10.0 9.38 0.02
Reference 59.10 0.98 8.55 7.35 8.15 7.00 0.01
OpenDayli
ght 1.30 0.30 0.01 0.01 0.04 0.03 0.03
9 72 64 Pox 69.10 0.90 9.76 8.38 9.50 8.16 0.04
Floodlight 27.60 0.55 8.92 7.66 7.34 6.31 0.02
Reference 33.10 0.73 9.17 7.87 9.42 8.09 0.01
OpenDayli
ght 2.53 1.50 0.01 0.01 0.01 0.01 0.15
17 272 256 Pox 60.60 0.91 9.60 8.60 9.76 8.38 0.01
Floodlight 59.90 2.08 6.52 5.60 7.55 6.48 0.03
Reference 119.00 1.62 5.38 4.62 5.62 4.83 0.01
OpenDayli
ght 4.67 2.90 0.002 0.001 0.004 0.002 0.32
33 1056 1024 Pox 146.00 1.01 3.75 3.22 2.58 2.21 0.02
Floodlight 94.30 1.06 6.23 5.35 4.86 4.18 0.03
Reference 76.90 0.76 6.69 5.75 6.31 5.41 0.01

4.3 Discussion
The results depict that Foodlight controller has an edge over all other controllers due
to consistent performance across all metrics. This was also experienced during exper-
imentation that this controller worked smoothly with all topologies and all sizes. On
the other hand both POX and OpenDaylight Controller were stressed out when either
the number of hosts or the number of links (linear topology) were increased. From the
current set of controllers thus, Floodlight is an ideal choice for designing SDN due to
is consistent performance across all chosen metrics. The controllers can further be
tested with benchmark tests to establish the relevance of this fact more robustly. In
this paper, the motive was to give baseline results for a beginner researcher starting to
program the SDNs in Mininet. The obtained results tally with those obtained in earlier
works discussed in related work section. However, the results given in this paper,
gives more detailed insights into where things go wrong for various controllers. For
 10

example, it has been clear that OpenDaylight Controller cannot handle load offered by
linear topology and POX controller suffers from high data loss in linear topologies.

5 Conclusion

Software Defined Networks is an upcoming networking paradigm and is a topic of

recent interest among researchers. In this paper, a much needed performance compari-
son study of open source OpenFlow based popular controllers has been done on vary-
ing topologies and network sizes. The detailed results give useful insights into the
performance of each controller on different metrics. It has been concluded that out of
the considered set of controllers, Floodlight controller performs consistently across
metrics and can be considered current best. It can thus be an ideal choice for further
design and testing of new SDN protocols.

References
1. Network Virtualization Report: SDN Controllers, Cloud Networking and More - Online
Edition, https://www.sdxcentral.com/sdn 2017, last accessed 2017/10/10.
2. Hu F., Hao Q. and Bao K.: A Survey on Software-Defined Network and OpenFlow: From
Concept to Implementation.IEEE Communications Surveys & Tutorials16(4), 2181-2206
(2014).
3. Wenfeng, X et al.: A Survey on Software-Defined Networking. IEEE Communications
Surveys & Tutorials 17(1), 27-51 (2015).
4. Kreutz, D. et al.: Software-Defined Networking: A Comprehensive Survey. Proceedings of
the IEEE 10391), 14-76 (2015).
5. Zhao, Y., Luigi, I., and Riguidel, M.: On the performance of SDN controllers: A reality
check. In: Proceedings of IEEE Conference on Network Function Virtualization and Soft-
ware Defined Network (NFV-SDN) (2015).
6. Tootoonchian, A., et al.: On Controller Performance in Software-Defined Networks. In:
Proceedings of 2nd USENIX Conf. Hot Topics Manage. Internet Cloud Enterprise Netw.
Services, pp. 1-6 (2012).
7. Zara, S. et al.: Performance Debugging in SDN Controllers—A Case Study (Short Pa-
per).In: Proceedings of 5th IEEE International Conference on Cloud Networking (Cloud-
net) (2016).
8. Bholebawa I. Z., Jha R. K., and Dalal U. D.: Performance Analysis of Proposed Open-
Flow-Based Network Architecture Using Mininet. Wireless Personal Communication,
86(2), 943-958, (January 2016).
9. Nadeau T., Gray K.: SDN: Software Defined Networks, O'Reilly Media (2013).
10. Kaur, K., Singh J., and Ghumman N. S.: "Mininet as Software Defined Networking Test-
ing Platform. In: Proceedings of International Conference on Communication, Computing
& Systems (ICCCS),(2014).
11. POX Controller, https://github.com/noxrepo/pox, last accessed on 2017/10/10.
12. OpenDaylight Controller, https://www.opendaylight.org/, last accessed on 2017/10/10.
13. Floodlight Project, http://www.projectfloodlight.org/floodlight/, last accessed on
2017/10/10.

View publication stats