IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 17, NO.

12, DECEMBER 2018 2953

Towards Optimal Network Planning for

Software-Defined Networks
Shih-Chun Lin , Member, IEEE, Pu Wang , Member, IEEE,
Ian F. Akyildiz, Fellow, IEEE, and Min Luo, Senior Member, IEEE

Abstract—Supporting on-line and adaptive traffic engineering in software-defined networks entails the fast, robust control message
forwarding from software-defined switches to the controller(s). In-band control using the existing infrastructure is cost-efficient, but
imposes a substantial barrier to timely transmissions of control messages. Also, due to the limited computational capability of a single
controller, only the use of multiple controllers is practically viable for large-scaled networks. Therefore, in this paper, the optimal
software-defined network planning is investigated with multi-controllers. First, the network planning problem is formulated as a
nonlinear multi-objective optimization, which aims to simultaneously minimize the number of controllers and the control traffic delay
for each switch. This planning problem is then partitioned into two sub-problems, i.e., multi-controller placement and control traffic
balancing, which are respectively solved by the proposed fast-convergent algorithms. Furthermore, an adaptive feedback control
mechanism is proposed to iteratively work out the two sub-problems and enable the dynamic network replanning, subject to the time-
varying traffic volume and network topology. Simulations validate the adaptivity of our control scheme, which significantly reduces
delay with maximum throughput for control flows, brings minimal impact to normal data flows, and requires the minimum controllers.

Index Terms—Traffic statistics, optimal network planning, in-band control, randomized rounding, control traffic balancing, software-defined
networks

Ç
1 INTRODUCTION

A Nemerging networking paradigm that separates the

network control plane from the data forwarding plane
is software-defined networks (SDNs). As a promising mech-
controller that defines the behavior and operation of the
network forwarding-infrastructure.
Through SDNs, when a new flow is initiated and no local
anism for dramatically improving network resource utiliza- forwarding policy is defined in the flow table, the switch
tion, simplifying network management, reducing operating must forward the first packet of the flow to the controller,
costs, and promoting innovation and evolution [1], [2], [3], which determines an appropriate forwarding path. As a
SDNs have shown great potential for data center networks result, the timely and reliably delivery of control messages
and the next-generation Internet [4], [5]. Recently, they (e.g., the first packet of every new flow, network traffic
have been extended to support wireless communication net- statistics, and flow instructions to all switches along the
works [6], [7]. The main functions of SDNs are (i) to separate selected path) for each software-defined or OpenFlow
the data plane from the control plane and (ii) to introduce switch largely impacts the performance of SDNs. However,
novel network control functionalities based on an abstract it is challenging to create scalable SDN solutions by only
representation of the network. In current instantiations of adopting a single controller. To this end, the placement of
SDNs, these functions are realized by (i) removing control multiple controllers across the entire network can address
decisions (e.g., routing) from the hardware (e.g., switches the performance limitation and also the “single point of fail-
or routers), (ii) enabling programmable flow tables in the ure” issue of a single controller while retaining the benefit
hardware through an open, standardized interface (e.g., of network control centralization. In this case, several fun-
OpenFlow [8]), and (iii) using a logically centralized network damental network planning problems have to be solved
regarding (i) the minimum number of controllers, (ii) their
optimal deployment locations, (iii) control domain assign-

S.-C. Lin is with the Department of Electrical and Computer Engineering, ments between switches and controllers, and (iv) the opti-
North Carolina State University, Raleigh, NC 27606. mal control traffic forwarding paths between switches and
E-mail: [email protected].

P. Wang is with the Department of Computer Science, University of North their corresponding controllers. When an in-band control
Carolina at Charlotte, Charlotte, NC 28223. E-mail: [email protected]. channel [8] is used, both control and data traffic have to

I.F. Akyildiz is with the Broadband Wireless Networking Laboratory, share the same forwarding infrastructure, and such a prob-
School of Electrical and Computer Engineering, Georgia Institute of Tech- lem becomes even more prominent.
nology, Atlanta, GA 30332. E-mail: [email protected].

M. Luo is with the Shannon Lab, Huawei Technologies Co., Ltd., Santa
This paper proposes an optimal network planning frame-
Clara, CA 95050. E-mail: [email protected]. work for SDNs. The objective of this framework is to find
the optimal controller placement and control traffic for-
Manuscript received 24 Oct. 2015; revised 7 Feb. 2018; accepted 3 Mar. 2018.
Date of publication 14 Mar. 2018; date of current version 1 Nov. 2018. warding paths for OpenFlow switches in such a way that
(Corresponding author: Shih-Chun Lin.) the required number of controllers is minimized and at
For information on obtaining reprints of this article, please send e-mail to: the same time, the control traffic delay is shorter than the
[email protected], and reference the Digital Object Identifier below. predefined threshold with a high probability. Towards this
Digital Object Identifier no. 10.1109/TMC.2018.2815691

1536-1233 ß 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See ht_tp://www.ieee.org/publications_standards/publications/rights/index.html for more information.

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 2954 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 17, NO. 12, DECEMBER 2018

objective, we first formulate the optimal planning problem different from [13] which only focuses on control traffic for-
as a nonlinear multi-objective (mixed integer and continu- warding in single-controller SDNs, this paper considers the
ous) optimization in which the number of controllers is an complete SDN planning which investigates the controller
integer-valued objective and the traffic delay belongs to a placement and control traffic balancing for more generic
continuous one. To solve such a complex optimization prob- multi-controller cases. In addition, while our previous work
lem in a timely manner, we partition the original problem studies the average network delay alone, both the average
into two sub-problems: multi-controller placement (MCP) and maximum delay are examined in this paper.
and control traffic balancing (CTB). The MCP problem aims To the best of our knowledge, this work is the first to
to find the minimum number of controllers and their respec- address the optimal network planning problem in multi-
tive deployment locations to cover all switches, while the controller SDNs, along with the provably fast-convergent
CTB problem aims at finding delay-optimal forwarding algorithms for near-optimal solutions. The rest of the paper
paths for control traffic between the switches and the is organized as follows. Section 2 provides related work,
deployed controllers. Finally, by considering the ever- and Section 3 presents the system model. Section 4 concerns
changing network dynamics, we introduces an adaptive the optimal network planning problem. To solve this plan-
feedback-control mechanism that iteratively solves the MCP ning problem, Section 5 proposes the optimal multi-control-
and CTB problems until the desired delay is met. ler placement, and Section 6 gives the control traffic
Specifically, we first prove that MCP is NP-Complete and balancing for multi-controller cases. Section 7 provides per-
show that MCP is an integer programming (IP) problem. formance evaluations, and Section 8 concludes the paper.
Then, we propose an approximation algorithm via random-
ized rounding, which yields a feasible optimal solution in at 2 RELATED WORK
most two iterations in average. Next, as the CTB sub-problem
belongs to a non-separable nonlinear continuous optimiza- SDNs, which require many signaling events and control
tion, its complexity is extremely high as a result of (i) its non- plane operations [2], [14], could easily generate a significant
linearity and (ii) massive variables of link traffic assignments amount of control traffic that must be addressed together
for large-size networks (i.e., combinatorial explosion of flow- with data traffic. However, existing studies [5], [15], [16],
to-link traffic assignments). To this end, we first analyze the [17], [18], [19] mainly focus on balancing data traffic in the
fundamental structure of CTB by proving its polynomial- data plane, such as prioritizing interactive, elastic, and back-
time complexity (i.e., its polynomiality [9]) under nonlinear ground traffic in [5]. In [15], the authors propose an integra-
optimization. Particularly, we prove that CTB is a strictly con- tion of dynamic load balancing, multi-path forwarding, and
vex optimization problem and the solution can be approxi- congestion control with the ability of per-flow and per-packet
mated by a polynomial-time fast algorithm. Furthermore, traffic splitting for data-center networks. A class-based traffic
motivated by the polynomiality analysis, we develop a poly- recovery with load balancing in [16] that supports SDN
nomial-time approximation algorithm (PTAA) for the CTB resilience for the automatic reconfigurability of traffic-path
problem that yields the optimal solution with convergence failures is introduced. Moreover, an adaptive resource man-
rate Oð1=cm Þ with constant c > 1 and iteration number m. agement framework with controller and manager placement
Such fast convergence is based on the adopted alternating in [17] that supports adaptive load balancing and energy
direction method of multipliers (ADMM) [10], an emerging management. Different from data traffic balancing, which
parallel and fast first-order method for solving large-scale aims to evenly distribute data traffic flows among network
convex optimization problems. Third, we propose the control links, control traffic balancing is much more challenging, par-
scheme that statistically ensures the control traffic delay is ticular for in-band control [13]. It aims to find the control mes-
lower than the desired threshold and only few controllers sage forwarding paths of each switch in such a way that the
are needed. control message delay can be minimized for the original data
We summarize our major contributions as follows: traffic, subject to acceptable performance. This control traffic
forwarding problem is extremely critical in SDNs, because
(1) We formulate the optimal SDN planning problem the timely delivery of control traffic initiated by OpenFlow
with respect to the minimization of required control- switches (i.e., the first packet of every new flow and the traf-
lers and network latency, e.g., the average network fic/congestion status) directly impacts the effectiveness of
delay and the maximum link delay. routing strategies determined by the controller.
(2) We partition the original problem into two sub-prob- In addition, several recent studies [20], [21], [22], [23],
lems and propose two fast-solving algorithms that [24], [25], [26], [27] have focused on the controller placement
give the optimal placement and the forwarding problem from various planning perspectives. The pioneer
paths, respectively. work of Heller et al. in [20] first adopts the distance between
(3) Regarding the time-varying traffic volume and net- a controller and switches as the performance metric and
work topology, we propose an adaptive control evaluates several well-known network topologies to find
scheme for dynamic network replanning, which iter- the optimal controller location. In [21], dynamic controller
atively exploits the fast-convergent algorithms for provisioning (i.e., changing controller placement according
MCP and CTB. to time-varying flows) is addressed through an NP-hard
Built upon the minimum required controllers (from our integer-linear programming problem and two heuristic
designated randomized rounding), simulation results show solving algorithms. In [22], NP-hardness of reliability-aware
that the proposed PTAA outperforms the conventional solu- controller placement is proven, and two heuristic algo-
tions (i.e., shortest-path and multi-path control traffic for- rithms are examined in terms of the reliability of the control
warding) with at least 73 percent delay reduction, for both traffic path. In [23], capacitated controller placement (i.e.,
Internet2 OS3E [11] and Sprint GIP [12]. Note that this paper considering the loads of controllers) is studied to reduce the
is based on our preliminary research in [13]. However, required number of controllers and the loads of the busiest

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 LIN ET AL.: TOWARDS OPTIMAL NETWORK PLANNING FOR SOFTWARE-DEFINED NETWORKS 2955

controller(s) [8]. Fig. 1 shows that each OF-switch forwards

the traffic from a variety of networks, such as IP data net-
works and cellular networks. Whenever a new data flow is
generated, the responsible OF-switch sends the routing
request to the assigned SDN controller. Once the controller
receives the control message, it calculates the optimal routing
path(s) to the destination OF-switch and sets up the routing
tables of switches along the optimal path(s). There are two
types of control channels: dedicated out-band control or in-
band control. Out-band control uses a dedicated network to
establish direct control links between each OF-switch and its
responsible controller. Although such an approach might
bring more design flexibilities in rate control or multi-path
routing, it is cost-prohibitive and not suitable for large-scale
networks, such as metropolitan-area-networks that span a
city. On the contrary, in-band control allows control and
data messages to share the same forwarding infrastructure.
It is more cost-efficient, and a desired tradeoff can be
achieved between service performance and infrastructure
cost. We consider in-band control throughout this paper.
The SDN is modeled by a graph G ¼ ðV; JÞ as in Fig. 1b,
where V is the set of OF-switches with total n switches (i.e.,
jV j ¼ n) and J is the set of links with total jJj links. A control-
ler can be placed at any location among all OF-switches. We
denote the set of SDN controllers K  V with a total of C con-
Fig. 1. Network topology of SDNs.
trollers, and the serving time capability of controller k is mod-
controllers. Through the non-zero-sum game approach eled as an exponential distribution with mean time value
in [24], a placement framework is proposed with con- 1=mkC ; 8k 2 K, where this valid assumption of exponential
trollers’ payoff functions to ensure the optimal controller distribution has also been made by many related work of traf-
number and their mapping to OpenFlow switches for cost fic modeling in SDNs (e.g., [28], [29], [30]). Note that as we
savings and QoS improvement. Regarding small-scale look at average service rates (time) of controllers, i.e.,
SDNs for enterprises in [25], a simulation solution that mkC ; 8k 2 K throughout the paper, the assumption of expo-
simultaneously determines the optimal number, locations, nential service-time distribution can actually be extended to
types of controllers, and the interconnections with switches a much-wide class of random processes, i.e., regenerative
is provided. Differently, heuristic approaches for large-scale processes, without affecting the optimization problem forma-
SDNs in [26] provide operators with Pareto-optimal place- tion and solving methods in this paper. Since the effective-
ment, with respect to planning-performance metrics. Thus, ness and scalability of SDNs highly depend on the timely
the existing efforts only examine heuristic schemes, which delivery of control messages from OF-switches to multiple
controllers, optimal controller placement and control traffic
cannot yield guaranteed performance bounds.
balancing with regard to control and data traffic statistics are
More importantly, all of these solutions concern control-
the focus of this paper. To simplify the readability, we refer
ler placement alone without involving control traffic for-
to switches as OF-switches in the remainder of the paper.
warding. Lately, novel work in fast failover [27] considers
Regenerative stochastic processes [31] are generic traffic
both resilience-aware controller placement and control traf-
models that generalize several widely adopted Internet
fic routing to improve the resiliency of SDNs. In particular,
traffic models including alternating renewal processes,
exhaustive and greedy search algorithms are provided for
recurrent Markov chains, and reflected Brownian motion.
controller placement; conventional routing tree in which
Without loss of generality, both control and data flows are
each switch has only one path to the controller is adopted
modeled by regenerative processes. Specifically, in SDNs,
for control traffic forwarding. However, single-controller
there are two main sources for control flows: (i) new coming
SDNs with in-band connections in [27] are limited for a
data flows to request forwarding routes and (ii) network
large amount of control traffic. Moreover, no load balancing
maintenance after initial setup. For case (i), when a new
on control traffic exists in the literature, losing the capability
data flow comes to an OF-switch, the switch will send a first
of multi-path control traffic forwarding. Towards this end,
packet of the flow, as a control message, to a controller and
different from all existing solutions, we jointly investigate
request the controller to setup the routes for the data flow.
multi-controller placement and control traffic balancing for
This type of control messages might relate to the arrival
the design of optimal network planning and develop an
rates of new data flows, but contribute little amount of traf-
adaptive control scheme that guarantees the optimum solu-
fic volume as compared to case (ii). On the other hand, case
tion with fast replanning of controller placement and for-
(ii) brings most amounts of control traffic volume and
warding paths over time-varying QoS requirements, traffic
includes all control messages after initial SDN setup, such
statistics, and network topology in SDNs.
as network/traffic statistic collection, flow-table mainte-
nance for switches, etc. [1], [2], [3], [17]. Different from the
3 SYSTEM MODEL control messages from case (i), control flows from case (ii)
A SDN generally consists of a number of OpenFlow-enabled are independent to data flows. What is more important,
switches (i.e., OF-switches) and the centralized SDN owing to the significant difference in traffic amounts of

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 2956 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 17, NO. 12, DECEMBER 2018

TABLE 1 Furthermore, to avoid long-distance assignments for delay-

Important Notations Utilized in This Paper ing control message transmissions, indicator variables
fIik ; 8i 2 V; k 2 V g are introduced to enable the localized
Notation Description
domain assignments. In particular
G ¼ ðV; JÞ A SDN graph with OF-switch set V (where jV j ¼ n) and link set J


KV The SDN controller set with total C controllers 1; if Distanceði; kÞ  dist;
mkC Average serving rate of controller k Iik ¼ (3)
0; otherwise,
si Average arrival rate of control traffic Ai from switch i
j Average arrival rate of data traffic Bj in link j
where Distanceði; kÞ denotes the distance, such as the hop-
mj Average serving rate of link j
count, between switch i and controller k. dist is the prede-
yk Controller location indicator for switch k
zik Control domain assignment indictor for switch i and controller k
termined value by network operators or service providers
Iik Local domain assignment indicator for switch i and controller k
from their specific concerns Q or requirements. From these
xkij Control traffic rate in link j from switch i to controller k variable definitions, yk ¼ 1  i2V ð1  zik Iik Þ and
Pik The set of available in-band paths for switch i to controller k
Tik The topology matrix for switch i to controller k yk  zik Iik 8 i 2 V; k 2 V: (4)
Dave and Dmax Average and maximum delay over SDNs
To let each switch be assigned to a single dedicated control-
ler, the following constraint is further provided as
cases (i) and (ii), most of the time in practice, the total con-
X
trol traffic which includes both independent and dependent zik Iik ¼ 1 8 i 2 V: (5)
traffic sources will behave like the independent traffic with k2V
respect to the data traffic. Therefore, in this paper, we con-
sider that the control flows are independent from data flows As focusing on the optimal networking planning, our work
without losing the practicability. In particular, the control could be extended to account for resiliency in the case of
traffic of each switch i is modeled by a regenerative arrival controller failure by assigning each switch to more than one
process Ai with mean value s i . For link j and j 2 J, the controller, i.e., in Eq. (5). Moreover, to avoid the infinite
existing data flow follows a regenerative arrival process Bj delay over controllers’ incoming queues, serving capability
with mean value j , and link serving time Sj follows of each controller should be enough for the arrival control
another regenerative process with mean time 1=mj . messages of the assigned switches
X
4 OPTIMAL NETWORK PLANNING PROBLEM s i zik Iik < mkC 8 k 2 K  V: (6)
i2V
We aim to provide a network planning scheme that deter-
mines the optimal placement for multiple controllers and Eq. (6) facilitates the workload distribution among control-
balances link traffic loads according to the control and data lers and eliminates the queue congestion.
traffic dynamics. Specifically, we formulate the traffic- On the other hand, regarding multi-controller scenarios,
driven optimal network planning as a nonlinear multi- the traffic assignment matrix x ¼ ½xkij i2V;j2J;k2V , where xkij
objective optimization. Note that in Table 1, we provide the denotes the amount of control traffic in link j that originates
notations used throughout the paper. from switch i to controller k, is obtained with respect to
minimizing the network delay. To achieve load balancing,
4.1 Problem Formulation multi-path routing is adopted, in which the given Pik is a set
As SDNs provide the centralized control capability with the of available paths for switch i to controller k and
global view of network status, we address the traffic-driven i 2 V; k 2 K. This means switch i can forward the control
design of multi-controller planing with two objectives: (i) messages to controller k via jPik j available paths. To charac-
minimize the required controllers to reduce infrastructure terize possible multi-path routings of control flows, for the
cost via a time-efficient placement and (ii) minimize the link flow from switch i to controller k, we define topology matrix
transmission delay via an optimized traffic scheduling. Tik with size jJj  jPik j as follows:
More specifically, the controller placement over SDNs

should determine the number of required controllers, their 1; if link j lies on path p;
Tik ½j; p ¼ (7)
individual locations, and the control domain assignments 0; otherwise.
for each switch. From Section 3, each switch i’s location can
be a controller k’s location (i.e., k 2 V ). The following varia- Matrix Tik maps the traffic amounts from end-to-end paths
bles are defined to address the placement problem. into each communication link and should be always full col-
fyk ; 8k 2 V g denotes the controller locations as umn-rank to avoid redundant paths. Its left-inverse matrix

T1ik ¼ ½ti1 ; . . . ; tijJj  exists and has the size jPik j  jJj, where
k k
1; if a controller picks switch k0 slocation; tkij is the column vector that maps link j to all possible paths
yk ¼ (1)
0; otherwise, of switch i’s flow to controller k. tkij is obtained by multiply-
P ing T1 1
ik with the jth standard basis ej (i.e., tij ¼ Tik ej ). While
k
and the number of total controllers C becomes k2V yk . each switch i brings a control flow with the mean value s i ,
fzik ; 8i 2 V; k 2 V g denotes the control domain assignments
the switch where the assigned controller sites can directly
between switches and controllers as
send its flow to controller without going through the net-


1; if switch i is assigned to controller k; work (i.e., if zkk ¼ 1, then xkkj ¼ 0, 8k 2 K; j 2 J). We set up
zik ¼ (2)
0; otherwise. the equalities for the control flow conservation of switches as
kT1 k T e
ik ½xi1 ; . . . ; xijJj  k1 ¼ s i zik Iik , 8k 2 K; i 2 V :¼ V n fzkk ¼
k

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 LIN ET AL.: TOWARDS OPTIMAL NETWORK PLANNING FOR SOFTWARE-DEFINED NETWORKS 2957

1; 8k 2 Kg, where k kT and k k1 denote the transpose and Note that the planning problem in Eq. (12) belongs to a
1-norm of vector, respectively. Let dkij ¼ kT1 ik ej k1 , such
mixed integer and continuous two-objective optimization,
equalities can be further simplified as along with several nonlinear constraints and tremendous
X variables. The entire problem for the optimal values is very
dkij xkij ¼ s i zik Iik 8 i 2 Ve; k 2 K; (8) complicated to solve in a time-efficient manner (i.e., even
j2J
finding a feasible solution will require a certain amount of
which is the flow conservation constraint, implying that the computing time). Therefore, to provide fast solving strategy
control flow initiated by each switch is split into multiple for this complex framework, we divide the original problem
outgoing flows on the selected transmission links. In addi- in Eq. (12) into two successive sub-problems as follows:
tion, to balance the traffic loads among all links, every link
should have finite transmission delay. Such finite link delay Multi  Controller Placement ðMCP Þ Problem
conditions are equivalent to Find: 8 i 2 V; k 2 V
yk ; zik
XX X
xkij < mj  j 8 j 2 J; (9) Minimize C¼ yk (13)
i2Ve
k2K
k2V

which ensure the incoming traffic rates are less than the link Subject to ð4Þ; ð5Þ; ð6Þ;
service rates and link delays remain nonnegative. Moreover,
regarding the second objective of minimizing network
Control Traffic Balancing ðCTBÞ Problem
delay, two metrics are considered in terms of link’s average
delay. Specifically, Dave denotes the average network delay Find: xk 8 i 2 Ve; j 2 J; k 2 K
ij
and Dmax denotes the maximum average delay among all (14)
Minimize Dave ðxkij Þ or Dmax ðxkij Þ
links. With the aid of Little’s law [32], Dave and Dmax for the
control messages are respectively obtained as Subject to ð8Þ; ð9Þ:
P P To this end, MCP problem in Eq. (13) first decides the opti-
X k2K xij þ j
k
1
i2Ve  ; (10)
Dave ¼P P P P mal locations for multiple controllers; CTB problem in
s i þ j2J j j2J m 
i2Ve e k2K xij þ j
k
j i2V Eq. (14) then examines the traffic scheduling for such a given
placement. Moreover, to introduce an adaptive feedback-
1 control mechanism for system QoS performance (e.g., net-
Dmax ¼ maxj2J P P : (11)
mj  xkij þ j work delay and throughput), the results obtained by execut-
i2Ve k2K
ing CTB can further activate another round of MCP for
different placement solution. Such iterations will continue
Remark 1. Eq. (10) considers the end-to-end link delay in a until the QoS requirements are fulfilled. In the following, we
path. In particular, it stands because
P P for each link j 2 J first apply the randomized rounding technique to solve
new packets arrive with rate ð e xkij þ j Þ and MCP problem and obtain the placement variables (i.e., yk
P i2 PV k2K
stay average time of 1=½mj  ð k2K xij þ j Þ, whose
k
and zik ) in Section 5. Next, with regard to the given solution
i2Ve
product is the queue backlogs of link j. Furthermore, the of MCP, the PTAA is proposed to solve CTB problem for con-
total external arrival rate of control and data traffic in the trol traffic scheduling parameters (i.e., xkij ) in Section 6. The
P P adaptive feedback control is also included in Section 6.3.
network is ð s i þ j2J j Þ. Hence, by Little’s law,
i2Ve
the average end-to-end (network) delay of packets (i.e.,
the duration time for a packet from entering the SDN sys- 5 OPTIMAL MULTIPLE CONTROLLER PLACEMENT
tem until exiting the system) is obtained as (10). Alterna- VIA RANDOMIZED ROUNDING
tively, Eq. (11) simply selects the maximum of average In this section, we aim to solve MCP problem and determine
link delay. Note that as multi-path, multi-hop forwarding the following: (i) the minimum required number of control-
is possible for each switch-controller pair from Eq. (7), lers, (ii) the locations of controllers, and (iii) the control
end-to-end packet transmissions are considered implic- domain assignments between switches and controllers. In
itly through Eq. (8). particular, we first prove that MCP is NP-Complete. Next,
by showing that MCP belongs to integer programming [33],
Therefore, with the above definitions, we define the Opti-
mal Network Planning Problem as follows. we propose an approximation algorithm that yields a feasi-
ble optimal solution through linear relaxation and random-
Definition 1 (Optimal Network Planning Problem). ized rounding techniques.
Given a SDN modeled by G ¼ ðV; JÞ with multiple controllers
k 2 K  V , control traffic arrival rates s i , a set of topology 5.1 NP-Completeness of MCP Problem
matrices Tik , 8i 2 V; k 2 K, data traffic rates j , and link serving The decision version [34] of the MCP problem is first intro-
rates mj , 8j 2 J, the network planning optimization problem is duced in the following Definition 2.
Find: 8 i 2 V; j 2 J; k 2 V
xkij ; yk ; zik Definition 2 (Decision Version of MCP Problem). Given
X a SDN graph G ¼ ðV; JÞ, in which switch set V provides a set
Minimize C¼ yk
of potential controller locations, and a positive integer b, the
k2V (12)
decision version of MCP problem determines whether there is
Minimize Dave ðxkij Þ or Dmax ðxkij Þ such a subset K  V with jKj ¼ b that (i) for each controller
Subject to ð4Þ; ð5Þ; ð6Þ; ð8Þ; ð9Þ: P2 K itsk incoming
k P queue has finite delay (i.e.,
k2K Uðm C  i2V s i zik I ik Þ ¼ jKj where Uð Þ is a step

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 2958 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 17, NO. 12, DECEMBER 2018

function) and (ii) for each switch i 2 V there exists at least one LPMCP is solved to get an optimal fractional solution (OPT),
controller k 2 K for zik Iik ¼ 1. denoted as y0k ; z0ik , 8i 2 V; k 2 V . Next, these fractional solu-
tions are rounded to integer values, denoted as yk ; zik ,
The NP-completeness of the MCP problem is then 8i 2 V; k 2 V , via a randomized rounding procedure. The
provided. rounding procedure consists of two steps: (i) set all zik to
Theorem 1. MCP problem is NP-complete. zero; then, (ii) let zik ¼ 1 with probability z0ik and execute
this step for log ðnÞ þ 2 times, where n is the number of
Proof. First, we argue that the decision version of MCP switches in the network. Step (ii) yields an integer solution
problem belongs to NP. Given an instance of MCP, a veri-  yk ; zik Þ, where C ¼ P
ðC;  yk ; zik ) is a fea-
k . To ensure (C;
k2V y
fication algorithm can effectively check whether each sible solution to IPMCP , Step (ii) is repeated until each con-
switch has at least one controller in its neighborhood (i.e., troller has finite queue delay and the minimum number of
within the range dist in Eq. (3) of the switch) and whether required controllers C satisfies the condition that C  aC 0 ,
the P
P minimum Pnumber of required controllers is b (i.e., 0 0
where C ¼ k2V yk and a is a constant provided in line 9 in
k2K UðmC  i2V s i zik Iik Þ ¼ jKj ¼ b). Thus, MCP2NP.
k
Algorithm 1. The result of proposed algorithm for IPMCP is
Next, we show that the Minimum Dominating Set provided in the Theorem 2.
(MDS) problem [34], which belongs to NP-complete, is
polynomial-time reducible to MCP problem (i.e., Algorithm 1. Randomized Rounding for MCP
MDSP MCP). An instance of MDS is given by a graph
Input : MCP problem in Eq. (13).
G ¼ ðV; JÞ and a positive integer b  1. The objective of  yk ; zik Þ % Optimal controller placement
Output: ðC;
MDS is to determine P whether there P is such a dominating 1: Solve LPMCP . Let ðy0k ; z0ik Þ be the optimum solution.
set V0  V that k2V0 UðmkC  i2V s i zik Iik Þ ¼ b  1 and 2: zik 0; 8i 2 V; k 2 V
each element i 2 V is a neighbor (i.e., with the range dist) 3: while t  log ðnÞ þ 2 do
of at least one element in V0 . Following the idea of MDS, 4: zik 1 with probability pik ¼ z0ik
we construct an instance of MCP problem from the one of 5: t tþ1
MDS as follows. The sets V and J are defined as 6: end
7: repeat
V ¼ V [ k0 and J ¼ J,  where k0 is a new controller element
k0
P 8: line 3-6P
with mC > i2V s i and is within the range dist of at least  0
i2V s i zik Iik < mC ; 8 k 2 V and C  aC , where
k
9: until
one element in V. Then, the instance of MCP is obtained as 0 0
ymax :¼ maxk2V yk and a ¼ log 1=y0max 4C 0

a graph G ¼ ðV; JÞ and a positive integer b. Furthermore,

we prove that the original instance of MDS is a “yes Theorem 2. Let OPT denote the optimal solution of MCP prob-
instance” if and only if the created MCP instance is also a lem. Algorithm 1 yields a solution of Oðlog nÞOPT with high
“yes instance”. On the one hand, suppose the created probability.
instance of MCP problem has a solution K  V with Proof. As C 0 is the optimal number of controllers and C is
P P
k2K UðmC  i2V s i zik Iik Þ ¼ jKj ¼ b. By our construc-
k
the corresponding solution after the randomized  round-

tion, k0 is one of the most powerful controllers within cer- ing, the expected value Eð  follows E P
CÞ yk ¼
P  Q  P  Q k2V 
tain switches’ neighborhoods and thus k0 should be E k 1  i2V ð1  zik Iik Þ ¼ k2V 1  i2V Eð1  zik Iik Þ .
included in K. This implies that k0 is the element of K The second equality holds from the assumption that the
that satisfies finite delay condition. While V ¼ V  k0 , the controller selection of each switch is independent to other
instance of MDS then has a dominating set V0  V with switches. Given a rounds of randomized rounding, the
jV0 j ¼ b  1. On the other hand, suppose there is a domi- probability that the controller k is selected by the switch i
nating set V0  V with jV0 j ¼ b  1 in the original MDS (i.e., zik ¼ 1) when the rounding is done is
instance. Through the similar arguments, the minimum
" #
number of required controllers is b in the constructed MCP [
instance. Therefore, we have shown that MDS problem can Pr½zik ¼ 1 ¼ Pr z0ik ¼ 1 at round t
be reduced to MCP problem by the proposed construction. ta (15)
As our construction takes polynomial-time and MCP2NP, ¼ 1  ð1  z0ik Þa :
we can conclude that MCP problem is NP-complete. u
t
With Eqs. (15) and (3), we have
5.2 Linear Relaxation (LPMCP ) and Randomized
Approximation Algorithm for MCP E½1  zik Iik  ¼ Pr½zik Iik ¼ 0
Considering MCP problem formulation in (13), the objective

1 when Iik ¼ 0 (16)
function is the number of required controllers in SDNs and ¼ 0 a
ð1  zik Þ when Iik ¼ 1:
three constraint functions are all linear. Thus, this integer
programming problem IPMCP is also linear. By relaxing var-
Therefore, the upper bound of the expected number of
iables yk ; zik 2 f0; 1g to yk ; zik 2 ½0; 1, we get the relaxed lin- P  Q
controllers is obtained as E C ¼ k2V 1  i2V ð1 
ear programming problem LPMCP . That is, it follows IPMCP P   P  
along with yk ; zik 2 ½0; 1, 8i 2 V; k 2 V . The solution of z0ik Iik Þa  ¼ 1  ð1  y0k Þa  k2V 1  ð1  ay0k Þ ¼
P k2V
LPMCP provides the optimal solution of IPMCP . a k2V y0k ¼ aC 0 .
Given an MCP instance modeled by IPMCP , Algorithm 1 Next, we derive the probability that the number of
is proposed to solve such an integer programming problem controllers is large than daC 0 , where d > 0 denotes a
as follows. First, the relaxed linear programming problem given constant. Applying Chernoff bound [35], the

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 LIN ET AL.: TOWARDS OPTIMAL NETWORK PLANNING FOR SOFTWARE-DEFINED NETWORKS 2959

following inequality is obtained as rounding is given as

" #
aC 0
X eðd1Þ Pr½Some controllers have infinite queue delay
0
Pr yk  daC  : (17) X X
dd 1 (23)
k2V  ðy0k Þa  ðy0max Þa ¼ C 0 ðy0max Þa  ;
k2V k2V
4
The best polynomial bound that can be achieved P via the
proposed randomized rounding is then Pr½ k2V yk  where y0max :¼ maxk2V y0k and a ¼ log 1=y0max 4C 0 . Combining
ðd1Þ
daC 0   e dd  n13  18 , provided that aC 0  1 and
Eqs. (18) and (19) with Eq. (23), the proposed algorithm
d ¼ Qðlog n=ðlog log nÞÞ. It implies the approximation
yields a solution that is dðlog 1=y0max 4C 0 Þ
Oðlog nÞ times
ratio Qðlog n=ðlog log nÞÞ is achieved, when we apply at
least two rounds for the rounding (i.e., a  2). Further- the solution of LPMCP with the probability at least 1=2.
more, we can achieve the better constant bound by con- That is,
sidering the general value d and applying more rounds.   1
P  Pr C < daC 0 ^ Each controller has finite delay  : (24)
For d > 2e, Eq. (17) becomes Pr k2V yk  daC 0  2
0
2ðd1ÞaC ; for d < 2e, the Chernoff bound is simplified as
P  ðd1Þ2 0 This completes the proof. Note that both events in
Pr k2V yk  daC 0  e 4 aC . The value of optimal Eq. (24) can be verified in the polynomial-time. Other-
controller number C 0 is further considered in the follow- wise, we simply repeat the entire rounding process (i.e.,
ing two cases. For C 0 < 1=e2 , we let d ¼ 1 þ 1=C 0 > 2e lines 7-9 in Algorithm 1), and the expected number of
and get repetitions is at most two due to 1=2 in Eq. (24). u
t
" #
X 1 6 CONTROL TRAFFIC BALANCING FOR MULTIPLE
0
 2a  e2 <
a
Pr yk  daC ; (18)
k2V
4 CONTROLLERS
pffiffiffiffiffiffiffiffiffiffiffiffiffiffi In this section, we aim to solve the CTB problem based on
for C 0 > 1=e2 , we select d ¼ 1 þ ð2=C 0 Þ < 2e and
obtain the results from MCP and provide a coherent solution to
traffic-driven network planning. We first prove the polyno-
" #
X 1 miality [9] of the CTB problem. Based on this mathematical
yk  daC  e2 < :
0 a
Pr (19) analysis, we propose the PTAA scheme with the aid of
k2V
4 ADMM [10] (a fast first-order method), yielding fast conver-
gence to the optimal solution, and further propose the adap-
For both above cases, it suggests that a  4; thus, the
tive feedback control in Section 6.3.
approximation ratio Oð1Þ is reached, when we apply at
least four rounds for the rounding.
Finally, we consider the probability that some control- 6.1 Polynomiality of CTB Problem
lers have infinite queue delay after the randomized First of all, the minimization objective function in the for-
rounding. By the fact that a controller’s queue delay can- mulated optimization problem (14) belongs to a non-separa-
not be infinite if it was not selected by any switch, the ble nonlinear continuous function. As indicated by [9], to solve
probability that a controller k 2 V has infinite queue such problems, the leading methodology is to develop itera-
delay at round t can be upper bounded by tive and numerical algorithms, whose performance are
characterized by the convergence rate. Moreover, different
" #
X from linear problems, for the nonlinear problems, the length
k
Pr s i zik Iik > mC at round t of the solution can be infinite (e.g., when a solution is irra-
i2V
" # tional). Hence, the polynomiality of nonlinear optimization
X (20) problems is determined by the existence of the polynomial-
¼ 1  Pr s i zik Iik  mkC at round t time converged algorithms which can approximate the opti-
i2V mal solution in the solution space.
 1  Pr½zik Iik ¼ 0 8 i at round t:
Definition 3 (Polynomiality of Nonlinear Problem). A
From Eq. (16), the probability that the controller k is not nonlinear optimization problem is of polynomiality, if it has
selected by the switch i at round t is polynomial-time converged algorithm that provides optimal

solutions with pre-specified accuracy in the solution space.
1 when Iik ¼ 0
Pr½zik Iik ¼ 0 at round t ¼ (21) Theorem 3. CTB problem in (14) is of polynomiality.
1  z0ik when Iik ¼ 1:
P  Proof. We first prove that both average network delay Dave
Therefore, we have Pr i2V s i zik Iik > mkC at round t  in Eq. (10) and maximum average delay Dmax in Eq. (11)
Q
1  i2V ð1  z0ik Iik Þ ¼ 1  ð1  y0k Þ ¼ y0k . This implies the are strictly convex. The basic form of Dave is provided as
probability that a controller k has infinite queue delay BDave ðxÞ ¼ ðx þ j Þ=½mj  ðx þ j Þ, where mj and j are
after the rounding is upper bounded by the given constants and x is a single variable. Applying the
" # second derivations to variable x, we obtain the following:
X
Pr s i zik Iik > mC  ðy0k Þa :
k
(22)
d2 BDave ðxÞ 2mj ½mj  ðx þ j Þ
i2V
¼ : (25)
By Eq. (22) and union bound [35], the probability that dx2 ½mj  ðx þ j Þ4
some controllers have infinite queue delay after the

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 2960 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 17, NO. 12, DECEMBER 2018

The convexity of Eq. (25) is determined by the sign of

numerator, specifically mj  ðx þ j Þ. Furthermore, as the Find: 8 i 2 Ve; j 2 J; k 2 K
xkij and bkij
affine mappings P preserve
P the convexity, we replace x in P P
X k2K bij þ j
k
Eq. (25) with k 1 i2Ve
i2Ve k2K ij and get the multiple-to-one
x Maximize P P P P
P P s i þ j2J j j2J mj  ð e k2K bij þ j Þ
k
strictly convex function (i.e., ð k2K xij þ j Þ=½mj  i2Ve
k
i2Ve
i2V
P P 8 k
e
ð k2K xij þ j Þ), where Eq. (9) guarantees the
k
< bij ¼ xij 8 i 2 V ; j 2 J; k 2 K
> k
i2Ve P
strictly convexity. Finally, since the summation of strictly Subject to
> dk bk ¼ s i zik Iik 8 i 2 Ve; k 2 K :
: j2J ij ij
convex function is still strictly convex, the average delay ð9Þ
Dave is a strictly convex function. Similar arguments are
followed for the maximum average delay. In particular,
Proof. A set of auxiliary variables is first introduced as
the basic form of Dmax is provided as BDmax ðxÞ ¼ 1=½mj 
b ¼ ½bkij  e and bkij ¼ xkij , 8i 2 Ve; j 2 J; k 2 K.
ðx þ j Þ, and the corresponding second derivations to i2V ;j2J;k2K
variable x is Then, the dual problem is obtained from the standard
derivations and thus omits here. u
t

d2 BDmax ðxÞ 2 Given the dual problem in Theorem 4 and the penalty
¼ : (26) parameter r > 0 for the augmented Lagrangian [10], we con-
dx 2
½mj  ðx þ j Þ3 sider the update rules for primal variables xkij , bkij and dual
variables g kij , 8i 2 Ve; j 2 J; k 2 K. For x-update, the follow-
P P
With the affine mapping from x to k
k2K xij and
ing iteration is obtained:
i2Ve
Eq. (9), the multiple-to-one strictly convex function
P P rXXX k
1=½mj  ð k2K xij þ j Þ is obtained. Moreover, as xðmþ1Þ :¼ arg min
kðmÞ kðmÞ
i2Ve
k
ðxij  bij þ g ij Þ2 : (27)
ð9Þ 2
the maximum of strictly convex functions is still strictly i2Ve
j2J k2K
convex, the maximum delay Dmax is proven as a strictly
convex function. Thus, (14) belongs to a strictly convex To simplify Eq. (27), let n e ¼ n  jfzkk ¼ 1; 8k 2 Kgj, xj ¼
optimization framework for either Dave or Dmax with lin- P P  ðmÞ P P kðmÞ ðmÞ P
k
nC, bj ¼
k2K xij =e

k2K bij =enC, g j ¼
i2Ve i2Ve i2Ve
ear constraint functions. P kðmÞ  Then, xk ¼ bkðmÞ  g kðmÞ þ xj  b ðmÞ þ g ðmÞ
While the non-separable nonlinear optimization prob- k2K g ij =enC. ij ij ij j j
lem is in general hard, CTB problem that belongs to a and the x-update of ADMM for the primal problem (14) with
non-separable convex continuous problem is solvable in Dave and the corresponding dual problem in Theorem 4 is
polynomial-time. In particular, based on the Ellipsoid
method [36], a solution approximating the optimal objec- Find: xj 8j2J
tive value to the convex continuous problem is obtain-
e  P ðmÞ ðmÞ
able in polynomial-time, provided that the gradient of Minimize nCr
2 j2J ð
xj b

j þ gj Þ2 (28)
the objective functions are available and that the value of eCxj < mj  j
Subject to n 8 j 2 J:
the optimal solution is bounded in a certain interval [37].
In other words, any information about the behavior of Eq. (28) has jJj single-variable problems and can be inde-
the objective at the optimum can always be translated to pendently implemented in parallel for each link j, greatly
a level of accuracy of the solution vector itself. The inter- decreasing the computation complexity.
est of solving such optimization problem is thus in terms For b-update, the iteration of bðmþ1Þ is
of the accuracy of the solution rather than the accuracy of
the optimal objective value. u
t P P
P e P bk þj
Pi2V k2K ij
Theorem 3 motivates our following work that approxi- j2J m ð
eP k2K bij þj Þ
k
j
mates the objective value of the balancing problem via a arg min P i2V
P e s i þ j2J j
polynomial-time fast algorithm. dkij bkij ¼s i zik Iik 8 i2Ve;k2K i2V (29)
j2J
rXXX k kðmþ1Þ kðmÞ
6.2 Polynomial-Time Approximation Algorithm þ ðbij  xij  g ij Þ2 ;
2
i2Ve
j2J k2K
To exploit a fast and possible parallel solving approach
for CTB problem with immense variables, we adopt kðmþ1Þ kðmÞ ðmþ1Þ ðmÞ
ADMM [10] for the proposed optimization problem with and bkij ¼ xij þ g ij þ b j  x
j  gj . To rewrite
the following two steps. First, we formulate the dual  j , 8j 2 J as usual, we deal with the con-
Eq. (29) in terms of b
problem from the given primal problem in Eq. (14). Then, straint functions by matrix operation and parameter rear-
we alternatively solve both problems for the optimal solu- rangement as follows. Given the intermediate variable
P P  the following equations hold
tion. Note that while Eq. (14) contains two possible delay ^j ¼ k k
b e k2K dij bij =e
i2V
nC,
objectives, in the following we focus on the derivations
for the case of average delay Dave . Similar procedures can 8P P
< k2K dij bij ¼ n
k k
eCb
^j
be done with maximum delay Dmax ; however, we omit i2Ve
P P (30)
the derivations and only include the results to simplify : k2K bij ¼ n
k
eCb
j
the readability. i2Ve

 yk ; zik ), the dual  j ¼ g1 b

j j þ gj bj , where gj
Theorem 4. Based on the MCP results, i.e., (C; Rearranging Eq. (30), we obtain b ^ 2 1
2
problem of Eq. (14) with Dave is given as and gj is from

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 LIN ET AL.: TOWARDS OPTIMAL NETWORK PLANNING FOR SOFTWARE-DEFINED NETWORKS 2961

With the above accomplishments, we propose PTAA in

ð g1j g2j Þ ¼ ð ½1 1 ½1 1 ½1 1 Þ Algorithm 2 via the update rules of ADMM to solve CTB
!y problem for multi-controller SDNs.
  
½d11j dC1j  ½d12j dC2j  ½d1 dC  (31)
 e
nj e
nj
½1 1 ½1 1 ½1 1 6.3 Dynamic Replanning via Adaptive Control
As network size and traffic flow dramatically changes, the
and ð Þy denotes the pseudo-inverse of matrix. The b-update controller placement, switch-controller association, and con-
of ADMM is then obtained by trol traffic forwarding paths may need to be replanned. In
this section, we assume that the network operator has
Find: j
b 8j2J already deployed servers at particular locations throughout
1 X eCb
n  j þ j the network. The controllers are running on these servers,
Minimize P P which can be in either active mode or inactive mode. A con-
e s i þ j2J j nCb
mj  ðe  þ j Þ
j
i2V j2J troller is active if at least one OF-switch is assigned to it and
n  X
eCr (32) the controller is inactive otherwise. Towards this end, we
þ   xðmþ1Þ  g ðmÞ Þ2
ðb
2 j2J j j j aim to propose an adaptive feedback-control mechanism
P P that adaptively activate the controllers at the optimal loca-
X 1  g2j s i zik Iik
Subject to  j ¼ i2Ve k2K
b :
tions and select the optimal forwarding paths for control
j2J
g 1
j eC
n traffic according to time-varying traffic volume.

Instead of having multiple single-variable problems, (32) is 6.3.1 Statistical Delay Guarantee
a jJj-variables problem due to the coupled constraint func- Because of the randomness features of control and data
tion among b  j , 8j 2 J. However, such a constraint function flows, it is practical to provide statistical guarantees (i.e., the
is simply a linear combination of jJj variables and can be probability that the packet violates its delay  constraint is
easily solved. Finally, the iteration of dual-update of ADMM bounded) in QoS control over SDNs, i.e., Pr W ðtÞ  W B 
is obtained t. W ðtÞ is the queueing delay, W B is the requisite bound,
kðmþ1Þ kðmÞ kðmþ1Þ kðmþ1Þ
and t characterizes the degree of guarantees. The violation
g ij :¼ g ij þ xij  bij probability can be upper-bounded as PrfW ðtÞ  W B g 
ðmþ1Þ ðmÞ ðmþ1Þ ðmþ1Þ
(33) fðW ðtÞ; W B Þ. We aim to formulate the function f with
) g j ¼ gj þ xj b

j 8 j 2 J:
respect to the average delay Dave and maximum delay Dmax
On the other hand, regarding the maximum average delay from previous sections, and obtain the network throughput
Dmax , the x-update of ADMM is given as of control messages within the statistical delay guarantees.
Find: xj 8 j 2 J More specifically, this system throughput characterizes the
 P
 2 allowable control traffic
 from switches,
 which satisfies the
Minimize e
nCr
j  bðmÞ þ g ðmÞ delay constraint: Pr W ðtÞ  W B  fðW ðtÞ; W B Þ  t.
2 j2J x j j
(  (34) First, regarding the average transmission delay Dave , we
eC xj < mj  j 8 j 2 J
n formulate the upper-bounded function f by Markov
Subject to ;
1
t 8j2J inequality [35]. In particular
mj ðenCxj þj Þ

the b-update is provided as follows:   EðW ðtÞÞ Dave

Pr W ðtÞ  W B  ¼ B  t ave ; (36)
Find: j 8 j 2 J
b WB W
 P
 2
where t ave is the predefined probability threshold. Moreover,
Minimize tþenCr
2 j2J
j  xðmþ1Þ  g ðmÞ
b j j
P P (35) by the network model in Section 3, the achievable P through-
P 1gj 2
e k2K s i 
zik Iik put under QoS guarantee ðW B ; t ave Þ is yielded as i2V s i .
j2J g1 bj ¼
Subject to  i2V
: In regard to the maximum average delay Dmax , let Wj ðtÞ
j e
nC
denote the transmission delay of link j 2 J and Wmax ðtÞ
The dual-update is the same as Eq. (33). denote the maximum link delay (i.e., Wmax ðtÞ ¼
maxj2J Wj ðtÞ). Then, this maximum delay can be upper-
Algorithm 2. Polynomial-Time Approximation Algorithm bounded as
(PTAA) for CTB    
Pr Wmax ðtÞ  Wmax
B
¼ 1  Pr W1 ðtÞ  Wmax B
Input : CTB problem in Eq. (14) with Dave or Dmax .  
Output: xkij ; 8 i 2 Ve; j 2 J; k 2 K   Pr WjJj ðtÞ  Wmax B

ð0Þ  
1: Set xj ¼ 0; b  ð0Þ ¼ 0; g ð0Þ ¼ 0; 8 j 2 J EðW1 ðtÞÞ EðWjJj ðtÞÞ
j j
1 1 1  (37)
2: for m ¼ 0; 1; . . . do WmaxB B
Wmax
ðmþ1Þ
3: Compute xj ; 8 j 2 J according to Eq. (28) for Dave or
jJj
Dmax jJjDmax
according to Eq. (34) for Dmax 1 1 B  B
 t max ;
4: Compute b  ðmþ1Þ ; 8 j 2 J according to Eq. (32) for Dave or Wmax Wmax
j
according to Eq. (35) for Dmax
ðmþ1Þ where t max is the predefined probability threshold. The first
5: Compute g j ; 8 j 2 J according to Eq. (33) equality comes from the independence assumption among
; 8 i 2 Ve; j 2 J; k 2 K
kðmþ1Þ ðmþ1Þ
6: Set xij from xj Wj ; 8j 2 J; the second inequality follows P the Markov
7: end inequality. Now, the achievable throughput i2V s i satis-
fies the QoS guarantee ðWmaxB
; t max Þ.

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 2962 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 17, NO. 12, DECEMBER 2018

6.3.2 Feedback-Control Loop

Now, we develop a feedback-control loop for dynamic net-
work replanning. Informally speaking, the idea is first to
derive the optimal multi-controller placement (i.e., activate
the controllers at proper servers), then to assign switches to
proper controllers, next to optimize the traffic scheduling
upon this switch-controller assignment, and finally to active
the feedback-control loop if the scheduling result doesn’t
meet the QoS requirements. More specifically, the required
QoS guarantee from upper applications gives three specific
parameters: the delay metric (i.e., the average packet latency
or the maximum link transmission latency), the correspond-
ing requisite delay bound W B and the degree of guarantee
t. Setting these parameters and the network planning
framework as the input, Algorithm 3 iteratively exploits
MCP solution in Algorithm 1 and CTB solution in Algo- Fig. 2. Linear-fast convergence of Algorithm 2: PTAA with rate Oð1=cm Þ.
rithm 2, until QoS guarantee is fulfilled. In particular,
multi-controller SDNs, the convergence rate of Algorithm 2
NSQG function in line 2 enables the feedback control that
to the optimal solutions is Oð1=cm Þ, where c > 1 is a con-
gives the true value if the statistical QoS guarantee is not
stant and m is the iteration number. Moreover, we also pro-
satisfied in the current round. In that case, the algorithm
vide numerical results to validate the theoretical proof. In
enables the successive round and stops whenever it reaches
particular, to demonstrate the fast convergence of Algo-
the optimal solutions. Therefore, the fast-convergent fea-
rithm 2, Fig. 2 shows that the rapid convergence rate of
tures of proposed algorithms for both MCP problem (i.e., at
PTAA follows our theoretical finding Oð1=cm Þ under opti-
most two rounds in average) and CTB problem (i.e.,
mal MCP in Table 2, where y-axis is our targeted Dave and
Oð1=cm Þ) allow us to fast reconfigure the entire system, spe-
the penalty parameter r in PTAA is set to 103 after an
cifically Algorithm 3, according to time-varying traffic sta-
empirical examination. The results imply that PTAA pro-
tistics and network topology.
vides the satisfactory values after 150 iterations, which serve
as a desired stopping point. Hence, we have proposed a lin-
Algorithm 3. Adaptive Feedback Control ear-fast convergent Algorithm 2 for CTB problem with mul-
Input : QoS Guarantee ðW B ; tÞ and Traffic-Driven Network tiple controllers.
Planning in Eq. (12) Finally, Algorithm 3 consists of a while loop that iterates
P
Output: i2V s i ; ðyk ; zik Þ; xk
ij ; 8i 2 V; j 2 J; k 2 K
Algorithms 1 and 2. It implies that the time complexity of
1: Set s i ¼ 1; 8i 2 V ; D ¼ 1 % Initialization Algorithm 3 is OðsðlogðnÞ þ 2 þ 1=cm ÞÞ ¼ OðlogðnÞ þ 1=cm Þ.
2: while NSQG D; W B ; t do In short, yielding optimal controller placement and traffic bal-
3: kfs i gk1 kfs i gk1  1 ancing, the above results confirm that the proposed random-
4:  yk ; zik Þ
ðC; Algorithm 1ðfs i g; Eq. ð13ÞÞ % MCP ized rounding provides suitable MCP in a timely manner, the
5: ðD; xkij Þ Algorithm 2ðyk ; zik ; Eq. ð14ÞÞ % CTB proposed PTAA significantly reduces delay via a fast and
6: end parallel computation approach, and the proposed feedback-
P   
i2V s i ¼ kfs i gk1 ; fyk ; zik ; xij g f
yk ; zik ; xkij g
7: k control loop inherits low complexity. Thus, these designs are
favored in practical large-scale SDN implementation.

7 PERFORMANCE EVALUATION
6.4 Complexity Analysis
We provide the performance analysis for the proposed algo- We evaluate the proposed solving algorithms, including
rithms in this paper. Specifically, we give detailed complex- Algorithm 1 for the MCP sub-problem, Algorithm 2 for the
ity analysis for Algorithms 1, 2, and 3 and confirm their CTB sub-problem, and Algorithm 3 for the network replan-
low-complexity designs. ning problem in both Internet2 OS3E network [11] and Sprint
First, from Theorem 2, we have proved that the proposed GIP backbone network [12]. As our objective is to minimize
Algorithm 1 yields a feasible solution with the probability of the control traffic latency (e.g., either average or maximum
at least 1=2. Thus, the expected number of repetitions for a delay), we focus on evaluating average delay Dave , while
feasible solution will be one over the above probability, maximum delay Dmax can be examined in a similar way.
which gives at most two repetitions in average. This reflects Also, we average over 30 samples for each evaluation point.
the algorithm complexity in terms of pseudo codes. Specifi-
cally, for Algorithm 1, the while loop in lines 3-6 is straight- 7.1 Performance for MCP and CTB
forward value assignments for variables zik ; 8i 2 V; k 2 V , In the following, we first evaluate Algorithm 1 for MCP in
and the expected repetition of this while loop (i.e., lines 7-9) Internet2 OS3E and Sprint GIP backbone. On top of that, we
is at most two as proved. Thus, we have successfully pro- compare Algorithm 2 with several control traffic forwarding
posed a fast-convergent approximation algorithm for the schemes, including (i) shortest-path routing [20], (ii) the
NP-Complete MCP problem. multi-path forwarding, and (iii) the benchmark. Specifically,
Second, regarding Algorithm 2, the convergence analysis (i) shortest-path solution [20] adopts hop-counts as routing
can be conducted by following the similar arguments in our metric and employs the shortest path strategy to guide con-
previous study [13] for single-controller SDNs. In particular, trol traffic from a switch to the controller. While adopting
due to the strictly convex framework of CTB problem for hop-count metric as well, (ii) the multi-path solution equally

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 LIN ET AL.: TOWARDS OPTIMAL NETWORK PLANNING FOR SOFTWARE-DEFINED NETWORKS 2963

TABLE 2
Optimal MCP in 22 Roachdale and 13 Fort Worth; Two Switch Groups with Controller Serving Capability
mkC ¼ s k þ 200  rand [pkts/ms], 8k 2 K  V (Actual Values Are Listed)

City mkC Group City mkC Group City mkC Group

1 Seattle,WA 43.7 1 14 Dallas,TX 170.53 1 27 Pittsburgh,PA 67.25 2
2 Tacoma,WA 80.7 1 15 Houston,TX 165.97 2 28 Fairfax,SC 101.18 1
3 Oroville,CA 132.12 2 16 Omaha,NE 135.86 1 29 Orlando,FL 53.1 2
4 Rancho Cordova,CA 163.05 2 17 Saint Paul,MM 82.72 2 30 Miami,FL 175.86 2
5 Stockton,CA 23.24 2 18 Kansas City,MO 169.32 1 31 Ashburn,VA 45.95 1
6 San Jose,CA 192.88 2 19 Lee’s Summit,MO 113.57 1 32 Harrison,NJ 52.18 1
7 Los Angeles,CA 162.14 2 20 ST. Louis,MO 77.15 2 33 Relay,MD 41.14 1
8 Anaheim,CA 104.36 2 21 Chicago,IL 194.8 1 34 Washington,DC 52.53 1
9 Rialto,CA 94.17 1 22 Roachdale,IN 182.19 1* 35 New York City,NY 94.14 1
10 Phoenix,AZ 96.36 1 23 Nashville,TN 117.03 1 36 Pennsauken,NJ 69.22 1
11 Cheyenne,WY 68.27 1 24 Atlanta,GA 131.5 1 37 Springfield,MA 191.68 1
12 Denver,CO 108.7 1 25 Detroit,MI 124.41 1 38 Boston,MA 93.04 1
13 Fort Worth,TX 109.15 2* 26 Akron,OH 48.55 1 *: Controller location

splits control traffic loads among all available next-hops of a to existing data traffic. With increasing data traffic, PTAA
switch and applies the shortest path strategy to guide the does not incur link overflow and outperforms shortest- and
corresponding multiple routes. (iii) The benchmark solution multi-path solutions with at least 73 percent delay reduction.
is implemented to solve the CTB problem in Eq. (14) Alternatively, given fixed data traffic rates 600 [pkts/ms] and
through the brute-force exhaustive search.

7.1.1 Internet2 OS3E

Internet2 OS3E [11] is launched to support research and net-
working in the United States. Being the next-generation inno-
vation platform, it has been selected as a representative setup
for evaluating controller placement in SDNs by many stud-
ies [13], [20], [22], [26], [27]. As shown in Fig. 3, this common
platform has 27 nodes and 36 links and is adopted for our
MCP and CTB evaluations. In particular, by first applying
Algorithm 1 with given controller serving capability, Fig. 3
shows that two controllers in Houston and Atlanta are
selected with two switch groups. These results come from the
joint consideration of traffic and topological attributes when
solving MCP problem. On the other hand, in [20], hop-count
is the only concerned attribute in single-controller placement,
and Chicago is selected as the optimal location in regard to
average hop number. However, this work does not consider Fig. 4. Average delay in Internet2 OS3E under optimal MCP in Fig. 3
traffic statistics and the obtained location will no longer be with respect to existing data traffic. Link serving rate mj ¼ 600 [pkts/ms],
optimal regarding time-varying control and data traffic. 8j 2 J and control traffic arrival s i ¼ 10 [pkts/ms], 8i 2 V .
Moreover, according to the acquired controller placement,
we execute Algorithm 2 for CTB in Internet2 OS3E. Specifi-
cally, given control traffic rates 10 [pkts/ms] from switches,
Fig. 4 shows the average delay for CTB solutions with respect

Fig. 5. Average delay in Internet2 OS3E under optimal MCP in Fig. 3

Fig. 3. Optimal MCP in Houston and Atlanta; two switch groups (i.e., in with respect to control traffic arrivals. Link serving rate mj ¼ 800 [pkts/
red and blue) with given controller serving capability as listed. ms] and existing data traffic rate j ¼ 600 [pkts/ms], 8j 2 J.

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 2964 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 17, NO. 12, DECEMBER 2018

Fig. 6. Sprint GIP network topology of North America with 38 nodes and
66 links [12].

increasing control traffic, Fig. 5 shows that PTAA signifi-

cantly reduces the delay and supports the performance close Fig. 7. Average delay in Sprint GIP backbone under optimal MCP in
to the benchmark. Thus, the capability of balancing control Table 2 with respect to control traffic arrivals. Link serving rate
traffic via the proposed PTAA has been validated. mj ¼ 1000 [pkts/ms], 8j 2 J.

7.1.2 Sprint GIP Backbone MCP with less controller serving capability, in which three
In addition to examining the proposed algorithms in a controllers in Roachdale, Lee’s Summit, and Rialto are
research-intensive setup in Section 7.1.1, we also evaluate selected with their respective switch groups. Similarly, these
our algorithms in a practical network scenario: Sprint GIP controllers are favored to serve as traffic hubs because of
backbone network [12]. Specifically, Sprint Corporation pro- their great serving capabilities and their central locations
vides the real backbone network topology and the actual link with many direct links to switches.
delay of data traffic. Such delay information is utilized to esti- Based on the obtained controller placement via MCP, we
mate the corresponding data traffic arrival and serving rates. further examine the transmission delay of control traffic
As shown in Fig. 6, the GIP network topology of North Amer- under the proposed CTB solution in Algorithm 2. Apart
ica with 38 nodes and 66 links is adopted for our evaluations. from comparing with shortest-path scheme [20] and multi-
Given control traffic rate s i from switches, Tables 2 and 3 path forwarding as before, two delay bounds relevant to
show the optimal multi-controller placement in Sprint GIP today’s networks [38] are considered. Specifically, (i) ring
network with respect to controller serving capabilities, in protection, 50 [ms], concerns the target restoration time of a
which rand denotes an uniformly distributed random vari- ring topology (e.g., SONET ring). It covers the time from
able between 0 and 1. Note that to enable localized control fault detection to when flowing traffic in the opposite direc-
domain assignments, i.e., Iik in Eq. (3), it is assumed that tion along the ring. (ii) Shared-mesh restoration, around 200
each controller can regulate its three-hop switch neighbors. [ms], serves as the point at which voice calls start to drop,
With better controller-serving capability, Table 2 shows that or ATM circuit rerouting may be triggered.
two controllers in Roachdale and Fort Worth are selected Given link serving rate 1000 [pkts/ms], Fig. 7 shows the
with their respective switch groups. Controllers with greater average delay of the proposed PTAA and of several possible
capabilities are not selected in optimal MCP, as not only con- solutions in the log scale. Specifically, with increasing control
troller computation capabilities but their topological attrib- traffic from switches, PTAA always has lower delay, close to
utes are concerned with the minimum controller the benchmark delay, than other solutions. While shortest-
requirement. Moreover, Table 3 further shows the optimal or multi-path scheme brings a dramatic delay increase from

TABLE 3
Optimal MCP in 22 Roachdale, 19 Lee’s Summit, and 9 Rialto; Three Switch Groups with Controller Serving
Capability mkC ¼ s k þ 150  rand [pkts/ms], 8k 2 K  V (Actual Values Are Listed)

City mkC Group City mkC Group City mkC Group

1 Seattle,WA 116.3 1 14 Dallas,TX 147.87 2 27 Pittsburgh,PA 33.57 2
2 Tacoma,WA 117.68 1 15 Houston,TX 52.2 2 28 Fairfax,SC 106.42 1
3 Oroville,CA 16.51 3 16 Omaha,NE 51.33 3 29 Orlando,FL 56.62 3
4 Rancho Cordova,CA 136.07 3 17 Saint Paul,MM 56.94 2 30 Miami,FL 141.77 3
5 Stockton,CA 147.16 3 18 Kansas City,MO 77.06 2 31 Ashburn,VA 24.72 1
6 San Jose,CA 154.66 3 19 Lee’s Summit,MO 104.23 2* 32 Harrison,NJ 155.26 1
7 Los Angeles,CA 135.84 3 20 ST. Louis,MO 10.78 1 33 Relay,MD 88 1
8 Anaheim,CA 124.83 2 21 Chicago,IL 133.33 2 34 Washington,DC 113.04 3
9 Rialto,CA 84.01 3* 22 Roachdale,IN 90.85 1* 35 New York City,NY 156.92 1
10 Phoenix,AZ 33.64 2 23 Nashville,TN 135.12 2 36 Pennsauken,NJ 50.18 1
11 Cheyenne,WY 66.79 2 24 Atlanta,GA 59.18 2 37 Springfield,MA 69.18 1
12 Denver,CO 27.09 2 25 Detroit,MI 73.9 1 38 Boston,MA 76.73 1
13 Fort Worth,TX 11.63 2 26 Akron,OH 15.14 3 *: Controller location

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 LIN ET AL.: TOWARDS OPTIMAL NETWORK PLANNING FOR SOFTWARE-DEFINED NETWORKS 2965

Fig. 9. Statistical QoS guaranteed control flow throughput with respect to

Fig. 8. Average delay in Sprint GIP backbone under optimal MCP in the degree of QoS guarantee t.
Table 2 with respect to control traffic arrivals. Link serving rate
mj ¼ 1200 [pkts/ms], 8j 2 J.

link overflow, our solution can tolerate such high loads by

distributing control traffic over links with lighter data loads.
Moreover, with better link-serving capability (i.e., serving
rate 1200 [pkts/ms]), Fig. 8 shows that the proposed PTAA
greatly surpasses other solutions with the delay close to the
benchmark performance. In this case, our solution greatly
reduces the delay, particularly for large arrivals, while short-
est- or multi-path routing induces link overflow even for lim-
ited control traffic volume. Also, similar to Fig. 7, Fig. 8
indicates that PTAA leads to the delay performance lower
than the latency of ring protection, and retains the slightly
increased delay for increasing traffic. In short, all of the
above observations suggest that by employing information
of traffic statistics, our solution brings better controller place-
ment and link resource utilizations and outperforms existing
schemes with at least 73 percent delay reduction.
Fig. 10. Statistical QoS guaranteed control flow throughput with respect
to control traffic arrivals.
7.2 Dynamic SDN Replanning
To evaluate the effectiveness of dynamic replanning with as compared with configuration in Table 2. The results in
regard to feedback-control strategy and QoS provisioning, Table 4 show that the replanning of controller placement is
we examine statistical delay-guaranteed throughput, from executed to cope with this increased control traffic, where two
executing Algorithm 3 in Sprint GIP network. The required additional controllers are respectively activated at the servers
delay bound is set as W B ¼ 100 [ms], the delay degree assign- in Tacoma and Chicago. Specifically, newly activated control-
ment t ¼ 0:16, and the same controller serving capability in lers, particularly the one in Chicago with great serving capa-
Table 2. First, we double the control traffic rate of each switch bility, considerably share and alleviate heavy traffic loads

TABLE 4
Dynamic Replanning of Optimal MCP in 22 Roachdale, 13 Fort Worth, 2 Tacoma, and 21 Chicago; Four Switch Groups
Under Same Controller Capability in Table 2 but Doubled Control Traffic Rates from Switches

City mkC Group City mkC Group City mkC Group

1 Seattle,WA 43.7 3 14 Dallas,TX 170.53 1 27 Pittsburgh,PA 67.25 3
2 Tacoma,WA 80.7 3* 15 Houston,TX 165.97 1 28 Fairfax,SC 101.18 4
3 Oroville,CA 132.12 1 16 Omaha,NE 135.86 4 29 Orlando,FL 53.1 2
4 Rancho Cordova,CA 163.05 4 17 Saint Paul,MM 82.72 3 30 Miami,FL 175.86 4
5 Stockton,CA 23.24 4 18 Kansas City,MO 169.32 2 31 Ashburn,VA 45.95 2
6 San Jose,CA 192.88 2 19 Lee’s Summit,MO 113.57 1 32 Harrison,NJ 52.18 3
7 Los Angeles,CA 162.14 4 20 ST. Louis,MO 77.15 3 33 Relay,MD 41.14 2
8 Anaheim,CA 104.36 2 21 Chicago,IL 194.8 4* 34 Washington,DC 52.53 1
9 Rialto,CA 94.17 4 22 Roachdale,IN 182.19 1* 35 New York City,NY 94.14 4
10 Phoenix,AZ 96.36 4 23 Nashville,TN 117.03 2 36 Pennsauken,NJ 69.22 1
11 Cheyenne,WY 68.27 1 24 Atlanta,GA 131.5 4 37 Springfield,MA 191.68 1
12 Denver,CO 108.7 1 25 Detroit,MI 124.41 1 38 Boston,MA 93.04 4
13 Fort Worth,TX 109.15 2* 26 Akron,OH 48.55 1 *: Controller location

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 2966 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 17, NO. 12, DECEMBER 2018

from two original controllers in Table 2 (i.e., in Roachdale and [8] ONF, “OpenFlow switch specification.” (2013). [Online]. Avail-
able: https://www.opennetworking.org/images/stories/
Fort Worth). The above observations validate the capability of downloads/sdn-resources/onf-specifications/openflow/
dynamic replanning subject to the possibly time-varying traf- openflow-spec-v1.4.0.pdf
fic volume and network topology. [9] D. S. Hochbaum, “Complexity and algorithms for nonlinear opti-
We also investigate delay-guaranteed throughput with mization problems,” Ann. OR, vol. 153, no. 1, pp. 257–296, 2007.
respect to the varying delay-guarantee degree and link serv- [10] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein,
“Distributed optimization and statistical learning via the alternat-
ing capabilities. Fig. 9 shows that, as expected, both loose ing direction method of multipliers,” Found. Trends Mach. Learn.,
delay guarantee degree t and higher serving capability bring vol. 3, no. 1, pp. 1–122, 2011.
higher delay-guaranteed throughput. Moreover, with given [11] Internet2, “Open science, scholarship and services exchange.” (2014).
t ¼ 0:16, Fig. 10 shows achievable throughput with respect [Online]. Available: http://www.internet2.edu/network/ose
[12] Sprint, Overland Park, KS, “Sprint IP network performance,”
to control traffic arrivals. As traffic arrivals increase, higher 2011. [Online]. Available: http://www.sprint.net/performance
serving capability per controller leads to higher control traffic [13] S.-C. Lin, P. Wang, and M. Luo, “Control traffic balancing in soft-
throughput. Hence, all of the above evaluations suggest that ware defined networks,” Comput. Netw., vol. 106, pp. 260–271,
our solution can instantly react to different network dynam- Sep. 2016.
ics, including time-varying QoS requirements, network [14] S. H. Yeganeh, A. Tootoonchian, and Y. Ganjali, “On scalability of
software-defined networking,” IEEE Commun. Mag., vol. 51, no. 2,
topologies and serving capabilities, and traffic statistics. pp. 136–141, Feb. 2013.
[15] A. Mallik and S. Hegde, “A novel proposal to effectively combine
multipath data forwarding for data center networks with conges-
8 CONCLUSION tion control and load balancing using software-defined network-
ing approach,” in Proc. Int. Conf. Recent Trends Inf. Technol., Apr.
This study addressed the traffic-driven SDN planning with 2014, pp. 1–7.
in-band control traffic as a nonlinear multi-objective (mixed [16] D. Adami, S. Giordano, M. Pagano, and N. Santinelli, “Class-
integer and continuous) optimization problem. The com- based traffic recovery with load balancing in software-defined
plex optimization was solved in a timely manner by parti- networks,” in Proc. IEEE GLOBECOM Workshops, Dec. 2014,
pp. 161–165.
tioning the original problem into two sub-problems (the
[17] D. Tuncer, M. Charalambides, S. Clayman, and G. Pavlou,
placement and traffic balancing) and iteratively solving the “Adaptive resource management and control in software defined
sub-problems through the proposed adaptive feedback con- networks,” IEEE Trans. Netw. Service Manage., vol. 12, no. 1,
trol. Performance evaluations confirmed that our control pp. 18–33, Mar. 2015.
scheme, based on the minimum number of required control- [18] S.-C. Lin, P. Wang, and M. Luo, “Jointly optimized QoS-aware vir-
tualization and routing in software defined networks,” Comput.
lers, outperformed existing solutions with at least 73 percent Netw., vol. 96, pp. 69–78, Feb. 2016.
delay reduction via a fast and low complexity approach. [19] S.-C. Lin, I. F. Akyildiz, P. Wang, and M. Luo, “QoS-aware adap-
Therefore, a novel paradigm was successfully presented, tive routing in multi-layer hierarchical software defined networks:
facilitating the on-line configurations of multi-controller A reinforcement learning approach,” in Proc. IEEE Int. Conf. Serv-
SDNs. In our future work, the hierarchical controller archi- ices Comput., Jun. 2016, pp. 25–33.
[20] B. Heller, R. Sherwood, and N. McKeown, “The controller place-
tecture and the control traffic delivery with regard to inter- ment problem,” in Proc. ACM Workshop Hot Topics Softw. Defined
controller communication will be investigated. Netw., Aug. 2012, pp. 7–12.
[21] M. F. Bari, A. R. Roy, S. R. Chowdhury, Q. Zhang, M. F. Zhani,
R. Ahmed, and R. Boutaba, “Dynamic controller provisioning in
ACKNOWLEDGMENTS software defined networks,” in Proc. 9th Int. Conf. Netw. Serv. Man-
This work was supported by the US National Science Foun- age., Oct. 2013, pp. 18–25.
[22] Y. Hu, W. Wang, X. Gong, X. Que, and S. Cheng, “On reliability-
dation (NSF) under Grant No. 1547353. optimized controller placement for software-defined networks,”
China Commun., vol. 11, no. 2, pp. 38–54, Feb. 2014.
REFERENCES [23] G. Yao, J. Bi, Y. Li, and L. Guo, “On the capacitated controller
placement problem in software defined networks,” IEEE Commun.
[1] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, Lett., vol. 18, no. 8, pp. 1339–1342, Aug. 2014.
L. Peterson, J. Rexford, S. Shenker, and J. Turner, “OpenFlow: [24] H. K. Rath, V. Revoori, S. Nadaf, and A. Simha, “Optimal control-
Enabling innovation in campus networks,” ACM SIGCOMM Com- ler placement in software defined networks (SDN) using a non-
put. Commun. Rev., vol. 38, no. 2, pp. 69–74, Apr. 2008. zero-sum game,” in Proc. IEEE Int. Symp. World Wireless Mobile
[2] I. F. Akyildiz, A. Lee, P. Wang, M. Luo, and W. Chou, “A road- Multimedia Netw., Jun. 2014, pp. 1–6.
map for traffic engineering in SDN-OpenFlow networks,” Comput. [25] A. Sallahi and M. S. -Hilaire, “Optimal model for the controller
Netw., vol. 71, pp. 1–30, Oct. 2014. placement problem in software defined networks,” IEEE Commun.
[3] D. Kreutz, F. M. V. Ramos, P. E. Verssimo, C. E. Rothenberg, Lett., vol. 19, no. 1, pp. 30–33, Jan. 2015.
S. Azodolmolky, and S. Uhlig, “Software-defined networking: [26] S. Lange, S. Gebert, T. Zinner, P. T. -Gia, D. Hock, M. Jarschel, and
A comprehensive survey,” IEEE Proc., vol. 103, no. 1, pp. 14–76, M. Hoffmann, “Heuristic approaches to the controller placement
Jan. 2015. problem in large scale SDN networks,” IEEE Trans. Netw. Serv.
[4] S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh, Manage., vol. 12, no. 1, pp. 4–17, Mar. 2015.
S. Venkata, J. Wanderer, J. Zhou, M. Zhu, J. Zolla, U. Holzle, [27] N. Beheshti and Y. Zhang, “Fast failover for control traffic in soft-
S. Stuart, and A. Vahdat, “B4: Experience with a globally- ware-defined networks,” in Proc. IEEE Global Commun. Conf., Dec.
deployed software defined WAN,” in Proc. ACM SIGCOMM, 2012, pp. 2665–2670.
Aug. 2013, pp. 3–14. [28] M. Jarschel, S. Oechsner, D. Schlosser, R. Pries, S. Goll, and P. Tran-
[5] C. Y. Hong, S. Kandula, R. Mahajan, M. Zhang, V. Gill, M. Nanduri, Gia, “Modeling and performance evaluation of an OpenFlow
and R. Wattenhofer, “Achieving high utilization with software- architecture,” in Proc. 23rd Int. Teletraffic Congr., 2011, pp. 1–7.
driven WAN,” in Proc. ACM SIGCOMM, 2013, pp. 15–26. [29] Z. Shang and K. Wolter, “Delay evaluation of OpenFlow network
[6] I. F. Akyildiz, P. Wang, and S.-C. Lin, “SoftAir: A software defined based on queueing model,” in CoRR, vol. abs/1608.06491, 2016.
networking architecture for 5G wireless systems,” Comput. Netw., [Online]. Available: http://arxiv.org/abs/1608.06491
vol. 85, pp. 1–18, Jul. 2015. [30] Y. Goto, H. Masuyama, B. Ng, W. K. G. Seah, and Y. Takahashi,
[7] I. F. Akyildiz, S.-C. Lin, and P. Wang, “Wireless software-defined “Queueing analysis of software defined network with realistic
networks (W-SDNs) and network function virtualization (NFV) OpenFlow-based switch model,” in Proc. IEEE 24th Int. Symp. Model.
for 5G cellular systems: An overview and qualitative evaluation,” Anal. Simul. Comput. Telecommun. Syst., Sep. 2016, pp. 301–306.
Comput. Netw., vol. 93, pp. 66–79, Dec. 2015.

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric
 LIN ET AL.: TOWARDS OPTIMAL NETWORK PLANNING FOR SOFTWARE-DEFINED NETWORKS 2967

[31] S. Karlin and H. M. Taylor, A First Course in Stochastic Processes. Ian F. Akyildiz (M’86-SM’89-F’96) received the
New York, NY, USA: Academic Press, 1975. BS, MS, and PhD degrees in computer engineer-
[32] D. Gross, J. F. Shortle, and C. M. Harris, Fundamentals of Queueing ing from the University of Erlangen-Nurnberg,
Theory, 4th ed. Hoboken, NJ, USA: Wiley, 2008. Germany, in 1978, 1981, and 1984, respectively.
[33] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, Since 2008, he has been an honorary professor
U.K.: Cambridge Univ. Press, 2004. with the School of Electrical Engineering, Univer-
[34] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduc- sitat Politcnica de Catalunya, Barcelona, Spain,
tion to Algorithms, 2nd ed. Cambridge, MA, USA: MIT Press, 2001. and the founding director of the NaNoNetworking
[35] A. Gut, Probability: A Graduate Course. Berlin, Germany: Springer- Center, Catalunya, Spain. Since 2011, he has
Verlag, 2005. been a consulting chair professor with the
[36] S. A. Vavasis, Nonlinear Optimization: Complexity Issues. New York, Department of Information Technology, King
NY, USA: Oxford Univ. Press, 1991. Abdulaziz University, Jeddah, Saudi Arabia. Since 2013, he has also
[37] A. S. Nemirovsky and D. B. Yudin, Problem Complexity and Method been a Finland distinguished professor program professor (supported by
Efficiency in Optimization. New York, NY, USA: Wiley, 1983. the Academy of Science) with the Department of Electronics and Com-
[38] ANSI, “Enhanced network survivability performance,” Alliance munications Engineering, Tampere University of Technology, Tampere,
Telecommun. Ind. Solutions, Washington, D.C., USA, Tech. Rep. Finland, and the founding director of the Nano Communications Center.
T1.TR.68–2001, 2001. He is currently the Ken Byers chair professor of telecommunications
with the School of Electrical and Computer Engineering, Georgia Insti-
Shih-Chun Lin (S’08-M’17) received the BS tute of Technology, Atlanta, Georgia, and the director of the Broadband
degree in electrical engineering and the MS degree Wireless Networking Laboratory and the chair of the Telecommunication
in communication engineering from National Tai- Group, Georgia Institute of Technology. His current research interests
wan University, Taipei, Taiwan, in 2008 and 2010, include wireless sensor networks in challenged environments, 5G cellu-
respectively, and the PhD degree in electrical and lar systems, nanonetworks, terahertz band, and software-defined net-
computer engineering from the Georgia Institute of works. He was a recipient of numerous awards from the IEEE and the
Technology, Atlanta, Georgia, in 2017. He is cur- ACM. His h-index is 110 and the total number of citations is above 99K
rently an assistant professor with the Department due to Google Scholar as of March 2018. He is the editor-in-chief of
of Electrical and Computer Engineering, North Car- Computer Networks and the founding editor-in-chief of Ad Hoc Net-
olina State University, Raleigh, North Carolina. His works, Physical Communication, and Nano Communication Networks.
current research interests include software-defined He became an ACM fellow in 1997.
networking, wireless underground sensor networks, 5G cellular networks,
machine-to-machine communications, and statistical scheduling in wire-
less systems. He was a recipient of the BWN Lab Researcher of the Year Min Luo received the BS and MS degrees in
Award from the Georgia Institute of Technology in 2015. He is a member of computer science, in 1982 and 1987, respec-
the IEEE. tively, and the PhD degree in electrical engineer-
ing from the Georgia Institute of Technology,
Atlanta, Georgia, in 1992. Currently, he is the
Pu Wang (M’05) received the BS degree in elec- head and chief architect of the Advanced Net-
trical engineering from the Beijing Institute of working, Huaweis Shannon (IT) Lab, leading the
Technology, Beijing, China, in 2003, the MEng research and development in Software Defined
degree in computer engineering from the Memo- Networking (SDN) and other future networking
rial University of Newfoundland, St. Johns, NL, initiatives. He served as chief/executive architect
Canada, in 2008, and the PhD degree in electrical for IBM SWGs Strategy and Technology, Global
and computer engineering from the Georgia Insti- Business Solution CenterGCG, Industry Solutions, and Center of Excel-
tute of Technology, Atlanta, Georgia, in 2013. He lence for Enterprise Architecture and SOA for more than 11 years. He
is currently an assistant professor with the also worked as a senior operations research analyst, senior manager,
Department of Computer Science, University of and director of Transportation Network Planning and Technologies for
North Carolina at Charlotte, Charlotte, North Car- two Fortune 500 companies for seven years. He is certified and was
olina. He was a fellow of the School of Graduate Studies, Memorial Uni- awarded as the distinguished lead/chief architect from Open Group in
versity of Newfoundland, St. Johns, NL, Canada, in 2008. His current 2008. He is an established expert in the field of next negation software
research interests include wireless sensor networks, cognitive radio net- defined networking (SDN), enterprise architecture and information sys-
works, software-defined networks, nanonetworks, multimedia communi- tems, whole life cycle software application and product development,
cations, wireless communications in challenged environment, and business intelligence, and business process optimization. He is also a
cyber-physical systems. He was a recipient of the BWN Laboratory pioneer and one of the recognized leading experts and educators in Ser-
Researcher of the Year Award in 2012 from the Georgia Institute of vice-oriented architecture (SOA), Model/business-driven architecture
Technology and the TPC Top Ranked Paper Award of IEEE DySPAN and development (MDA-D), and component/object-oriented technolo-
2011. He is a member of the IEEE. gies. He coauthored two books, including the pioneering Patterns: Ser-
vice Oriented Architecture and Web Services in 2004, and published
more than 20 research papers. He has been serving on the organizing
committee for IEEEs ICWS, SCC, and CC (Cloud Computing) conferen-
ces, chaired sessions, presented several tutorials on SOA and Enter-
prise Architecture and their best practices, and gave lectures with the
Service University. He has served as adjunct professors in several USA
and Chinese universities since 1996. He is a senior member of the IEEE.

" For more information on this or any other computing topic,

please visit our Digital Library at www.computer.org/publications/dlib.

icensed use limited to: SVKM¿s NMIMS Mukesh Patel School of Technology Management & Engineering. Downloaded on August 20,2021 at 09:33:16 UTC from IEEE Xplore. Restric