Received 11 December 2019; revised 9 May 2020; accepted 22 May 2020.

Date of publication 26 May 2020;

date of current version 9 June 2020.
Digital Object Identifier 10.1109/OAJPE.2020.2997665

Software-Defined Microgrid Control: The

Genesis of Decoupled Cyber-Physical
Microgrids
LIZHI WANG 1 (Student Member, IEEE), YANYUAN QIN 2 (Student Member, IEEE),
ZEFAN TANG 1 (Student Member, IEEE),
AND PENG ZHANG 1 (Senior Member, IEEE)
1 Department of Electrical and Computer Engineering, Stony Brook University, Stony Brook, NY 11794 USA
2 Department of Computer Science and Engineering, University of Connecticut, Storrs, CT 06269 USA

(CORRESPONDING AUTHOR: P. ZHANG ([email protected])

This work was supported in part by the National Science Foundation under Grant ECCS-1831811.

ABSTRACT Nowadays, microgrid controllers are often embedded in specialized hardware such as PLC
and DSP. The hardware-dependency and fit-and-forget design make it difficult and costly for microgrid
controllers to evolve and upgrade under frequent changes such as plug-and-play of microgrid components.
Furthermore, different distributed energy resources in a microgrid require customized controllers, leading
to long development cycles and high operational costs for deploying microgrid services. To tackle the
challenges, a software-defined control SDC) architecture for microgrid is devised, which virtualizes tradi-
tionally hardware-dependent microgrid control functions as software services decoupled from the underlying
hardware infrastructure, fully resolving hardware dependence issues and enabling unprecedentedly low costs.
A generic SDC prototype is designed to generate microgrid controllers autonomously in edge computing
facilities such as distributed virtual machines. Extensive experiments verify that SDC outperforms traditional
hardware-based microgrid control in that it empowers a decoupled cyber-physical microgrid and thus makes
microgrid operations unprecedentedly affordable, autonomic, and secure.

INDEX TERMS Microgrids, software-defined control, decoupled cyber-physical microgrids, virtual

machine, plug-and-play.

I. INTRODUCTION microgrid controllers [7], [8]. Thus, any failure in microgrid

M ICROGRID is a paradigm shifting solution which

enhances electricity resiliency and supports the
ever-increasing integration of distributed energy resources
controller can lead to severe impacts on both microgrid and
the main grid. Adding to those challenges is the fact that
there is no universally applicable tool for designing microgrid
(DERs) and energy storage at grid edges [1], [2]. However, control efficiently and optimally, making the designing and
it is oftentimes prohibitively difficult to build and oper- deployment of new hardware controllers daunting tasks.
ate a microgrid mainly due to the hardware-dependence of Over the years, despite a plethora of literature in improving
microgrid protection, automation and control (PAC) where the DER control in microgrids (e.g. droop control, PQ control
the microgrid intelligence resides. Especially, the existing and V/f control [9]), a majority of research efforts [10], [11]
microgrid control is always implemented on specific hard- only focused on the simulation-based performance analysis
ware such as DSP or PLC [3], [4], making it difficult and for control algorithms, assuming a hardware-dependent con-
costly to evolve and update when the microgrid configuration trol architecture. Few of the existing publications address how
changes and thus the controller parameters have to be re- to improve the response of a microgrid to hardware anomalies
tuned [5], [6]. In the state-of-the-practice, the high CAPEX such as failures or being sabotaged. Recently, the software-
and OPEX of hardware controllers multiplying the large defined networking (SDN) is attracting increasing attentions
number of DERs prohibit nearly any redundancy or backup in from the microgrid community as it offers a programmable,

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
VOLUME 7, 2020 173
 FIGURE 1. SDC-enabled microgrid architecture.

flexible and reliable solution to operate the microgrid, sup- frame is designed in the SDC architecture, which auto-
porting diverse quality-of-service (QoS) requirements and matically generates controllers for microgrid, providing
making it much easier to develop new applications and enable high reliability and easily-deployable redundancy.
fast innovation in microgrid [12]. However, although a few • A discrete model of virtual droop control, which incor-
works have been done on developing SDN-enabled microgrid porates double-loop controllers using the trapezoidal
or even networked microgrids [12], [13], the existing litera- rule, is derived as an example for implementing inter-
ture is unfortunately largely silent on the topic of developing faces between DERs and virtualized controllers.
software-defined microgrid controls. Part of the reason for • A testbed is built in an RTDS environment to evaluate
this stems from the fact that achieving the transition – replac- the SDC’s performance. Extensive experimental results
ing hardware-dependent controllers with software-based vir- validate the robustness, plug-and-play capability and
tual ones – is not straightforward. It was unclear how to reliability of the SDC-based system. Results also ver-
guarantee the software-based controllers perform as well as ify that the software-defined controllers perform better
the hardware-dependent ones, and how to provide the redun- when microgrid encounters hardware failures.
dancy for controllers with software-based solutions. The rest of this paper is organized as follows: Section II
In this paper, we present a software-defined control (SDC) presents the SDC-enabled microgrid architecture, the SDC’s
architecture for microgrid. It decouples the hardware infras- work flow and implementation, and the benefits of using SDC
tructure with microgrid control functions. Decoupling soft- for microgrid. The derived discrete model of virtual droop
ware from dedicated hardware enables easier modifications, control is presented in Section III. Section IV provides the
managements and updates. Virtualization allows multiple experimental results, and Section V concludes the paper.
independent users to efficiently use computational and net-
work resources (e.g., processing power and communication II. SDC-ENABLED MICROGRID
bandwidth) by abstracting them into logical units. Specifi- Virtualization provides a promising opportunity to develop
cally, the controllers are implemented in software, and run on novel applications flexibly in microgrid. To implement
white-box hardware platforms via an ultra-fast network [14]. software-based control functions through virtualization on
As an illustration purpose, a primary DER microgrid con- general-purpose servers, a critical challenge is to design a
troller, droop controller, is virtualized and implemented in the framework for SDC with no performance degradation. In this
SDC architecture to demonstrate the system’s performance. section, the devised SDC-enabled microgrid architecture is
Extensive experiments in an RTDS environment verify that first presented, and this is followed by the description of
virtualization of microgrid controllers can get rid of the SDC’s work flow and implementation. The benefits of using
restriction of hardware implementation and at the same time SDC for microgird are also provided.
achieve the goal of hot standby [15] and seamless switching
of controllers at zero cost. The contributions of this paper are A. ARCHITECTURE OF THE SDC-ENABLED MICROGRID
threefold: The high-level design of the SDC-enabled microgrid archi-
• A novel SDC architecture is devised for microgrid, tecture is illustrated in Fig. 1, where the physical and cyber
where the concept and validation of virtualized micro- layers are fully decoupled. The control plane and its corre-
grid controller are addressed. A generalized control sponding functions are pushed to the edge of the microgrid,

174 VOLUME 7, 2020

Wang et al.: Software-Defined Microgrid Control: The Genesis of Decoupled Cyber-Physical Microgrids

TABLE 1. Control configuration of different DERs.

FIGURE 2. Framework of a generalized DER controller.

and the virtual controllers run in local servers. The compo-

nents in the physical plane include various DERs such as
solar panels, wind turbines, diesels and storages, and others
like smart meters, loads, and transformers. Each DER com-
municates with its controller through an IP-address-assigned
interface, which can adopt different communication protocols which means the switches a1 , a3 , a5 , a6 and a9 are turned on.
to transfer DER’s data such as control signals. DERs typically operate as grid-following sources using the
The measurements from the physical plane (e.g., power P/Q control when microgrid works in grid-connected mode.
generation, load, etc.) are transferred to the control plane When microgrid works in islanded mode, some DERs operate
via a communication network, and are used for various tasks as grid-forming sources to actively control their frequency
such as eigenvalue analysis, power flow calculation, transient output, making it possible for them to naturally support
analysis, formal/reachability analysis, and parameters learn- the system frequency while sharing a portion of the load
ing [16]. The results obtained from the analyses are used change [18]. DERs operate as grid-following sources to track
for designing control parameters and updating controllers’ the voltage angle of the microgrid to control their output.
models. On the top of the architecture is the SDC manager. Table 1 illustrates the control configurations of DERs based
It manages different virtual controllers in the control plane, on the features of DERs’ functions, which provides the spe-
such as implementing control functions, managing the han- cific guidance on how to design DER controllers for SDC
dover of controllers, and providing design parameters for users based on the generalized controller. The configuration
controllers. A critical component for the SDC manager is the table is stored in SDC library, which provides SDC manager
SDC library, which consists of parameters from a generalized reference to decide which control (e.g., PQ control, droop
DER controller. control, V/f control, etc.) should be selected from the library.
The framework of a generalized DER controller is given Meanwhile, a backup controller is also established to provide
in Fig. 2 [17]. It consists of frequency, power, and voltage the control redundancy for microgrid, and will be discussed
control modules, and an inner loop module. A polymorphic later.
switch a1 is used to select the active power reference, and To implement new virtualized control methodologies,
its state is initially off. Each controller’s parameters, i.e., the flexible computational resources such as memories and
references that are marked in red in Fig. 2, are tuned when processors have to be virtualized and allocated to the new
the controller is created. The tuning process is based on the controller. This is achieved using technologies such as Hyper-
analysis of the system with the goal of guaranteeing the visor or Container by the SDC manager. Once the new
system’s stability. The combinations of different modules methodology is created, the description on the virtualized
and switches’ states generate different control functions. This controller will be stored in the SDC library, which contains
characteristic ensures the software-defined controllers inherit information including its constitutions and the computational
from the generalized controller easily. resource requirements. Meanwhile, the communication net-
The relationship between the control objective and the work resource information such as the bandwidth and delay
switches’ states can be described as follows: is obtained by the SDC manager via the interface with the
communication network. New communication channels will
c = b · a, (1) be created for new methodologies once the requirements of
where c = {c1 , c2 , . . . , cN } denotes the set of N microgrid computational and communication network resources are sat-
controllers, b = {b1 , b2 , . . . , bN } is the set of binary numbers isfied. It is to be noted that the handshake and communication
that control the states of different switches in the N microgrid of different units are managed by the SDC manager.
controllers, and a = {a1 , a2 , . . . , a10 } is the set of the
10 switches in the generalized control architecture in Fig. 2. B. SDC’s WORK FLOW
For instance, if bit i of bn is one, switch ai will be turned on to SDC provides the flexibility to develop new applications
implement the function of controller cn . In particular, for the such as optimization, distributed control and system parame-
droop controller, the binary number is b1 = (0100110101)2 , ters learning. Based on the demand of microgrid operation,

VOLUME 7, 2020 175

 FIGURE 4. An overview of the implementation for SDC in
microgrid.
FIGURE 3. SDC’s work flow.

different software-defined controllers can be installed for

different DERs. For example, an islanded microgrid needs DERs and controllers. To guarantee the performance of
diesels or batteries to stabilize the system frequency, and DER controls, the allowed time delay of communication
therefore the software-defined droop controllers can be cre- between DERs and controllers should be within tens of
ated to achieve the goal. The procedures of implementing the microseconds [19], which can be achieved using the existing
software-defined controllers are described as follows: widely-used network switches.
• Step 1: Select controllers from the SDC library, and set
the controllers’ corresponding parameters based on each C. IMPLEMENTATION OF SDC IN MICROGRID
DER’s requirements. The implementation of SDC in microgrid is described in
• Step 2: The SDC manager installs the software-defined this subsection. As shown in Fig. 4, the implementation
controllers on general computing platforms, i.e., servers. process consists of three key components. The first one is
As the controllers are installed in software, one or mul- the implementation of SDC manager. The SDC manager has
tiple backup controllers can be installed to provide the the ability of creating new master and backup controllers,
redundancy. All the controllers are installed on servers, managing controller handovers, transferring parameters for
which can be close to or far away from DERs. master and backup controllers, and monitoring DERs for
• Step 3: Run the virtual controllers on servers. Each mas- plug-in and plug-out. The second component is the estab-
ter controller receives its corresponding DER’s states lishment of a generalized DER controller library. This library
and sends out control signals to the DER. Meanwhile, is in charge of creating different DER controllers. A variety
each backup controller runs in the ‘‘hot standby’’ mode, of controllers share some common functions, which can
which improves the system’s modularity and robustness. be inherited from the generalized controller, e.g., functions
• Step 4: A software-defined controller controls its DER like Phase Lock Loop (PLL), coordinate transformation, low
via a data transmission network. For instance, the three pass filtering, and double-loop control. The last component
phases of the voltage or current from each DER can is the virtualization of software-defined controllers, which
be transferred to the corresponding controller. Various can be designed as classes based on the characteristics of
encryption protocols can be used for the data transmis- object-oriented programming, and are inherited from the
sion to ensure its reliability and security. generalized DER controller.
The devised SDC’s work flow is illustrated in Fig. 3. The communications between DERs and virtual con-
The SDC manager is in charge of the events including trollers are managed by the SDC manager. Specifically, when
initialization, scaling, termination, and updating. Once the the SDC manager receives a request for creating a new con-
SDC manager detects that a new controller for a certain troller for a DER, the function of create_new_controller is
DER needs to be created, a new connection between the executed. The DER’s IP address and port are sensed by the
controller and the DER will be initialized. The SDC man- SDC manager, and the IP address and port of the GCU,
ager abstracts the physical resources of each controller, and on which the new created controller is implemented, are
creates the corresponding functions within the virtualized transferred to the DER. Meanwhile, the SDC manager sends
infrastructure. It ensures that the life cycle of each con- commands to the generalized DER controller to request a new
troller is independent of hardware platforms such as the controller class. The controller then enters the listening mode
DERs with standardized interfaces between the controllers after being initialized and connected to the DER. At this state,
and DERs. A software-defined controller can run either on the controller is receiving any data packet whose destination
a physical server, or on a virtual machine (VM). IP and port match the controller, respectively [20]. Once a
Network protocols such as TCP/IP can be used to estab- packet arrives, the controller begins to send corresponding
lish low-latency and loss-tolerating connections between control signals to the DER, e.g., when the SDC manager

176 VOLUME 7, 2020

Wang et al.: Software-Defined Microgrid Control: The Genesis of Decoupled Cyber-Physical Microgrids

receives a request from a battery storage, a software-defined framework for a droop-controlled inverter-based microgrid
droop controller is created from the generalized DER con- are presented in reference [24], where the double-loop con-
troller, and corresponding control signals are generated. The trollers are not included. Moreover, the PI regulator is shown
server then enters the sending mode and starts to send out con- using the z-transformation, which however cannot be applied
trol signals whose destination IP and port are the IP and port directly for software evolutions. In this paper, a droop con-
of the DER, respectively. The handover function is executed troller considering the power calculation, frequency control,
when the SDC manager detects one controller stops working. voltage-current loop control is presented in detail using the
The IP and port of that controller will be transferred to a trapezoidal rule, which can be directly applied with the soft-
backup controller, and the corresponding control parameters ware. Discrete models for other controllers, such as PQ and
will also be set for the backup controller. V/f controls, can be generated in a similar way.
For a droop control, the frequency and voltage magnitudes
D. BENEFITS AND COST OF USING SDC FOR at time k can be obtained as follows:
MICROGRID
f (k) = f ∗ − m(P(k) − P∗ ) = f ∗ − m1P(k), (2)
Virtualized controllers running on the commodity hard-
ware use virtualized computational resources to facilitate and
the implementation of the SDC framework. A virtualized
controller has the ability to produce optimal results with E(k) = E ∗ − n(Q(k) − Q∗ ) = E ∗ − n1Q(k), (3)
minimum resources. Integrating the intervention into exist- where f (k) and E(k) are the frequency and voltage magni-
ing structures greatly requires the replacement of existing tudes at time k, respectively. P(k) and Q(k) are the discrete
controllers with virtualized resources and the establishment samples of the active and reactive powers, respectively. f ∗ and
of connections between virtualized controllers and DERs. E ∗ are the frequency and voltage references, respectively;
Compared to the existing hardware-dependent control archi- and P∗ and Q∗ are the active and reactive power references,
tectures, SDC offers the following benefits for microgrid: respectively. 1P(k) and 1Q(k) are the corresponding active
• The separation of software from hardware infrastruc- and reactive power error inputs for the droop controller,
ture allows the software to evolve independently, which and m and n are frequency and voltage droop coefficients,
provides microgrid with great flexibility such as the respectively.
flexible deployment of control functions for different For a converter-based DER, the droop control is typically
requirements in microgrid. implemented using a double-loop control diagram, where
• The dynamic service provided by the SDC manager the outer control loop is designed to provide current refer-
offers more intensive monitoring for controllers. ences for inner loop, and the inner loop generates modulation
• The generalized DER control framework offers a high waves for SPWM generation [25]. Let vd (k) and vq (k) be the
efficiency and great convenience to develop specific new voltage components in dq coordinate transformed from the
controllers for DERs. three-phase voltage in abc coordinate at time k. Then, Er (k)
• The convenient creation and deployment of software- can be defined as follows:
defined backup controllers provide high reliability for X4 1
microgrid to operate with redundancy. Er (k) = vd (k)(1 + (−1)n 82n (k))
n=1 2n
X4 1
III. VIRTUAL DROOP CONTROLLER +vq (k)(8(k)+ (−1)n 82n+1 (k)), (4)
The communication process between a DER and a virtual
n=1 2n+1
controller can be summarized as follows: 1) the controller where 8(k) is the output of the PLL function at time k, and
establishes communication with the DER using information can be calculated recursively as follows:
provided by the SDC manager; 2) at the DER side, measure-
8(k + 1) = 8(k) + (w(k) + w(k + 1))T /2, (5)
ments such as three phases of the output current and voltage
of DER are sampled and transferred to the controller; and where T is the sampling time, and is set at 0.2 ms in this study.
3) control signals such as those used to generate SPWM are w(k) can be calculated recursively as follows:
sent back to the DER.
w(k + 1) = kp Er (k + 1)
k
A. VIRTUAL DROOP CONTROLLER MODEL X
Discrete control algorithms deployed in the processor are the +ki T ((Er (0)+Er (k + 1))/2+ Er (n)), (6)
n=1
main components to operate DERs’ controllers. In [21]–[23],
different distributed discrete secondary control approaches where kp and ki are the PI parameters.
are proposed to minimize frequency and voltage devia- The phase of the voltage at time k, θ(k), in discrete domain
tions and ensure accurate active and reactive power sharing can be calculated recursively as follows:
for either radial- or mesh-structured microgrids. However,
θ(k + 1) = θ(k) + 2π(f (k) + f (k + 1))T /2, (7)
the primary discrete controllers are not addressed in these
papers. A discrete-time mathematical model and an analytical where f (k) can be obtained from (2).

VOLUME 7, 2020 177

 With Er (k) and θ(k) obtained from (4)-(7), the error of the
input voltage for the outer loop can be calculated as follows:
X4 1

 1vdref = Er (k)(1 + (−1)n θ 2n (k)) − vd (k))


 n=1 2n
X4 1
1v qref = Er (k)(θ(k) + (−1)n
θ 2n+1 (k)) (8)


 n=1 2n + 1
−vq (k).


With vdref and vqref obtained from (11), the current refer-
ences for the inner loop can be calculated as follows:


 idref (k + 1) = kpvd (1vdref (k + 1)) + kivd T (1vd (0)
 Xk
FIGURE 5. The cyber and physical components in the simulation
1vd (n))

+1vd (k + 1)/2 +


n=1 (9) testbed.
 iqref (k + 1) = kpvq (1vqref (k + 1)) + kivq T (1vq (0)

as follows:
 Xk
1vq (n)),

+1vq (k + 1)/2 +


1ẋMG (t) = AMG0 1xMG (t) + BMG0 1yMG (t)

n=1 

S
where kpvd , kivd , kpvq and kivq are PI parameters, and


 X
(A 1x (t − τ )

1idref (k) = idref (k) − id (k). Then, the voltage modulation

MGi MG i
(11)
i=1
references vdm and vqm are obtained as follows: 
+BMGi 1yMG (t − τi ))





0 = CMG0 1xMG (t) + DMG0 1yMG (t)
 

 vdm (k + 1) = vd (k + 1) + kpid (1idref (k + 1))
+kiid T (1id (0) + 1id (k + 1)/2

where τi = τmi + τei (i = 1, . . . , m) refers to the time



 Xk
1id (n)) + iq (k + 1)wL delay constants, and 0 < τ1 < . . . < τ1 , τmax . In the

+


n=1 (10) SDC-enabled architecture, the measurement and execution
 vqm (k + 1) = vq (k + 1) + kpiq (1iqref (k + 1))

 delays are trivial compared with the switching and communi-
+kiiq T (1iq (0) + 1iq (k + 1)/2




 Xk cation delays caused by master and backup controllers. The
1iq (n)) − id (k + 1)wL,


+ delay’s impact will be further discussed in Section IV.
n=1

where kpid , kiid , kpiq and kiiq are PI parameters. id (k) and iq (k) IV. TEST AND VALIDATION OF SDC
are the current components in dq coordinate transformed A. EXPERIMENTAL SETUP
from the three-phase current in abc coordinate at time k. w To test the SDC’s performance, a real-time simulation testbed
is the frequency reference, i.e., 120π, and L is the inductance is built in an RTDS environment. As illustrated in Fig. 5,
of the filter in the droop controller. this testbed consists of the RTDS hardware, its auxiliary
A variety of functions in a droop control such as inner loop facilities such as the GTNET cards, and the servers. A console
and outer loop, can be used for other software-defined con- PC is used to develop and compile the microgrid model in
trols. Similar methodologies can be applied for the derivation the RSCAD, a power system simulation software designed
of discrete models for other software-defined controls such to interact with the RTDS hardware. The GTNET cards
as PQ and V/f controls. can be used to transmit data between RTDS and external
equipment through a LAN/WAN using protocols such as the
B. TIME DELAY FACTORS IN VIRTUAL DROOP GTNET-SKT (Socket) Protocol. The RTDS and servers are
CONTROLLER connected through switches and a campus network. Mean-
In the SDC architecture, the flow of the information consists while, each server has a specific port, which is linked to
of three steps: 1) measurement, 2) communication among the campus network and can be used for the console PC to
controllers, and 3) execution. In the measurement process, access the server. In this study, the simulator in RDTS has
transmitting the incremental frequency in the f/P droop con- 16 cores for physical-layer simulation running in real time.
trol from a DER to a virtualized controller can introduce a It provides Gigabit Ethernet ports for all IP-based commu-
time-delay τm in the feedback path. The execution process nications including data transmissions between RTDS and
uses an interface to send control signals from a virtualized external servers.
controller to a DER. The control delay τe , which can affect A typical microgrid shown in Fig. 6 is used to test and
the stability of the distributed system, should be considered. validate the SDC’s performance. This test system contains
The impact of the control algorithms’ iterations in the SDC two diesel units, one photovoltaic (PV) unit, one wind tur-
can be neglected due to the large computing capacity of the bine, and one battery. The microgrid can operate in islanded
SDC. The delay in the closed-loop τ , which is the sum of τm or grid-connected mode, depending on whether the circuit
and τe , forms the total delay of the SDC-enabled microgrid. breaker S1 is open or closed. The detailed parameters of the
The small signal model [26] of the time-delayed microgrid is microgrid can be found in Appendix.

178 VOLUME 7, 2020

Wang et al.: Software-Defined Microgrid Control: The Genesis of Decoupled Cyber-Physical Microgrids

FIGURE 7. Comparison of the two diesels’ active powers.

software-defined droop controller performs as well as a

FIGURE 6. A typical microgrid used to test and validate the
traditional hardware-dependent one; there is almost no
SDC’s performance in this paper [29]. performance degradation.

In the simulation, the sampling time step is set at 2 µs C. IMPACT OF AN SDC-ENABLED DER ON MICROGRID
for converter-based components and 50 µs for other compo- In this test case, we demonstrate how the microgrid reacts
nents in microgird. The sampling rate of transmission data when an SDC-enabled DER is plugged into the system.
is 600 packets per second. A software-defined virtual droop Initially, the microgrid operates in islanded mode with the
control is designed for the battery in Fig. 6, whose parameters battery disconnected. At time t = 1.2 s, the battery is plugged
are given in table 2 in Appendix V. The battery is connected into the system. Meanwhile, the SDC manager receives a
to Bus 2 through a 0.48/13.2 kV step-up transformer. As the handshake request, and immediately generates and installs
current constraint is commonly set at 1.1-1.3 times of the a virtual droop controller on the remote server. A new con-
rated current [27], [28], a constraint of 1.25 p.u. is set for the nection is built between the battery and the virtual droop
converter’s current in the test cases. controller. The three phases of voltage and current of the
battery’s output are sampled and transferred to the virtual
B. COMPARISON WITH TRADITIONAL DROOP droop controller on the server through the campus network.
CONTROLLER At time t = 6.8 s, the active power of Load 6 suddenly
The performances of the SDC and a traditional droop con- increases from 1 MW to 5 MW.
troller are compared in this subsection. In this test case, The current and voltage responses of the battery before and
the two diesels share the same droop factor and capacity. after the battery is plugged in the microgrid are illustrated
Diesel 1 is controlled by a software-defined droop controller in Fig. 8. It can be seen that:
running on a remote server, and Diesel 2 is controlled by a • The virtual droop controller immediately starts to con-
traditional droop controller, which runs in the RTDS. trol the battery when the battery is plugged in the micro-
The operation of the microgrid works as follows: Initially, grid at time t = 1.2 s. The current of the battery is
the microgrid is connected to the main grid, and the bat- stabilized in a very short time.
tery and capacitor banks are disconnected. At time t = • The voltage of the battery has a little fluctuation but
4 s, the switch S1 is open, making the microgrid operate becomes stable within only around 0.05 s after the bat-
in islanded mode. Meanwhile, the two diesels immediately tery is connected to the microgrid.
start to regulate their output powers for balancing the power • When Load 6 increases at time t = 6.8 s, the voltage
generation with the load consumption. At time t = 11.5 s, magnitude of the battery maintains constant, while the
the active power of Load 6 (see Fig. 6) suddenly increases current magnitude of the battery increases. It indicates
from 1 MW to 5 MW. that the SDC-enabled battery has started to participate
The comparison results of the two diesels’ active powers in the power sharing within the microgrid.
are given in Fig. 7. It can be observed that:
• From time t = 4 s to around t = 7 s, the two diesels have D. IMPACT OF SWITCHING BETWEEN MASTER AND
almost the same dynamic performances. The maximum BACKUP SOFTWARE-DEFINED CONTROLLERS RUNNING
difference between the active powers of the two diesels ON THE SAME SERVER
is only around 0.05 MW. A backup controller can be implemented in microgrid to pro-
• At time t = 11.5 s, the two diesels increase their vide redundancy. In this subsection, the impact of the switch-
active power to an almost same degree during the reg- ing between a master controller and a backup controller
ulation of the active power output. It verifies that the running on the same server is evaluated. Specifically, at time

VOLUME 7, 2020 179

FIGURE 8. Current and voltage responses of the battery before FIGURE 10. Current and voltage responses of the battery with
and after the battery is plugged in the microgrid. (a) Current master and backup virtual controllers running on the same
response of the battery. (b) Voltage response of the battery. server before and after the master controller fails. (a) Current
response of the battery. (b) Voltage response of the battery.

transferred to the backup controller, making the battery be

controlled by the backup controller. At time t = 3.8 s,
the active power of Load 6 is adjusted from 1 MW to 5 MW.
From Fig. 10, it can be seen that there are no overcurrent or
fluctuation issues during the controllers handover when the
controllers are running on the same server.

E. IMPACT OF SWITCHING BETWEEN MASTER AND

BACKUP SOFTWARE-DEFINED CONTROLLERS RUNNING
ON DIFFERENT SERVERS
The impact of the switching between master and backup
FIGURE 9. Current response of the battery with the traditional
hardware-dependent controller before and after the controller software-defined controllers running on different servers
fails. is evaluated in this subsection. Specifically, the master
software-defined droop controller runs on server 1, and the
t = 2 s, the master controller fails, meaning it cannot pro- backup software-defined droop controller runs on server 2.
cess data and achieve the droop function. The battery’s cur- At time t = 2 s, the master controller on server 1 fails. The
rent response using a traditional hardware-dependent droop states of the master controller are transferred to the backup
controller with no backup controller is illustrated in Fig. 9. controller on server 2. The current and voltage responses of
It can be seen that when the traditional droop controller the battery before and after the master controller fails are
fails, no control signals are sent back to the battery because illustrated in Fig. 11.
of the lack of redundancy. As there is no feedback control It can be observed that the current magnitude becomes
signal, no modulation wave can be used to generate the PWM slightly higher, i.e., increasing from 0.5 kA to around 0.8 kA,
signals. The insulated-gate bipolar transistors (IGBTs) in the during the controllers handover. This is mainly caused by
converter are isolated due to the overcurrent at time t = 2 s, the communication latency between two servers. However,
and the converter stops working. the increased current is still within the limit of constraints,
The current and voltage responses of the battery with promising the normal operation of the system during the
the master and backup virtual droop controllers are given handover between controllers on different servers.
in Fig. 10. At time t = 2 s, the master controller fails and
the backup controller immediately starts to work. Meanwhile, F. IMPACT OF DELAY
the new connection between the battery and the backup con- For an SDC-enabled system, a critical issue is the impact of
troller is established. The states of the master controller are the communication latency on a software-defined controller’s

180 VOLUME 7, 2020

Wang et al.: Software-Defined Microgrid Control: The Genesis of Decoupled Cyber-Physical Microgrids

TABLE 2. Power loads (KVA) at different buses in Fig. 6.

TABLE 3. DERs’ generation(MW/MVA) and control parameters.

FIGURE 11. Current and voltage responses of the battery with

V. CONCLUSION
master and backup virtual controllers running on different
servers before and after the master controller fails. (a) Current In this paper, a software-defined control architecture is
response of the battery. (b) Voltage response of the battery. devised for microgrid. Compared with the existing microgrid
control systems that heavily rely on hardware infrastructure
which is inconvenient and costly to evolve and upgrade,
the presented SDC architecture provides robustness and plug-
and-play capability for DER controllers. It is also capable
of supporting a variety of applications such as secondary
and tertiary controls, and other advanced control schemes in
microgrid. The experimental results in an RTDS environment
validate that SDC is a universal and plug-and-play platform
for microgrid, where new DER controllers can be easily and
flexibly defined and deployed, and high redundancy can also
be provided to deal with DER controllers’ failures.

FIGURE 12. System’s performance with different time delays. APPENDIX

Table 2 gives the three-phase power loads at different buses.
performance. Delays caused by measurement, execution, and Loads on buses 1-4 are modeled as switched RL passive
algorithm iteration can have different impacts on the system’s loads while loads on buses 5-7 are modeled as non-switchable
performance. In this paper, it is found that the measurement dynamic loads representing the critical loads in the system.
and execution cause very small delays due to the high rate The DERs’ generation and control schemes are summarized
of date sampling and large communication capability of the in Table 3. The diesels and the battery use droop control
test bed. Fig. 12 illustrates the system’s performance with to achieve frequency and power sharing when the microgrid
different time delays caused by the controllers’ handover in operates in islanded mode, while the PV and wind turbine use
different servers, where the current constraint of the battery the PQ control. For more details on the microgrid, readers are
is denoted as Imn , and the maximum current magnitude of the referred to [29].
battery during the controllers handover with a certain time
delay is denoted as Imd . REFERENCES
It can be seen that the nominal delay caused by the con- [1] R. R. Kolluri, I. Mareels, T. Alpcan, M. Brazil, J. de Hoog, and
trollers’ handover in different servers is tolerated, which D. A. Thomas, ‘‘Power sharing in angle droop controlled microgrids,’’
IEEE Trans. Power Syst., vol. 32, no. 6, pp. 4743–4751, Nov. 2017.
indicates that the system is still stable. When different time
[2] Y. Mohamed and E. F. El-Saadany, ‘‘Adaptive decentralized droop con-
delays are introduced, the current of the battery is below the troller to preserve power sharing stability of paralleled inverters in dis-
constraint of the converter’s current when the delay is less tributed generation microgrids,’’ IEEE Trans. Power Electron., vol. 23,
than 40 ms. Note that the communication delay with existing no. 6, pp. 2806–2816, Nov. 2008.
[3] K. De Brabandere, K. Vanthournout, J. Driesen, G. Deconinck, and
Ethernet switches is only several milliseconds, which is much R. Belmans, ‘‘Control of microgrids,’’ in Proc. IEEE Power Eng. Soc. Gen.
smaller than 40 ms. Meeting, vol. 7, Jun. 2007, pp. 1–7.

VOLUME 7, 2020 181

 [4] P. Danzi, M. Angjelichinoski, C. Stefanovic, T. Dragicevic, and [28] A. A. Bani-Ahmed, ‘‘Design and implementation of a true decentralized
P. Popovski, ‘‘Software-defined microgrid control for resilience against autonomous control architecture for microgrids,’’ Ph.D. dissertation, Dept.
denial-of-service attacks,’’ IEEE Trans. Smart Grid, vol. 10, no. 5, Elect. Comput. Eng., Univ. Wisconsin-Milwaukee, Milwaukee, WI, USA,
pp. 5258–5268, Sep. 2019. 2017.
[5] T. Dragičević, X. Lu, J. C. Vasquez, and J. M. Guerrero, ‘‘DC microgrids— [29] O. Nzimako, ‘‘Real-time simulation of a microgrid system with distributed
Part I: A review of control strategies and stabilization techniques,’’ IEEE energy resources,’’ M.S. thesis, Dept. Elect. Comput. Eng., Univ. Mani-
Trans. Power Electron., vol. 31, no. 7, pp. 4876–4891, Jul. 2016. toba, Winnipeg, MB, Canada, 2015.
[6] M. N. D. P. D. Bhuyar and A. Jadhav, ‘‘Review on IoT based smart solar
photovoltaic plant remote monitoring and control unit,’’ Int. J., vol. 3, no. 3,
pp. 1–5, 2018. LIZHI WANG (Student Member, IEEE) received
[7] P. Basak, S. Chowdhury, S. H. N. Dey, and S. P. Chowdhury, ‘‘A literature the B.S. and M.S. degrees in electrical engi-
review on integration of distributed energy resources in the perspective
neering from Zhejiang University, Hangzhou,
of control, protection and stability of microgrid,’’ Renew. Sustain. Energy
China, in 2015 and 2018, respectively. He is
Rev., vol. 16, no. 8, pp. 5545–5556, Oct. 2012.
[8] Y. Li, D. M. Vilathgamuwa, and P. C. Loh, ‘‘Design, analysis, and real-time currently pursuing the Ph.D. degree in electrical
testing of a controller for multibus microgrid system,’’ IEEE Trans. Power engineering with Stony Brook University, Stony
Electron., vol. 19, no. 5, pp. 1195–1204, Sep. 2004. Brook, NY, USA. His current research interests
[9] P. Piagi, ‘‘Microgrid control,’’ Ph.D. dissertation, Dept. Elect. Comput. include stability analysis and distributed control
Eng., Univ. Wisconsin–Madison, Madison, WI, USA, 2005. of microgrid, cyber physical security for electric
[10] W. Liu, W. Gu, Y. Xu, Y. Wang, and K. Zhang, ‘‘General distributed sec- power networks, and quantum computation and its
ondary control for multi-microgrids with both PQ-controlled and droop- application in power grid.
controlled distributed generators,’’ IET Gener., Transmiss. Distrib., vol. 11,
no. 3, pp. 707–718, Feb. 2017.
[11] S. Adhikari and F. Li, ‘‘Coordinated v-f and p-q control of solar photo- YANYUAN QIN (Student Member, IEEE)
voltaic generators with MPPT and battery storage in microgrids,’’ IEEE received the B.S. degree in automation from the
Trans. Smart Grid, vol. 5, no. 3, pp. 1270–1281, May 2014. Nanjing University of Aeronautics and Astronau-
[12] L. Ren, Y. Qin, B. Wang, P. Zhang, P. B. Luh, and R. Jin, ‘‘Enabling
tics, China, in 2011, and the M.S. degree in con-
resilient microgrid through programmable network,’’ IEEE Trans. Smart
trol science and engineering from Shanghai Jiao
Grid, vol. 8, no. 6, pp. 2826–2836, Nov. 2017.
[13] L. Ren et al., ‘‘Enabling resilient distributed power sharing in networked Tong University, China, in 2014. He is currently
microgrids through software defined networking,’’ Appl. Energy, vol. 210, pursuing the Ph.D. degree with the Computer
pp. 1251–1265, Jan. 2018. Science and Engineering Department, University
[14] R. Guerzoni, ‘‘Network functions virtualisation: An introduction, benefits, of Connecticut.
enablers, challenges and call for action, introductory white paper,’’ in Proc. His research interests are in software-defined
SDN OpenFlow World Congr., vol. 1, 2012, pp. 5–7. networking and wireless networks.
[15] S. Batra and G. Taneja, ‘‘Reliability modeling and optimization of the
number of hot standby units in a system working with two operative units,’’
Compusoft, vol. 8, no. 2, pp. 3059–3068, 2019. ZEFAN TANG (Student Member, IEEE) received
[16] J. Hwang, K. K. Ramakrishnan, and T. Wood, ‘‘NetVM: High performance the B.S. degree in mechanical engineering
and flexible networking using virtualization on commodity platforms,’’ from Zhejiang University, Zhejiang, China,
IEEE Trans. Netw. Service Manage., vol. 12, no. 1, pp. 34–47, Mar. 2015. in 2014, and the M.S. degree in electrical and
[17] C. Wang, Analysis Simulations Theory Microgrids. Beijing, China: Science
computer engineering from the University of
Press, 2013.
[18] D. Pattabiraman, R. H. Lasseter., and T. M. Jahns, ‘‘Comparison of grid Michigan–Shanghai Jiao Tong University Joint
following and grid forming control for a high inverter penetration power Institute, Shanghai Jiao Tong University, Shang-
system,’’ in Proc. IEEE Power Energy Soc. Gen. Meeting (PESGM), hai, China, in 2017. He is currently pursuing the
Aug. 2018, pp. 1–5. Ph.D. degree in electrical engineering with Stony
[19] C. Macana, A. Abdou, H. Pota, J. Guerrero, and J. Vasquez, ‘‘Cyber Brook University, Stony Brook, NY, USA.
physical energy systems modules for power sharing controllers in inverter His current research interests include distributed renewable energy
based microgrids,’’ Inventions, vol. 3, no. 3, p. 66, Sep. 2018. systems, cyber physical security for electric power networks, quantum
[20] V. Venkataramanan, A. Srivastava, and A. Hahn, ‘‘Real-time co-simulation security, microgrid, power system resilience, and cyberattack-resilient load
testbed for microgrid cyber-physical analysis,’’ in Proc. Workshop Model- forecasting.
ing Simulation Cyber-Phys. Energy Syst. (MSCPES), Apr. 2016, pp. 1–6.
[21] X. Lu, X. Yu, J. Lai, Y. Wang, and J. M. Guerrero, ‘‘A novel distributed
secondary coordination control approach for islanded microgrids,’’ IEEE PENG ZHANG (Senior Member, IEEE) received
Trans. Smart Grid, vol. 9, no. 4, pp. 2726–2740, Jul. 2018. the Ph.D. degree in electrical engineering from the
[22] W. Gu, G. Lou, W. Tan, and X. Yuan, ‘‘A nonlinear state estimator-based
University of British Columbia, Vancouver, BC,
decentralized secondary voltage control scheme for autonomous micro-
grids,’’ IEEE Trans. Power Syst., vol. 32, no. 6, pp. 4794–4804, Nov. 2017. Canada, in 2009.
[23] J. Schiffer, T. Seel, J. Raisch, and T. Sezi, ‘‘Voltage stability and reactive He was a Francis L. Castleman Associate
power sharing in inverter-based microgrids with consensus-based dis- Professor and a Centennial Associate Professor
tributed voltage control,’’ IEEE Trans. Control Syst. Technol., vol. 24, no. 1, with the University of Connecticut, Storrs, CT,
pp. 96–109, Jan. 2016. USA, from 2017 to 2019. He was a System Plan-
[24] M. B. Delghavi and A. Yazdani, ‘‘An adaptive feedforward compensation ning Engineer at BC Hydro and Power Authority,
for stability enhancement in droop-controlled inverter-based microgrids,’’ Vancouver, from 2006 to 2010. He is currently a
IEEE Trans. Power Del., vol. 26, no. 3, pp. 1764–1773, Jul. 2011. SUNY Empire Innovation Professor with Stony Brook University, Stony
[25] A. Yazdani and R. Iravani, Voltage-Sourced Converters in Power Systems, Brook, NY, USA. His research interests include networked microgrids,
vol. 34. Hoboken, NJ, USA: Wiley, 2010.
programmable microgrids, cyber resilience, formal methods and reachability
[26] H. Ye, Y. Liu, and P. Zhang, ‘‘Efficient eigen-analysis for large delayed
cyber-physical power system using explicit infinitesimal generator dis-
analysis, and quantum engineering.
cretization,’’ IEEE Trans. Power Syst., vol. 31, no. 3, pp. 2361–2370, Dr. Zhang is an individual member of CIGRÉ. He is an Editor of the IEEE
May 2016. TRANSACTIONS ON POWER SYSTEMS, the IEEE TRANSACTIONS ON SUSTAINABLE
[27] M. Abdolkarimzadeh, M. Nazari-Heris, M. Abapour, and M. Sabahi, ENERGY, and the IEEE POWER AND ENERGY SOCIETY LETTERS. He is also an
‘‘A bridge-type fault current limiter for energy management of AC/DC Associate Editor of the IEEE JOURNAL OF OCEANIC ENGINEERING and the IEEE
microgrids,’’ IEEE Trans. Power Electron., vol. 32, no. 12, pp. 9043–9050, TRANSACTIONS ON INDUSTRIAL ELECTRONICS.
Dec. 2017.

182 VOLUME 7, 2020