Real-time Co-simulation Platform using OPAL-RT

and OPNET for Analyzing Smart Grid Performance

D. Bian, Student, IEEE, M. Kuzlu, Senior Member, IEEE, Y. Wu, Student, IEEE
M. Pipattanasomporn, Senior Member, IEEE, and S. Rahman, Fellow, IEEE Industrial Information and Control
Advanced Research Institute System Department
Virginia Tech KTH Royal institute of technology
Arlington, VA, USA 22203 Stockholm, Sweden 10044

Abstract— The objective of this paper is to present a real-time properties, such as Quality of Service (QoS), latency, etc.
co-simulation platform for analyzing electric power grid This will allow realistic studies of smart grid operation
operation, taking into account integrated communication integrated with communication systems.
systems. This paper discusses how a co-simulation platform can
be set up using OPNET and OPAL-RT as communication and
There are several co-simulation platforms presented in the
power system simulators, respectively. A simplified distribution literature, including VPNET, OpenDSS-NS2, ADEVS-NS2
automation case study is demonstrated to show how the co- and NMLab. The VPNET platform – the combination of
simulation platform can be used to analyze performance of the VTB (Virtual Test Bed) and OPNET – is used to analyze the
smart grid operation in real-time. influence of data blocking on the output voltage of a boost
converter in [5]. A co-simulation platform based on
Keywords— real-time, co-simulation, Java Eclipse, OPNET, RT-
LAB, OPAL-RT, System-in-the-Loop, smart grid applications. OpenDSS and NS2 network simulator is presented in [6], and
this platform has been used to examine the effect of
communication failures on smart grid operation. NS2
I. INTRODUCTION
cooperated with ADEVS (A Discrete Event System
A smart grid is regarded as a complex network where simulator) is also used to simulate the communication delay
electric power systems and communication systems are on power system monitoring and control in [7]. In [8], a co-
interdependent [1]. Relying on communication technologies, simulation framework called NMLab, which is a combination
many smart grid applications can be implemented in electric of MATLAB and NS2, is introduced. This paper introduces a
power grids, such as demand response, distribution real-time co-simulation platform using OPNET and OPAL-
automation, etc. Since communication systems have intrinsic RT. This platform is unique in that it allows simultaneous
characteristics, such as latency, data error and package lost, simulations of power system and communication networks in
this will affect the reliability and security of smart grid real-time.
operation. In [2, 3, 4], benefits and potential impacts that Section II summarizes popular simulators for both
communication networks have introduced to a smart grid are communication and power system analyses. In Section III, the
discussed. Because smart grid operation relies on classification of co-simulation platforms is discussed, and then
communication technologies, analyzing its performance the process to set up a real-time co-simulation platform based
needs to consider operation of both power and on OPNET and OPAL-RT is described. In Section IV, a
communication systems. distribution automation test case based on a remote control of
PowerWorld, MATLAB/Simulink, GridLAB-D, RT-LAB a switch is simulated on this real-time co-simulation platform.
are popular power system simulation tools. OPNET, NS2/ Simulation results are then discussed.
NS3 and OMNeT++ are popular communication and network
simulators. Nonetheless, neither simulation tools for power II. SIMULATION TOOLS FOR POWER AND
system analysis consider the impact from communication COMMUNICATION SYSTEMS
systems, nor can communication system simulators provide In this section, widely used power and communication
power system analysis. By combining both power system and system simulators are summarized, including PowerWorld,
communication network simulators, a co-simulation platform OpenDSS, SimuLink, RTDS and OPAL-RT for power
can be developed that enables simulations of electric power system analysis, and NS2/NS3, MONeT++ and OPNET for
systems that take into account communication network communication system simulation.

978-1-4673-8040-9/15/$31.00 ©2015 IEEE

 2

A. Simulation tools for Power System Analysis OPNET Modeler is a commercial discrete event network
PowerWorld from PowerWorld Corp. is an interactive simulator which focuses on network performance simulation.
power system simulation package designed to simulate high OPNET provides toolsets, such as: node models that specify
voltage power system operation. This software contains interface of a network component; process models that
power flow analysis package capable of efficiently solving abstract the behavior of a network component; a project
power system models [9]. window that defines network topology and link connections;
OpenDSS, an open-source simulator from EPRI, is a and a simulation window that captures and displays
comprehensive electrical power system simulation tool for simulation results. OPNET Modeler has a high-level user
analyzing power distribution systems. OpenDSS supports interface offering access to a large library of blocks, which
nearly all frequency domain analyses commonly performed represents different models and functions. In addition,
on electric utility power distribution systems. Additionally, it OPNET Modeler provides System-in-the-Loop (SITL) that
supports many new types of application analyses, which are allows physical hardware devices and a simulator to interact
designed to meet future needs related to smart grid, as a unified system. SITL can also allow multiple physical
renewable energy research, etc. [10]. networks to communicate through a simulation [14].
Simulink, developed by MathWorks, is data-flow C. Choosing Simulators for Real-Time Co-Simulation
graphical programming software for modeling, simulating In this paper, OPNET and OPAL-RT are chosen to
and analyzing dynamic systems. It supports simulation, develop a real-time co-simulation platform. OPNET is chosen
automatic code generation, continuous test and verification of because it is well designed and widely used commercial
embedded systems. Integrated with MATLAB, it can software that provides more standard packages than other
incorporate MATLAB algorithms into models and export communication simulators discussed above. It also offers a
simulation results to MATLAB for further analysis [11]. user-friendly interface. For power system real-time
Several kinds of real-time hardware-based simulation simulation, OPAL-RT is chosen because RT-LAB, coming
platforms are available, i.e., Real-Time Digital Simulator with OPAL-RT, has a number of ready-to-use models and it
(RTDS) and OPAL-RT. These platforms enable model can be integrated with MATLAB/Simulink. Having access to
execution at the same rate as actual time. a number of standard packages and ready-to-use models will
RTDS is a real-time power system simulator. High-speed facilitate simulations of a number of smart grid applications.
simulation, closed-loop testing of protection and control
equipment and hardware-in-the-loop applications are III. CO-SIMULATION PLATFORM
supported by RTDS [12]. In this section, the classification of co-simulation platform
OPAL-RT can be used to design, test and optimize control is discussed. Components of the real-time co-simulation
and protection systems for power grids, power electronics, platform are presented, together with discussion on the co-
etc. In addition, the RT-LAB, core OPAL-RT software, simulation platform set up.
enables users to develop models suitable for real-time A. Co-Simulation Platform Classification
simulation. RT-LAB models are fully integrated with
The co-simulation platform can be classified into non real-
MATLAB/Simulink [13].
time (off-line) and real-time, as discussed below.
B. Communication System Simulators 1) Offline Co-Simulation (Non Real-time)
Network Simulator versions 2 and 3 (NS2/NS3) are open- In a non real-time co-simulation, two simulators are
source discrete event network simulators, targeting generally runs separately. In order to realize accurate
communication and network research. They support information exchange between power and communication
simulations of IP-based applications (including TCP, UPD, systems, it is necessary to synchronize the simulation time
etc.), routing and multicast protocols over all kinds of step, while the operation of two simulators does not need to
networks (both wired and wireless). NS2 core is written in be synchronized. For a non real-time platform, a
C++ programming language. Therefore, C++ is needed to communication simulation runs and simulation results are
create new network models or protocols. NS3, the successor saved into a file, which is shared with the power system
of NS2, includes some new features, such as support for the simulator. This file is used when the power system simulator
Python programming language, improved scalability, and etc. needs information about communication simulation results.
[14]. Easy setting up is the advantage of this non real-time co-
OMNeT++, as a discrete event simulation environment, is simulation platform. However, the accuracy of simulation
an open-source software. Since OMNeT++ is component- results can be limited due to asynchronous operation of two
based and provides a generic and flexible architecture, it is simulators.
widely used to simulate complex information technology 2) Real-Time Co-Simulation
systems, queuing networks, and etc. Communication network A real-time co-simulation platform needs an interface that
simulation is one of the most popular applications of can enable the information exchange between two simulators.
OMNeT++ [15]. The synchronization of two simulators relies on this interface,
 3

which serves as a data buffer that allows real-time packet that has RT-LAB installed are used. The power system model
exchanges using protocols, like TCP or UDP. In a real-time is built using readily available blocks from RT-LAB library.
co-simulation platform, two simulators run synchronously. Once the simulated model is compiled successfully in RT-
As an advantage, a real-time co-simulation platform can LAB, the power system model can be loaded into the real-
handle large and complex systems without manual time simulator, i.e., OPAL-RT.
intervention. However, the initial set up is time-consuming 3) Communication System Simulator
and multiple PCs along with the real-time simulator are A PC with OPNET installed is used to simulate the
necessary to set up a real-time co-simulation platform. communication system. The communication system model is
configured to enable information exchanging between the
B. Components of a Real-Time Co-Simulation Platform
control center and the power system model. Simulation
The following components, as shown in Figure 1, are results can provide communication latency, data dropping,
necessary to establish a real-time co-simulation platform for etc.
simulating smart grid operation. This paper discusses this 4) Connections among the Control Center Model, and the
real-time co-simulation platform in the context of a Electric Power and Communication System Simulators
distribution automation application. The connection between the power system simulator (i.e.,
OPAL-RT) and the communication system simulator (i.e., the
PC that has OPNET installed) is set up using an Ethernet
interface. Similarly, the connection between the OPNET PC
and the control center (i.e., the PC that has Java Eclipse
installed) is also set up using an Ethernet interface.
SITL, a model provide by OPNET is used to enable
exchange of packets between OPNET and OPAL-RT/Java
Eclipse. As shown in Figure 1, the PC with OPNET installed
requires two sets of a SITL model and a network card. Each
set, i.e., one SITL model and one network card, functions as
the data exchange interface. With this interface, the data
Figure 1. Connection among OPAL-RT, OPNET and Java Eclipse. exchanging between Java Eclipse and OPNET can be
realized. Similarly, OPAL-RT and OPNET are connected
1) Control Center Model through another set of data exchange interface.
A distribution automation control center is simulated using
a PC that has Java Eclipse installed. This control center is C. Network Configuration and Operation of the Real-time
responsible for monitoring voltage levels and controlling Co-simulation Platform
selected devices in a distribution system. Based on the The detailed IP and Subnet Mast set up of the real-time co-
voltage information monitored, the control center makes a simulation platform are illustrated in Figure 2. The operation
decision and sends control commands to selected devices. sequence of this platform can be divided into two steps.
2) Electric Power System Simulator In the first step, building and compiling the power system
To set up an electric power system, OPAL-RT and a PC model in the PC with RT-LAB installed; and loading the

Figure 2. Real-time Co-simulation Platform with detailed IP and Subnet Mask

 4

compiled power system model into OPAL-RT. In this It is assumed that the transmission medium between the
experiment, the connection between OPAL-RT and RT-LAB control center and the switch for capacitor bank control is a
is established using Wi-Fi interface. See Fig. 2. fiber-optic cable. This is typical for any distribution
The second step is to run control center, OPNET and automation application. This connection is modeled as a 100-
OPAL-RT at the same time to co-simulate smart grid km fiber-optic cable that connects the control center and the
operation. Simulations performed in this step run in real-time. power system model developed in OPAL-RT, as shown in
The package exchanging among these three components uses Figure 4. In this simulation, two background traffic
the UDP protocol. generators (one is attached to the control center; the other is
IV. CASE STUDY attached to the OPAL-RT node) are used to create
background traffic representing communication traffic of
In this section, a case study of a simplified distribution other smart grid applications that shares the same channel.
automation application is simulated using the real-time co- Load data used in the case study is derived from a real-
simulation platform discussed in Section III. world distribution substation of an electric utility in Virginia.
In the simulation, a simplified distribution system model is The data resolution is in one-minute intervals. The load
set up in RT-LAB, whose single-line diagram is shown in profile at Bus 4 used in this study is shown in Figure 5 for a
Figure 3. As shown, a capacitor bank is connected at Bus 4 24-hour period.
through a switch, which is remotely controlled by the control
center located 100 km away. The control center monitors 6
x 10
voltage at Bus 4 and sends control commands to switch 4
ON/OFF the capacitor based on the voltage level. 3.5

2.5
Load (W)

1.5

1
Figure 3. Single-line diagram of the simulated power system model 0.5

Two scenarios are simulated in this study. In the first 0

0 4 8 12 16 20 24
scenario, the control center monitors the voltage at Bus 4, but Time (hour)
there is no control command sent to the switch. In this case, Figure 5. Load profile at Bus 4
the switch keeps its open status, implying that the capacitor
bank is always disconnected. In the second scenario, the The real-time co-simulation platform is used to simulate
voltage at Bus 4 is monitored, and a control command is sent the two scenarios discussed above. Simulation results
to switch ON the capacitor bank if the monitored voltage is showing voltage profiles at Bus 4 are shown in Figure 6.
lower than 0.95 per unit (pu). When the monitored voltage is
higher than 0.98 pu, a control command is sent to switch OFF
the capacitor bank. 1
Without Capacitor Bank
In order to simulate the system in the developed real-time 0.99 With Capacitor Bank
co-simulation platform, a communication model associated
0.98
Voltage (pu)

with this power system model is developed in OPNET, as

shown in Figure 4. 0.97

0.96

0.95

0.94

0.93
0 4 8 12 16 20 24
Time (hour)
Figure 6. Voltage profiles at Bus 4 w/ and w/o the capacitor bank

For the first scenario, the capacitor bank is always

Figure 4. Communication system model in OPNET disconnected. Voltage at Bus 4 without the capacitor bank is
shown as the dotted blue line in Figure 6. The voltage is
 5

lower than 0.95 pu during the morning peak period, which up using OPNET and OPAL-RT. In this platform, the
begins at 7 am. simulation of power system operation takes into account the
In the second scenario, voltage at Bus 4 is shown as the communication system performance. The detailed network
solid red line in Figure 6. The voltage drops below 0.95 pu at configuration and operation of this platform are discussed. A
around 7:30 am. Therefore, the switch is closed to connect simplified distribution automation case study is simulated on
the capacitor bank when the command from the control center this real-time co-simulation platform. Results of the case
is received. The recorded latency is around 0.002 seconds. study verify that this platform can provide a real-time co-
This represents the duration from when the voltage is simulation analysis for smart grid applications, such as the
measured at Bus 4 to when the switch receives the command impact of communication system failures.
from control center. Once the switch is closed, voltage at Bus
4 is increased above 0.95 pu as expected. Note that this REFERENCES
0.002-second latency is obtained assuming that the occupied [1] Fang, Xi; Misra, Satyajayant; Xue, Guoliang; Yang, Dejun, "Smart
Grid -The New and Improved Power Grid: A
bandwidth of the fiber-optic communication channel is less Survey," Communications Surveys & Tutorials, IEEE , vol.14, no.4,
than 50% using the background traffic generators. pp.944-980, 2012.
Figure 7 illustrates communication latencies for the switch [2] U.S. Department of Energy, “A survey of wireless communications
to receive commands from the control center. Using the afor the electric power system” [Online], Available:
https://www.pnnl.gov/nationalsecurity/technical/secure_cyber_systems/
background traffic generators, three cases with different pdf/power_grid_wireless.pdf. Retrieve: Nov. 2014.
background traffic are simulated, i.e., 0%, 50% and 100% of [3] ABB North America, “Power systems automation and communication
the channel bandwidth. protection and control for reliable and optimized power delivery”
[Online], Available:
http://www05.abb.com/global/scot/scot400.nsf/veritydisplay/ae01e7a2e
47c7a9fc1257bf3004681e9/$file/9AKK105713A8759_LoRes_en_Pow
er_Systems_Automation_and_Communication_ABB_North_America.
pdf. Retrieve: Nov. 2014.
[4] Karlheinz Schwarz, “Monitoring and control of power systems and
ocmmunication infrastructures based on IEC 61850 and IEC 61400-25”
[Online], Available:
http://www.nettedautomation.com/download/pub/dt-tampa-
paper_2010-03-24.pdf. Retrieve: Nov. 2014.
[5] Li, W.; Monti, A.; Luo, M.; Dougal, R.A., "VPNET: A co-simulation
framework for analyzing communication channel effects on power
systems," IEEE Electric Ship Technologies Symposium (ESTS), pp.143-
149, April 2011.
[6] Godfrey, T.; Mullen, S.; Dugan, R.C.; Rodine, C.; Griffith, D.W.;
Golmie, N., "Modeling Smart Grid Applications with Co-
Simulation," IEEE Smart Grid Communications (SmartGridComm)
Conference, pp.291,296, Oct. 2010.
Figure 7. Communication latency
[7] Nutaro, J.; Kuruganti, P.T.; Miller, L.; Mullen, S.; Shankar, M.,
"Integrated Hybrid-Simulation of Electric Power and Communications
When the background traffic is lower than the channel Systems," IEEE Power Engineering Society General Meeting, pp.1-8,
bandwidth, the latency relies on the channel length. It can be June 2007
seen that with 0% and 50% bandwidth utilization, the [8] Heimlich, O.; Sailer, R.; Budzisz, L., "NMLab: A Co-simulation
latencies are around 0.002 seconds, as shown as the solid line Framework for Matlab and NS-2," Advances in System Simulation
(SIMUL) Conference, pp.152-157, Aug. 2010
and the line with dots in Figure 7. [9] PowerWorld Corporation, [Online], Available:
With smart grid development, communication channels http://www.powerworld.com/products/simulator/overview. Retrieve:
may need to be shared with many other smart grid Nov, 2014.
applications. In some situations, the channel utilization can be [10] Electric Power Research Institute, [Online], availabe:
http://smartgrid.epri.com/SimulationTool.aspx. Retrieve: Nov, 2014.
close to 100% of its capacity during certain peak periods. The
[11] RTDS Technologies, [Online], Available:
latency may also increase due to the queueing delay caused http://www.rtds.com/hardware/hardware.html. Retrieve: Nov, 2014.
by high data transfer. An example of latency with 100% [12] OPAL-RT Technologies, [Online], availabe: http://www.opal-
channel utilization is shown as the dashed line in Figure 7. rt.com/product/rt-lab-professional-real-time-digital-simulation-
Under this situation, poor communication performance may software. Retrieve: Nov, 2014.
potentially have negative impacts on the operation of the [13] Jianli Pan, “A Survey of Network Simulation Tools: Current Status and
Future Developments”, [Online], Available:
smart grid, depending on criticality of smart grid applications. http://www.cs.wustl.edu/~jain/cse567-08/ftp/simtools/index.html
Therefore, the real-time co-simulation platform is helpful for [14] “Introduction to SITL” [Online], Available:
analyzing the impact of communication failure on smart grid https://support.riverbed.com/bin/support/static//doc/opnet/17.5.A/online
/modeler_17.5_PL5/Tutorials/wwhelp/wwhimpl/common/html/wwhelp
operation. .htm#href=sitl_tut_1.html&single=true Retrieve: Nov, 2014.
V. CONCLUSION [15] OMNeT++, [Online], Available: http://www.omnetpp.org/models
Retrieve: Nov, 2014.
This paper discusses a real-time co-simulation platform set