sensors

Article
F-DCS: FMI-Based Distributed CPS Simulation
Framework with a Redundancy Reduction Algorithm
Seokjoon Hong 1,† , Ducsun Lim 1,† , Inwhee Joe 1 and WonTae Kim 2, *
1 The Department of Computer and Software, Hanyang University, 222 Wangsimni-ro, Seoul 04763, Korea;
[email protected] (S.H.); [email protected] (D.L.); [email protected] (I.J.)
2 The Department of Computer Science and Engineering, Korea University of Technology and Education,
Cheonan-si 31253, Korea
* Correspondence: [email protected]; Tel.: +82-41-560-1485
† These authors contributed equally to this work.




Received: 30 October 2019; Accepted: 29 November 2019; Published: 1 January 2020


Abstract: A cyber physical system (CPS) is a distributed control system in which the cyber part and
physical part are tightly interconnected. A representative CPS is an electric vehicle (EV) composed
of a complex system and information and communication technology (ICT), preliminary verified
through simulations for performance prediction and a quantitative analysis is essential because
an EV comprises a complex CPS. This paper proposes an FMI-based distributed CPS simulation
framework (F-DCS) adopting a redundancy reduction algorithm (RRA) for the validation of EV
simulation. Furthermore, the proposed algorithm was enhanced to ensure an efficient simulation
time and accuracy by predicting and reducing repetition patterns involved during the simulation
progress through advances in the distributed CPS simulation. The proposed RRA improves the
simulation speed and efficiency by avoiding the repeated portions of a given driving cycle while
still maintaining accuracy. To evaluate the performance of the proposed F-DCS, an EV model was
simulated by adopting the RRA. The results confirm that the F-DCS with RRA efficiently reduced the
simulation time (over 30%) while maintaining a conventional accuracy. Furthermore, the proposed
F-DCS was applied to the RRA, which provided results reflecting real-time sensor information.

Keywords: cyber-physical system; electric vehicle; distributed co-simulation; functional mock-up

interface; zero crossing; driving cycle

1. Introduction
Along with the progress and innovations in information and communication technology (ICT) in
recent years, its application range has been gradually expanding. A cyber physical system (CPS) is a
distributed control system in which the cyber part and physical part are tightly interconnected and
it is defined as a next-generation paradigm that has real-time and intelligent characteristics as well
as predictability and advanced adaptability [1–4]. A CPS can realize seemingly impossible tasks by
tightly combining physical objects, including machines, automobiles, factory facilities, and humans,
with technologies defined as the cyber part. It can acquire various types of information through
monitoring by using sensors in heterogeneous devices and facilitate interaction and collaboration,
by which information is shared with other sensors and actuators through a network. Based on such
shared information, it facilitates accurate analysis for the physical part and appropriate operation
of desired functions in order to use them efficiently. Accordingly, cyber-physical systems (CPS) are
being applied to a broad range of research areas, such as intelligent transport systems, smart grids,
medical equipment, smart manufacturing systems, nuclear power plants, aviation, and the automobile
industry [5–9]. In the automobile industry, a CPS is usually applied as a system that monitors and

Sensors 2020, 20, 252; doi:10.3390/s20010252 www.mdpi.com/journal/sensors

Sensors 2020, 20, 252 2 of 27

controls a combination of sensors, actuators, and physical components through a network. In the rapidly
growing present society, academics and industries are adopting plans of using green energy, such as
hydrogen, as solutions to reduce environmental problems, transportation problems, and dependency
on conventional fossil fuels [10,11].
In this regard, cyber physical energy systems (CPESs), an extension of CPSs, are applied to the
energy domain [12,13]. A CPES consists of an energy resource, network, application, and consumers,
and based thereupon, the requirements for next-generation energy are satisfied. The goal of CPES is
to further enhance real-time sensing and dynamic monitoring in a distributed system and wide area
network (WAN) environment with respect to the use of an energy resource [14]. New CPES-related
information technologies have been applied to electric vehicles (EVs), smart grids, home automation
systems, etc., with the aim of achieving efficient energy usage and environmental protection. Tehrani
and Maurice [13] presented a design for a cyber energy system to enhance the autonomy of EVs.
Kim and Mosse [15] provided a general framework for the design, modeling, and simulation of a CPS.
A conventional vehicle system consists of mechanical elements such as an engine and transmission.
In an electric vehicle (EV), however, ICT is applied in the form of software and communication
technologies, which are combined with electrical parts such as battery management systems (BMS) and
electric machine; they are monitored and controlled through feedback loops of sensing and actuation.
Therefore, the best approach towards improving the efficiency performance prediction of EVs would
be to optimize the system and reduce the development time through CPS modeling based on combined
ICTs. This can be accomplished by selecting each component of EVs based on CPS characteristics and by
collecting and applying the information. Preliminary verification through simulations for performance
prediction and quantitative analysis are essential before application to a real model because a complex
system comprises various highly coupled components [16–19]. Such simulations are considered very
important because they can continually improve the model throughout the development process of the
system model to facilitate the efficient operation of the system.
Co-simulation [20] is a simulation technique that facilitates the execution of global simulation by
combining an existing simulation tool and a simulator generated from different simulation tools in a
whole system. Instead of using only one type of tool and language when performing a simulation,
a specialized tool called a solver (a software component that includes event processing and integration
algorithm to solve a given problem in a model) is used to perform the modeling separately for
independent parts of the system; then, these models are linked via integration and modification,
and co-simulation is performed. In comparison with various conventional monolithic simulations,
co-simulation can facilitate the independent modeling of subsystems and then integrate and modify
the whole system. Therefore, co-simulation is more effective in the aspect of merging domains and
reusing and sharing the model. Furthermore, co-simulation offers advantages for the verification of
CPS and multi-domain systems as it provides a method of considering multiple domains with mutually
different time steps.
Research on co-simulation has been conducted in various areas such as automobiles [21–24],
electric power systems [25], and aviation [26]. When simulating a complex CPS model in the EV and
various other areas, a user can perform the simulation by additionally using a model developed with a
different simulation tool.
The functional mock-up interface (FMI) standard [27], which is an international standard for
methods of interfaces in simulation models, supports co-simulation to facilitate easy integration of
models developed through various simulation tools. The FMI defines a standardized application
programming interface (API) such that simulators can operate together and a model that implements
it is defined as a functional mock-up unit (FMU). The FMU is a “black box” that is equipped with
input port, output port, and status variables set, and uses an API to implement the initialization of
status variables, setting of the input port value and searching for the output port value. Continuously
developing and managing this standard has many potential benefits. First, in general, the details
included in the “black box” are not available to users. This can significantly reduce the time required
Sensors 2019, 20,
Sensors 2020, 19, 252
x 33 of
of 26
27

for gaining knowledge on operation details and the amount of relevant knowledge required. It is thus
for gaining knowledge on operation details and the amount of relevant knowledge required. It is
a potentially excellent method for preserving the intellectual property (IP) of manufacturers.
thus a potentially excellent method for preserving the intellectual property (IP) of manufacturers.
Therefore, a dynamic model developed in a different simulation tool needs not be converted into a
Therefore, a dynamic model developed in a different simulation tool needs not be converted into a
model directly suitable for the host simulation tool. Secondly, it features simulator independence
model directly suitable for the host simulation tool. Secondly, it features simulator independence with
with respect to compiling, linking, and distributing FMU. Thus, application is possible even without
respect to compiling, linking, and distributing FMU. Thus, application is possible even without the
the information of the target simulator. Thirdly, there is a clear possibility that a technology exclusive
information of the target simulator. Thirdly, there is a clear possibility that a technology exclusive to
to a certain area can be widely applied to other areas as well by properly using the advantages of the
a certain area can be widely applied to other areas as well by properly using the advantages of the
FMI standard. In the Figure 1, the respective components were developed through EV research [28],
FMI standard. In the Figure 1, the respective components were developed through EV research [28],
and we implemented as virtual prototypes.
and we implemented as virtual prototypes.

Figure 1.
Figure 1. FMI-based simulation in the electric vehicle
vehicle (EV)
(EV) domain.
domain

The FMI [27] standard specifies an API for interconnected FMUs and defines a master algorithm
(MA) in terms of calling the API and orchestrating the co-simulation, but it does not designate any
MA. The
specific MA. The design
designand anduseuseofofan anappropriate
appropriateMA MAare areimportant
important because
because thethe requirements
requirements of
of simulation
simulation differ
differ depending
depending on the
on the situation.
situation. If a simulation
If a simulation is performed
is performed without
without a necessary
a necessary MA,
MA, functions
functions for basic
for basic modeling,
modeling, including
including simple
simple discrete-eventsimulation
discrete-event simulationand and variable
variable step-size
numerical integration
numerical integration algorithm,
algorithm, cannot be used.
cannot be The MA controls
used. The MA datacontrols
exchangedata and synchronization
exchange and
between FMUs. The
synchronization most FMUs.
between important Therequirements
most important for requirements
the implementation for theof the MA are accuracy
implementation and
of the MA
are accuracy and efficiency. Usually, these two requirements are contradictory. In a simulation, the
efficiency. Usually, these two requirements are contradictory. In a simulation, the MA can reduce
allowable
MA error bythe
can reduce improving
allowable theerror
accuracy through repeated
by improving steps, which
the accuracy through significantly increases
repeated steps, the
which
runtime. If a step
significantly is not the
increases repeatedly
runtime.performed,
If a step the efficiency
is not can be
repeatedly improved by
performed, theincreasing
efficiency thecanstep
be
size according
improved to the stepthe
by increasing performed.
step size This will, however,
according to the step decrease the accuracy.
performed. This will, Therefore,
however,despite
decreaseits
important
the role,
accuracy. it is difficult
Therefore, to devise
despite an MA. role, it is difficult to devise an MA.
its important
Accordingly, attempts
Accordingly, attempts havehave been
been made
made to to design MAs having the characteristics of accuracy and
efficiency [29,30]. In InLee
Lee[29],
[29],step
stepdetermination
determinationwas was considered
considered a major
a major characteristic
characteristic of the
of the MA.MA.
In
In their
their simulation,
simulation, thethe largest
largest stepsize
step sizethat
thatcancanbebeaccommodated
accommodatedbefore before performing
performing the the step
step was
determined for every FMU, and among them, the minimum value was selected to determine the step
However, even
size. However, evenafter
afterfinishing
finishingthe thesimulation
simulationstep, step,thethe
step
stepsize was
size stillstill
was found to be
found to large. On On
be large. the
other
the hand,
other [30] [30]
hand, presented a stepa revision
presented step revisionthat enhances
that enhancesthe stepthedetermination,
step determination, and it included rolling
and it included
back to back
rolling a previous timepoint
to a previous before moving
timepoint the step the
before moving forward in the whole
step forward simulation.
in the Based onBased
whole simulation. these
studies,
on thesethis paperthis
studies, proposes
paper an FMI-based
proposes distributeddistributed
an FMI-based CPS simulation frameworkframework
CPS simulation (F-DCS) adopting
(F-DCS)a
redundancy
adopting reduction algorithm
a redundancy reduction(RRA) algorithmfor the validation
(RRA) for theofvalidation
EV simulation.
of EV simulation.
The remainder
remainderofofthis paper
this paper is organized
is organized as follows. Section
as follows. 2 describes
Section the FMI,the
2 describes MAFMI,
that performs
MA that
the rollback
performs theand data distribution
rollback and data service (DDS)service
distribution framework. (DDS)Section 3 explains
framework. the Distributed
Section 3 explains CPS the
simulation framework with RRA in detail. Section 4 presents a
Distributed CPS simulation framework with RRA in detail. Section 4 presents a Distributed CPSDistributed CPS model based on
model based on hybrid modeling and Section 5 describes the validation of EV simulation of the
Sensors 2020, 20, 252 4 of 27

hybrid modeling and Section 5 describes the validation of EV simulation of the proposed FMI-based
Distributed CPS simulation framework, and lastly, Section 6 concludes the paper.

2. Related Work

2.1. Function Mock-Up Interface

The FMI standard is an independent interface tool standard developed through MODELISAR
project research managed by the Modelica Association. It supports standardized data exchange
between dynamic models designed in mutually different simulation environments by using C codes
and xml files [27]. The main goal of FMI is to make improvements such that models designed with
different modeling tools can exchange data and components or sub-systems that can interoperate.
The models are packaged in the form of a black box called the FMU, which can be connected to a larger
upper-level model owing to the standard interface that can access the state and derivatives included in
the equations. In the abstract aspect, the FMU can be viewed as a timed Mealy machine, a finite-state
machine in which the output value is determined by the status of the input value only, and an access
from an outside part can interact with the FMU through the API only. The FMI standard has adopted
an API that the FMU should comply with.
We introduce some essential functions of FMI that allow communications between a master and a
slave when running a simulation.

• fmiDoStep: This function is provided by the FMI for a method of advancing time in co-simulation.
There are three arguments in a call: currentCommunicaiton, which shows the current click
time of the master; communicationStepSize, the step size that the solver has to calculate;
and noSetFMUStatePriorToCurrentPoint, which shows whether the master will call/not call
fmiSetFMUState for the previous communication point. If the fmiOK call is returned, the step has
been successfully executed, and if it cannot be executed, fmiError is returned. If the slave returns
FMIDiscard, only some parts of the communication step have been calculated, and here, the MA
should be re-executed with a smaller communication step size.
• fmiSetXXX and fmiGetXXX: The MA provides input data to the FMU by calling a procedure called
fmiSetXXX provided by the FMU. It calls fmiGetXXX to retrieve an output value from the FMU
and provides an argument for the ID of the output and a pointer to a storage location (“XXX” is
defined with the data type of the input and output).
• fmiSetFMUState and fmiGetFMUState: These functions allow the master to store and restore the
whole state of the slave. fmi2SetFMUstate restores the internal state of the FMU. This requires the
input pointer previously copied for the FMU state and the current state of the FMU is replaced
with the copied one. fmi2GetFMUstate stores the internal state of the FMU. It receives the pointer
related to the FMU state as an input and copies the current FMU state.

These functions allow the master to roll back the communication step. The implementation of
these functions is optional. They deal with a legacy system. The FMU can wrap the legacy simulator
that does not allow storing and restoring.
As the FMI includes various APIs which allow reading and writing of the FMU’s state, large support
can be provided to the MA, but several factors need to be reinforced. First, because overhead can
occur if a state is stored and restored, the design should consider a method for reducing the overhead.
Secondly, because the implementation of the rollback procedure by the FMU is optional, it may be
quite difficult to implement it according to the characteristics of each model. Thirdly, as shown in
our research, although the APIs supported in the FMI are quite helpful, the supported APIs are not
sufficient in terms of designing to prevent the occurrence of rollback in a section where it should not
occur. Therefore, this study implemented the rollback function by complying with the API supported
in the FMI standard, such that it will be supported in the FMU. Determining the sequence of calling
these procedures is an essential factor in the design of the MA.
Sensors 2020, 20, 252 5 of 27
Sensors 2019, 19, x 5 of 26

2.2.
2.2. Master
Master Algorithm
Algorithm for
for Rollback
Rollback
Many
Many MAsMAsforfor orchestrating
orchestrating FMI-based
FMI-based co-simulation
co-simulation haveproposed.
have been been proposed.
In [29], theInsimulation
[29], the
simulation
was was through
performed performed through
doStep with doStep withsize
a basic step a basic
firststep size first
for several for several
FMUs. FMUs.
If an FMU hasIfnotanfound
FMU
has not found the ZCP, a rollback to a previous time instance is performed. Afterwards,
the ZCP, a rollback to a previous time instance is performed. Afterwards, the maximum step size that the maximum
step
can besize that can be
proceeded proceeded
with with by
by each FMU each FMU
is checked, andis checked,
among them, and among them, the
the minimum minimum
value value
is selected to
is selected
proceed to the
with proceed with the
simulation. simulation.
In contrast, [30]In contrast,
proposes the[30] proposes
following the following
method. method.
The procedure The
of step
procedure of step revision
revision algorithm algorithm
is illustrated by theissequence
illustrated in by the sequence
Figure in Figure 2. When
2. When co-simulation co-simulation
is performed using
is performed
several FMUs,using several FMUs,
the simulation the simulation
is carried out with theisbasic
carried
stepout
sizewith the basic
(h) first (T2 =step
T0 +size
h). (h)
Among (𝑇 =
first them,
𝑇 + ℎ).
if an Among
invalid them,
state if anflag
or error invalid state
occurs inor
anyerror flagFMUs,
of the occursthein any of the
whole FMUs, the
simulation whole back
is rolled simulation
to the
is rolled back
previous to the previous
simulation simulation
time T0 . Then, time 𝑇 . Then,
after adjusting after
the step adjusting
size of all thethe stepto
FMUs size of all the
a smaller FMUs
step size
to a smaller
(h’), step size
the simulation is (h’), the simulation
repeated is repeated
until the ZCP is reached.until the ZCP is reached.

Figure 2. Step
Step revision
revision algorithm with rollback.

2.3. Middleware
2.3. Middleware for
for Distributed
Distributed Co-Simulation
Co-Simulation
The concept
The concept of of distributed
distributed co-simulation
co-simulation is is based
based on on the
the idea
idea that
that simulations
simulations are are performed
performed on on
multiple computers
multiple computers and and can
can bebe distributed
distributed notnot only
only logically
logically but
but also
also physically
physically through
through aa network.
network.
If distributed co-simulation is performed using more than one remote
If distributed co-simulation is performed using more than one remote simulator, the simulation simulator, the simulation
execution time
execution time cancan be
be reduced
reduced by by balancing
balancing thethe workloads
workloads at at multiple
multiple nodes
nodes through
through parallel
parallel tasks.
tasks.
Subsequently, the results of executing at distributed domains should be
Subsequently, the results of executing at distributed domains should be integrated with the integrated with the framework.
A middleware
framework. is used to perform
A middleware is usedthe to co-simulation, in which a multi-agent
perform the co-simulation, in which aarchitecture
multi-agentfacilitates
architecture the
execution the
facilitates of parallel andofdistributed
execution parallel and tasks.
distributed tasks.
Many open-source tools have
Many open-source tools have been been developed
developed andand researched
researchedfor FMU
for FMUsimulations. Among
simulations. them
Among
them are tools that support distributed co-simulation of FMU, such as DACCOSIM [31] and FMI [32].
are tools that support distributed co-simulation of FMU, such as DACCOSIM [31] and FMI Go! Go!
DACCOSIM is an FMI-based co-simulation tool implemented in
[32]. DACCOSIM is an FMI-based co-simulation tool implemented in Java. A user can design andJava. A user can design and
execute multi-simulation
execute multi-simulation based based on on cooperation
cooperation between
between multiple
multiple FMUs
FMUs in in multi-core
multi-core or or cluster
cluster
environments. The
environments. The technology
technology used used forfor exchanging
exchanging data data between
between FMUWrappers
FMUWrappers varies varies depending
depending
on the
on the corresponding
correspondinglocation.
location.InIn FMUWrapper,
FMUWrapper, a shared
a shared queue
queue is used
is used for communication
for communication to a
to a local
local master, and messages are sent through the ZeroMQ middleware
master, and messages are sent through the ZeroMQ middleware for communication between a local for communication between
a local and
master master and amaster.
a global global FMImaster.Go! FMI
is an Go! is an open-source
open-source software with software with permissive
permissive license, andlicense,
it was
and it was designed for using components compatible with
designed for using components compatible with the FMI standard in the distributed the FMI standard in the distributed
simulation
simulation environment
environment and platform. andFMI-Go!
platform. FMI-Go!
uses uses a server-client
a server-client architecturearchitecture in which
in which servers hostservers
FMU
host FMU individually, and uses the ZeroMQ middleware to send
individually, and uses the ZeroMQ middleware to send messages for mapping of various FMI messages for mapping of various
FMI functions.
functions.
This paper
This papersuggests
suggestsan alternative
an alternative or replacement
or replacementof conventional solutions from
of conventional the middleware
solutions from the
perspective, verifying the distributed co-simulation environment by
middleware perspective, verifying the distributed co-simulation environment by using DACCOSIM, using DACCOSIM, FMIGO!.
We applied the data distribution service (DDS), a data communication
FMIGO!. We applied the data distribution service (DDS), a data communication middleware, to middleware, to facilitate
efficient and
facilitate autonomous
efficient data exchange
and autonomous datainexchange
various platforms
in various andplatforms
communication environments
and communication
demanded by users. The DDS is a publish/subscribe-based network
environments demanded by users. The DDS is a publish/subscribe-based network communicationcommunication middleware and
an Object Management
middleware and an Object Group (OMG) standard
Management Group that supports
(OMG) real-time,
standard thatscalable,
supportsand dependable
real-time, data
scalable,
exchanges [33].
and dependable data exchanges [33].
An application used for communication in DDS is DomainParticipant. Multiple domains can
exist in a same network, but DomainParticipant can subscribe to only one of them and can be a
 Sensors 2019, 19, x 6 of 26
Sensors 2020, 20, 252 6 of 27

publisher, subscriber, or both. A publisher includes one or more DataWriter, and a subscriber
manages one or more DataReader. DataWriter and DataReader are bound with a topic when
An application used for communication in DDS is DomainParticipant. Multiple domains can exist
generated from the publisher and the subscriber, respectively. If bound with the same topic, they are
in a same network, but DomainParticipant can subscribe to only one of them and can be a publisher,
interconnected logically. However, the subscriber does not need to know the location of the
subscriber, or both. A publisher includes one or more DataWriter, and a subscriber manages one
publisher. Multiple publishers can post for the same topic and multiple subscribers can subscribe to
or more DataReader. DataWriter and DataReader are bound with a topic when generated from the
the same topic.
publisher and the subscriber, respectively. If bound with the same topic, they are interconnected
The pub/sub implemented in the DDS is a data distribution technology in which a user
logically. However, the subscriber does not need to know the location of the publisher. Multiple
participating in a domain becomes a publisher/subscriber to produce and consume the desired data
publishers can post for the same topic and multiple subscribers can subscribe to the same topic.
only, unlike the server-client system. If a large number of devices provide data frequently in a
The pub/sub implemented in the DDS is a data distribution technology in which a user participating
distributed manner in the distributed co-simulation, it may not be efficient. As the pub/sub
in a domain becomes a publisher/subscriber to produce and consume the desired data only, unlike the
distribution technology facilitates the free participation and withdrawal of domain and provides
server-client system. If a large number of devices provide data frequently in a distributed manner in
functions for producing and collecting desired data only, it can provide a suitable environment.
the distributed co-simulation, it may not be efficient. As the pub/sub distribution technology facilitates
Furthermore, because it facilitates access to data regardless of location, time, and synchronization
the free participation and withdrawal of domain and provides functions for producing and collecting
with respect to the service, it provides a suitable environment for real-time communication. This will
desired data only, it can provide a suitable environment. Furthermore, because it facilitates access to
increase the possibility of making progress when activating connections and integration of the
data regardless of location, time, and synchronization with respect to the service, it provides a suitable
Internet of Things (IoT) for a digital twin.
environment for real-time communication. This will increase the possibility of making progress when
activating connections and integration of the Internet of Things (IoT) for a digital twin.
3. Distributed CPS Simulation Framework with RRA
3. Distributed CPS Simulation Framework with RRA
3.1. Distributed CPS Simulation Framework
3.1. Distributed
We propose CPS Simulation Framework
a framework consisting of DDS middleware and FMUWrapper for the FMI-based
distributed co-simulation.
We propose a framework Here, FMUWrapper
consisting of DDSismiddleware
an FMI interface for simulatingfor
and FMUWrapper the
theFMU at each
FMI-based
distributed co-simulation.
distributed node. Moreover, theFMUWrapper
Here, DDS middleware is an not
FMIonly sendsfor
interface the simulation
simulating thedata
FMU produced
at each
between FMUs,
distributed but also sends
node. Moreover, the DDS control messages
middleware for sends
not only simulation status information
the simulation data produced and
synchronization
between FMUs, but of also
rollback
sendsperformed between
control messages forone Global Master
simulation node and several
status information Local Master
and synchronization
nodes.
of rollback performed between one Global Master node and several Local Master nodes.
Inthe
In thedistributed
distributedCPS CPSsimulation
simulationframework
frameworkproposed
proposedin inFigure
Figure3,3,thethedistributed
distributednodes
nodesare arethe
the
Domain Participants.
Domain Participants. Moreover, a structure of of Ctrl,
Ctrl, Status,
Status, andand Data
Data types
types isis used
used toto transmit
transmit the the
control data and simulation data between the nodes. First, the Ctrl type can be used
control data and simulation data between the nodes. First, the Ctrl type can be used when the global when the global
nodecontrols
node controlsother
otherlocal
localnodes.
nodes. For
For example,
example, the
the global
global node
node cancan use
use Ctrl
Ctrl type
type messages
messages to to request
request
thestarting,
the starting,stopping,
stopping,and androlling
rollingback
backofofsimulation
simulationto tothe
thelocal
localnodes.
nodes. The The Status
Status type
typeallows
allowseacheach
localnode
local nodetotoshare
shareits
itssimulation
simulationperforming
performingstatus
statusinformation
informationto tothe
theglobal
globalnode.
node.TheTheData
Datatype
typeisis
usedwhen
used whenthethesimulation
simulationdatadataare
aretransmitted
transmittedbetween
betweenthethenodes.
nodes.

Figure3.3. FMI-based
Figure FMI-based distributed
distributed cyber
cyber physical
physical system
system(CPS)-simulation
(CPS)-simulationframework.
framework.

First, if the simulation is performed normally, the global node performs the simulation using the
initial step size value for the simulation time t through FMU Wrapper and sends the result to the
Sensors 2020, 20, 252 7 of 27

First, if the simulation is performed normally, the global node performs the simulation using
the initial step size value for the simulation time t through FMU Wrapper and sends the result to the
connected nodes through Data-type messages. A node that has received the data sets the reference
value according to the data I/O connection setting between the FMUs of co-simulation; afterwards,
it sets the input value with the setReal function. Here, a data message sent from the global node
contains the current step size information as well and when each local node receives this message,
it changes the current simulation step size with the step size from the data message and performs the
simulation. Hence, the global node and local nodes are synchronized. If the simulation for each FMU is
performed without any problem at the current simulation time (t) of all the connected nodes, the global
node increases the next simulation time (t + hh) and performs the simulation again for the FMUs.
If a rollback occurs while performing the simulation from time t to t+hh at a certain node,
the rollback to time t is performed first. Subsequently, the global node is informed about the rollback
through a Status message. Then, the global node performs the rollback and sends a Rollback Ctrl
message to all connected local nodes such that all nodes can be synchronized by rolling back. Among
nodes that have received the Ctrl message, there may be nodes where the simulation time has not
proceeded to t+hh yet. In this case, the rollback is not performed.
Furthermore, some nodes have a sensing data management module and a sensor provides the
collected data from outside to the node through a cloud in real-time. FM Wrapper inputs the provided
sensing information in the FMU with the setReal function, and the simulation is performed by applying
the provided value. In this manner, the simulation can reflect real-time sensing information, such as
temperature and humidity.
Next, a method of predicting and applying the step size at each node is examined. At each
node, the step size can be determined by using the two proposed algorithms for each FMU. It is
important to determine the step size that can be commonly applied to every FMU in the co-simulation,
in which FMUs in multiple nodes interoperate. For this purpose, the next_hh value is used in DDS
Data messages and Status messages. At the current simulation time t, the step size determined using
the algorithm at each node, is stored as the next_hh value and delivered along with the simulation data
in a DDS Data message from the global node to immediately before the final local node. The receiving
node compares its determined step size with the next_hh value and continually updates it with the
smaller value. Then, the value is delivered to the final node. When the current location is the final
node, its determined step size is compared with the next_hh value and it is updated with the smaller
value. Next, this value is sent to the global node through a Status message. Finally, the global node
determines the step size with the minimum value among the next_hh values received from the final
nodes, and the simulation of t + next_hh is performed.

3.2. Redundancy Reduction Algorithm

Among the most widely used techniques for testing combinations of vehicle components, a typical
method applies a driving cycle to the simulation [34,35]. By applying a driving cycle to the simulation
of an EV model, it is possible to determine the vehicle’s driving distance by time and battery SOC
(state of charge) by power consumption based on the EV’s operation [36–38]. When several driving
cycles are examined, a repetition section can be observed in a cycle, as shown in Figure 4.
3.2. Redundancy Reduction Algorithm
Among the most widely used techniques for testing combinations of vehicle components, a typical
method applies a driving cycle to the simulation [34,35]. By applying a driving cycle to the simulation
of an EV model, it is possible to determine the vehicle’s driving distance by time and battery SOC (state
of charge)
Sensors by 252
2020, 20, power consumption based on the EV’s operation [36–38]. When several driving cycles
8 ofare
27
examined, a repetition section can be observed in a cycle, as shown in Figure 4.

Sensors 2019, 19, x 8 of 26

(a) (b)
Figure 4. Driving cycles for EV simulation; (a) New European Driving Cycle (NEDC), (b) 10–15 mode
Figure 4.
cycle.
Figure 4. Driving
Drivingcycles
cyclesfor EVEV
for simulation; (a) (a)
simulation; New European
New Driving
European CycleCycle
Driving (NEDC), (b) 10–15
(NEDC), mode
(b) 10–15
cycle.
mode cycle.
If the model is simulated by accepting such a driving cycle as an input value and combining it
If
with otherthe model is simulated
components by accepting
in the such a driving
EV, the simulation cyclecan
graph as an input
have value and
positive andcombining
negative it with
values
other components
depending on thein the EV,
time, andthe simulation
many graph can
zero-crossing have(ZCPs)
points positive and
[39] arenegative
produced. values
To depending
find a sectionon
the
thattime,
has and manyofzero-crossing
a value 0 exactly, a points
bisection(ZCPs) [39] are[40]
algorithm produced.
can be To findWhen
used. a section that has
a ZCP a value
occurs of
in the
0simulation
exactly, a of bisection algorithm [40] can be used. When a ZCP occurs in the simulation
a certain model, as shown in Figure 5, this algorithm checks the associated error. If the of a certain
model,
error isas shown
larger thanin the
Figure 5, this
signal algorithm
threshold, thechecks
previousthe step
associated error.with
is repeated If thetheerror
stepis size
larger than the
reduced to
signal threshold, the previous step is repeated with the step size reduced to half,
half, as shown in Figure 5a. Here, if the simulation data value is smaller than the signal threshold, the as shown in Figure 5a.
Here, if the simulation
simulation data is
of the next step value is smaller
performed; thanthe
if not, theabove
signalprocess
threshold, the simulation
is repeated. Suppose of thethenext step is
simulation
performed;
is for Figureif5b, not,
in the
whichabovethe process
graph ofisthe repeated.
simulationSuppose
modelthe simulation
does is for Figure
not go through the point 5b, 0indirectly
which
the
andgraph
the value of thechanges
simulation frommodel
positivedoestonot go through
negative or vicetheversa.
point Then,
0 directly
the and
timethe value changes
threshold can be
from positive
assigned to negative
for the simulation or vice versa.(𝑇Then,
interval −𝑇 the) time
and thethreshold can beofassigned
exact time changing forthethesign
simulation
can be
interval
determined 1 − Tn ) andby
(Tn+accurately theusing
exactthe
time of changing
bisection the sign
algorithm. Oncanthebe determined
other hand, if theaccurately
bisection byalgorithm
using the
bisection algorithm. On the other hand, if the bisection algorithm is used
is used to find the ZCP, this method will be considerably inefficient as the simulation time to find the ZCP, this method
will
will be considerably
increase in the simulation inefficient
modelaswiththe simulation
a repeating time will increase in the simulation model with a
pattern.
repeating pattern.

(a) (b)
Figure 5.
Figure 5. Zero crossing point
Zero crossing point and
and bi-section
bi-section search;
search; (a)
(a) continuous
continuous model,
model, (b)
(b) discontinuous
discontinuous model.
model.

Considering these
Considering these issues,
issues, this
this paper
paper proposes
proposes aa method
method that
that can
can reduce
reduce the simulation
simulation time
time more
more
efficiently while
efficiently whilemaintaining
maintainingthe theconventional
conventionalaccuracy
accuracybybyusing
using the
the repetitive
repetitive pattern
pattern information
information of
of driving
driving cycles
cycles in anin
EVan EV simulation.
simulation. First,performing
First, when when performing a simulation,
a simulation, informationinformation for the
for the simulation
simulation
values values through
is obtained is obtained
the MAthrough
and thetherepetition
MA and section
the repetition sectioncycle
in the driving in the driving cycle
is predicted. Then,is
predicted.
the step sizeThen, the step
is predicted size the
using is predicted using the obtained
obtained information information
and the simulation and the simulation is
is performed.
performed.
This paper proposes two algorithms for reducing the time of FMU-based EV simulation using a
This paper
repetitive proposes
and cyclic two
driving algorithms for reducing the time of FMU-based EV simulation using a
cycle.
repetitive andacyclic
First, for driving
case with cycle.from a cycle in a simulation graph, an adaptive step size algorithm
a pattern
First, for
is proposed toareduce
case with
the asimulation
pattern from timea cycle
usinginthat
a simulation
pattern. Ifgraph, an adaptive
the simulation step size algorithm
is performed multiple
is proposed to reduce the simulation time using that pattern. If the simulation is performed multiple
times with a small step size as the simulation data value is repeated, it may be more efficient to
perform the simulation with a large step size instead of the small step size.
Secondly, for a case with ZCP occurring in a different simulation model in the first starting cycle
of the driving cycle, rollback pattern information is saved. Then, if the ZCP is predicted to occur in
Sensors 2020, 20, 252 9 of 27

times with a small step size as the simulation data value is repeated, it may be more efficient to perform
the simulation with a large step size instead of the small step size.
Secondly, for a case with ZCP occurring in a different simulation model in the first starting cycle
of the driving cycle, rollback pattern information is saved. Then, if the ZCP is predicted to occur in
the same graph cycle in the model, the simulation value in the saved rollback pattern information is
compared with the step size value, thereby finding a mapping step size for the current simulation
value. By applying it, the second proposed algorithm can minimize the frequency of rollbacks in
the simulation.
The
Sensors sequence
2019, 19, x diagram describing this method is shown in Figure 6. 9 of 26

Figure 6. Sequence
Figure 6. Sequence diagram
diagram of
of the
the redundancy
redundancy reduction
reduction algorithm.
algorithm.
3.2.1. Adaptive Step Size Algorithm
3.2.1. Adaptive Step Size Algorithm
Algorithm 1 shows the process of saving level patterns in the adaptive step size algorithm. First,
Algorithm 1: adaptive step size algorithm (saving level patterns)
when the simulation begins, the step size is initialized as 1 and it checks changes by the time of
1 level array is initialized with zero values;
simulation to be measured during the first small repetitive cycle Tp1 while performing the simulation
2 lvIdx = 0;
Figure3 7. while
Here,time
the step
≤ End size
dowas set to 1 initially because the data information of the driving cycle was
given4 in 1 s intervals.
fmiDoStep(c,time, same
If the hh); simulation value is repeated in every step, the length of the repeating
section
5 is checked when the1, repeating
fmiGetReal(c,&vr, &currVal); section is completed, and the value is stored in a structure
designed
6 to store simulation
if prevVal pattern
== currVal theninformation. The structure for storing pattern information has
value,7 duration, and zeroSlopeCnt ++;
order information, representing the simulation value, maintained duration and
8 else
the order of appearing in the cycle, respectively. Here, the duration can be obtained by subtracting the
9
first simulation if from
time the last ≥
zeroSlopeCnt 2 AND time
simulation < 𝑇Furthermore,
time. then when the same value appears again
10 level[lvIdx].duration is calculated;
during the repetition cycle Tp1 , the order is used to distinguish it. In other words, for a simulation
11 lvIdx is increased by one;
value appearing first when saving in the array, the order is 0, and if the same value appears repeatedly,
12 if lvIdx is zero then
2, . . . , n, and stored.
their13orders are set to 1,level[lvIdx].order is set to zero;
14 else
15 Find index i of level array which has the same value as the last one. Then
level[i].order is increased;
16 end
17 end
18 end
19 prevVal = currVal;
20 end

Algorithm 1 shows the process of saving level patterns in the adaptive step size algorithm. First,
when the simulation begins, the step size is initialized as 1 and it checks changes by the time of
 Sensors 2020, 20, 252 10 of 27
Figure 6. Sequence diagram of the redundancy reduction algorithm.

3.2.1. Adaptive Step Size Algorithm

Algorithm 1: adaptive step size algorithm (saving level patterns)
Algorithm 1: adaptive step size algorithm (saving level patterns)
1 level array is initialized with zero values;
2 lvIdx = 0;
3 while time ≤ End do
4 fmiDoStep(c,time, hh);
5 fmiGetReal(c,&vr, 1, &currVal);
6 if prevVal == currVal then
7 zeroSlopeCnt ++;
8 else
9 if zeroSlopeCnt ≥ 2 AND time < 𝑇 then
10 level[lvIdx].duration is calculated;
11 lvIdx is increased by one;
12 if lvIdx is zero then
13 level[lvIdx].order is set to zero;
14 else
15 Find index i of level array which has the same value as the last one. Then
level[i].order is increased;
16 end
17 end
18 end
19 prevVal = currVal;
20 end

Sensors 2019, 19, x 11 of 26

Algorithm 1 shows the process of saving level patterns in the adaptive step size algorithm. First,
when the simulation begins, the step size is initialized as 1 and it checks changes by the time of
simulation to be measured during the first small repetitive cycle 𝑇 while performing the
simulation Figure 7. Here, the step size was set to 1 initially because the data information of the
driving cycle was given in 1 s intervals. If the same simulation value is repeated in every step, the
length of the repeating section is checked when the repeating section is completed, and the value is
stored in a structure designed to store simulation pattern information. The structure for storing
pattern information has value, duration, and order information, representing the simulation value,
maintained duration and the order of appearing in the cycle, respectively. Here, the duration can be

Figure
Figure 7. New
7. New European Driving
European Driving Cycle
Cycle(typical repetition
(typical cycle). cycle).
repetition
Algorithm 2 shows the process of the search for level patterns and step size application in the
3.2.2. Efficient
adaptive Zero Crossing
step size Detection Algorithm
algorithm.

The second algorithm can be used in the co-simulation using ZCD and repeating rollbacks can
be minimized. The ZCD technique is useful because zero crossing (ZC) can be determined. When the
bi-section algorithm is used for ZCD, rollback is performed by reducing the step size by half if ZCP
occurs and the error with 0 is large; rollback is stopped only if the result is below the threshold value.
However, in the case of many ZCPs occurring in a short time, as shown in Figure 8, the simulation
time will be lengthy due to frequent rollbacks if the bi-section algorithm is used.
 obtained by subtracting the first simulation time from the last simulation time. Furthermore, when
the same value appears again during the repetition cycle 𝑇 , the order is used to distinguish it. In
other words, for a simulation value appearing first when saving in the array, the order is 0, and if the
same2020,
Sensors value
20, appears
252 repeatedly, their orders are set to 1, 2, …, n, and stored. 11 of 27
Algorithm 2 shows the process of the search for level patterns and step size application in the
adaptive step size algorithm.
Algorithm 2: adaptive step size algorithm (search of level patterns and step size application)
Algorithm 2: adaptive step size algorithm (search of level patterns and step size application)
1 if zeroSlopeCnt == 1 then
2 if time <= 𝑇 then
3 level[lvIdx].val = currVal;
4 else
5 Find index i of tmplevel array which has the same value as the last one
(level[tmplvIdx].val). Then the level[tmplvIdx].order is increased;
6 for i=0 to lvIdx -1 do
7 if (level[i].val == tmplevel[templeIdx].val) && (level[i].order == tmplevel[templeIdx].order)
then
8 foundIdx = i;
9 end
10 end
11 if The value is found with index as foundIdx then
12 nexthh = level[foundIdx].duration;
13 nextTime = time + nexthh;
14 end
15 end
16 end
17 time = time + hh;
18 if nexthh is set by algorithm then
19 hh = nexthh;
20 end
21 if now time is nextTime then
22 hh=1;
23 end

Next, if the first repetition cycle 𝑇 is completed, the current value is added to the simulation
pattern
Next,information
if the first using a new
repetition structure
cycle Tp1 is variable
completed, for pattern information
the current value is comparison.
added to theIfsimulation
the current
order isinformation
pattern the same asusingthe previous order, the
a new structure step size
variable forispattern
as largeinformation
as the duration in the previous
comparison. pattern
If the current
structure
order is theinformation
same as the is applied.order,
previous In this themanner,
step sizerepetitive
is as largesimulation sections
as the duration in can be reduced.
the previous If the
pattern
value is information
structure not the same, the step In
is applied. is set
thisto 1 by rolling
manner, repetitivebacksimulation
to the previous
sections step
canagain, and then
be reduced. the
If the
simulation
value is not istheperformed.
same, the Usually,
step is setbecause
to 1 bythe simulation
rolling back tovalues are repeated
the previous several
step again, andtimes
theninthe 𝑇
inside the driving
simulation cycle, there
is performed. is a high
Usually, becauseprobability that the values
the simulation same value will appear
are repeated eventimes
several after passing
in Tp2
the step
inside theby as much
driving as there
cycle, the duration.
is a highFurthermore,
probability that because the value
the same same pattern of the
will appear simulation
even value
after passing
canstep
the reappear in the as
by as much repetition cycle, the
the duration. existence of
Furthermore, the same
because thesimulation
same pattern value
of in
thethe stored structure
simulation value
is checked.
can reappearIfinsuch a value exists,
the repetition cycle,ittheis updated
existenceby of adding
the same 1 simulation
to the ordervaluein theinstructure.
the storedThereafter,
structure
isthe existence
checked. of thea same
If such valuevalueexists,initthe previously
is updated bysaved
adding pattern
1 to the information
order in the and the orderThereafter,
structure. is checked.
If there
the is any
existence of such value,
the same the in
value duration is applied
the previously savedas apattern
new step size. In this
information andmanner,
the orderefficiency
is checked. and
Ifspeed
there of the such
is any simulation
value, can the be improved
duration because
is applied as the simulation
a new step size.can In be
thisperformed by maximally
manner, efficiency and
expanding
speed of thethe step size for
simulation canthe
be same
improvedsimulation
becausevaluethein the remaining
simulation can becycles after excluding
performed the first
by maximally
cycle. Furthermore,
expanding the step size because
for thethe
sameEVsimulation
simulationvalue method usually
in the measures
remaining cyclestheafter
EV excluding
driving distance
the firstor
battery
cycle. SOC by repeating
Furthermore, becausethe thewhole cycle 𝑇 method
EV simulation , the patternusually information
measures saved driving𝑇distance
the EVduring can be
orcontinually
battery SOC used.
by repeating the whole cycle Tp2 , the pattern information saved during Tp1 can be
continually used.

3.2.2. Efficient Zero Crossing Detection Algorithm

The second algorithm can be used in the co-simulation using ZCD and repeating rollbacks can be
minimized. The ZCD technique is useful because zero crossing (ZC) can be determined. When the
bi-section algorithm is used for ZCD, rollback is performed by reducing the step size by half if ZCP
occurs and the error with 0 is large; rollback is stopped only if the result is below the threshold value.
 3.2.2. Efficient Zero Crossing Detection Algorithm
The second algorithm can be used in the co-simulation using ZCD and repeating rollbacks can
be minimized. The ZCD technique is useful because zero crossing (ZC) can be determined. When the
bi-section
Sensors algorithm
2020, 20, 252 is used for ZCD, rollback is performed by reducing the step size by half if12 ZCP
of 27
occurs and the error with 0 is large; rollback is stopped only if the result is below the threshold value.
However, in the case of many ZCPs occurring in a short time, as shown in Figure 8, the simulation
However, in the case of many ZCPs occurring in a short time, as shown in Figure 8, the simulation
time will be lengthy due to frequent rollbacks if the bi-section algorithm is used.
time will be lengthy due to frequent rollbacks if the bi-section algorithm is used.

(a) (b)

Figure
Figure 8.8. Examples
Examples of
ofsimulation
simulationusing
usingthe
theNew
NewEuropean
EuropeanDriving
DrivingCycle;
Cycle;(a)(a)
tractive effort
tractive or or
effort force Ft
force
(N), (b)(b)
Ft (N), power consumption
power consumptionfrom thethe
from battery PbcP(W).
battery bc (W).

Therefore,
Therefore, the the proposed
proposed algorithm
algorithm saves
saves the
the rollback
rollback pattern
pattern ifif ZCP
ZCP occurs
occurs in
in the
the first
first repetition
repetition
cycle T𝑇p1 .. If
cycle If ZCP
ZCP occurs
occurs afterwards,
afterwards, aa small
small step
step size
size is
is applied
applied by
by using
using the
the saved
saved rollback
rollback pattern
pattern
information in order to reduce unnecessary rollbacks. To accomplish this, the
information in order to reduce unnecessary rollbacks. To accomplish this, the proposed algorithm proposed algorithm
uses
uses the
the following
following method.
method. As As shown
shown in in Figure
Figure 9,
9, the
the cases
cases of
of ZCP
ZCP occurrence
occurrence can
can be
be divided
divided into
into
two
two cases: occurrence in a graph of increasing or decreasing shape, and continuously maintaining the
cases: occurrence in a graph of increasing or decreasing shape, and continuously maintaining the
same
same value
value and and then
then suddenly
suddenly changing
changing from
from aa positive
positive value
value to
to aa negative
negative value
value or
or vice
vice versa
versa in
in aa
stepwise
Sensors 2019, 19, manner.
stepwise x
manner. 12 of 26

Figure
Figure9.9.Example
Example of zero
zerocrossing.
crossing.

First,First, in the
in the casecase
of aofsequential
a sequentialgraph
graphshape
shape with
with aaslope,
slope,thethesimulation
simulation graph
graphis continuous in in
is continuous
each section; however, the same value is not maintained and the value continuously changes, as shown
each section; however, the same value is not maintained and the value continuously changes, as
in Figure 7. Here, sections are divided into down crossing sections, for each of which the value is
shown in Figure 7. Here, sections are divided into down crossing sections, for each of which the value
changed from a positive value to a negative value, and up crossing sections, in which a negative value
is changed from a positive value to a negative value, and up crossing sections, in which a negative
is changed into a positive value.
value is changed
In the case into a positive
of ZC value.
repeating with such a pattern, a “list” is created to store a certain number of
In the casevalues
simulation of ZC repeating every
continuously with time
suchthe
a pattern,
simulation a is
“list” is created
performed to store
in order to applya certain
the step number
size by of
simulation values continuously every time the simulation is performed in order to apply
predicting the next ZC, as shown in Figure 10. During the first small repetition cycle Tp1 , the simulation the step size
by predicting
values before thethe
next ZC, as of
occurrence shown
ZC areincontinuously
Figure 10. During
stored inthe first small
the “list”. Then,repetition
when down cycle 𝑇 , the
crossing
or up crossing occurs, they are stored in the front part of the array space designed
simulation values before the occurrence of ZC are continuously stored in the “list”. Then, when down to predict each
down/up
crossing or upcrossing.
crossingThe simulation
occurs, values
they are and step
stored size
in the (h) values
front part offrom
the the start
array to thedesigned
space end of rollback
to predict
each down/up crossing. The simulation values and step size (h) values from the start to the end of
rollback are stored next in continuation. Because this study assumed the use of the bisection
algorithm, the step size change while performing the rollback is always smaller than 1, i.e., 2 (n >=
1).
simulation values continuously every time the simulation is performed in order to apply the step size
by predicting the next ZC, as shown in Figure 10. During the first small repetition cycle 𝑇 , the
simulation values before the occurrence of ZC are continuously stored in the “list”. Then, when down
crossing or up crossing occurs, they are stored in the front part of the array space designed to predict
eachSensors 2020, 20, crossing.
down/up 252 The simulation values and step size (h) values from the start to13the of 27
end of
rollback are stored next in continuation. Because this study assumed the use of the bisection
algorithm,
are stored the step
next size change while
in continuation. performing
Because the rollback
this study assumed is of
the use always smaller
the bisection than 1, i.e.,
algorithm, 2 (n >=
the step
1). size change while performing the rollback is always smaller than 1, i.e., 2 (n >= 1).
−n

Figure
Figure10.
10.Zero
Zerocrossing predictionwith
crossing prediction withupup and
and down
down crossing
crossing array.
array.

After
After ZC,ZC,thethe currentsimulation
current simulation value
value is
iscompared
comparedwith thethe
with first value
first in the
value in down/up crossing
the down/up crossing
array saved previously to check if they are the same. If they have the same value, the remaining part of
array saved previously to check if they are the same. If they have the same value, the remaining part
the array are all compared as well, and if all values are consistent, the step sizes saved in the previously
of the array are all compared as well, and if all values are consistent, the step sizes saved in the
performed rollbacks are applied sequentially, thereby preventing unnecessary rollbacks.
previously performed rollbacks are applied sequentially, thereby preventing unnecessary rollbacks.
Secondly, in the case of ZC occurring repeatedly, the simulation graph has a continuous constant
Secondly,
value in eachin the case
section, butofthe
ZCvalue
occurring repeatedly,
changes thevalue
to a different simulation graph
cyclically, has aincontinuous
as shown Figure 9. constant
value inIn each
the section,
case of ZCbutrepeating
the valuewith
changes
such toaa different
pattern, thevalue cyclically,
following as shown
processes in Figure
(Algorithm 3) are9.
performed to predict the next ZC. First, the current simulation value is compared with the previous
value to check if they are the same. When the same value is maintained continuously, the current value
is saved in the array storing the “level history” and the array index is increased. Here, the starting time
of the level was saved, and afterwards, when the ZC reappeared and the simulation value changed,
the ending time of the level was saved as well. Furthermore, the duration from the starting time to the
ending time of the level was saved.
If the current simulation value is the same as the first value of the previous array, a search
is performed to check whether the previously saved level value is in the array. If the level value
exists, the saved duration time is added to the starting time of the current level in order to predict
the time when ZC will occur in the future. If the remaining predicted ZC time is smaller than the
current step size, the simulation is performed by changing the remaining time to the current step size,
thereby moving to the timepoint immediately before the occurrence of ZC. Next, the previously saved
step size value is applied to the next step size. If the step size is applied, the rollbacks can be reduced
in the case of ZC occurrence. However, if ZC does not occur, the simulation is performed by reverting
to the original step size.
Sensors 2020, 20, 252 14 of 27

Sensors 2019, 19, x 13 of 26

Algorithm 3: level zero crossing algorithm
Algorithm 3: level zero crossing algorithm
1 while time ≤ End do
2 if prevVal * currVal < 0 then // zero crossing
3 if hh > 𝑇 then
4 Rollback to previous simulation time.
5 else
6 if preVal == lastLevel then
7 This is end of Rollback;
8 Save last simulation time and step size, and duration to zcLevel array.
9 end
10 hh=1; // Recover to original time step size
11 end
12 end
13 if prevVal == currVal then
14 if lastLevel ≠ currVal then
15 This is first time when zclevel array for different level is saved;
16 Save start time and value of level;
17 else
18 if There is zclevel array saved. AND It is not searched then
19 Compare currVal to level values in zcLevel array.
20 end
21 if It is found then
22 Predict remained time when the step size will be reduced using zcLevel array.
23 if The predicted time is current time then
24 Apply hh with saved hh in zcLevel array.
25 end
26 end
27 end
28 end
29 end

In the case of ZC repeating with such a pattern, the following processes (Algorithm 3) are
4. Distributed to
performed CPS predict
Model theBased
next ZC. First, the
on Hybrid current simulation value is compared with the previous
Modeling
value to check if they are the same. When the same value is maintained continuously, the current
4.1. value
EV Co-Simulation
is saved in Model
the array storing the “level history” and the array index is increased. Here, the
starting
The EVtime of the levelmodel
co-simulation was saved,
was used and to afterwards,
simulate thewhen the ZC reappeared
measurement and of
of the SOC thebatteries
simulation
value changed, the ending time of the level was saved as well. Furthermore,
according to the driving cycle. Figure 11 presents a block diagram of the co-simulation model, the duration from the
starting time to the ending time of the level was saved.
showing the input and output of each FMU. As shown in the figure, each subsystem implements
an FMU Ifofthe current simulation
co-simulation type andvalue
performsis thethesame as the first
compiling using value of theSDK
the FMU previous array,
tool and a FMU
each search is
performed to check whether the previously saved level value is in
calculates and outputs a simulation value based on the input value. Every FMU was implemented the array. If the level value exists,
using fmiSetFMUState and fmiGetFMUState to provide the rollback function, and the performance oftime
the saved duration time is added to the starting time of the current level in order to predict the
the when
proposedZC will
MAsoccur in the through
was tested future. Ifthe
theco-simulation.
remaining predicted ZC time
In previous is smaller
studies [36–38],than
the the current step
components
size, the simulation is performed by changing the remaining time to
of EV were modeled mathematically and based on these, simulations were performed to particularly the current step size, thereby
moving
measure to theconsumption
battery timepoint immediately
according before
to the the occurrence
driving of ZC. Next,
cycle inputted theEV
in the previously saved step
model. Herein,
size value is applied to the next step size. If the step size is
the modeling of the subsystem through a mathematical model by the FMU is explained. applied, the rollbacks can be reduced in
the case of ZC occurrence. However, if ZC does not occur, the simulation is performed by reverting
to the original step size.

4. Distributed CPS Model Based on Hybrid Modeling

4.1. EV Co-Simulation Model

The EV co-simulation model was used to simulate the measurement of the SOC of batteries
according to the driving cycle. Figure 11 presents a block diagram of the co-simulation model,
showing the input and output of each FMU. As shown in the figure, each subsystem implements an
FMU of co-simulation type and performs the compiling using the FMU SDK tool and each FMU
 calculates and outputs a simulation value based on the input value. Every FMU was implemented
using fmiSetFMUState and fmiGetFMUState to provide the rollback function, and the performance
of the proposed MAs was tested through the co-simulation. In previous studies [36–38], the
components of EV were modeled mathematically and based on these, simulations were performed
to particularly measure battery consumption according to the driving cycle inputted in the EV model.
Sensors 2020, 20, 252 15 of 27
Herein, the modeling of the subsystem through a mathematical model by the FMU is explained.

Figure 11.Functional
Figure11. Functionalmock-up
mock-upinterface-based
interface-basedelectric
electricvehicle
vehicleco-simulation
co-simulationmodel.
model.

4.1.1. Driving Cycle FMU

4.1.1. Driving Cycle FMU
This FMU calculates the EV speed and acceleration value by time. The calculation formula is as
This FMU calculates the EV speed and acceleration value by time. The calculation formula is as
follows. In other words, if the vehicle’s driving cycle is referenced, the speed value by current time (s)
follows. In other words, if the vehicle’s driving cycle is referenced, the speed value by current time
can be obtained. However, as this value is a discrete value of the second unit, a continuous simulation
(s) can be obtained. However, as this value is a discrete value of the second unit, a continuous
was facilitated through the following formula for each section by referencing the driving cycle. Here,
simulation was facilitated through the following formula for each section by referencing the driving
if the acceleration a(t) is not provided in the driving cycle, it can be simply calculated as follows.
cycle. Here, if the acceleration a(t) is not provided in the driving cycle, it can be simply calculated as
follows.
v(t) = v0 + a × t (1)
𝑣(𝑡) = 𝑣 + 𝑎 × 𝑡 (1)
v(t) = v(k ) + a(k ) × t (k ≤ t < k + 1) (2)
𝑣(𝑡) = 𝑣(𝑘) + 𝑎(𝑘) × 𝑡 (𝑘 ≤ 𝑡 < 𝑘 + 1) (2)
a(k) = v(k + 1) − v(k) (k = 0, 1, 2, . . . (sec))a (3)

4.1.2. Tractive effort FMU 𝑎(𝑘) = 𝑣(𝑘 + 1) − 𝑣(k) (𝑘 = 0, 1,2, … (sec))𝑎 (3)

Next, the following is a formula used in the FMU model to find the vehicle’s tractive effort or
4.1.2.FTractive effort FMU
force t , traction torque Tt and traction power Pt . Based on these, the vehicle wheel’s angular velocity
and revolution
Next, the per minuteis(RPM)
following are determined.
a formula used in the FMU model to find the vehicle’s tractive effort or
force 𝐹 , traction torque 𝑇 and traction power 𝑃 . Based on these, the vehicle wheel’s angular
velocity and revolution per minuteF(RPM) t = Frr +
areFad +Fhc + Fla + Fwa
determined. (4)

𝐹 =𝐹 F µrr mg
+rr𝐹= +𝐹 +𝐹 +𝐹 (5)
(4)
1
Fad = ρACd v2 (6)
𝐹 2= 𝜇 𝑚𝑔 (5)
Fhc = mgsin(α) (7)
1
𝐹Fla =
= ma𝜌𝐴𝐶 𝑣 (6)
(8)
2
Fwa = 0.05 × Fla (9)
𝐹 = 𝑚𝑔𝑠𝑖𝑛(𝛼) (7)
Tt = Ft × rw (10)
𝐹 = 𝑚𝑎 (8)
Pt = Ft × v (11)
v
ωw = (12)
rw
30
Sw = × ωw (13)
π
Sensors 2020, 20, 252 16 of 27

4.1.3. Gear Box FMU

For a gear box model, the following formulas are used to calculate the electric machine (electric
motors, electric generators)’s shaft power (Ps ) and shaft torque (Ts ) based on the vehicle’s tractive
effort or force. If the tractive force is a negative number, i.e., If brake is applied, power is generated by
regenerative braking; therefore, a negative value is obtained when calculated with the following formula:

Tt
Ts = (Pt > 0) (14)
ηg × G

Tt
Tsr = −η g × (Pt < 0) (15)
G
Ss = Ssr = G × Sw (16)
π
Ps = Ts × Ss × (17)
30
π
Psr = Tsr × Ss × (18)
30

4.1.4. Electric-Machine FMU

We used an induction machine (asynchronous machine) to model the electric machine and assumed
that this model used an AC-DC converter for regenerative braking mode [41,42]. The efficiency map
data determined by this model and the electric machine’s speed and torque data were used to obtain
the efficiency of power consumed in the motoring mode and the efficiency of power generated in the
recuperation mode, which are indicated by ηm and ηr , respectively. Then, based on these, the power
consumed in the motoring mode and the power generated in the recuperation mode were calculated by
the following formulas. The Lookup2d function was implemented in the FMU. This function performs
a role similar to that of lookup2d in MATLAB/Simulink. We calculated efficiency according to torque
and the speed of the electric machine using efficiency curves given in [36].

Efficiency(η) = lookup2d(Torque(Nm), Speed(rpm)) (19)

ηm = lookup2d(Ts , Ss ) (20)

ηr = lookup2d(Tsr , Ssr ) (21)

Ps
Pbm = (22)
ηm
Pbr = ηr × Psr (23)

4.1.5. Power Consumption FMU

Suppose auxiliary load Paux is the power used for other purposes such as radio or heater, not the
power used for tractive force through vehicle wheels in an EV. Then, this model is for a formula that
calculates the total power consumed in the EV. If the tractive force is a positive number, then the total
power consumed in the EV is the sum of the auxiliary power consumption and the power consumed
in the motoring mode for wheel rotation. If the tractive force is a negative number, then it is the sum
of the auxiliary power consumption and the power generated in the recuperation mode because the
vehicle is in a braking situation.

Pbm (t) + Paux , Ft (t) > 0

(
Pbc (t) = (24)
Pbr (t) + Paux , Ft (t) < 0
Sensors 2020, 20, 252 17 of 27

4.1.6. Battery Management System (BMS) FMU

The battery management system model consists of a battery pack and a module that measures the
SOC and SOH of the battery [43]. A battery current IB can be obtained through the battery’s no-load
voltage EB0 , internal resistance RB0 , and power consumption calculated by (24). If this is integrated,
the total consumed electric charge Q can be obtained. Furthermore, the battery cell capacity C may
change depending on the ambient temperature, as revealed by the following formula [34]. Through
this, the current SOC value of the battery can be finally be obtained.
s
!2
EB0 EB0 Pbc
IB ( t ) = − − (25)
2 × RBi 2 × RBi RBi
Z t
Q(t) = IB dt (26)
0
  
C = C0 ∗ 1 + alphaC × Tcurrent − Tre f (27)

C − Q(t)
SOC(t) = (28)
C
In addition, the SOH of the battery can be calculated as follows:

SOH = 1 − SOS = SOHc × SOHz (29)

For a new battery, SOHc = 1 and SOHz = 1 therefore, also SOH = 1. In this paper, it is assumed
that the value of SOH is 1.

4.2. Energy Conversion Chain of EV Co-Simulation Model

The EV simulation model has different energy conversion chains because of different energy flows
depending on the motoring and the regenerative mode [37]. In Figure 12, the energy required to move
the EV for 1 s is the same as Pt . First, in the case of motoring mode, the energy of each part is calculated
as follows:
PEM_OUT
PEM_IN = (30)
ηm
Pt
PEM_OUT = (Pt > 0) (31)
ηg
Sensors 2019, 19, x 17 of 26

Figure 12.
Figure 12. Energy
Energy conversion
conversion chain
chain of
of each
each modes;
modes; (a)
(a) motoring
motoring mode,
mode, (b)
(b) regenerative
regenerative braking mode.
braking mode.

4.3. Distributed CPS Simulation Model Considering Sensor Information

Moreover, the EV model can be simulated with the following structure in order to use real-time
sensing data. In other words, the F-DCS simulates the EV model and the sensor node collects sensed
temperature data and saves them in a file, as shown in Figure 13. The data values saved in the file
were applied to the FMU in real-time through a cloud service in order to perform the simulation.
Sensors 2020, 20, 252 18 of 27

Next, in the regenerative braking mode, the energy of each part is calculated as follows:

PEM_IN = PEM_OUT × ηr (32)

Figure 12. Energy conversion chainPofEM_OUT = Pt(a)

each modes; η g (Pt < mode,
× motoring 0) (b) regenerative braking mode. (33)

4.3. Distributed CPS

4.3. Distributed CPS Simulation
Simulation Model
Model Considering
Considering Sensor
Sensor Information
Information
Moreover,
Moreover, the
the EV
EV model
model can
can be
be simulated
simulated with
with the
the following
following structure
structure in
in order
order to
to use
use real-time
real-time
sensing
sensing data. In other
data. In other words,
words, the
the F-DCS
F-DCS simulates
simulates the
the EV
EV model
model and
and the
the sensor
sensor node
node collects
collects sensed
sensed
temperature
temperature data and saves them in a file, as shown in Figure 13. The data values saved in the file
data and saves them in a file, as shown in Figure 13. The data values saved in the file
were
were applied
applied to
to the
the FMU
FMU in
in real-time
real-time through
through aa cloud
cloud service
servicein
inorder
orderto toperform
performthe
thesimulation.
simulation.

Figure 13. Simulation model including sensor information.

5. Simulation Experiments
5. Simulation Experiments andand Analysis
Analysis
We
We experimentally
experimentallyvalidated
validatedthe
theEVEVsimulation
simulation model
model with thethe
with proposed algorithms
proposed to verify
algorithms the
to verify
results by employing driving cycle data. In the test, the new European driving cycle (NEDC),
the results by employing driving cycle data. In the test, the new European driving cycle (NEDC), where V max
equals
where 𝑉 120 km/h and V equals
equals 120 km/h and 𝑉
avg 32 km/h, was adopted as a standard driving cycle. The NEDC,
equals 32 km/h, was adopted as a standard driving cycle. The one of
the standard driving cycles that includes specifications of urban, rural, and highway driving conditions,
NEDC, one of the standard driving cycles that includes specifications of urban, rural, and highway
is composed of four consecutive ECE-15 urban driving cycles (UDC) and one extra-urban driving cycle
driving conditions, is composed of four consecutive ECE-15 urban driving cycles (UDC) and one
(EUDC). Although the NEDC was originally designed to assess gasoline-driven vehicles, it is now used
extra-urban driving cycle (EUDC). Although the NEDC was originally designed to assess gasoline-
to evaluate and measure the power consumption and driving range of diesel-driven, hybrid and electric
driven vehicles, it is now used to evaluate and measure the power consumption and driving range
vehicles. The NEDC reflects the necessary conditions for evaluating the EV system as it provides
of diesel-driven, hybrid and electric vehicles. The NEDC reflects the necessary conditions for
average velocity, certain ratio of braking time, and acceleration similar to those in actual driving.
evaluating the EV system as it provides average velocity, certain ratio of braking time, and
Therefore, it provides cases that closely resemble real-world situations in EV model simulations.
acceleration similar to those in actual driving. Therefore, it provides cases that closely resemble real-
We used FMU SDK [44] and OpenDDS [45] for implementing F-DCS. For these experiments, we
world situations in EV model simulations. We used FMU SDK [44] and OpenDDS [45] for
used six personal computers and an Ethernet switch. Table 1 shows the EV simulation parameter
implementing F-DCS. For these experiments, we used six personal computers and an Ethernet switch.
setting values.
Table 1 shows the EV simulation parameter setting values.
Sensors 2019, 19, x 18 of 26

Sensors 2020, 20, 252 Table 1. EV simulation parameter setting values. 19 of 27

Description Value (Default)

m Table Mass of the vehicle
1. EV simulation (kg) setting values.
parameter 1000
𝑟 Wheel radius (m) 0.2736
Description Value (Default)
g Gravity acceleration (m/s2) 9.81
m𝜌 MassAirof the vehicle (kg)
density 1000
1.2
rw Wheel radius (m) 2 0.2736
g
A Front area of vehicle (m2 ) 2.36
9.81
Gravity acceleration (m/s )
ρ𝛼 Angle of driving surface (rad)
Air density 01.2
𝜇
A Rolling
Frontresistance coefficient
area of vehicle (m2 ) 0.015
2.36
αCd Angle of driving
Aerodynamic dragsurface (rad)
coefficient 0.30
µ𝜂rr Rolling resistance
Gearbox efficiencycoefficient 0.015
0.98
Cd Aerodynamic drag coefficient 0.3
G Gearbox ratio 8.59
ηg Gearbox efficiency 0.98
GC0 Initial capacity
Gearbox ratio(C) 720000
8.59
CR0bi Internal
Initialresistance
capacity (C)(Ω) 0.008
720000
REbib0 Internal resistance
Open-circuit voltage(Ω) (V) 0.008
53.6
E
alphaC
b0 Open-circuit voltage (V)
Linear temperature coefficient of capacity (K -1 ) 53.6
0.03 (0)
alphaC Linear temperature coefficient of capacity (K−1 ) 0.03 (0)
Tref Reference temperature 20
Tref Reference temperature 20
SOHc c
SOH Capacity
Capacitystate
state of health
of health 11
SOHz z
SOH Impedance
Impedancestatestate of health
health 11

Several studies involving conventional EV simulations first modeled the subsystems of an EV

Several studies involving conventional EV simulations first modeled the subsystems of an EV
mathematically, after which they implemented and simulated the models using MATLAB/Simulink.
mathematically, after which they implemented and simulated the models using MATLAB/Simulink.
This approach allowed the power system and driving distance of the EV to be predicted more
This approach allowed the power system and driving distance of the EV to be predicted more
accurately [36–37,41–42]. In our study, we implemented the subsystems of an EV based on the
accurately [36,37,41,42]. In our study, we implemented the subsystems of an EV based on the
mathematical modeling method of existing studies. Therefore, the entire model shown in Figure 11
mathematical modeling method of existing studies. Therefore, the entire model shown in Figure 11 was
was implemented and simulated with MATLAB/Simulink first. Then, the results were compared
implemented and simulated with MATLAB/Simulink first. Then, the results were compared with the
with the simulation results that were derived by applying the method proposed in this paper, thereby
simulation results that were derived by applying the method proposed in this paper, thereby evaluating
evaluating the accuracy of the proposed simulation.
the accuracy of the proposed simulation.
Figure 14 shows the results of EV model simulation for 1180 s considering the driving cycle. The
Figure 14 shows the results of EV model simulation for 1180 s considering the driving cycle.
results presented in Figure 14 appear to agree well with the results of a simulation based on
The results presented in Figure 14 appear to agree well with the results of a simulation based on
MATLAB/Simulink [46]. This is because it adopts the driving cycle data where most of the average
MATLAB/Simulink [46]. This is because it adopts the driving cycle data where most of the average
velocities and key values are the same.
velocities and key values are the same.

(a) (b)

Figure 14. Cont.

Sensors 2020, 20, 252 20 of 27
Sensors 2019, 19, x 19 of 26

(c) (d)

Figure 14. Results

Figure 14. Results of
of EV
EV simulation using driving
simulation using driving cycle
cycle data;
data; (a)
(a) Velocity
Velocity (m/s)
(m/s) for time t,
for time t, (b)
(b) tractive
tractive
effort or force Ft (N), (c) power consumption from the battery
effort or force Ft (N), (c) power consumption from the battery Pbc P (W), (d) SOC of the battery.
bc (W), (d) SOC of the battery.

Furthermore, to quantitatively measure the simulation accuracy of the proposed method, the mean
Furthermore, to quantitatively measure the simulation accuracy of the proposed method, the
absolute percentage error (MAPE) value of the simulation values was determined as follows:
mean absolute percentage error (MAPE) value of the simulation values was determined as follows:
n 
11 X x̂𝑥i −−xi𝑥

𝑀𝐴𝑃𝐸(%) =
MAPE ( % ) =  x𝑥i  ××100
 100 (34)
(34)
n𝑛
i=1

wherennisisthe
where the number
number of simulation
of simulation datadata values,
values, 𝑥 ith
x̂i is the is simulation
the ith simulation
result whenresult whenalgorithm
a certain a certain
algorithm
was was xused,
used, and i is the ith 𝑥simulation
and is the ithvalue
simulation
when value when MATLAB/Simulink
MATLAB/Simulink was used. For wastheused. For the
simulation
simulation
data, data,measured
the values the values in measured in 1 from
1 s increments s increments
0 to 1180from
s were 0 toused1180to sevaluate
were used to evaluate the
the performance of
performance of the proposed algorithm. Because the MAPE can be applied
the proposed algorithm. Because the MAPE can be applied only when the value of xi is not 0, as shown only when the value of
𝑥 the
in is not 0, asequation,
above shown inwe themeasured
above equation, we measured
the simulation results the
Ft , Psimulation
bc , and SOC results Ft, Pnot
that did bc, and
haveSOC that
a value
did0 not
of have
in the a value of
simulation 0 in the simulation section.
section.
The simulation
The simulationvaluesvaluesofofFMUFMUSDK SDKand andF-DCS
F-DCS withwith RRA,
RRA, which
which areare provided
provided in Table
in Table 2,less
2, are are
less than
than 5%. 5%. Furthermore,
Furthermore, the two
the two algorithms
algorithms yielded
yielded the same
the same MAPE MAPE values,
values, and and
this this is because
is because the
the F-DCS with the RRA algorithm was implemented on the basis of FMU
F-DCS with the RRA algorithm was implemented on the basis of FMU SDK and the repeating sections SDK and the repeating
sections
were were removed
removed while maintaining
while maintaining the same the same simulation
simulation accuracy accuracy
of FMU SDK. of FMUThe SDK.
MAPE The MAPE
values of
values of DACCOSIM
DACCOSIM were smaller were smaller
than thosethan those and
of F-DCS, of F-DCS,
it wasand
shown it wasthatshown
accuratethatsimulation
accurate simulation
is feasible.
is feasible.
Table 2. MAPE (%) of simulation values.
Table 2. MAPE (%) of simulation values.
Simulation Value DACCOSIM FMU SDK F-DCS with RRA
Simulation
Ft Value DACCOSIM
6.7 FMU4.8SDK F-DCS with4.8RRA
Pbc Ft 11.76.7 4.8
1.25 4.8 1.25
SOCPbc 0.075
11.7 0.66
1.25 1.250.66
SOC 0.075 0.66 0.66
Next, each algorithm was evaluated by using the following equation to measure the zero crossing
pointNext, each algorithm
error (ZCPE), was evaluated
which indicates the errorby using
in the the following
simulation time forequation to m
ZCP. Here, measure the zero
is the frequency
crossing
of point
ZCP that errorin(ZCPE),
occurs which indicates
all simulation sections. the error in the simulation time for ZCP. Here, m is the
frequency of ZCP that occurs in all simulation Pm sections.
i=1 Tn+1 − Tn
Avg ZCPE = ∑ 𝑇 − 𝑇 ( f or all ZCP) (35)
𝐴𝑣𝑔 𝑍𝐶𝑃𝐸 = m (𝑓𝑜𝑟 𝑎𝑙𝑙 𝑍𝐶𝑃) (35)
𝑚
The
The simulation
simulation results
results inin Table
Table 33 indicate
indicate that
that the
the average
average ZCPE
ZCPE of of DACCOSIM
DACCOSIM and and FMU
FMU SDKSDK
is
is the same as the step size. This is because in the case of a fixed step size, the time error for ZCP is
the same as the step size. This is because in the case of a fixed step size, the time error for ZCP is
proportional
proportional to to the
the step
step size,
size, as
as shown
shown in in Figure
Figure 9. However, in
9. However, in the
the case
case of
of F-DCS
F-DCS with
with RRA,
RRA, itit is
is
confirmed
confirmed that
that the
the simulation
simulation can can be
be performed
performed forfor ZCP
ZCP with
with aa value
value smaller
smaller than
than the
the time threshold
time threshold
value by using the bisection algorithm.
value by using the bisection algorithm.
Sensors 2020, 20, 252 21 of 27

Table 3. Average zero crossing point error (ZCPE) of simulation values.

DACCOSIM/FMU SDK F-DCS with RRA

Fixed Step Size Adaptive Step Size
Simulation Value
1 0.01 0.0001 (T thres = 0.0001)

Average ZCPE
Ft 1 0.01 0.0001 0.000061
Pbc 1 0.01 0.0001 0.000061

The simulation results using the NEDC driving cycle data, where the validation procedure with
the measurement of the SOC value changing with simulation duration until the value of 0.1 was applied,
is shown in Figure 15. In the validation with the application of the F-DCS with RRA for effectively
reducing the overall simulation time, the calculated amount of battery SOC consumed was confirmed
to agree with the results of general simulation cases (MATLAB/Simulink, DACOSSIM, FMU SDK
default co-simulation algorithm [47]). An examination of the simulation result in Figure 15a reveals
that the resulting battery SOC was almost the same as that of other conventional simulation tools.
This result verifies the accuracy of the simulation measurement of the battery SOC of the proposed
F-DCS. The simulation running time was evaluated in comparison with other FMI-based simulation
tools. As shown in Figure 15b, the simulation time required by F-DCS with RRA was more than 30%
less than that required by other methods. This means that if a driving cycle pattern with repetitive
cycles exists in the simulation, it is safe to perform the simulation by proceeding with the rollback
using that pattern. In addition, comparing the values of the stored rollback pattern information with
the values generated during the progress of the simulation, looking for mapped values and applying
them can improve efficiency in terms of simulation time.
Figure 16a shows the results of the performance of the number of DDS data message transmissions
and in an environment that considers distributed co-simulation. As data are transmitted in the
pub/sub format, as described by the distributed CPS simulation frameworks in Section 4, the number
of messages generated by a simulation with a connected structure, as shown in Figure 11, can be
calculated by the following formula.

Total data message = Simulation loop numalgorithm × numlink (36)

Total control message = Rollback numalgorithm × (numnode − 1) (37)

Figure 16b shows that the result of DDS control messages decreased by more than 45%. With the
FMU SDK algorithm, the occurrence of message generation increases significantly as all nodes that
subscribe to a rollback receive a message containing the rollback information when the rollback occurs.
By applying the F-DCS with RRA algorithm, the time for exchanging messages with the progress of
simulation appears to be reduced by more than 30%, according to the reduction of the number of
rollbacks using the patterns.
Finally, the result of applying the sensor to the simulation and linking it is demonstrated in
Figure 17. The simulation test was conducted using the test model shown in Figure 11 and a wide
range of ambient temperatures was used for the information generated from the sensor. The simulation
test was performed by using the measurement environments of 20 ◦ C and 25 ◦ C for the temperature
measured with the sensor. In the results of battery SOC, the sensor information was not reflected
without the F-DCS; nevertheless, the proposed algorithm could obtain simulation results by reflecting
the sensor information value.
based simulation tools. As shown in Figure 15b, the simulation time required by F-DCS with RRA
was more than 30% less than that required by other methods. This means that if a driving cycle
pattern with repetitive cycles exists in the simulation, it is safe to perform the simulation by
proceeding with the rollback using that pattern. In addition, comparing the values of the stored
rollback pattern
Sensors 2020, 20, 252 information with the values generated during the progress of the simulation, looking
22 of 27
for mapped values and applying them can improve efficiency in terms of simulation time.

Sensors 2019, 19, x 21 of 26

Figure 16a shows the results of the performance of the number of DDS data message
transmissions and in an environment that considers distributed co-simulation. As data are
transmitted in the pub/sub format, as described(a) by the distributed CPS simulation frameworks in
Section 4, the number of messages generated by a simulation with a connected structure, as shown
in Figure 11, can be calculated by the following formula.
𝑇𝑜𝑡𝑎𝑙 𝑑𝑎𝑡𝑎 𝑚𝑒𝑠𝑠𝑎𝑔𝑒 = 𝑆𝑖𝑚𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑙𝑜𝑜𝑝 𝑛𝑢𝑚 × 𝑛𝑢𝑚 (36)

𝑇𝑜𝑡𝑎𝑙 𝑐𝑜𝑛𝑡𝑟𝑜𝑙 𝑚𝑒𝑠𝑠𝑎𝑔𝑒 = 𝑅𝑜𝑙𝑙𝑏𝑎𝑐𝑘 𝑛𝑢𝑚 × (𝑛𝑢𝑚 − 1) (37)

Figure 16b shows that the result of DDS control messages decreased by more than 45%. With the
FMU SDK algorithm, the occurrence of message generation increases significantly as all nodes that
subscribe to a rollback receive a message containing the rollback information when the rollback
occurs. By applying the F-DCS with RRA algorithm, the time for exchanging messages with the
progress of simulation appears to be reduced by more than 30%, according to the reduction of the
(b)
number of rollbacks using the patterns.
Figure 15. Results
Results of
ofmodel
modelvalidation;
validation;(a)(a) result
result of battery
of battery SOC,
SOC, (b) average
(b) average totaltotal simulation
simulation time
time (sec).
(sec).

Table 3. Average zero crossing point error (ZCPE) of simulation values.

DACCOSIM / FMU SDK F-DCS with RRA

Simulation Fixed step size Adaptive step size
value 1 0.01 0.0001 (Tthres = 0.0001)
Average ZCPE
Ft 1 0.01 0.0001 0.000061
Pbc 1 0.01 0.0001 0.000061

(a) (b)

Figure 16.
Figure Results of
16. Results of data
data distribution
distribution service
service (DDS)
(DDS) validation;
validation; (a)
(a) DDS
DDS data
data message,
message, (b)
(b) DDS
DDS
control message.
control message.

Finally, the result of applying the sensor to the simulation and linking it is demonstrated in
Figure 17. The simulation test was conducted using the test model shown in Figure 11 and a wide
range of ambient temperatures was used for the information generated from the sensor. The
simulation test was performed by using the measurement environments of 20 °C and 25 °C for the
temperature measured with the sensor. In the results of battery SOC, the sensor information was not
reflected without the F-DCS; nevertheless, the proposed algorithm could obtain simulation results by
reflecting the sensor information value.
Sensors 2020, 20, 252 23 of 27
Sensors 2019, 19, x 22 of 26

Figure
Figure 17. Test results
17. Test results on
on distributed
distributed co-simulation
co-simulation linked
linked with
with sensor
sensor information.
information.

6. Conclusions
6. Conclusions
Co-simulation has significant potential in terms of improving the accuracy and efficiency of
Co-simulation has significant potential in terms of improving the accuracy and efficiency of CPS
CPS simulations. However, because modeling and simulation differ between industrial areas, it is
simulations. However, because modeling and simulation differ between industrial areas, it is
reasonable to apply appropriate techniques to each area. This paper presents a DDS middleware-based
reasonable to apply appropriate techniques to each area. This paper presents a DDS middleware-
distributed co-simulation framework and proposes a method of determining the step size that can
based distributed co-simulation framework and proposes a method of determining the step size that
be commonly applied in all FMUs. When a repetitive pattern was observed in the frequently used
can be commonly applied in all FMUs. When a repetitive pattern was observed in the frequently used
driving cycle in EV simulation, simulations were performed by predicting the pattern. Based on
driving cycle in EV simulation, simulations were performed by predicting the pattern. Based on this
this method, the proposed algorithms facilitated more efficient simulations while still maintaining
method, the proposed algorithms facilitated more efficient simulations while still maintaining
conventional accuracy.
conventional accuracy.
The proposed algorithms were applied to the EV simulation model and the results confirm that
The proposed algorithms were applied to the EV simulation model and the results confirm that
the proposed algorithms improved the performance by reducing the simulation time by over 30%
the proposed algorithms improved the performance by reducing the simulation time by over 30%
compared to conventional algorithms. Furthermore, real-time data were received from the sensor
compared to conventional algorithms. Furthermore, real-time data were received from the sensor
node and applied to the EV simulation model. The results confirm that the real environment could be
node and applied to the EV simulation model. The results confirm that the real environment could
considered in the simulations. If an algorithm applicable to models of other areas in addition to the EV
be considered in the simulations. If an algorithm applicable to models of other areas in addition to
model in this study were developed, it would become the foundation of digital twin research.
the EV model in this study were developed, it would become the foundation of digital twin research.
Author Contributions: Conceptualization,
Author Contributions: Conceptualization, S.H.;
S.J.H.;formal analysis,
formal S.H.;
analysis, investigation,
S.J.H.; S.H.;S.J.H.;
investigation, methodology, S.H.,
methodology,
D.L.; writing—review and editing, D.L.; supervision, I.J., W.K.; funding acquisition, W.K. All authors have read
S.J.H., D.S.L.; writing—review and editing, D.S.L.; supervision, I.W.J., W.T.K; funding acquisition, W.T.K
and agreed to the published version of the manuscript.
Funding: This
Funding: This research
research was
was supported
supported by
by the
the Institute
Institute Information
Information & & communications
communications Technology
Technology Promotion
Promotion
(IITP) (NO.2019-0-01347 and No. 2018-0-01456)
(IITP) (NO.2019-0-01347 and No. 2018-0-01456).
Conflicts
Conflicts of Interest: The
of Interest: Theauthors
authorsdeclare
declarenono
conflict ofof
conflict interest.
interest.

Nomenclature
Nomenclature
Abbreviations
Abbreviations
CPS cyber physical system
CPS cyber physical system
EV electric vehicle
EV electric vehicle
ICT information and communication technology
ICT information and communication technology
CPESs cyber physical energy systems
CPESs wide areacyber
WAN
physical energy systems
network
WAN
ZC wide
zero crossing area network
ZC zero crossing
ZCP zero crossing point
ZCPE zero crossing point error
FMI Functional mock-up interface
Sensors 2020, 20, 252 24 of 27

ZCP zero crossing point

ZCPE zero crossing point error
FMI Functional mock-up interface
API application programming interface
FMU functional mock-up unit
IP intellectual property
MA master algorithm
DDS data distribution service
OMG object management group
IoT internet of things
RPM revolution per minute
BMS battery management system
MAPE mean absolute percentage error
NEDC new European driving cycle
UDE urban driving cycle
SOC State of charge
SOH State of health
SOS State of sickness
Variables
v Velocity of the vehicle, m/s
A Frontal area of the vehicle, m2
a Acceleration of the vehicle, m/s2
Ft Traction force of the vehicle, N
Frr Rolling resistance force of the wheels, N
Fad Aero dynamic drag, N
Fhc Hill climbing force, N
Fla Force required to give linear acceleration, N
Fwa Force required to give angular acceleration to the rotating motor, N
µrr Coefficient of rolling resistance
m Vehicle mass, kg
g Gravity acceleration, m/s2
ρ Density of the air
Cd Aerodynamic drag coefficient
α Angle of the driving surface, rad
rw Wheel radius, m
Pt Traction power, W
wW Angular velocity of the wheels, rad/s
SW Angular velocity of the wheels, rpm
Tt Traction torque, Nm
Ts Shaft torque of electric machine (motoring mode), Nm
Tsr Shaft torque of electric machine (regenerative braking mode), Nm
ηg Gearbox efficiency g
G Gear ratio of differential
Ss Shaft angular velocity of electric machine (motoring mode), rpm
Ssr Shaft angular velocity of electric machine (regenerative breaking mode), rpm
Ps Shaft power of electric machine (motoring mode), W
Psr Shaft power of electric machine (regenerative breaking mode), W
ηm Efficiency of power consumed (motoring mode)
ηr Efficiency of power generated (regenerative breaking mode)
Pbm Power consumed by electric machine (motoring mode), W
Pbr Power generated by electric machine (regenerative breaking mode), W
Paux Power consumed by auxiliary loads, W
Pbc Total power consumed in the EV, W
Sensors 2020, 20, 252 25 of 27

Eb0 Open-circuit voltage of the battery, V

RBi Internal Resistance of the battery, Ω
Q Total charge of the battery
IB Battery current,
C Battery capacity, Ah
alphaC Linear temperature coefficient of the capacity C, K−1
Tref Reference temperature, K

References
1. Lee, E.A. Cyber physical systems: Design challenges. In Proceedings of the 11th IEEE Symposium on Object
Oriented Real-Time Distributed Computing (ISORC), Orlando, FL, USA, 5–7 May 2008; IEEE Computer
Society: Washington, DC, USA, 2008; pp. 363–369.
2. Derler, P.; Lee, E.A.; Vincentelli, A.S. Modeling cyber–physical systems. In Proceedings of the 18th IEEE
International Conference and Workshops on Engineering of Computer-Based Systems, Las Vegas, NV, USA,
27–29 April 2011; Volume 100, pp. 13–28.
3. Eidson, J.C.; Lee, E.A.; Matic, S.; Seshia, S.A.; Zou, J. Distributed real-time software for cyber–physical
systems. Proc. IEEE 2011, 100, 45–59. [CrossRef]
4. National Science Foundation of the United States. Cyber-Physical System (CPS) Program Solicitation.
Available online: http://www.nsf.gov/pubs/2010/nsf10515/nsf10515.htm (accessed on 18 August 2017).
5. Guan, X.; Yang, B.; Chen, C.; Dai, W.; Wang, Y. A comprehensive overview of cyber-physical systems: From
perspective of feedback system. IEEE/CAA J. Autom. Sin. 2016, 3, 1–14.
6. Sharma, A.; Rathee, G.; Kumar, R.; Saini, H.; Vijaykumar, V.; Nam, Y.; Chilamkurti, N. A Secure, Energy-and
SLA-Efficient (SESE) E-Healthcare Framework for Quickest Data Transmission Using Cyber-Physical System.
Sensors 2019, 19, 2119. [CrossRef] [PubMed]
7. Jiang, Y.; Yin, S.; Kaynak, O. Data-driven monitoring and safety control of industrial cyber-physical systems:
Basics and beyond. IEEE Access 2018, 6, 47374–47384. [CrossRef]
8. Roy, D.; Zhang, L.; Chang, W.; Mitter, S.K.; Chakraborty, S. Semantics-preserving cosynthesis of cyber-physical
systems. Proc. IEEE 2017, 106, 171–200. [CrossRef]
9. Yu, Z.; Zhou, L.; Ma, Z.; El-Meligy, M.A. Trustworthiness modeling and analysis of cyber-physical
manufacturing systems. IEEE Access 2017, 5, 26076–26085. [CrossRef]
10. Emadi, A.; Lee, Y.J.; Rajashekara, K. Power electronics and motor drives in electric, hybrid electric, and plug-in
hybrid electric vehicles. IEEE Trans. Ind. Electron. 2008, 55, 2237–2245. [CrossRef]
11. Iora, P.; Tribioli, L. Effect of ambient temperature on electric vehicles’ energy consumption and range: Model
definition and sensitivity analysis based on nissan leaf data. World Electr. Veh. J. 2019, 10, 2. [CrossRef]
12. Morris, T.H.; Srivastava, A.K.; Reaves, B.; Pavurapu, K.; Abdelwahed, S.; Vaughn, R.; Wesley, S.; Dandass, Y.
Engineering future cyber-physical energy systems: Challenges, research needs, and roadmap. In Proceedings
of the 41st North. American power symposium, Starkville, MS, USA, 4–6 October 2009; pp. 1–6.
13. Tehrani, K.; Maurice, O. A cyber physical energy system design (CPESD) for electric vehicle applications.
In Proceedings of the 12th System of Systems Engineering Conference (SoSE), Waikoloa, HI, USA, 18–21
June 2017; pp. 1–6.
14. Ge, Y.; Dong, Y.; Zhao, H. A cyber-physical energy system architecture for electric vehicles charging
application. In Proceedings of the 12th International Conference on Quality Software, Xi’an, China, 27–29
August 2012; pp. 246–250.
15. Kim, J.E.; Mosse, D. Generic framework for design, modeling and simulation of cyber physical systems.
ACM SIGBED Rev. 2008, 5, 1. [CrossRef]
16. Gao, D.W.; Mi, C.; Emadi, A. Modeling and simulation of electric and hybrid vehicles. Proc. IEEE 2007, 95,
729–745. [CrossRef]
17. Muta, K.; Yamazaki, M.; Tokieda, J. Development of New-Generation Hybrid System THS II-Drastic Improvement of
Power Performance and Fuel Economy; No. 2004-01-0064; SAE Technical Paper; SAE International: Warrendale,
PA, USA, 2004.
Sensors 2020, 20, 252 26 of 27

18. Maia, R.; Silva, M.; Araújo, R.; Nunes, U. Electric vehicle simulator for energy consumption studies in electric
mobility systems. In Proceedings of the 2011 IEEE Forum on Integrated and Sustainable Transportation
Systems, Vienna, Austria, 29 June–1 July 2011; pp. 227–232.
19. Mahseredjian, J.; Dennetière, S.; Dubé, L.; Khodabakhchian, B.; Gérin-Lajoie, L. On a new approach for the
simulation of transients in power systems. Electr. Power Syst. Res. 2007, 77, 1514–1520. [CrossRef]
20. Gomes, C.; Thule, C.; Broman, D.; Larsen, P.G.; Vangheluwe, H. Co-simulation: A survey. ACM Comput. Surv.
2018, 51, 49. [CrossRef]
21. Faruque, M.O.; Dinavahi, V.; Steurer, M.; Monti, A.; Strunz, K.; Martinez, J.A.; Chang, G.W.; Jatskevich, J.;
Iravani, R.; Davoudi, A. Interfacing issues in multi-domain simulation tools. IEEE Trans. Power Deliv. 2011,
27, 439–448. [CrossRef]
22. Zhao, H.; Wang, B.; Zhang, G.; Feng, Y. Energy Saving Design and Control of Steering Wheel System of
Steering by Wire Vehicle. IEEE Access 2019, 7, 44307–44316. [CrossRef]
23. Chen, Y.; Li, X.; Wiet, C.; Wang, J. Energy management and driving strategy for in-wheel motor electric
ground vehicles with terrain profile preview. IEEE Trans. Ind. Inform. 2013, 10, 1938–1947. [CrossRef]
24. Zhang, Z.; Eyisi, E.; Koutsoukos, X.; Porter, J.; Karsai, G.; Sztipanovits, J. A co-simulation framework for
design of time-triggered automotive cyber physical systems. Simul. Model. Pract. Theory 2014, 43, 16–33.
[CrossRef]
25. Li, W.; Zhu, X.Y.; Ju, J. Hierarchical Braking Torque Control of In-Wheel-Motor-Driven Electric Vehicles Over
CAN. IEEE Access 2018, 6, 65189–65198. [CrossRef]
26. Yu, W.; Xue, Y.; Luo, J.; Ni, M.; Tong, H.; Huang, T. An UHV grid security and stability defense system:
Considering the risk of power system communication. IEEE Trans. Smart Grid 2015, 7, 491–500. [CrossRef]
27. Blochwitz, T. Functional Mock-Up Interface for Model Exchange and Co-Simulation. Available online:
https://www.fmi-Standard.org/downloads/ (accessed on 25 July 2014).
28. Hyundai Mobis. Available online: https://www.mobis.co.kr/main/index.do (accessed on 29 November 2019).
29. Broman, D.; Brooks, C.; Greenberg, L.; Lee, E.A.; Masin, M.; Tripakis, S.; Wetter, M. Determinate composition
of FMUs for co-simulation. In Proceedings of the Eleventh ACM International Conference on Embedded
Software, Montreal, QC, Canada, 29 September–4 October 2013; p. 2.
30. Cremona, F.; Lohstroh, M.; Broman, D.; Di Natale, M.; Lee, E.A.; Tripakis, S. Step revision in hybrid
co-simulation with FMI. In Proceedings of the 2016 ACM/IEEE International Conference on Formal Methods
and Models for System Design (MEMOCODE), Kanpur, India, 18–20 November 2016; pp. 173–183.
31. Galtier, V.; Vialle, S.; Dad, C.; Tavella, J.P.; Lam-Yee-Mui, J.P.; Plessis, G. FMI-based distributed multi-simulation
with DACCOSIM. In Proceedings of the Symposium on Theory of Modeling & Simulation: DEVS Integrative
M&S Symposium, San Diego, CA, USA, 7–10 April 2015; Society for Computer Simulation International: San
Diego, CA, USA, 2015; pp. 39–46.
32. Lacoursière, C.; Härdin, T. FMI Go! A simulation runtime environment with a client server architecture over
multiple protocols. In Proceedings of the 12th International Modelica Conference, Prague, Czech Republic,
15–7 May 2017; Linköping University Electronic Press: Linköping, Sweden, 2017; pp. 653–662.
33. Tijero, H.P.; Gutierrez, J.J. On the schedulability of a data-centric real-time distribution middleware.
Comput. Stand. Interfaces 2012, 34, 203–211. [CrossRef]
34. Einhorn, M.; Conte, F.V.; Kral, C.; Niklas, C.; Popp, H.; Fleig, J. A modelica library for simulation of electric
energy storages. In Proceedings of the 8th International Modelica Conference, Dresden, Germany, 20–22
March 2011; Linköping University Electronic Press: Linköping, Sweden, 2011; pp. 436–445.
35. Chen, M.; Rincon-Mora, G.A. Accurate electrical battery model capable of predicting runtime and IV
performance. IEEE Trans. Energy Convers. 2006, 21, 504–511. [CrossRef]
36. Bhatt, A. Planning and application of Electric Vehicle with MATLAB® /Simulink® . In Proceedings of the 2016
IEEE International Conference on Power Electronics, Drives and Energy Systems (PEDES), Trivandrum, India,
14–17 December 2016; pp. 1–6.
37. Larminie, J.; Lowry, J. Electric Vehicle Technology Explained; John Wiley & Sons: Hoboken, NJ, USA, 2012; ISBN
978-1-119-94273-3.
38. Soylu, S. (Ed.) Electric Vehicles: Modelling and Simulations; BoD–Books on Demand: Norderstedt, Germany,
2011; ISBN 978-953-307-447-1.
39. Niederjohn, R.J.; Stick, P.P. A computer interface for efficient zero-crossing interval measurement.
IEEE Trans. Comput. 1975, 100, 329–331. [CrossRef]
Sensors 2020, 20, 252 27 of 27

40. Cellier, F.E.; Kofman, E. Continuous System Simulation; Springer Science & Business Media: Berlin, Germany,
2006; ISBN 978-0-387-26102-7.
41. Pejovic, P.; Kolar, J.W.; Nishida, Y. Bidirectional AC DC Converter for Regenerative Braking. Electronics 2012,
16, 3–8. [CrossRef]
42. Karotiya, R.M.; Gaidhane, M.A. Regenerative Braking Method Used in Converter for Traction Application.
2015. Available online: https://pdfs.semanticscholar.org/84ca/bf9fb0fd498dcaf455e524c4f5993a4511c7.pdf
(accessed on 6 September 2015).
43. Cheng, K.W.E.; Divakar, B.P.; Wu, H.; Ding, K.; Ho, H.F. Battery-management system (BMS) and SOC
development for electrical vehicles. IEEE Trans. Veh. Technol. 2010, 60, 76–88. [CrossRef]
44. Qtronic, FMU SDK. version 2.0.6. Available online: https://www.qtronic.de/doc.fmusdk.zip (accessed on 25
July 2014).
45. OpenDDS. version 3.13. Available online: https://opendds.org/ (accessed on 23 August 2018).
46. Butler, K.L.; Ehsani, M.; Kamath, P. A MATLAB-based modeling and simulation package for electric and
hybrid electric vehicle design. IEEE Trans. Veh. Technol. 1999, 48, 1770–1778. [CrossRef]
47. FMU SDK Co-Simulation Master Algorithm. Available online: https://github.com/schteppe/fmusdk.git
(accessed on 28 November 2012).

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).