1

Exploration of AI-Oriented Power System

Dynamic Simulations
Tannan Xiao, Ying Chen, Jianquan Wang, Shaowei Huang, Weilin Tong, Tirui He

 University, etc., and real-time simulators such as RTDS [11] by

Abstract—With the rapid development of Artificial Intelligence Manitoba, HYPERSIM [12] by OPAL-RT, ADPSS [13] by
(AI), it is foreseeable that the accuracy and efficiency of future CEPRI, etc. The commercial simulators are well tested in the
power system dynamic analysis will be greatly improved by the practical power system, which means they support a lot of
integration of dynamic simulators and AI. To explore the functions and are very reliable. However, the commercial
interaction mechanism of power system dynamic simulations and
AI, a general design of an AI-oriented power system dynamic simulators are usually designed and implemented years ago,
simulator is proposed, which consists of a high-performance which means their architecture might be old and the application
simulator with neural network supportability and flexible programming interfaces (APIs) may be stiff or even be
external and internal application programming interfaces (APIs). unavailable. On the other hand, the free and open-source
With the support of APIs, simulation-assisted AI and AI-assisted simulators are commonly not as functionally mature as the
simulation form a comprehensive interaction mechanism between commercial ones but are much more flexible. The source code
power system dynamic simulations and AI. A prototype of this
design is implemented and made public based on a highly efficient can be directly modified, so APIs can be developed as needed.
electromechanical simulator. Tests of this prototype are carried Research on artificial intelligence (AI) has achieved a growth
out under four scenarios including sample generation, AI-based spurt in the past few years. AI algorithms such as graph neural
stability prediction, data-driven dynamic component modeling, networks (GNN), reinforcement learning (RL), etc., have been
and AI-aided stability control, which prove the validity, flexibility, applied to a variety of power system studies such as
and efficiency of the design and implementation of the AI-oriented measurement enhancement [14], dynamic component modeling
simulator.
[15], parameter inference [16], optimization and control [17],
Index Terms—Power system dynamic simulation, artificial stability assessment [18], etc. AI models can learn and
intelligence, application programming interface, parallel approximate any functions with enough samples [19]. AI
computing. technology will be more and more important in the research
field of power systems, especially with the rapid development
of renewable generation and power electronics. The safe and
I. INTRODUCTION efficient operation of power systems is facing great challenges,
A. Motivations e.g., we may need to model new devices to analyze stability

P
although some devices’ operating mechanisms are still under
OWER Power system dynamic simulation is still the most
research, dimensionality reduction is needed to scale down the
reliable and widely used approach for power system
complexity, the significantly increased uncertainty of power
stability analysis [1]. Electric power companies and developers
systems requires fast and flexible stability analysis and control,
from all over the world have developed many dynamic
etc. AI-assisted power system analysis and control might be a
simulators including electromechanical simulators such as
solution to these challenges, or at least a mitigation measure.
PSASP [2] and PSD-BPA [3] by the China Electric Power
Currently, although some commercial software such as
Research Institute (CEPRI), PSS/E [3] by Siemens, DSATools
DIgSILENT PowerFactory and CloudPSS have supported AI
[4] by Powertech, DIgSILENT PowerFactory [5] by
applications to a certain extent, the relationship between power
DIgSILENT GmbH, Eurostag [6] by Tractebel,
system dynamic simulation and AI is still relatively fragmented.
PYPOWER-Dynamics [7] by Susanto, STEPS [8] by
The simulator usually works only as a data generator and
Shandong University, etc., electromagnetic simulators such as
provides limited prior knowledge, whereas the trained AI
PSCAD/EMTDC [9] by Manitoba, CloudPSS [10] by Tsinghua
model usually works independently as a black-box model with
poor interpretability and cannot be easily integrated into
_
____________________________________
This work was supported in part by the National Natural Science Foundation simulators. In one word, the simulator is not AI-friendly
of China under Grant 52107104, (Corresponding Author: Ying Chen). enough. In [20], the idea of a learning simulation engine that
Tannan Xiao, Ying Chen, Shaowei Huang, Tirui He are with the Department combines AI and simulation is proposed. Simulation-assisted
of Electrical Engineering, Tsinghua University, Beijing, 100084, China (Email:
[email protected], [email protected], AI and AI-assisted simulation mutually support each other.
[email protected], [email protected]). With the support of AI, the simulator can evolve autonomously
Jianquan Wang is with the College of Electrical Engineering, Zhejiang and become more accurate and efficient, which is also crucial to
University, Hangzhou, 310027, China (Email: [email protected]).
Weilin Tong is with Wuxi Power Supply Company of State Grid, Wuxi,
the realization of power system digital twins [21]. In [22], a
China (Email: [email protected]). similar and more comprehensive concept of simulation
intelligence is proposed. This can be the future of power system data-driven AI models can cooperate to achieve the task of data
dynamic simulators. enhancement [23], awareness enhancement [14], analysis
enhancement [24], decision-making enhancement [25], etc.,
B. Contributions
and may finally lead to the creation of a power system digital
Inspired by the learning simulation engine proposed in [20] twin.
and the simulation intelligence discussed in [22], we explore
the cooperation of AI and power system dynamic simulations in
this paper. The contributions are as follows.
1) A general design of an AI-oriented power system dynamic
simulator is proposed, which consists of a high-performance
simulator with neural network supportability and flexible
external and internal APIs. External API-based
Simulation-assisted AI and internal API-based AI-assisted
simulation form a comprehensive interaction mechanism
between power system dynamic simulations and AI.
Fig. 1. The overall architecture of the AI-oriented simulator.
2) A prototype of the proposed design is implemented based
on a self-developed power system electromechanical simulator.
In this section, the designs of the simulator and APIs, as well
The efficiency of the simulator is improved by traditional
as the interactions of the simulator and AI models are explained
approaches including sparsity techniques, parallel computing,
in detail.
and memory allocation optimization. External and internal
APIs are developed with Python and the source code is made A. Simulator
public on GitHub1. Power system dynamic simulators can be used to generate
3) Four typical cases of utilizing the developed simulator, i.e., massive scenarios and simulation results, i.e., physics-based
sample generation, GNN-based stability prediction, data-driven data. The general structure of a power system simulator is
dynamic component modeling, and RL-based stability control, displayed in Fig. 1, which consists of four parts, namely, model,
are illustrated to prove the validity, flexibility, and efficiency of parameter, solution, and function. Firstly, a dynamic model is
the proposed design and implementation. All four cases are needed for each power system component. It can be a
supported by at least one paper or the source code we published physics-based model, a data-driven model, or a
on GitHub. physics-data-integrated model. A model conversion function
C. Paper Organization for different models of different simulators is preferred.
Secondly, parameters of the selected model need to be
The remainder of the papers is as follows. Section II
measured or estimated, i.e., model calibration [26] is needed.
introduces the design of the AI-oriented power system dynamic
Thirdly, all the models with parameters are formulated together
simulator and discusses the interaction of dynamic simulations
in a group of high-dimensional equations. Power flow can be
and AI. The implementation details of a prototype simulator
solved with the Newton method to obtain the operation state.
based on the proposed design are illustrated in Section III. In
Power system dynamics are formulated with ordinary
section IV, the typical examples of the implemented simulator
differential equations (ODEs) in the electromagnetic simulation
are explained and tested. Conclusions are drawn in Section V.
and differential-algebraic equations (DAEs) in the
electromechanical simulation. They can both be solved with a
II. DESIGN OF AI-ORIENTED DYNAMIC SIMULATOR
numerical integration method and a linear solver [27]. Finally,
In Fig. 1, the overall architecture design of the AI-oriented simulation-based functions are realized based on the solutions
simulator is demonstrated. The idea is intuitive. In order to of power flow and power system dynamics.
support the interactions between the simulator and AI, a Here are two required features of the simulator to support the
reasonable choice is to develop AI-friendly APIs to bridge the integrations of simulations and AI models, i.e., rapid simulation
two. The simulator, APIs, AI models, and power system speed and neural network supportability.
operator form a bionic interaction mechanism similar to the 1) Rapid Simulation Speed
musculoskeletal system, nerves, spinal cord, and brain. Via the Simulation speed is essentially the basis of AI-assisted power
APIs, the simulator can provide massive data and prior system analysis and control. The training of AI models requires
knowledge for AI models, whereas AI models mine the data, massive data. The data generation is very time-consuming. The
discover the hidden patterns, and return well-trained models simulation speed is a bottleneck in successfully utilizing AI
and posterior knowledge. Meanwhile, AI models can provide algorithms and training a model with sufficient performance.
predictions and suggestions to power system operators based Therefore, the simulator must be well optimized and highly
on their demands. Therefore, a closed-loop interaction efficient. Algorithm-level and task-level parallelism, which is
mechanism is established. With the support of this interaction solution-level and function-level in Fig. 1, is required to fulfill
mechanism, the traditional physics-based simulator and the the efficiency needs in different situations.
2) Neural Network Supportability
Another requirement of the simulator is neural network
1
https://github.com/xxh0523/Py_PSOPS.
 3

supportability, i.e., being able to load the structure and example, massive samples of tens or even hundreds of
parameters of neural networks and perform at least forward gigabytes are generated. The data exchange better happens in
propagation of neural networks. The simulator should be able RAM instead of on hard drives. If there is not enough RAM, the
to integrate AI models into any part of the simulator so that the data could be cut into several pieces and transferred
efficiency and accuracy of the simulator can be improved by sequentially, or the data could be exchanged using a database.
the cooperation of traditional algorithms and AI. 3) Interpreted Language-Written
It is recommended to develop the APIs of an AI-oriented
B. Application Programming Interface (API)
power system simulator with interpreted languages such as
Flexible APIs are crucial to the cooperation of power system Python. These programming languages are easy to learn and
simulations and AI models. The APIs of an AI-oriented power use, and have very mature developer communities. There are a
system simulator are divided into external APIs and internal tremendous number of Python-written open-source AI projects
APIs. External APIs are used to get data and invoke functions on GitHub. With Python APIs, the simulator can be easily
of the simulator, whereas internal APIs are used to modify data modified for AI applications and cooperate with AI models.
and control the simulation process. Corresponding to the four
parts of the simulator shown in Fig. 1, the external and internal C. Interactions of Simulations and AI
APIs can further be divided into four categories, namely, model In contrast to the simulator, AI produces data-driven models
APIs, parameter APIs, solution APIs, and function APIs. based on existing data, as illustrated in Fig. 1. Firstly, a training
External model APIs and parameter APIs are used to output dataset is needed. The quality and representativeness of the
structures and parameters of different models. The model samples will seriously affect the performance of AI models.
expression should be easy to understand and modify, e.g., Secondly, a hypothesis set is established, i.e., a learning
diagrams, JSON files, etc. External solution APIs are used to framework is selected based on the task and the training data.
output parameters related to solution methods and intermediate Thirdly, optimization algorithms are utilized to train the model.
results during the solution process, e.g., the node ordering At last, the final hypothesis, i.e., an AI model, is obtained.
method, the admittance matrix, the iteration number of the The interactions between the simulator and AI models are
power flow solution, the integration step, etc. External function further illustrated in Fig. 2. The right part demonstrates
APIs are used to invoke functions e.g., power flow solutions, simulation-assisted AI and the left part denotes AI-assisted
short-circuit calculations, transient simulations, etc., as well as simulation.
output required simulation results, e.g., the maximum rotor
angle difference, nodal voltages, currents of transmission lines,
etc.
Correspondingly, the internal APIs are used to alter model
types, modify parameters, change solution methods, and adjust
boundary conditions of functions. Through internal APIs, AI
models can be used to model dynamic components, estimate Fig. 2. Interactions between simulation and AI.
parameters, accelerate solutions, and surrogate functions.
Overall, compared with the APIs of existing commercial 1) Simulation-Assisted AI
simulation tools, the APIs of an AI-oriented power system Simulation-assisted AI is realized based on the external APIs.
simulator make it easy to develop AI models for power system Firstly, simulations can be used to generate training data. The
applications and realize the cooperation of traditional actual training dataset also needs sample selection or
algorithms and AI algorithms. More detailed information on augmentation, e.g., stability prediction needs simulation results
power system components and simulations can be provided to with balanced stability labels, etc. Sampling methods are very
AI models via the external APIs and AI models can be deeply important since the data quality determines the performance of
integrated into power system simulations via the internal APIs. AI models [28]. Secondly, physics-based models can be used as
Other suggested features of the APIs are as follows. a strong prior knowledge for AI model design. For example, the
1) No Impact on Simulation Efficiency power network topology can be used to design GNN [18].
The implementation of APIs must not affect the efficiency of Thirdly, physical laws such as conservation laws can be used as
the simulator. As mentioned before, the simulator focuses on constraints in optimization algorithms to limit the feasible
efficiency. The source code is usually written with efficient region, improve the interpretability of AI models, and speed up
programming languages such as C++, Java, FORTRAN, etc., the training process [29], [30]. Finally, the simulator can be used
i.e., the implementation is highly organized and optimized. It as the benchmark for the performance verification of AI models.
should not be disturbed by the APIs. Therefore, a suggested 2) AI-Assisted Simulation
way is to rewrap the needed internal functions as external AI-assisted simulation is realized based on the internal APIs.
functions. These external functions can be invoked by other Firstly, data-driven AI models can be used for dynamic
programming languages. component modeling [31], [32]. Although the model may suffer
2) Efficient Memory Exchange from the problem of interpretability, the measurement-based AI
Data are frequently exchanged between the simulator and AI model can also be accurate and adaptive. Via the internal model
models. Taking neural network-based stability prediction as an APIs, physics-based models and data-driven models can be
integrated seamlessly and be simulated simultaneously [33]. (AMD-MNSP) algorithm and the multi-path sparse vector
Secondly, AI models can be used for power system model method are utilized to enhance the efficiency of the sparse
calibration [34]. Power system dynamic modeling and vector method while maintaining the sparsity of the factorized
parameter estimation are facing increasing challenges because matrix [39].
of the rapid development of renewable generation and power In terms of parallel computing, a fully parallel BBDF method
electronics. Data-driven power system dynamic modeling could [40], a fully parallel nested BBDF method [41], and an efficient
be a potential solution. Thirdly, AI models can be used to computing task allocation scheme based on subnet-core
discover and formulate the hidden patterns in the solution mapping and mixed programming of MPI and OpenMP [42] are
procedures and improve solution efficiency, e.g., convergence adopted to improve the BBDF method at the algorithmic and
prediction of power network equations, variable integration step implementational levels.
prediction, switch state prediction of power electronic devices As for memory allocation, a node ordering-based memory
[35], etc. Finally, AI models can be used as surrogate models for allocation technique is utilized, which reorders components
power system analysis [36] and control [37]. Power system such as transmission lines, transformers, generators, and loads
computation can be very time-consuming. Using a surrogate based on the node ordering and saves the admittance matrix, the
model as an approximation of the actual computation can equation coefficient matrix, and the independent vector in a
significantly increase the efficiency of analysis and contiguous memory block to increase the cache hit rate when
decision-making. solving power network equations.
In TABLE I, the basic information and average time
III. IMPLEMENTATION OF AN AI-ORIENTED SIMULATOR consumption of 10-second transient simulations after utilizing
In Fig. 3, a prototype of the AI-oriented simulator explained the above techniques are displayed. The 2383wp system is a
in the former section is implemented based on a widely used test system in MATPOWER. Sys13490 and
high-performance electromechanical simulator called Power Sys24886 are two practical power systems. All the dynamic
System Optimal Parameter Selection (PSOPS). After components are modeled in detail. The test HPC platform is
developing some external functions to support Python APIs, the Sugon I950r-G installed with 8 Intel Xeon E7-8837 2.67 GHz
simulator is compiled as a dynamic link library PSOPS.dll in processors. Each processor is integrated with 8 CPU cores, i.e.,
Windows and a shared object file PSOPS.so in Linux. The the total number of CPU cores is 64. As can be seen,
Python APIs of the prototype are developed using the ctypes simulations are significantly accelerated.
TABLE I
library [38]. The PSOPS.dll, PSOPS.so, and the open-source BASIC INFORMATION AND AVERAGE TIME CONSUMPTION OF 10-SECOND
Python APIs, can be found in the repository called Py_PSOPS SIMULATIONS OF THREE TEST SYSTEMS
on GitHub. Time Consumption
Test Number of components
(seconds)
Systems
Bus Branch Generator Load Serial Parallel
2383wp 2383 2892 327 1822 2.655 0.365
Sys13490 13490 22544 1797 3550 9.911 0.587
Sys24886 24886 39512 1919 5646 13.525 0.639

2) C++ Libraries for Neural Network Supportability

Meanwhile, neural network supportability is realized using
Fig. 3. Implementation and tests of Py_PSOPS. two open-source C++ libraries, namely, the Eigen library [44]
and the LibTorch library [45]. The difference between the two
In this section, the implementation details of the PSOPS libraries is the granularity of neural network implementation.
simulator and the APIs are illustrated. As for the Eigen library, neural networks are built in the source
code of the PSOPS simulator using the vector class and the
A. Implementation of PSOPS
matrix class. The structure of neural networks is loaded by
PSOPS can perform AC power flow considering PV-PQ reading a JSON file and the parameters are loaded by reading
switching and electromechanical transient stability simulations. binary files saved by PyTorch [46]. On the other hand, the
It is developed using C++ based on previous studies [39], [40], neural modules saved by PyTorch, i.e., both the structures and
[41], and [42]. Power system dynamics are modeled with a the parameters, are directly loaded for the LibTorch library.
group of high-dimensional nonlinear differential-algebraic The Eigen library is computationally more efficient but
equations. The alternating approach proposed in [43] is adopted structural changes of neural networks require modification of
in PSOPS due to its simplicity, reliability, and robustness [27]. the source code of PSOPS, whereas the LibTorch library is
1) Traditional Techniques for Efficiency Improvement simpler to use but the computation speed is lower than the
In PSOPS, traditional algorithms including improved sparsity Eigen library. Therefore, the LibTorch library is recommended
techniques, improved bordered block diagonal form (BBDF) for model evaluation and the Eigen library is recommended for
method, and memory allocation techniques are utilized to model deployment. By changing the version of LibTorch,
accelerate transient simulations. Py_PSOPS can adapt to different versions of PyTorch.
As for the sparsity techniques, the Approximate Minimum As mentioned before, neural network supportability makes it
Degree Minimum Number of Source Predecessors
 5

possible to integrate AI with traditional simulations. Currently, 4) Function API

the integration and simultaneous simulations of AI-based The function API supports calling power flow solutions and
power system dynamic models and physics-based models have transient stability simulations and gets simulation results
been realized in PSOPS. Other aforementioned applications of including rotor angles, rotation speed, inner electric potential,
AI-assisted simulations are still under development. active and reactive power, regulators’ outputs, nodal voltages,
etc. Meanwhile, task-level parallelism is realized using the ray
B. Implementation of APIs
library [48] of Python.
As shown in Fig. 3, the PSOPS simulator is compiled as a
dynamic link library file PSOPS.dll in Windows and a shared IV. CASE STUDIES
object file PSOPS.so in Linux. Only the external functions,
In this section, four typical cases of utilizing the prototype,
which are realized by rewrapping the models and functions of
namely, sample generation, spatiotemporal graph
PSOPS, can be accessed. The external functions of PSOPS and
convolutional networks (STGCN)-based stability prediction,
the Python APIs are connected using the ctypes library.
neural ODE-based [49] power system dynamic component
The Python APIs load the external functions from PSOPS.dll
modeling, and RL-based stability-constrained optimal power
and PSOPS.so and reorganize the data into a NumPy [47] style.
flow (SOPF), are demonstrated to show the simulator-AI
The source code is organized in a component-based manner,
interactions based on Py_PSOPS, as shown in Fig. 3. Sample
which means the functions of the same kind of component are
generation is one of the most basic applications of Py_PSOPS.
put together. In the Python APIs, computational functions such
The STGCN-based stability prediction, the neural ODE-based
as power flow solution and transient simulation have names
dynamic modeling, and the RL-based SOPF are typical
that begin with “cal”. Functions with names that start with “get”
examples of simulation-assisted AI, AI-assisted simulation, and
and “set” denote the functions of external APIs and internal
mutual assistance between simulation and AI, respectively.
APIs respectively. The Python APIs can be extended easily to
More applications can be developed using Py_PSOPS. All four
fulfill the needs in different situations. A more well-rounded
cases are supported by at least one paper or open-source code
API will be a future working direction. The details of external
we developed on GitHub. The success of these tasks proves the
and internal model APIs, parameter APIs, solution APIs, and
validity, flexibility, and efficiency of the design and
function APIs are as follows.
implementation.
1) Model API
The test system is the IEEE-39 system. The high-performance
As mentioned before, when using the Eigen library, the
server used for testing consists of an NVIDIA P100 GPU, 250
structure of neural networks can be established in the simulator
gigabytes of RAM, and two Intel Xeon Gold 5118 processors,
by modifying the basic computation data file and reading a
which contain 24 CPU cores in total and hyperthreading is
JSON file containing the names and structure of layers of the
enabled, i.e., there are up to 48 threads available.
neural network. When using the LibTorch library, the whole
neural model can be established by directly loading the modules A. Sample Generation
saved by PyTorch. 1) Step-wise Power Flow Sampling Scheme
2) Parameter API Sample generation can be used for any AI application. It is
Components’ parameters such as the name, the total number, supported by the rapid simulation speed of PSOPS. As for
the constraints, the default settings, dynamic model parameters, power flow sampling, simple random sampling, grid sampling,
connectivity, etc., can be got or set. However, currently, the and a step-wise sampling scheme are implemented. The
parameters of neural networks are directly loaded by the pseudo-code of the step-wise scheme is shown as follows.
simulator via modifying the basic computation data file and Pseudo-code 1: Step-wise Power Flow Sampling Scheme
reading a binary file. input the total number of required samples N
3) Solution API set n  1
The intermediate results during simulation processes can be get the limits of PD , Q D , PG , and VG using external parameter APIs
reached. The simulated power system can be set to state at any while n  N :
integration step. Basic data of the solutions such as the iteration set the connectivity states of all the transmission lines to true using internal
number, the simulation time, the integration step, faults, solution APIs
disturbances, etc., can be accessed. More importantly, the set the connectivity state of one or two randomly chosen transmission lines
network topology accessibility is realized. Network topology to false using internal solution APIs
set PD and Q D randomly within limits using internal parameter APIs
data such as the admittance matrix, the impedance matrix, the
calculate sum(PD )
number of fill-ins, and the factorized lower and upper triangular
do
matrices can be obtained. The connectivity of each component, set PG randomly within limits using internal parameter APIs
i.e., whether the component is connected to the power network, until sum(PG )  sum(PSlack )  sum(PD )  sum(PG )  sum(PSlack )
can be changed and network connectivity check is supported, set VG randomly within limits using internal parameter APIs
i.e., asynchronous subsystems can be identified. Other settings power flow calculation using external function APIs
such as power flow solution methods, integration methods, node get power flow convergence using external solution APIs
ordering algorithms, and sparse vector methods can be modified if power flow calculation converges:
by changing the basic computation data file. save ( PD , PG , VG )
set n  n  1
 end if temporal data of selected state variables obtained by a short-time
end while
simulation from t  0 to t  T , which can be obtained using
where PD and Q D are the active power vector and reactive external function APIs. The fault is cleared at the instant t  tc .
power vector of loads, respectively, sum() denotes the sum of
The output of the model is the predicted stability label of the
elements in the vector, PG is the active power vector of
input cases.
generators, PSlack and PSlack are the upper limit vector and the
lower limit vector of slack generators, VG is the nodal voltage After training, the STGCN model and the transient simulation
vector of generators other than slack generators. The specific function of the Py_PSOPS can be integrated using APIs and
APIs used in each step are highlighted in bold. perform efficient transient stability analysis. The pseudo-code
After power flow sampling, contingencies are sampled by of training an STGCN model and integrating the STGCN model
randomly choosing a component, a fault type, a fault location, with transient simulations is as follows.
and a fault clearing time. Transient simulations of these Pseudo-code 2: STGCN-based Stability Prediction
contingencies are carried out to generate simulation samples. Training Procedures
2) Test Results input the total number of training epochs N E , the training dataset, the test
The source code of the proposed sampling scheme can be dataset, mini-batch size m
found on GitHub in the Py_PSOPS repository. On the test server, initiate the parameters of the STGCN model
set epoch  1
over 1.29 million power flow samples and over 50 million
while epoch  N E :
simulation samples of the IEEE-39 system are generated using
do
40 threads within 9 hours. This sample dataset is used to get a mini-batch of m samples from the training dataset
support the research on STGCN and neural ODE. STGCN.forward(mini-batch)
B. STGCN-based Stability Prediction calculate the cross-entropy loss
loss.backward()
This is a typical example of simulation-assisted AI. The update parameters of STGCN
simulator provides training data as well as prior knowledge to until all the samples in the training dataset have been selected
support AI model design. evaluate the STGCN model in the test dataset
1) STGCN set epoch  epoch  1
An STGCN-based stability prediction model is proposed in end while
[50]. The idea is to predict transient stability based on the Integration of the STGCN model and transient simulations
input the STGCN model, the contingency, the integration step t , and the
power network changes and state variable changes within a
short simulation time T
short time after faults occur. The STGCN can extract features power flow calculation using external function APIs
from these changes and learn the correlation of these changes set contingency using internal function APIs
with power system stability. Only a short-time simulation is set the integration step to t and the simulation time to T using internal
required and the efficiency of stability analysis can be improved. solution APIs
transient simulation using external function APIs
get Y0 , Y1 , Y2 , and f 0 , f1 ,..., f n using external solution APIs and external
function APIs
stability label = STGCN.forward( Y0 , Y1 , Y2 , and f 0 , f1 ,..., f n )

2) Test Results

Fig. 4. The architecture of the STGCN-based stability prediction model.

The implementation is supported by the network topology

accessibility of Py_PSOPS. The STGCN-based stability
prediction model is shown in Fig. 4. Each STGCN is composed
of a one-dimensional convolutional neural network (1D-CNN)
layer, a graph convolutional network (GCN) layer, and another
1D-CNN layer. The input of the ( l  1 )-th layer of STGCN is Fig. 5. Results of STGCN、CNN、LSTM、MLP models.
f l and Y and the output is f l 1 . The input features include Y0 ,
Y1 , and Y2 , i.e., the admittance matrices before the fault, during The STGCN-based stability prediction model is trained.
the fault, and after clearing the fault, respectively, which can be Samples in the training dataset are randomly selected in the
obtained via external solution APIs, and f 0 , f1 ,..., f n , i.e., sample dataset. The training dataset contains 10240 samples,
 7

whereas the testing dataset contains 33600 samples. The until all the samples in the training dataset have been selected
comparison results of the STGCN model, the convolutional evaluate the NODE model in the test dataset
neural network (CNN) model, the long short-term memory set epoch  epoch  1
(LSTM) model, and the multi-layer perceptron (MLP) model end while
NODE model-integrated transient simulation
are displayed in Fig. 5. Each time of STGCN-based stability
input the NODE model, the contingency, the integration step t , and the
prediction averagely costs 5 milliseconds, whereas the complete
simulation time T
simulation averagely costs 25 milliseconds.
set the component model to the NODE model using internal model APIs
C. Neural ODE-based Dynamic Modeling power flow calculation using external function APIs
set contingency using internal function APIs
This is an example of AI-assisted simulation. The trained AI
set the integration step to t and the simulation time to T using internal
model is integrated into the simulator and is simulated solution APIs
simultaneously with physics-based dynamic models. transient simulation using external function APIs
1) Neural ODE-based Power System Dynamic Modeling 2) Test Results
A neural ODE-based dynamic modeling method is proposed The source code of common neural ODE modules for power
in [51]. The idea is to build data-driven dynamic models based system dynamic modeling is developed and published on
on neural ODE and accessible measurement data when there is a GitHub2. A test case of modeling a generator with the published
lack of prior knowledge of the component, e.g., equivalent neural ODE module is conducted. The generator at bus 31 is
modeling of load areas [52] and renewable plants [53]. While modeled with the classical generator model. There are 3,200
learning a global approximation of the derivative functions with samples in the training dataset and 800 samples in the test
easily trainable neural networks, neural ODE also keeps the dataset. Samples are randomly selected in the sample dataset
classical framework of numerical integration, which is a very including stable contingencies and unstable contingencies. The
important prior knowledge and makes neural ODE highly neural model is trained using only the state variables before the
adaptive to scientific computations and industrial applications, maximum rotor angle difference exceeds 360 degrees.
as shown in : After training, the neural ODE-based dynamic model is
Y(x, V; )  x  f (x, V ) (1) integrated into the simulator using the LibTorch Library. The
where x denotes the state variables, whose time derivatives are comparative results of the original classical generator
equal to f (x, V ) , V denotes the nodal voltages, Y denotes the model-based simulations and the neural ODE model-integrated
parametric derivative functions, and  denotes the parameters simulations are shown in Fig. 6. As can be seen, the modeling
of the parametric derivative functions. After inputting the initial errors are within an acceptable range.
value x  x(0) , the variation of x can be calculated with a
numerical integration method. The parameters of neural ODE
can be trained using a set of sampled curves of x and V ,
which can be obtained through external APIs. The loss function
is the mean squared error between the predicted curves and the
ground-truth curves of x .
After training, the neural ODE-based dynamic models are (a) Stable contingency (b) Unstable contingency
directly integrated with physics-based models and transient Fig. 6. Comparative results of the classical generator model-based simulations
and the neural ODE model-integrated simulations under stable and unstable
simulations are carried out to prove the efficacy of the neural contingencies.
model. The pseudo-code of training a neural ODE model and
neural ODE model-integrated transient simulations is as
follows. D. RL-based SOPF
Pseudo-code 3: Neural ODE-based Power System Dynamic Modeling This is an example of the simulator and AI mutually assisting
Training Procedures each other.
input the total number of training epochs N E , the training dataset, the test 1) Framework Design
dataset SOPF is one of the traditional control problems of power
initiate the parameters of the neural ODE (NODE) model systems. In SOPF formulation, a target function needs to be
set epoch  1 optimized under the equality constraints of the power flow
while epoch  N E : equations and the differential-algebraic equations of power
do
ˆ (t )}, t  {t , t ,..., t } from the training dataset
system dynamics, as well as the inequality constraints of static
get a sample {xˆ (ti ), V i
ˆ (t )
i 0 1 n
security constraints and dynamic security constraints [54], [55].
set x  xˆ (t0 ) and V  V
for ti in t1 , t 2 ,..., t n  :
0
RL can solve this problem in a simulation-based optimization

x(ti )  x(ti1 )  (ti  ti1 )  NODE.forward() x(ti1 ), V ˆ (t ) manner as displayed in Fig. 7. Based on OpenAI Gym [56] and
i 1
end for Py_PSOPS, an environment of SOPF solutions is established,
cal the mean squared loss between x and x̂ which parses actions and outputs state and reward after
loss.backward() performing power flow calculation, transient simulation, and
update the parameters of the NODE model
2
https://github.com/xxh0523/Py_PSNODE.
constraints check. An AI-based agent determines actions the new state, and the reward
according to the state. The simulator-based environment and the end while
AI-based agent form an interactive mechanism by exchanging 2) Test Results
states, rewards, and actions. The training process is demonstrated in Fig. 8. After the agent
is trained, further tests are carried out to check the control
effectiveness. 50,000 power flow samples with dynamic
constraint violations are sampled. The agent gets the operation
state and outputs the control strategy. The agent cost 122.525
seconds, which includes the time consumption of generating a
strategy and performing the transient simulation once to check
the strategy. After control, 49602 samples return to safe
operating points, whereas 398 samples violate static stability
Fig. 7. The framework of the RL-based SOPF. constraints. The success rate is 99.204 percent and the new
operating points are 100 percent sure to maintain dynamic
After training, the AI-based agent can adjust power flow states security.
rapidly and is suitable for online applications. An RL-based
optimal power flow solution method has been proposed in [57]
using PSOPS and the twin-delayed deep deterministic policy
gradient (TD3) algorithm [58]. In this paper, a TD3-based SOPF
solution program is realized using Py_PSOPS. The reward
design is in :

(2)
Fig. 8. The training process of the agent with the TD3 algorithm.
where TE denotes the total simulation time, TS denotes the time
instant when the system loses stability and is set to TE if the
system remains stable, L(V, PG ) denotes the sum of the V. CONCLUSION
out-of-limit parts of V and PG , C denotes the generation cost, To conclude, based on the illustration of the interaction
and  and  represents the scaling factors. The generation cost mechanism between power system dynamic simulations and AI,
minimization problem is converted to the maximization an AI-oriented power system transient stability simulator called
problem of the reward. The pseudo-code of RL-based SOPF is Py_PSOPS is designed, implemented, tested, and made public.
as follows. Although it is currently an exploration of AI-oriented power
Pseudo-code 4: RL-based SOPF system dynamic simulations, the four test cases demonstrate
input the total number of training epochs N E and anticipated contingencies promising capabilities of Py_PSOPS to support the
initiate the parameters of the agent development of AI-assisted simulations and
set epoch  1 simulation-assisted AI applications in power system stability
while epoch  N E :
analysis and control. It should be noted the development of
reset the operation state using the Step-wise Power Flow Sampling
Py_PSOPS will continue in the future.
Scheme
get the original state using external function APIs
action = Agent.actor.forward(state) REFERENCES
adopt the action and set the operation state to the new state using internal [1]N. Hatziargyriou et al., “Definition and Classification of Power System
parameter APIs Stability – Revisited amp; Extended,” IEEE Transactions on Power Systems,
vol. 36, no. 4, pp. 3271–3281, Jul. 2021, doi: 10.1109/TPWRS.2020.3041774.
power flow calculation using external function APIs
[2]W. Zhongxi and Z. Xiaoxin, “Power System Analysis Software Package
get power flow convergence using external solution APIs (PSASP)-an integrated power system analysis tool,” in POWERCON ’98. 1998
if power flow calculation converges: International Conference on Power System Technology. Proceedings (Cat.
get the new state using external function APIs No.98EX151), Aug. 1998, vol. 1, pp. 7–11 vol.1. doi:
transient simulations for the anticipated contingencies using external 10.1109/ICPST.1998.728602.
[3]H. Song, R. Na, S. Ting, and P. Xiaojun, “Study on conversion between the
function APIs
common models of PSD-BPA and PSS/E,” in 2013 IEEE 11th International
get the time instants TS when the system loses stability using external Conference on Electronic Measurement Instruments, Aug. 2013, vol. 1, pp.
solution APIs 64–69. doi: 10.1109/ICEMI.2013.6743040.
check constraints using external function APIs [4]K. Morison, L. Wang, and P. Kundur, “Power system security assessment,”
end if IEEE Power and Energy Magazine, vol. 2, no. 5, pp. 30–39, Sep. 2004, doi:
10.1109/MPAE.2004.1338120.
calculate reward
[5]G. Lammert, L. D. P. Ospina, P. Pourbeik, D. Fetzer, and M. Braun,
update the parameters of the agent based on the original state, the actions, “Implementation and validation of WECC generic photovoltaic system models
 9

in DIgSILENT PowerFactory,” in 2016 IEEE Power and Energy Society IEEE Transactions on Power Systems, pp. 1–1, 2021, doi:
General Meeting (PESGM), Jul. 2016, pp. 1–5. doi: 10.1109/TPWRS.2021.3133611.
10.1109/PESGM.2016.7741608. [25] J. Liu, Z. Yang, J. Zhao, J. Yu, B. Tan, and L. Wenyuan, “Explicit
[6]S. Cole and B. Haut, “Robust Modeling Against Model-Solver Interactions Data-Driven Small-Signal Stability Constrained Optimal Power Flow,” IEEE
for High-Fidelity Simulation of VSC HVDC Systems in EUROSTAG,” IEEE Transactions on Power Systems, pp. 1–1, 2021, doi:
Transactions on Power Systems, vol. 28, no. 3, pp. 2632–2638, Aug. 2013, doi: 10.1109/TPWRS.2021.3135657.
10.1109/TPWRS.2012.2230194. [26] F. Ding, J. Pan, A. Alsaedi, and T. Hayat, “Gradient-Based Iterative
[7]J. Susanto, PYPOWER-Dynamics. 2021. Accessed: Sep. 26, 2021. [Online]. Parameter Estimation Algorithms for Dynamical Systems from Observation
Available: https://github.com/susantoj/PYPOWER-Dynamics Data,” Mathematics, vol. 7, no. 5, Art. no. 5, May 2019, doi:
[8]C. Li, Y. Wu, H. Zhang, H. Ye, Y. Liu, and Y. Liu, “STEPS: A Portable 10.3390/math7050428.
Dynamic Simulation Toolkit for Electrical Power System Studies,” IEEE [27] P. Kundur, N. J. Balu, and M. G. Lauby, Power system stability and
Transactions on Power Systems, vol. 36, no. 4, pp. 3216–3226, Jul. 2021, doi: control, vol. 7. McGraw-hill New York, 1994.
10.1109/TPWRS.2020.3045102. [28] Y. Zhang et al., “Encoding Frequency Constraints in Preventive Unit
[9]J. Xu, C. Zhao, W. Liu, and C. Guo, “Accelerated Model of Modular Commitment Using Deep Learning with Region-of-Interest Active Sampling,”
Multilevel Converters in PSCAD/EMTDC,” IEEE Transactions on Power IEEE Transactions on Power Systems, pp. 1–1, 2021, doi:
Delivery, vol. 28, no. 1, pp. 129–136, Jan. 2013, doi: 10.1109/TPWRS.2021.3110881.
10.1109/TPWRD.2012.2201511. [29] B. Lusch, J. N. Kutz, and S. L. Brunton, “Deep learning for universal
[10] Y. Liu, Y. Song, Z. Yu, C. Shen, and Y. Chen, “Modeling and Simulation linear embeddings of nonlinear dynamics,” Nature Communications, vol. 9, no.
of Hybrid AC-DC System on a Cloud Computing Based Simulation Platform - 1, Art. no. 1, Nov. 2018, doi: 10.1038/s41467-018-07210-0.
CloudPSS,” in 2018 2nd IEEE Conference on Energy Internet and Energy [30] X. Hu, H. Hu, S. Verma, and Z.-L. Zhang, “Physics-Guided Deep Neural
System Integration (EI2), Oct. 2018, pp. 1–6. doi: 10.1109/EI2.2018.8582334. Networks for Power Flow Analysis,” IEEE Trans. Power Syst., vol. 36, no. 3,
[11] R. Kuffel, J. Giesbrecht, T. Maguire, R. P. Wierckx, and P. McLaren, pp. 2082–2092, May 2021, doi: 10.1109/TPWRS.2020.3029557.
“RTDS-a fully digital power system simulator operating in real time,” in [31] L. D. Pabon Ospina, V. Usuga Salazar, and D. Pabon Ospina, “Dynamic
Proceedings 1995 International Conference on Energy Management and equivalents of nonlinear active distribution networks based on
Power Delivery EMPD ’95, Nov. 1995, vol. 2, pp. 498–503 vol.2. doi: Hammerstein-Wiener models: an application for long-term power system
10.1109/EMPD.1995.500778. phenomena,” IEEE Transactions on Power Systems, pp. 1–1, 2022, doi:
[12] A. Kumar, A. Bahjaoui, S. K. Musunuri, and B. Rout, “Design and 10.1109/TPWRS.2022.3153117.
Tuning of Multi-Band Based Power System Stabilizer and Implementation in [32] Y. Zhou and P. Zhang, “Neuro-Reachability of Networked Microgrids,”
HYPERSIM,” in 2019 20th International Conference on Intelligent System IEEE Transactions on Power Systems, vol. 37, no. 1, pp. 142–152, Jan. 2022,
Application to Power Systems (ISAP), Dec. 2019, pp. 1–6. doi: doi: 10.1109/TPWRS.2021.3085706.
10.1109/ISAP48318.2019.9065952. [33] T. Xiao, Y. Chen, S. Huang, T. He, and H. Guan, “Feasibility Study of
[13] Y. Wang, S. Xu, Y. Xu, Q. Mu, and X. Zhang, “The Research and Neural ODE and DAE Modules for Power System Dynamic Component
Implementation of Power CPS Simulation Platform Based on ADPSS,” in The Modeling,” arXiv:2110.12981 [cs, eess], Apr. 2022, Accessed: Apr. 20, 2022.
16th IET International Conference on AC and DC Power Transmission (ACDC [Online]. Available: http://arxiv.org/abs/2110.12981
2020), Jul. 2020, vol. 2020, pp. 706–711. doi: 10.1049/icp.2020.0054. [34] Z. Bi, F. Wang, and C. Liu, “Getting parameters in power systems based
[14] Z. Wang, Y. Chen, S. Huang, X. Zhang, and X. Liu, “Temporal Graph on adaptive linear neural network,” in The 2006 IEEE International Joint
Super Resolution on Power Distribution Network Measurements,” IEEE Conference on Neural Network Proceedings, Jul. 2006, pp. 1458–1462. doi:
Access, vol. 9, pp. 70628–70638, 2021, doi: 10.1109/ACCESS.2021.3054034. 10.1109/IJCNN.2006.246866.
[15] P. Zhang and Y. Zhou, “Neuro-Reachability of Networked Microgrids,” [35] K. Wang, W. Wei, S. Gao, S. Huang, X. Sun, and B. Zhou, “Parametric
IEEE Transactions on Power Systems, pp. 1–1, 2021, doi: Average-Value Modeling of Diode Rectifier Systems Based on Neural
10.1109/TPWRS.2021.3085706. Network,” in 2022 4th Asia Energy and Electrical Engineering Symposium
[16] R. Nagi, X. Huan, and C. Chen, “Bayesian Inference of Parameters in (AEEES), Mar. 2022, pp. 609–613. doi: 10.1109/AEEES54426.2022.9759625.
Power System Dynamic Models Using Trajectory Sensitivities,” IEEE [36] L. Zhu, D. J. Hill, and C. Lu, “Hierarchical Deep Learning Machine for
Transactions on Power Systems, pp. 1–1, 2021, doi: Power System Online Transient Stability Prediction,” IEEE Transactions on
10.1109/TPWRS.2021.3104536. Power Systems, vol. 35, no. 3, pp. 2399–2411, May 2020, doi:
[17] H. Nie, Y. Chen, Y. Song, and S. Huang, “A General Real-time OPF 10.1109/TPWRS.2019.2957377.
Algorithm Using DDPG with Multiple Simulation Platforms,” in 2019 IEEE [37] I.-I. Avramidis, F. Capitanescu, S. Karagiannopoulos, and E. Vrettos, “A
Innovative Smart Grid Technologies - Asia (ISGT Asia), May 2019, pp. Novel Approximation of Security-Constrained Optimal Power Flow With
3713–3718. doi: 10.1109/ISGT-Asia.2019.8881174. Incorporation of Generator Frequency and Voltage Control Response,” IEEE
[18] J. Huang, L. Guan, Y. Su, H. Yao, M. Guo, and Z. Zhong, Transactions on Power Systems, vol. 36, no. 3, pp. 2438–2447, May 2021, doi:
“System-Scale-Free Transient Contingency Screening Scheme Based on 10.1109/TPWRS.2020.3028427.
Steady-State Information: A Pooling-Ensemble Multi-Graph Learning [38] “ctypes — A foreign function library for Python — Python 3.9.7
Approach,” IEEE Transactions on Power Systems, pp. 1–1, 2021, doi: documentation.” https://docs.python.org/3/library/ctypes.html (accessed Sep.
10.1109/TPWRS.2021.3097331. 29, 2021).
[19] K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward [39] T. Xiao, J. Wang, Y. Gao, and D. Gan, “Improved Sparsity Techniques
networks are universal approximators,” Neural Networks, vol. 2, no. 5, pp. for Solving Network Equations in Transient Stability Simulations,” IEEE Trans.
359–366, Jan. 1989, doi: 10.1016/0893-6080(89)90020-8. Power Syst., vol. 33, no. 5, pp. 4878–4888, Sep. 2018, doi:
[20] L. von Rueden, S. Mayer, R. Sifa, C. Bauckhage, and J. Garcke, 10.1109/TPWRS.2018.2803200.
“Combining Machine Learning and Simulation to a Hybrid Modelling [40] T. Xiao, W. Tong, and J. Wang, “A New Fully Parallel BBDF Method in
Approach: Current and Future Directions,” in Advances in Intelligent Data Transient Stability Simulations,” IEEE Trans. Power Syst., vol. 35, no. 1, pp.
Analysis XVIII, Cham, 2020, pp. 548–560. doi: 304–314, Jan. 2020, doi: 10.1109/TPWRS.2019.2933637.
10.1007/978-3-030-44584-3_43. [41] T. Xiao, W. Tong, J. Wang, and Y. Chen, “A Fully Parallel Nested
[21] H. Pan, Z. Dou, Y. Cai, W. Li, X. Lei, and D. Han, “Digital Twin and Its BBDF Method for Power System Transient Stability Simulations,” in 2021
Application in Power System,” in 2020 5th International Conference on Power IEEE Power Energy Society General Meeting (PESGM), Jul. 2021, pp. 01–05.
and Renewable Energy (ICPRE), Sep. 2020, pp. 21–26. doi: doi: 10.1109/PESGM46819.2021.9638100.
10.1109/ICPRE51194.2020.9233278. [42] T. Xiao, W. Tong, and J. Wang, “Study on Reducing the Parallel
[22] A. Lavin et al., “Simulation Intelligence: Towards a New Generation of Overhead of the BBDF Method for Power System Transient Stability
Scientific Methods,” Dec. 2021, Accessed: Jan. 05, 2022. [Online]. Available: Simulations,” IEEE Trans. Power Syst., vol. 35, no. 1, pp. 539–550, Jan. 2020,
https://arxiv.org/abs/2112.03235v1 doi: 10.1109/TPWRS.2019.2929775.
[23] L. Zhu, D. Hill, and C. Lu, “Semi-Supervised Ensemble Learning [43] H. W. Dommel and N. Sato, “Fast Transient Stability Solutions,” IEEE
Framework for Accelerating Power System Transient Stability Knowledge Transactions on Power Apparatus and Systems, vol. PAS-91, no. 4, pp.
Base Generation,” IEEE Transactions on Power Systems, pp. 1–1, 2021, doi: 1643–1650, Jul. 1972, doi: 10.1109/TPAS.1972.293341.
10.1109/TPWRS.2021.3117402. [44] “Eigen: Main Page.” https://eigen.tuxfamily.org/dox/ (accessed Sep. 28,
[24] C. Ren, Y. Xu, and R. Zhang, “An Interpretable Deep Learning Method 2021).
for Power System Dynamic Security Assessment via Tree Regularization,”
[45] “PyTorch C++ API — PyTorch master documentation.”
https://pytorch.org/cppdocs/ (accessed Feb. 20, 2022).
[46] “PyTorch.” https://pytorch.org/
[47] “NumPy.” https://numpy.org/ (accessed Sep. 29, 2021).
[48] “Ray - Scaling Python made simple, for any workload,” Ray.
https://www.ray.io/ (accessed Sep. 29, 2021).
[49] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud,
“Neural Ordinary Differential Equations,” Advances in Neural Information
Processing Systems, vol. 31, pp. 6571–6583, 2018.
[50] Y. Zhuang, T. Xiao, L. Cheng, Y. Chen, and H. Guan, “Transient
Stability Assessment of Power System Based on Spatio-Temporal Graph
Convolutional Networks,” Automation of Electric Power Systems, vol. pp, no.
pp, p. pp, Accepted.
[51] T. Xiao, Y. Chen, S. Huang, T. He, and H. Guan, “Neural ODE and DAE
Modules for Power System Dynamic Component Modeling,”
arXiv:2110.12981 [cs, eess], Dec. 2021, Accessed: Dec. 28, 2021. [Online].
Available: http://arxiv.org/abs/2110.12981
[52] A. Arif, Z. Wang, J. Wang, B. Mather, H. Bashualdo, and D. Zhao,
“Load Modeling—A Review,” IEEE Transactions on Smart Grid, vol. 9, no. 6,
pp. 5986–5999, Nov. 2018, doi: 10.1109/TSG.2017.2700436.
[53] J. Liu, W. Yao, J. Fang, J. Wen, and S. Cheng, “Stability analysis and
energy storage-based solution of wind farm during low voltage ride through,”
International Journal of Electrical Power & Energy Systems, vol. 101, pp.
75–84, Oct. 2018, doi: 10.1016/j.ijepes.2018.03.013.
[54] S. Abhyankar, G. Geng, M. Anitescu, X. Wang, and V. Dinavahi,
“Solution techniques for transient stability-constrained optimal power flow –
Part I,” Transmission Distribution IET Generation, vol. 11, no. 12, pp.
3177–3185, 2017, doi: 10.1049/iet-gtd.2017.0345.
[55] G. Geng, S. Abhyankar, X. Wang, and V. Dinavahi, “Solution techniques
for transient stability-constrained optimal power flow – Part II,” Transmission
Distribution IET Generation, vol. 11, no. 12, pp. 3186–3193, 2017, doi:
10.1049/iet-gtd.2017.0346.
[56] G. Brockman et al., “OpenAI Gym,” arXiv:1606.01540 [cs], Jun. 2016,
Accessed: Nov. 14, 2020. [Online]. Available: http://arxiv.org/abs/1606.01540
[57] H. Zhen et al., “Design and tests of reinforcement-learning-based
optimal power flow solution generator,” Energy Reports, vol. 8, pp. 43–50, Apr.
2022, doi: 10.1016/j.egyr.2021.11.126.
[58] S. Fujimoto, H. Hoof, and D. Meger, “Addressing Function
Approximation Error in Actor-Critic Methods,” in International Conference on
Machine Learning, Jul. 2018, pp. 1587–1596. Accessed: Apr. 06, 2021.
[Online]. Available: http://proceedings.mlr.press/v80/fujimoto18a.html