Journal of Intelligent & Robotic Systems (2024) 110:94

https://doi.org/10.1007/s10846-024-02125-z

REGULAR PAPER

Multi-Encoder Spatio-Temporal Feature Fusion Network for Electric

Vehicle Charging Load Prediction
Yufan Chen1,2 · Mengqin Wang1,2 · Yanling Wei1,2 · Xueliang Huang3 · Shan Gao3

Received: 2 March 2024 / Accepted: 28 May 2024 / Published online: 9 July 2024
© The Author(s) 2024

Abstract
Electric vehicles (EVs) have been initiated as a preference for decarbonizing road transport. Accurate charging load prediction
is essential for the construction of EV charging facilities systematically and for the coordination of EV energy demand with
the requisite peak power supply. It is noted that the charging load of EVs exhibits high complexity and randomness due to
temporal and spatial uncertainties. Therefore, this paper proposes a SEDformer-based charging road prediction method to
capture the spatio-temporal characteristics of charging load data. As a deep learning model, SEDformer comprises multiple
encoders and a single decoder. In particular, the proposed model includes a Temporal Encoder Block based on the self-attention
mechanism and a Spatial Encoder Block based on the channel attention mechanism with sequence decomposition, followed
by an aggregated decoder for information fusion. It is shown that the proposed method outperforms various baseline models
on a real-world dataset from Palo Alto, U.S., demonstrating its superiority in addressing spatio-temporal data-driven load
forecasting problems.

Keywords Electric vehicle · Load forecasting · Transformer · Spatio-temporal features · Series decomposition

1 Introduction demands. Then, unplanned and unregulated charging prac-

tices may lead to a stochastic peak load on the power grid,
As a mode of low-carbon energy transportation, EVs offer
exacerbating fluctuations in peak and off-peak electricity
numerous advantages, including low noise, energy efficiency,
consumption, and potentially compromising the overall qual-
and environmental friendliness. With the increasing adop-
ity of the power grid [1]. Attempts have been made to address
tion of EVs, there is a corresponding surge in user charging
these issues from the perspective of EVs. For instance, Mei
B Yanling Wei et al. optimize the energy management of hybrid EVs dur-
[email protected] ing both the initial driving stage [2] and regenerative braking
Yufan Chen stage [3] to enhance energy efficiency. It is noted that the
[email protected] energy optimization merely based on EVs is insufficient for
Mengqin Wang well addressing the charging load issues. In contrast, global
[email protected] regulation from the grid side is expected to improve the effi-
Xueliang Huang ciency of energy utilization. In addition, unlike fuel-powered
[email protected] vehicles, a longer time of energy supply for EVs naturally
Shan Gao poses greater demands on the management and construc-
[email protected] tion of public charging stations. Currently, many countries
and regions have a limited number of charging stations, with
1 School of Automation, Southeast University, Nanjing ongoing construction efforts. Correspondingly, an accurate
210096, China
estimate of the charging demand is essential for the plan-
2 Key Laboratory of Measurement and Control of Complex ning of EV infrastructure. Therefore, the precise prediction
System of Engineering, Southeast University, Nanjing
210096, China of EV charging loads is of extreme importance for facilitat-
3 ing the enhancement and optimization of the power system
School of Electrical Engineering, Southeast University,
Nanjing 210096, China and ensuring the stable operation of the power grid [4, 5].

0123456789().: V,-vol 123

94 Page 2 of 11 Journal of Intelligent & Robotic Systems (2024) 110:94

The existing research methodologies on EV charging non-Euclidean spaces, raising challenges for most RNN-
load prediction can be broadly categorized into probability- based or CNN-based methods. The increasing interest in
based methods and artificial intelligence-based methods [6]. leveraging the expressive abilities of graph structures for
Probability-based methods are first widely used in study- machine learning-based graph analysis has brought atten-
ing the operation patterns of EVs. For example, a statistical tion to graph convolutional networks (GCN) [19]. Spatio-
model [7], relying on a small number of samples of user daily temporal graph convolutional network (STGCN) [20, 21]
driving data, is used to predict the temporal distribution of generates the prediction results by stacking spatio-temporal
charging loads, where the impact of large-scale EV charg- convolutional blocks, which are composed of temporal
ing behavior on the power grid is then evaluated through a gated convolutional layers and GCN layers. Attention based
small-scale simulation. Ikegami et al. [8] analyze the impact spatial-temporal graph convolutional network (ASTGCN)
of high penetration rates of EVs on the overall power demand [22] is built upon STGCN by incorporating an attention
of the electrical system. By employing probabilistic methods, mechanism into the spatio-temporal convolutional block to
Faridimehr et al. [9] utilize the Markov chain to model the enhance the modeling of dynamic spatio-temporal correla-
EV charging network as a two-stage stochastic problem to tions. Global spatio-temporal dynamic capturing network
estimate the charging load. It is noted that these methods (GSTDCN) [23] describes dynamic traffic flow information
often require the involvement of complex influencing factors through a combination of a global encoding block and GCN.
during modeling, however, the variables included in these It is worth mentioning that the over-smoothing issue
models are usually difficult to observe directly. For exam- for GCN makes it difficult to stack multiple layers [24],
ple, the battery state of charge (SOC), which serves as a which exhibits slow convergence speeds with large-scale
fundamental variable, can only be estimated through inter- node datasets. Transformer [25] initially achieves remark-
nal resistance, charge-discharge current, and battery voltage able success in natural language processing (NLP) [26].
[10]. This process tends to introduce biases and increase the Its encoder-decoder architecture is capable of capturing
difficulty of prediction. dependencies between sequence inputs. In particular, there
In recent years, machine-learning-based approaches has been an increasing amount of Transformer-based mod-
emerged, and it has been proven to be simple and effective els on time series forecasting. AutoFormer [27] embeds a
for load prediction. Traditional machine learning methods, decomposition architecture in both the encoder and decoder,
known for their ease of execution and rapid computation, are also replacing the multi-head attention mechanism of the
widely employed in time series forecasting. For instance, vanilla Transformer with an auto-correlation mechanism to
Liao et al. [11] propose support vector machine (SVM) better extract features in long time series. Informer [28]
for modeling the load forecasting model. Buzna et al. [12] reduces the time and space complexity of Transformer com-
employ two classical approaches, i.e., random forests (RF) putation through ProbSparse self-attention and leverages a
and gradient boosting regression trees (GBRT), to com- generative style decoder to mitigate the prediction errors of
pare predictive performance using EV load data from 169 long sequence time-series forecasting (LSTF). While models
charging stations in the Netherlands. Nevertheless, as these based on Transformers often prioritize temporal information.
methods typically involve numerous parameters, the predic- Therefore, the insufficient utilization of spatial information
tion accuracy tends to decline significantly as the temporal leaves much space for improvement.
scale increases. Motivated by the Transformer-based models, we develop
More recently, by means of deep learning (DL) theories a spatio-temporal prediction network, SEDformer, to tackle
and the abundance of computing resources, an increasing the task of EV load forecasting. The primary contributions
number of neural network models have been presented for of this study are outlined as follows:
larger-scale spatio-temporal prediction. Specifically, long
short-term memory neural network (LSTM) [13] is first • We propose SEDformer to address the EV long-term load
applied in intelligent transportation systems (ITS) for traf- prediction problem by capturing features in both the tem-
fic flow prediction. Convolution and LSTM are combined poral and spatial dimensions.
to create a Conv-LSTM [14, 15] module for extracting • A multi-encoder architecture is employed to maxi-
the spatial-temporal features of traffic flow, where LSTM mize the utilization of information. Temporal features
captures temporal features and convolutional neural net- are extracted through a self-attention mechanism-based
works (CNN) [16] captures the spatial features of gridded Temporal Encoder Block, while spatial features, after
discrete data. DGCRN [17] is a recurrent neural network sequence decomposition, are extracted through a Squeeze-
(RNN)-based model, where gate recurrent unit (GRU) [18] and-Excitation block.
addresses the problem of gradient disappearing and dynamic • Experiments on various time scales are conducted on
graph convolution replaces matrix multiplication in GRU. an EV charging dataset in Palo Alto, U.S. SEDformer
Nonetheless, real-world scenarios usually generate data in demonstrates excellent predictive performance, surpass-

123
Journal of Intelligent & Robotic Systems (2024) 110:94 Page 3 of 11 94

ing all the baseline models based on different kinds of S-Encoder utilizes trend decomposition to capture spatial
architectures. information by yielding trend and fluctuation subsequences.
Subsequences of two modes are fed into individual S-E
Blocks, which essentially employ a form of channel atten-
2 Methodology tion to establish positional connections between different
charging stations. Then the outputs of T-Encoder and S-
2.1 Problem Formulation Encoder are fused through attention and gating mechanism.
Finally, the aggregated Decoder Block takes fusional output
As a time series prediction problem, we aim to forecast EV sequences from the encoder and masked sequences as inputs,
charging load data for the next T time steps based on the his- ultimately generating the sequential prediction as output. The
torical data spanning H time steps, which can be formulated architecture of SEDformer is depicted in Fig. 1.
as follows:
2.3 Temporal Encoder Block
Ŷ t+1 , Ŷ t+2 ,..., Ŷ t+T =
  (1) Embedding To ensure comprehensive understanding and
F Y t , Y t−1 , ..., Y t−H +1 ,
effective utilization of information, preprocessing of the
  input data is required before feeding it into the model. For
where Y i = y1i , y2i , ..., y N
i . Y i ∈ R N is a vector of length
the initial input X raw ∈ R N ×L , a one-dimensional convo-
N , and y ij signifies the load value of the j-th charging station lution is applied to map the N -channel input tensor to a
at time step i. The mapping F denotes the proposed model, higher dimension, enhancing non-linear expressive capabil-
which endeavors to minimize prediction errors as much as ities. Meanwhile, the temporal features, including month,
possible. weekday, and date information of the input values, are also
extracted using the convolution network for time embedding.
2.2 Overall Architecture Additionally, as the attention mechanism cannot inherently
utilize the positional features in sequences, we employ sine
We design SEDformer, featuring S-E block and trend decom- and cosine functions for positional encoding [25]. The entire
position block, to effectively harness the spatio-temporal embedding module can be represented as:
information in the charging load data. SEDformer is struc-
tured as an encoder-decoder, which takes the raw data from X in = P E (X raw ) + Conv1d (X raw )
the embedding layer as inputs and outputs the prediction of +T imeEmbed (X raw ) , (2)
sequence data. The encoder consists of Temporal Encoder
Block (T-Encoder) and Spatial Encoder Block (S-Encoder).
  
T-Encoder employs a self-attention mechanism to preserve sin pos/10000ieven /dmodel 
P E ( pos, i) = , (3)
temporal information of the input data. Meanwhile, the cos pos/10000iodd /dmodel

Fig. 1 The complete structure of SEDformer. The input sequence is processed in parallel by the temporal encoder block and spatial encoder Block,
then fused and input into the aggregated decoder block to produce the predicted sequence

123
94 Page 4 of 11 Journal of Intelligent & Robotic Systems (2024) 110:94

where X in ∈ Rdmodel ×L , dmodel is the dimension of the fea- comprising linear layers and an activation function, formu-
tures after the whole embedding. P E represents positional lated as follows:
encoding embedding. Decoder inputs share the same way of
embedding with encoder inputs. F F N (x) = r elu (0, x W1 + b1 ) W2 + b2 , (8)
Multi-Head Attention The attention mechanism assists
to selectively filter out more important information from a
where W1 , b1 , W2 , and b2 are the parameters of the linear
vast pool of data and focuses on these crucial details while
layer. Residual connections [30] are utilized in the encoder.
disregarding much of the remaining information. Consider-
ing the elements in the input source as key (K ) and value (V ),
for query (Q) of each input element, V is calculated based 2.4 Spatial Encoder Block
on the similarity between Q and K . The attention score is
obtained by weighted summation of V . The attention mech- Trend Decomposition Time series decomposition [31] can
anism can be formulated as follows: partition the original data into trend components and peri-
odic components. The trend components encapsulate the
QK T low-frequency characteristics of the data, while the peri-
Att (Q, K , V ) = so f tmax √ V, (4) odic components capture the high-frequency features. Trend
dk
decomposition block is devised to effectively draw distinct
characteristics of the sequence. Since the training data is
where dk is the dimension of K . Self-attention is employed
obtained using a sliding window, to prevent potential data
here, hence Q, K , and V are identical elements. Addition-
leakage issues during the process of model training, a mov-
ally, the sequence is decomposed, utilizing multiple heads to
ing average strategy [27] is employed for data decomposition.
create subspaces that focus on different aspects of informa-
The process of trend decomposition is depicted as follows:
tion, namely multi-head attention mechanism [25], which is
formulated as follows:
X tr = Moving Avg (Padding (X )) , (9)
Multi head (Q, K , V ) = X pe = X − X tr , (10)
Concat (head1 , head2 , ..., head H ) W O , (5)
where X , X tr , X pe ∈ R N ×L . A padding strategy is required
to maintain the same dimensions between the subsequences
  and the original load sequence.
head j = Attention Q j , K j , V j , S-E Block We introduce two Squeeze-and-Excitation
j = 1, ..., H , (6) blocks (S-E blocks) to extract inter-channel dependen-
cies within sub-sequences at different scales. Squeeze-
and-excitation networks (SENet) [32] are widely used in
Q CNN-based models to capture relationships between differ-
Q j = QW j , K j = K W jK , V j = V W jV ,
ent channels in images. Modifications have been made here
j = 1, ..., H , (7) to enable the application of SENet for sequential data. The
workflow of a single S-E block is illustrated in Fig. 2. For
where W O , W jQ , W jK , and W jV are the weight matrices of the input X in ∈ Rdmodel ×L , one-dimensional average pooling
linear layers. layer is utilized to compress the temporal information. Sub-
Feed-Forward Neural Network Layer normalization sequently, two fully connected layers are applied to obtain
(LayerNorm) [29] operates independently on the batch size, weights for each channel, representing the weights for each
computing the mean and variance for individual samples charging station. The obtained weight vector is multiplied
and normalizing across different channels of the sample. element-wise with X in through the broadcasting mecha-
Following LayerNorm, a feed-forward network is utilized, nism, assigning different attention to charging stations. The

Fig. 2 The framework of a

Squeeze-and-Excitation block.
Fsq (·) represents
one-dimensional average
pooling. Fex (·, W ) represents
the fully connected layer used
for feature scaling. Fbr (·) refers
to the weight mapping using
broadcasting

123
Journal of Intelligent & Robotic Systems (2024) 110:94 Page 5 of 11 94

Algorithm 1 SEDformer forward propagation algorithm.

Input: historical observations X raw = Y 0 , Y 1 , ..., Y k ,

masked target value X m = Y k+1 , Y k+2 , ..., Y k+t , 0, ..., 0


Output: predicted value X̂ = Ŷ k+1 , Ŷ k+2 , ..., Ŷ k+T
(1)
1: X en ← P E (X raw ) + Conv1d (X raw ) + T imeEmbed (X raw )
// L 1 -layers Temporal Encoder
2: for i = 1, ..., L1 do   
(i) (i) (i)
3: X en ← L N M H A X en + X en {MHA: Multi-Head Attention; LN: LayerNorm}
   
(i+1) (i) (i)
4: X en ← L N F F N X en + X en {FFN: Feed-Forward Network}
5: end for
// Spatial Encoder  
(1)
6: X tr , X pe ← Decomp X en
(1)   
7: X se ← sigmoid W f c12 · r elu W f c11 · Avg Pool (X tr )  X tr
(2)    
8: X se ← sigmoid W f c22 · r elu W f c21 · Avg Pool X pe  X pe
(1) (2)
9: X se ← X se + X se
// L 2 -layers Aggregated
 Decoder

(L )
10: C x = Concat X en 1 , X se
 
11: g ← sigmoid Wg · C x + bg
12: X g ← g · C x + (1 − g) · X se
(1)
13: X de ← P E (X m ) + Conv1d (X m ) + T imeEmbed (X m )
14: for j = 1, ..., L2 do   
( j) ( j) ( j)
15: X de ← L N M H A X de + X de
   
( j) ( j) ( j)
16: X de ← L N M H A X g , X de + X de
   
( j+1) ( j) ( j)
17: X de ← L N F F N X de + X de
18: end for  
(L )
19: X̂ ← Linear X de 2
20: return X̂

calculation formula for a single S-E block is as follows: aggregate encoded information as follows:

  C x = Concat (X att , X se ) , (12)

X se = sigmoid W f c2 · r elu W f c1 ·  
g = sigmoid Wg C x + bg , (13)
Avg Pool1d (X in )))  X in , (11)
X en = gC x + (1 − g) X se . (14)

where {X in , X se } ∈ Rdmodel ×L . X se is the output of the whole

For the outputs X te from T-Encoder and X se from S-Encoder,
S-Encoder. W f c1 and W f c2 represents the weight matrices of
X att is calculated through multi-head attention. C x can be
the fully connected layers.  represents channel-wise multi-
obtained by concatenating X att and X se . Wg and bg are
plication.
parameters of linear transformation. The aggregated result
X en will be used in the decoder. Algorithm 1 illustrates the
flow of the forward pass of SEDformer.
2.5 Aggregated Decoder Block

The decoder consists of two attention modules, a feed-

forward layer, and intermediate connection layers. The main 3 Experiments
components of the decoder resemble to those of the T-
Encoder, as shown in Fig. 1. In this section, we conduct a resampling of the real-world
Gating Mechanism It is necessary to integrate the out- EV charging dataset and undertake a comparative analysis
put information from different encoders before input into between several baseline models and our proposed SED-
the decoder. A gating mechanism is applied to selectively former to evaluate the model’s performance.

123
94 Page 6 of 11 Journal of Intelligent & Robotic Systems (2024) 110:94

3.1 Dataset 3.3 Baseline Models

We conduct experiments on a dataset containing usage We select five different neural network models, includ-
records of EV charging stations in Palo Alto, U.S. which span ing LSTM, CNN-based model, GCN-based model, and the
from July 2011 to December 2020 [33]. To address poten- Transformer families. They have all demonstrated satis-
tial data sparsity arising from the limited charging behavior factory performance in time series prediction tasks. These
of EVs at the early stage, the sampling interval is set to models are evaluated on the same dataset as our proposed
1-day. The dataset is divided into training, validation, and SEDformer for comparison.
testing sets in a ratio of 7:2:1. The location of several selected
charging stations, along with the historical load curve, are (1) Long Short-Term Memory (LSTM): LSTM addresses
illustrated in Fig. 3. the issue of gradient disappearing and exploding in
traditional RNN models by introducing memory cells,
3.2 Implementation Details enabling it to capture complex dependencies in long
sequences.
We conduct experiments using PyTorch [34] on a personal (2) Convolutional Neural Network and Long Short-Term
computer equipped with an AMD Ryzen 5 7500F CPU, Memory (Conv-LSTM): Conv-LSTM transforms spa-
32GB of RAM, and an NVIDIA GeForce RTX 4060Ti 16GB tially distributed data into grid structures, then employs
GPU. In the experiments, mean square error (MSE) loss and Conv2d and LSTM to extract spatial and temporal fea-
the ADAM optimizer [35] are employed with an initial learn- tures from traffic data.
ing rate of 10−4 . Batch size is set to 32. The number of (3) Temporal Graph Convolutional Network (T-GCN) [36]:
epochs is configured as 1000, and to prevent overfitting, the T-GCN incorporates GCN into the temporal model to
early stopping strategy is implemented during the training capture complex geographical topological structures.
process. According to practical demands, we set the size of (4) Transformer: Transformer features an encoder-decoder
the look-back sliding window as L ∈ {30, 60, 120}, corre- framework, where the attention mechanism is capable of
sponding to the prediction length O ∈ {7, 15, 30}. For the capturing complex temporal dependencies.
7-day and 15-day prediction length, SEDformer includes 1 (5) Informer: Informer introduces ProbSparse self-attention
layer of T-Encoder and 1 layer of Decoder. For the 30-day mechanism along with self-attention distilling operation
prediction length, SEDformer includes 2 layers of T-Encoder to specifically address LSTF, further enhancing the pro-
and 1 layer of Decoder. cess of inference through a generative style decoder.

Fig. 3 Charging station nodes and historical load curves. (a) A partial map of Palo Alto city, with several charging stations denoted by a blue circle.
(b) The charging loads over a 600-day period commencing in July 2011 of five distinct charging stations

123
Journal of Intelligent & Robotic Systems (2024) 110:94 Page 7 of 11 94

3.4 Results larger than T-GCN. Overall, SEDformer maintains its status
as the top-performing model in the EV load forecasting task.
Experiments are conducted on three different time-scale The comparative analysis of all models is as follows. First, it
datasets. Root mean square error (RMSE), mean absolute is observed that, in most cases, an increase in the prediction
error (MAE), and weighted mean absolute percentage error length contributes to higher complexity in long-term load
(WMAPE) are used as three evaluation metrics to compre- forecasting, which results in the model’s accuracy tending
hensively assess the model’s performance. The results are to decrease. However, when the prediction length increases
presented in Table 1. The metrics are computed as follows: from 7 days to 15 days, LSTM, T-GCN, and SEDformer show
 a slight decrease in prediction errors. This phenomenon can
 N  2 be attributed to the enlarged look-back window size, which
 1 L 
RMSE =  y ij − ŷ ij , (15) provides the models with more spatio-temporal information
NL
i=1 j=1 from historical data. Second, Conv-LSTM exhibits relatively
weaker performance in the prediction task, owing to the dif-
1    i 
L N ficulty introduced in predicting sparse grid data in the dataset

M AE = y j − ŷ ij , (16) from Palo Alto city, resulting in lower accuracy compared to
NL SEDformer and other baseline models. Third, it is observed
i=1 j=1
that LSTM and Transformer experience a significant perfor-
mance decline when the prediction length increases to 30
W M AP E =
  i  days, while SEDformer shows a relatively small increase in
L 
N y ij  y − ŷ i 
 j j errors, indicating its better capability in long-term load fore-
L N i 
  , (17)
i=1 j=1 i=1 j=1 y j y ij  casting.

where y ij represents the actual load value, and ŷ ij represents

the predicted load value of a single charging station. 4 Discussion
As shown in Table 1, our proposed model demonstrates
superior performance when the prediction length is 7 days 4.1 Contrast Analysis
and 30 days. At a prediction length of 15 days, SEDformer
has the smallest RMSE, while MAE and WMAPE are slightly In the above sections, the details of SEDformer are explained,
alongside the quantification of prediction errors across vari-
ous models. To visually showcase the predictive capabilities
Table 1 Prediction results of varivous models for different prediction of different models, we plot partial load prediction curves as
horizons O ∈ {7, 15, 30} shown in Fig. 4. Figure 4(a), (b), and (c) visualize the pre-
Models RMSE MAE WMAPE(%) diction results of the charging load for a single station over
7 days. Due to the short prediction horizon, almost all meth-
LSTM 7 23.466 18.874 0.554
ods can reflect the trend of the load variations. In Fig. 4(a),
15 23.310 18.647 0.548
T-GCN accurately captures the decreasing trend in load on
30 26.013 20.468 0.614
the 3rd day, while SEDformer aligns better with the peak
Conv-LSTM 7 25.420 20.245 0.598 value on the 5th day. Figure 4(d), (e), and (f) visualize the
15 25.661 20.484 0.608 prediction results of the charging load over 15 days. Partial
30 30.152 24.442 0.733 baseline models present a smoothed trend when the actual
T-GCN 7 23.433 18.925 0.556 values fluctuate minimally. In Fig. 4(e), for the two peak
15 22.997 18.367 0.538 points in the actual load curve, SEDformer closely matches
30 25.522 20.680 0.604 the real values compared to other models, and LSTM exhibits
Transformer 7 24.077 19.309 0.568 a notable prediction error on the 13th day. For the 30-day pre-
15 25.008 20.204 0.597 diction horizon depicted in Fig. 4(g), (h), and (i), all models
30 27.523 22.085 0.659 show larger deviations, with Conv-LSTM struggling to cap-
Informer 7 24.385 19.451 0.572 ture load variations, leading to predicted values converging
15 23.257 18.680 0.545 towards a constant value. In contrast, SEDformer can track
30 25.860 20.634 0.611 the changing trends of the ground truth more accurately, and
SEDformer 7 23.247 18.741 0.550 when there are peaks or valleys in the actual load values, the
15 22.659 18.522 0.540 prediction values of SEDformer are more proximate. The
30 24.242 19.405 0.575 findings indicate that the proposed approach can predict the
load of EV charging stations under three different predic-

123
94 Page 8 of 11 Journal of Intelligent & Robotic Systems (2024) 110:94

100 100 100

80

EV Charging Load(kW)
80

EV Charging Load(kW)
80
EV Charging Load(kW)

60 60 60

40 40 40

20 20 20

0 0 0
1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
Day(d) Day(d) Day(d)

(a) (b) (c)

100 100 100

80

EV Charging Load(kW)
80
EV Charging Load(kW) 80
EV Charging Load(kW)

60 60 60

40 40 40

20 20 20

0 0 0
2 4 6 8 10 12 14 2 4 6 8 10 12 14 2 4 6 8 10 12 14
Day(d) Day(d) Day(d)

(d) (e) (f)

100 100 100

80
EV Charging Load(kW)

80
EV Charging Load(kW)

80

EV Charging Load(kW)
60 60 60

40 40 40

20 20 20

0 0 0
0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30
Day(d) Day(d) Day(d)

(g) (h) (i)

Fig. 4 Visualization and comparison of prediction results. (a) to (c) represents a prediction horizon of 7 days, (d) to (f) for 15 days, and (g) to (i)
for 30 days

tion window sizes. Furthermore, compared to several similar encoder, the feature aggregation module is unnecessary and
methods, SEDformer exhibits the smallest prediction error. thus excluded.
The results in Fig. 5 reveal that the removal of any
module can result in a varying degree of performance
4.2 Ablation Studies degradation for SEDformer, which becomes more pro-
nounced as the prediction horizon increases. Particularly
To demonstrate the effectiveness of the proposed model, for long-term load forecasting, such as 15 days, SED-
we design ablation experiments targeting each module. former w/o T-Encoder exhibits a conspicuous performance
Experiments are conducted on four models: SEDformer, decline, indicating the necessity of T-Encoder for capturing
SEDformer without T-Encoder (SEDformer w/o T-Encoder), long temporal features and improving predictive capabil-
SEDformer without S-Encoder (SEDformer w/o S-Encoder), ity. It is also shown that S-Encoder helps extract implicit
and SEDformer without the feature aggregation module similar relationships among charging stations and enhance
(SEDformer w/o gating) using RMSE, MAE, and WMAPE predictive performance. The gap between SEDformer w/o
metrics. It is noted that if the model involves only a single gating and SEDformer increases with the extension of the

123
Journal of Intelligent & Robotic Systems (2024) 110:94 Page 9 of 11 94

Fig. 5 The ablation experiments

of SEDformer, SEDformer w/o
T-Encoder, SEDformer w/o
S-Encoder, and SEDformer w/o
gating models at different
temporal scales

123
94 Page 10 of 11 Journal of Intelligent & Robotic Systems (2024) 110:94

prediction horizon, suggesting the pivotal role of the aggre- Open Access This article is licensed under a Creative Commons
gation module in combining spatio-temporal information Attribution 4.0 International License, which permits use, sharing, adap-
tation, distribution and reproduction in any medium or format, as
in complex scenarios. As a data-driven approach, the col-
long as you give appropriate credit to the original author(s) and the
laborative dual-encoder architecture of the proposed model source, provide a link to the Creative Commons licence, and indi-
effectively explores the dependencies between long-range cate if changes were made. The images or other third party material
and cross-regional information. in this article are included in the article’s Creative Commons licence,
unless indicated otherwise in a credit line to the material. If material
is not included in the article’s Creative Commons licence and your
intended use is not permitted by statutory regulation or exceeds the
permitted use, you will need to obtain permission directly from the copy-
5 Conclusion right holder. To view a copy of this licence, visit http://creativecomm
ons.org/licenses/by/4.0/.

This paper introduces a novel Transformer-based deep learn-

ing model to address the EV load forecasting problem.
Specifically, a multi-encoder structure is designed to thor-
References
oughly explore the spatio-temporal characteristics of EV 1. Yong, J.Y., Ramachandaramurthy, V.K., Tan, K.M., Mithulanan-
charging load distribution and enhance prediction accuracy. than, N.: A review on the state-of-the-art technologies of electric
Further, to capture spatial similarity features among charging vehicle, its impacts and prospects. Renew. Sustain. Energy Rev. 49,
stations, after decomposing the input load sequences using 365–385 (2015)
2. Mei, P., Karimi, H.R., Xie, H., Chen, F., Huang, C., Yang, S.: A deep
the moving average strategy, a channel attention mechanism reinforcement learning approach to energy management control
is employed to extract multidimensional spatial information. with connected information for hybrid electric vehicles. Eng. Appl.
Finally, a fusion decoding mechanism is employed to select Artif. Intell. 123, 106239 (2023)
and integrate both temporal and spatial features. Experiments 3. Mei, P., Karimi, H.R., Yang, S., Xu, B., Huang, C.: An adaptive
fuzzy sliding-mode control for regenerative braking system of elec-
on real-world data spanning time scales of 7 days, 15 days, tric vehicles. Int. J. Adapt. Control Signal Process. 36(2), 391–410
and 30 days demonstrate that SEDformer excels at capturing (2022)
the dynamic nature of load variations and validate the effec- 4. Al-Ogaili, A.S., Hashim, T.J.T., Rahmat, N.A., Ramasamy, A.K.,
tiveness of the proposed model. The ablation studies validate Marsadek, M.B., Faisal, M., Hannan, M.A.: Review on schedul-
ing, clustering, and forecasting strategies for controlling electric
the impact of each module on the experimental outcomes vehicle charging: challenges and recommendations. IEEE Access
and their effectiveness in EV load forecasting. Our findings 7, 128353–128371 (2019)
can potentially contribute to optimizing power resources and 5. Khan, Z.A., Ullah, A., Haq, I.U., Hamdy, M., Mauro, G.M.,
expanding the EV market in practical applications. Muhammad, K., Hijji, M., Baik, S.W.: Efficient short-term electric-
ity load forecasting for effective energy management. Sustainable
In future work, we aim to gather a more comprehensive Energy Technol. Assess. 53, 102337 (2022)
dataset on EV charging loads to enhance the model’s general- 6. Ge, G., Tao, X., Yinghua, Z., Qiang, Z.: A survey of cooperative
ization capabilities for deployment in longer-scale prediction optimization of traffic-grid networks in the era of electric vehicles.
tasks. Moreover, our model has not fully capitalized on Control and Decision 36(09), 2049–2062 (2021)
7. Lee, T.-K., Bareket, Z., Gordon, T., Filipi, Z.S.: Stochastic model-
precise geographical information, such as coordinates. The ing for studies of real-world PHEV usage: driving schedule and
incorporation of dynamic graph neural networks may provide daily temporal distributions. IEEE Trans. Veh. Technol. 61(4),
a superior modeling approach for evolving charging station 1493–1502 (2011)
nodes. 8. Ikegami, T., Ogimoto, K., Yano, H., Kudo, K., Iguchi, H.: Balancing
power supply-demand by controlled charging of numerous electric
Author Contributions All authors contributed to the study and analysis. vehicles. In: 2012 IEEE International Electric Vehicle Conference,
Coding, Y. Chen; Writing- original draft, Y. Chen and Y. Wei; formal pp. 1–8 (2012)
analysis, Y. Wei and M. Wang; investigation, X. Huang and S. Gao; 9. Faridimehr, S., Venkatachalam, S., Chinnam, R.B.: A stochas-
writing- review and editing, Y. Wei and M. Wang. All authors have read tic programming approach for electric vehicle charging network
and agreed to the final manuscript. design. IEEE Trans. Intell. Transp. Syst. 20(5), 1870–1882 (2018)
10. Huang, C., Ding, W., Gao, R., Mei, P., Karimi, H.R.: Distributed
Funding This project is supported by National Key Research and Devel- state-of-charge estimation for lithium-ion batteries with random
opment Program of China (Grant/Award Number: 2021YFB2501600). sensor failure under dynamic event-triggering protocol. Inform.
Fusion 95, 293–305 (2023)
Data Availability All data analyzed during this study is available from 11. Liao, X., Kang, X., Li, M., Cao, N.: Short term load forecasting
the website of Open Data, City of Palo Alto [33]. and early warning of charging station based on PSO-SVM. In: 2019
International Conference On Intelligent Transportation, Big Data
& Smart City, pp. 305–308 (2019)
Declarations 12. Buzna, L., De Falco, P., Khormali, S., Proto, D., Straka, M.: Elec-
tric vehicle load forecasting: a comparison between time series
and machine learning approaches. In: 2019 1st International Con-
Competing Interests The authors have no competing interests to ference on Energy Transition in the Mediterranean Area, pp. 1–5
declare that are relevant to the content of this article. (2019)

123
Journal of Intelligent & Robotic Systems (2024) 110:94 Page 11 of 11 94

13. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural 33. CityofPaloAlto: Electric vehicle charging station usage (July
Comput. 9(8), 1735–1780 (1997) 2011 - Dec 2020). https://data.cityofpaloalto.org/dataviews/
14. Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., Woo, W.- 257812/electric-vehicle-charing-station-usage-july-2011-dec-
c.: Convolutional LSTM network: a machine learning approach 2020/ (2023)
for precipitation nowcasting. Adv. Neural Inform. Process. Syst. 34. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan,
28 (2015) G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch:
15. Liu, Y., Zheng, H., Feng, X., Chen, Z.: Short-term traffic flow pre- an imperative style, high-performance deep learning library. Adv.
diction with Conv-LSTM. In: 2017 9th International Conference on Neural Inform. Process. Syst. 32 (2019)
Wireless Communications and Signal Processing, pp. 1–6 (2017) 35. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization.
16. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature ArXiv:1412.6980 (2014)
521(7553), 436–444 (2015) 36. Hüttel, F.B., Peled, I., Rodrigues, F., Pereira, F.C.: Deep spatio-
17. Li, F., Feng, J., Yan, H., Jin, G., Yang, F., Sun, F., Jin, D., Li, Y.: temporal forecasting of electrical vehicle charging demand.
Dynamic graph convolutional recurrent network for traffic predic- ArXiv:2106.10940 (2021)
tion: benchmark and solution. ACM Trans. Knowl. Discov. Data
17, 1–21 (2023)
18. Cho, K., Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares,
F., Schwenk, H., Bengio, Y.: Learning phrase representations using Publisher’s Note Springer Nature remains neutral with regard to juris-
RNN encoder–decoder for statistical machine translation. In: Pro- dictional claims in published maps and institutional affiliations.
ceedings of the 2014 Conference on Empirical Methods in Natural
Language Processing, pp. 1724–1734 (2014)
19. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neu-
ral networks on graphs with fast localized spectral filtering. Adv. Yufan Chen received his B.S. degree from the School of Information
Neural Inform. Process. Syst. 29 (2016) and Control Engineering, China University of Mining and Technology,
20. Yu, B., Yin, H., Zhu, Z.: Spatio-temporal graph convolutional Xuzhou, China in 2022. He is currently pursuing his M.S. degree in
networks: a deep learning framework for traffic forecasting. the School of Automation at Southeast University. His current interests
ArXiv:1709.04875 (2017) include time series prediction and intelligent transportation systems.
21. Zhang, D., Peng, Y., Zhang, Y., Wu, D., Wang, H., Zhang, H.: Train
time delay prediction for high-speed train dispatching based on
spatio-temporal graph convolutional network. IEEE Trans. Intell.
Mengqin Wang received her B.S. degree from Changsha University
Transp. Syst. 23(3), 2434–2444 (2021)
of Science & Technology, Changsha, China in 2022. She is currently
22. Guo, S., Lin, Y., Feng, N., Song, C., Wan, H.: Attention based
pursuing her master’s degree in the School of Automation at South-
spatial-temporal graph convolutional networks for traffic flow fore-
east University. Her current research interest is electric vehicle routing
casting. In: Proceedings of the AAAI Conference on Artificial
problems with charging scheduling.
Intelligence, vol. 33, pp. 922–929 (2019)
23. Sun, H., Wei, Y., Huang, X., Gao, S., Song, Y.: Global spatio-
temporal dynamic capturing network-based traffic flow prediction.
IET Intell. Trans. Syst. (2023) Yanling Wei received the Ph.D. degree in control science and engineer-
24. Chen, L., Wu, L., Hong, R., Zhang, K., Wang, M.: Revisiting graph ing from the Harbin Institute of Technology, Harbin, China, in June
based collaborative filtering: A linear residual graph convolutional 2014. He was a Research Fellow with the Department of Electrical
network approach. In: Proceedings of the AAAI Conference on Engineering and Computer Science, Technical University of Berlin,
Artificial Intelligence, vol. 34, pp. 27–34 (2020) Berlin, Germany, and a Senior Research Fellow with Katholieke Uni-
25. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., versiteit Leuven (KU Leuven), Leuven, Belgium. He currently serves
Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. as a Full Professor with the School of Automation, Southeast Univer-
Advances in Neural Information Processing Systems 30 (2017) sity, Nanjing, China. His current research interests include collective
26. Basly, H., Zayene, M.A., Sayadi, F.E.: Spatiotemporal self- behaviors of complex networks, intelligent decision-making and con-
attention mechanism driven by 3d pose to guide rgb cues for daily trol, and biorobotics.
living human activity recognition. J. Intell. & Robot. Syst. 109(1),
2 (2023)
27. Wu, H., Xu, J., Wang, J., Long, M.: Autoformer: decomposition
Xueliang Huang received the B.S., M.S., and Ph.D. degrees in electri-
transformers with auto-correlation for long-term series forecasting.
cal engineering from Southeast University, Nanjing, China, in 1991,
Adv. Neural. Inf. Process. Syst. 34, 22419–22430 (2021)
1994, and 1997, respectively. From 2002 to 2004, he was a Postdoc-
28. Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., Zhang,
toral Researcher with the University of Tokyo. Since 2004, he has
W.: Informer: beyond efficient transformer for long sequence time-
been a Professor with the Electrical Engineering Department, South-
series forecasting. In: Proceedings of the AAAI Conference on
east University. His research interests include novel wireless power
Artificial Intelligence, vol. 35, pp. 11106–11115 (2021)
transfer systems, analysis of electromagnetic field, applied electro-
29. Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization.
magnetics, intelligent electricity technology, etc.
ArXiv:1607.06450 (2016)
30. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image
recognition. In: Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, pp. 770–778 (2016) Shan Gao received the Ph.D. degree from Southeast University, Nan-
31. Cleveland, R.B., Cleveland, W.S., McRae, J.E., Terpenning, I.: jing, China, in 2000. He is currently an Associate Professor with
STL: A seasonal-trend decomposition. J. Off. Stat. 6(1), 3–73 the School of Electrical Engineering, Southeast University, and the
(1990) Jiangsu Provincial Key Laboratory of Smart Grid Technology and
32. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Pro- Equipment, Southeast University. His research interests include power
ceedings of the IEEE Conference on Computer Vision and Pattern system planning and operation, renewable energy integration, and
Recognition, pp. 7132–7141 (2018) active distribution networks.

123