This article has been accepted for inclusion in a future issue of this journal.

Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1

Data-Driven Iterative Learning Model Predictive

Control With Self-Modified Prior Knowledge
Lele Ma , Xiangjie Liu , Furong Gao , and Kwang Y. Lee , Life Fellow, IEEE

Abstract— Iterative learning model predictive control (ILMPC) can be iteratively utilized for the control correction of
has become an excellent data-driven intelligent control strategy subsequent trials. With the embedded batch-wise iterative
for digitized batch manufacturing, featured by the progressive learning control (ILC) [6], [7], [8], the tracking performance
improvement of tracking performance along trials, and the
persistent rejection of stochastic disturbance along time. The of ILMPC can be improved by excavating the information
point-to-point learning mechanism of existing ILMPC generally contained in batch process data. In virtue of the time-domain
relies on identical operating conditions along trials to guarantee receding horizon optimization of model predictive control
the integrity and accuracy of historical data. However, the vari- (MPC), the stability and robustness of the ILMPC system are
ations of production requirements usually lead to trial-varying guaranteed under stochastic uncertainties.
operating references and durations, resulting in incomplete and
inaccurate historical information for the iterative learning of Over the past two decades, significant progress has been
subsequent trials. To promote the adaptability and flexibility of made in ILMPC within a wide range of research fields such
ILMPCs with unconformable prior information, a data-driven as nonlinear systems [9], disturbed systems [10], and uncertain
self-modification scheme is originally embedded into ILMPC systems [11]. However, ILMPC has always been considered
in this article to transfer the prior knowledge contained in as an intelligent control strategy relying on the strict identity
the historical operating data into the form consistent with the
condition of each current trial. The control actions are imitated of operating conditions due to its point-to-point learning
along trials by an adaptive deep neural network (DNN), which mechanism. In a typical batch process, a reference trajectory is
is then utilized to generate reference control signals for iterative usually generated via real-time optimization (RTO), which is
learning in each trial. For attenuating the influence of the con- then delivered to an ILMPC controller for tracking [12]. The
siderable DNN approximation error in early trials with limited
changes in product type/specification/cost and the volatility of
data accumulation, the 2-D optimization of ILMPC is performed
under a tube control frame to ensure the time-domain bounded environmental disturbance will cause the reference trajectory
stability. Based on the intrinsic recursive feasibility and the optimized by RTO to vary in both shape and length [13],
guaranteed time-domain stability, the iteration-domain bounded which can result in inaccurate and incomplete prior knowledge
convergence of the developed ILMPC system is theoretically for iterative learning. The classical ILMPC will be enforced
validated. Simulations on the nonlinear injection molding process
to reinitialize the iterations when the operating condition
verify the superiority of the proposed method in adapting to
significant changes in operating reference and duration. suffers significant disorders, indicating the suspension and
even retrogression of the learning progress.
Index Terms— 2-D optimization, iterative learning model
predictive control (ILMPC), self-modified prior knowledge, trial-
Concerning the nonrepetitive circumstances, many impres-
varying operating conditions. sive research efforts have been made in the area of
ILC-based batch process control, in which robust and adaptive
I. I NTRODUCTION control schemes are extensively integrated. Regarding the
iteration-varying reference trajectory shape, Chi et al. [14]

I TERATIVE learning model predictive control (ILMPC)

has received much attention as an intelligent 2-D con-
trol strategy for batch manufacturing [1], [2], [3], [4], [5].
originally developed an adaptive ILC strategy for discrete-time
higher order system, which directly incorporated the known
function of reference trajectory into an explicit adaptive ILC
Characterized by the finite-period repetitive operation, batch law through the recursive least squares (RLSs) algorithm.
processes accumulate control experiences along trials, which Under the similar frame, a variety of improved adaptive ILC
Manuscript received 17 May 2023; revised 21 November 2023 and 31 May laws have been sequentially raised for data-driven nonlinear
2024; accepted 29 August 2024. This work was supported in part by the systems [15], 2-D linear Fornasini–Marchesini systems [16],
National Natural Science Foundation of China under Grant 62203170 and and second-order nonlinear systems [17]. Different from the
Grant 62073136 and in part by the Fundamental Research Funds for
the Central Universities under Grant 2023JC002 and Grant 2023YQ002. parametric ILC methods, a state-space optimal ILC method
(Corresponding author: Xiangjie Liu.) was proposed in [18] for linear stochastic systems, in which
Lele Ma and Xiangjie Liu are with the State Key Laboratory of Alternate a batch-domain/time-domain Kalman filter was employed to
Electrical Power System with Renewable Energy Sources, North China Elec-
tric Power University, Beijing 102206, China (e-mail: [email protected]; realize the iterative state estimation in accordance with the
[email protected]). variations of the reference trajectory. For the particular case
Furong Gao is with the Department of Chemical and Biological Engineering, of initial state shifts, Jin [19] introduced a class of modifier
Hong Kong University of Science and Technology, Hong Kong (e-mail:
[email protected]). functions into the ILC law, which adjusted the reference
Kwang Y. Lee is with the Department of Electrical and Com- trajectory beyond a small initial time interval to accommodate
puter Engineering, Baylor University, Waco, TX 76798 USA (e-mail: the initial state misalignment. The above existing treatments
[email protected]).
Digital Object Identifier 10.1109/TNNLS.2024.3453380 used in the sole ILC rely on the explicit expression of learning
2162-237X © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

2 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS

laws, which cannot be extended to constrained ILMPC with Since batch processes accumulate operation data gradually
an implicit learning law. Regarding the reference variations as trial by trial, the DNN-based self-modification will suffer from
the external 2-D disturbance of batch processes, Liu et al. inadequate accuracy during the early trials due to insufficient
[20] embedded the H∞ control technique into the ILMPC control experience. The consequent uncertain approximation
to restrain the impacts of changing reference trajectory on error can degrade the tracking accuracy of ILMPC system
the tracking accuracy and first achieved the iteration-varying and also poses a barrier for theoretical convergence analysis.
reference tracking over the entire time domain under the To suppress the influence of the primitive DNN approximation
framework of constrained ILMPC. error on the subsequent trials, the robustness of ILMPC against
Against the issue of iteration-varying trial lengths, a static the DNN approximation error should be enhanced by utilizing
constant compensation method was primarily designed in [21], a disturbance-rejection scheme. Because the approximation
[22], and [23] to fill the incomplete operation data with zero, error tends to decrease along trials with accumulating train-
so as to adjust all trials into the same desired length. Shen and ing samples, it presents as a trial-varying external additive
Xu [24], Yu and Hou [25], Liu and Hou [26], and Shen et al. interference. As an excellent MPC strategy for coping with
[27] then replaced the compensated zero with the terminal bounded additive disturbance, tube MPC [34], [35] divides
value of each trial to avoid the tracking oscillation caused by the actual disturbed system into a separated nominal system
the abrupt return to zero. With increased emphasis on compen- and the external interference, which guarantees the bounded
sation accuracy, Lin et al. [28], Ma et al. [29], and Hou et al. stability of the actual system by restraining the actual dynamic
[30] turned to utilize a dynamic compensation method to trajectory within the designed invariants set around the nom-
fill the absent duration with neural-network-based predictive inal one. By extending the tube MPC stability theory to
values. Beyond the intuitive idea of data compensation, the the 2-D ILMPC, the bounded convergence of tracking error
upgrade of learning structure also becomes an alternative way along trials can be ensured by utilizing the 2-D recursive
to capacitate the progressive learning under iteration-varying feasibility and the tube invariant property. Meanwhile, the
trial lengths. Jin [31] proposed an indicator-based learning tube boundary can be regulated flexibly in accordance with
structure that enforces the system to learn from the nearest the renewed DNN approximation error of each current trial to
trial of comparative length. Ma et al. [29] further developed reduce the conservativeness. As the accuracy of DNN-based
an event-based switching learning structure to reconcile the self-modification increases with the growing available training
accuracy and the optimality of prior information by deter- samples, the effect of the tube scheme will diminish in stages
mining the learning order according to the direction of trial to achieve the optimality of ILMPC.
length variation. However, these works focus on coping with Concerning the incomplete and inaccurate historical infor-
the reference variations either in shape or length, which mation caused by varying operation conditions in batch
fail to guarantee the iteration-domain learning efficiency and processes, a data-driven tube ILMPC with self-modified prior
the time-domain tracking accuracy when the tendency and knowledge is proposed in this article. The contributions are in
duration of each trial change simultaneously. The tracking threefold.
deviations resulting from the noticeable variations of operating 1) To cope with the information incompatibility caused by
conditions can last for numerous trials and even cause the trial-varying operation conditions in the batch process,
divergence of control signal. an adaptive DNN-based approximator is designed to
Under the situation that significant differences in shape and transfer the prior knowledge into the form consistent
length exist between the operating trajectories along trials, the with the circumstance of the current trial.
original process data of historical trials cannot be directly used 2) For restraining the influence of DNN approximation
to iteratively update the ILMPC law of the current trial due error in early trials, a tube-based control framework is
to the noncorrespondence in system dynamics and time range. utilized in ILMPC to restrict the actual tracking error
It is thus critical to convert the prior knowledge contained within a predetermined invariant set.
in historical data into the form consistent with the operating 3) Based on the 2-D recursive feasibility and the tube
conditions of the current trial, which signifies creating a virtual invariant property, the bounded convergence of the
excellent example for iterative learning to substitute for the data-driven ILMPC system with self-modified prior
real historical trials used in classical ILMPC. The deep neural knowledge is guaranteed.
network (DNN) [32], [33], featured by extraordinary function The rest of this article is organized as follows. The DNN-
approximation abilities, can be applied to extract the deep based self-modification mechanism is presented in Section II.
features of prior operating knowledge and generate the new The tube ILMPC strategy is formulated in Section III, while
learning example for all operating points of the current trial. a convergence analysis is provided in Section IV. Section V
Through repetitive training with the control actions of ILMPC gives the simulation results of an injection molding process,
and the feedback from the controlled process, the DNN with the conclusions are drawn in Section VI.
approximator generates the control reference corresponding to II. DNN-BASED S ELF -M ODIFICATION U NDER VARYING
the target trajectory of each current trial for the implementation O PERATING C ONDITIONS
of ILMPC. Based on this self-modified prior knowledge with A. Problem Formulation
high compatibility, the valid updating of ILMPC can be
ensured despite the trial-varying operating conditions, thus Consider a typical nonlinear batch process
contributing to the convergence acceleration along trials. xk (t + 1) = f (xk (t), u k (t)) + ω(t) (1)

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MA et al.: DATA-DRIVEN ITERATIVE LEARNING MPC WITH SELF-MODIFIED PRIOR KNOWLEDGE 3

where k denotes the number of trials and t ∈ [0, Tk ) represents

the operating time with Tk denoting the length of the kth
trial. xk (t) ∈ X and u k (t) ∈ U are n x × 1 state vector and
n u × 1 input vector at time t of the kth trial, with X and
U denoting the state constraint set and input constraint set,
respectively. f (xk (t), u k (t)) represents the nonlinear system
dynamic satisfying Lipschitz continuity, and ω(t) denotes the
unknown repetitive disturbance. The control objective of the
batch process (1) is to track the reference state trajectory xkr (t)
over the time span (0, Tk ] in the current kth trial.
Define the tracking error as
xke (t) = xk (t) − xkr (t). (2)
Substituting (2) into (1) gives
xke (t + 1) = f xke (t) + xkr (t), u k (t)

+ ω(t) − xkr (t + 1), t ∈ [0, Tk ). (3)

The typical ILMPC utilizes the experience of the former
Fig. 1. Scheme of ILMPC with DNN-based self-modification.
trial to update the control law of the current trial. Therefore,
the iteration relationship between adjacent trials needs to be
explicitly described. According to (3), the system dynamic in
the (k − 1)th trial can be expressed as
e
(t + 1) = f xk−1
e
(t) + xk−1
r
(t), u k−1 (t) + ω(t)


xk−1
r
(t + 1), t ∈ 0, Tk−1 ).

− xk−1 (4)
Since the reference xkr (t) and the operation duration Tk vary
randomly with the trial index k, there are xkr (t) ̸ = xk−1
r
(t) and
Tk ̸ = Tk−1 . In consequence, an explicit iteration relationship
cannot be directly derived from (3) and (4), which indicates
that the operation experience of the previous trials does not
accord with the circumstance of the current trial. To address
this problem, it is essential to realize an accurate transfer
of the prior knowledge between trials to facilitate batch-wise
learning progress. In other words, a reference control signal Fig. 2. Architecture of DNN training for prior knowledge transformation.
u rk (t)(t ∈ [0, Tk )) needs to be solved according to the newly
acquired xkr (t) and Tk in each trial k. For this purpose, where W j and c j are the connection weight and the bias
an adaptive DNN is established to imitate the actions of the between the ( j − 1)th hidden layer and the jth hidden layer,
ILMPC controller and generate the prerequisite u rk (t) for the respectively.
online 2-D optimization. The relationship between the DNN- The network output is calculated by
based self-modification and the ILMPC control is depicted in
Y = N X ; W̄ = Wo σ (W L σ (W L−1 σ (· · · ) + c L−1 ) + c L ) + co


Fig. 1.
(6)
B. DNN for Prior Knowledge Transformation where W̄ = [W1 , c1 , W2 , c2 , . . . ,W L , c L , Wo , co ], with Wo and
The DNN training architecture for prior knowledge trans- co being the connection weight and the bias between the Lth
formation is shown in Fig. 2, where the network inputs are hidden layer and the output layer, respectively.
composed of a current state and its next state, while the output In accordance with the limited and nonstationary sample
is the corresponding control signal. At the current kth trial, data of batch processes, an extended Kalman filter (EKF) [36]
the state data and input data of the recent l trials are gathered is adopted to train the DNN in Fig. 2. At the beginning of
to formulate the training sample set, which is composed of l the (k + 1)th trial, the network weight matrix W̄ is updated
small-scale discontinuous datasets with different time spans. adaptively by
The input sample is then expressed as X k = [xi (t)xi (t + 1)]T
W̄ k+1 = W̄ k − K k E k
and the output sample is given as Yk∗ = u i (t), with i =
−1
k − l, k − l + 1, . . . , k − 1 and t = 0, 1, . . . , Ti − 1. T
K k = P̄ k JFk R N k + JFk P̄ k JFk
With σ (·) denoting the activation function, the output of the T


P̄ k+1 = Q N k + 1 − K k JFk P̄ k (7)
jth hidden layer h j ( j = 1, 2, . . . , L) in DNN is computed by
where K k is the Kalman filter gain; P̄ k is the covariance
h j = σ W j h j−1 + c j


(5) matrix of estimation error; Q N k and R N k = r N k I represent

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS

the covariance matrices of artificial process noise and mea- of ILMPC. Based on (12) and (13), the tube strategy is
surement noise, respectively; JFk denotes the Jacobian matrix incorporated into nonlinear ILMPC to restrain the influence
obtained through orderly taking derivatives of the network of the DNN approximation error.
output Yk with respect to W̄ k ; and E k is the modeling error Define dk (t) = x̂ rk (t + 1) − xkr (t + 1). Based on (1), (14),
matrix formulated as and (15), the batch-wise incremental model can be expressed
as
E k = Yk − Yk∗ (8)
xk (t + 1) = f a (xk (t), u k (t)) + dk (t) (17)
where Yk = N (X k ; W̄ k ). The covariance matrix Q N k can be
updated using the maximum likelihood method to promote the where
training convergence speed
  f a (xk (t), u k (t)) = xkr (t + 1) + f e (xk (t), u k (t))
k
1  X  
QNk = 1W̄ j 1W̄ Tj + P̄ k − P̄ k−n w (9) with f e (xk (t), u k (t)) = f (xk (t), u k (t)) − f (xkr (t), û rk (t)), and
nw 
dk (t) can be regarded as a bounded disturbance satisfying

j=k−n w +1
|dk (t)| ≤ ςkx .
where 1W̄ j = W̄ j − W̄ j−1 and n w is the selected window
The nominal model of (17) is correspondingly given as
size for updating Q N .
Referring to [36, Th. 1], if the following condition is z k (t + 1) = f a (z k (t), vk (t)) (18)
satisfied:
! where z k (t) and vk (t) denote the nominal state and input,
1 3Yk |E k |2
, when |E k | > 192ζk . (10)
p
rNk = + respectively.
2 no 64n o On account of the nominal model (18), a nominal ILMPC
controller is designed to drive z k (t) to the reference trajectory
The weight matrix W̄ k subject to the EKF-based updating (7)
xkr (t) through receding horizon 2-D optimization. An auxiliary
converges to the optimum along trials. Here, in (10), n o is the
controller is then utilized to maintain xk (t) in the tube invariant
number of neurons in the output section, ζk is the trace of
T set around the reference trajectory xkr (t) by minimizing the
JFk P̄ k JFk , and |·| denotes the Euclidean norm.
deviation between the actual system (17) and the nominal
With the network input given as X k = [xkr (t)xkr (t + 1)]T ,
system (18).
the well-trained DNN is then used to generate an approximate
reference control signal û rk (t)(t ∈ [0, Tk )) for the kth trial
through the following equation: A. Nominal ILMPC
û rk (t) = Yk = N xkr (t), xkr (t + 1); W̄ k The nominal control input vk (t) is composed of two vari-


(11)
ables: the ILC control signal vkILC (t) and the MPC control
which is taken as the prior value of control signal when signal vkMPC (t), which can be expressed as
performing ILMPC optimization in the kth trial.
vk (t) = vkILC (t) + vkMPC (t). (19)
III. T UBE ILMPC S TRATEGY W ITH S ELF -M ODIFIED
At the interval between the (k − 1)th trial and the kth trial,
P RIOR K NOWLEDGE
the following ILC optimization problem is solved offline:
Since the DNN shown in Fig. 2 can approximate any
2 2
nonlinear mapping to arbitrary accuracy with sufficient hidden min JkILC = zek Q
+ 1vILC
k R
1vILC
units, there must exist a bound ςku such that the approximation k

s.t. z k (t + 1) = f a z k (t), û rk (t) + 1vkILC (t)

error of the reference control signal satisfies (20)

û rk (t) − u rk (t) ≤ ςku . (12) where

z k (1) − xkr (1)
 
In virtue of the Lipschitz continuity, it can be derived
from (1) that  z k (2) − xkr (2) 
zek =  ..
 
.

x̂ rk (t + 1) − xkr (t + 1) ≤ ςkx (13)  
z k (Tk ) − xkr (Tk )
where
vkILC (0) − û rk (0)
 

x̂ rk (t + 1) = f xkr (t), û rk (t) + ω(t) vkILC (1) − û rk (1)

(14)  
1vILC = ..
 
xkr (t + 1) = f xkr (t), u rk (t) + ω(t) k


.

(15)  
ςkx ≤ L u û rk (t) − u rk (t) ≤ L u ςku (16) vkILC (Tk − 1) − û rk (Tk − 1)

with L u signifying the Lipschitz constant. and ∥·∥2Q and ∥·∥2R represent the quadratic forms with Q
When the available historical data are limited in early and R being the weight matrices, respectively. The relative
trials, the insufficient accuracy of the DNN approximator magnitudes of Q and R decide the importance levels of
will lead to a relatively large error bound ςku , which can tracking precision and control smoothness in the process of
result in considerable damage to the convergence performance optimizing.

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MA et al.: DATA-DRIVEN ITERATIVE LEARNING MPC WITH SELF-MODIFIED PRIOR KNOWLEDGE 5

At each sampling time t, the MPC functions through the and V f (xke (Tk )) = xke (Tk )T P f xke (Tk ) (P f is positive definite)
following receding horizon optimization problem: is the terminal cost with the corresponding control invariant
2 2 set given as X f = {xke |V f (xke ) ≤ α}(α > 0), which satisfies
Tk
min JkMPC (t) = zek t+1 Q
+ δvk |tTk −1 n
2
o
min V f′ ( f e (xk , u k ))+ xke Q 1 +∥δu k ∥2R1 xke ∈ X f ≤ V f′ xke


vMPC
k |tTk −1 R
u k ∈U
s.t. z k (t + 1) = f a z k (t), vkILC (t) + vkMPC (t)


V f = βV f′ (β ≥ 1) (24)
z k ∈ Z, vk ∈ V, z ke (Tk ) = 0 (21)
for any xke ∈ X f , where Q 1 and R1 are diagonal matrices
where intercepted from Q and R, respectively; β is a constant
for justifying the omission of terminal constraint; and δu k
z k (t + 1) − xkr (t + 1)
 
 z k (t + 2) − xkr (t + 2)  represents the time-wise increment of u k . Here, β should
Tk be appropriately chosen to reach a compromise between the
zek = ..
 
t+1
.

  stability performance and the tracking accuracy.
z k (Tk ) − xk (Tk )
r
The auxiliary optimization problem is then formulated as

vk (t) − vk (t − 1)
 follows to solve the actual control input u k (t):
 vk (t + 1) − vk (t) 
min JkAU (t) (25)
δvk |tTk −1 = ..
  T −1
uk |t k
.

 
vk (Tk − 1) − vk (Tk − 2) subject to (22) and u k ∈ U.
The first element of uk |tTk −1 is taken as the current control
and V ⊆ U and Z ⊆ X are tightened constraint sets of the signal to be implemented on the controlled process, expressed
nominal input v and state z, respectively. as u k (t) = ϕ(xk (t), t).
From a general point of view, the data-driven tube ILMPC
B. Auxiliary Controller with prior knowledge self-modification is comprised of both
offline and online procedures in each trial as follows.
An auxiliary controller is utilized to drive the actual state
Step 1: Set k = 1 and t = 0.
xk (t) close to the nominal state z k (t), which is developed from
Offline:
the following composite system:
Step 2: Update the parameters of the DNN approximator
through (7) on account of the operation data collected from
(
xk (t + 1) = f a (xk (t), u k (t))
(22) the former l trials. Generate the reference control signal û rk
z k (t + 1) = f a (z k (t), vk (t))
based on (11).
where the disturbance dk (t) is omitted from the actual uncer- Step 3: Solve vkILC from the nominal ILC optimization
tain system (17). By designing a stabilizing controller for the problem (20).
composite system (22), the deviation between the trajectories Step 4: Solve vkMPC (t) from the MPC optimization prob-
of the actual uncertain systems (17) and the nominal sys- lem (21).
tem (18) will be accordingly restrained within a limited range Step 5: Compute the nominal control signal vk (t) based
decided by dk (t) owing to the resultant stability property. on (19). Apply vk (t) to the nominal system (18). If t < Tk −1,
Since the system z k (t + 1) = f a (z k (t), vk (t)) has already set t = t + 1, and go to Step 4. Otherwise, go to the online
been stabilized by the nominal ILMPC in Section III-A, the steps and set t = 0.
target of stabilizing the composite system (22) turns to be Online:
minimizing the distance between the dynamic trajectories of Step 6: Solve u k (t) from the auxiliary optimization
the two systems in (22). Thus, the objective function of the problem (25).
auxiliary controller is formulated as Step 7: Impose u k (t) on the actual controlled system (1).
If t < Tk − 1, and go to Step 6 and set t = t + 1. Otherwise,
Tk Tk 2 2
JkAU (t) = xek t+1
− zek t+1 Q
+ uk |tTk −1 − vk |tTk −1 go to Step 2 and set k = k + 1.
R
+ V f xke (Tk )


(23) IV. P ROPERTIES OF DATA -D RIVEN T UBE ILMPC W ITH
S ELF -M ODIFICATION
where
A. Time-Domain Stability
xk (t + 1) − xkr (t + 1)
 
 xk (t + 2) − xkr (t + 2)  The time-domain stability is a prerequisite for the
Tk
xek = .. iteration-domain convergence of ILMPC system, which is
 
t+1
.

  directly related to the end-product quality and produc-
xk (Tk ) − xk (Tk )
r
tion security. By treating the DNN approximation error as
u k (t) an external disturbance, the time-domain stability of the
 
 u k (t + 1)  proposed ILMPC can be analyzed theoretically based on
uk |tTk −1 = .. the mature tube control theory. Some rational assumptions
 
.

are presented for the nominal system (18) and the actual
 
u k (Tk − 1) system (17).

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS

Assumption 1:
1) The nonlinear function f e (·) is twice continuously dif-
ferentiable.
2) The state constraint set X and the input constraint set U
are compact subsets of R n x and R n u , respectively.
3) The DNN approximation error set D derived from
|dk (t)| ≤ ςkx is compact and convex, which contains
Fig. 3. Tracking trajectories along time under tube-based ILMPC.
the origin.
4) The cost functions JkILC , JkMPC (t), and JkAU (t) are invariant set around the optimum. When the approximation
quadratic and positive definite. error bound ςku approaches zero through repetitive training
5) The linearized system of (18) can be stabilized along based on accumulated operation data, the actual control signal
the reference trajectory. u k (t) will eventually converge to the optimum along trials.
6) There exist b > h ≥ h ∗ and β ≥ max{b/α, 1}, Based on the inherent 2-D recursive feasibility of the opti-
where b > 0, h ∈ (0, b), h ∗ = b3 ςkx /(1 − γ ), and mization problems (20) and (21), the bounded convergence of
γ = 1−b1 /b2 . (The definitions of coefficients b1 , b2 , the established ILMPC system is reached by utilizing the tube
and b3 can be found in [34].) invariant set.
7) There exist tightened constraint sets Z ⊆ X and V ⊆ U Theorem 2: The nominal control input vk (t) solved accord-
for the nominal system (18) such that, for any feasible ing to (20) and (21) converges to the optimum along trials.
∗
initial state xk (0), Skh (xk (t)) ⊂ X holds for each t ∈ Proof: It is evident that 1vILC = 0 is a feasible solution
h∗ k
[0, Tk ], where Sk (xk (t)) = {xk (t)|JkAU (t) ≤ h ∗ }. for the ILC optimization problem (20). Representing the
Theorem 1: If Assumption 1 is satisfied, the data-driven optimal nominal ILC control signal in the kth trial with vILC∗ k ,
tube ILMPC system is robust and stable, that is, for each the following inequality must hold:
feasible initial state xk (0), the error trajectory and the control
 r
JkILC vILC∗ ILC


k ≤ Jk ûk (29)
trajectory of the actual system (17) lie in the state tube
Tek = {Skh (xke (t)) = {xke (t)|JkAU (t) ≤ h}, t ≥ 0} and the control r
where ûk = [û rk (0)û rk (1), . . . , û rk (Tk−1 )]T .
tube Tuk = {ϕ(Skh (xk (t)), t), t ≥ 0}, respectively. Substituting vILC
r
= ûk into (20), there is
k
Proof: Based on [34, Corollary 1], it can be inferred that  r  r 2
JkILC ûk = zek ûk . (30)
JkAU (t + 1) ≤ γ JkAU (t) + b3 ςkx (26) Q
Combining (20), (29), and (30) results in
for all t ≥ 0, xke (t) ∈ Skb (t) = {xke |JkAU (t) ≤ b}, and
xke (t + 1) ∈ f e (xk (t), ϕ(xk (t), t)) + dk (t). Therefore, if b3 ςkx + 2

2 2
 r
zek vILC∗
k + 1vILC∗
k ≤ zek ûk . (31)
ε ≤ (1 − γ )JkAU (t), there will be Q R Q

JkAU (t + 1) ≤ γ JkAU (t) + b3 ςkx ≤ JkAU (t) − ε. (27) Assuming that vMPC∗k,t |tTk −1 is the optimal solution of the
MPC optimization problem (21) at time t of the kth trial, the
With ε = 0, the following inequality can be derived corresponding objective function is expressed as
from (27) with JkAU (t) ≤ h and b3 ςkx ≤ (1 − γ )JkAU (t) ≤     2 2
Tk
(1 − γ )h: JkMPC vMPC∗
k,t
Tk −1
t = zek t+1 vMPC∗
k,t
Tk −1
t + δvk |tTk −1
Q R
JkAU (t + 1) ≤ JkAU (t) ≤ h. (28) = z ke (t
+ 1) +
2
∥δvk (t)∥2R
Q
 
Therefore, it is proved that the state tube Tek
and the control + JkMPC vMPC∗
k,t
Tk −1
(32)
t+1
tube Tuk are both invariant for the actual system (17), that is,
the actual tracking error trajectory will remain in the tube Tek , where JkMPC (vMPC∗
k,t
Tk −1
|t+1 ) = Tk
∥zek |t+2 (vMPC∗
k,t |t+1 )∥ Q +
Tk −1 2

while the corresponding actual input trajectory stays within the Tk −1 2

∥δvk |t+1 ∥ R .
tube Tuk . As the approximation error of the DNN approaches Thus, it can be deduced from (32) that
zero, indicating that h ∗ → 0, both tubes will contract to the
  
2
JkMPC vMPC∗
k,t
Tk −1
t ≥ z ke (t + 1) Q
+ JkMPC vMPC∗
k,t
Tk −1
t+1 .
nominal trajectories. As shown in Fig. 3 (with n x = 2), the
actual tracking trajectory belongs to the original constraint set, (33)
while the nominal one is restricted in the tightened constraint Tk −1
In the same manner, vMPC∗
k,t must be a feasible solution
|t+1
set.
to the MPC optimization problem (21) at time t + 1. With
Tk −1
vMPC∗
k,t+1 |t+1 denoting the optimal solution at time t + 1, the
B. Iteration-Domain Convergence
following inequality holds:
The iteration-domain convergence is a critical property for    
evaluating the closed-loop performance of ILMPC along trials. JkMPC vMPC∗ Tk −1
k,t+1 t+1 ≤ JkMPC vMPC∗
k,t
Tk −1
t+1 . (34)
The convergence property of the proposed data-driven tube Incorporating (33) with (34) makes
ILMPC is highly associated with the accuracy of the DNN 2
   
approximator. In the early trials with considerable error bound z ke (t + 1) Q + JkMPC vMPC∗ Tk −1
k,t+1 t+1 ≤ Jk
MPC
vMPC∗
k,t
Tk −1
t .
ςku , the actual control signal u k (t) converges to the tube (35)

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MA et al.: DATA-DRIVEN ITERATIVE LEARNING MPC WITH SELF-MODIFIED PRIOR KNOWLEDGE 7

Because vMPC
k |0Tk −1 = 0 is a feasible solution at t = 0, it is
concluded that
 
2
JkMPC vMPC∗
k,0
Tk −1
t+1 ≤ JkMPC (0) = zek vILC∗
k Q
. (36)

It is derived from the recursive calculation of (35) and (36)

that

2
2
zek v∗k Q ≤ zek vILC∗
k Q
(37)

where v∗k = vILC∗

k +vMPC∗
k represents the optimal nominal input
vector.
From (31) and (37), one can get

2
2  r 2
zek v∗k Q ≤ zek vILC∗
k Q
≤ zek ûk . (38)
Q Fig. 4. Tracking error convergence of the tube ILMPC along trials.
r
As illustrated in Section II, ûk
denotes the prior knowledge
transformed from the previous trials. In the nominal system
analysis, by regarding the disturbance dk (t) as zero, the term
r
∥zek (ûk )∥2Q can completely represent the learning progress until
the (k − 1)th trial. Therefore, (38) can be regarded as

2  r 2
2
lim zek v∗k Q ≤ lim zek ûk = lim zek v∗k−1 Q (39)
k→∞ k→∞ Q k→∞

which means that the limit of ∥zek (v∗k )∥2Q must exist.
Applying the squeeze rule to (39), there is

2
2  r 2
lim zek vILC∗
k Q
= lim zek v∗k Q = lim zek ûk .
k→∞ k→∞ k→∞ Q Fig. 5. Varying reference trajectories from the first trial to the eighth trial.
(40)
bound of the DNN approximation error ςku can significantly
Incorporating (31) into (40) gives affect the convergence property by influencing the form of
 r 2
2 2 tube invariant set. The increase of the DNN approximation
lim 1vILC∗ e
zek vILC∗


≤ lim z ûk − lim =0
k→∞
k R k
k→∞ Q k→∞
k Q error will aggravate divergence of u k (t) due to the enlarged
(41) boundary of tube invariant set.
Fig. 4 depicts the tracking error trajectory of the tube
where limk→∞ ∥1vILC∗ k ∥2R = 0 implies that the nominal ILMPC system with n x = 3. The green trajectory represents
control vector vk (t) converges to the optimum along trials. the nominal tracking error, while the red one denotes the actual
Theorem 3: The actual control vector u k (t) solved through tracking error. As the trial index increases, with the DNN
the auxiliary optimization problem (25) converges to the parameter matrix W̄ k converging to the optimum according
control tube invariant set Tuk centered on the optimum along to (10), the DNN approximation error bound ςku will finally
trials. decrease to zero, resulting in h ∗ → 0. In consequence,
Proof: In accordance with the time-domain stability the actual tracking error trajectory will eventually coincide
results in Section IV-A, the actual control vector u k (t) remains with the nominal one and approach the actual optimum after
within the control tube Tuk around the nominal control vector adequate trials. At this stage, the auxiliary optimization prob-
vk (t). With vk (t) gradually approaching the optimum vk∗ (t), lem (25) can be omitted since the nominal system (18) turns
the actual control vector u k (t) will finally converge to the tube equivalent to the actual one.
Tuk∗ = ϕ Skh xk∗
 e
(t) , t , t ≥ 0



(42) Remark 1: When h ∗ finally reaches zero, the ILC optimiza-
tion problem (20) will be unavailing because the reference
where control signal û rk generated by the DNN approximator turns
∗
equal to the optimal ILC signal u ILC . In other words, the ILC
e
xk∗ (t) ∈ Tek∗ k
 controller can be fully replaced by the DNN approximator.
Tk Tk 2 Tk −1 2
e
Tk∗ = xk∗ e
xek∗ − vek∗ + uk |tTk −1 − vek∗ In the meantime, the MPC optimization (21) will be kept
t+1 t+1 Q t R
 effective to resist real-time disturbance and guarantee system
stability.
e
(Tk ) ≤ h, t ≥ 0 .


+V f xk∗
V. S IMULATIONS
Based on Theorems 1 and 3, the convergence performance In the presented simulation verification, a typical nonlinear
of the actual ILMPC system is closely related to the tube injection molding process [37] is studied. In the filling phase
boundary h. With h ≥ h ∗ = b3 ςkx /(1 − γ ) and ςkx ≤ L u ςku , the of the injection molding, the injection velocity needs to track

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

8 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS

a given trajectory with rather high accuracy, which plays a molding process from the second trial on, to drive the state
crucial role in final product quality. The various product spec- xk (t) to the reference xkr (t) in each trial. For initial trials, the
ifications and inaccuracy transmissions can lead to batch-wise network structure of DNN is determined through an empirical
reference variations in both shape and length, making it estimation, with the number of hidden layers set as 3 and
challenging to progressively improve tracking accuracy along the number of neurons in each hidden layer set as 5. Until
trials through point-to-point learning. the seventh trial, the preferable network hyperparameters are
The nonlinear dynamics in the filling phase can be described solved using a genetic algorithm based on historical operation
by the following differential equations [38]: data, where the number of hidden layers is 3 and the number
of neurons in each hidden layer is 6. At the beginning of
βh
 
d Ph dZ


 = q h − A h , each trial k, the parameter matrix W̄ k of the DNN approxi-
 dt Vh dt

mator is updated according to (7), where the estimation error


dvz 1
= (Ph Ah − Pn An − f v ) (43) covariance matrix P̄ k is retrieved from the modeling result
dt M
of the former trial, and the process noise covariance matrix
 n
(s − 1)vz
 
 dZ


 = vz , f v = 2πη Rn1−n (l0 + Z ) 1−s (Q N k ) and the measurement noise covariance matrix (R N k ) are
dt kr − 1
calculated by the adaptive equations (9) and (10), respectively.
where Ph is the hydraulic pressure, vz is the injection velocity, The sliding window size for DNN training and the adaption
Z is the screw displacement, and qh is the hydraulic flowrate. window size of Q N are selected as l = n w = 2 using a
The physical interpretations and values of the rest parameters correlation analysis based on historical operating data, which
can be found in [39]. can be further adjusted through trial and error along batches.
Define the system state as x = [x1 x2 x3 ]T = [Ph vz Z ]T The reference trajectory xkr is then input to the trained DNN
and the input as u = qh . With a sampling time of 0.25 s, the in sequence to generate the reference flowrate signal û rk ,
discrete state-space model of (43) is given as as shown in (11). The approximation error bound ςku in (12)
is redetermined in each trial based on the testing results of the
xk (t + 1) = f (xk (t), u k (t)) + ω (44)
adaptive DNN.
where By selecting the weight matrices as Q = I3Tk ×3Tk and
R1 = 0.005ITk ×Tk , the nominal ILC and MPC optimiza-
f (xk (t), u k (t)) = Axk (t) + Bu k (t) + g xk,2 (t)


tion problems (20) and (21) are solved offline to obtain
the nominal control input vk (t). For the second trial, the
   
1 −0.236 0 0.2225
A =  1.0805 1 0 , B =  0  tightened constraints are designed by employing the Monte
0 1 1 0 Carlo method [34] to simulate the trajectory fluctuations with
T sufficient samples of dk (t), which is finally obtained as
g xk,2 (t) = 0 − 4.3059xk,2 0 0.7



ω = [0 31.3187 0]T .
5.0 ≤ u k (t) ≤ 95.0
The input constraints on the hydraulic flowrate are given as −18.3 ≤ 1u k (t) ≤ 18.3
0 ≤ u k (t) ≤ 100 −9.8 ≤ δu k (t) ≤ 9.8 (46)
−20 ≤ 1u k (t) ≤ 20
−10 ≤ δu k (t) ≤ 10 (45) where δu k (t) = u k (t) − u k (t − 1), and the tube boundary is
computed as h = h ∗ = 3.67. With an increasing trial index
where δu k (t) = u k (t) − u k (t − 1). k, the constraint (46) is progressively relaxed and the tube
The reference trajectory xkr is assumed to change with boundary h contracts accordingly.
the production requirements and the external environmental The actual control vector u k (t) is solved online through
interferences in each trial. Fig. 5 displays the iteration-varying the auxiliary optimization problem (25) and then implemented
xkr (t) from the first trial to the eighth trial, where the trial in the injection molding process (43) at each sampling point
lengths are set as 80, 75, 85, 75, 85, 80, 75, and 85. The sixth t ∈ [0, Tk ).
trial and the eighth trial desire a low injection velocity, while The trajectories of injection velocity vz and hydraulic
the fourth and fifth trials demand a high injection velocity. The flowrate qh under the proposed data-driven tube ILMPC with
first–third trials and the seventh trial are supposed to maintain self-modification are depicted in Figs. 6 and 7, respectively,
a medium velocity. The reference velocity beyond the eight while the DNN approximation results are provided in Fig. 8.
trial varies periodically in the order of the first eight trials. With the trial index increasing from 1 to 16, the DNN approx-
The initial state of each trial is xk (0) = [−26.5 0 18]T . The imation error decreases gradually, and the tracking accuracy of
initial input trajectory is set as u 0 (t) = 30 (t ∈ [0, 80)), while injection velocity improves significantly, despite the changing
the initial reference trajectory keeps identical to the first trial. reference shapes and lengths. The DNN approximator supports
A classical nonlinear ILMPC is implemented in the first trial the ILMPC controller with transferred prior knowledge to
to generate the optimal control policy for training the DNN overcome the nonrepetitive operating conditions along trials.
approximator. The tube strategy embedded in ILMPC helps to guarantee
Based on model (44), the data-driven tube ILMPC strategy system stability by restraining the influence of the DNN
with DNN-based self-modification is applied to the injection approximation error. The tube invariant set contracts trial by

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MA et al.: DATA-DRIVEN ITERATIVE LEARNING MPC WITH SELF-MODIFIED PRIOR KNOWLEDGE 9

Fig. 7. Hydraulic flowrate qh under the tube ILMPC with self-modification

from the 1st trial to the 16th trial. (a) qh from the first trial to the eighth trial.
(b) qh from the 9th trial to the 16th trial.

Fig. 6. Injection velocity vz under tube ILMPC with self-modification

from the 1st trial to the 16th trial. (a) vz from the first trial to the eighth To validate the superiority of the DNN-based prior knowl-
trial (solid lines—actual trajectories and dashed lines—reference trajectories). edge self-modification in facilitating the learning progress of
(b) vz from the 9th trial to the 16th trial (solid lines—actual trajectories and
dashed lines—reference trajectories).
ILMPC under iteration-varying operation conditions, compar-
isons are made with the widely used compensation methods
trial due to the enhanced precision of the DNN approximator, for dealing with variable trial lengths, including the constant
as displayed in Fig. 9. compensation [21] and the predictive compensation [28]. The
The tendency of the tracking root-mean-square error constant compensation fills the deficient data with the same
(RMSE) along trials is presented in Fig. 10, where the RMSE constant value to adjust prior information into the same length.
of each trial is defined as In contrast, the predictive compensation uses model-based
v predictive data to complement the unavailable interval. The
u Tk
u1 X data-driven tube ILMPC strategy is applied under the pro-
RMSEk = t x e (i)2 . (47) posed DNN-based self-modification scheme and the above two
Tk i=1 k
compensation methods, with the initial conditions and weight
The simulation result suggests that when the trial index matrices kept the same. The comparative tracking trajectories
grows to 20, with the DNN approximation error bound ςku of the 9th–12th trials are displayed in Fig. 12, with the
approaching zero, the nominal trajectory starts to coincide with corresponding tracking RMSE shown in Table I.
the actual one, which signifies that the ILC optimization (20) Under the same initial conditions and controller design, the
and the auxiliary optimization (25) become unavailing. From tracking error when adopting the DNN-based self-modification
the 25th trial on, the control signal maintains at the optimum, is obviously smaller than that using the compensation mech-
indicating that the minimization of tracking error is achieved anism, especially in the trials suffering from insufficient
under the various types of reference trajectories. prior knowledge, such as the 3rd, 5th, 8th, 9th, and 11th
Fig. 11 depicts the comparison modeling error and tracking trials. The tracking RMSE under the DNN-based self-
error in the seventh trial with different settings of the num- modification decreases progressively, while that under the
ber of hidden layers and neurons, where the initial network compensation mechanism is trapped in the periodical fluctu-
structure remains the same from the second trial to the sixth ations. By adopting the dynamic compensation scheme with
trial. It is observed that the hyperparameters solved through a higher precision, the tracking RMSE under the predictive
the genetic algorithm contribute to the lowest modeling error. compensation decreases more quickly than that under the
With a much smaller DNN approximation error bound ςku , the constant compensation, showing stronger adaptability against
tracking error under the tube ILMPC is accordingly reduced the changing trial lengths. However, due to failure in dealing
due to the contracted tube invariant set. with the iteration-varying shape of the reference trajectory,

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

10 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS

Fig. 10. Average tracking error along trials under the tube ILMPC with
self-modification.

Fig. 8. Approximation results of the DNN from the 1st trial to the 16th trial.
(a) 2nd trial. (b) 3rd trial. (c) 7th trial. (d) 8th trial. (e) 15th trial. (f) 16th
trial.

Fig. 11. Comparison of modeling error and tracking error. (a) Modeling
RMSE and (b) tracking RMSE under different numbers of hidden layers and
neurons.

optimizations (20) and (21) is directly implemented on the

injection molding system, which resembles the control scheme
Fig. 9. Contracting tube invariant sets at 7.5 s along trials. in [40]. The comparative tracking trajectories of the third–sixth
trials, during which the reference trajectory varies notably both
the tracking performance under the predictive compensation in shape and length, are plotted in Fig. 13. The corresponding
can hardly match that under the DNN-based self-modification. tracking RMSE and the average CPU time of online optimiza-
Owing to the accurate knowledge transfer between successive tion (collected from a personal computer with an Intel Core
trials with varying operation trajectories and durations, the i7 CPU and 8-GB RAM) are given in Table II.
DNN-based self-modification contributes to avoiding severe Fig. 13 indicates that the tube ILMPC has a significantly
trajectory deviations and helps realizing the rapid convergence higher tracking accuracy than classical ILMPC in early trials
along trials. with considerable DNN approximation error. Table II also
Moreover, the proposed tube ILMPC is compared with reveals that the tracking RMSE of the tube ILMPC is restricted
the classical ILMPC under the same DNN-based self- within an acceptable range in the first eight trials due to its
modification, to verify its effectiveness in avoiding the excellent robustness against the DNN approximation error,
divergence of tracking error in early trials. In the classical while the intense increase of RMSE intermittently occurs
ILMPC, the nominal control signal vk (t) solved through under the classical ILMPC. Since the main steps of the

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MA et al.: DATA-DRIVEN ITERATIVE LEARNING MPC WITH SELF-MODIFIED PRIOR KNOWLEDGE 11

Fig. 12. Comparing tracking trajectories of the 9th–12th trials between the
DNN-based self-modification and the compensation methods. (a) 9th trial. Fig. 13. Comparing tracking trajectories of the third–sixth trials between
(b) 10th trial. (c) 11th trial. (d) 12th trial. tube-based ILMPC and classical ILMPC. (a) 3rd trial. (b) 4th trial. (c) 5th
trial. (d) 6th trial.
TABLE I
RMSE OF T UBE ILMPC U NDER DNN-BASED S ELF -M ODIFICATION
AND C OMPENSATION M ETHODS

TABLE II
C OMPARING T RACKING RMSE AND CPU T IME OF T UBE
ILMPC AND C LASSICAL ILMPC

Fig. 14. Comparison of the proposed ILMPC and the sole ILC: (a) ILMPC
and (b) ILC under state disturbance [0.3 0.5 0]T in the third–sixth trials.
(a) ILMPC (solid lines—actual trajectories and dashed lines—reference tra-
tube ILMPC are accomplished through offline procedures at jectories). (b) ILC (solid lines—actual trajectories and dashed lines—reference
trajectories).
the interval of trials, the online CPU time of both ILMPC
strategies maintains at a similar level.
Furthermore, a comparative simulation between the pro- law u k (t) = vkILC (t) is imposed on the injection molding
posed ILMPC and the sole ILC is conducted based on the system in each trial. Fig. 14 shows the comparing tracking
same self-modified prior knowledge, where the optimal ILC trajectories of the two methods along trials, where a state

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

12 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS

disturbance of [0.3 0.5 0]T acts on the controlled system [5] L. Ma, X. Liu, X. Kong, and K. Y. Lee, “Iterative learning model
from 6.25 to 10 s since the third trial. The results indicate predictive control based on iterative data-driven modeling,” IEEE Trans.
Neural Netw. Learn. Syst., vol. 32, no. 8, pp. 3377–3390, Aug. 2021.
that the tracking convergence of ILC strategy is severely [6] S. Arimoto, S. Kawamura, and F. Miyazaki, “Bettering operation
deteriorated by the abrupt state disturbance since its open-loop of robots by learning,” J. Robot. Syst., vol. 1, no. 2, pp. 123–140,
control constructure fails to guarantee time-domain stability Mar. 1984.
[7] X. Li, D. Shen, and B. Ding, “Iterative learning control for output
when the nonrepetitive disturbance appears in the third trial. tracking of nonlinear systems with unavailable state information,” IEEE
In contrast, the proposed ILMPC method can quickly restrain Trans. Neural Netw. Learn. Syst., vol. 33, no. 9, pp. 5085–5092,
the disturbance within the third trial it occurs so that the Sep. 2022.
tracking accuracy of the subsequent trials is not significantly [8] X. Yu, Z. Hou, M. M. Polycarpou, and L. Duan, “Data-driven iterative
learning control for nonlinear discrete-time MIMO systems,” IEEE
affected. Besides, the convergence rate of the proposed ILMPC Trans. Neural Netw. Learn. Syst., vol. 32, no. 3, pp. 1136–1148,
is also obviously higher than that of ILC during early trials Mar. 2021.
since the integration of the tube MPC scheme contributes to [9] X. Liu, L. Ma, X. Kong, and K. Y. Lee, “An efficient iterative learning
predictive functional control for nonlinear batch processes,” IEEE Trans.
less dependence on the accuracy of the DNN approximator. Cybern., vol. 52, no. 6, pp. 4147–4160, Jun. 2022.
In summary, the proposed data-driven tube ILMPC with [10] U. Rosolia, X. Zhang, and F. Borrelli, “Robust learning model-predictive
DNN-based self-modification has evident superiority in cop- control for linear systems performing iterative tasks,” IEEE Trans.
Autom. Control, vol. 67, no. 2, pp. 856–869, Feb. 2022.
ing with trial-varying operation conditions, which facilitates
[11] D. Li et al., “Synthesis of ILC–MPC controller with data-driven
improving the production benefits of batch manufacturing such approach for constrained batch processes,” IEEE Trans. Ind. Electron.,
as injection molding process. vol. 67, no. 4, pp. 3116–3125, Apr. 2020.
[12] P.-C. Lu, J. Chen, and L. Xie, “Iterative learning control (ILC)-based
economic optimization for batch processes using helpful disturbance
VI. C ONCLUSION information,” Ind. Eng. Chem. Res., vol. 57, no. 10, pp. 3717–3731,
Feb. 2018.
Aimed at relieving the classical ILMPC strategy from the [13] J. H. Lee and K. S. Lee, “Iterative learning control applied to
restriction of identical operating conditions, a data-driven tube batch processes: An overview,” Control Eng. Pract., vol. 15, no. 10,
ILMPC scheme with DNN-based self-modification is proposed pp. 1306–1318, Oct. 2007.
in this article to learn from the unconformable historical [14] R. Chi, Z. Hou, and J. Xu, “Adaptive ILC for a class of discrete-time
systems with iteration-varying trajectory and random initial condition,”
data. Based on an adaptive DNN approximator, the effective Automatica, vol. 44, no. 8, pp. 2207–2213, Aug. 2008.
conversion of prior knowledge is achieved between successive [15] R. Chi, Z. Hou, and S. Jin, “A data-driven adaptive ILC for a
trials with significant differences in both shape and length class of nonlinear discrete-time systems with random initial states and
iteration-varying target trajectory,” J. Franklin Inst., vol. 352, no. 6,
of reference trajectories. On account of the embedded tube pp. 2407–2424, Jun. 2015.
framework, the influence of the inevitable DNN approximation [16] K. Wan and X.-D. Li, “High-order internal model-based iterative
error in early trials is restrained within the designed tube learning control for 2-D linear FMMI systems with iteration-varying
trajectory tracking,” IEEE Trans. Syst., Man, Cybern., Syst., vol. 51,
invariant set. The simulation results on a typical injection no. 3, pp. 1462–1472, Mar. 2021.
molding process verify that the proposed ILMPC strategy can [17] C. E. Boudjedir, M. Bouri, and D. Boukhetala, “Model-free iterative
guarantee high-accuracy tracking under significant variations learning control with nonrepetitive trajectories for second-order MIMO
in the amplitude and duration of reference trajectories with an nonlinear systems—Application to a delta robot,” IEEE Trans. Ind.
Electron., vol. 68, no. 8, pp. 7433–7443, Aug. 2021.
ensured iteration-wise convergence. [18] S.-K. Oh and J. M. Lee, “Stochastic iterative learning control for discrete
The tube-based ILMPC scheme contributes to the bounded linear time-invariant system with batch-varying reference trajectories,”
convergence in early trials with a noticeable DNN approx- J. Process Control, vol. 36, pp. 64–78, Dec. 2015.
imation error. Nevertheless, as the accuracy of the DNN [19] X. Jin, “Fault tolerant nonrepetitive trajectory tracking for MIMO output
constrained nonlinear systems using iterative learning control,” IEEE
approximator increases with accumulated historical data, the Trans. Cybern., vol. 49, no. 8, pp. 3180–3190, Aug. 2019.
issue of excessive intertrial optimization burden will emerge [20] X. Liu, L. Ma, X. Kong, and K. Y. Lee, “Robust model predictive iter-
gradually. Establishing an explicit iterative relationship of the ative learning control for iteration-varying-reference batch processes,”
IEEE Trans. Syst., Man, Cybern., Syst., vol. 51, no. 7, pp. 4238–4250,
nominal ILMPC solution along trials can be a future research Jul. 2021.
direction to improve computational efficiency. A breaking [21] X. Li, J.-X. Xu, and D. Huang, “An iterative learning control approach
criterion is also anticipated to be developed to completely for linear systems with randomly varying trial lengths,” IEEE Trans.
Autom. Control, vol. 59, no. 7, pp. 1954–1960, Jul. 2014.
replace the tube ILMPC with the typical ILMPC once the [22] D. Shen, W. Zhang, Y. Wang, and C.-J. Chien, “On almost sure and mean
DNN approximator reaches sufficient precision. square convergence of P-type ILC under randomly varying iteration
lengths,” Automatica, vol. 63, pp. 359–365, Jan. 2016.
[23] D. Meng and J. Zhang, “Deterministic convergence for learning control
R EFERENCES systems over iteration-dependent tracking intervals,” IEEE Trans. Neural
[1] K. S. Lee, I.-S. Chin, H. J. Lee, and J. H. Lee, “Model predictive control Netw. Learn. Syst., vol. 29, no. 8, pp. 3885–3892, Aug. 2018.
technique combined with iterative learning for batch processes,” AIChE [24] D. Shen and J. Xu, “Adaptive learning control for nonlinear systems with
J., vol. 45, no. 10, pp. 2175–2187, Oct. 1999. randomly varying iteration lengths,” IEEE Trans. Neural Netw. Learn.
[2] J. Shi, F. Gao, and T.-J. Wu, “Single-cycle and multi-cycle generalized Syst., vol. 30, no. 4, pp. 1119–1132, Apr. 2019.
2D model predictive iterative learning control (2D-GPILC) schemes [25] Q. Yu and Z. Hou, “Adaptive fuzzy iterative learning control for
for batch processes,” J. Process Control, vol. 17, no. 9, pp. 715–727, high-speed trains with both randomly varying operation lengths and sys-
Oct. 2007. tem constraints,” IEEE Trans. Fuzzy Syst., vol. 29, no. 8, pp. 2408–2418,
[3] S.-K. Oh and J. M. Lee, “Iterative learning model predictive control for Aug. 2021.
constrained multivariable control of batch processes,” Comput. Chem. [26] G. Liu and Z. Hou, “Adaptive iterative learning fault-tolerant control
Eng., vol. 93, pp. 284–292, Oct. 2016. for state constrained nonlinear systems with randomly varying itera-
[4] S.-K. Oh, B. J. Park, and J. M. Lee, “Point-to-point iterative learning tion lengths,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 2,
model predictive control,” Automatica, vol. 89, pp. 135–143, Mar. 2018. pp. 1735–1749, Feb. 2024.

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

MA et al.: DATA-DRIVEN ITERATIVE LEARNING MPC WITH SELF-MODIFIED PRIOR KNOWLEDGE 13

[27] M. Shen, X. Wu, J. H. Park, Y. Yi, and Y. Sun, “Iterative learning Xiangjie Liu received the Ph.D. degree in automatic
control of constrained systems with varying trial lengths under alignment control from the Research Center of Automation,
condition,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 9, Northeastern University, Shenyang, China, in 1997.
pp. 6670–6676, Sep. 2023. He subsequently held a post-doctoral position with
[28] N. Lin, R. Chi, and B. Huang, “Auxiliary predictive compensation-based China Electric Power Research Institute (CEPRI),
ILC for variable pass lengths,” IEEE Trans. Syst., Man, Cybern., Syst., Beijing, China, until 1999, and acted as an Associate
vol. 51, no. 7, pp. 4048–4056, Jul. 2021. Professor with CEPRI. He was a Research Associate
[29] L. Ma, X. Liu, F. Gao, and K. Y. Lee, “Event-based switching with The University of Hong Kong, Hong Kong, and
iterative learning model predictive control for batch processes with a Professor with the National University of Mexico,
randomly varying trial lengths,” IEEE Trans. Cybern., vol. 53, no. 12, Mexico City, Mexico. He is currently a Professor
pp. 7881–7894, Dec. 2023. with North China Electric Power University, Beijing.
[30] R. Hou, L. Jia, X. Bu, and C. Zhou, “Dynamic neural network His current research interests include model predictive control with its
predictive compensation-based point-to-point iterative learning control application in industrial processes.
with nonuniform batch length,” IEEE Trans. Neural Netw. Learn. Syst., Prof. Liu is a member of the Technical Committee on both control theory
vol. 35, no. 9, pp. 13005–13016, Sep. 2024. and process control at Chinese Association of Automation. He is an Editor of
[31] X. Jin, “Iterative learning control for MIMO nonlinear systems with Acta Automatica Sinica, Journal of Control and Decision, and Chinese Journal
iteration-varying trial lengths using modified composite energy func- of Electric Power Automation Equipment. He has been in the Program for
tion analysis,” IEEE Trans. Cybern., vol. 51, no. 12, pp. 6080–6090, New Century Excellent Talents at the University since 2006. He was awarded
Dec. 2021. Beijing Distinguished Teacher Winner in 2019.
[32] G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of
data with neural networks,” Science, vol. 313, no. 5786, pp. 504–507,
Jul. 2006.
[33] R. Chai, A. Tsourdos, A. Savvaris, S. Chai, Y. Xia, and C. L. P. Chen, Furong Gao received the B.Eng. degree in automa-
“Design and implementation of deep neural network-based control for tion from China University of Petroleum, Beijing,
automatic parking maneuver process,” IEEE Trans. Neural Netw. Learn. China, in 1985, and the M.Eng. and Ph.D. degrees
Syst., vol. 33, no. 4, pp. 1400–1413, Apr. 2022. in chemical engineering from McGill Univer-
[34] D. Q. Mayne, E. C. Kerrigan, E. J. van Wyk, and P. Falugi, “Tube-based sity, Montreal, QC, Canada, in 1989 and 1993,
robust nonlinear model predictive control,” Int. J. Robust Nonlinear respectively.
Control, vol. 21, no. 11, pp. 1341–1353, 2011. He was a Senior Research Engineer with Mold-
[35] M. Cannon, J. Buerger, B. Kouvaritakis, and S. Rakovic, “Robust tubes flow International Company Ltd., Melbourne, VIC,
in nonlinear model predictive control,” IEEE Trans. Autom. Control, Australia. Since 1995, he has been with Hong Kong
vol. 56, no. 8, pp. 1942–1947, Aug. 2011. University of Science and Technology, Hong Kong,
where he is currently the Chair Professor with the
[36] X. Wang and Y. Huang, “Convergence study in extended Kalman
Department of Chemical and Biological Engineering. Meanwhile, he holds
filter-based training of recurrent neural networks,” IEEE Trans. Neural
positions, including the Director of the Center for Polymer Processing
Netw., vol. 22, no. 4, pp. 588–600, Apr. 2011.
and Systems, the Director of the Institute of Advanced Manufacturing and
[37] Y. Wang, D. Zhou, and F. Gao, “Iterative learning model predictive Automation, and the Vice President of the Fok Ying Tung Research Institute
control for multi-phase batch processes,” J. Process Control, vol. 18, (FYTRI). His current research interests include process monitoring and control
no. 6, pp. 543–557, Jul. 2008. and polymer processing.
[38] J. Lu, Z. Cao, and F. Gao, “Multipoint iterative learning model predictive Prof. Gao was elected as a fellow of the Society of Plastics Engineers (SPE)
control,” IEEE Trans. Ind. Electron., vol. 66, no. 8, pp. 6230–6240, for his great contributions in the field of quality control and optimization for
Aug. 2019. batch processes in 2010. He is the President of the Society of Advanced
[39] M. Rafizadeh, W. I. Patterson, and M. R. Kamal, “Physically-based Molding Technology and the Vice President of Guangdong Association of
model of thermoplastics injection molding for control applications,” Int. Automation.
Polym. Process., vol. 11, no. 4, pp. 352–362, Dec. 1996.
[40] L. Jia, C. Han, and M.-S. Chiu, “Dynamic R-parameter based integrated
model predictive iterative learning control for batch processes,” J. Pro-
cess Control, vol. 49, pp. 26–35, Jan. 2017. Kwang Y. Lee (Life Fellow, IEEE) received the B.S.
degree in electrical engineering from Seoul National
University, Seoul, South Korea, in 1964, the M.S.
degree in electrical engineering from North Dakota
State University, Fargo, ND, USA, in 1968, and
the Ph.D. degree in systems science from Michigan
Lele Ma received the B.S. degree in automation and State University, East Lansing, MI, USA, in 1971.
the Ph.D. degree in control theory and control engi- He has been working in the area of power plants
neering from North China Electric Power University, and power systems control for over 30 years at
Beijing, China, in 2016 and 2021, respectively. Michigan State University; Oregon State University,
She was with Baylor University, Waco, TX, USA, Corvallis, OR, USA; the University of Houston,
as a Visiting Scholar, from 2019 to 2020. She is Houston, TX, USA; The Pennsylvania State University; and Baylor University,
currently an Associate Professor with North China Waco, TX, USA, where he is a Professor and the Chairperson of the
Electric Power University. Her research interests Department of Electrical and Computer Engineering. His research interests
include iterative learning model predictive control include control, operation, and planning of energy systems; computational
and its application in industrial processes. intelligence, intelligent control and their applications to energy and envi-
Dr. Ma won the “Outstanding Student Paper ronmental systems; and modeling, simulation, and control of renewable and
Award” in the 30th China Process Control Conference and was awarded as distributed energy sources.
the “Excellent Ph.D. Student” by Beijing Association of Automation in 2020. Dr. Lee was elected as a fellow of IEEE in January 2001 for his contributions
She was selected for the Young Talent Lifting Project held by Beijing Science to the development and implementation of intelligent system techniques for
and Technology Association from 2023 to 2025. power plants and power systems control.

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on March 09,2025 at 10:55:47 UTC from IEEE Xplore. Restrictions apply.