This article has been accepted for inclusion in a future issue of this journal.

Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON CYBERNETICS 1

Neural Network-Based Passivity Control

of Teleoperation System Under
Time-Varying Delays
Da Sun, Fazel Naghdy, and Haiping Du

Abstract—In this paper, a novel neural network (NN)-based ent environmental conditions. Ideal transparency means that
four-channel wave-based time domain passivity approach (TDPA) the medium between the operator and the environment is
is proposed for a teleoperation system with time-varying delays. not felt and the dynamics of the master and the slave are
The designed wave-based TDPA aims to robustly guarantee the
channels passivity and provide higher transparency than the pre- canceled out.
vious power-based TDPA. The applied NN is used to estimate With the network technologies advancing at a staggering
and eliminate the system’s dynamic uncertainties. The system rate, teleoperation can be conducted by using commercially
stability with linearity assumption on human and environment available communication networks. When the local and remote
has been analyzed using Lyapunov method. The proposed algo- platforms are connected via commercial networks, the for-
rithm is validated through experimental work based on a 3-DOF
bilateral teleoperation platform in the presence of different time ward and feedback control signals between the master and
delays. the slave will be inevitably associated with time delays. In
remote control and manipulation, without proper control algo-
Index Terms—Bilateral teleoperation, neural network (NN),
passivity, time domain passivity approach (TDPA), time-varying rithms, even a small time delay may destabilize and degrade
delays, wave variable. the tracking performance of a teleoperation system. Numerous
methods have been proposed to balance the tradeoff between
the system stability and transparency in the presence of time
delays. A system designed by Lee and Spong [8] uses direct
I. I NTRODUCTION
position feedback to eliminate position drift. Nuño et al. [9]
N THE last five decades, teleoperation technologies have
I been widely applied and developed all around the world.
Teleoperation system is defined as electromechanical mech-
deployed P-like, PD-like and scattering controllers to ana-
lyze the stability of the nonlinear teleoperation systems with
the classic assumptions of passivity. Later, they introduce
anism that extends human’s sensing, decision making, and a general Lyapunov-like function to unify stability analysis
manipulation capability to the remote environment. A conven- on the passivity-based control for the nonlinear teleoperation
tional teleoperation system consists of the human operator, systems [10]. An adaptive coordination control law based on
the master robot, the communication networks, the slave the scattering approach is introduced by Chopra et al. [11] to
robot, and the environmental tasks. Teleoperation systems have ensure position synchronization in the nonlinear teleoperation
numerous applications ranging from space exploration [1], systems. Yang et al. [12] designed a new fuzzy PD-like con-
underwater operation [2], mining [3], and nuclear reactor [4] troller to deal with uncertainties of system dynamics. However,
where human operators are protected from dangerous situ- all of the P-like and PD-like systems require preset dampers
ations, to medical training [5], rehabilitation [6], and min- with constant gains associated with the value of time delays
imally invasive surgery [7] where a patient suffers less to guarantee the system stability by reducing transparency.
trauma through key-hole surgery. A teleoperation system is Due to different types of time delays, these methods may be
called unilateral if only the master’s control signals are trans- over-conservative in some situations.
mitted to the slave side. If there exists the motion or force In recent times, the neural networks (NNs) have attracted
feedback from the slave side to the master, this system can much attention due to their prominent properties such as learn-
be called bilateral. Bilateral teleoperation is assessed through ing capability mapping and parallel processing. NNs have been
the two critical indices of stability and transparency. Stability deployed in the control of the robotic systems and have sig-
requires the closed loop system to be stable under differ- nificantly improved their performance [13], [14]. In bilateral
teleoperation research, a control system with acceleration mea-
Manuscript received January 4, 2016; revised March 21, 2016 and surement is designed in [15] using NN to estimate nonlinear
April 6, 2016; accepted April 12, 2016. This paper was recommended by uncertainties. In [16], the NN is applied in a prescribed perfor-
Associate Editor P. X. Liu.
The authors are with the Faculty of Engineering and Information Sciences, mance control system. A terminal sliding mode control system
University of Wollongong, Wollongong, NSW 2500, Australia (e-mail: with NN is also designed in [17]. These systems, however,
[email protected]). perform only under extremely restricting assumptions that the
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org. time delay is constant and the external force is zero, both
Digital Object Identifier 10.1109/TCYB.2016.2554630 of which are against the reality of these systems. In [18],
2168-2267 c 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

2 IEEE TRANSACTIONS ON CYBERNETICS

NN is deployed for systems with time-varying delays, but the delays without rate restrictions. Compared with the previous
approach requires precise knowledge of the external force as power-based TDPA, the proposed wave-based TDPA can more
well as the coefficients of mass and damper of the external efficiently monitor the power flows under the condition of
force. References [19] and [20] extend the application of NN arbitrary time delays. The proposed passivity controllers do
to multilateral teleoperation. The major drawbacks of these not influence position and force tracking. NN is applied to
studies are some of the assumptions underlying them. For the proposed system to estimate and eliminate the dynamic
example, the positive constraints including disturbances are uncertainties. The proposed control algorithm is deployed in
assumed to be restricted by large velocity signals, and the rate the absence of the knowledge of the upper bound of the
of time delays must be less than one. In [21], the state-of-art NN approximation error and external disturbance. Lyapunov
NN control systems is reviewed. functions are used to prove the system stability. Finally, the
The idea of passivity characterized by mechanical energy, experimental work is performed to show the effectiveness of
which uses force and velocity as efforts and flow variables, proposed control system in comparison with other systems in
is an effective tool for establishing stability of bilateral tele- different scenarios.
operation interaction under time delays. Compared with the The remainder of this paper is structured as follows.
methods based on absolute stability, most passivity-based After providing a background in Section II on the dynam-
methods are more conservative and sacrifice system trans- ics of teleoperation system and its related properties, and the
parency, but can easily accommodate communication time knowledge of the radial basis function (RBF) NN, the pro-
delays. Among the numerous passivity-based approaches, the posed four-channel (4-CH) wave-based TDPA is described
wave variable method, introduced by Niemeyer and Slotine, is in Section III. In Section IV, the NN-based controller
a classic approach to guarantee the time delayed channel pas- design is introduced and the delay-based stability is also
sivity. However, the traditional wave variable transformation studied. Results of the experimental work are presented in
has many drawbacks. For example, it can hardly guaran- Section V. Section VI draws some conclusions.
tee the system stability when time delay varies. In addition,
the two intrinsic problems in a wave variable system, posi- II. BACKGROUND
tion drift and wave reflections can cause inaccurate position
tracking and large signal variations, respectively. Numerous A. Model of Teleoperation System
approaches have been proposed to overcome one or some of In this paper, the local (master) and the remote (slave)
these shortcomings [22]–[30]. Nevertheless, according to the robots are modeled as a pair of n-DOF serial links with
literature, none of the previous work addresses all the problems revolute joints. Their corresponding nonlinear dynamics are
associated with the wave variable transformation. Specially, modeled as
the time-varying delay issue is still the main drawback of the
Mm (qm )q̈m + Cm (qm , q̇m )q̇m + Fm q̇m + fcm (q̇m ) + gm (qm )
previous work on wave-based systems.
∗
Another classic passivity-based approach is time domain − Fm = τm + τh (1)
∗
passivity approach (TDPA) that was first introduced by Ms (qs )q̈s + Cs (qs , q̇s )q̇s + Fs q̇s + fcs (q̇s ) + gs (qs ) − Fs
Ryu et al. [31], consisting of passivity observers and pas- = τs − τe (2)
sivity controllers, to adaptively dissipate energy. The pas-
sivity observers are used to monitor the channel passivity where i = m, s for the master and slave. q̈i (t), q̇i (t), and
and the passivity controllers are used to dissipate the active qi (t) ∈ Rn×1 are the joint acceleration, velocity, and posi-
energy. This method is later extended in [32] to deal with tion, respectively. Mi (qi (t)) ∈ Rn×n are the inertia matrices
the time-varying delay issues. In [33] and [34], the energy- and Ci (qi (t), q̇i (t)) ∈ Rn×n are Coriolis/centrifugal effects.
based TDPA is extended to the power-based TDPA, which gi (qi (t)) ∈ Rn are the vectors of gravitational forces and τi
can dissipates energy as soon as any active energy is produced. are the control signals. τh (t) and τe (t) are the actual human
The power-based TDPA proposed in [34] is further extended and environmental torques applied to the robots. Fi q̇i (t) denote
in [35] to tackle the position drift issue. However, although the the viscous friction and fci (q̇i (t)) denote the Coulomb fric-
above TDPA-based system are capable to guarantee channels tion. Fi∗ (t) ∈ Rn×1 are the bounded unknown disturbances.
passivity under time-varying delays, transparency degradation In this paper, the Coulomb friction function fci (q̇i (t)) on the
is still their main drawback, especially in the presence of small master and slave sides are bounded and piecewise continuous
constant or no time delay where high transparency can be functions.
easily derived by many nonpassivity-based schemes. In [36], Important properties of the above nonlinear dynamic model,
we proposed a wave-based TDPA system to more accurately which are used in this paper, are as follows.
observe the power flow during different time delay scenarios. 1) P1: The inertia matrix Mi (qi ) for a manipulator is sym-
However, assumption of the rate of time delays less than one metric positive-definite as: 0 < σmin (Mi (qi (t)))I ≤
and degraded position and torque tracking owning to passivity Mi (qi (t)) ≤ σmax (Mi (qi (t)))I ≤ ∞, where I ∈ Rn×n
controllers in the presence of sharply-varying delays were two is the identity matrix. σ min and σmax denote the strictly
major weaknesses of that method. positive minimum (maximum) eigenvalue of Mi for all
In this paper, a new wave-based TDPA system is proposed to configurations qi .
guarantee the communication channels’ passivity and achieve 2) P2: Under an appropriate definition of the
high tracking performance in the presence of time-varying Coriolis/centrifugal matrix, the matrix Ṁi − 2Ci is
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

SUN et al.: NN-BASED PASSIVITY CONTROL OF TELEOPERATION SYSTEM UNDER TIME-VARYING DELAYS 3

skew symmetric, which can also be expressed as which can be written as: f (X) = f1 (X) + f2 (X), where f1 (X) is
the continuous part and f2 (X) is the bounded piecewise term,
Ṁi (qm (t)) = Ci (qi (t), q̇i (t)) + CiT (qi (t), q̇i (t)). (3) respectively. Therefore
3) P3: For a manipulator with revolute joints, there exists
f (X) = W ∗T (X) + ξ ∗ (X) + f2 (X) = W ∗T (X) + ξ̄ ∗ (X)
a positive constant Z bounding the Coriolis/centrifugal
matrix as (10)

Ci (qi (t), x(t))y(t)2 ≤ Zx(t)2 y(t)2 . (4) where ξ̄ ∗ (X) = ξ ∗ (X) + f2 (X), ξ̄ ∗ (X) ≤ ξ̄up
∗ . ξ̄ ∗ is the upper
up
bound of the approximation error.
4) P4: The time derivative of Ci (qi (t), q̇i (t)) is bounded if
qi (t) and q̇i (t) are bounded.
III. WAVE -BASED TDPA
In this paper, the external human and environmental torques
are modeled as (5) and (6), where τh,e ∗ (t) stand for, respec- The system passivity in a traditional power-based
tively, the positive and bounded human operator and the TDPA system can be defined as [34], [35]
environment exogenous input. Kh,e , Bh,e , and Mh,e represent 1 T
the non-negative constant scalars corresponding to the mass, P(t) = τm (t)q̇m (t) − τs (t)q̇s (t) = τ (t)τm (t)
2b m
damping, and stiffness of human and environment. kh,e , 1
bh,e , and mh,e are the unknown bounded variables relat- −(τm (t) − bq̇m (t))2 + bq̇Ts (t)q̇s (t)
2b
ing to Kh,e , Bh,e , and Mh,e . Moreover, we use the extended 1 1
active observer (EAOB) to measure the human and environ- − (τs (t) + bq̇s (t))2 − Ṫ2 (t)τmT (t)τm (t)
2b 2b
mental torques as well as acceleration signals [26]. Compared b dE dE
with other force observers, EAOB possesses the advantage of − Ṫ1 (t)q̇s (t)q̇s (t) +
T
= Pdiss + (11)
2 dt dt
external noise suppression by deploying Kalman filter, and is t t
suitable for nonlinear systems 1 b
E(t) = τs (η)τs (η)dη +
T
q̇Tm (η)q̇m (η)dη
2b 2
τh (t) = τh∗ (t) − (Kh + kh )qm (t) − (Bh + bh )q̇m (t) t−T2 (t) t−T1 (t)
− (Mh + mh )q̈m (t) (5) (12)
τe (t) = τe∗ (t) + (Ke
+ ke )qs (t) + (Be + be )q̇s (t) where b is a positive constant that relates to the unit of torque
+ (Me + me )q̈s (t). (6) and velocity. T1 (t) and T2 (t) are the forward and backward
time delays, respectively. Since Pdiss is not observable at any
B. Neural Networks single port of the 2-port network, in order to facilitate real-time
monitoring of the network’s passivity, Pdiss can be written as
The main advantage of the NNs is its ability to approximate
any smooth nonlinear function with arbitrary precision due to Pdiss (t) = Pm
diss (t) + Pdiss (t)
s
(13)
its inherent approximate capabilities [15], [38]. In this paper,
where Pm diss (t) and Pdiss (t) are the power dissipation com-
s
the RBF NN is applied to approximate a continuous function
f (X) : Rq → Rp expressed as ponents which are observable at the master and slave ports,
respectively
f (X) = W T (X) + ξ(X) (7)
1 T 1
where X ∈ x ⊂ Rq is the input vector. W ∈ Rn×p is the diss (t) =
Pm τ (t)τm (t) − (τm (t) − bq̇m (t))2
b m 2b
weight matrix. n is the number of the neurons. ξ(X) is the 1
− Ṫ2 (t)τmT (t)τm (t) (14)
approximation errors. 2b
(X) = [1 (X), 2 (X), . . . , k (X), . . . , n (X)], where 1
Psdiss (t) = bq̇Ts (t)q̇s (t) − (τs (t) + bq̇s (t))2
k (X) is the RBF Gaussian function 2b
  b
1 − Ṫ1 (t)q̇s (t)q̇s (t).
T
(15)
k (X) = exp − 2 X − Ck  2
(8) 2
2Hk
Ṫ1,2 is replaced by constant parameters μ̄1,2 in [34] and [35].
where Ck and Hk are the center and the width of the kth Their values are set to be the upper bound of Ṫ1,2 . The
neuron, respectively. According to the universal approximation passivity observers on the master and the slave side can be
property of NNs, for any continuous function f (X), there exists written as
an NN such that 1 T 1
  Pmobs (t) = τm (t)τm (t) − (τm (t) − bq̇m (t))2
f (X) = W ∗T (X) + ξ ∗ (X), ξ ∗ (X) ≤ ξup
∗
(9) b 2b
1
where W ∗ and ξ ∗ (X) are the ideal weight and error in the − μ̄2 τmT (t)τm (t) (16)
∗ is ξ ∗ (X)’s upper bound. The
2b
approximation, respectively. ξup 1
dynamic functions of the manipulators can be considered to Psobs (t) = bq̇Ts (t)q̇s (t) − (τs (t) + bq̇s (t))2
2b
be piecewise continuous functions due to the existing fric- b
tion and backlash. Assume that f (X) is a piecewise function − μ̄1 q̇Ts (t)q̇s (t). (17)
2
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON CYBERNETICS

signals IA2 and IB2 , where VA1 (t) = βδqm (t) + αδ q̇m (t) +
γ δ q̈m (t), VB1 (t) = α1 δ q̇m (t) + γ1 δ q̈m (t), IA2 (t) = α1 δ q̇s (t) +
γ1 δ q̈m (t), IB2 (t) = −βδqs (t) − αδ q̇s (t) − γ δ q̈s (t). α, α1 , γ , γ1 ,
and β, are diagonal positive-definite matrices. δ is a positive
constant. In this system, position, velocity and acceleration are
transmitted between the two robots.
The wave variables in the two schemes are defined as
follows:
b1 VA1 (t) + λ11 IA2 (t − T2 (t))
um1 (t) = √
2b1
b1 VA2 (t) + λ11 IA2 (t)
us1 (t) = √ (20)
2b1
Fig. 1. 4-CH wave variable transformation. IA2 (t − T2 (t)) IA2 (t)
vm1 (t) = √ , vs1 (t) = √ (21)
2b1 2b1
By applying the passivity observers, the power flows can be b2 VB1 (t) b2 VB1 (t − T1 (t))
um2 (t) = √ , us2 (t) = √ (22)
detected in each port. Two passivity controllers attached at 2b2 2b2
each port are activated when Pm s
obs and Pobs are negative so that b2
Pctr = −Pobs and Pctr = −Pobs , where Pm
m m s s s λ2 VB1 (t) − IB1 (t)
ctr and Pctr are the vm2 (t) = √
dissipated power from the passivity controllers. By using the 2b2
two passivity controllers, the torque perceived by the operator b2
λ2 VB1 (t − T1 (t)) − IB2 (t)
τm (t) and the command velocity of slave q̇s (t) can be derived vs2 (t) = √ (23)
as [34] 2b2
where b1,2 and λ1,2 are the positive characteristic impedances.
τm (t) = τs (t − T2 (t)) + τPC (t) (18) The traditional wave√ variable transformation is √ written as
q̇s (t) = q̇m (t − T1 (t)) − q̇PC (t) (19) um (t) = −vm (t) + 2bq̇m (t), vs (t) = −us (t) + 2/bFe (t)
in [11], where the incoming wave variables vm and us in
where τPC (t) is the output of the master side passivity con- this relationship are reflected and returned as the outgoing
troller and q̇PC (t) is the output of the slave side passivity wave variable um and vs . This phenomenon is called wave
controller. reflection. Wave reflections can last several cycles in the com-
The power-based TDPA using the passivity observers and munication channels and then gradually disappear, which can
passivity controllers can robustly guarantee the passivity of easily produce unpredictable interference and disturbances that
the communication channels in the presence of time varying significantly influence transparency. Unlike the conventional
delays. However, as a conservative method for system passiv- wave variable, the outgoing wave variables vs1 and um2 do not
ity, this method can largely degrade the system’s transparency contain any unnecessary information from the incoming wave
in the presence of the constant time delays or even no delay variables us1 and vm2 as shown in (21) and (22). Therefore, the
(μ̄2 = 0). During the free space movement (τm,s = 0), (16) signal variations caused by wave-reflections can be efficiently
can be simplified as Pm obs (t) = −(b/2)q̇m (t), and during
2 reduced. In addition, direct position information is transmitted
the hard environmental contact (q̇m,s = 0), (17) can be between the master and the slave, and position drift does not
simplified as Psobs (t) = −(1/2b)τs2 (t). Under these condi- occur in this system. The control signals after transmission in
tions, Pm obs (t) and Pobs (t) are negative to the extent that
s Fig. 1 can be derived as
accurate torque and trajectory tracking performances can- IA1 = α1 δ q̇s (t − T2 (t)) + γ1 δ q̈s (t − T2 (t))
not be achieved due to the adverse effect of the passivity
controllers. + b1 λ1 (βδqm (t) + αδ q̇m (t) + γ δ q̈m (t)) (24)
Remark: In this paper, the differentials of unsymmetri-
IB1 = −βδqs (t − T2 (t)) − αδ q̇s (t − T2 (t))
cal time delays T1 (t) and T2 (t) are bounded by μ̄1,2 . That
is, |Ṫ1,2 (t)| ≤ μ̄1,2 . μ̄1,2 are arbitrary positive constants. − γ δ q̈s (t − T2 (t))
Moreover, the time-varying delays T1,2 (t) are considered to be b2 α1
the sum of the constant time delays T̄1,2 with their bounded + (δ q̇m (t) − δ q̇m (t − T1 (t) − T2 (t − T1 (t))))
λ2
perturbations T1,2 (t). That is, T1,2 (t) = T̄1,2 + T1,2 (t) ≤ (25)
T̄1,2 + ε̄1,2 = T1,2max , where ε̄
1,2 is the upper bounds of the
perturbations and T1,2 max is the upper bounds of the T (t).
1,2 VA2 (t) = βδqm (t − T1 (t)) + αδ q̇m (t − T1 (t))
Fig. 1 shows the proposed 4-CH wave variable transforma- + γ δ q̈m (t − T1 (t))
tion which contains two wave transformation schemes. α1
− (δ q̇s (t) − δ q̇s (t − T2 (t) − T1 (t − T2 (t))))
The two wave transformation schemes are applied to encode b1 λ1
the feed-forward signals VA1 and VB1 with the feedback (26)
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

SUN et al.: NN-BASED PASSIVITY CONTROL OF TELEOPERATION SYSTEM UNDER TIME-VARYING DELAYS 5

VB2 (t) = α1 δ q̇m (t − T1 (t)) + γ1 δ q̈m (t − T1 (t)) P2 (t) = VB1 (t)IB1 (t) − VB2 (t)IB2 (t)
λ2 2 T
+ (βδqs (t) + αδ q̇s (t) + γ δ q̈s (t)). (27) = u (t)um2 (t) + 2λ2 vTs2 (t)vs2 (t)
b2 λ2 m2
 
In order to simplify this expression, we define Tl1 (t) = T2 (t)+ − 2 uTm2 (t)vm2 (t) + uTs2 (t)vs2 (t)
T1 (t − T2 (t)) and Tl2 (t) = T1 (t) + T2 (t − T1 (t)). We also 1 T 
= u (t)um2 (t) − uTs2 (t)us2 (t)
have the constraints Ṫl1 (t) ≤ μ̄1 , Ṫl2 (t) ≤ μ̄2 . The proposed λ2 m2
 
4-CH wave variable transformation can actually be seen as the + λ2 vTs2 (t)vs2 (t) − vTm2 (t)vm2 (t)
combination of two 2-port networks. Therefore, the power flow  T
1 1
in the 4-CH wave variable transformation can be defined as + λ2 vm2 (t) − um2 (t) vm2 (t) − um2 (t)
λ2 λ2
 T
P4CH (t) = P1 (t)[scheme1] + P2 (t)[scheme2] 1 1
+ λ2 vs2 (t) − us2 (t) vs2 (t) − us2 (t)
P1 (t) = VA1 (t)IA1 (t) − VA2 (t)IA2 (t) λ2 λ2
t
2 T d 1 T
= 2λ1 uTm1 (t)um1 (t) + v (t)vs1 (t) = u (η)um2 (η)dη
λ1 s1 dt λ2 m2
  t−T1 (t)
− 2 uTm1 (t)vm1 (t) + uTs1 (t)vs1 (t) t
  +
d
λ2 vTs2 (η)vs2 (η)dη
= λ1 uTm1 (t)um1 (t) − uTs1 (t)us1 (t) dt
t−T2 (t)
1 T  
+ vs1 (t)vs1 (t) − vTm1 (t)vm1 (t) T
λ1 1 1
+ λ2 vm2 (t) − um2 (t) vm2 (t) − um2 (t)
λ2 λ2
1 
+ (vm1 (t) − λ1 um1 (t))T (vm1 (t) − λ1 um1 (t)) T
λ1 1 1
+ λ2 vs2 (t) − us2 (t) vs2 (t) − us2 (t)
λ2 λ2
1
+ (vs1 (t) − λ1 us1 (t))T (vs1 (t) − λ1 us1 (t)) 1
λ1 − Ṫ1 (t)us2 (t)us2 (t) − λ2 Ṫ2 (t)vm2 (t)vm2 (t)
T T
λ2
t dE2 (t)
d = + Pdiss
= λ1 uTm1 (η)um1 (η)dη 2 (t) (31)
dt dt
t−T1 (t) t
1 T
E2 (t) = u (η)um2 (η)dη
t λ2 m2
d 1 T t−T1 (t)
+ v (η)vs1 (η)dη
dt λ1 s1 t
t−T2 (t)
+ λ2 vTs2 (η)vs2 (η)dη (32)
1
+ (vm1 (t) − λ1 um1 (t))T (vm1 (t) − λ1 um1 (t)) t−T2 (t)
λ1  T
1 1
1 Pdiss (t) = λ 2 v m2 (t) − um2 (t) vm2 (t) − um2 (t)
+ (vs1 (t) − λ1 us1 (t))T (vs1 (t) − λ1 us1 (t))
2
λ2 λ2
λ1  T
1 1
1 + λ2 vs2 (t) − us2 (t) vs2 (t) − us2 (t)
− λ1 Ṫ1 (t)uTs1 (t)us1 (t) − Ṫ2 (t)vTm1 (t)vm1 (t) λ2 λ2
λ1 1
dE1 (t) − Ṫ1 (t)uTs2 (t)us2 (t) − λ2 Ṫ2 (t)vTm2 (t)vm2 (t).
= + Pdiss λ2
1 (t) (28)
dt (33)
t According to (29) and (32), the net energy flows are absolutely
E1 (t) = λ1 uTm1 (η)um1 (η)dt positive to guarantee passivity of the communication network.
t−T1 (t) Based on the definition of passivity and assuming E1 (0) =
t E2 (0) = 0, the energy flow is derived as
1 T t t
+ v (η)vs1 (η)dt (29) dE1
λ1 s1 Eflow (t) = P4CH (η)dη = (Pdiss 1 (η) + P2 (η) +
diss
(η)
t−T2 (t) dt
1 0 0
1 (t) =
Pdiss (vm1 (t) − λ1 um1 (t))T (vm1 (t) − λ1 um1 (t)) dE2
λ1 + (η))dη = E1 (t) + E2 (t) − E1 (0) − E2 (0)
1 dt
+ (vs1 (t) − λ1 us1 (t))T (vs1 (t) − λ1 us1 (t)) t t
λ1
1 + P1 (η) + P2 (η)dη ≥ Pdiss
diss diss
1 (η)
− λ1 Ṫ1 (t)uTs1 (t)us1 (t) − Ṫ2 (t)vTm1 (t)vm1 (t)
λ1 0 0
(30) + Pdiss
2 (η)dη. (34)
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON CYBERNETICS

1
Psobs (t) = (vs1 (t) − λ1 us1 (t))T (vs1 (t) − λ1 us1 (t))
λ1
 T
1 1
+ λ2 vs2 (t) − us2 (t) vs2 (t) − us2 (t)
λ2 λ2
Ṫ 1 T
− λ1 Ṫ 1 uTs1 (t)us1 (t) − u (t)us2 (t) (38)
λ2 s2
⎧ estimate
⎪
⎪ Ṫ , estimate < M̄
if Ṫ1,2 1,2
⎨ 1,2
M̄1,2 , else, if Ṫ1,2 estimate ≥ M̄
Ṫ 1,2 = 1,2
(39)
⎪
⎪ M̄ , else, if um1 (t − T1 (t)) = 0 or
⎩ 1,2
Fig. 2. Time delay differential estimator. vs1 (t − T1 (t)) = 0
where M̄1,2 is the estimated constant upper bound that satisfy
1 (t)+P2 (t) ≥ 0, according
Therefore, in the situation that Pdiss diss

to (34), the energy flow Eflow (t) is no less than zero and the M̄1,2 > Ṫ1,2 .
passivity of the time delayed network can be guaranteed. By using the passivity observer, we design the passivity
controller to be
1 (t)+P2 (t) can also be defined as the
Similar with (11), Pdiss diss

sum of master power dissipation components Pm diss (t) and slave V̂s (t) = VA2 (t) − VB2 (t) − 2 (t) (40)
power dissipation components Psdiss (t) based on (30) and (33) Îm (t) = −IA1 − IB1 − 1 (t) (41)
1 where V̂s (t) and Îm (t) are the output control signals from the
diss (t) =
Pm (vm1 (t) − λ1 um1 (t))T (vm1 (t) − λ1 um1 (t))
λ1 passivity controllers on the slave and master sides, respec-
 T
1 1 tively. 1 (t) and 2 (t) are designed as (42) and (43), where
+ λ2 vm2 (t) − um2 (t) vm2 (t) − um2 (t) σ1,2 ,  are positive constants
λ2 λ2
⎧
−
1
Ṫ2 (t)vTm1 (t)vm1 (t) − λ2 Ṫ2 (t)vTm2 (t)vm2 (t) ⎪
⎪ 0, if Pm obs (t) ≥ 0
⎪
⎪ − α
(α 1 )μ̄2 δ q̇s (t − T2 (t))q̇s (t − T2 (t))
T
λ1 ⎪
⎪  
⎪
⎪
(35) ⎪
⎨ 2 q̇m (t) + e−t
Pdiss (t) =
s 1
(vs1 (t) − λ1 us1 (t)) (vs1 (t) − λ1 us1 (t))
T 1 (t) = (γ − γ1 )μ̄2 δ q̈Ts (t − T2 (t))q̈s (t − T2 (t))
⎪
⎪ +  
λ1 ⎪ 2 q̇m (t) + e−t
 T
⎪
⎪ 
1 1 ⎪
⎪ b2 α1 μ̄2 δ q̇Tm (t)q̇m (t) + σ1
+ λ2 vs2 (t) − us2 (t) vs2 (t) − us2 (t) ⎪
⎪  
⎩+ obs (t) < 0
, if Pm
λ2 λ2 2λ2 q̇m (t) + e−t
1 (42)
− λ1 Ṫ1 (t)uTs1 (t)us1 (t) − Ṫ1 (t)uTs2 (t)us2 (t). (36) ⎧
λ2 ⎪ 0, if P s (t) ≥ 0
⎪
⎪ obs
According to (35) and (36), the proposed passivity observers ⎪
⎪ (α − α1 )μ̄1 δ q̇m (t − T1 (t))q̇m (t − T1 (t))
T
⎪
⎪
can observe the power dissipation components in real time, ⎪
⎪ 2 q̇s (t) + e−t
⎨
as Pmdiss (t) and Pdiss (t) only contain the signals observed at
s
2 (t) = (γ − γ1 )μ̄1 δ q̈Tm (t − T1 (t))q̈m (t − T1 (t))
the master and slave ports, respectively. The proposed 4-CH ⎪
⎪ +  
⎪
⎪  2 q̇s (t) + e−t
wave transformation is proposed to guarantee the passivity ⎪
⎪
⎪
⎪ α1 μ̄1 σ2 δ q̇Ts (t)q̇s (t) + σ2
of the communication channels in the presence of constant ⎪
⎩+   , if Psobs (t) < 0.
2b1 λ1 q̇s (t) + e−t
delays so that the Pm diss (t) and Pdiss (t) are required to be pos-
s
(43)
itive when Ṫ1 (t) = Ṫ2 (t) = 0. Therefore, for constant time
delays, the final two terms in (35) and (36) can be treated
as zero and then (35) and (36) are definitely non-negative. IV. D ESIGN AND A NALYSIS OF THE P ROPOSED
Therefore, the communication channels’ passivity can be guar- T ELEOPERATION S YSTEM
anteed by the proposed 4-CH wave transformation and the Based on the external force models (5), (6), the teleoperation
passivity controllers will not be launched to degrade the system dynamics can be rewritten as the following form:
transparency.
Mm (qm )δ q̈m + Cm (qm , q̇m )δ q̇m
The value of Ṫ1,2 can be measured by using the time delay
differential estimator in Fig. 2. When this estimator is used, = τm + τh∗ (t) − Bh δ q̇m (t) − Mh δ q̈m (t) + Fm
∗
− fm (Xm )
the integral of um1 (t) and vs1 (t) should be sent outside the (44)
wave transformation. The passivity observers are designed as Ms (qs )δ q̈s + Cs (qs , q̇s )δ q̇s
1 = τs − τe∗ (t) − Be δ q̇s (t) − Me δ q̈s (t) + Fs∗ − fs (Xs ) (45)
obs (t) =
Pm (vm1 (t) − λ1 um1 (t))T (vm1 (t) − λ1 um1 (t))
λ1 where Xi (t) = [q̈Ti (t), q̇Ti (t), qTi (t)]T . fi (Xi ) are defined as
 T
1 1
+ λ2 vm2 (t) − um2 (t) vm2 (t) − um2 (t) fi (Xi ) = Fi q̇i + fci (q̇i ) + gi (qi ) + Mi (qi )(1 − δ)q̈i
λ2 λ2
+ Ci (qi , q̇i )(1 − δ)q̇i + (Kh,e
Ṫ 2 T
− v (t)vm1 (t) − λ2 Ṫ 2 vTm2 (t)vm2 (t) (37) + kh,e )qi + bh,e q̇i + mh,e q̈i . (46)
λ1 m1
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

SUN et al.: NN-BASED PASSIVITY CONTROL OF TELEOPERATION SYSTEM UNDER TIME-VARYING DELAYS 7

According to NNs approximation property, the functions The adaptive control laws Yi (t) are mainly used to deal with
f̂i (Xi ) are applied in this paper to approximate fi (Xi ) with the approximation error, external positive input, and unknown
disturbance. ˆ i (t) are applied to compensate for the sum of NN
f̂i (Xi ) = ŴiT i (Xi ) (47) approximate error, the bounded external disturbance Fi∗ and
the exogenous input τh,e ∗ (t). That is,  ≥ ξ̄ ∗ (X)+F ∗ ±τ ∗ .
i i i h,e
where Ŵi is the NN adaption parameters and i (Xi ) is the NN In the ideal situation where f̃i (Xi ) = 0, the adaptive con-
basis functions. We define trollers can be considered as damping terms which may
  influence transparency. However, by setting 0 < δ < 1,
f̃i (Xi ) = f̂i (Xi ) − fi (Xi ) = ŴiT − Wi∗T i (Xi ) the adverse influence can be effectively reduced. The adap-
tive control laws Gi (t) are applied to guarantee the stability
= W̃iT i (Xi ) + ξ̄i∗ (X). (48) of acceleration transmission. hi (t) are the states of the aux-
iliary system in (58) (hi (t) = 0). k is a positive constant.
Due to the piecewise continuous function fci (q̇i ), we assume setting k < 1 can efficiently reduce the influence of (52) on
that ξ̄i∗ (X) are made up of ξi∗ and fci (q̇i ). transparency.
Combine the proposed wave-based TDPA control method Based on the control laws (49), (50), f̂i (Xi ) and Yi (t)
and the NN control method, the control laws of the overall are deployed to diminish the side effects of system uncer-
teleoperation systems are given as follows: tainties as well as the external disturbance and input. The
two terms of −b1 λ1 αδ q̇m (t) and −((λ2 α)/b2 )δ q̇s (t) are
τm (t) = f̂m (Xm ) + Îm (t) − (α − α1 )δ q̇m (t) applied to guarantee the system’s stability and the two rela-
− (1 − b1 λ1 )βδqm (t) − (1 − b1 λ1 )(γ − γ1 )δ q̈m (t) tionships of −((b2 α1 )/λ2 )(δ q̇m (t) − δ q̇m (t − Tl1 (t))) and
− Gm (t) − Ym (t) = f̂m (Xm ) − IA1 − IB1 − 1 (t) −(α1 /(b1 λ1 ))(δ q̇s (t) − δ q̇s (t − Tl2 (t))) can strengthen tracking
performance and system stability. Under small time delays,
− (α − α1 )δ q̇m (t) − (1 − b1 λ1 )βδqm (t)
these two terms are close to zero. For large time delays, they
− (1 − b1 λ1 )(γ − γ1 )δ q̈m (t) − Gm (t) − Ym (t) can be treated as dampers that can enhance the system stability.
= f̂m (Xm ) + β(δqs (t − T2 (t)) − δqm (t)) Also setting small value of α1 can efficiently reduce the value
+ (α − α1 )(δ q̇s (t − T2 (t)) − δ q̇m (t)) of the two terms. The remaining parts produce accurate posi-
tion, velocity and acceleration signals tracking. The proposed
+ (γ − γ1 )(δ q̈s (t − T2 (t)) − δ q̈m (t))
control laws allow the operator to feel the position, velocity
b2 α1
− (δ q̇m (t) − δ q̇m (t − Tl2 (t))) and acceleration information of the remote environment, thus,
λ2 highly accurate torque tracking is expected to achieve. When
− b1 λ1 αδ q̇m (t) − 1 (t) − Gm (t) − Ym (t) (49) large and sharply varying delays occur, 1,2 (t) will be immedi-

λ2 ately launched to guarantee the whole system’s stability. More
τs (t) = f̂s (Xs ) + V̂s (t) − (α − α1 )δ q̇s (t) − 1 − βδqs (t)
b2 details on setting control parameters will be introduced later.

λ2 The total block diagram of the proposed teleoperation system
− 1− (γ − γ1 )δ q̈s (t) − Gs (t) − Ys (t)
b2 is shown in Fig. 3.
= f̂s (Xs ) + VA2 (t) − VB2 (t) − 2 (t) − (α − α1 )δ q̇s (t) Theorem 1: Consider the teleoperation system (1), (2). If the
  control laws are constructed by (49) and (50), the NN adaptive
λ2 λ2
− 1− βδqm (t) − 1 − (γ − γ1 )δ q̈s (t) laws are
b2 b2
− Gs (t) − Ys (t) = f̂s (Xs ) ˙ =   (X )δ q̇
Ŵ (53)
i i i i
i
+ β(δqm (t − T1 (t)) − δqs (t))
+ (α − α1 )(δ q̇m (t − T1 (t)) − δ q̇s (t)) where i is the positive definite matrices. The position and
+ (γ − γ1 )(δ q̈m (t − T1 (t)) − δ q̈s (t)) velocity tracking errors will asymmetrically converge to zero
α1 λ2 α in the presence of arbitrary time delays.
− (δ q̇s (t) − δ q̇s (t − Tl1 (t))) − δ q̇s (t) Proof: Consider a positive semi-definite function V(t) for
b1 λ1 b2
− 2 (t) − Gs (t) − Ys (t). (50) the system as V(t) = V1 (t) + V2 (t) + V3 (t) + V4 (t) + V5 (t)
where
Yi (t) and Gi (t) are the designed adaptive control laws as 1 T 1
⎧ V1 (t) = δ q̇ (t)Mm (qm (t))δ q̇m (t) + δ q̇Ts (t)Ms (qs (t))δ q̇s (t)
2 m 2
⎨ δ q̇i (t) ˆ ˆ i (t) = δ q̇i (t)  1  T −1  1  T −1 
 (t), if q̇i (t) = 0,  + tr W̃m m W̃m + tr W̃s s W̃s
Yi (t) = δ q̇i (t) i (54)
⎩ 0, if q̇i (t) = 0
2 2
β
(51) V2 (t) = (δqm (t) − δqs (t))T (δqm (t) − δqs (t))
 2
Gi (t) = khi (t) 1 2 1  2
+ m −  ˆ m (t) + s −  ˆ s (t)
ḣi (t) = − γ −γ1 δ q̈i (t) +δ q̇i (t)
2 2
2 h (t)2
hi (t) + kδ q̇i (t). 2 2
i
1 T 1 T
(52) + hm (t)hm (t) + hs (t)hs (t) (55)
2 2
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

8 IEEE TRANSACTIONS ON CYBERNETICS

Fig. 3. Total block diagram.

0 t of V1 (t) can be written as

β
V3 (t) = δ 2 q̇Tm (η)q̇m (η)dηdπ 
2
−T1 (t) t+π V̇1 (t) = δ q̇m (t) f̂m (Xm ) − fm (Xm ) − Bh q̇m (t) − Mh q̈m (t)
T

0 t + τh∗ (t) + Fm
∗
+ β(δqs (t − T2 (t)) − δqm (t))
β
+ δ 2 q̇Ts (η)q̇s (η)dηdπ (56) + (α − α1 )(δ q̇s (t − T2 (t)) − δ q̇m (t))
2
−T2 (t) t+π
+ (γ − γ1 )(δ q̈s (t − T2 (t)) − δ q̈m (t))
t b2 α1
α − α1 − 2b1 λ1 αδ q̇m (t) − (δ q̇m (t)
V4 (t) = δ 2 q̇Tm (η)q̇m (η)dη λ2
2
t−T1 (t) − δ q̇m (t − Tl2 (t)))
t δ q̇m (t)
α − α1 − 1 (t) − m (t) − Gm (t)
+ δ 2 q̇Ts (η)q̇s (η)dη δ q̇m (t)
2 
t−T2 (t)
+ δ q̇Ts (t) f̂s (Xs ) − fs (Xs ) − Be q̇s (t) − Me q̈s (t)
t
γ − γ1 + Fs∗ − τe∗ (t) + β(δqm (t − T1 (t)) − δqs (t))
+ δ 2 q̈Tm (η)q̈m (η)dη
2
t−T1 (t) + (α − α1 )(δ q̇m (t − T1 (t)) − δ q̇s (t))
t + (γ − γ1 )(δ q̈m (t − T1 (t)) − δ q̈s (t))
γ − γ1 2λ2 α α1
+ δ 2 q̈Ts (η)q̈s (η)dη − δ q̇s (t) − (δ q̇s (t)
2 b2 b1 λ1
t−T2 (t)
− δ q̇s (t − Tl1 (t)))
t
b2 α1 δ q̇s (t)
+ δ 2 q̇Tm (η)q̇m (η)dη − 2 (t) − s (t) − Gs (t)
2λ2 δ q̇s (t)
t−Tl2 (t)  T   T 
− tr W̃m m (Xm )δ q̇m (t) − tr W̃s s (Xs )δ q̇s (t) . (59)
t
α1
+ δ 2 q̇Ts (η)q̇s (η)dη (57) Also, the time derivative of V2 (t) is given by
2b1 λ1
t−Tl1 (t) V̇2 (t) = βδ q̇Tm (t)(δqm (t) − δqs (t − T2 (t)))
Mh /δ + γ − γ1 + βδ q̇Ts (t)(δqs (t) − δqm (t − T1 (t)))
V5 (t) = δ 2 q̇Tm (t) q̇m (t)
2 t t
Me /δ + γ − γ1
+ δ 2 q̇Ts (t) q̇s (t). (58) − βδ q̇Tm (t) δ q̇s (η)dη − βδ q̇Ts (t) q̇m (η)dη
2
t−T (t) t−T (t)
 2
  1
Using property 2 in Section II, the control laws (49), (50), ˆ ˆ ˆ ˆ s (t)
+ m (t) − m m (t) + s (t) − s 
the modeled human and environmental torques (5), (6),
and the NNs adaptive laws (53), the time derivative + hm (t)ḣm (t) + hs (t)ḣs (t). (60)
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

SUN et al.: NN-BASED PASSIVITY CONTROL OF TELEOPERATION SYSTEM UNDER TIME-VARYING DELAYS 9

After algebraic manipulations, time derivative of V3 (t) is found t t

to satisfy −2q̇Ts (t) q̇m (η)dη − q̇Tm (η)q̇m (η)dη
t t−T1 (t) t−T1 (t)
βδ 2 β
V̇3 (t) ≤ T1 q̇m (t)
max T
q̇m (t) − δ 2 q̇Tm (η)q̇m (η)dη ≤ T1max q̇Ts (t)q̇s (t) (67)
2 2
t−T1 (t) and with the adaptive laws (51), (52), it can be derived that
t 
βδ 2 β β 
+ T2max q̇Ts (t) q̇s (t) − δ 2 q̇Ts (η)q̇s (η)dη. V̇(t) < −δ q̇m (t) b1 λ1 α − T 1 + T 2 + ε1 + ε2 δ q̇m (t)
T
2 2 2

t−T2 (t) λ2 α β 
(61) − δ q̇Ts (t) − T 1 + T 2 + ε1 + ε2 δ q̇s (t)
b2 2
The time derivative of V4 (t) can also be written as α − α1
− (δ q̇m (t) − δ q̇s (t − T2 (t)))T
α − α1  2 T 2
V̇4 (t) = δ q̇m (t)q̇m (t) × (δ q̇m (t) − δ q̇s (t − T2 (t)))
2
α − α1
− δ 2 q̇Tm (t − T1 (t))q̇m (t − T1 (t)) − (δ q̇s (t) − δ q̇m (t − T1 (t)))T
 2
+ Ṫ1 (t)δ 2 q̇Tm (t − T1 (t))q̇m (t − T1 (t)) × (δ q̇s (t) − δ q̇m (t − T2 (t)))
α − α1  2 T −
γ − γ1
(δ q̇m (t) − δ q̈s (t − T2 (t)))T
+ δ q̇s (t)q̇s (t) 2
2
× (δ q̇m (t) − δ q̈s (t − T2 (t)))
− δ 2 q̇Ts (t − T2 (t))q̇s (t − T2 (t))
 γ − γ1
− (δ q̇s (t) − δ q̈m (t − T1 (t)))T
+ Ṫ2 (t)δ 2 q̇Ts (t − T2 (t))q̇s (t − T2 (t)) 2
γ − γ1  2 T × (δ q̇s (t) − δ q̈m (t − T2 (t)))
+ δ q̈m (t)q̈m (t) b2 α1
2 − (δ q̇m (t) − δ q̇m (t − Tl2 (t)))T
− δ 2 q̈Tm (t − T1 (t))q̈m (t − T1 (t)) 2λ2
 × (δ q̇m (t) − δ q̇m (t − Tl2 (t)))
+ Ṫ1 (t)δ 2 q̈Tm (t − T1 (t))q̈m (t − T1 (t)) α1
− (δ q̇s (t) − δ q̇s (t − Tl1 (t)))T
γ − γ1  2 T 2b1 λ1
+ δ q̈s (t)q̈s (t) × (δ q̇s (t) − δ q̇s (t − Tl1 (t)))
2
− δ 2 q̈Ts (t − T2 (t))q̈s (t − T2 (t)) (α − α1 )Ṫ1 (t)δ 2 T
 + q̇m (t − T1 (t))q̇m (t − T1 (t))
2
+ Ṫ2 (t)δ 2 q̈Ts (t − T2 (t))q̈s (t − T2 (t))
(α − α1 )Ṫ2 (t)δ 2 T
b2 α1  2 T +
2
q̇s (t − T2 (t))q̇s (t − T2 (t))
+ δ q̇m (t)q̇m (t)
2λ2 (γ − γ1 )Ṫ1 (t)δ 2 T
+ q̈m (t − T1 (t))q̈m (t − T1 (t))
− δ 2 q̇Tm (t − Tl2 (t))q̇m (t − Tl2 (t)) 2

+ Ṫl2 (t)δ 2 q̇Tm (t − Tl2 (t))q̇m (t − Tl2 (t)) (γ − γ1 )Ṫ2 (t)δ 2 T
+ q̈s (t − T2 (t))q̈s (t − T2 (t))
α1  2 T 2
+ δ q̇s (t)q̇s (t) b2 α1 Ṫl2 (t)δ 2  T 
2b1 λ1 + q̇m (t)q̇m (t) + σ1
2λ2
− δ 2 q̇Ts (t − Tl1 (t))q̇s (t − Tl1 (t))
 α1 Ṫl1 (t)δ 2  T 
+Ṫl1 (t)δ 2 q̇Ts (t − Tl1 (t))q̇s (t − Tl1 (t)) . + q̇s (t)q̇s (t) + σ2 − δ q̇Tm (t)1 (t)
2b1 λ1
(62) − δ q̇Ts (t)2 (t) − δ q̇Tm (t)Bh q̇m (t) − δ q̇Ts (t)Be q̇s (t)

The differential of V5 (t) is ∗ q̇m (t)
+ δ q̇m (t) ξ m (X) + τh∗ (t) + Fm
T ∗
− m (t)
V̇5 (t) = q̇Tm (t)δMh q̈m (t) + q̇Ts (t)δMe q̈s (t) q̇m (t)

∗ δ q̇s (t)
+ q̇Tm (t)δ 2 (γ − γ1 )q̈m (t) + q̇Ts (t)δ 2 (γ − γ1 )q̈s (t). + δ q̇Ts (t) ξ s (X) − τe∗ (t) + Fs∗ − s (t)
δ q̇s (t)
(63)    
+ m (t) − m δ q̇m (t) + s (t) − s δ q̇m (t).
By setting σ1−2 large enough to make sure (68)
q̇Tm (t)q̇m (t) + σ1 > q̇Tm (t − Tl2 (t))q̇m (t − Tl2 (t)) (64)
Substituting the upper bounds i of + ξ̄i∗ (X)
± Fi∗
q̇Ts (t)q̇s (t) + σ2 > q̇Ts (t − Tl1 (t))q̇s (t − Tl1 (t)) (65) ∗  into (68), the final four terms in (68) can be
τh,e
using the following inequalities from [37, Lemma 1]: removed. The Lyapunov approach requires V̇(t) to be neg-
t t ative semi-definite. In the presence of constant time delays,
−2q̇m (t)
T
q̇s (η)dη − q̇Ts (η)q̇s (η)dη Ṫ1,2 (t) and ε̄1,2 are zero. Also, the passivity controllers
do not take effect so that 1 (t) and 2 (t) are zero.
t−T2 (t) t−T2 (t) −((b2 α1 )/(2λ2 ))(δ q̇m (t) − δ q̇m (t − Tl2 (t)))T (δ q̇m (t) − δ q̇m (t −
≤ T2max q̇Tm (t)q̇m (t) (66) Tl2 (t))) and −(α1 /(2b1 λ1 ))(δ q̇s (t) − δ q̇s (t − Tl1 (t)))T (δ q̇s (t) −
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

10 IEEE TRANSACTIONS ON CYBERNETICS

δ q̇s (t − Tl1 (t))) can be seen as −((b2 α1 )/2λ2 )ζa δ 2 q̇Tm (t)q̇m (t)
and −(α1 /(2b1 λ1 ))ζb δ 2 q̇Ts (t)q̇s (t) with ζa,b ≥ 0. Under small
time delays, ζa,b → 0. V̇(t) can be guaranteed to be negative
semi-definite by properly tuning b1,2 , λ1,2 , α, α1 , and β to
make sure
b2 α1 β 
b1 λ1 α + ζa ≥ T̄1 + T̄2 (69)
2λ2 2
λ2 α α1 β  Fig. 4. Experimental setup.
+ ζb ≥ T̄1 + T̄2 . (70)
b2 2b1 λ1 2
When the time delay is varying, the passivity controllers are The system’s dynamic model can also be written as
launched by the passivity observers, substituting (42) and (43)
into (68), the biased terms (((α − α1 )Ṫ1 (t)δ 2 )/2)q̇Tm (t − 
δ q̈i = Mi−1 (qi ) τi ± τh,e
∗
(t) − Bh,e q̇i (t) − Mh,e q̈i (t) + Fi∗
T1 (t))q̇m (t − T1 (t)) + (((α − α1 )Ṫ2 (t)δ 2 )/2)q̇Ts (t − 
T2 (t))q̇s (t − T2 (t)) + (((γ − γ1 )Ṫ1 (t)δ 2 )/2)q̈Tm (t − −fi (Xi ) − Ci (qi , q̇i )δ q̇i . (72)
T1 (t))q̈m (t − T1 (t)) + (((γ − γ1 )Ṫ2 (t)δ 2 )/2)q̈Ts (t −
T2 (t))q̈s (t − T2 (t)) + ((b2 α1 Ṫl2 (t)δ 2 )/2λ2 )(q̇Tm (t)q̇m (t) + Differentiating both sides of (72)
σ1 ) + ((α1 Ṫl1 (t)δ 2 )/2b1 λ1 )(q̇Ts (t)q̇s (t) + σ2 ) caused by the
d d  −1 
∗
time varying delays in (68) are directly compensated by δ q̈i = Mi (qi ) τi ± τh,e (t) − Bh,e q̇i (t) − Mh,e q̈i (t)
−δ q̇Tm (t)1 (t) − δ q̇Ts (t)2 (t). No extra parameters need to be dt dt 
∗
tuned when the time delays vary and V̇(t) is still negative + Fi − fi (Xi ) − Ci (qi , q̇i )δ q̇i
d
semi-definite. + Mi−1 (qi ) τi ± τh,e∗
(t) − Bh,e q̇i (t) − Mh,e q̈i (t)
Integrating both sides of (68), we get dt 
+Fi∗ − fi (Xi ) − Ci (qi , q̇i )δ q̇i . (73)
+∞ > V(0) ≥ V(0) − V(t)
t  For the first term of the right side of (73), we have
α − α1
> (δ q̇m (t) − δ q̇s (t − T2 (t)))T
2 d  −1   
0 Mi = −Mi−1 Ṁi Mi−1 = −Mi−1 Ci + CiT Mi−1 . (74)
dt
× (δ q̇m (t) − δ q̇s (t − T2 (t)))
α − α1
+ (δ q̇s (t) − δ q̇m (t − T1 (t)))T According to Properties 1 and 3, (d/dt)(Mi−1 ) is bounded.
2 Based on Property 4, the terms in bracket of (74) are also
× (δ q̇s (t) − δ q̇m (t − T2 (t)))
bounded. Therefore,
t (d/dt)q̈i (t) ∈ L∞ and q̈i (t) are uniformly
γ − γ1
+ (δ q̇m (t) − δ q̈s (t − T2 (t)))T continuous ( 0 q̈i (η)dη = q̇i (t) − q̇i (0)). Since q̇i (t) → 0,
2 it can be concluded that q̈i (t) → 0 based on Barbǎlat’s
× (δ q̇m (t) − δ q̈s (t − T2 (t)))
γ − γ1 lemma.
+ (δ q̇s (t) − δ q̈m (t − T1 (t)))T
2
× (δ q̇s (t) − δ q̈m (t − T2 (t)))
V. E XPERIMENTAL W ORK
b2 α1
+ (δ q̇m (t) − δ q̇m (t − Tl2 (t)))T The teleoperation system used to validate the proposed
2λ2
× (δ q̇m (t) − δ q̇m (t − Tl2 (t))) algorithm consists of two 3-DOF Phantom manipulators:
α1 1) Phantom Omni and 2) Phantom Desktop (Sensible
+ (δ q̇s (t) − δ q̇s (t − Tl1 (t)))T Technologies Inc., Wilmington, MA, USA) as shown in Fig. 3.
2b1 λ1
The two haptic device are connected by two computers that are
× (δ q̇s (t) − δ q̇s (t − Tl1 (t))) + δ q̇Tm (t)Bh q̇m (t)
directly connected via a network cable and network cards. The
+ δ q̇Ts (t)Be q̇s (t) dt. (71) MATLAB software is applied to establish the proposed con-
trol system. To further enlarge and tune the value of the time
delays, Simulink time delay blocks are also applied. During
Therefore, from V(t) ≥ 0 and V̇(t) ≤ 0, it is true that
the experimental process, the control loop is configured as
W̃m and W̃s ∈ L∞ , q̇m (t) and q̇s (t) ∈ L2 . (q̇m (t) − q̇s (t −
a 1 kHz sampling rate. The general control parameters are con-
T2 (t))), (q̇s (t) − q̇m (t − T2 (t))), (δ q̇m (t) − δ q̈s (t − T2 (t))),
figured as: b1 = b2 = 2, α = [15, 15, 15]T , β = [15, 15, 15]T ,
(δ q̇s (t) − δ q̈m (t − T2 (t))), (q̇m (t) − q̇m (t − Tl2 (t))), and
α1 = [2.5, 2.5, 2.5]T , γ = [60, 60, 60]T , γ1 = [10, 10, 10]T ,
(q̇s (t) − q̇s (t − Tl1 (t))) ∈ L2 . Using the fact that qm (t) −
t δ = 0.2, s = m = [25, 25, 25]T . σ1 = σ2 = 0.25.  = 4,
qs (t − T2 (t)) = qm (t) − qs (t) + t−T2 (t) q̇s (t)dt, qs (t) −
t and k = 0.1. We set the number of the neuron κ = 7. The
qm (t − T1 (t)) = qs (t) − qm (t) + t−T1 (t) q̇1 (t)dt, and using center of the RBF is set as C = 0.5 × ones(9, 7) and the
t √
Cauchy–Schwarz inequality t−T2 (t) q̇s (t)dt ≤ T2 (t)q̇s (t) and width of the RBF is set as H = 0.1 × ones(7, 1). The param-
t √
t−T1 (t) q̇m (t)dt ≤ T1 (t)q̇m (t), we can get qm (t)−qs (t−T2 (t)), eters relating to the time delays will be introduced in each
qs (t) − qm (t − T1 (t)) ∈ L∞ . experiment.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

SUN et al.: NN-BASED PASSIVITY CONTROL OF TELEOPERATION SYSTEM UNDER TIME-VARYING DELAYS 11

Fig. 5. Free motion under constant time delays (comparison between [35] and our system).

Fig. 6. Hard contact under constant time delays (comparison between [35] and our system).

A. Innovative Passivity Observers on the slave side in the system in [35] is definitely non-
The experiments conducted clearly demonstrate the novelty positive. Therefore, large torque tracking errors are caused
and contribution of the proposed passivity observers. The pro- by the passivity controllers. By contrast, the power signals
posed system is compared with a TDPA-based system in [35]. observed in the proposed system are non-negative resulting in
The time delays are constant and T1 is 200 ms and T2 is accurate torque tracking during hard contact. From these two
100 ms. According to (69) and (70), we set λ1 = 0.075, diagrams, it can be observed that the proposed wave-based pas-
λ2 = 0.3, and μ̄1 = μ̄2 = ε̄1 = ε̄2 = 0. In the system sivity observers makes the proposed system less conservative
in [35], the slave PD controllers are chosen as Kp = 10 and compared the conventional power-based systems and guaran-
Kd = 5, and b in the passivity observers is set to 2.5. tees the system high tracking performance in the presence of
Fig. 5 shows the position tracking, torque tracking, and constant time delays.
observed power of the two systems during free motion. Even
under small constant time delays whilst the rate of the time B. Eliminating Wave Reflection
delays are zero, based on (16), the power observed in the mas- In this experiment, the proposed system is compared against
ter side is still nonpositive so that the passivity controllers are the wave-based system proposed in [11] in order to show its
still launched to reduce tracking performance. The launched effectiveness in eliminating wave reflection. Figs. 7 and 8 show
passivity controllers regrade the position tracking and make the the position and torque tracking and their relating tracking
system over-damped so that the human operator can feel large errors when the slave is in contact with a reverse wall. The
feedback forces during free motion. As shown in Fig. 5, the time delay in this experiment is around 900 ms with 100 varia-
slave cannot quickly and closely track the master in the pres- tions, and its rate is around 0.2. According to (69) and (70), we
ence of such small delays in the system in [35]. Unlike [35], set λ1 = 0.45, λ2 = 1.8, μ̄1 = μ̄2 = 0.2, and ε̄1 = ε̄2 = 0.2.
the power signals observed on the master and the slave sides The wave-based system in [11] uses a traditional wave trans-
of the proposed system are positive owning to the designed formation with impedance matching to encode the velocity
wave-based passivity observer, and the passivity controllers are and position signals. Based on recommended values, we set
not launched. Therefore, the slave caclosely track the master Km = Ks = b = 4. The extra energy caused by time-varying
and the human operator can hardly feel the feedback force. delays in their system is eliminated by applying the scaling
On the other hand, Fig. 6 displays the position tracking, gain 1 − μ̄1,2 .
torque tracking, and observed power signals of the two sys- At first, the slave robots in both systems are in free motion
tems during hard contact. Based on (17), the power observed and both have accurate position tracking. Then the slave robots
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

12 IEEE TRANSACTIONS ON CYBERNETICS

Fig. 7. Contact to a reverse wall under slowly varying delays (wave-based system in [11]).

Fig. 8. Contact to a reverse wall under slowly varying delays (our system).

come in contacts with the reverse wall, in which both systems and velocities cannot be transmitted. Figs. 9 and 10 demon-
achieve accurate torque tracking. Therefore, it can be stated strate position tracking, position errors, and torque tracking of
that both systems have good steady state performance. After the two systems. The key element in a PD+d system is the
about 2 s, the wall is suddenly removed, causing a sudden velocity damper that can guarantee the system’s stability but,
change in the environment. As a result, the impedances- instead, degrade the system’s transparency.
matching approach of the system in [11] fails to work at As shown in Fig. 9, with large time delays, the velocity
the transient state to the extent that the wave-reflections damper Br and Bl in the system in[10] have to be set large
are restored causing large perturbations that adversely affect enough to guarantee stability, and 1 − μ̄1,2 also affect the
the position and torque tracking performances of the system velocity transmission to the extent that the position tracking
in [11] as shown in Fig. 7. By contrast, based on the designed is affected and large position errors occur. Also, the opera-
wave variables in (20)–(23), the outgoing signals do not con- tor feels the system over-damped and achieves large feedback
tain necessary signals and the wave reflections are eliminated. forces even under free motion.
Therefore, the proposed system has better performance on the By contrast, passivity controllers in our system vary based
transient state and the position error directly returned to zero on the observed power signals at each port. Based on Fig. 10,
after removing the wall as shown in Fig. 8. the observed power signals are not definitely negative, so that
the passivity controllers keep varying between activation and
C. Performance in the Presence of Time-Varying Delays deactivation modes. Therefore, the proposed system is not as
In this section, we compare our system against classic conservative as the classical PD+d system in [10] and can
PD+d system proposed in [10] in order to show its unique achieve more accurate position tracking performance under
performance in dealing with the time-varying delays. The time large time-varying delays. In addition, since the proposed sys-
delays in this experiment are approximately 1 s with 500 ms tem is not over-damped, the feedback force felt by the operator
variations, and the rate is around 0.5. We set λ1 = 0.5, is not as large as that in the system in [10].
λ2 = 2, μ̄1 = μ̄2 = 0.5, and ε̄1 = ε̄2 = 0.5. In
order to guarantee stability, the parameters in the system D. Performance in the Presence of Fast-Varying Delays
2 2
in [10] are required to satisfy 4Br Bl > (T1max + T2max )Kl Kr , The novelty of the proposed system is also reflected by
Br ≥ Bl , and Kr ≥ Kl . Therefore, we set the differential gain its ability to deal with fast-varying delays. In this section,
Kd = 3, and the prortional gains Kr = Kl = 3. Hence, we compare the new system with the performance of a sys-
Br = Bl =3.5. The PD+d system in [10] also uses the tem we developed previously [36]. The time delays for the
scaling gain 1 − μ̄1,2 for velocity transmission. It is notice- experiment of are set to be very large (around 2 s with 1 s
able that when μ̄1,2 ≥ 1, this approach is too conservative variations) and with a large rate (around 1.5). We set λ1 = 1,
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

SUN et al.: NN-BASED PASSIVITY CONTROL OF TELEOPERATION SYSTEM UNDER TIME-VARYING DELAYS 13

Fig. 9. Free motion under time varying delays (PD+d system in [10]).

Fig. 10. Free motion under time varying delays (our system).

λ2 = 4, μ̄1 = μ̄2 = 1.5, and ε̄1 = ε̄2 = 1. The slave robots As shown in Fig. 11, the slave robot cannot closely and
in these two systems are controlled to have a free motion first rapidly track the master robot during free motion, and large
and then to come in contact with a solid wall in the reverse torque tracking errors exist during hard contact with the
direction. Figs. 11 and 12 show the position and torque track- environment.
ing as well as the related tracking errors of the two systems. In contrast, the passivity controllers in the new system are
The previous system [36] primarily has two deficiencies. First, actually velocity dampers with the value varying according to
the parameters of this system are seriously restricted by the the observed powers. Therefore, according to Fig. 12, even
assumption that Ṫ1,2 (t) ≤ 1. If Ṫ1,2 (t) > 1, and the stability with the higher rate of time delays, the position tracking in
of the whole system cannot be guaranteed. The time delay for free motion and torque tracking in hard contact are still bet-
system in [36] is set to 2 s with 1 s variation but at a rate of 0.9. ter that those of the system in [36]. The experimental results
The related parameters are set as recommended in [36]. The illustrate that our new system is more suited to practical appli-
second drawback of system in [36] is how it guarantees system cation than the system proposed in [36] for the worst-case
stability by largely reducing the position and torque signals. scenario.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

14 IEEE TRANSACTIONS ON CYBERNETICS

Fig. 11. Free motion and hard contact under sharply-varying delays (system in [36]).

Fig. 12. Free motion and hard contact under sharply-varying delays (our system).

VI. C ONCLUSION damping velocity but not reducing position and torque sig-
A new 4-CH wave-based TDPA teleoperation system with nals. Therefore in the worst case scenario where the rate
new passivity observers and controllers was proposed in this of time delay is larger than one, the proposed system can
paper RBF NN was also deployed to eliminate the nonlinear still have fine position and torque tracking performances.
uncertainties in the dynamic models. Compared with conven- The system stability with external human and environment
tional observer, the new observer can more efficiently and inputs was also analyzed using Lyapunov functions. The pro-
accurately monitor the power flow to guarantee that the system posed algorithm was validated using a 3-DOF teleoperation
is not affected by the passivity controllers under constant time system under different time delay situations, and was shown
delays. Also, unlike the conventional passivity controllers, the to be superior over previous algorithm we proposed reported
new passivity controllers guarantee the system stability by earlier.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

SUN et al.: NN-BASED PASSIVITY CONTROL OF TELEOPERATION SYSTEM UNDER TIME-VARYING DELAYS 15

R EFERENCES [29] L. Bate, C. D. Cook, and Z. Li, “Reducing wave-based teleopera-

tor reflections for unknown environments,” IEEE Trans. Ind. Electron.,
[1] G. Hirzinger, B. Brunner, J. Dietrich, and J. Heindl, “Sensor-based vol. 58, no. 2, pp. 392–397, Feb. 2011.
space robotics-ROTEX and its telerobotic features,” IEEE Trans. Robot. [30] D. Sun, F. Naghdy, and H. Du, “Transparent four-channel bilateral con-
Autom., vol. 9, no. 5, pp. 649–663, Oct. 1993. trol architecture using modified wave variable controllers under time
[2] T. B. Sheridan, “Space teleoperation through time delay: Review and delays,” Robotica, vol. 34, no. 4, pp. 859–875, 2014.
prognosis,” IEEE Trans. Robot. Autom., vol. 9, no. 5, pp. 592–606, [31] J.-H. Ryu, D.-S. Kwon, and B. Hannaford, “Stable teleoperation with
Oct. 1993. time-domain passivity control,” IEEE Trans. Robot. Autom., vol. 20,
[3] S. Hirose and K. Kato, “Development of quadruped walking robot with no. 2, pp. 365–373, Apr. 2004.
the mission of mine detection and removal-proposal of shape-feedback [32] J. Artigas, J. H. Ryu, and C. Preusche, “Time domain passivity con-
master-slave arm,” in Proc. IEEE Int. Conf. Robot. Autom., vol. 2. trol for position-position teleoperation architectures,” Presence Teleoper.
Leuven, Belgium, May 1998, pp. 1713–1718. Virtual Environ., vol. 19, no. 5, pp. 482–497, 2010.
[4] L. Cragg and H. Hu, “Application of mobile agents to robust teleopera- [33] Y. Ye, Y. J. Pan, Y. Gupta, and J. Ware, “A power-based time domain pas-
tion of Internet robots in nuclear decommissioning,” in Proc. IEEE Int. sivity control for haptic interfaces,” IEEE Trans. Control Syst. Technol.,
Conf. Ind. Technol., vol. 2. Maribor, Slovenia, 2003, pp. 1214–1219. vol. 19, no. 4, pp. 874–883, Jul. 2011.
[5] B. Barnes et al., “Virtual reality extensions into surgical training and [34] Y. Ye, Y. J. Pan, and T. Hilliard, “Bilateral teleoperation with time-
teleoperation,” in Proc. 4th Int. IEEE EMBS Spec. Topic Conf. Inf. varying delay: A communication channel passification approach,”
Technol. Appl. Biomed., Birmingham, U.K., Apr. 2003, pp. 142–145. IEEE/ASME Trans. Mechatronics, vol. 18, no. 4, pp. 1431–1434,
[6] Y. Rybarczyk, E. Colle, and P. Hoppenot, “Contribution of neuroscience Aug. 2013.
to the teleoperation of rehabilitation robot,” in Proc. IEEE Int. Conf. Syst. [35] V. Chawda and M. K. O’Malley, “Position synchronization in bilateral
Man Cybern., vol. 4. Hammamet, Tunisia, Oct. 2002, p. 6. teleoperation under time-varying communication delays,” IEEE/ASME
[7] A. E. Quaid and R. A. Abovitz, “Haptic information displays Trans. Mechatronics, vol. 20, no. 1, pp. 245–253, Feb. 2015.
for computer-assisted surgery,” in Proc. IEEE Int. Conf. Robot. [36] D. Sun, F. Naghdy, and H. Du, “Wave-variable-based passivity con-
Autom. (ICRA), vol. 2. Washington, DC, USA, 2002, pp. 2092–2097. trol of four-channel nonlinear bilateral teleoperation system under time
[8] D. Lee and M. W. Spong, “Passive bilateral teleoperation with constant delays,” IEEE/ASME Trans. Mechatronics., vol. 21, no. 1, pp. 238–253,
time delay,” IEEE Trans. Robot., vol. 22, no. 2, pp. 269–281, Apr. 2006. Feb. 2016.
[9] E. Nuño, L. Basañez, R. Ortega, and M. W. Spong, “Position tracking [37] C. C. Hua and X. P. Liu, “Delay-dependent stability criteria of tele-
for non-linear teleoperators with variable time delay,” Int. J. Robot. Res., operation systems with asymmetric time-varying delays,” IEEE Trans.
vol. 28, no. 7, pp. 895–910, 2009. Robot., vol. 26, no. 5, pp. 925–932, Oct. 2010.
[10] E. Nuño, L. Basañez, and R. Ortega, “Passivity-based control for bilat- [38] C. C. Hua, Y. Yang, and X. Guan, “Neural network-based adaptive
eral teleoperation: A tutorial,” Automatica, vol. 47, no. 3, pp. 485–495, position tracking control for bilateral teleoperation under constant time
2011. delay,” Neurocomputing, vol. 113, pp. 204–212, Aug. 2013.
[11] N. Chopra, M. W. Spong, and R. Lozano, “Synchronization of
bilateral teleoperators with time delay,” Automatica, vol. 44, no. 8, Da Sun received the B.Sc. degree in
pp. 2142–2148, 2008. telecommunication engineering from the University
[12] Y. Yang, C. Hua, and X. Guan, “Coordination control for bilateral tele- of Wollongong, Wollongong, NSW, Australia,
operation with kinematics and dynamics uncertainties,” Robot. Comput. in 2012, where he is currently pursuing the Ph.D.
Integr. Manuf., vol. 30, no. 2, pp. 180–188, 2014. degree in electrical engineering.
[13] F. L. Lewis, A. Yesildirek, and K. Liu, “Multilayer neural-net robot His current research interests include robotics
controller with guaranteed tracking performance,” IEEE Trans. Neural
Netw., vol. 7, no. 2, pp. 388–399, Mar. 1996. and teleoperation systems.
[14] M. K. Ciliz, “Adaptive control of robot manipulators with neural network
based compensation of frictional uncertainties,” Robotica, vol. 23, no. 2,
pp. 159–167, 2005.
[15] L. P. J. Veelenturf, Analysis and Applications of Artificial Neural
Networks. London, U.K.: Prentice-Hall, 1995. Fazel Naghdy received the first degree from Tehran
[16] Y. Yang, C. Ge, H. Wang, X. Li, and C. Hua, “Adaptive neural network University, Tehran, Iran, in 1976, and the M.Sc.
based prescribed performance control for teleoperation system under and Ph.D. degrees from the Postgraduate School
input saturation,” J. Frank. Inst., vol. 352, no. 5, pp. 1850–1866, 2015. of Control Engineering, University of Bradford,
[17] Y. Yang, C. Hua, and X. Guan, “Finite time control design for bilateral
teleoperation system with position synchronization error constrained,” Bradford, U.K., in 1979 and 1982, respectively.
IEEE Trans. Cybern., vol. 46, no. 3, pp. 609–619, Mar. 2015. He is currently a Professor of Robotics
[18] A. Forouzantabar, H. A. Talebi, and A. K. Sedigh, “Adaptive neural and Intelligent Systems with the University of
network control of bilateral teleoperation with unsymmetrical stochastic Wollongong, Wollongong, NSW, Australia. He
delays and unmodeled dynamics,” Int. J. Robust Nonlin. Control, vol. 24, is the Director of the Centre for Intelligent
no. 11, pp. 1628–1652, 2012. Mechatronics Research. He has a demonstrated
[19] Z. Li and C. Y. Su, “Neural-adaptive control of single-master–multiple- track record and leadership in research, teaching,
slaves teleoperation for coordinated multiple mobile manipulators with and management. His research has had its focus on machine intelligence
time-varying communication delays and input uncertainty,” IEEE Trans. and control particularly in embedded mechatronics and robotics systems.
Neural Netw. Learn. Syst., vol. 24, no. 9, pp. 1400–1413, Sep. 2013.
[20] Z. Li et al., “Neural network-based control of networked trilateral teleop- His current research interests include haptic rendered virtual manipulation
eration with geometrically unknown constraints,” IEEE Trans. Cybern., of clinical and mechanical systems, and intelligent control and learning in
vol. 46, no. 5, pp. 1051–1064, May 2016. nonlinear and nonstructured systems.
[21] Z. Li, Y. Xia, and C. Y. Su, Intelligent Networked Teleoperation Control. Prof. Naghdy is also a contributing Reviewer of the IEEE T RANSACTIONS
Heidelberg, Germany: Springer, 2015. ON M ECHATRONICS E NGINEERING , the International Journal of Intelligent
[22] Y. Yokokohji, T. Imaida, and T. Yoshikawa, “Bilateral teleoperation Automation, and Soft Computing.
under time-varying communication delay,” in Proc. IEEE/RSJ Int. Conf.,
vol. 3. Gyeongju, South Korea, 1999, pp. 1854–1859. Haiping Du received the Ph.D. degree in
[23] S. Munir and W. J. Book, “Internet-based teleoperation using wave vari- mechanical design and theory from Shanghai Jiao
ables with prediction,” IEEE/ASME Trans. Mechatronics, vol. 7, no. 2, Tong University, Shanghai, China, in 2002.
pp. 124–133, Jun. 2002.
He was a Research Fellow with the University
[24] Y. Ye and P. X. Liu, “Improving haptic feedback fidelity in wave-
variable-based teleoperation orientated to telemedical applications,” of Technology Sydney, Ultimo, NSW, Australia,
IEEE Trans. Instrum. Meas., vol. 58, no. 8, pp. 2847–2855, Aug. 2009. from 2005 to 2009, and a Post-Doctoral Research
[25] H. Li and K. Kawashima, “Achieving stable tracking in wave-variable- Associate with Imperial College London, London,
based teleoperation,” IEEE/ASME Trans. Mechatronics, vol. 19, no. 5, U.K., from 2004 to 2005, and the University of
pp. 1574–1582, Oct. 2014. Hong Kong, Hong Kong, from 2002 to 2003. He
[26] L. Chan, F. Naghdy, and D. Stirling, “Extended active observer for force is currently an Associate Professor with the School
estimation and disturbance rejection of robotic manipulators,” Robot. of Electrical, Computer and Telecommunications
Auton. Syst., vol. 61, no. 12, pp. 1277–1287, 2013. Engineering, University of Wollongong, Wollongong, NSW, Australia. His
[27] Y. Ye and P. X. Liu, “Improving trajectory tracking in wave-variable-
based teleoperation,” IEEE/ASME Trans. Mechatronics, vol. 15, no. 2, current research interests include vibration control, vehicle dynamics and
pp. 321–326, Apr. 2010. control systems, robust control theory and engineering applications, electric
[28] A. Aziminejad, M. Tavakoli, R. V. Patel, and M. Moallem, “Transparent vehicles, robotics and automation, and smart materials and structures.
time-delayed bilateral teleoperation using wave variables,” IEEE Trans. Dr. Du was a recipient of the Australian Endeavour Research Fellowship
Control Syst. Technol., vol. 16, no. 3, pp. 548–555, May 2008. in 2012.