Proceedings of 2017 IEEE

International Conference on Mechatronics and Automation

August 6 - 9, Takamatsu, Japan

Step Length Adaptation for Walking Assistance

Qiming Chen1 , Student Member, IEEE, Hong Cheng1 , Senior Member, IEEE, Chunfeng Yue1 , Rui Huang1 ,
Student Member, IEEE, Hongliang Guo1

Abstract— Lower exoskeleton has gained considerable inter- patient to walk with a reference gait. Most of these control
ests in rehabilitation and health care applications, since it has methods are combined with Finite State Machines (FSM),
the potential to help the paraplegia patients to walk again. which contains several gait trajectories in different walking
On the control of these lower exoskeletons, the predefined gait
control methods are commonly used, which requires the pilot patterns, such as in ATLAS[4], ReWalk[5], elegs[6], etc.
to adjust his/her movements to follow the predefined motion These methods achieve efficient results in walking assistance
trajectories. In the meanwhile, the desired trajectories are of paraplegic patients. However, there are two significant
predefined from healthy persons or extrapolated from clinical disadvantages of these methods which with predefined gait
gait analysis (CGA) datasets. However, in individual walking trajectories. Firstly for different patients, the gait trajectories
assistance situations, the exoskeleton should have the ability
to adapt with variant motions of the pilot, since the pilot of the exoskeleton should be corrected manually through trial
will change his motion in different walking situations. This and error. Secondly during normal walking, the patient can
paper presents a novel step length adaptation method to adapt not adjust his/her gait motion even slightly to adapt to the
the pilot’s motion for walking assistance exoskeletons. In this environment, e.g., before sitting down or stepping up/down
paper, the exoskeleton robot is model as a special Hybrid stairs. Moreover, the stability of the exoskeleton and the
Human-Exoskeleton Agent (HHEA), which considers both the
exoskeleton and the pilot. The Dynamic Movement Primitives energy cost of the patient also needs to be considered in
(DMP) is utilized to model the exoskeleton gait trajectories the gait trajectories.
dynamically, in which the relationship between the gait length In this paper, we present a novel step length adaptation
and Center of Mass (CoM) of HHEA is considered. In the method with dynamic gait models to model the exoskeleton
training process, a Reinforcement Learning (RL) method is
employed to update the parameters of dynamic gait model
gait flexibly, which enables the exoskeleton with the ability to
online. We demonstrate the efficiency of the proposed step learn/adapt the patient’s motion online. As a human-coupled
length adaptation method in simulation environment as well system, we modeled the exoskeleton with the pilot as a
as a lower limb exoskeleton system named as AssItive DEvice Hybrid Human-Exoskeleton Agent (HHEA). The Dynamic
for paRaplegics (AIDER). Experimental results shows that the Movement Primitive (DMP) architecture is employed to
proposed step length adaptation method is able to adapt variant
motions of the pilot during walking.
model the exoskeleton gait, which combining the relationship
Index Terms - Step Length Adaptation, Walking Assis- between the Center of Mass (CoM) of the HHEA and the
tance, Dynamic Movement Primitives, Reinforcement Learning, gait length. In order to learn the dynamic gait models online,
AIDER. a Reinforcement Learning (RL) method is utilized to update
the parameters of the gait models, in which the energy cost
I. INTRODUCTION
of the patient and the stability of the HHEA are considered
Many orthoses and lower exoskeletons have been devel- as cost terms. The paper’s contribution can be summarized
oped to help the paraplegic and quadriplegic people to regain as follows:
their locomotion abilities in lower limbs [1][4][5][6].
1) A novel step length adaptation method is proposed to
In recent years lower limb exoskeletons have gained
adapt to the patient’s motion, in which the exoskeleton
considerable interests since they can not only be utilized
with the pilot is modeled as a HHEA;
in clinical rehabilitation [2], also have the ability to serve
2) We employ a DMP architecture to model the exoskele-
the patients’ daily lives [1]. Lower limb exoskeletons have
ton gait flexibly, which combining the CoM of the
progressed from the stuff of science to commercialized
HHEA and the gait length;
products over last few decades [7]. However, lower limb
3) Parameters of the gait models are updated online with
exoskeletons also face challenge problems when serving
RL method.
patients in their daily lives. Gait planning is one of the critical
issues in walking assistance since the gait of the exoskeleton We first validated the proposed step length adaptation
will affect the patient’s movements directly. method in a simulation environment, and then test it on
The predefined gait trajectory control method is commonly a walking assistance exoskeleton system which named as
used in lower exoskeletons for walking assistance. With AssIstive DEvice for paRaplegics (AIDER). Experimental
predefined gait trajectories, the exoskeleton will force the results show that the advantages of the proposed step length
adaptation method over the traditional predefined gait tra-
1 Qiming Chen, Hong Cheng, ChunFeng Yue, Rui Huang, Hongliang
jectory control method, especially with consideration of the
Guo are with Center for Robotics, School of Automation Engineering,
University of Electronic Science and Technology of China, Chengdu, China
patient’s energy cost and the stability of the HHEA.
kipochen@163.com The structure of this paper is organized as follows. We
978-1-5090-6759-6/17/$31.00 ©2017 IEEE 644
first introduce the dynamic gait models for step length the patient and ‘hinder legs’ are actuated by the exoskeleton.
adaptation in Section II, in which the HHEA and DMP Fig. 1 shows the schematic diagram of the HHEA. As same
are also introduced. Section III lay down the reinforcement as most human-exoskeleton systems, the HHEA model has
learning algorithm for online adaptation of gait parameters. only four active joints on hips and knees. In order to simplify
Experimental results on both simulation environment and the the model, the DOF of the patient’s wrist is ignored, so the
AIDER system are presented and analyzed in Section IV. ‘front legs’ only have two DOFs in each shoulder joint and
This paper ends with conclusion and future work in Section one DOF in each elbow joint.
V. During normal walking of HHEA, a whole gait cycle
can be separated into two phases: double support phase and
II. DYNAMIC G AIT M ODELS FOR S TEP L ENGTH triple support phase. Fig. 2 illustrate normal walking gait of
A DAPTATION HHEA. In double support phase, both of feet is contacting
This section presents the methodology details of the with the ground and the patient move two sticks forward.
proposed dynamic gait models for step length adaptation. After two sticks touch the ground, one of the exoskeleton’s
Before introducing the models of dynamic gait, we regard leg swings front with the support of other three ‘stance legs’.
the assistance exoskeleton system as a Hybrid Human-
Exoskeleton Agent (HHEA) in Section II-A. In Section II- LF

B, based on the HHEA scheme, the Dynamic Movement LH

RF
Primitives (DMP) are employed to model the exoskeleton
RH
gait, which combining a dynamic compensation with CoM
of HHEA (introduced in Section II-C). Fig. 2. Gait bars of HHEA during normal walking (black bars indicate
stance phases and white bars indicate swing phases).
A. Gait of Hybrid Human-Exoskeleton Agent
Most lower exoskeletons are commonly under-actuated During each phase of normal walking, gait of HHEA is
since it can make the patient participate more in the walking described with motion trajectories of each joint. With the
assistance process [8].In these exoskeleton systems, there are kinematic constraints, the joint motion trajectories can be
only four active Degree-Of-Freedoms (DOFs) which provide determined by the foot trajectories and hip trajectories [16].
active torques on hips and knees in rotative orientation. Fig. 3 describe gait trajectories of HHEA in sagittal plane
Therefore, the patient should utilize two sticks to keep during a walking step. However, in the gait description of
balance with the exoskeleton system. HHEA, the hip trajectories is determined by the patient’s
upper body since there is not active torques provided on
the ankle joints. Therefore, we only consider the ankle
joints’ trajectories in sagittal plane, which indicate that the
trajectories of {x(t), z(t)} can express the gait of the HHEA.

Hip Trajectory

Z
Y

X
Foot Trajectory

Fig. 1. The schematic diagram of the HHEA (joints in blue belong to the
pilot and joints in black belong to the exoskeleton).

Hence, we regard the human-exoskeleton system as a

Hybrid Human-Exoskeleton Agent (HHEA), which actuated
by both the exoskeleton and the patient. In the HHEA
scheme, the human-exoskeleton system is regarded as a Fig. 3. Gait trajectories of HHEA in sagittal plane during a walking step.
special quadruped robot, which ‘front legs’ are actuated by
645
B. Model Gait with DMP Fig. 4 shows different step length situations during normal
DMP has been widely employed in robotic applications walking. It indicates that if the step length is too long for a
since it can solve flexible modelling problems with coupled particular step, the CoM’s projection will out of the support
terms [11][12][15]. Moreover, DMP is easy to learn with polygon in the next step (which means the system will fall
statistical methods and can be adapted through a few param- down). Another important issue is that if the step length is
eters after imitation learning [11]. In this paper, we employed too short, the patient should provide more strength to support
discrete DMPs to model the gait dynamically. The system the system. This feature will be utilized as cost terms in the
equations of DMP are modeled as a linear spring system learning process and will be introduced in Section III.
perturbed by an external force term:
Back-Left Front-Right
τ v̇ = K(g − x) − Dv + (g − x0 ) f , (1) CoM
Step Length
τ ẋ = v, (2) (0.2m)

where x and v indicate the position and velocity of the Back-Right Front-Left

system, respectively. x0 and g are the start and goal position.

τ is a temporal scaling factor. K and D are the spring and Step Length
damping factors of the system. f is a nonlinear function (0.1m)
combining with Gaussian kernels which leads DMP can
model complex movements:
Back-Right Swing Back-Left Swing
∑Ni=1 ωi ψi (s)s
Front Legs Swing
f (s) = , (3)
∑Ni=1 ψi (s) Fig. 4. Different step length situations during normal walking (blue points
where ψ = exp(−hi (s − ci )2 ) are Gaussian basis functions indicate positions of the CoM of HHEA).
with centers ci and width hi . wi are weights which should
be learned. The phase variable in the nonlinear function (3) The relationship between the CoM’s projection and the
is utilized to avoid f directly dependence on time. Here we step length of HHEA in sagittal plane is depicted in Fig. 5. In
utilize a first-order dynamics to define the phase variable x: this paper, we utilize a dynamic compensation term to online
adapt the goal state of each step (which also regarded as step
τ ṡ = αs. (4) length of HHEA). The compensated goal g is represented as
When the goal g is close to the start position x0 , a (8):
XCoM
small change in g may lead huge accelerations, which can g = ginit + α(X̂CoM − ), (8)
achieve the limitation of the exoskeleton system. Therefore, XSP
we employed modified system equations introduced in [17]: where α(X̂CoM − XXCoM ) is the dynamic compensation term.
SP
τ v̇ = K(g − x) − Dv + K(g − x0 )s + K f (s), (5) α and X̂CoM are compensation parameters which should
be learned online. The details of learning process will be
τ ẋ = v. (6) introduced in Section III.
The third term in (5) can avoid jump movements at the
III. R EINFORCEMENT L EARNING FOR G AIT A DAPTATION
beginning of each step. After obtaining the target function
ftarget through (7), the weighted parameters ωi can be learned In the previous modulated DMP architectures, many im-
via statistical learning methods [18]. itation learning methods are employed to learn/determine
τ v̇ + Dv the parameters of the modulation term of DMPs [11][9].
ftarget (s) = − (g − x) + (g − x0 )s. (7) However, during normal walking of HHEA, the exoskeleton
K
should adapt the patient’s gait online, which leads the gait
With specified start position x0 and goal position g, the models should be updated online if the patient want to adjust
gait model can be generated through the learn weights ωi . his/her gait patterns.
C. Dynamic Compensation with CoM In this paper, we employ a model-free reinforcement
During normal walking of HHEA, the most important learning algorithm with parameterized policies, which name
for the exoskeleton is keeping balance. In most cases, the is Policy Improvement and Path Integrals (PI2 ) [10]. The
patient will use sticks to keep balance. However, if the patient PI2 algorithm utilizes a dynamic control system (DCS) to
want to adjust his/her steps because of the environment or describe the parameterized policies:
specifical tasks, the exoskeleton should have the ability to ẋt = f (xt ,t) + G(xt )(ut + εt ), (9)
online adapt the patient’s gait. In this paper, we embedded
a dynamic gait compensation into the original DMP archi- where xt is the state of the DCS. ut is input of the DCS which
tecture based on the CoM of HHEA, which described as the is a function of parameters θ (θ is a vector of parameters
relationship between the step length and the projection of [α, X̂CoM ]T which should be learned) and system state xt . εt
CoM. is a Gaussian noise with variance δ . G(xt ) is the system
646
 TABLE I
R EINFORCEMENT L EARNING P ROCESS FOR G AIT PARAMETERS
A DAPTATION .
truck

Input: θinit , δ , N, K
arm
Output: θ
1. while the system cost R is not converged do
2. Create K roll-outs of the system from the same start with
thigh noise εk ∼ N(0, δ 2 ).
3. for k = 1 → K do
4. for i = 1 → N do
5. Update the stochastic cost S through (12)
stick 6. Update the probability P through (13)
7. end for
shank 8. end for
9. for i = 1 → N do
10. Update δ θti for each time step through (14)
z 11. end for
12. Update δ θ through (15)
x
13. Update θ ← θ + δ θ
X CoM 14. Create one noiseless roll-out to evaluate the trajectory cost
X sp R of updated θ .
15. end while
16. return θ

Fig. 5. The relationship between the CoM’s projection and the step length
of HHEA.
The parameter λ in (13) is set automatically within (0, 1].
After obtaining the updated parameters θ of dynamic gait
control matrix, and f (xt ,t) is the passive dynamics of the models, the HHEA should take a noiseless roll-out which
control system. without exploration noise εt to compute the cost R. With the
In the implementation of HHEA, the immediate cost reinforcement learning process , the parameters of dynamic
function is defined based on the patient energy cost: gait models can be updated online to adapt the patient’s gait,
�t
f dt which aiming to reduce the energy cost of the patient as well
0
rt =
, (10) as stable walking.
t
where f is ground reaction forces of sticks. For each roll- IV. E XPERIMENTS AND D ISCUSSIONS
out in reinforcement learning process, the immediate cost
In this section, experiments are carried out to validate the
will be calculated in each step of the patient. The total cost
proposed step length adaptation method on both simulation
of each roll-out is combined with the immediate cost and a
platform and real-time exoskeleton system. Experimental
final cost. In this paper, we utilize the balance of HHEA as
setup, results and discussions are provided in the following
the final cost of each roll-out. Hence, the cost function of
two subsections.
each roll-out can be defined as:
N −m N A. Dynamic Simulation
R = λ1 + λ2 ∑ ri , (11)
N i=1 1) HHEA in Simulation Platform: In the simulation en-
vironment, the exoskeleton with the patient are modeled as
where λ1 and λ2 are weighted parameters. N indicate the
HHEA which described as Fig. 1. The HHEA is modeled as
number of steps for each roll-out and m means the HHEA
a special quadruped robot which has total 18 DOFs. As same
fall down after m steps. Here the discrete form of immediate
with the exoskeleton in real-time applications, the HHEA in
cost of each step is utilized to calculate the total cost.
simulation environment has 10 active joints, in which 4 active
After defining the policies and cost function, the learning
joints are in exoskeleton hips and knees, and rest 6 active
process is shown in Tab. I, with equations (12) to (15).
joints are aiming to actuate the ‘front legs’ of HHEA. The
1 N−1 simulation platform is based on Gazebo environment, Fig. 6
S(i, k) = Rk + ∑ (θ + εk )2 ,
2 j=1
(12)
shows the HHEA model in simulation environment.
The HHEA’s motion in simulation environment is planned
1
e− λ S(i,k) separately in two parts. The first part is the motion planners
P(i, k) = 1 , (13) for exoskeleton legs, which include motion trajectories of
∑Kk=1 [e− λ S(i,k) ]
knee joints and hip joints. Another part is the planners of
K the patient, which aiming to move the sticks (‘front legs’ of
δ θti = ∑ [P(i, k)εk ], (14) HHEA) to given goal places.
k=1
In order to obtain some basic information of HHEA, three
∑N−1
i=1 (N − i)δ θti kinds of sensors are utilized to measure the feedback which
δθ = . (15)
∑N−1
i=1 (N − i)
can be employed in the proposed step length adaptation
647
 the learned dynamic gait.

9000
Initial Param: A
8000 Initial Param: B

7000

Cost
6000

5000

4000

3000
0 2 4 6 8 10 12 14 16
Updates
(a) Cost during learning process

Fig. 6. The HHEA in simulation environment.

10
Initial Param: A

Step Number
8 Initial Param: B
method. The Inertial Measurement Unit (IMU) sensor at-
tached on the upper body is utilized to estimate whether the 6
HHEA is fall down or not. On each stick, pressure sensors are
installed to measure the ground reaction force on sticks. For 4
each joint in HHEA, a joint encoder is embedded to obtain
2
joint states, and also can be utilized to calculate joint’s power 0 2 4 6 8 10 12 14 16
of the pilot. Updates
2) Experimental Setup: In the dynamic simulation exper- (b) Average stable steps of one roll-out
iments, the pilot’s intention (goal places of sticks) during
Fig. 7. Experimental results with different initial parameters in simulation
normal walking is set as a noisy parameter with Gaussian environment.
white noise, which makes each step of the pilot is not
fixed in the simulation environment. The dynamic gait of
HHEA is modeled through DMP with CoM compensation, After obtaining optimal parameters of dynamic gait mod-
which introduced in Section II. In the reinforcement learning els, we compare the performance of proposed step length
process, the parameters of dynamic gait models are updated adaptation method with traditional fixed step length methods.
every 5 roll-outs. During each roll-out, the HHEA takes 10 In the experiment, the torque of HHEA’s upper body is
gait cycles with the updated parameters including exploration utilized to validate the efficiency of the proposed dynamic
noises. gait, which indicate that the strength from the patient. Fig. 8
illustrates the performances of proposed step length adapta-
TABLE II
tion method with fixed step length methods, the results show
I NITIAL PARAMETERS IN SIMULATION EXPERIMENTS .
that the proposed method can reduce the provided torque of
Parameters X̂CoM α
the patient after learning.
Initial parameters A 0.6 -1.0
Initial parameters B 0.9 -0.4 Dynamic Gait
Torque of Upper Body (N.M)

800 Fixed Step Length :0.1m

Fixed Step Length :0.12m
700
3) Experimental Results: The first step in simulation Fixed Step Length :0.14m
Fixed Step Length :0.16m
experiment is learning the optimal parameters of dynamic 600
Fixed Step Length :0.18m
gait models. In the learning process, we choose two different 500 Fixed Step Length :0.20m
initial parameters to validate the efficiency of the learning 400
algorithm. The initial parameters are given in Tab. II.
300
The experimental results are shown in Fig. 7. As depicted
in Fig. 7(a), the parameters of dynamic gait will be converged 200

to optimal parameters (X̂CoM = 0.80, α = −8.4) after almost 100

2 4 6 8 10
10 updates. The experimental results indicate that with Step Number
different initial parameters, the learning algorithm has the
ability to obtain the optimal parameters of dynamic gait. Fig. Fig. 8. Comparison of proposed step length adaptation method with fixed
7(b) gives the average stable steps of one roll-out during each step length methods in simulation environment.
update, the results show that the HHEA can walk stably with
648
B. Experiments on the AIDER System ensure safety of the pilot during the experiment. In order to
1) Introduction to AIDER System: The AIDER system keep walking in the experiment, the pilot can turn around
is design for walking assistance of paraplegic patients with with a rotary table. The initial parameters of dynamic gait
lower extremities, which aiming to help the paraplegia to are chosen as the same with parameters A in simulation
stand up and walking again. The AIDER system has total experiments. In the experiments, the pilot put the sticks
four active joints in hips and knees, which activated by DC randomly in each step in order to validate the efficiency of
servo motors. Fig. 9 illustrate the AIDER system with a pilot. the proposed step length adaptation method.
The ankle joints of AIDER system are designed as energy-
storage mechanism which can store energy in stance phase
1
and release it in swing phase during normal walking. Besides
the joints and rigid links, many compliant connections at
waist, thighs, shanks and feet are provided for semi-rigid
connecting AIDER to the pilot.
2

2 3

Fig. 10. The experimental environment of the AIDER system:1. Ceiling

rail; 2. Safety rope 3. Rotary table;

3) Results and Discussions: Fig. 11 illustrates the aver-

4 age pressure of pressure sensors on sticks during learning
process. As shown in Fig. 11, the pilot should take almost
300 steps to obtain optimal parameters of the dynamic gait
Fig. 9. The AIDER system with a pilot: 1. Battery and the main controller; models. The results also indicate that after learning the
2. IMU sensor and pressure sensor; 3. DC motors and node controllers; 4.
Plantar sensors. optimal dynamic gait, the pilot can provide less torques to
support the whole system.
A distributed control system with four node controllers and
a main controller is built for real-time control applications 500
of the AIDER system. The main controller is set on the Dynamic Gait
Pressure of Sticks (N)

backpack, which running the control algorithm of the AIDER 450

system. Nearby each node controller, a node controller is
400
utilized to collect sensor data and execute commands from
the main controller. AIDER has total four kinds of sensor 350
to measure its current state. Encoders are interpreted in
joint actuators to measure the current state of each joint. 300
IMU sensors and pressure sensors on sticks are employed
250
to measure orientation of the pilot’s arm and contact forces
between the stick and the ground respectively. The plantar 200
sensors are installed on the shoes of the AIDER system to 0 100 200 300 400 500 600 700
estimate gait phases. Step Number
2) Experimental Setup: In experiments on the AIDER Fig. 11. Average pressure of pressure sensors on sticks during learning
system, a healthy pilot is chosen to operate the AIDER process.
system. Thus, the pilot is requested to not move his lower
limbs actively, which regarded as a paraplegic patient. The After the learning process, we compare the proposed step
experimental environment of the AIDER system is shown length adaptation method with fixed step length methods on
in Fig. 10. The ceiling rail and safety rope are installed for the AIDER system. In the experiments, the pressure of sticks
649
are measured to validate the efficiency of those methods. on both in simulation and real system demonstrates that the
Fig. 12 shows the experimental results of comparison of efficiency of the proposed method.
proposed step length adaptation method with fixed step In the future, we will focus on the online learning of tra-
length methods. As shown in Fig. 12 (a), the proposed step jectories in different walking situations. Furthermore, com-
length adaptation method can reduce the energy cost of the bining with perception of environment, we can learn the gait
pilot. However, if the step length is fixed in a large value, of exoskeleton online to adapt with different environment.
e.g. 0.2m, the energy cost of the pilot is also small as the VI. ACKNOWLEDGMENTS
proposed step length adaptation method. However, a large
step length will make the AIDER system fall down after This work was made possible by support from NSFC (No.
few steps, which shown in Fig. 12 (b). If the step length is 6150020696, 6157021026) and Fundamental Research Funds
fixed in 0.2m, the AIDER system will fall down in 6 steps. for the Central Universities (ZYGX2014Z009).
Hence, the experimental results indicate that the proposed R EFERENCES
step length adaptation method has better performance than [1] Y. Sankai, “HAL: Hybrid Assistive Limb based on Cybernics”,
traditional fixed step length methods, not only can reduce the Robotics Research, 2011, pp 25-34
energy cost of the pilot, but also can ensure stable walking. [2] H.K. Kwa, J.H. Noorden, M. Missel, et al., “Development of the IHMC
Mobility Assist Exoskeleton” In IEEE International Conference on
Robotics and Automation (ICRA), 2009, pp 2556-2562
[3] S. Lee, Y. Sankai, “Power Assist Control for Walking Aid with HAL-3
500 Based on EMG and Impedance Adjustment around Knee Joint”, In
0.1m
International Conference on Intelligent Robots and Systems (IROS),
Pressure of Sticks (N)

0.12m
450
0.14m 2002, pp 1499-1504
0.16m [4] D. Sanz Merodio, M. Cestari Soto, J.C. Arevalo, et al., “Control
400 0.18m Motion Approach of a Lower Limb Orthosis to Reduce Energy
0.2m Consumption, International Journal of Advanced Robotic Systems,
350 Dynamic Gait 2012, 9:1-18
[5] M. Talaty, A. Esquenazi, J.E. Briceno, “Differentiating Ability in Users
of the ReWalk TM Powered Exoskeleton: An Analysis of Walking
300 Kinematics”, In IEEE International Conference on Rehabilitation
Robotics (ICORR), 2013, pp 1-5
250 [6] K.A. Strausser, H. Kazerooni, “The Development and Testing of a
Human Machine Interface for a Mobile Medical Exoskeleton”, In
0 100 200 300 400 IEEE International Conference on Intelligent Robots and Systems
Ticks (20ms) (IROS), 2011, pp 4911-4916
(a) Measured pressure of sticks [7] T. Yan, M. Cempini, C.M. Oddo, et al., “Review of Assistive Strategies
in Powered Lower-Limb Orthoses and Exoskeletons”, Robotics and
Autonomous Systems, 2015, 64:120-136
10
[8] S. Wang, L. Wang, C. Meijneke, et al., “Design and Control of the
MINDWALKER Exoskeleton”, IEEE transactions on neural systems
8 and rehabilitation engineering, 2015, 23(2): 277-286
Step Number

[9] J. Buchli, F. Stulp, E. Theodorou, et al., “Learning Variable Impedance

6 Control”, The International Journal of Robotics Research, 2011, 30(7):
820-833
4 [10] E. Theodorou, J. Buchli, S. Schaal, “A Generalized Path Integral
Control Approach to Reinforcement Learning”, Journal of Machine
Learning Research, 2010, 11(nov): 3137-3181
2
[11] A.J. Ijspeert, J. Nakanishi, S. Schaal, “Movement Imitation with
Nonlinear Dynamical Systems in Humanoid Robots”, In IEEE In-
0 ternational Conference on Robotics and Automation (ICRA), 2002, pp
ASL 0.1 0.12 0.14 0.16 0.18 0.2 1398-1430
Step Length (m) [12] A.J. Ijspeert, J. Nakanishi, S. Schaal, “Learning Rhythmic Movements
(b) Stable step number in 10 steps by Demonstration using Nonlinear Oscillators”, In IEEE International
Conference on Intelligent Robots and Systems (IROS), 2002, pp 958-
Fig. 12. Comparison of proposed step length adaptation method with 963
fixed step length methods on the AIDER system (ASL means adaptive step [13] J. Nakanishi, J. Morimjoto, G. Endo, et al., “Learning from Demon-
length). stration and Adaptation of Biped Locomotion”, Robotics and Au-
tonomous Systems, 2004, 47(2): 79-91
[14] H.B. Amor, G. Neumann, S. Kamthe, et al., “Interation Primitives for
Human-Robot Cooperation Tasks”, In IEEE International Conference
on Robotics and Automation (ICRA), 2014, pp 2831-2837
V. C ONCLUSIONS AND F UTURE W ORK [15] A. Gams, B. Nemec, A.J. Ijspeert, “Coulpling Movement Primitives:
Interaction with the Environment and Bimanual Tasks”, IEEE Trans-
In this paper, we modeled the exoskeleton robot as a spe- action on Robotics, 2014, 30(4): 816-830
cial Hybrid Human-Exoskeleton Agent (HHEA), a new step [16] Q. Huang, K. Yokoi, S. Kajita, et al., “Planning Walking Patterns for a
Biped Robot”, IEEE Transactions on Robotics and Automation, 2001,
length adaptation method of HHEA based on reinforcement 17(3): 280-289
learning and dynamic movement primitives (DMP). The [17] Pastor, Peter, et al. “Learning and Generalization of Motor Skills by
Dynamic Movement Primitives (DMP) is utilized to model Learning from Demonstration”, In IEEE International Conference on
Robotics and Automation (ICRA), 2009, pp 763-768
the exoskeleton gait trajectories dynamically. Reinforcement [18] D. Nguyen-Tuong, M. Seeger, J. Peters, “Model Learning with
learning methods are utilized to learn the parameters of DMP, Local Gaussian Process Regression”, Advanced Robotics, 2009,
which aims to adapt variant motions of the pilot. Experiments 23(15):2015-2034

650