Journal of Intelligent and Robotic Systems 37: 117–141, 2003.

117
© 2003 Kluwer Academic Publishers. Printed in the Netherlands.

Survey of Intelligent Control Techniques

for Humanoid Robots

DUŠKO KATIĆ and MIOMIR VUKOBRATOVIĆ

Robotics Laboratory, Mihailo Pupin Institute, Volgina 15, 11000 Belgrade, Yugoslavia;
e-mail: {dusko,vuk}@robot.imp.bg.ac.yu

(Received: 29 July 2002; in final form: 31 December 2002)

Abstract. This paper focusses on the application of intelligent control techniques (neural networks,
fuzzy logic and genetic algorithms) and their hybrid forms (neuro-fuzzy networks, neuro-genetic
and fuzzy-genetic algorithms) in the area of humanoid robotic systems. It represents an attempt to
cover the basic principles and concepts of intelligent control in humanoid robotics, with an outline
of a number of recent algorithms used in advanced control of humanoid robots. Overall, this survey
covers a broad selection of examples that will serve to demonstrate the advantages and disadvantages
of the application of intelligent control techniques.

Key words: humanoid robots, neural networks, fuzzy logic, genetic algorithms.

1. Introduction

Many aspects of modern life involve the use of intelligent machines capable of op-
erating under dynamic interaction with their environment. In view of this, the field
of biped locomotion is of special interest when human-like robots are concerned.
Humanoid robots as anthropomorphic walking machines have been in operation
for more than twenty years. Currently, research on humanoid robots and biped
locomotion is one of the most exciting topics in the field of robotics. There are
more than 50 major humanoid robot projects around the world, along with many
other bipedal walking projects (an extensive list of projects is given at the site
www.androidworld.com). The reason for increasing research interest in this do-
main is that major application areas have become self-evident. Humanoid robots
are expected to be servants and maintenance machines with the main task to assist
human activities in our daily life and to replace humans in hazardous operations. It
is as obvious as interesting that anthropomorphic biped robots are potentially capa-
ble to effectively move in all unstructured environments where humans do. Hence,
particularly, the fields of service robotics, medical applications, and operation in
hazardous environments are of primary importance. Another important reason for
the growth of humanoid robots research represents the development of advanced
118 D. KATIĆ AND M. VUKOBRATOVIĆ

technologies in design and production of robot sensors, actuators and computing

units.
Recently, significant progress has been made in the design of a hardware plat-
form for humanoid robots and control of humanoid robots, particularly in the re-
alization of dynamic walking in several full-body humanoids (www.world.honda.
com/robot; Yamazaki et al., 2000; Hirai et al., 1998; Yamaguchi et al., 1998; Hirai,
1997; Sardain et al., 1998). The major class of humanoid robots use electric motors
as actuators, while some types operate with fluidic (pneumatic) actuators (Schulz
and Bretthauer, 2001; Klute et al., 1999). For example, still in 1986, HONDA
(www.world.honda.com/robot) commenced the humanoid research and devel-
opment program, which resulted in a series of robot prototypes (P1, P2, P3 and
ASIMO). Key elements of the HONDA humanoid robot development included
“intelligence and mobility” with intention to use humanoid robot in daily life,
rather than a robot purpose-built for special operations. The design includes two-
foot/leg mobility technology to make it compatible with most types of terrain,
including very rough surfaces. Kitano’s Symbiotic Systems Project team (www.
symbio.jst.go.jp) has joined electronic component maker Murata Manufactur-
ing in developing “Morph”, Bluetooth-enabled humanoid robot. With full duplex
data transmission at 720 kbps, Bluetooth could allow robots to be remotely con-
trolled in real time while sending video images or sensor data to an operator, an
advantage as mobile humanoid robots are envisioned for dangerous search and
rescue operation. At the Institute of Applied Mechanics, Technical University of
Munich, Germany, humanoid robot JOHNNIE (Pfeiffer et al., 2002) was designed
that can already walk with intention to run.
Humanoid robot applications usually demand the robot be highly intelligent.
Intelligent humanoid robots are functionally oriented devices, built to perform sets
of tasks instead of humans. They are autonomous systems capable of extracting
information from their environments and using knowledge about the world and
intelligence of their duties and proper governing capabilities. Intelligent humanoid
robots should be autonomous to move safely in a meaningful and purposive man-
ner, i.e. to accept high-level descriptions of tasks (specifying what the user wants
to be done, rather than how to do it) and would execute them without further
human intervention. They have to be intelligent to determine all possible actions
in an unpredictable dynamic environment using information from various sensors.
Human operator can transfer to the robot his knowledge, experience and skill in
advance, to make it capable of solving complex tasks. Future humanoid robots are
likely to have greater sensory capabilities, more intelligence for valid reasoning
and decision making, higher levels of manual dexterity and adequate mobility as
compared to humans.
Naturally, the first approach to making humanoid robots more intelligent was
the integration of sophisticated sensor systems as computer vision, tactile sensing,
ultrasonic and sonar sensors, laser scanners and other smart sensors. However,
today’s sensor products are still very limited in interactivity and adaptability to
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 119

changing environments. As the technology and algorithms for real-time 3D vision

and tactile sensing improve, humanoid robots will be able to perform tasks that
involve complex interaction with the environment (e.g., grasping and manipulating
the objects). A major reason is that uncertainty and dynamic changes make the de-
velopment of reliable artificial systems particularly challenging. On the other hand,
to design robots and systems that best adapt to their environment, the necessary
research includes investigations in the field of mechanical robot design (intelligent
mechanics), environment perception systems and embedded intelligent control that
ought to cope with the task complexity, multi-objective decision making, large
volume of perception data and substantial amount of heuristic information. Also, in
the case when the robot performs in an unknown environment, the knowledge may
not be sufficient. Hence, the robot has to adapt to the environment and to be capable
of acquiring new knowledge through the process of learning. The robot learning is
essentially concerned with equipping robots with the capacity of improving their
behaviour over time, based on their incoming experiences.
There are several intelligent paradigms that are capable of solving intelligent
control problems in humanoid robotics. Connectionist theory (NN – neural net-
works), fuzzy logic (FL), and theory of evolutionary computation (GA – genetic
algorithms), are of great importance in the development of intelligent humanoid ro-
bot control algorithms. Due to their strong learning and cognitive abilities and good
tolerance of uncertainty and imprecision, intelligent techniques have found wide
applications in the area of advanced control of humanoid robots. Also, of great
importance in the development of efficient algorithms are the hybrid techniques
based on the integration of particular techniques such as neuro-fuzzy networks,
neuro-genetic algorithms and fuzzy-genetic algorithms.
Intelligent control systems can benefit from the advances in artificial neural net-
works (Rumelhart and McClelland, 1986; Haykin, 1994) as a tool for on-line learn-
ing optimisation, and optimal policy making. The connectionist systems (neural
networks) represent massively parallel distributed networks with the ability to serve
in advanced robot control loops as learning and compensation elements using the
abilities of nonlinear mapping, learning, parallel processing, self-organizing and
generalization.
The fuzzy control systems (Zimmermann, 1990; Terano et al., 1992) based on
mathematical formulation of fuzzy logic have the capability of representing human
knowledge and experience as a set of fuzzy rules. Fuzzy robot controllers use
human know-how or heuristic rules in the form of linguistic if-then rules, while
fuzzy inference engine computes the efficient control action for a given purpose.
The technique of evolutionary computation with genetic algorithms (Goldberg,
1989; Haupt and Haupt, 1998) represents an approach to global optimization search
which is based on the mechanics of natural selection and natural genetics. It com-
bines survival of the fittest among string structures with a structured yet random-
ized information exchange to form a search algorithm with the expected ever-
improving performance.
120 D. KATIĆ AND M. VUKOBRATOVIĆ

Each of the proposed paradigms has its own merits and drawbacks. To overcome
the drawbacks, certain integration and synthesis of hybrid techniques (symbiotic
intelligence) are needed for efficient application in humanoid robotics. Symbiotic
Intelligence incorporates a new type of humanoid robotics system having many
degrees of freedom (DOFs) and multi-modal sensory inputs. The underlying idea is
that the richness of inputs to and outputs from the system, along with co-evoluting
complexity of the environment mixed with various intelligent control paradigms,
is the key to the emergence of intelligence. For example, neuro-fuzzy networks
represent a combined tool where human operators are able to give their knowledge
by means of membership functions. On the other hand, membership functions
are modified through learning process as fine tuning by neural networks. After
learning, the human operator can understand the acquired rules in the network.
Neuro-fuzzy networks are faster than the conventional neural networks in terms of
convergence of the learning. Also, fuzzy logic and neural networks can be eval-
uation functions for the genetic algorithms. At the same time, genetic algorithms
can be structure optimizers for fuzzy and neural algorithms. The computational in-
telligence techniques map well onto nonlinear problems and are better at handling
uncertainties like those that can be encountered in running over irregular terrain
with obstacles.
All these techniques may be incorporated in advanced and sophisticated control
systems of humanoid robots that were inspired in general by biological designs
and neurobiological principles (Giszter et al., 2000; Garcia et al., 2000; Kiria-
zov, 2001). In the last two decades, many researchers explored the design of au-
tonomous systems, swarms of intelligent agents, and biologically inspired control
designs and actuators (Doya et al., 2001; Guihard and Gorce, 2001; Vijayakumar
and Schaal, 2000; Billard and Mataric, 2000; Mataric et al., 1998; Kawato et al.,
1987; Arkin, 1999; Brooks, 1997; Kawato, 1999). The research in this area is spe-
cially oriented toward the ideas of artificial life and adaptive behavour. An ultimate
goal is the creation of a humanoid robot as autonomous agent which is capable of
mimicking all aspects of human action, perception and cognition in everyday life
and in remote and unfriendly environments.
The purpose of this survey is to present the main control problems in humanoid
robotics and the background of some intelligent techniques as new paradigms and
tools for solving control problems in humanoid robotics. An account of the basic
principles and concepts is given, with the outlines of a number of relevant algo-
rithms that have been shown to simulate or use the diversity of intelligent concepts
for sophisticated humanoid robot control systems.

2. Control Problems in Humanoid Robotics

In spite of a significant progress and accomplishments achieved in the design of a
hardware platform of humanoid robot and synthesis of advanced intelligent control
of humanoid robots, a lot of work has still to be done in order to improve actuators,
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 121

sensors, materials, energy accumulators, hardware, and control software that can
be utilized to realize user-friendly humanoid robots. We are still in an initial stage
when the understanding of the motor control principles and sensory integration
subjacent to human walking is concerned.
There are various sources of control problems and various tasks and criteria that
must be solved and fulfilled in order to create valid walking and other functions of
humanoid robots. Previous studies of biological nature, theoretical and computer
simulation, have focussed on the structure and selection of control algorithms ac-
cording to different criteria such as energy efficiency, energy distribution along the
time cycle, stability, velocity, comfort, mobility, and environment impact. Never-
theless, in addition to these aspects, it is also necessary to consider some other
issues: capability of mechanical implementation due to the physical limitations
of joint actuators, coping with complex highly-nonlinear dynamics and uncertain-
ties in the model-based approach, complex nature of periodic and rhythmic gait,
inclusion of learning and adaptation capabilities, computation issues, etc.
The major problems associated with the analysis and control of bipedal systems
are the high-order highly-coupled nonlinear dynamics and furthermore, the discrete
changes in the dynamic phenomena due to the nature of the gait. Irrespective of the
humanoid robot structure and complexity, the basic characteristic of all bipedal
systems are: (a) the DOF formed between the foot and the ground is unilateral and
underactuated (Goswami, 1999); (b) the gait repeatability (symmetry) and regular
interchangeability of the number of legs that are simultaneously in contact with the
ground. During the walk, two different situations arise in sequence: the statically
stable double-support phase in which the mechanism is supported on both feet
simultaneously, and statically unstable single-support phase when only one foot
of the mechanism is in contact with the ground. Thus, the locomotion mechanism
changes its structure during a single walking cycle from an open to a closed kine-
matic chain. Also, it is well known that through the process of running the robot
can be most of the time in no-support phase. In this case, the control schemes that
are successful for walking problem are not necessarily successful for the running
problem. All the mentioned characteristics have to be taken into account in the
synthesis of advanced control algorithms that accomplish stable, fast and reliable
performance of humanoid robots.
The stability issues of humanoid robot walking are the crucial point in the
process of control synthesis. In view of this humanoid walking robots can be
classified in three different categories (Marchese et al., 2001). First category rep-
resents static walkers, whose motion is very slow so that the system’s stability
is completely described by the normal projection of the centre of gravity, which
only depends on the joint’s position. Second category represents dynamic walkers,
biped robots with feet and actuated ankles. Postural stability of dynamic walkers
depends on joint’s velocities and acceleration too. These walkers are potentially
able to move in a static way provided they have large enough feet and the motion
is slow. The third category represents purely dynamic walkers, robots without feet.
122 D. KATIĆ AND M. VUKOBRATOVIĆ

In this case the support polygon during the single-support phase is reduced to a
point, so that static walking is not possible. In the walk with dynamic balance,
the projected centre of mass is allowed outside of the area inscribed by the feet,
and the walker may essentially fall during parts of the walking gait. The control
problems of dynamic walking are more complicated than in walking with static
balance, but dynamic walking patterns provide higher walking speed and greater
efficiency, along with more versatile walking structures.
For all the mentioned categories of walking robots, the issue of stable and reli-
able bipedal walk is the most fundamental and yet unsolved with a high degree of
reliability. This subject has been studied mainly through the following two classes
of walking pattern generators and robot controllers. The first approach is to gener-
ate a dynamically consistent periodic walking pattern off-line. It is done assuming
that the models of robot and environment are available, and the kinematic and
dynamic parameters of the robot model are precisely defined (Hirai et al., 1998;
Yamaguchi et al., 1998). On the other hand, the second approach uses limited or
simplified knowledge of the system’s dynamics (Raibert, 1986; Zheng and Shen,
1990). However, in this case, the control relies much on the feedback control, and it
is necessary to develop methods without high computation resources for real-time
implementation.
The rotational equilibrium of the foot is the major factor of postural instability
with legged robots. The question has motivated the definition of several dynamic-
based criteria for the evaluation and control of balance in biped locomotion. The
most common criteria are the centre of pressure (CoP), the zero-moment point
(ZMP) and the foot-rotation indicator (FRI) (Sardain and Bessonet, 2001; Vukobra-
tović et al., 2002; Goswami, 1999). Of these criteria, the ZMP concept has gained
widest acceptance and played a crucial role in solving the biped robot stability
and periodic walking pattern synthesis (Vukobratović et al., 2002). The ZMP is
defined as the point on the ground about which the sum of all the moments of
the active forces equals zero. If the ZMP is within the convex hull of all contact
points between the foot and the ground, the biped robot can walk.
For a legged robot walking on complex terrain, such as a ground consisting of
soft and hard uneven parts, a statically stable walking manner is recommended.
However, in the cases of soft terrain, up and down slopes or unknown environ-
ment, the walking machine may lose its stability because of the position planning
errors and unbalanced foot forces. Hence, position control alone is not sufficient for
practical walking, position/force control being thus necessary. Foot force control
(Zhou and Low, 2001) can overcome these problems, so that foot force control is
one of the ways to improve the terrain adaptability of walking robots. For example,
in the direction normal to the ground, foot force has to be controlled to ensure firm
foot support and uniform foot force distribution among all supporting legs; foot
force in the tangential direction has to be monitored to avoid slippage.
On the other hand, biological investigations suggest that human’s rhythmic
walking is a consequence of combined inherent patterns and reflexive actions. The
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 123

inherent dynamic pattern is rhythmic and periodic. It is considered as an optimal

feedforward motion pattern acquired through the development in the typical walk
environments without disturbances. The reflexive action is a rapid response due to
the feedback control using sensory information. The reflexive action determines
stability against unexpected events such as external disturbances or ground irreg-
ularity (Huang et al., 2001). A biped humanoid should not only perform rhythmic
walk in a known environment but also adapt itself to real world uncertainties. In
this case, the adaptability and ability of compensating external disturbances must
be included in advanced control algorithms.
A practical biped needs to be more like a human – capable of switching between
different known gaits on familiar terrain and learning new gaits when presented
with unknown terrain. In this sense, it seems essential to combine force control
techniques with more advanced algorithms such as adaptive and learning strategies.
Inherent walking patterns must be acquired through the development and refine-
ment by repeated learning and practice as one of important properties of intelligent
control of humanoid robots. Learning enables the robot to adapt to the changing
conditions and is critical to achieving autonomous behaviour of the robot.
Many studies have given weight to biped walking which is based only on sta-
bility of the robot: steady-state walking, high-speed dynamic walking, jumping,
and so on. A humanoid robot is however, a kind of integrated machine: a two-
arm and two-leg mechanism. Hence, we must not only focus on the locomotion
function but also on arm’s function with this kind of machines; manipulation and
handling being major functions of robot’s arms. The concept of mobile manipula-
tion (Yoshikawa et al., 2001) considers that the leg motion is capable of increasing
manipulation ability by changing stand position or adjusting posture of the body
with bending and stretching of the legs. From this point of view, it is necessary to
develop advanced control methods of mobile manipulation of humanoid robots.
When the ground conditions and stability constraint are satisfied, it is desirable
to select a walking pattern that requires small torque and velocity of the joint
actuators. Humanoid robots are inevitably restricted to a limited amount of en-
ergy supply. It would therefore be advantageous to consider the minimum energy
consumption, when cyclic movements like walking are involved. With this in mind,
an important approach in research is to optimise simultaneously both the humanoid
robot morphology and control, so that the walking behaviour is optimised instead
of optimising walking behaviour for the given structure of humanoid robot. Opti-
mum structures can be designed when the suitable components and locomotion for
the robot are selected appropriately through evolution.
It is well known that real-time generation of control algorithms based on highly-
complex nonlinear model of humanoid robot commonly suffers from a large amount
of computation. Hence, new time-efficient control methods need to be discovered
to control humanoid robots in real time, to overcome the mentioned difficulty.
In summary, conventional control algorithms for humanoid robots can run into
some problems related to mathematical tractability, optimisation, limited extend-
124 D. KATIĆ AND M. VUKOBRATOVIĆ

ability and limited biological plausibility. The presented intelligent control tech-
niques have a potential to overcome the mentioned constraints.

3. Connectionist Control Algorithms in Humanoid Robotics

Recently, some researchers have begun considering the use of neural networks for
control of humanoid walking (Doerschuk et al., 1998; Miller, 1994; Miller et al.,
1987; Kun and Miller, 1999; Wang et al., 1992). This approach makes possible the
learning of new gaits which are not weighted combinations of predefined biped
gaits. Various types of neural networks are used for gait synthesis and control
design of humanoid robots such as multilayer perceptrons, CMAC (Cerebellar
Model Arithmetic Controller) networks, recurrent neural network, RBF (Radial
Basis Function) networks or Hopfield networks, which are trained by supervised or
unsupervised (reinforced) learning methods. The majority of the proposed control
algorithms have been verified by simulation, while there were few experimental
verifications on real biped and humanoid robots. Neural networks have been used
as efficient tools for the synthesis and off-line and on-line adaptation of biped gait.
Another important role of connectionist systems in control of humanoid robots has
been the solving of static and dynamic balance during the process of walking and
running on terrain with different environment characteristics.
Kitamura et al. (1988) proposed a walking controller based on Hopfield neural
network in combination with an inverted pendulum dynamic model. The optimiza-
tion function of the Hopfield network was based on complete dynamic model of
biped.
Salatian et al. (Salatian and Zheng, 1992a, 1992b; Salatian et al., 1997) studied
off-line and on-line reinforcement techniques for adapting the gait designed for
horizontal surfaces to be executed on sloping surfaces. They considered humanoid
robot SD-2 with 8 DOFs and two force sensors on both feet. These control al-
gorithms without considering kinematic and dynamic models of humanoid robot
were evaluated using a biped dynamic simulation (Salatian and Zheng, 1992a,
1992b), as well as on real biped SD-2 (Salatian et al., 1997). The control structure
includes gait trajectory synthesizer (with the memory of previously stored and
learned gaits) and neural networks that are tuned by reinforcement signal from
force sensors at the feet. The joint positions of the robot are adjusted until the
force sensors indicate that the robot has a stable gait. The neural network has the
task to map the relation between foot forces and adjustment of the joint positions.
The reinforcement learning is used because the neural network receives no direct
instruction on which joint position needs to be modified. The neural network is not
a conventional type of network (perceptrons) and includes a net of more neurons
with inhibitory/excitory inputs from the sensor unit. Every joint of the robot is
associated with a neuron called joint neuron. Every joint neuron is further attached
to two pairs of neurons, called direction neurons. Each neuron possesses a value
of activation function called neuron value. During the learning process, a joint
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 125

neuron with the maximum neuron value is selected to modify the position of the
corresponding joint, while direction neuron is selected to determine the direction
of modification. If the selected joint and direction neuron result in a correct mo-
tion (the robot becomes more stable), the selection is reinforced by increasing the
neuron value. Otherwise, the neuron value is reduced. Using previously mentioned
“regard-and-punish” strategy, the neural network converges quickly and generates
a stable gait for the sloping surface. In this way, reinforcement learning is very
attractive because the algorithm does not require an explicit feedback signal. The
computation issues for reinforcement learning are simple, while the noise from
the feet sensors is taken in the process of learning. During one step of the biped,
there are 8 static configurations, which are called the primitive points. The neural
network is only responsible for the motion in the sagittal plane. Hence, including
the redundancy at the hip joint. there are 3 · 8 = 24 joint neurons. Because of
the nature of reinforcement learning, each time only one joint neuron is active.
Static and pseudo-dynamic learning are demonstrated to prove that the proposed
mechanism is valid for robot walking on the sloping surface. In this approach,
kinematic and dynamic models were not used, hence it would be a problem for real
dynamic walking with a high speed. Also, the real terrain is more complex than the
environments used in test experiments, so that more studies need to be conducted
to make the robot walk robustly on different sorts of terrain.
More recently, Miller (Miller, 1994; Miller et al., 1987; Kun and Miller, 1999)
has developed a hierarchical controller that combines simple gait oscillators, classi-
cal feedback control techniques and neural network learning, and does not require
detailed equations of the dynamics of walking. The emphasis is on the real-time
control studies using an experimental ten-axis biped robot with foot force sensors.
The neural network learning is achieved using CMAC controller, where CMAC
neural networks were used essentially as context sensitive integral errors in the
controller, the control context being defined by the CMAC input vector. There are
3 different CMAC neural networks for humanoid posture control. The front/back
balance CMAC neural network was used to provide front/back balance during
standing, swaying and walking. The training of this network is realized using
data from foot sensors. The second CMAC neural network is used for right/left
balance, to predict the correct knee extension required to achieve sufficient lateral
momentum for lifting the corresponding foot for the desired length of time. The
training of this network is realized using temporal difference method based on the
difference between the desired and real time of foot rising. The third CMAC net-
work is used to learn kinematically consistent robot postures. In this case, training
is also realized by data from foot sensors.
The results indicated that the experimental biped was able to learn the closed-
chain kinematics necessary to shift body weight side-to-side while maintaining
good foot contact. Also, it was able to learn the quasi-static balance required to
avoid falling forward or backward while shifting body weight side-to-side at dif-
ferent speeds. It was able to learn the dynamic balance in order to lift a foot off
126 D. KATIĆ AND M. VUKOBRATOVIĆ

Figure 1. The UNH biped walking.

the floor for a desired length of time and different initial conditions. There were,
however, many limitations (limited step length, slow walking, no adaptation for
left–right balance, no possibility of walking on sloping surfaces). Hence upgrad-
ing and improvement of this approach were proposed in (Kun and Miller, 1999).
The new dynamically balanced scheme for handling variable-speed gait was pro-
posed based on the preplanned but adaptive motion sequences in combination with
closed-loop reactive control. This allows the algorithm to improve the walking
performance over consecutive steps using adaptation, and to react to small errors
and disturbances using reactive control. New sensors (piezoresistive accelerom-
eters and two solid-state rate gyroscopes) are mounted on the new UNH biped
(Figure 1). The complete control structure consists of high-level and low-level
controllers (Figure 2) The control structure on high-level control includes 7 compo-
nents (Figure 3): gait generator, simple kinematics block and 5 CMAC controllers.
The operation of the gait generator is based on simple heuristics and an appropriate
biped model. The CMAC neural networks are used for compensation of right and
left lift-lean angle correction, reactive front-back offset, right–left lean correction,
right and left ankle-Y correction and front-back lean correction. Training of neural
networks is realized through the process of temporal difference learning using in-
formation about ZMP from robot foot sensors. The five CMAC neural networks
were first trained during repetitive foot-lift motion similar to marching in place.
Then, training was carried out during the attempts at walking for increased step
length and gait speeds. The control structure on the lower control level includes
reactive lean angle control, together with a PID controller.
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 127

Figure 2. Block diagram of overall biped control system.

Figure 3. High-level control architecture.

128 D. KATIĆ AND M. VUKOBRATOVIĆ

The experimental results indicate that the UNH biped robot can walk with for-
ward velocities in the range of 21–72 cm/min, with sideways leaning speed in the
range of 3.6–12.5 deg./s. The main characteristic of this controller is the synthesis
of the control signal without dynamic model of biped. The proposed controller
could be used as a basis for similar controllers of more complex humanoid robots in
the future research. However, this controller is not of a general nature, because it is
suitable only for the proposed structure of biped robot and must be adapted for the
bipeds with different structures. More research efforts are needed to simplify the
controller structure, to increase the gait speed, and to ensure stability of dynamic
walking.
The previously used CMAC controller is a particularly good option for ro-
botic motor control. It has the quality of fast learning and simple computation
in comparison with multilayer perceptrons and similar approximation capabilities
such as radial basis function networks. However, there are problems with large
memory requirements, function approximation and stability of dynamic walking.
These problems have been addressed in (Hu et al., 1999), where self-organizing
CMAC neural network structure was proposed for biped control based on a data
clustering technique together with adaptation of the basic control algorithm. In
this case, memory requirements are drastically reduced and globally asymptotic
stability is achieved in a Lyapunov sense. The structural adaptation of the network
centres is realized to ensure adaptation to unexpected dynamics. Unsupervised
learning using CMAC can be implemented with a Lyapunov trajectory index. The
distance between the input vector and the centre vectors of the CMAC is calculated,
then the memory cells corresponding to the centres (hit by the input) are found,
and finally, computation of the CMAC output by a linear combination of CMAC
basis functions and weights of the memory cell is achieved. The weights in the
fired memory cells are updated by unsupervised learning. The approach is verified
through simulation experiments on a biped with 7 DOFs. An important character-
istic of this approach is the inclusion of adaptation for CMAC and PID controllers
with a moderate increase of controller complexity to handle disturbances and en-
vironmental changes. Although the robustness was enhanced in terms of height
and pitch tracking along with external disturbance rejection, the proposed adaptive
controller does not guarantee the long-term stability of the walking gait, where the
following posture parameters are presented in Figure 3: H – height; Rff and Lff are
right and left foot forward; Rfl and Lfl are right and left foot lift; Fbl is front–back
lean and Rll is right–left lean.
Wang et al. (1992) have developed a hierarchical controller for a three-link two-
legged robot. The approach uses the equations of motion, but only for the training
of the neural networks, rather than to directly control the robot. The authors used a
very simplified model of biped with decoupled frontal and sagittal planes. There are
3 neural networks (multilayer perceptrons) for control of leg on the ground, control
of leg in the air, and for body regulation. This approach uses off-line training and
on-line adaptation. Training algorithm is a standard back-propagation algorithm
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 129

based on the difference between the decoupled supervising control law and output
of all three neural networks. There are no feedback in real-time control, and this is
a great problem in the case when the system uncertainties exist.
Apart from considering the walking control problem, very little research has
been done on the problem of intelligent control of running. Doerschuk et al. (1998)
presented an adaptive controller to control the movement of simulated jointed leg
during a running stride (uniped control). The main idea of this approach is the
application of modularity, i.e. the use of separate controllers for each phase of the
running stride (take-off, ballistic, landing), thus allowing each to be optimized for
the specific objective of its phase. In the take-off phase, the controller’s objective
is to realize inverse feedforward control (for desired height, distance and angular
momentum it is necessary to produce control signals that achieve these objectives).
The controller learns from experience to produce the control signals which will
produce the desired height, distance and angular momentum. Three different types
of neural networks are investigated (multilayer perceptrons, CMAC, and neuro-
fuzzy nets). It was concluded that neuro-fuzzy nets achieve more accurate results
than the other two methods. It is not needed for off-line global training because of
using local learning. The neuro-fuzzy take-off controller controls very accurately
the value of angular momentum of the stride after only two learning iterations.
The ballistic controller controls the movement of the leg while the foot is in the
air. In this case, ballistic controller combines neural network learning with the
conventional PD control. It is a typical feedback error learning scheme where a
PD controller generates the torques that are applied to the joints, producing move-
ment of the leg. The controller learns the dynamic model of leg from experience
generated by the PD controller and improved upon its performance. The CMAC
controller is used for neural network learning part with the possibility to very
accurately control the movement of the leg along a target trajectory even during
the first attempt. Ballistic learning is accomplished on-line without the need for
precomputed examples. This enables effective adaptability of humanoid robot to
various changes and new conditions.
The neural networks can be effectively used to generate trajectories (gait) of
humanoid robots (Kurematsu et al., 1991; Juang and Lin, 1996). Kurematsu (1991)
proposed a multi-layered network by using the centre of gravity concept in trajec-
tory generation. For example, Juang and Lin (1996) used the back propagation
through time algorithm for gait synthesis of a biped robot. Due to a high number of
DOFs of the biped, it is difficult to get a high nonlinear model of the biped. Hence,
the complex inverse dynamic computations were eliminated by using linearised
inverse biped model. The neural controller is a three-layer feedforward network.
The simulation results show that the neural network as open-loop controller can
generate control sequences to drive the biped along a prespecified trajectory. This
algorithm can also be used for the slope surface training.
130 D. KATIĆ AND M. VUKOBRATOVIĆ

4. Fuzzy Control Algorithms in Humanoid Robotics

Some researchers used the fuzzy logic (Vukobratovic and Timčenko, 1995; Zhou
and Meng, 2000; Yang and Low, 2002; Ivanescu et al., 2001) as the methodology
for biped gait synthesis and control of biped walking. Fuzzy logic was used mainly
as part of control systems on the executive control level, for generating and tun-
ing PID gains, fuzzy control supervising, direct fuzzy control by supervised and
reinforcement error signals.
In (Vukobratovic and Timčenko, 1995), fuzzy logic is applied at the level of
local control for tuning of local PID gains, while the complete control structure
includes nominal feedforward control (based on the biped’s dynamics model), too.
It has been shown that the aggregation-decomposition method for stability analysis
of the overall biped system is applicable in the cases when local subsystems are
stabilized with fuzzy regulators. For the synthesis of fuzzy regulators a method of
parallel distributed compensation was utilized.
The problem of biped gait synthesis using the reinforcement learning with fuzzy
evaluative feedback is considered in (Zhou and Meng, 2000). As first, initial gait
from fuzzy rules is generated using human intuitive balancing scheme. Simula-
tion studies showed that the fuzzy gait synthesizer can only roughly track the
desired trajectory. A disadvantage of the proposed method is the lack of practical
training data. In this case there are no numerical feedback teaching signal, only
evaluative feedback signal exists (failure or success), exactly when the biped robot
falls (or almost falls) down. Hence, it is a typical reinforcement learning problem.
The dynamic balance knowledge is accumulated through reinforcement learning
constantly improving the gait during walking. Exactly, it is fuzzy reinforcement
learning that uses fuzzy critical signal. For human biped walk, it is typical to
use linguistic critical signals such as “near-fall-down”, “almost-success”, “slower”,
“faster”, etc. In this case, the gait synthesizer with reinforcement learning is based
on a modified GARIC (Generalized Approximate Reasoning for Intelligent Con-
trol) method. This architecture of gait synthesizer consists of three components:
action selection network (ASN), action evaluation network (AEN), and stochastic
action modifier (SAM) (Figure 4). The ASM maps a state vector into a recom-
mended action using fuzzy inference. The training of ASN is achieved as with
standard neural networks using error signal of external reinforcement. The AEN
maps a state vector and a failure signal into a scalar score which indicates the
state goodness. It is also used to produce internal reinforcement. The SAM uses
both recommended action and internal reinforcement to produce a desired gait for
the biped. The reinforcement signal is generated based on the difference between
desired ZMP and real ZMP in the x–y plane. In all cases, this control structure
includes on-line adaptation of gait synthesizer and local PID regulators. The ap-
proach is verified using simulation experiments. In the simulation studies, only
even terrain for biped walking is considered, hence the approach should be veri-
fied for irregular and sloped terrain. where Xzmp, Yzmp are the ZMP coordinates;
d d
θzmp , θzmp are the desired joint angles of the biped gait.
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 131

Figure 4. The architecture of the reinforcement learning-based gait synthesizer.

In (Yang and Low, 2002), conventional fuzzy controller for position/force con-
trol of robot leg is proposed and experimentally verified. This intelligent walking
strategy is specially intended for walking on rough terrain.
A main problem in the synthesis of fuzzy control algorithms for biped robots
remains the inclusion of dynamic model and learning capabilities in order to ob-
tain exact tracking of biped trajectories as well as the steps with greater speed,
preserving dynamic stability of the biped gait.

5. Genetic Approach in Humanoid Robotics

It is considered that GA can be efficiently applied for trajectory generation of the
biped natural motion on the basis of energy optimisation (Arakawa and Fukuda,
1997; Capi et al., 2001), as well as for walking control of biped robots (Cheng
and Lin, 1997) and for generation of behaviour-based control of these systems
(Pettersson et al., 2001).
The hierarchical trajectory generation (Arakawa and Fukuda, 1997) method
consists of two layers, one is the GA level which minimises the total energy of
all actuators and the other is the evolutionary programming (EP) layer which op-
timises the interpolated configuration of biped locomotion robots. The trajectory
of biped is generated using ZMP stability conditions. The chromosome on the
EP level represents the interpolated configuration expressed by 12 state variables
(angles) of the biped. Also, a chromosome in a GA level consists of two parts, the
first of them representing the set of interpolated configurations, while the second
132 D. KATIĆ AND M. VUKOBRATOVIĆ

part includes a bit which represents the effectiveness of the configuration (0 or 1).
The process runs in a cyclic procedure through the application of mutation and
selection at the EP level, transfer of generated interpolated configuration into the
GA level, and complete evolution process through crossover, mutation, evaluation
and selection at the GA level. The fitness function at the GA level is connected to
the optimisation of total robot energy in order to ensure the natural movement of
the biped. The fitness function also contains some constraints related to the robot
motion. The final result represents an optimised trajectory similar to natural human
walking, which was demonstrated by the simulation experiment.
A typical example of the application of GA in humanoid robotics was presented
in (Capi et al., 2001), where the main intention was the optimal gait synthesis for
biped robots. The proposed method can easily be applied onto other tasks like
overcoming obstacles, going down stairs, etc. In solving these optimization tasks,
the most important constraint included is the stability, which is verified through
the ZMP concept. To ensure a stable motion, the jumping of the ZMP is realized
by accelerating the body link. GA makes easy handling of the constraints by us-
ing the penalty function vector, which transforms a constrained problem into an
unconstrained one. The optimisation process is based on considering two different
cost functions: minimisation of consumed energy (CE) and minimisation of torque
change (TC). In this optimisation process, some constraints are included such as
the stability conditions defined by ZMP to be within the sole length. The block
diagram of the GA optimisation method is presented in Figure 5.
Based on the initial conditions, the initial population, represented by the angle
trajectory in the form of a polynomial of time, is created. Its range is determined
on the basis of the number of angle trajectory constraints and the coefficients are
calculated to satisfy these constraints. In the simulation experiments, the parame-
ters of real humanoid robot “Bonten-Maru I” are used. For the optimisation of the
cost function, a real-value GA was employed in conjunction with the selection,
mutation and crossover operators. GA converges within 40 generations, while the
maximum number of generations is used as the termination function. Based on
simulation, the biped robot posture is straighter, like the human walking when the
CE is used as cost function. The torques change more smoothly when minimum
TC is used as a cost function.
However, for the real-time applications, some process of GA optimisation is
time-consuming (in this case, optimisation process needs 10 min). Hence, the au-
thor considered teaching a RBFNN (Radial Basis Function Neural Networks) based
on GA data. When the biped robot was to walk with a determined velocity and step
length, the RBFNN input variable would be step length and step time, while the
output variables of the RBFNN were the same as the variables generated by GA.
Simulations showed good results generated by RBFNN in a very short time (only
50 ms).
Another example is the application of GA to PD local gain tuning and deter-
mination of nominal trajectory for dynamic biped walking (Cheng and Lin, 1997).
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 133

Figure 5. Block diagram of the GA optimization process.

The biped with 5 links is considered. In the proposed GA, 19 controller gains and
24 final points for determination of nominal trajectory are taken into account. In or-
der the biped body be in the vertical plane during walking, some constraints related
to the fixation of joint angles are realised. Hence, it is possible to reduce the number
of parameters of nominal trajectory for optimisation by 6 parameters. Designs to
attain different goals, such as the capability of walking on an inclined surface,
walking at high speed, or walking with specified step size, have been evolved with
the use of GA. The fitness functions are related to total time of effective walking,
average speed of the biped body, and the size of the walking step. The total number
of generations for problem solving was between 10 and 60. The research showed
excellent simulation results in the evaluation of control parameters, as well as in
optimisation of the mechanical design of biped.
The main problem of GA application in humanoid robotics represents the cop-
ing with the reduction of GA optimisation process in real time.

6. Hybrid Intelligent Approaches in Humanoid Robotics

Because their complementary capabilities hybrid intelligent methods have also
found their place in the research of gait synthesis and control of humanoid robots.
134 D. KATIĆ AND M. VUKOBRATOVIĆ

In (Juang, 2000), a learning scheme based on a neuro-fuzzy controller to gener-

ate walking gaits, is presented. The learning scheme uses a neuro-fuzzy controller
combined with a linearised inverse biped model. The training algorithm is back
propagation through time. The linearised inverse biped model provides the error
signals for back propagation through the controller at control time instants. For
the given prespecified constraints such as the step length, crossing clearance, and
walking speed, the control scheme can generate the gait that satisfies all the men-
tioned constraints. Simulation results are verified for a simple structure of five-link
biped robot.
The GA has been efficiently applied in robotic neural approaches, as in the
case of the neuro-GA controller for visually-guided swing motion of a biped with
16 DOFs (Nagasaka et al., 1997). The aim of this robot task is learning of swing
motion by neural network using visual information from a virtual working envi-
ronment. As is known, GA requires a lot of computing time in order to evaluate the
fitness function for each individual in a population. Hence, it is not desirable to use
direct execution of the working task on a real biped because of task complexity and
inaccuracies of the sensors. Instead of a real biped, virtual working environment is
used for acceleration of the learning process. As the learning process is transferred
from the virtual environment to the real robot, the difference existing between these
two systems is neutralized by generalisation capabilities of the neural network. The
aim of learning for visually guided swing motion is increasing the swing amplitude
by skillful change of the gravity centre of the biped robot in the direction of swing
radius, caused by dynamic change of the environment recognised by the vision
sensor. The input to the network represents sensor information from the vision
sensor, while the output of the neural network is the knee angles of the biped

Figure 6. Neuro-GA approach for optimization of robot swing motion.

INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 135

(Figure 6). GA optimises the three sets of weighting factors of this 4-layer neural
network. At the output of the network, the data are transformed into joint angles and
then using limiters of angular velocities (to avoid extreme changes of joint angles),
the knee joint angle is calculated. The genotype is represented by a sequence of
weighting factors. The number of individuals in the initial population is 200. The
fitness function is represented by the height of the centre of gravity in the initial and
final pose. The evolution simulation experiment is terminated when the number of
alternations in generations reaches 50. The results show the efficient learning of
swing motion through successive generation that is verified through generalisation
experiments on the real robot biped.
In (Fukuda et al., 1997) the authors deal with a GA application for the deter-
mination of weighting factors of a recurrent neural network in order to generate
a stable biped gait. When the biped robot walks on the ground which has some
gradients, the optimal trajectory is not known, hence the optimal trajectory of ZMP
is not realised. Because of that, the reinforcement learning is used by applying a
recurrent neural network. Recurrent neural network is chosen in order to select best
biped configuration (desired joint position and velocity) using ZMP as stabilisation
index. This type of neural network was chosen because the output of the network
generates the dynamic output data for static inputs and can describe time records
easily. The input to the network is the information about position of ZMP taken
from the force sensor, while the output of the network is the correction angles
and correction velocities needed for a stable motion. The ZMP is calculated using
the values from force sensors at each sole and values of joint angles. Only self-
mutation is used from the set of genetic operators based on addition of the Gauss
noise with multiplication by the value of fitness function. The elite selection is
chosen, while the fitness function is defined by the sum of squares of the devia-
tions of the desired coordinates from the ZMP coordinates. In both single-support
and double-support phases of walking the algorithm calculates the ZMP by using
values from four force sensors at each sole, while correction to actuation angles and
velocities is determined by recurrent neural networks with the ZMP being within
the supporting area of the sole of the robot. The block diagram of the stabilisation
biped control is shown in Figure 7, where θ, θ̇ are the joint angles and velocities;
θD , θ̇D are the desired joint angles and velocities; U is the control signal; F is the
foot force. The motion on inclined surfaces is investigated with initial population
of 50 different individuals. It has been shown that the use of this approach yields a
stable biped gait.
Reil and Husbands (2002) proposed an evolutionary approach for the biped con-
troller based on dynamic recurrent neural network. Each neural network consists
of 10 fully interconnected neurons. The first 6 neurons represent motor neurons
because that control biped actuators (the biped has 6 DOFs). Their outputs are
scaled to map to the angle limits. The GA has a task to optimize the weighting
factors, time constant, and bias of activation functions for the chosen neurons.
Parameter values are coded using real numbers with different ranges for each type.
136 D. KATIĆ AND M. VUKOBRATOVIĆ

Figure 7. Block diagram of stabilization biped control.

Each population consists of 50 individuals. The rank-based selection is used for

generating new generations with a fittest fraction. From the genetic operators, only
mutation with small rate is applied. The fitness function considers two components:
(1) the minimisation of travelling distance from the origin; (2) the gravity centre
can not be below a certain height. The fraction of evolutionary runs leading to
stable walkers was 10%, of which the average walking distance was 20.577 m.
All controllers in the simulation experiments walked in a straight line without the
use of proprioceptive inputs and without active balance control. because of the
application of the mentioned fitness function. The authors proposed an additional
criterion for fitness function in order to reward cycling activity. In this way, the
proportion of successful runs is increased to 80% but without improvement of the
overall time efficiency. In order to achieve walking on a rough terrain, some set
of simulation experiments with inclusion of sensor signals as input to neurons of
the neural network, are realized. These preliminary experiments on the integration
indicate that cyclic walking activity can indeed be modified by external stimuli in a
meaningful way. The quality of simulation results indicates a further improvement
by a refined fitness function, together with inclusion of coupled neural oscillators
instead of a single neural network. Also, it is desirable to incorporate biomechan-
ical knowledge about human walking. However, because of the existence of only
simulation results, a real problem is the implementation of the proposed theory on
the embodied humanoid robots.
A very interesting approach was proposed in (Endo et al., 2002), using the ideas
of artificial life. The main idea is to optimise both the morphology and control
of biped walking at the same time, instead of optimising the walking behaviour
for the given hardware. It was shown that the generated robots have diverse mor-
phologies and control systems, while their walking is fast and efficient. Both the
morphology and neural systems are represented as simple large tree structures that
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 137

are optimised simultaneously. From the morphology side, the lengths of the lower
and upper limbs are optimised. Two types of control systems are analysed: the one
based on neural network and the other based on neural oscillator. The input to the
neural network represents the velocity, acceleration and ZMP position, while the
output of the network represents the joint angles. It is a layered neural network
with a pair of hidden layers. The chromosome includes the following parameters
for optimization: information on initial angle and velocity, length of each link
and weights of each neuron in the neural network. The simulation experiments
with population size of 200 individuals and 600 generations were realized using
standard genetic operators. In the first phase of GA, the fitness function was the
distance between the centre of mass of the robot and the initial point. In the second
phase, two fitness functions were evaluated based on the efficiency and stability
of walking. The preferred solution has appropriate locomotion and morphology.
As the other solution for control algorithm, neural oscillator was used, because the
biped walking is a periodical and symmetrical solution. Neural oscillator generates
the rhythm for the biped walking. In this case, it is not necessary to use a large-
size GA chromosome, as was the case with neural networks. The structure of the
neural oscillator represents some kind of recurrent neural network dynamic state,
while the other parameters of GA optimisation process are the same as in the
previous case. The walking patterns for the neural network and neural oscillator
are similar, even though the sizes of chromosomes are much different (the neural
network chromosome has 1000 bits size, while the size of the neural oscillator is
300 bits). Therefore, a larger dynamic model of biped can be applied to the model
with a neural oscillator. It has been shown that there is a close relation between
the morphology and locomotion. The proposed co-evolution of the morphology
and control systems may be a potentially powerful method for designing the real
humanoid robots.

7. Conclusions
In spite of the intensive development and experimental verification of various hu-
manoid robots, it is important to further improve their capabilities using advanced
hardware and control software solutions to make humanoid robots more auton-
omous, intelligent and adaptable to the environment and humans. The presented
survey indicates that the intelligent techniques, if applied in an appropriate manner,
can be very powerful tools for attaining these goals.
The neural networks were used for the synthesis and on-line adaptation of
biped gait, as well as for the control of humanoid robots to ensure static and
dynamic balance during the process of walking and running on the terrain with
different environment characteristics. The main advantages are the compensation
of system’s uncertainties and the inclusion of learning capabilities. The majority of
the proposed control algorithms were verified by simulation, while there were few
experimental verification on real biped and humanoid robots. Besides, the inclusion
138 D. KATIĆ AND M. VUKOBRATOVIĆ

of complex nonlinear models in real-time control, limited realized steps and slow
walking are the problems in implementation of connectionist control algorithms.
Fuzzy logic was used mainly as part of control systems on the executive control
level, for generation and efficient tuning of PID gains and direct fuzzy control by
supervised and reinforcement error signals. The main problem in using fuzzy con-
trol algorithms for biped robots remains the inclusion of a complex dynamic model
and learning capabilities. The GA represents an efficient tool for searching the op-
timised solutions of gait synthesis and biped control, the main problem being how
to cope with the reduction of GA optimisation process in real time and preserve
stability of the motion. The hybrid methods using complementary characteristics
of intelligent techniques have a great potential in the field of intelligent humanoid
robots. An important idea from the area of artificial life is the use of simultaneous
evolution of the robot design and control.

References
Arakawa, T. and Fukuda, T.: 1997, Natural motion generation of biped locomotion robot using
hierarchical trajectory generation method consisting of GA, EP layers, in: Proc. of the IEEE
Internat. Conf. on Robotics and Automation, Albuquerque, USA, April, pp. 211–216.
Arkin, R. C.: 1999, Behaviour-Based Robotics, MIT Press, Cambridge, MA, USA.
Billard, A. and Mataric, M.: 2000, Learning human arm movements by imitation: Evaluation
of a biologically-inspired connectionist algorithms, in: Proc. of the IEEE Internat. Conf. on
Humanoid Robots HUMANOIDS 2000, Cambridge, MA, USA, September.
Brooks, R. A.: 1997, From earwigs to humans, Robotics Autonom. Systems 20(2–4), 291–304.
Capi, G., Nasu, Y., Barolli, L., Yamano, M., Mitobe, K., and Takeda, K.: 2001, A neural network
implementation of biped robot optimal gait during walking generated by genetic algorithm, in:
Proc. of the 9th Mediterranean Conf. on Control and Automation, Dubrovnik, Croatia, June.
Cheng, M.-Y. and Lin, C.-S.: 1997, Genetic algorithm for control design of biped locomotion,
J. Robotic Systems 14(5), 365–373.
Doerschuk, P. I., Simon, W. E., Nguyen, V., and Li, A.: 1998, A modular approach to intelligent
control of a simulated jointed leg, IEEE Robotics Automat. Mag. 5(2), 12–21.
Doya, K., Kimura, H., and Kawato, M.: 2001, Neural mechanisms of learning and control, IEEE
Control Systems Mag. 21(4), 42–54.
Endo, K., Yamasaki, F., Maeno, T., and Kitano, H.: 2002, A method for co-evolving morphology and
walking pattern of biped humanoid robot, in: Proc. of the 2002 IEEE Internat. Conf. on Robotics
and Automation, Washington, DC, USA, May, pp. 2159–2164.
Fukuda, T., Komata, Y., and Arakawa, T.: 1997, Stabilization control of biped locomotion robot based
learning with GAs having self-adaptive mutation and recurrent neural network, in: Proc. of the
IEEE Internat. Conf. on Robotics and Automation, Albuquerque, USA, April, pp. 217–222.
Garcia, L.-M., Oliveira, A. A. F., Grupen, R. A., Wheeler, D. S., and Fagg, A. H.: 2000, Trac-
ing patterns and attention: Humanoid robot cognition, IEEE Intelligent Systems (July/August),
70–77.
Giszter, S. F., Moxon, K. A., Rybak, I. A., and Chaplin, J. K.: 2000, Neurobiological and neurorobotic
approaches to a control architecture for a humanoid motor system, IEEE Intelligent Systems
(July/August), 64–69.
Goldberg, D. E.: 1989, Genetic Algorithms in Search, Optimization, and Machine Learning,
Addison-Wesley, Reading, MA, USA.
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 139

Goswami, A.: 1999, Postural stability of biped robots and the foot-rotation indicator point, Internat.
J. Robotics Res. 18(6), 523–533.
Guihard, M. and Gorce, P.: 2001, Dynamic control of a biomechanical inspired biped: BIPMAN,
in: Proc. of the 4th Internat. Conf. on Climbing and Walking Robots, Karlsruhe, Germany,
September.
Haupt, R. and Haupt, S. E.: 1998, Practical Genetic Algorithms, Wiley, New York.
Haykin, S.: 1994, Neural Networks – A Comprehensive Foundation, Vols. 1, 2, Mackmillan,
Englewood Cliffs, NJ, USA.
Hirai, K.: 1997, Current and future perspective of Honda humanoid robot, in: Proc. of the IEEE/RSJ
Internat. Conf. on Intelligent Robotics and Systems IROS, pp. 500–508.
Hirai, K., Hirose, M., Haikawa, Y., and Takenaka, T.: 1998, The development of Honda humanoid
robot, in: Proc. of the IEEE Internat. Conf. on Robotics and Automation, Leuven, Belgium, May,
pp. 1321–1326.
Hu, J., Pratt, J., and Pratt, G.: 1999, Stable adaptive control of a bipedal walking robot with CMAC
neural networks, in: Proc. of the 1999 IEEE Internat. Conf. on Robotics and Automation, Detroit,
USA, May, pp. 1950–1956.
Huang, Q., Nakamura, Y., and Inamura, T.: 2001, Humanoids walk with feedforward dynamic pattern
and feednack sensory reflection, in: Proc. of the 2001 IEEE Internat. Conf. on Robotics and
Automation, Seoul, Korea, May, pp. 4220–4225.
Ivanescu, M., Popescu, A. M., and Popescu, D.: 2001, Moving target interception for a walking
robot by fuzzy observer and fuzzy controller, in: Proc. of the 4th Internat. Conf. on Climbing and
Walking Robots, Karlsruhe, Germany, September.
Juang, J.-G.: 2000, Fuzzy neural network approaches for robotic gait synthesis, IEEE Trans. Systems
Man Cybernet., Part B: Cybernetics 30(4), 594–601.
Juang, J.-G. and Lin, C.-S.: 1996, Gait synthesis of a biped robot using backpropagation through
time algorithm, in: Proc. of the IEEE Internat. Joint Conf. on Neural Networks, Washington, DC,
USA, pp. 1710–1715.
Kawato, M.: 1999, Internal models for motor control and trajectory planning, Current Opinion
Neurobiology 12, 718–727.
Kawato, M., Uno, Y., Isobe, R., and Suzuki, R.: 1987, Hierarchical neural network model for
voluntary movement with application to robotics, IEEE Control Systems Mag. 57, 169–185.
Kiriazov, P.: 2001, Learning robots to move: Biological control concepts, in: Proc. of the 4th Internat.
Conf. on Climbing and Walking Robots, Karlsruhe, Germany, September.
Kitamura, S., Kurematsu, Y., and Nakai, Y.: 1988, Application of the neural network for the trajectory
planning of a biped locomotion robot, Neural Networks 1, 344–356.
Klute, G. K., Czerniecki, J. M., and Hannaford, B.: 1999, Pneumatic actuators with biomechanical
intelligence, in: Proc. of the IEEE/ASME Internat. Conf. on Advanced Intelligent Mechatronics,
Atlanta, USA, September.
Kun, A. L. and Miller, III, W. T.: 1999, Control of variable – speed gaits for a biped robot, IEEE
Robotics Automat. Mag. 6(3), 19–29.
Kurematsu, Y., Katayama, O., Iwata, M., and Kitamura, S.: 1991, Autonomous trajectory generation
of a biped locomotive robot, in: Proc. of the IEEE Internat. Joint Conf. on Neural Networks,
pp. 1983–1988.
Marchese, S., Muscato, G., and Virk, G. S.: 2001, Dynamically stable trajectory synthesis for a
biped robot during the single-support phase, in: Proc. of the 2001 IEEE/ASME Internat. Conf. on
Advanced Intelligent Mechatronics, Como, Italy, May, pp. 953–958.
Mataric, M. et al.: 1998, Behaviour-based primitives for articulated control, in: Proc. of the 5th
Internat. Conf. on Soc. for Adaptive Behaviours, MIT Press, Cambridge, MA, USA, pp. 165–170.
Miller, III, W. T.: 1994, Real-time neural network control of a biped walking robot, IEEE Control
Systems Mag. (February), 41–48.
140 D. KATIĆ AND M. VUKOBRATOVIĆ

Miller, III, W. T., Glanz, F. H., and Kraft, L. G.: 1987, Application of a general learning algorithm to
the control of robotic manipulators, Internat. J. Robotics Reserach 6(2).
Nagasaka, K., Konno, A., Inaba, M., and Inoue, H.: 1997, Acquisition of visually guided swing
motion based on genetic algorithms and neural networks in two-armed bipedal robot, in: Proc. of
the IEEE Internat. Conf. on Robotics and Automation, Albuquerque, USA, April, pp. 2944–2949.
Pettersson, J., Sandholt, H., and Wahde, H.: 2001, A flexible evolutionary method for the generation
and implementation of behaviours for humanoid robots, in: Proc. of the IEEE–RAS Internat.
Conf. on Humanoid Robots HUMANOIDS 2001, Tokyo, Japan, November.
Pfeiffer, F., Loffler, K., and Gienger, M.: 2002, The concept of jogging JOHNNIE, in: Proc. of the
2002 IEEE Conf. on Robotics and Automation, Washington, DC, USA, May, pp. 3129–3135.
Raibert, M.: 1986, Legged Robots that Balance, MIT Press, Cambridge, MA, USA.
Reil, T. and Husbands, P.: 2002, Evolution of central pattern generators for bipedal walking in a
real-time physics environment, IEEE Trans. Evolutionary Comput. 6(2), 159–168.
Rumelhart, D. E. and McClelland, J. L.: 1986, Parallel Distributed Processing (PDP): Exploration
in the Microstructure of Cognition, Vols. 1, 2, MIT Press, Cambridge, MA, USA.
Salatian, W., Yi, K. Y., and Zheng, Y. F.: 1997, Reinforcement learning for a biped robot to climb
sloping surfaces, J. Robotic Systems 14(4), 283–296.
Salatian, W. and Zheng, Y. F.: 1992a, Gait synthesis for a biped robot climbing sloping surfaces
using neural networks I. Static learning, in: Proc. of the IEEE Internat. Conf. on Robotics and
Automation, Nice, France, May, pp. 2601–2606.
Salatian, W. and Zheng, Y. F.: 1992b, Gait synthesis for a biped robot climbing sloping surfaces
using neural networks II. Dynamic learning, in: Proc. of the IEEE Internat. Conf. on Robotics
and Automation, Nice, France, May, pp. 2607–2611.
Sardain, P. and Bessonnet, G.: 2001, Gait analysis of a human walker wearing robot feet as shoes,
in: Proc. of the 2001 IEEE Internat. Conf. on Robotics and Automation, Seoul, Korea, May,
pp. 2285–2292.
Sardain, P., Rostami, M., and Bessonnet, G.: 1998, An anthropomorphic biped robot: Dynamic
concepts and technological details, IEEE Trans. Systems Man Cybernet., Part A: Systems and
Humans 28(6), 823–838.
Schulz, S. and Bretthauer, G.: 2001, A fluidic humanoid robothand, in: Proc. of the IEEE–RAS
Internat. Conf. on Humanoid Robots HUMANOIDS 2001, Tokyo, Japan, November.
Terano, T., Asai, K., and Sugeno, M.: 1992, Fuzzy Systems Theory and Its Applications, Academic
Press, Boston, USA.
Vijayakumar, S. and Schaal, S.: 2000, Real time learning in humanoids: A challenge for scalability
algorithms, in: Proc. of the IEEE Internat. Conf. on Humanoid Robots HUMANOIDS 2000,
Cambridge, MA, USA, September.
Vukobratović, M., Borovac, B., and Surdilovic, D.: 2002, Zero-moment point – proper interpretation
and the application in gait control, Intelligent J. Engrg. Automat. Problems 3, 3–14.
Vukobratović, M. and Timčenko, O.: 1995, Stability analysis of certain class of bipedal walking
robots with hybridization of classical and fuzzy control, in: Proc. of the 1st ECPD Internat.
Conf. on Advanced Robotics and Intelligent Automation, Athens, Greece, pp. 290 – 295.
Wang, H., Lee, T. T., and Gruver, W. A.: 1992, A neuromorphic controller for a three link biped
robot, IEEE Trans. Systems Man Cybernet. 22(1), 164–169.
www.androidworld.com
www.symbio.jst.go.jp
www.world.honda.com/robot
Yamaguchi, J. J. et al.: 1998, Development of a bipedal humanoid robot – control method of whole
body cooperative dynamic biped walking, in: Proc. of the IEEE Internat. Conf. on Robotics and
Automation, Leuven, Belgium, May, pp. 3801–3806.
Yamazaki, F., Matsui, T., Miyashita, T., and Kitano, H.: 2000, PINO – the humanoid that walks, in:
Proc. of the 1st IEEE–RAS Internat. Conf. on Humanoid Robotis.
INTELLIGENT CONTROL TECHNIQUES FOR HUMANOID ROBOTS 141

Yang, A. and Low, K. H.: 2002, Fuzzy position/force control of a robot leg with a flexible gear
system, in: Proc. of the 2002 IEEE Conf. on Robotics and Automation, Washington, DC, USA,
May, pp. 2159–2164.
Yoshikawa, Y., Asada, M., and Hosoda, K.: 2001, Development approach to spatial perception for
imitation learning incremental demonstrator’s view, in: Proc. of the IEEE–RAS Internat. Conf.
on Humanoid Robots HUMANOIDS 2001, Tokyo, Japan, November.
Zheng, Y. P. and Shen, J.: 1990, Gait synthesis for the SD-2 biped robots to climb sloping surface,
IEEE Trans. Robotics Automat. 6(1), 86–96.
Zhou, D. and Low, K. H.: 2001, Combined use of ground learning model and active compliance to
the motion control of walking robotic legs, in: Proc. of the 2001 IEEE Internat. Conf. on Robotics
and Automation, Seoul, Korea, May, pp. 3159–3164.
Zhou, C. and Meng, Q.: 2000, Reinforcement learning with fuzzy evaluative feedback for a biped
robot, in: Proc. of the IEEE Internat. Conf. on Robotics and Automation, San Francisko, CA,
USA, pp. 3829–3834.
Zimmermann, H.-J.: 1990, Fuzzy Sets Theory and Its Applications, Kluwer Academic Publishers,
Dordrecht, The Netherlands.