64 IEEE TRANSACTIONS ON ROBOTICS, VOL. 35, NO.

1, FEBRUARY 2019

Quadratic Programming for Multirobot and

Task-Space Force Control
Karim Bouyarmane , Member, IEEE, Kevin Chappellet , Joris Vaillant,
and Abderrahmane Kheddar , Senior Member, IEEE

Abstract—We have extended the task-space multiobjective ported in a large variety of robots, especially redundant ones
controllers that write as quadratic programs (QPs) to handle [4], [5], achieving multiobjective complex tasks under various
multirobot systems as a single centralized control. The idea is constraints.
to assemble all the “robots” models and their interaction task
constraints into a single QP formulation. By multirobot, we mean Recent implementations formulate the task-space control as
that whatever entities a given robot will interact with (solid or a quadratic program (QP) where multiple objective tasks are
articulated systems, actuated, partially or not at all, fixed-base ordered by means of a weighted, strict, or hybrid strict-weighted
or floating-base), we model them as clusters of robots and the priority. The controller reduces to a QP solver for a problem that
controller computes the state of each cluster as an overall system is built online, at each control loop, e.g., [6]–[10], and where
and their interaction forces in a physically consistent way. By doing
this, the tasks specification simplifies substantially. At the heart the tasks are expressed as a part of the QP cost function or part
of the interactions between the systems are the contact forces; of its constraints, e.g., [11], [12].
methodologies are provided to achieve reliable force tracking by In our previous work [13], [14], we have devised a multicon-
our multirobot QP controller. The approach is assessed by a large tact planner that considers robots and objects as multirobot sys-
panel of experiments on real complex robotic platforms (full-size tems. It also gathers nongaited locomotion and manipulation in
humanoid, dexterous robotic hand, fixed-base anthropomorphic
arm) performing whole-body manipulations, dexterous manipu- a single multicontact planning framework. However, until now
lations, and robot–robot comanipulations of rigid floating objects we have not proposed a controller that can deal with generated
and articulated mechanisms, such as doors, drawers, boxes, or plans, nor had we experimented with common ground planning
even smaller mechanisms like a spring-loaded click pen. on real robots. We propose to extend the QP control methods
Index Terms—Humanoid robot manipulation, manipulation to encompass the idea that other objects and entities can be in-
force control, multirobot control, robot–robot comanipulation, tegrated as parts of a single controller when they interact with
task specification. the robot. We have introduced this idea and demonstrated its ap-
I. INTRODUCTION plicability in graphic animation of avatars [15], summarized in
Section II-B. In this paper, we present its adaptation to robotics.
ASK-SPACE sensory control [1] has reached a consid-
T erable level of maturity and has diverse implementa-
tions in kinematics and inverse dynamics [2], [3]. It has been
The contribution of this paper over the material presented in
[15] is to add all missing components for implementation on real
robots, namely contact sensing and force control, and to demon-
strate its applicability in real-application, real-world, real-robots
Manuscript received February 13, 2018; revised August 20, 2018; accepted scenarios with thorough experimentation on various platforms.
September 13, 2018. Date of publication November 9, 2018; date of current We believe that this idea will be largely adopted in robotics
version February 4, 2019. This paper was recommended for publication by
Associate Editor M. Schwager and Editor T. Murphay upon evaluation of the as, first, it is easy to implement—we provide the software im-
reviewers’ comments. This work was supported in part by the EU H2020 CO- plementation of this framework in open-source1 —and, second,
MANOID project www.comanoid.eu and in part by the JSPS Kakenhi B No. it allows us to ease task specification to its simplest expression,
16H02886. (Corresponding author: Abderrahmane Kheddar.)
K. Bouyarmane is with the Université de Lorraine, CNRS, Inria Nancy-Grand i.e., at the level of interactions. For example, when a robot has
Est, Loria UMR 7503, Larsen team, Vandoeuvre-les-Nancy 54506, France to open a fridge, our method does not ask to build specific ge-
(e-mail:,[email protected]). ometric constraints [16] or virtual mechanisms [17]–[20], nor
K. Chappellet and A. Kheddar are with the CNRS-UM LIRMM Interactive
Digital Humans group, Montpellier 34090, France, and also with the CNRS- to implement a specific planning or control strategy [21]–[25].
AIST Joint Robotics Laboratory, UMI3218/RL, Tsukuba 305-8560, Japan Instead, we model the fridge as a “robot” with as many degrees
(e-mail:,[email protected]; [email protected]). of freedom (DoFs) as possible. The user must design the fridge
J. Vaillant is with the Navya, Paris 75009, France (e-mail:, joris.vaillant@
gmail.com). model (e.g., as a ROS urdf file) and our controller integrates it
This paper has supplementary downloadable multimedia material available with that of the robot and considers interaction tasks as defined
at http://ieeexplore.ieee.org provided by the authors. This includes a video that through areas of interaction (contacts). The core idea here is
describes all the experiments achieved in multi-robot and force serving using
QP. The robots that are showcased are the humanoid robot HRP-4, the Romeo that the model already embeds the constraints (the kinematics
Arm from SoftBak Robotics and the Shadow dextrous hand. This material is
45.9 MB in size.
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org. 1 Available at github.com/jrl-umi3218/Tasks and a more complete version is
Digital Object Identifier 10.1109/TRO.2018.2876782 available at https://gite.lirmm.fr/multi-contact/mc_rtc upon request

1552-3098 © 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
BOUYARMANE et al.: QUADRATIC PROGRAMMING FOR MULTIROBOT AND TASK-SPACE FORCE CONTROL 65

and the dynamics ones) instead of explicitly defining them as in tact posture. In [29], the approach is enhanced by accounting
[26] and [27]. for bilateral grasp contact and more complex balancing strate-
Integrating the kinematic model of the manipulated mecha- gies; Lasa et al. [30] combined the QP controller with higher-
nism has been proposed in previous work [23], [24]. However, level finite state machine and used it for locomotion with cyclic
they remain at the geometric level, and only the kinematics of feet contact switching, whereas Bouyarmane and Kheddar [6]
the planned mechanisms are accounted for. The planned con- applied the approach for humanoid robot in acyclic multicon-
figurations and motions in these studies do not account for the tact locomotion, applied later in DARPA robotics challenge-like
dynamics and inertia of manipulated mechanism(s), although scenarios in simulation in [31] and to real robot HRP-2 climbing
these will influence robot balance through motion. In [25], the a vertical ladder in [12]. In [32] and [33], the continuous task
dynamics of the articulated mechanism is accounted for, yet they activation/deactivation within this scheme is proposed by con-
restrict the study to one-DoF mechanisms, and robot balance is tinuous variation of weights. There are many other studies that
not an issue (manipulator), we also refer the reader to the refer- use the QP in other schemes, such as force control distribution,
ences therein for a review of previous door and drawer opening e.g., [34]–[36]. In the following, we extend this framework to
studies and their limitations. A Cartesian impedance method for multicontact manipulation of articulated mechanisms and float-
the opening of a door is proposed in [21] for a mobile manipula- ing objects by humanoids, and to multirobot collaboration (e.g.,
tor without balancing issues and with the door opening motion robot–robot comanipulation).
being designed for the specific task at hand.
Instead, we compute desired states that have coherent con- B. Multirobot QP Formalism
tact interaction forces. Many of the intended manipulation and
comanipulation applications rely on friction (manipulation of a In this section, we briefly recall the multirobot QP formalism
free-floating box for example) and necessitates the generation from [15]. Let us consider a system of n “robots” that can be
of the right amount of normal and tangential forces. Therefore, actual robots, free-floating rigid objects, or passive articulated
it is important to master force control under a QP controller mechanisms, such as a door, a drawer, or a valve for example.
framework, even on position-controlled robots, which has not A typical minimal manipulation system would consist of n = 2
been previously proposed to our knowledge. Hence, we propose “robots”: the actual manipulating robot and the manipulated
it as another contribution in this paper. object or mechanism; a typical minimal collaboration system
The rest of this paper is organized as follows. Section II recalls would consist of n = 3 “robots”: the two collaborating robots
the multirobot QP formalism from [15]. Section III introduces and the collaboratively manipulated object; a dexterous hand
QP force control to track the manipulation forces resulting from with m fingers manipulating a rigid object would consist of
multirobot QP when applied on position-controlled robots. Fi- n = m + 1 “robots,” each finger and the object. We use the
nally, Section IV presents experimentation results where our unified term “robot” here to refer to all these systems, since
new controller is applied in very challenging scenarios involv- they are all instances of the general multibody model. Indeed,
ing Kawada’s HRP-4 humanoid robot, Softbank’s ROMEO arm, each of these systems i ∈ {1, . . . , n} can be modeled as a fixed
and Shadow’s dextrous hand. base or free-floating base kinematic tree structure for which the
DoFs qi obey the following equation of motion (EoM)
II. MULTIROBOT QP Mi (qi )q̈i + Ni (qi , q̇i ) = JiT fi + Si τi . (1)
A. QP Control: A Brief Background
Equation (1) encompasses all types of robots and accounts for all
QP control has been proposed in the robotics and computer underactuation possibilities (free-floating base for humanoids
graphics communities to solve the control problem of multibody and for free-floating rigid objects, nonactuated joints of passive
systems with floating bases subject to friction limitations. The mechanisms) through the actuation-to-DoFs mapping matrix
approach appeared particularly suited to humanoid robots and Si . Note that we use Newton–Euler-based algorithms for the
humanoid virtual characters that typically feature such proper- derivation of (1) in our implementation [37]. In this framework,
ties. A QP is instantiated at every control/simulation time-step the parts of q̇i and q¨i corresponding to a free-floating link (i.e.,
minimizing the error of multiple desired task accelerations un- the whole object in case of a free-floating rigid object or the
der all physical and structural constraints of the robot, which base link of a humanoid) are abusive notations for Vi and V̇i ,
have the characteristic of being linear in the optimization vector respectively, where Vi is the SE(3) velocity of the free-floating
variable composed of the control torques, contact force coef- link.
ficients along the linearized friction cones, and joint acceler- The vector fi stacks all point contact forces applied on the
ations. The multitask problem is cast as a multiobjective op- surfaces of robot i. These contact forces are either applied by
timization program that can be solved with a weighted-sum the fixed inertial environment (e.g., at the feet of humanoids)
scalarization or a lexicographic ordering scheme, among other or by another robot j (e.g., the forces applied on the hands of
possible multiobjective optimization or multicriteria decision a humanoid by a manipulated object). The latter forces come
making resolution techniques. Of the studies that opted for the in pairs of action/reaction forces among the system of robots
weighted-sum scalarization, the work in [28] is worth citing in according to Newton’s third law, and opposite forces applied
the field of computer animation as one of the firsts that proposed by the robot i on the robot j appear inside vector fj . We, thus,
the method for tracking in physics simulation a motion capture decompose the forces fi as fi = (fi0 , fi− , −fi+ ) such that fi0
data clip with a standing humanoid character in a multicon- stacks the forces applied by the fixed environment on the robot
Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
66 IEEE TRANSACTIONS ON ROBOTICS, VOL. 35, NO. 1, FEBRUARY 2019

The kinematic constraint that expresses the coincidence of

the contacts points corresponding to an action/reaction pair can
be synthetically written using the matrix Ψ and the principle of
virtual work as

J+ q̇ = ΨJ− q̇ (14)

which is equivalent to, given that a permutation matrix is or-

thogonal ΨT Ψ = I3K
Fig. 1. Force notation illustration.

J− − ΨT J+ q̇ = 0 . (15)
i, fi−stacks the forces applied by the robots j < i on robot i,
and fi+ stacks forces applied by robot i on robots j > i. We This latter form of the constraint is consistent with the fact that
then denote F 0 , F − , F + , respectively, the vectors stacking all F − can be interpreted as the constraint’s Lagrange multiplier
the vectors fi0 , fi− , fi+ . Let K be the total number of forces in in (13). This constraint has to be complemented with the fixed
F − , i.e., such that F − ∈ R3K . By virtue of Newton’s third law, environment contact kinematic constraint that writes
there exists a permutation matrix Π ∈ RK ×K such that
F + = (Π ⊗ I3 )F − (2) J0 q̇ = 0 (16)

where ⊗ denotes the Kronecker product, see Fig. 1. for which F 0 also appears as the corresponding Lagrange mul-
We denote Ψ = Π ⊗ I3 (itself a permutation matrix). Let tiplier in (13).
Ki be the number of forces in fi+ , i.e., such that fi+ ∈ R3K i . Note that the proposed mathematical Lagrange multiplier in-
The permutation matrix Ψ is decomposed into selection matrix terpretations of F − and F 0 do not oppose the fact that both
blocks Ψi ∈ R3K i ×3K in the form F − and F 0 consist of physical contact forces (as they had been
⎛ ⎞ initially constructed earlier in the section by concatenation of
Ψ1
⎜ ⎟ point contact forces). As a consequence of their physical na-
Ψ = ⎝ ... ⎠ (3) ture, F − and F 0 are indeed the correct subjects of the Coulomb
Ψn friction cone constraints F − ∈ C− and F 0 ∈ C0 (which would
not have been necessarily a justified hypothesis if we had de-
such that for each i we can write fi+ = Ψi F − . Finally the equa-
rived (13) directly using a Lagrangian approach on the whole
tions of motions (1) take the form
system made from all of the robots). These friction cones are
T
Mi (qi)q̈i + Ni (qi , q̇i) = Ji,0 fi0 + Ji,−
T
fi− − Ji,+
T
Ψi F − + Si τi then approximated as polyhedral cones with generators stacked
(4) as columns of matrices denoted C− and C0 , respectively [38].
where Ji,0 and Ji,− and Ji,+ are the matrices obtained by ex- The coefficients of F − and F 0 along the generators are de-
tracting from Ji the columns corresponding to the positions of noted λ− and λ0 , respectively, such that F − = C− λ− and F 0 =
f 0 , f − , f + in f , respectively.1 We stack together all the equa- C0 λ0 . These coefficients are constrained to be nonnegative
tions (4) with the following matrices and vectors: component wise
q = (q1 , . . . , qn ) (5)
λ = (λ− , λ0 ) ≥ 0 . (17)
τ = (τ1 , . . . , τn ) (6)
The constraints of the problem are completed with the append-
M (q) = blockdiag(M1 (q1 ), . . . , Mn (qn )) (7)
ing of joint limits, velocity limits, torque limits, and velocity-
J0 (q) = blockdiag(J1,0 (q1 ), . . . , Jn ,0 (qn )) (8) damper-based collision avoidance constraints between any links
la and lb , all of the initial forms
J+ (q) = blockdiag(J1,+ (q1 ), . . . , Jn ,+ (qn )) (9)
J− (q) = blockdiag(J1,− (q1 ), . . . , Jn ,− (qn )) (10) qm in ≤ q ≤ qm ax (18)
S = blockdiag(S1 , . . . , Sn ) (11) q̇m in ≤ q̇ ≤ q̇m ax (19)
 T τm in ≤ τ ≤ τm ax (20)
N (q, q̇) = N1 (q1 , q̇1 )T · · · Nn (qn , q̇n )T (12)
to get our synthetic Newton’s third law-consistent EoM for the ˙ a , lb ) ≥ ξ dist(la , lb ) − δs
dist(l (21)
whole system of robots δi − δs

M (q)q̈ + N (q, q̇) = J0T F 0 + J− − ΨT J+ F − + Sτ .
T
where the parameters ξ, δi , and δs represent, respectively, the
(13) velocity damping coefficient, the influence distance between
links below that the constraint starts to act, and the security
1 we use the index notations 0, +, − in the superscript of vectors and subscript distance between links that the constraint ensures will never be
of matrices, to avoid conflict with the transpose notation of matrices. reached. The constraints (18), (19), and (21) are rewritten in

Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
BOUYARMANE et al.: QUADRATIC PROGRAMMING FOR MULTIROBOT AND TASK-SPACE FORCE CONTROL 67

terms of constraints on q̈ as follows:

q̇m in − q̇ q̇m ax − q̇
 q̈  (22)
Δt Δt
qm in − q − q̇Δt qm ax − q − q̇Δt
1 2
 q̈  1 2
(23)
2 Δt 2 Δt

¨ ≥ 1 dist −δs ˙
dist −ξ − dist . (24)
Δt δi − δs
These formulations allow us to write the control problem for
the system of robots as a single QP
M
min wk ||g̈k − g̈kd ||2
q̈ ,τ ,λ
k =1

subject to (13), (15)–(17), (20), (22), (23), (24) (25)

where gk denote the tasks (possibly multidimensional) and g¨k d

the desired task accelerations that can for example take the
following form:
Fig. 2. Predicted forces (green) in planned contact state {Contact 1, Contact
g̈kd = g̈kref − Pk ek − Dk ėk , ek = gk − gkref (26) 2, Contact 3} versus sensor forces (yellow) in real contact state {Contact 1,
Contact 2} (the predicted forces are resultant at the sensor locations of the point
with Pk and Dk denoting the task gain matrices designed such forces in dashed lines computed at the vertices of the contact prints). In this
situation, the QP controller assumes that the robot is in the planned contact
that ( −P0 k −D
I
k
) is a stable (Hurwitz) matrix, and where gkref is a state and, therefore, predicts forces that do not correspond to the actual force
reference trajectory or a fixed set-point of the task [39]. repartition, since the hand contact (Contact 3) is not yet established.
Once a contact state for the system of robots has been spec-
ified, the effectiveness of the formulation (25) lies in the fact
that a task can be specified for any feature of any single robot or
1) it is based on the QP-used models of the robots and the
group of robots of the system in a uniform way. For illustration,
objects, implying that any discrepancy in these models
it is sufficient to specify a task in terms of position and orienta-
would result in inexact predicted forces;
tion of a free-floating manipulated object; the control commands
2) it supposes that the robot is in a given contact state that was
for the manipulating robot (or the comanipulating robots) will
planned beforehand, without actually knowing whether
automatically be induced from the contact constraints through
the robot has effectively reached that contact state and
(25), without the need of explicitly specifying any task for the
whether the contact has been established. If not, the QP
manipulating end-effectors. Similarly, if it is a mechanism that
would still base its calculations on the assumption that the
is being manipulated, it is sufficient to specify a task in terms
robot is in its planned contact state and will output contact
of the configuration of the mechanism (opening angle of a door,
forces that are in reality null, see Fig. 2.
rotation angle of a valve) rather than tasks for the manipulating
Therefore, we need a method that feeds back the information
end-effectors. As an further illustration of the expressiveness of
from the force sensors and realizes the tracking of the predicted
(25), the balance of a biped robot manipulating an object with
forces by the sensored ones. Such a tracking method should also
a nonnegligible mass can be written in terms of a single task on
be able to ensure that the actual contact states match the planned
the center of mass (CoM) of the whole system.
ones by making sure that any planned contact has effectively
been established in the current contacts state.
III. QP FORCE CONTROL
Force control has been extensively studied in robotics, see a
The QP controller (see Section II) outputs accelerations q̈, thorough review in the monographs [40], [41] and in the hand-
forces coefficients λ, and joint torques τ for the robots. We book of robotics [27]. Force control in the task space for fixed-
use it in our applications with position-controlled robots (see base robots was also developed and experimented in [26], [42],
Section IV), by double integrating the output q̈ and feeding the and [43]. Contrarily to [16] and [26], task specification includ-
resulting q to the low-level motor position controller. However, ing force control is simplified and made straightforward with
in view of the effective use of the multirobot QP in interaction the QP built-in multirobot constraints specification, since inter-
tasks (e.g., robots comanipulations), it is necessary to ensure that action forces are part of the QP decision variables. Active force
the planned manipulation contact forces are adequately matched control can be achieved either directly, through explicit clo-
during the execution, even when the framework is applied on sure on the force, or indirectly through compliance, impedance,
position-controlled robots. As demonstrated in Section II, the or admittance control [27]. The multirobot QP control frame-
formulation (25) does produce accelerations and, hence, po- work allows us to have both, and also allows us to consider
sition commands, which are consistent with the QP-predicted floating-base under-actuated or fixed-base robots. As compared
contact forces λ at a given control time-step. However, there are to exiting controllers, the added value of the QP control is not
following two issues with this prediction: in a structural way or in how basic force control is made, but
Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
68 IEEE TRANSACTIONS ON ROBOTICS, VOL. 35, NO. 1, FEBRUARY 2019

Fig. 3. Block diagram for force control with the QP. The error between the target force and the sensed force is converted into velocity with diagonal matrix gain
K I . This velocity is low-pass filtered into a reference velocity for the end-effector QP tracking task. The target force comes either from the force output of the
QP (switch 2 up) or from a external user command (switch 2 down). That external user command can also alternatively be incorporated inside the QP (switch 1
closed) in order to influence the force output of the QP when using the latter as a target force (i.e., with switch 2 up).

rather in the way force control is integrated into the whole mul- proceed with an admittance controller that takes as an input
tiobjective task-space control, accounting for limitations in the the error between a target force ftarget and the corresponding
force wrench explicitly and in full multi-unilateral-contact set- sensed force fsensor , and transforms it into a QP end-effector
tings. For example, prohibiting sliding (unilateral contact) is task through the following stages. First, we convert the force
made simply by adding built-in nonsliding constraint task; lim- error into a velocity command with a diagonal gain matrix KI
iting the force or moment in any direction is easy. Yet, the real (inverse of a damping)
added-value with respect to existing force control frameworks,
v = KI (ftarget − fsensor ) (27)
is the ability for the controller to suggest (i.e., plan) the refer-
ences forces to be used in a given set of tasks and current state then we clamp that value between (vm in , vm ax ) to prevent the
configuration and use that output to close the loop on such gen- end-effector from moving too fast nearby the contact surface
erated reference forces. For example, consider a multicontact if not reached yet (i.e., if fsensor = 0, which happens when the
setting where we ask a humanoid robot to move one leg and its end-effector is “searching” the surface it is supposed to be in
body in multi-unilateral-contacts using the other foot and one contact with), this is a guarded motion with
hand as supporting contacts. In order to shift the CoM and mov-
vclamp = min (max (vm in , v) , vm ax ) (28)
ing the leg, we may let the QP decide what force to generate
on the supporting hand under torque and nonsliding constraints to which we apply a low-pass filter (order 3 and cutoff frequency
and close the loop on such computed force wrench during the 20 Hz Butterworth in our implementations). The filtered signal
movement. We could also suggest a threshold force on the hand ṽ is converted into a QP end-effector task by taking it as the ref-
or a desired force behavior that must be met at best. If not, we erence velocity trajectory (ġref = ṽ) and by deriving it from the
would have to select by hand what the force trajectory is (for reference position gref and acceleration g̈ref trajectories. The lat-
all components) that need to be generated. As we will see later, ter are then sent to the QP as an end-effector trajectory tracking
our controller can servo on a user-specified or otherwise planed task (here, similar to an impedance). Note that the contact zero-
desired force wrenches, or on its own computed force-wrenches acceleration constraint is dropped for any link that is subject to
(if no desired force wrench is specified) and finally, it can servo the admittance task.
on both (i.e., on user-specified or planed desired force wrenches We retained three possible strategies to incorporate the ad-
while driving the controller to generate a behavior and conse- mittance scheme in our framework, depending on the states of
quently contact forces as close as possible to the desired ones, the switches 1 and 2 that appear in Fig. 3. Fig. 4 illustrates a
but keeping physical and constraint consistencies). simplified representation of the different switch combinations
To achieve the previously described force control behaviors in in the block diagram of Fig. 3.
the multirobot QP controller, we propose the scheme represented The configuration as it appears in the case of Fig. 3, i.e., with
in Fig. 3. For a given end-effector (or more generally any link) switch 1 open and switch 2 up, implements an autonomous be-
of the robot equipped with a force/torque sensor/observer, we havior where the controller tracks the force output by the QP
Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
BOUYARMANE et al.: QUADRATIC PROGRAMMING FOR MULTIROBOT AND TASK-SPACE FORCE CONTROL 69

Fig. 5. Effect of the gains for x-axis in diag(K I ) on the force time response
of the HRP-4 right hand to human interactive manipulation. The same behavior
are observed for the other axis and also in both hands and feet that are force-
control to zero-force reference (fd = 0 in all components). The admittance
gains are changed online three times during the manipulation from its initial
(high damped) value to double and double again from the values. The higher the
admittance gain, the faster the response, hence the more interactive the robot is
to human guidance.

TABLE I
ADMITTANCE RANGE FOR FORCE CONTROL: (SLOW/SAFE) LOWER BOUND;
FAST/INTERACTIVE (HIGHER BOUND)

Fig. 4. Simplified representations of the different switch positions in the block

diagram of Fig. 3. From top to bottom: Both switches 1 and 2 open, switch 1
open and switch 2 down, switch 1 closed and switch 2 up, switch 1 closed and
switch 2 down.

of gains matrices KI and, second, the clamping speeds and

as the QP figures it out from the other tasks of the problem. direction for each controlled limb.
However, the user might want to have some control on interac- Each value of KI correlates to the response time of the de-
tion forces, that might not turn out to be satisfactory for them sired force/moment. To not account for dynamic equilibrium
(typically, in the applications and experiments of Section IV, constraints, we hung the robot in the air, fixed the waist, and
we considered that the forces output by the QP on the hands of manipulated interactively and simultaneously (two operators are
the HRP-4 robot can be too important given the relative fragility needed) every terminal point (both hands and feet). The higher
of the hands, regardless of the nominal manufacturer’s torque the admittance gain for a given component of force, the faster
limits, so we wanted to produce less force on the hand). Hence, the response, this is noticeable in Fig. 5. The task gains for the
we offer the user the possibility to specify a desired force fd EF trajectory task (impedance on resulting position/orientation
that can be used in two ways. The simplest one is with switch errors g in Fig. 3) are found to be P = 0.1 and D = 10 for
1 open and switch 2 down, this allows the sensor force to track all the limbs and all directions. This was the case for the in-
fd independently of the other physical constraints of the robot. teractive experiments and in the trials and final multicontact
This is not a safe strategy as the user might specify unrealistic experiments we conducted later. Increasing task PD gains do
forces fd given the current configuration and contact state of not have a predominant effect w.r.t to the admittance gains KI .
the robot, it can be used as a last resort only if the user is sure For the admittance gains, the Table I sums-up the values for the
that the specified fd is safe/consistent. The other way to use fd controlled limbs (they are in fact the same for the hands and for
is through the QP, with switch 1 closed and switch 2 up by each foot).
adding the term ||f − fd ||2 to the cost function of the QP, this Once the gain ranges are determined, the robot is put in four
we call a QP force task. This ensures that the user-specified contacts configuration, the CoM is shifted to lie within the cen-
force fd is filtered through physical constraints that are taken ter of the right foot. Fig. 6 shows the experimental setup that was
into account in the QP and produces an ftarget that is as close as used to assess the QP force control. It is put near a table in a half-
possible to fd while remaining physically consistent. sitting initial posture (i.e., the knees are slightly bent). A posture
First, we conducted preliminary QP force tasks experi- is computed so that both hands are just hovering above the table
ments with a humanoid HRP-4 equipped with built-in 6-DoF while the feet are firmly on the ground. The force control goal
force/torque sensors on each foot; in addition, our robot has is specified for each hand and the left foot independently, but
customized 6-DoF force/torque sensor on each hand (Nano 40 simultaneously. The right foot is kept as a supporting noncon-
from ATI). In these experiments (that can be made with any trolled contact for the coherency of the force control. As we only
robot), the reference force is put to zero fd = 0 for each limb have unilateral contacts, we must insure that the forces are not
and in all directions. The goal is to determine, first, the range controlled in a nonconsistant way. Obviously, one cannot control
Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
70 IEEE TRANSACTIONS ON ROBOTICS, VOL. 35, NO. 1, FEBRUARY 2019

in practice). For instance, we found a torsional friction of the

left leg (around z, the vertical axis aligned with gravity and the
contact normal to ground) to be  0.5; but this z-axis torsional
friction decreased to 0.4 and up to 0.3 when we asked the feet to
also be controlled with reference moments on the x- or y-axes.
All in all, nonsliding constraints and bounds on forces can be
integrated as part of the whole-body controller explicitly. After
experiments with the reference force wrenches in force tasks,
i.e., without using the QP computed forces, the plots in Figs. 8
Fig. 6. Base experiment for comparing the different proposed QP force control and 9 illustrate a comparative study of all combinations that can
paradigms. Each hand is controlled with a different paradigm.
be achieved with our controller. The transient phase to converge
to the desired command is due to the fact that the same uni-
a force fi of a given contact Ci and, at the same time-control, a fied control is used both for searching for the contact and for
force in another contact Cj = i that lies in the opposite direction regulating the magnitude of the contact force once contact has
of fi = −fj = i , to support the reaction. Another example, with been established. As for the previous canonical experiments, the
only feet as the contact, controlling the force along the z-axis speed of convergence can be mainly tuned with the admit-
(normal to the ground aligned with the gravity field vector) of tance gain matrix KI but also to some extent with task ad-
both feet at the same time will result in oscillatory behaviors mittance gains P and D (see Fig. 3). The later also correlate to
and none convergence up to instability; this is because the CoM the time response as for any other tasks of the QP. Fig. 9 il-
shift that results in controlling the wrench of one foot, will have lustrates how the regulation scheme does not track forces when
a direct consequence on the other, based on the action-reaction they are not consistent with the constraints and automatically
principle. This problem is not encountered in force control of compute the closest desired force wrench to the user-defined
fixed-based robots. This is the reason why accounting for the QP one.
computed forces and encompassing nonsliding and other bound
forces in the multirobot QP allows us to keep the concistency of IV. EXPERIMENTATIONS
the force control in multi-unilateral-contacts in most cases.
Fig. 7 illustrates the result of the simultaneous force control We experimented with the multirobot QP controller on var-
in all directions for the remaining three limbs. For these experi- ious challenging scenarios that were recorded in an accompa-
ments, we used the admittance gains diag(KI ) = [fx = 0.001, nying video. The scenarios use three different robots: Kawada
fy = 0.001, fz = 0.002, mx = 0.15, my = 0.15, mz = 0.15] HRP-4 humanoid robot, SoftBank Robotics ROMEO arm (with
for the hands and diag(KI ) = [fx = 0.001, fy = 0.001, fz = a hand), and Shadow dextrous multifinger hand. In all three
0.002, mx = 0.007, my = 0.007, mz = 0.007] for the left foot. scenarios, the control was performed in real-time, as the mul-
Given that we are in a multi-unilateral contacts setting, desired tirobot QP was consistently solved in times below 5 ms per
forces are kept small to avoid sliding (we also identified the iteration.
sliding coefficients prior to the experiments) and also the time
response is purposely set to be moderately slow (in the order of A. Box Manipulation Experiment
hundreds of milliseconds) to avoid overshooting in the force We instantiate the multirobot QP setting on a box manipula-
control. Overshooting in force response is not problematic, but tion experiment in Fig. 10. The box is modeled as a one-link
it may excite the robot internal flexibilities and this may result free-floating robot with unknown mass and CoM, only the geo-
in nondesired damped oscillations throughout the body of the metric model of the box is known to the controller. The force-
robot. Also each time a reference force wrench is changed for control scheme with autonomous QP (switch 1 open, switch 2 up
a given contact, its effects are perceived through a light instant combination) is used to ensure that the robot applies sufficient
deviation of the remaining contact forces (due to light changes force to avoid box slippage. The box is made with cardboard
in the postures); this is highlighted in Fig. 7. and is filled with various arbitrary objects tightly occupying all
We previously mentioned the friction issue in unilateral con- the space inside the box (to have a constant CoM). We used an
tacts. In our experiments, we could easily identify the friction Extended Kalman Filter method to estimate the mass and CoM
coefficients of the feet and hands in all directions using incre- position of the box online while manipulating it.
mental reference translational forces. When we provide the QP Once the dynamic parameters of the box are identified, the
with desired forces that lie outside the identified friction cone for contact forces during manipulation can be computed from and
a given contact, the latter slides and this is noticeable when we controlled by the MQP to achieve the same trajectory of the
plot the measured forces: the desired forces cannot be reached. box with less internal forces instead of using the user-defined
Torsional frictions are more difficult to identify. We could exper- contact forces.
imentally validate in part the observations in [44]: the torsional
friction and bounds on rotational sliding depend not only on the
B. Manipulation of Articulated Mechanisms
normal force and the Coulomb friction coefficient, but also on
the distance between the center of pressure and its distance to In these experiments, we illustrate the capabilities of the
the closest vertex of the contact surface (that we cannot estimate controller to manipulate every-day-life objects with articulated

Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
BOUYARMANE et al.: QUADRATIC PROGRAMMING FOR MULTIROBOT AND TASK-SPACE FORCE CONTROL 71

Fig. 7. Simultaneous multi-unilateral-contact force control of the humanoid robot HRP-4 in the setting of Fig. 6 (here the desired force wrench fd is given
directly to the QP and fQP is not used). The vertical bar indicates the moment where a desired force wrench fd is changed from either one of the hands or feet.
Horizontal lines indicate the desired values and continuous curves are measured taking their values from the appropriate force sensor. When there is no dotted
lines, it means that there is no explicit desired force sent to the robot. From up to down: left, right hands, and left foot three components of forces and moments.

mechanisms. Two manipulation scenarios were used in these hand on the printer is suggested by the planner [14] to create
experiments: door opening and printer tray opening. The robot a closed kinematic chain so as to pull the tray without causing
is the HRP-4 humanoid that is provided with the urdf models equilibrium or force application problems.
of the objects. The door is a regular (not self-closing) door of the Fig. 12 illustrates snapshots from the door opening experi-
laboratory room intended for everyday use. It is modeled in the ment with the same controller. The accompanying video shows
multirobot QP as a “robot” with a two-DoF fixed-base mech- an additional door opening experiment using a position control
anism. One DoF is the passive revolute joint at the hinges of scheme of the handle, with another robot posture where the door
the door; the second one is a spring-loaded revolute joint at the is opened with the left arm, pushed with the right one and finally
handle of the door. The printer is a commercial printer (model crossed using a walking controller [46]. This is an example of
Canon i-sensys LPB7680Cx). It is also modeled in the mul- sequencing the multirobot QP controller with other controllers
tirobot QP as “robot” with a fixed-base mechanism (although such as a walking controller in this case.
it could have been more accurately modeled as a floating-base
mechanism in unilateral contact with a support table as in [45])
C. Robot–Robot Comanipulation
with one passive prismatic joint for the tray (the model can
also include the other nonused trays and also all the buttons as We experimented with the multirobot QP paradigm for ac-
prismatic joints). tual multirobot collaborative tasks between a humanoid robot
In the printer experiment, the user provides a desired force ROMEO’s left arm from SoftBank Robotics and the hu-
fd = 10 N along the local z-axis on the left hand in order to manoid robot HRP-4. The two robots use different and un-
prevent its slippage (to compensate for the modeling approx- related low-level control and communication architectures, as
imation that we make consisting in defining a planar surface well as different control frequencies (respectively, 100 Hz and
on the hand of HRP-4 that is not perfectly planar) and a de- 200 Hz), constituting a challenging setting for the multirobot QP
sired force fd = 5 N along the local z-axis on the right hand to controller, see Fig. 13.
firmly insert it inside the tray handle prior to the tray pulling The multirobot QP controller computations were completed
motion, and fd = 0 N for the remaining axes, see Fig. 11. Both on an external computer and sent to both robots using dedicated
force commands are sent using the “switch 1 closed, switch communication architecture. We plan in the future to embark
2 up” combination of the controller in Fig. 3. Putting the left the multirobot QP control computations on one of the robots

Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
72 IEEE TRANSACTIONS ON ROBOTICS, VOL. 35, NO. 1, FEBRUARY 2019

Fig. 8. Three different executions of the experiment in Fig. 6 to compare the different proposed QP force control methods. Each row represents the data for one
run of the experiment.

and use the other robot’s computational resources for auxiliary removed from the box) and putting it down in a different loca-
tasks such as vision for example. tion. The task was only specified in terms of positions of the box
The task consisted in a collaborative pick-and-place oper- using three way points: lifting up, moving to the right (of HRP-
ation, collaboratively lifting a box (a random parcel package 4), putting down. When putting down, we specified a slightly
delivered by the post containing electronic parts that were not lower height than the lifted height to ensure that the contact

Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
BOUYARMANE et al.: QUADRATIC PROGRAMMING FOR MULTIROBOT AND TASK-SPACE FORCE CONTROL 73

Fig. 9. Comparison between the two methods that account for a desired force command fd , in two additional instances of the experiment in Fig. 6. In the first
method, the robot follows the user command but reaches torque limits (over-torque errors appeared on the robot during both executions). The second method (right
hand), the QP autonomously “saturates” the exaggerated force command to keep the robot within the torque limit constraint. The second method is thus the safest
for the robot.

and the box being sufficient. Force control was additionally

implemented in this experiment to maintain the contact force
between the HRP-4 hand and the box, and the ROMEO hand
and the box. Not using the QP force control scheme resulted in
slippage of the box and the robots were unable to lift it. Fig. 14
tracks the motions of fixed points on the contact surface frames
of the hands of the robots in comparison with the motion of a
Fig. 10. Example of online dynamic inertial parameter estimation experiment fixed point in the box frame representative of the task.
for a manipulated box. Each image shows a posture way-point.

D. Dexterous Manipulation
between the box and the table has been well established and to Both HRP-4 and ROMEO robots are equipped with gripper
avoid dropping the box from a nonzero height. This one-task mechanisms at the hand that do not implement anthropomor-
specification is illustrative of the multirobot QP coordination phic dexterous hand capabilities. So we chose to demonstrate
capabilities, since no explicit task is needed for the hands of the the multirobot QP applicability to dexterous manipulation prob-
two robots, the unilateral contact constraints between the hands lems on a Shadow dexterous hand with 19 DoFs. We chose an

Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
74 IEEE TRANSACTIONS ON ROBOTICS, VOL. 35, NO. 1, FEBRUARY 2019

and the cylindrical body of the pen were faceted (approximated

with planar surface patches).
The task in this example was specified on the configura-
tion of the pen “robot” such that the position of the clicking
part reaches its joint limit to trigger the exit of the writing tip
(see Figs. 15 and 16).

E. Practical Considerations
All the experiments presented are performed with a consistent
methodology: we use the MQP control framework without cus-
tomization for all the experiments and we use the same desired
force specification tasks and coding template with the switches
in Fig. 3 for each robot/contact. The differences lie in the urdf
of the robots in play and eventually the tasks (CoM, posture,
force, etc.) for each scenario. For the previous experiments,
Table II provides a list of the tasks that have been used (all
coded as templates).
Since objects in the environment that are interacting with
robots for given tasks are considered as “robots,” an increasing
number of objects come with an increase in the computational
cost. This is partly true because of the following reasons.
1) We do not systematically include all robots—or objects
considered as such, of a scene in a single MQP. Instead,
we define clusters of MQPs on the basis of effective in-
teractions between the robots/objects of interest; each of
which is computed separately. Given the application and
the context, we may consider the MQP computation as
Fig. 11. Printer tray opening with HRP-4. being remote, cloud computed, or distributed;
2) The MQP is sparse; therefore, we can benefit from any
QP solver that exploits the sparsity of the problem; this is
being currently investigated and since our framework is
independent from the QP solver per se, any efficient sparse
QP solver can be used with little development efforts.
The box manipulation experiment highlights the benefits in
having an estimation of the parameters such as inertia and the
pose of objects that are considered as “robots,” whereas they
Fig. 12. Door opening with HRP-4 using the force control scheme of the are not. For the case of the box experiment—or any well struc-
multirobot QP. tured articulated rigid objects, it is possible to add the inertia
parameters identification in the task space (as they can write as a
QP [47]) that allow more precise contact force computation and
control during the manipulation when this is possible. As for
the pose and configuration, such objects do not have encoders
and, hence, estimating their configuration can be made using the
robot embedded camera as we did very recently in [45]. If such
two identification features can be gathered, then we may con-
Fig. 13. Multirobot collaborative manipulation between HRP-4 and sider extending our MQP framework to deal with human–robot
ROMEO’s left arm. cooperative tasks.
Although our control framework is able to identify a large
illustrative manipulation problem where the manipulated object, number of nonconsistent multicontact force control, there is no
a click pen, is again an articulated mechanism, but as opposed guarantee, as for now, that we handle all of them properly. For
to the door and printer this time it is a free-floating base mech- example, if we ask a humanoid robot to achieve a desired same
anism. The cardboard boxes in the previous experiments were fd force on both foot in the same x-axis direction (front), the
also free floating but without articulations. Hence, with this last robot will bend behind up to falling in trying to fulfill such
example, we cover all typologies of manipulated objects. The force references if there are no tasks to constraint the CoM.
clicking articulation is modeled with a spring-loaded prismatic Moreover, we need through an explicit formalism to detect in
joint in the multirobot QP. The contact surfaces on the fingertips a multi-unilateral-contacts setting if we are asking in specific

Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
BOUYARMANE et al.: QUADRATIC PROGRAMMING FOR MULTIROBOT AND TASK-SPACE FORCE CONTROL 75

Fig. 14. Robot–robot comanipulation motion. Resulting coordinated motion (position of HRP-4 and that of ROMEO’s hand link frames) from single task
command (position of box frame).

directions for force control of both the action and its reaction.
Automating the detection of such user-specified inconsistencies
is an interesting issue to investigate further.

V. CONCLUSION
We have shown the benefit of integrating task-space QP con-
trollers as a single problem to handle multirobot interactions. By
Fig. 15. Dexterous hand clicking a pen.
multirobot, we mean that the controller can deal with any clus-
ter of objects or robots or mechanisms that are passive, partially
passive, or totally capable of interaction. Not only does such an
approach ease the specification of the tasks (as a complement to
planning) to its simplest expression (i.e., the interaction level),
but it also computes physically consistent contact interaction
forces. Subsequently, we devised force control algorithms for
QP controllers and show that we can achieve reliable closed-
loop force control where the QP can track at best the desired
forces (and does its best when they are not feasible) but also plan
them if not specified. The implementation code of the multirobot
controller is open source; it has been interfaced with vRep and
Gazebo simulators. The code is already distributed to several
Fig. 16. Green spots are the predefined contact areas. teams worldwide, it is sustained, and has been implemented
on other humanoid and robotic platforms (e.g., ARMAR, Nao,
TABLE II Pepper, HRP-2Kai, KuKa arms, etc.).
LIST OF TASKS FOR EACH EXPERIMENT
QP controllers are currently emerging as a gold standard to
handle multiobjective tasks in redundant robots, such as hu-
manoids. We have proven that they are capable of control-
ling position, torque and now multirobots and force. Recently
our MQP controller has been enhanced with visual servoing
[45] that makes it a multimodal controller as vision, force,
impedance/admittance, and position tasks can be specified to-
gether [48]. Investigations in terms of stability have been con-
ducted in [39]. Singularity problems are also considered in [49].
We are also conducting promising research using QP control
as an adaptive controller, where gains of the actuators [50] and
tasks are also part of decision variables. Additional issues ap-
pear to be important for further investigations. For example,
what is the granularity of the objects that need to be considered
and integrated as “robots” when we consider tasks such as gath-
ering small pieces in a box (more linked to perception); how
to extend the MQP framework to deal with deformable objects;
and finally, how the formalism can be extended to other types of
robots, such as complex wheeled robots, flying and sea robots,
and cable robots.

Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
76 IEEE TRANSACTIONS ON ROBOTICS, VOL. 35, NO. 1, FEBRUARY 2019

REFERENCES [24] F. Burget, A. Hornung, and M. Bennewitz, “Whole-body motion planning

for manipulation of articulated objects,” in Proc. IEEE Int. Conf. Robot.
[1] C. Samson, M. Le Borgne, and B. Espiau, Robot Control—The Task Func- Automat., Karlsruhe, Germany, May 6–10, 2013, pp. 1656–1662.
tion Approach. Oxford, U.K.: Clarendon, 1991. [25] Y. Karayiannidis, C. Smith, F. E. V. Barrientos, P. Ögren, and D. Kragic,
[2] N. Mansard, O. Khatib, and A. Kheddar, “A unified approach to integrate “An adaptive control approach for opening doors and drawers under un-
unilateral constraints in the stack of tasks,” IEEE Trans. Robot., vol. 25, certainties,” IEEE Trans. Robot., vol. 32, no. 1, pp. 161–175, Feb. 2016.
no. 3, pp. 670–685, Jun. 2009. [26] H. Bruyninckx and J. De Schutter, “Specification of force-controlled ac-
[3] L. Sentis, J. Park, and O. Khatib, “Compliant control of multi-contact tions in the “task frame formalism”—A synthesis,” IEEE Trans. Robot.
and center-of-mass behaviors in humanoid robots,” IEEE Trans. Robot., Automat., vol. 12, no. 4, pp. 581–589, Aug. 1996.
vol. 26, no. 3, pp. 483–501, Jun. 2010. [27] L. Villani and J. De Schutter, “Force control,” in Handbook of Robotics.
[4] Y. Nakamura, H. Hanafusa, and T. Yoshikawa, “Task-priority based re- Berlin, Germany: Springer-Verlag, 2008, pp. 161–185.
dundancy control of robot manipulators,” Int. J. Robot. Res., vol. 6, no. 2, [28] Y. Abe, M. da Silva, and J. Popović, “Multiobjective control with fric-
pp. 3–15, Jun. 1987. tional contacts,” in Proc. Eurographics/ACM SIGGRAPH Symp. Comput.
[5] B. Siciliano and J.-J. E. Slotine, “A general framework for managing Animation., San Diego, CA, USA, Aug. 2–4, 2007, pp. 249–258.
multiple tasks in highly redundant robotic systems,” in Proc. Int. Conf. [29] C. Collette, A. Micaelli, C. Andriot, and P. Lemerle, “Dynamic balance
Adv. Robot., Pisa, Italy, Jun. 19–22, 1991, vol. 2, pp. 1211–1216. control of humanoids for multiple grasps and non coplanar frictional
[6] K. Bouyarmane and A. Kheddar, “Using a multi-objective controller to contacts,” in Proc. IEEE/RAS Int. Conf. Humanoid Robots, Pittsburgh,
synthesize simulated humanoid robot motion with changing contact con- PA, USA, Nov. 29–Dec. 1, 2007, pp. 81–88.
figurations,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., San Fran- [30] M. de Lasa, I. Mordatch, and A. Hertzmann, “Feature-based locomotion
sico, CA, USA, Sep. 25–30, 2011, pp. 4414–4419. controllers,” ACM Trans. Graph. (SIGGRAPH), vol. 29, no. 4, pp. 1–10,
[7] O. Kanoun, F. Lamiraux, and P.-B. Wieber, “Kinematic control of 2010.
redundant manipulators: Generalizing the task-priority framework to [31] K. Bouyarmane, J. Vaillant, F. Keith, and A. Kheddar, “Exploring hu-
inequality task,” IEEE Trans. Robot., vol. 27, no. 4, pp. 785–792, manoid robots locomotion capabilities in virtual disaster response scenar-
Aug. 2011. ios,” in Proc. IEEE RAS Int. Conf. Humanoid Robots, Osaka, Japan, Nov.
[8] P. M. Wensing and D. E. Orin, “Generation of dynamic humanoid behav- 2012, pp. 337–342.
iors through task-space control with conic optimization,” in Proc. IEEE [32] J. Salini, S. Barthélemy, and P. Bidaud, LQP-Based Controller Design
Int. Conf. Robot. Automat., Karlsruhe, Germany, May 2013, pp. 3088– for Humanoid Whole-Body Motion. Berlin, Germany: Springer, 2010,
3094. pp. 177–184.
[9] A. Escande, N. Mansard, and P.-B. Wieber, “Hierarchical quadratic pro- [33] J. Salini, V. Padois, and P. Bidaud, “Synthesis of complex humanoid
gramming: Fast online humanoid-robot motion generation,” Int. J. Robot. whole-body behavior: A focus on sequencing and tasks transitions,” in
Res., vol. 33, no. 7, pp. 1006–1028, 2014. Proc. IEEE Int. Conf. Robot. Automat., Shanghai, China, May 9–13, 2011,
[10] S. Kuindersma, F. Permenter, and R. Tedrake, “An efficiently solvable pp. 1283–1290.
quadratic program for stabilizing dynamic locomotion,” in Proc. IEEE [34] A. Herzog, L. Righetti, F. Grimminger, P. Pastor, and S. Schaal, “Balancing
Int. Conf. Robot. Automat., Hong Kong, 2014, pp. 2589–2594. experiments on a torque-controlled humanoid with hierarchical inverse
[11] S. Feng, X. Xinjilefu, W. Huang, and C. G. Atkeson, “Optimization-based dynamics,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Chicago, IL,
full body control for the DARPA robotics challenge,” J. Field Robot., USA, Sep. 2014, pp. 981–988.
vol. 32, no. 2, pp. 293–312, Mar. 2015. [35] M. Liu, A. Micaelli, P. Evrard, A. Escande, and C. Andriot, “Interactive
[12] J. Vaillant et al.,“Multi-contact vertical ladder climbing by an HRP-2 virtual humans: A two-level prioritized control framework with wrench
humanoid,” Auton. Robots, vol. 40, no. 3, pp. 561–580, Mar. 2016. bounds,” IEEE Trans. Robot., vol. 28, no. 6, pp. 1309–1322, Dec. 2012.
[13] K. Bouyarmane and A. Kheddar, “Multi-contact stances planning for [36] L. Righetti and S. Schaal, “Quadratic programming for inverse dynamics
multiple agents,” in Proc. IEEE Int. Conf. Robot. Automat., Shanghai, with optimal distribution of contact forces,” in Proc. IEEE RAS Int. Conf.
China, May 9–13, 2011, pp. 5246–5253. Humanoid Robots, Osaka, Japan, Nov. 2012, pp. 538–543.
[14] K. Bouyarmane and A. Kheddar, “Humanoid robot locomotion and ma- [37] R. Featherstone, Rigid Body Dynamics Algorithms. New York, NY, USA:
nipulation step planning,” Adv. Robot., vol. 26, no. 10, pp. 1099–1126, Springer, 2008.
2012. [38] D. Stewart and J. C. Trinkle, “An implicit time-stepping scheme for rigid
[15] J. Vaillant, K. Bouyarmane, and A. Kheddar, “Multi-character physical body dynamics with coulomb friction,” in Proc. IEEE Int. Conf. Robot.
and behavioural interactions controller,” IEEE Trans. Vis. Comput. Graph., Automat., 2000, vol. 1, pp. 162–169.
vol. 23, no. 6, pp. 1650–1662, Jun. 2017. [39] K. Bouyarmane and A. Kheddar, “On weight-prioritized multitask control
[16] G. Borghesan, E. Scioni, A. Kheddar, and H. Bruyninckx, “Introducing of humanoid robots,” IEEE Trans. Autom. Control, vol. 14, no. 6, pp. 2343–
geometric constraint expressions into robot constrained motion specifica- 2352, Jun. 2018.
tion and control,” IEEE Robot. Automat. Lett., vol. 1, no. 2, pp. 1140–1147, [40] D. M. Gorinevsky, A. M. Formalsky, and A. Y.-U. Schneider, Force Con-
Jul. 2016. trol of Robotics Systems. Boca Raton, FL, USA: CRC Press, 1997.
[17] K. Kosuge, T. Itoh, T. Fukuda, and M. Otsuka, “Tele-manipulation sys- [41] B. Siciliano and L. Villani, Robot Force Control (The Springer Interna-
tem based on task-oriented virtual tool,” in Proc. IEEE Int. Conf. Robot. tional Series in Engineering and Computer Science 540). New York, NY,
Automat., Nagoya, Japan, May 21–27, 1995, pp. 351–356. USA: Springer, 1999.
[18] L. D. Joly and C. Andriot, “Imposing motion constraints to a force re- [42] J. De Schutter et al., “Estimating first-order geometric parameters and
flecting telerobot through real-time simulation of a virtual mechanism,” in monitoring contact transitions during force-controlled compliant motion,”
Proc. IEEE Int. Conf. Robot. Automat., Nagoya, Japan, May 21–27, 1995, Int. J. Robot. Res., vol. 18, no. 12, pp. 1161–1184, 1999.
pp. 357–362. [43] J. De Schutter et al., “Constraint-based task specification and estimation
[19] A. Kheddar, “Teleoperation based on the hidden robot concept,” IEEE for sensor-based robot systems in the presence of geometric uncertainty,”
Trans. Syst. Man Cybern. A, vol. 31, no. 1, pp. 1–13, Jan. 2001. Int. J. Robot. Res., vol. 26, no. 5, pp. 433–455, 2007.
[20] S. A. Bowyer, B. L. Davies, and F. Rodriguez y Baena, “Active con- [44] S. Caron, Q.-C. Pham, and Y. Nakamura, “Stability of surface contacts
straints/virtual fixtures: A survey,” IEEE Trans. Robot., vol. 30, no. 4, for humanoid robots: closed-form formulae of the contact wrench for
pp. 138–157, Feb. 2014. rectangular support areas,” in Proc. IEEE RAS Int. Conf. Robot. Automat.,
[21] C. Ott, B. Bäuml, C. Borst, and G. Hirzinger, “Employing cartesian Seattle, WA, USA, May 26–30, 2015, pp. 5107–5112.
impedance control for the opening of a door: A case study in mobile [45] A. Paolillo, K. Chappellet, A. Bolotnikova, and A. Kheddar, “Interlinked
manipulation,” in Proc. IFAC Symp. Intell. Auton. Veh., Toulouse, France, visual tracking and robotic manipulation of articulated objects,” IEEE
2007, pp. 349–354. Robot. Automat. Lett., vol. 3, no. 4, pp. 2746–2753, Oct. 2018.
[22] S. Dalibard, A. Nakhaei, F. Lamiraux, and J.-P. Laumond, “Manipulation [46] D. J. Agravante, A. Sherikov, P.-B. Wieber, A. Cherubini, and A. Kheddar,
of documented objects by a walking humanoid robot,” in Proc. IEEE “Walking pattern generators designed for physical collaboration,” in Proc.
RAS Int. Conf. Humanoid Robots, Nashville, TN, USA, Dec. 6–8, 2010, IEEE Int. Conf. Robot. Automat., Stockholm, Sweden, May 16–21, 2016,
pp. 518–523. pp. 1573–1578.
[23] D. Berenson, S. Srinivasa, and J. Kuffner, “Task space regions: A frame- [47] J. Jovic, A. Escande, K. Ayusawa, E. Yoshida, A. Kheddar, and G. Venture,
work for pose-constrained manipulation planning,” Int. J. Robot. Res., “Humanoid and human inertia parameter identification using hierarchical
vol. 30, no. 12, pp. 1435–1460, 2011. optimization,” IEEE Trans. Robot., vol. 32, no. 3, pp. 726–735, Jun. 2016.

Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.
BOUYARMANE et al.: QUADRATIC PROGRAMMING FOR MULTIROBOT AND TASK-SPACE FORCE CONTROL 77

[48] A. Bolotnikova et al., “A circuit-breaker use-case operated by a humanoid Joris Vaillant received the Master’s degree in interac-
in aircraft manufacturing,” in Proc. IEEE Conf. Automat. Sci. Eng., X’ian, tive systems from Paul Sabatier University, Toulouse,
China, Aug. 20–23, 2017, pp. 15–22. France, in 2010 and the Ph.D. degree in robotics from
[49] K. Pfeiffer, A. Escande, and A. Kheddar, “Singularity resolution in equal- the University of Montpellier, Montpellier, France, in
ity and inequality constrained hierarchical task-space control by adap- May 2015.
tive nonlinear least squares,” IEEE Robot. Automat. Lett., vol. 3, no. 4, He conducted his Ph.D. research at the Interac-
pp. 3630–3637, Oct. 2018. tive Digital Human group, CNRS-UM Laboratory of
[50] V. Samy, K. Bouyarmane, and A. Kheddar, “QP-based adaptive-gains Informatics, Robotics, and Microelectronics, Mont-
compliance control in humanoid falls,” in Proc. IEEE Int. Conf. Robot. pellier, France. He was with Abolis Biotechnologies,
Automat., Singapore, May 29–Jun. 3, 2017, pp. 4762–4767. Paris, and he is currently Research and Development
Engineer with Navya, Paris, France. His research in-
terests include contact planning and whole body motion for humanoids and
Karim Bouyarmane (M’17) received the double virtual avatars.
Ingénieur Diploma from the École Polytechnique,
Palaiseau, France, in 2007 and from the École Na-
tionale Supérieure des Mines de Paris, Paris, France,
in 2008, and the Ph.D. degree from the University of
Montpellier, Montpellier, France, in 2011.
Before receiving the Ph.D. degree, he completed
a full-time Ph.D. program in the Joint Robotics Lab-
oratory, National Institute of Advanced Industrial Abderrahmane Kheddar (SM’12) received the
Science and Technology, Tsukuba, Japan. He sub- B.S. in computer science from the Institut National
sequently held a Japan Society for the Promotion of d’Informatique (ESI), Bab Ezzouar, Algeria, and the
Science (JSPS) Postdoctoral Fellowship with the Computational Neuroscience M.Sc. and Ph.D. degrees in robotics, from the Univer-
Laboratories Department, Advanced Telecommunications Research Institute sity of Pierre and Marie Curie, Paris 6, Paris, France
International, Kyoto, Japan, working on brain–robot interfaces. He conducted in 1993 and 1997.
a CNRS fixed-term contract researcher with the Laboratory of Informatics, He is currently the Director of Research with
Robotics and Microelectronics of Montpellier (LIRMM). He is currently an the CNRS, Paris, France. He is the Co-Director of
Assistant Professor with the University of Lorraine, Lorraine, France. the CNRS-AIST Joint Robotic Laboratory (JRL),
UMI3218/RL, Tsukuba, Japan, and a Leader of the
Interactive Digital Humans (IDH) team, CNRS-UM
LIRMM, Montpellier, France. His research interests include humanoid robotics,
Kevin Chappellet received the Master’s degree haptics, and thought-based control using brain machine interfaces.
in computer sciences and applied mathematics Dr. Kheddar is currently a founding member of the IEEE/RAS chapter on
(Geometry, Image, and CAD) from Joseph Fourier haptics (acting also as a senior advisor), the Co-Chair and co-founding mem-
University, Grenoble, France, in 2014. He is cur- ber of the IEEE/RAS Technical committee on model-based optimization, and a
rently pursuing the Ph.D. degree in manufacturing member of the steering committee of the IEEE Brain Initiative. He is currently
humanoid technologies from the Interactive Digi- an Editor of the IEEE TRANSACTIONS ON ROBOTICS, the Journal of Intelligent
tal Human Group, CNRS-UM Laboratory of Infor- and Robotic Systems, and Frontiers in Bionics; he is a founding member of the
matics, Robotics, and Microelectronics, Montpellier, IEEE TRANSACTIONS ON HAPTICS and served on its editorial board during three
France. years (2007–2010), he was also an Associate Editor for the MIT Press Presence
He is currently a Research Engineer with the Inter- journal. He is currently a member of the National Academy of Technologies of
active Digital Human Group, CNRS-UM Laboratory France (NATF), knight in the National Order of Merit, and a Senior Member of
of Informatics, Robotics, and Microelectronics, where he is responsible for the IEEE Society.
the software development and experiments on the HRP-4 humanoid robot. He
was on an internship with the French National Institute for computer science
and applied mathematics, Grenoble, France. He was with the IMAGINE group
working on the calculation of the pressure forces of a garment on a character in
a WebGL rendering.

Authorized licensed use limited to: Bibliothèque ÉTS. Downloaded on January 08,2025 at 02:58:47 UTC from IEEE Xplore. Restrictions apply.