Cooperative Control

of Multi-Agent
Systems
 Cooperative Control
of Multi-Agent
Systems A CONSENSUS REGION
APPROACH

Z hong k u i L i • Z h is h e n g Du a n

Boca Raton London New York

CRC Press is an imprint of the

Taylor & Francis Group, an informa business

K16462_FM.indd 1 9/17/14 10:01 AM

 AUTOMATION AND CONTROL ENGINEERING
A Series of Reference Books and Textbooks

Series Editors
FRANK L. LEWIS, Ph.D., SHUZHI SAM GE, Ph.D.,
Fellow IEEE, Fellow IFAC Fellow IEEE
Professor Professor
The Univeristy of Texas Research Institute Interactive Digital Media Institute
The University of Texas at Arlington The National University of Singapore

PUBLISHED TITLES

Cooperative Control of Multi-agent Systems: A Consensus Region Approach,

Zhongkui Li; Zhisheng Duan
Nonlinear Control of Dynamic Networks,
Tengfei Liu; Zhong-Ping Jiang; David J. Hill
Modeling and Control for Micro/Nano Devices and Systems,
Ning Xi; Mingjun Zhang; Guangyong Li
Linear Control System Analysis and Design with MATLAB®, Sixth Edition,
Constantine H. Houpis; Stuart N. Sheldon
Real-Time Rendering: Computer Graphics with Control Engineering,
Gabriyel Wong; Jianliang Wang
Anti-Disturbance Control for Systems with Multiple Disturbances,
Lei Guo; Songyin Cao
Tensor Product Model Transformation in Polytopic Model-Based Control,
Péter Baranyi; Yeung Yam; Péter Várlaki
Fundamentals in Modeling and Control of Mobile Manipulators, Zhijun Li;
Shuzhi Sam Ge
Optimal and Robust Scheduling for Networked Control Systems, Stefano Longo;
Tingli Su; Guido Herrmann; Phil Barber
Advances in Missile Guidance, Control, and Estimation, S.N. Balakrishna;
Antonios Tsourdos; B.A. White
End to End Adaptive Congestion Control in TCP/IP Networks,
Christos N. Houmkozlis; George A Rovithakis
Robot Manipulator Control: Theory and Practice, Frank L. Lewis;
Darren M Dawson; Chaouki T. Abdallah
Quantitative Process Control Theory, Weidong Zhang
Classical Feedback Control: With MATLAB® and Simulink®, Second Edition,
Boris Lurie; Paul Enright
Intelligent Diagnosis and Prognosis of Industrial Networked Systems,
Chee Khiang Pang; Frank L. Lewis; Tong Heng Lee; Zhao Yang Dong
Synchronization and Control of Multiagent Systems, Dong Sun
Subspace Learning of Neural Networks, Jian Cheng; Zhang Yi; Jiliu Zhou
Reliable Control and Filtering of Linear Systems with Adaptive Mechanisms,
Guang-Hong Yang; Dan Ye

K16462_FM.indd 2 9/17/14 10:01 AM

 Reinforcement Learning and Dynamic Programming Using Function
Approximators, Lucian Busoniu; Robert Babuska; Bart De Schutter; Damien Ernst
Modeling and Control of Vibration in Mechanical Systems, Chunling Du;
Lihua Xie
Analysis and Synthesis of Fuzzy Control Systems: A Model-Based Approach,
Gang Feng
Lyapunov-Based Control of Robotic Systems, Aman Behal; Warren Dixon;
Darren M. Dawson; Bin Xian
System Modeling and Control with Resource-Oriented Petri Nets,
MengChu Zhou; Naiqi Wu
Sliding Mode Control in Electro-Mechanical Systems, Second Edition,
Vadim Utkin; Juergen Guldner; Jingxin Shi
Autonomous Mobile Robots: Sensing, Control, Decision Making and
Applications, Shuzhi Sam Ge; Frank L. Lewis
Linear Control Theory: Structure, Robustness, and Optimization,
Shankar P. Bhattacharyya; Aniruddha Datta; Lee H.Keel
Optimal Control: Weakly Coupled Systems and Applications, Zoran Gajic
Deterministic Learning Theory for Identification, Recognition, and Control,
Cong Wang; David J. Hill
Intelligent Systems: Modeling, Optimization, and Control, Yung C. Shin;
Myo-Taeg Lim; Dobrila Skataric; Wu-Chung Su; Vojislav Kecman

FORTHCOMING TITLES
Modeling and Control Dynamic Sensor Network,
Silvia Ferrari; Rafael Fierro; Thomas A. Wettergren
Optimal Networked Control Systems, Jagannathan Sarangapani; Hao Xu

K16462_FM.indd 3 9/17/14 10:01 AM

MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not
warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MATLAB® soft-
ware or related products does not constitute endorsement or sponsorship by The MathWorks of a particular
pedagogical approach or particular use of the MATLAB® software.

CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742

© 2015 by Taylor & Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Version Date: 20140916

International Standard Book Number-13: 978-1-4665-6997-3 (eBook - PDF)

This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com

and the CRC Press Web site at

http://www.crcpress.com
Contents

Preface vii

1 Introduction and Mathematical Background 1

1.1 Introduction to Cooperative Control of Multi-Agent Systems 1
1.1.1 Consensus . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.2 Formation Control . . . . . . . . . . . . . . . . . . . . 6
1.1.3 Flocking . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 Overview of This Monograph . . . . . . . . . . . . . . . . . . 7
1.3 Mathematical Preliminaries . . . . . . . . . . . . . . . . . . . 9
1.3.1 Notations and Definitions . . . . . . . . . . . . . . . . 9
1.3.2 Basic Algebraic Graph Theory . . . . . . . . . . . . . 11
1.3.3 Stability Theory and Technical Tools . . . . . . . . . . 16
1.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2 Consensus Control of Linear Multi-Agent Systems: Continuous-

Time Case 19
2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . 20
2.2 State Feedback Consensus Protocols . . . . . . . . . . . . . . 21
2.2.1 Consensus Condition and Consensus Value . . . . . . 22
2.2.2 Consensus Region . . . . . . . . . . . . . . . . . . . . 24
2.2.3 Consensus Protocol Design . . . . . . . . . . . . . . . 29
2.2.3.1 The Special Case with Neutrally Stable Agents 30
2.2.3.2 The General Case . . . . . . . . . . . . . . . 31
2.2.3.3 Consensus with a Prescribed Convergence
Rate . . . . . . . . . . . . . . . . . . . . . . . 33
2.3 Observer-Type Consensus Protocols . . . . . . . . . . . . . . 35
2.3.1 Full-Order Observer-Type Protocol I . . . . . . . . . . 35
2.3.2 Full-Order Observer-Type Protocol II . . . . . . . . . 40
2.3.3 Reduced-Order Observer-Based Protocol . . . . . . . . 41
2.4 Extensions to Switching Communication Graphs . . . . . . . 43
2.5 Extension to Formation Control . . . . . . . . . . . . . . . . 45
2.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3 Consensus Control of Linear Multi-Agent Systems: Discrete-

Time Case 53
3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . 54

v
vi Contents

3.2 StateFeedback Consensus Protocols . . . . . . . . . . . . . . 54

3.2.1 Consensus Condition . . . . . . . . . . . . . . . . . . . 55
3.2.2 Discrete-Time Consensus Region . . . . . . . . . . . . 57
3.2.3 Consensus Protocol Design . . . . . . . . . . . . . . . 59
3.2.3.1 The Special Case with Neutrally Stable Agents 60
3.2.3.2 The General Case . . . . . . . . . . . . . . . 62
3.3 Observer-Type Consensus Protocols . . . . . . . . . . . . . . 65
3.3.1 Full-Order Observer-Type Protocol I . . . . . . . . . . 65
3.3.2 Full-Order Observer-Type Protocol II . . . . . . . . . 67
3.3.3 Reduced-Order Observer-Based Protocol . . . . . . . . 68
3.4 Application to Formation Control . . . . . . . . . . . . . . . 69
3.5 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

4 H∞ and H2 Consensus Control of Linear Multi-Agent Sys-

tems 73
4.1 H∞ Consensus on Undirected Graphs . . . . . . . . . . . . . 74
4.1.1 Problem Formulation and Consensus Condition . . . . 74
4.1.2 H∞ Consensus Region . . . . . . . . . . . . . . . . . . 77
4.1.3 H∞ Performance Limit and Protocol Synthesis . . . . 80
4.2 H2 Consensus on Undirected Graphs . . . . . . . . . . . . . 83
4.3 H∞ Consensus on Directed Graphs . . . . . . . . . . . . . . 85
4.3.1 Leader-Follower Graphs . . . . . . . . . . . . . . . . . 85
4.3.2 Strongly Connected Directed Graphs . . . . . . . . . . 87
4.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

5 Consensus Control of Linear Multi-Agent Systems Using Dis-

tributed Adaptive Protocols 93
5.1 Distributed Relative-State Adaptive Consensus Protocols . . 95
5.1.1 Consensus Using Edge-Based Adaptive Protocols . . . 96
5.1.2 Consensus Using Node-Based Adaptive Protocols . . . 100
5.1.3 Extensions to Switching Communication Graphs . . . 101
5.2 Distributed Relative-Output Adaptive Consensus Protocols . 103
5.2.1 Consensus Using Edge-Based Adaptive Protocols . . . 104
5.2.2 Consensus Using Node-Based Adaptive Protocols . . . 107
5.2.3 Simulation Examples . . . . . . . . . . . . . . . . . . . 109
5.3 Extensions to Leader-Follower Graphs . . . . . . . . . . . . . 111
5.4 Robust Redesign of Distributed Adaptive Protocols . . . . . 114
5.4.1 Robust Edge-Based Adaptive Protocols . . . . . . . . 115
5.4.2 Robust Node-Based Adaptive Protocols . . . . . . . . 119
5.4.3 Simulation Examples . . . . . . . . . . . . . . . . . . . 121
5.5 Distributed Adaptive Protocols for Graphs Containing Direct-
ed Spanning Trees . . . . . . . . . . . . . . . . . . . . . . . . 123
5.5.1 Distributed Adaptive Consensus Protocols . . . . . . . 123
Contents vii

5.5.2 Robust Redesign in the Presence of External Distur-

bances . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

6 Distributed Tracking of Linear Multi-Agent Systems with a

Leader of Possibly Nonzero Input 137
6.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . 138
6.2 Distributed Discontinuous Tracking Controllers . . . . . . . . 139
6.2.1 Discontinuous Static Controllers . . . . . . . . . . . . 139
6.2.2 Discontinuous Adaptive Controllers . . . . . . . . . . 142
6.3 Distributed Continuous Tracking Controllers . . . . . . . . . 144
6.3.1 Continuous Static Controllers . . . . . . . . . . . . . . 144
6.3.2 Adaptive Continuous Controllers . . . . . . . . . . . . 147
6.4 Distributed Output-Feedback Controllers . . . . . . . . . . . 152
6.5 Simulation Examples . . . . . . . . . . . . . . . . . . . . . . 156
6.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

7 Containment Control of Linear Multi-Agent Systems with

Multiple Leaders 159
7.1 Containment of Continuous-Time Multi-Agent Systems with
Leaders of Zero Inputs . . . . . . . . . . . . . . . . . . . . . 160
7.1.1 Dynamic Containment Controllers . . . . . . . . . . . 161
7.1.2 Static Containment Controllers . . . . . . . . . . . . . 164
7.2 Containment Control of Discrete-Time Multi-Agent Systems
with Leaders of Zero Inputs . . . . . . . . . . . . . . . . . . . 164
7.2.1 Dynamic Containment Controllers . . . . . . . . . . . 165
7.2.2 Static Containment Controllers . . . . . . . . . . . . . 168
7.2.3 Simulation Examples . . . . . . . . . . . . . . . . . . . 168
7.3 Containment of Continuous-Time Multi-Agent Systems with
Leaders of Nonzero Inputs . . . . . . . . . . . . . . . . . . . 170
7.3.1 Distributed Continuous Static Controllers . . . . . . . 171
7.3.2 Adaptive Continuous Containment Controllers . . . . 176
7.3.3 Simulation Examples . . . . . . . . . . . . . . . . . . . 180
7.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

8 Distributed Robust Cooperative Control for Multi-Agent

Systems with Heterogeneous Matching Uncertainties 183
8.1 Distributed Robust Leaderless Consensus . . . . . . . . . . 184
8.1.1 Distributed Static Consensus Protocols . . . . . . . . 185
8.1.2 Distributed Adaptive Consensus Protocols . . . . . . . 190
8.2 Distributed Robust Consensus with a Leader of Nonzero Con-
trol Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
8.3 Robustness with Respect to Bounded Non-Matching Distur-
bances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
viii Contents

8.4 Distributed Robust Containment Control with Multiple Lead-

ers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
8.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

9 Global Consensus of Multi-Agent Systems with Lipschitz

Nonlinear Dynamics 207
9.1 Global Consensus of Nominal Lipschitz Nonlinear Multi-Agent
Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
9.1.1 Global Consensus without Disturbances . . . . . . . . 208
9.1.2 Global H∞ Consensus Subject to External Disturbances 211
9.1.3 Extensions to Leader-Follower Graphs . . . . . . . . . 214
9.1.4 Simulation Example . . . . . . . . . . . . . . . . . . . 216
9.2 Robust Consensus of Lipschitz Nonlinear Multi-Agent Systems
with Matching Uncertainties . . . . . . . . . . . . . . . . . . 219
9.2.1 Distributed Static Consensus Protocols . . . . . . . . 219
9.2.2 Distributed Adaptive Consensus Protocols . . . . . . . 224
9.2.3 Adaptive Protocols for the Case without Uncertainties 230
9.2.4 Simulation Examples . . . . . . . . . . . . . . . . . . . 231
9.3 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

Bibliography 235

Index 251
Preface

In the past two decades, rapid advances in miniaturizing of computing, com-

munication, sensing, and actuation have made it feasible to deploy a large
number of autonomous agents to work cooperatively to accomplish civilian
and military missions. This, compared to a single complex agent, has the ca-
pability to significantly improve the operational effectiveness, reduce the costs,
and provide additional degrees of redundancy. Having multiple autonomous
agents to work together efficiently to achieve collective group behaviors is usu-
ally referred to as cooperative control of multi-agent or multi-vehicle systems.
Due to its potential applications in various areas such as satellite formation
flying, distributed computing, robotics, surveillance and reconnaissance sys-
tems, electric power systems, cooperative attack of multiple missiles, and intel-
ligent transportation systems, cooperative control of multi-agent systems has
received compelling attention from various scientific communities, especially
the systems and control community.
For a cooperative control problem, the main task is to design appropriate
controllers to achieve the desired group objective. Due to the large number of
agents, the spatial distribution of actuators, limited sensing capability of sen-
sors, and short wireless communication ranges, it is considered too expensive
or even infeasible in practice to implement centralized controllers. Thus, dis-
tributed control, depending only on local information of the agents and their
neighbors, appears to be a promising tool for handling multi-agent systems.
Designing appropriate distributed controllers is generally a challenging task,
especially for multi-agent systems with complex dynamics, due to the inter-
connected effect of the agent dynamics, the interaction graph among agents,
and the cooperative control laws.
In this book, we address typical cooperative control problems, including
consensus, distributed tracking, and containment control, for multi-agent sys-
tems with general linear agent dynamics, linear agent dynamics with uncer-
tainties, and Lipschitz nonlinear agent dynamics. Different from the exist-
ing works, we present a systematic consensus region approach to designing
distributed cooperative control laws. One benefit of the proposed consensus
region is that it decouples the design of the feedback gain matrices of the
consensus protocols from the communication graph and can serve in a certain
sense as a measure for the robustness of the consensus protocols to variations
of the communication graph. We give algorithms built on consensus regions to
design the consensus protocols. In particular, by exploiting the decoupling fea-
ture of the consensus region approach, we present several distributed adaptive

ix
x Cooperative Control of Multi-Agent Systems

consensus protocols, which adaptively tunes the weights on the communica-

tion graph and can be designed by each agent in a fully distributed fashion.
This is one unique feature of the book.
This book consists of nine chapters. Chapter 1 introduces the background
of cooperative control and reviews graph theory and some technical tools.
Chapters 2 and 3 are concerned with the consensus control problem for
continuous-time and discrete-time linear multi-agent systems, respectively,
and present several fixed-gain consensus protocols. Chapter 4 studies the H∞
and H2 consensus problems for linear multi-agent systems subject to exter-
nal disturbances. Chapter 5 continues Chapter 2 to investigate the consensus
protocol design problem for continuous-time linear multi-agent systems and
presents several fully distributed adaptive consensus protocols. Chapter 6 con-
siders the distributed tracking control problem for linear multi-agent systems
having a leader with nonzero control input. Chapter 7 studies the distributed
containment control problem for the case with multiple leaders. Chapter 8 is
concerned with the robust cooperative control problem for multi-agent systems
with linear nominal agent dynamics subject to heterogeneous matching un-
certainties. The global consensus problem for Lipschitz nonlinear multi-agent
systems is finally discussed in Chapter 9.
The results in this book would not have been possible without the efforts
and help of our colleagues, collaborators, and students. In particular, we are
indebted to Professor Lin Huang, a member of the Chinese Academy of Sci-
ences, the supervisor and also a mentor of both of us, for leading us to the
research field of control and for his enduring support. We are also indebted
to Professors Guanrong Chen, Wei Ren, Lihua Xie, Gang Feng, and Frank
L. Lewis for their constant support and inspirations. We as well acknowledge
Guanghui Wen, Yu Zhao, Jingyao Wang, and Miao Diao, former and current
students of the second author, for their assistance in reviewing parts of the
manuscript. We would like to extend our thanks to Professor Frank L. Lewis
for inviting us to write this book and to Nora Konopka, Karen Simon, and
Joselyn Banks at Taylor & Francis for their professionalism. We wish to thank
our families for their support, patience, and endless love.
Finally, we gratefully acknowledge the financial support from the Nation-
al Natural Science Foundation of China under grants 61104153, 11332001,
61225013, and a Foundation for the Author of National Excellent Doctoral
Dissertation of PR China. This work is also partially supported by the State
Key Laboratory for Turbulence and Complex Systems, College of Engineering,
Peking University.
All the numerical simulations are done using MATLAB. MATLABr is a
registered trademark of The MathWorks, Inc. For product information please
contact
The MathWorks, Inc.
3 Apple Hill Drive, Natick, MA, 01760-2098 USA
Tel: 508-647-7000; Fax: 508-647-7001
E-mail: [email protected]; Web: www.mathworks.com
1
Introduction and Mathematical Background

CONTENTS
1.1 Introduction to Cooperative Control of Multi-Agent Systems . . . . . . . 1
1.1.1 Consensus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.2 Formation Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.3 Flocking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Overview of This Monograph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Mathematical Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.1 Notations and Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.2 Basic Algebraic Graph Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3.3 Stability Theory and Technical Tools . . . . . . . . . . . . . . . . . . . . . . . 16
1.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

1.1 Introduction to Cooperative Control of Multi-Agent

Systems
The subject of this monograph is cooperative control of multi-agent sys-
tems, where an agent denotes a dynamical system which can be a
ground/underwater vehicle, an aircraft, a satellite, a smart sensor with mi-
croprocessors, and so forth. In some literature, multi-agent systems are also
called multi-vehicle systems, with efforts to avoid causing confusion with the
multi-agent systems in computer science. The multi-agent systems concerned
in this monograph and in the systems and control community are quite differ-
ent from those in the computer science community, regarding the meanings,
the objectives, and the commonly used tools, even though they share the same
name.
The objective of cooperative control is to have multiple autonomous agents
work together efficiently to achieve collective group behavior via local interac-
tion. Recent advances in miniaturizing of computing, communication, sensing,
and actuation have made it feasible to deploy a large number of autonomous
agents to work cooperatively to accomplish civilian and military missions.
This, compared to a single complex agent, has the capability to significantly
improve the operational effectiveness, reduce the costs, and provide additional
degrees of redundancy. Cooperative control of multi-agent systems consists of
many different and yet related research topics and problems. Typical coopera-

1
2 Cooperative Control of Multi-Agent Systems

tive control problems include consensus, formation control, flocking, coverage

control, distributed estimation, task and role assignment, among others.
The study of cooperative control of multi-agent systems is stimulated by
at least the following two reasons.

• Observations and descriptions of collective behavior in nature: For in-

stance, flocking is the collective motion regularly seen in a flock of birds,
a school of fish, or a swarm of insects [118]. A group of birds or fishes
moving in flocks are believed to have the advantage of seeking foods, mi-
grating, or avoiding predators and obstacles. Flocking behavior was first
simulated on a computer by Reynolds [140] in the 1980s by proposing
three simple rules. The mathematical study of the flocking behavior can
be well carried out in the framework of cooperative control of multi-agent
systems. Several algorithms consisting of consensus components and ar-
tificial potential functions have been presented in the literature, e.g., in
[29, 119, 162, 165], which have been theoretically shown to produce the
flocking behavior observed by Reynolds. These algorithms only require in-
dividuals to coordinate with their nearby neighbors and do not involve
any central coordination. Recently, ten autonomous drones flying in for-
mation and coordinating flight patterns without central control have been
reported as the first outdoor demonstration of how biologically inspired
control rules can be used to create resilient yet dynamic flocks [184].
• Motivation and stimulus of many industrial applications and social sys-
tems: We take satellite formation flying as a typical example. Satellite
formation flying is the concept that multiple satellites work together in a
group to accomplish the objective of one larger, usually more expensive,
satellite. Coordinating smaller satellites has several advantages, includ-
ing unprecedented image resolution, reduced cost, better reconfigurability,
and higher redundancy [9, 167]. Some potential applications for satellite
formation flying include space interferometers and military surveillance
instruments. Satellite formation flying missions, e.g., the space interfer-
ometry for the Terrestrial Planet Finder mission [6], generally require au-
tonomous precise attitude and position maintenance of multiple satellites.
These problems can be formulated as formation control and attitude syn-
chronization, which are typical cooperative control problems in multi-agent
systems. Another example is wireless sensor networks, for which two im-
portant problems are clock synchronization [153] and deployment in an
unknown environment [59]. One efficient protocol for synchronizing the
clocks of a wireless sensor network is based on a cascade of two consen-
sus algorithms, which is shown to be adaptive to time-varying clock drifts
and changes of the communication topology, and robust to packet drop
and sensor node failure [145]. Deploying a mobile sensor network to cover
an unknown environment is known as coverage control [27], which is an
extensively studied cooperative control problem [61, 76]. Apart from the
aforementioned two examples, coordination of multi-agent systems has
Introduction and Mathematical Background 3

many other potential applications, such as distributed computing [170],

robotics [113], surveillance and reconnaissance systems [49], electric power
systems [12], cooperative attack of multiple missiles [66], and intelligent
transportation systems [73].

Given a group objective, the cooperative control problem of multi-agent

systems is mainly composed of three components, namely, the agent dynamics,
the interactions among the agents, and the cooperative control laws required
to achieve the group objective. The configuration of these three components is
depicted in FIGURE 1.1. The selection of the cooperative control laws depends
on the agent dynamics and the interaction topology. The interplay of these
three components generally renders the design of the control cooperative laws
troublesome, especially for the case with complex agent dynamics.

Agent Dynamics

Inter-Agent Interactions Cooperative Control Laws

FIGURE 1.1: Configuration of multi-agent systems.

For different scenarios, the dynamics of the employed agents may also be
different. The simplest agent dynamics are those modeled by first-order inte-
grators, which can be adopted if the velocities of the agents such as mobile
sensors and robots can be directly manipulated. When the accelerations rather
than the velocities of the agents can be directly manipulated, it is common
to use second-order integrators to describe the agent dynamics. Most existing
works on cooperative control of multi-agent systems focus on first-order and
second-order integrator agent dynamics. More complicated agent dynamics are
described by high-order linear systems, which contain integrator-type agents
as special cases and can also be regarded as the linearized models of nonlin-
ear agents dynamics. If we are concerned with cooperative control of multiple
robotic manipulators or attitude synchronization of satellite formation flying,
the agents are generally described by second-order nonlinear Euler-Lagrangian
equations. In some cases, the agents may be perturbed by parameter uncer-
tainties, unknown parameters, or external disturbances, which should also be
taken into account.
For multi-agent systems concerned by the systems and control community,
4 Cooperative Control of Multi-Agent Systems

the agents are generally dynamically decoupled from each other, which im-
plies the necessity of cooperation in terms of information exchange between the
agents to achieve collective behavior. Specifically, each agent needs to receive
information from other agents via a direct sensing or communication network.
The interaction (information exchange) topology among the agents is usually
represented as a graph, where each agent is a node and the information ex-
changes between the agents are represented as edges. Detailed knowledge of
the graph theory is summarized in Section 1.3. An edge in a graph implies
that there exists information flow between the nodes on this edge. The infor-
mation flows among the agents can be directed or undirected. The interaction
graph among the agents can be static or dynamic. For static graphs, the edges
are not time varying, i.e., the graph remains unchanged throughout the whole
process. In contrast, the edges in dynamic graphs are time-varying, due to
disturbances and/or subject to communication range limitations.
For the cooperative control problem, the main task is to design appropriate
controllers to achieve a desired coordination objective. For cooperative control
design, two approaches, namely, the centralized approach and the distributed
approach, are commonly adopted. The centralized approach assumes that at
least a central hub (which can be one of the agents) is available and has the
ability to collect information from and send control signals to all the other
agents. The distributed approach is based on local interactions only, i.e., each
agent exchanges information with its neighbors. Due to the large number of
agents, the spatial distribution of actuators, limited sensing capability of sen-
sors, and short wireless communication ranges, it is considered too expensive
or even infeasible in practice to implement centralized controllers. Thus, dis-
tributed control, depending only on local information of the agents and their
neighbors, appears to be a promising resolution for multi-agent systems. In
some cases, only the relative states or output information between neighboring
agents, rather than the absolute states or outputs of the agents, is accessible
for controllers design. One typical case is that the agents are equipped with
only ultrasonic range sensors. Another instance is deep-space formation flying,
where the measurement of the absolute position of the spacecraft is accurate
only in the order of kilometers, which is thus useless for control purposes [155].
In the past decade, cooperative control of multi-agent systems has been
extensively studied and quite a large amount of research works has been
reported; see the recent surveys [4, 20, 115, 120, 134], the monographs
[8, 74, 112, 127, 138, 139], and the references therein. In the following sub-
sections, we will briefly introduce some typical cooperative control problems:
consensus, formation control, and flocking.

1.1.1 Consensus
In the area of cooperative control of multi-agent systems, consensus is an
important and fundamental problem, which is closely related to formation
control, flocking, distributed estimation, and so on. Consensus means that a
Introduction and Mathematical Background 5

team of agents reaches an agreement on a common value by interacting with

each other via a sensing or communication network.
Consensus problems have a long history in computer science and form the
foundation of distributed computing. Pioneering works on the consensus prob-
lems for distributed decision making and parallel computing include [14, 170].
The emerging interest in consensus control in the last decade is mainly stim-
ulated by [65] and [121]. In particular, a theoretical explanation was provided
in [65] for the alignment behavior observed in the Vicsek model [174] and a
general framework of the consensus problem for networks of integrators was
proposed in [121]. Since then, a large body of works has been reported on the
consensus problems; refer to the surveys [4, 20, 120, 134].
In the following, we briefly introduce some recent works on consensus. In
[133], a sufficient condition was derived to achieve consensus for first-order
integrator multi-agent systems with jointly connected communication graphs.
Consensus of networks of second- and high-order integrators was studied in
[68, 97, 130, 132, 135, 202]. The consensus problem of multi-agent systems with
general linear dynamics was addressed in [83, 86, 93, 106, 146, 173, 185, 195].
Consensus of multiple rigid bodies or spacecraft described as Euler-Lagrange
systems was investigated in [7, 24, 25, 79, 109, 142, 176]. In [90, 156, 189],
conditions were derived for achieving consensus for multi-agent systems with
Lipschitz-type nonlinearity. Consensus algorithms were designed in [22, 75]
for multi-agent systems with quantized communication links. The effect of
various communication delays and input delays on consensus was considered
in [13, 97, 114, 122, 166]. Sampled-data control protocols were proposed to
achieve consensus for fixed and switching agent networks in [17, 47]. Distribut-
ed consensus protocols with event-triggered communications were designed in
[34, 149]. Distributed H∞ consensus and control problems were investigated
in [85, 99] for networks of agents subject to external disturbances.
Existing consensus algorithms can be roughly categorized into two classes,
namely, consensus without a leader (i.e., leaderless consensus) and consensus
with a leader. The latter is also called leader-follower consensus or distributed
tracking. The authors in [56, 57] designed a distributed neighbor-based estima-
tor to track an active leader. Distributed tracking algorithms were proposed in
[131] for a network of agents with first-order dynamics. In [21], discontinuous
controllers were studied in the absence of velocity or acceleration measure-
ments. The authors in [109] addressed the distributed coordinated tracking
problem for multiple Euler-Lagrange systems with a dynamic leader. The dis-
tributed tracking of general linear multi-agent systems with a leader of nonzero
control input was addressed in [92, 95].
The distributed tracking problem deals with only one leader. However, in
some practical applications, there might exist more than one leader in agent
networks. In the presence of multiple leaders, the containment control prob-
lem arises, where the followers are to be driven into a given geometric space
spanned by the leaders [67]. The study of containment control has been mo-
tivated by many potential applications. For instance, a group of autonomous
6 Cooperative Control of Multi-Agent Systems

vehicles (designated as leaders) equipped with necessary sensors to detect the

hazardous obstacles can be used to safely maneuver another group of vehicles
(designated as followers) from one target to another, by ensuring that the
followers are contained within the moving safety area formed by the leaders
[19, 67]. A hybrid containment control law was proposed in [67] to drive the
followers into the convex hull spanned by the leaders. Distributed contain-
ment control problems were studied in [16, 18, 19] for a group of first-order or
second-order integrator agents under fixed or switching directed communica-
tion topologies. The containment control was considered in [104] for second-
order multi-agent systems with random switching topologies. A hybrid model
predictive control scheme was proposed in [46] to solve the containment and
distributed sensing problems in leader-follower multi-agent systems. The au-
thors in [36, 110, 111] studied the containment control problem for a collection
of Euler-Lagrange systems. The containment control for multi-agent systems
with general linear dynamics was considered in [88, 94].

1.1.2 Formation Control

Compared with the consensus problem where the final states of all agents typ-
ically become a singleton, the final states of all agents can be more diversified
under the formation control scenario. The objective of formation control is to
stabilize the relative distances/positions among the agents to prescribed val-
ues. Formation control finds natural applications in satellite formation flying,
cooperative transportation, sensor networks, and so forth. Roughly speaking,
formation control can be categorized into formation producing and formation
tracking. Formation producing refers to the algorithm design for a group of
agents to reach some pre-desired geometric pattern in the absence of a group
reference while formation tracking refers to the same task following a pre-
designated group reference [20]. Due to the existence of the group reference,
formation tracking is usually much more challenging than formation produc-
ing.
The consensus algorithms lay a firm basis for the formation control prob-
lem. By adding appropriate offset variables, the consensus protocols can be
modified to produce desired formation structures [44, 86, 136]. Also, as an
extension of the consensus algorithms, some coupling matrices were intro-
duced [100, 123, 137] for systems with single-integrator kinematics and double-
integrator dynamics to offset the corresponding control inputs by some an-
gles. The collective motions for nonholonomic mobile robots were studied in
[147, 148] for both all-to-all and limited communications. The inverse agree-
ment problem was studied in [35] and was used to force a team of agents
to disperse in space. Motivated by the graph rigidity, research has been con-
ducted in [45, 54, 188] to drive a group of agents to a desired inter-agent
configuration by ensuring that a certain number of edge distances are identi-
cal to the desired ones. Compared with other formation producing algorithms
which require edge vector information, the formation control algorithms based
Introduction and Mathematical Background 7

on rigidity only require edge distance information. As a tradeoff, some unsta-

ble equilibria, rather than the desired inter-agent configuration, might exist.
Formation control in the presence of a group reference was studied in [38, 40]
for a group of nonholonomic mobile robots and in [141] for multiple rigid
bodies. More works on formation control were reported in [20].

1.1.3 Flocking
Flocking is a typical collective behavior in autonomous multi-agent systems,
which can be widely observed in nature in the scenes of grouping of birds,
schooling of fish, swarming of bacteria, and so on. In practice, understanding
the mechanisms responsible for the emergence of flocking in animal groups can
help develop many artificial autonomous systems such as formation control of
unmanned air vehicles, motion planning of mobile robots, and scheduling of
automated highway systems.
The classical flocking model proposed by Reynolds [140] in the 1980s con-
sists of three heuristic rules: 1) collision avoidance: attempt to avoid collisions
with nearby flockmates; 2) velocity matching: attempt to match velocity with
nearby flockmates; and 3) flock centering: attempt to stay close to nearby
flockmates. These three rules are usually known as separation, alignment, and
cohesion, respectively. In 1995, Vicsek et al. proposed a simple model for phase
transition of a group of self-driven particles and numerically observed the col-
lective behavior of synchronization of multiple agents’ headings [174]. In a
sense, Vicsek’s model can be viewed as a special case of Reynolds’ flocking
model, because only velocity matching is considered in the former. To sys-
tematically investigate the flocking control problem in multi-agent systems,
distributed algorithms consisting of a local artificial potential function and a
velocity consensus component were proposed in [119, 165]. The local artificial
potential function guarantees separation and cohesion whereas the velocity
consensus component ensures alignment in such multi-agent systems. This
kind of three-rule combined flocking algorithms were then extended to deal
with obstacles avoidance in [28]. The flocking control problem of multiple non-
holonomic mobile robots was investigated in [39] by using tools from nonlinear
control theory. Distributed protocols to achieve flocking in the presence of a
single or multiple virtual leaders were designed in [160, 161, 162]. The flocking
control problem of multi-agent systems with nonlinear velocity couplings was
investigated in [52, 179]. Distributed hybrid controllers were proposed in [194],
which can ensure that the flocking behavior emerges and meanwhile retain the
network connectivity.
8 Cooperative Control of Multi-Agent Systems

1.2 Overview of This Monograph

The objective of this monograph is to address typical cooperative con-
trol problems of multi-agent systems, including consensus, distributed track-
ing, and containment control. Although there exist several monographs
[8, 74, 112, 127, 138, 139] on recent advances in cooperative control of multi-
agent systems, it is worth noting that the agent dynamics considered in those
monographs are assumed to be first-order, second-order integrators, Euler-
Lagrange systems, or systems with passivity properties. These assumptions
might be too restrictive under many circumstances. This motivates us to write
this monograph, which focuses on cooperative control of multi-agent systems
with more general and complex agent dynamics.
In this monograph, we address the consensus, distributed tracking and
containment problems for multi-agent systems with general linear agent dy-
namics, linear agent dynamics with uncertainties, or Lipschitz nonlinear agent
dynamics. Differing from the existing works, in this monograph we present
a systematic consensus region approach to designing distributed cooperative
control laws, which is a quite challenging task due to the interplay of the agent
dynamics, the interaction graph among agents, and the cooperative control
laws. The proposed consensus region decouples the design of the feedback
gain matrices of the consensus protocols from the communication graph and
can serve in a certain sense as a measure for the robustness of the consensus
protocols to variations of the communication graph. By exploiting the decou-
pling feature of the proposed consensus region approach, distributed adaptive
consensus protocols are proposed, which adaptively tune the weights on the
communication graph and can be designed and implemented by each agent in
a fully distributed fashion. This is one main contribution of this monograph.
The results in this monograph also extend many existing results in the area
of cooperative control.
This monograph consists of nine chapters. A brief introduction of these
chapters is given as follows.
• Chapter 1 introduces the background of cooperative control and reviews
the basic graph theory and some technical tools which will be used later.
• Chapter 2 is concerned with the consensus control problem of multi-agent
systems with general continuous-time linear agent dynamics. The notion
of continuous-time consensus region is introduced. Algorithms based on
the consensus region will be presented to design several consensus pro-
tocols based on relative states or relative outputs of neighboring agents.
Extensions of consensus algorithms to formation control are also discussed.
• Chapter 3 presents the discrete-time counterpart of Chapter 2. The notion
of discrete-time consensus region is introduced and algorithms are present-
ed to design consensus protocols based on relative states or relative outputs
Introduction and Mathematical Background 9

of neighboring agents. The main differences between continuous-time and

discrete-time consensus problems are highlighted.

• Chapter 4 studies the H∞ and H2 consensus problems for linear multi-

agent systems subject to external disturbances. The consensus regions
with H∞ and H2 performance indices are introduced and algorithms to
design consensus protocols to achieve consensus with desired H∞ and H2
performances are presented.

• Chapter 5 continues Chapter 2 to investigate the consensus protocol design

problem for continuous-time linear multi-agent systems. By exploiting the
decoupling feature of the proposed consensus region approach, distributed
adaptive consensus protocols are proposed, which can be designed by each
agent in a fully distributed fashion. The robustness issue of the adaptive
protocols is also discussed.
• Chapter 6 considers the distributed tracking control problem for linear
multi-agent systems with a leader of nonzero control input. Several dis-
tributed discontinuous and continuous tracking controllers are designed,
which contains additional terms to deal with the effect of the leader’s
nonzero control input.

• Chapter 7 studies the distributed containment control problem for lin-

ear multi-agent systems with multiple leaders. The containment control
problem reduces to distributed tracking for the case with only one leader.
Several distributed controllers are designed by using local information of
neighboring agents.

• Chapter 8 is concerned with the robust cooperative control problem for

multi-agent systems with linear nominal agent dynamics but subject to
heterogeneous matching uncertainties. Under some further assumptions on
the uncertainties, distributed controllers are designed for both the cases
with and without a leader having nonzero control input.

• Chapter 9 considers the global consensus problem for Lipschitz nonlinear

multi-agent systems. Both the cases with and without matching uncer-
tainties are investigated, for which distributed protocols are designed.

1.3 Mathematical Preliminaries

1.3.1 Notations and Definitions
The notations used in this book are standard: Rn×n and Cn×n denote the
set of n × n real matrices and complex matrices, respectively. R+ denotes
10 Cooperative Control of Multi-Agent Systems

the set of positive real numbers. The superscript T means transpose for real
matrices and H means conjugate transpose for complex matrices. IN repre-
sents the identity matrix of dimension N . Matrices, if not explicitly stated,
are assumed to have compatible dimensions. Denote by 1 a column vector
with all entries equal to one. kxk denotes its 2-norm of a vector x and kAk
denotes the induced 2-norm of a real matrix A. For ζ ∈ C, denote by Re(ζ)
its real part and by Im(ζ) its imaginary part. rank(A) denotes the rank of a
matrix A. diag(A1 , · · · , An ) represents a block-diagonal matrix with matrices
Ai , i = 1, · · · , n, on its diagonal. det(B) denotes the determinant of a matrix
B. For real symmetric matrices X and Y , X > (≥)Y means that X − Y
is positive (semi-)definite. For a symmetric matrix A, λmin (A) and λmax (A)
denote, respectively, the minimum and maximum eigenvalues of A. range(B)
denotes the column space of a matrix B, i.e, the span of its column vectors. ι
denotes the imaginary unit.
Definition 1 A matrix A ∈ Cn×n is neutrally stable in the continuous-time
sense if it has no eigenvalue with positive real part and the Jordan block cor-
responding to any eigenvalue on the imaginary axis is of size one, while is
Hurwitz if all of its eigenvalues have strictly negative real parts.
Definition 2 A matrix A ∈ Cn×n is neutrally stable in the discrete-time
sense if it has no eigenvalue with magnitude larger than 1 and the Jordan block
corresponding to any eigenvalue with unit magnitude is of size one, while is
Schur stable if all of its eigenvalues have magnitude less than 1.
Definition 3 A matrix A ∈ Rn×n is a Hermite matrix if A = AH . Real
Hermite matrices are called symmetric matrix. A is a unitary matrix if AH A =
AAH = I. Real unitary matrices are called orthogonal matrices.
Definition 4 Matrix P ∈ Rn×n is an orthogonal projection onto a subspace
S, if range(P ) = S, P 2 = P , and P T = P . The orthogonal projection onto a
subspace is unique. For an orthogonal projection P onto a subspace S, if the
columns of C ∈ Rn×m are an orthonormal basis for S, then P = C T C and
S = range(P ) [168].
Definition 5 The Kronecker product of matrices A ∈ Rm×n and B ∈ Rp×q
is defined as  
a11 B · · · a1n B
 ..  ,
A ⊗ B =  ... ..
. . 
am1 B ··· amn B
which satisfies the following properties:
(A ⊗ B)(C ⊗ D) = (AC) ⊗ (BD),
(A ⊗ B)T = AT ⊗ B T ,
A ⊗ (B + C) = A ⊗ B + A ⊗ C,
where the matrices are assumed to be compatible for multiplication.
Introduction and Mathematical Background 11

Definition 6 A square matrix A ∈ Rn×n is called a singular (nonsingular)

M -matrix, if all its off-diagonal elements are non-positive and all eigenvalues
of A have nonnegative (positive) real parts.

Definition 7 Let C be a set in a real vector space V ⊆ Rp . The set C is

called convex if, for any x and y in C, the point (1 − z)x + zy is in C for
any z ∈ [0, 1]. The convex hull for a set of points X = {x1 , · · · , xn } in V is
PN
the minimal convex set containing all points in X, given by { i=1 αi xi | xi ∈
PN
X, αi ∈ R ≥ 0, i=1 αi = 1}.

1.3.2 Basic Algebraic Graph Theory

A team of agents interacts with each other via communication or sensing
networks to achieve collective objectives. It is convenient to model the infor-
mation exchanges among agents by directed or undirected graphs. A directed
graph G is a pair (V, E), where V = {v1 , · · · , vN } is a nonempty finite node
set and E ⊆ V × V is an edge set of ordered pairs of nodes, called edges. A
weighted graph associates a weight with every edge in the graph. A subgraph
Gs = (Vs , Es ) of G is a graph such that Vs ⊆ V and Es ⊆ E. Self loops in the
form of (vi , vi ) are excluded unless otherwise indicated. The edge (vi , vj ) in
the edge set E denotes that agent vj can obtain information from agent vi ,
but not necessarily vice versa. For an edge (vi , vj ), node vi is called the parent
node, vj is the child node, and vi is a neighbor of vj . The set of neighbors of
node vi is denoted as Ni , whose cardinality is called the in-degree of node vi .
A graph is defined as being balanced when it has the same number of
ingoing and outgoing edges for all the nodes (Edge (vi , vj ) is said to be out-
going with respect to node vi and incoming with respect to vj ). A graph with
the property that (vi , vj ) ∈ E implies (vj , vi ) ∈ E for any vi , vj ∈ V is said
to be undirected, where the edge (vi , vj ) denotes that agents vi and vj can
obtain information from each other. Clearly, an undirected graph is a special
balanced graph.
A directed path from node vi1 to node vil is a sequence of ordered edges
of the form (vik , vik+1 ), k = 1, · · · , l − 1. An undirected path in an undirected
graph is defined analogously. A cycle is a directed path that starts and ends
at the same node. A directed graph is strongly connected if there is a directed
path from every node to every other node. Note that for an undirected graph,
strong connectedness is simply termed connectedness. A directed graph is
complete if there is an edge from every node to every other node. A undirected
tree is an undirected graph where all the nodes can be connected by the way
of a single undirected path. A (rooted) directed tree is a directed graph in
which every node has exactly one parent except for one node, called the root,
which has no parent and has directed paths to all other nodes. A directed tree
is defined as spanning when it connects all the nodes in the graph. It can be
demonstrated that this implies that there is at least one root node connected
12 Cooperative Control of Multi-Agent Systems

with a simple path to all the other nodes. A graph is said to have or contain
a directed spanning tree if a subset of the edges forms a directed spanning
tree. This is equivalent to saying that the graph has at least one node with
directed paths to all other nodes. For undirected graphs, the existence of a
directed spanning tree is equivalent to being connected. However, in directed
graphs, the existence of a directed spanning tree is a weaker condition than
being strongly connected. A strongly connected graph contains at least one
directed spanning tree.

1 6
1 6

2 5 2 5

3 4 3 4

(a) (b)

1 6 1 6

2 5 2 5

3 4 3 4

(c) (d)

FIGURE 1.2: Different types of graphs of six nodes: (a) an undirected con-
nected graph, (b) a strongly connected graph, (c) a balanced and strongly
connected graph, (d) a directed spanning tree.

In FIGURE 1.2 are several examples of different types of graphs with six
nodes. In this book, a directed edge in a graph is denoted by a line with
a directional arrow while an undirected edge is denoted by a line with a
bidirectional arrow or no arrows.
The adjacency matrix A = [aij ] ∈ RN ×N associated with the directed
Introduction and Mathematical Background 13

graph G is defined such that aii = 0, aij is a positive value if (vj , vi ) ∈ E and
aij = 0 otherwise. The adjacency matrix of an undirected graph is defined
analogously except that aij = aji for all i 6= j. Note that aij denotes the
weight for the edge (vj , vi ) ∈ E. If the weights are not relevant, then aij is set
N ×N
equal to 1 if (vj , vi ) ∈ E.
PThe Laplacian matrix L = [Lij ] ∈ R of the graph
G is defined as Lii = j6=i aij and Lij = −aij , i 6= j. The Laplacian matrix
can be written into a compact form as L = D−A, where D = diag(d1 , · · · , dN )
is the degree matrix with di as the in-degree of the i-th node.

Example 1 For the graphs depicted in FIGURE 1.2, we compute the corre-
sponding Laplacian matrices as
   
3 −1 0 0 −1 −1 3 0 0 −1 −1 −1
−1 3 −1 −1 0 0 −1 1 0 0 0 0
   
 0 −1 2 0 0 −1  −1 −1 2 0 0 0 
La =    ,
0 −1 0 2 0 −1  , Lb = −1 0 0 1 0 0 
   
−1 0 0 0 1 0 0 0 0 −1 1 0
−1 0 −1 −1 0 3 0 0 0 0 −1 1
   
2 0 −1 −1 0 0 0 0 0 0 0 0
−1 2 0 0 0 −1 −1 1 0 0 0 0
   
 0 −1 1 0 0 0   −1 0 1 0 0 0
Lc = −1 0 , L = 
 0 −1 0 1 0 0 .

 0 2 −1 0  
d
 
0 0 0 −1 2 −1 0 0 −1 0 1 0
0 −1 0 0 −1 2 0 0 0 0 −1 1

From the definition of the Laplacian matrix and also the above example,
it is easy to see that L is diagonally dominant and has nonnegative diagonal
entries. Since L has zero row sums, 0 is an eigenvalue of L with an associ-
ated eigenvector 1. According to Gershgorin’s disc theorem [58], all nonzero
eigenvalues of L are located within a disk in the complex plane centered at
dmax and having radius of dmax , where dmax denotes the maximum in-degree
of all nodes. According to the definition of M -matrix in the last subsection,
we know that the Laplacian matrix L is a singular M -matrix.

Lemma 1 ([2, 133]) The Laplacian matrix L of a directed graph G has at

least one zero eigenvalue with 1 as a corresponding right eigenvector and all
nonzero eigenvalues have positive real parts. Furthermore, zero is a simple
eigenvalue of L if and only if G has a directed spanning tree. In addition, there
exists a nonnegative left eigenvector r of L associated with the zero eigenvalue,
satisfying rT L = 0 and rT 1 = 1. Moreover, r is unique if G has a directed
spanning tree.

Lemma 2 ([121]) For an undirected graph, zero is a simple eigenvalue of L

if and only if the graph is connected. The smallest nonzero eigenvalue λ2 of L
T
satisfies λ2 = min xxTLx x
.
x6=0,1T x=0
14 Cooperative Control of Multi-Agent Systems

For an undirected graph, it is known that L ≥ 0, i.e., y T Ly ≥ 0, ∀y ∈ RN .

In this case, it is not difficult to verify that L satisfies the following sum-of-
squares (SOS) property:
N N
1 XX
T
y Ly = aij (yi − yj )2 ,
2 i=1 j=1

where yi denotes the i-th item of y. For strongly connected and balanced
directed graphs, y T Ly ≥ 0, ∀y ∈ RN , still holds, due to the property that
1 T
2 (L + L ) represents the Laplacian matrix of the undirected graph obtained
from the original directed graph by replacing the directed edges with undirect-
ed ones [121]. However, for general directed graphs which are not balanced,
the Laplacian matrix L is not symmetric and y T Ly can be sign-indefinite.
Fortunately, by exploiting the property of L being a singular M -matrix, we
have the following result.

Lemma 3 ([10, 127, 189]) Suppose that G is strongly connected. Then,

there is a positive left eigenvector r = [r1 , · · · , rN ]T of L associated with the
zero eigenvalue and RL + LT R ≥ 0, where R = diag(r1 , · · · , rN ).

The smallest nonzero eigenvalue λ2 of the Laplican matrix L for a con-

nected undirected graph is usually called the Fiedler eigenvalue, also known
as the graph algebraic connectivity. For a strongly connected directed graph,
a generalized algebraic connectivity can be defined as follows.

Lemma 4 ([189]) For a strongly connected graph G with Laplacian matrix L,

T T
define its generalized algebraic connectivity as a(L) = min x (RL+L
2xT Rx
R)x
,
r T x=0,x6=0
where r and R are defined as in Lemma 3. Then, a(L) > 0. For balanced
T T
graphs, a(L) = λ2 ( L+L
2 ), where λ2 ( L+L
2 ) denotes the smallest nonzero
T
L+L
eigenvalue of 2 .

The Laplacian matrix is very useful in the study of consensus of continuous-

time multi-agent systems.

Lemma 5 ([133, 138]) Suppose that z = [z1 , · · · , zN ]T with zi ∈ R. Let

A ∈ RN ×N and L ∈ RN ×N be, respectively, the adjacency matrix and the
Laplacian matrix associated with the directed graph G. Then, consensus is
reached in the sense of limt→∞ kzi (t) − zj (t)k = 0, ∀ i, j = 1, · · · , N , for the
PN
closed-loop system ż = −Lz or equivalently żi = − j=1 aij (xi − xj ), where
aij denotes the (i, j)-th entry of A, if and only if G has a directed spanning
tree. Furthermore, the final consensus value is given by rT z(0), where r is the
normalized left eigenvector of L associated with the zero eigenvalue.

Apart from the Laplacian matrix L defined as above, another form of

Introduction and Mathematical Background 15

Laplacian matrix is Lb = I − Ab = I − D−1 A. In the event that D is singu-

lar due to a vertex vi with zero in-degree, we set d−1i = 0 to complete the
definition. Clearly, Lb = D−1 L. In order to avoid confusion, we refer to Lb as
the normalized Laplacian matrix and Ab as the normalized adjacency matrix.
From the definitions of D and A, it follows that the (i, j)-th entry âij of Ab is
a
equal to PN ij a . Note that even for undirected graphs, Lb is not necessarily
j=1 ij

b we have the following lemma.

symmetric. Regarding the spectrum of L,

Lemma 6 ([44]) Zero is an eigenvalue of Lb with 1 as a corresponding right

eigenvector and all nonzero eigenvalues lie in a disk of radius 1 centered at
the point 1 + ι0 in the complex plane. Furthermore, zero is a simple eigenvalue
of L if and only if G has a directed spanning tree.

The stochastic matrix is more convenient to deal with the consensus prob-
lem for discrete-time multi-agent systems, as detailed in Chapter 3.
For a directed graph G with N nodes, the row-stochastic matrix D ∈
N ×N
R
Pm is defined as dii > 0, dij > 0 if (j, i) ∈ E but 0 otherwise, and
j=1 d ij = 1. The stochastic matrix for undirected graphs can be defined
analogously except that dij = dji for all i 6= j. For undirected graphs, the
stochastic matrix D is doubly stochastic, since it is symmetric and all its row
sums and column sums equal to 1. Because the stochastic matrix D has unit
row sums, 1 is clearly an eigenvalue of D with an associated eigenvector 1.
According to Gershgorin’s disc theorem [58], all of the eigenvalues of D are
located in the unit disk centered at the origin.

Lemma 7 ([133]) All of the eigenvalues of the stochastic matrix D are either
in the open unit disk centered at the origin or equal to 1. Furthermore, 1 is a
simple eigenvalue of D if and only if the graph G contains a directed spanning
tree.

Lemma 8 ([120, 133]) Suppose that z = [z1 , · · · , zN ]T with zi ∈ R. Then,

consensus is reached in the sense of limk→∞ kzi (k) − zj (k)k = 0, ∀ i, j =
1, · · · , N , for the closed-loop system z(k + 1) = (I − ǫL)z or equivalently
PN
zi (k+1) = zi (k)− j=1 aij (xi (k)−xj (k)), where aij denotes the (i, j)-th entry
of the adjacency matrix A, ǫ ∈ (0, dmax ], and dmax denotes the maximum in-
degree of all nodes, if and only if G has a directed spanning tree. Furthermore,
the final consensus value is given by r̂T z(0), where r̂ is the normalized left
eigenvector of I − ǫL associated with the one eigenvalue.

Note that in the above lemma I − ǫL with ǫ ∈ (0, dmax ] is actually a row-
stochastic matrix, since I −ǫL has nonnegative entries and has row sums equal
to 1 which follows from the fact that (I − ǫL)1 = 1.
16 Cooperative Control of Multi-Agent Systems

1.3.3 Stability Theory and Technical Tools

Lemma 9 (Theorem 4.2 in [71]) Consider the autonomous system
ẋ = f (x), (1.1)
n n
where f : D → R is a locally Lipschitz map from a domain D ∈ R into
Rn . Suppose that x = 0 is an equilibrium point of (1.1). Let V : Rn → R
be a continuously differentiable function such that V (0) = 0 and V (x) > 0,
∀x 6= 0, V (x) is radially unbounded (i.e., V (x) → ∞ as kxk → ∞), and
V̇ < 0, ∀x 6= 0. Then, x = 0 is globally asymptotically stable.
Lemma 10 (LaSalle’s Invariance Principle, Theorem 4.4 in [71]) Let
Ω ⊂ D be a compact set that is positively invariant with respect to (1.1). Let
V : D → R be a continuously differentiable function such that V̇ ≤ 0 in Ω.
Let E be the set of all points in Ω where V̇ = 0 and M be the largest invariant
set in E. Then, every solution starting in Ω approaches M as t → ∞.
Note that in many applications the construction of V (x) will guarantee the
existence of a set Ω. In particular, if Ωc = {x ∈ Rn : V (x) ≤ c} is bounded
and V̇ ≤ 0, then we can choose Ω = Ωc .
The following lemma is a corollary of Lemma 10.
Lemma 11 (Corollary 4.2 in [71]) Let x = 0 be an equilibrium point for
(1.1). Let V : Rn → R be a continuously differentiable, radially unbounded,
positive definite function such that V̇ ≤ 0 for all x ∈ Rn . Let S = {x ∈ Rn :
V̇ (x) = 0} and suppose that no solution can stay identically in S, other than
the trivial solution x(t) ≡ 0. Then, the origin is globally asymptotically stable.
Lemma 12 (Theorems 4.8 and 4.9 in [71]) Consider the nonautonomous
system
ẋ = f (x, t), (1.2)
where f : [0, ∞) × D → Rn is a piecewise continuous in t and locally Lipschitz
in x on [0, ∞) × D, and D ∈ Rn is a domain that contains the origin x = 0.
Suppose that x = 0 is an equilibrium point of (1.2). Let V : [0, ∞) × D → R
be a continuously differentiable function such that
W1 (x) ≤ V (t, x) ≤ W2 (x),
∂V ∂V (1.3)
V̇ (t, x) = + f (t, x) ≤ 0
∂t ∂x
for any t ≥ 0 and x ∈ D, where W1 (x) and W2 (x) are continuous positive
definite functions on D. Then, x = 0 is uniformly stable. Suppose that V (t, x)
satisfies (1.3) and
V̇ (t, x) ≤ −W3 (x)
for any t ≥ 0 and x ∈ D, where W3 (x) is a continuous positive definite
function on D. Then, x = 0 is uniformly asymptotically stable. Moreover, if
D = Rn and W1 (x) is radically unbounded, then x = 0 is globally uniformly
asymptotically stable.
Introduction and Mathematical Background 17

Lemma 13 (Barbalat’s Lemma, Lemma 4.2 in [154]) If the differen-

tiable function f (t) has a finite limit as t → ∞, and if f˙ is uniformly contin-
uous, then f˙ → 0 as t → ∞.

A simple sufficient condition for a differentiable function to be uniformly

continuous is that its derivative is bounded.

Lemma 14 (LaSalle–Yoshizawa Theorem, Theorem A.8 in [72]) Sup-

pose that x = 0 is an equilibrium point of (1.2) and f is locally Lipschitz in
x uniformly in t. Let V : [0, ∞) × Rn → R be a continuously differentiable
function such that
W1 (x) ≤ V (t, x) ≤ W2 (x)
V̇ (t, x) ≤ −W3 (x) ≤ 0
n
for any t ≥ 0 and x ∈ R , where W1 (x) and W2 (x) are continuous positive
definite functions and W1 (x) is radically unbounded. Then, all the solutions
of (1.2) are globally uniformly bounded and satisfy

lim W3 (x(t)) = 0.
t→∞

Lemma 15 ([26]) For a system ẋ = f (x, t), where f (x, t) is locally Lipschitz
in x and piecewise continuous in t, assume that there exists a continuously
differentiable function V (x, t) such that along any trajectory of the system,

α1 (kxk) ≤ V (x, t) ≤ α2 (kxk),

(1.4)
V̇ (x, t) ≤ −α3 (kxk) + ǫ,

where ǫ > 0 ∈ R, α1 and α2 are class K∞ functions, and α3 is a class

K function 1 . Then, the solution x(t) of ẋ = f (x, t) is uniformly ultimately
bounded.

For the special case of autonomous system ẋ = f (x), the condition (1.4)
can be modified to that there exists a continuous positive definite and radically
unbounded function V (x) such that V̇ (x) ≤ −W (x) + ǫ, where W (x) is also
positive definite and radically unbounded.

Lemma 16 (Comparison Lemma, Lemma 3.4 in [71]) Consider the s-

calar differential equation

u̇ = f (t, u), u(t0 ) = u0 ,

where f (t, u) is continuous in t and locally Lipschitz in u, for all t ≥ 0 and

all u ∈ J ⊂ R. Let [t0 , T ) (T could be infinity) be the maximal interval of
1 A continuous function α : [0, a) → [0, ∞) is said to belong to class K if it is strictly

increasing and α(0) = 0. It is said to belong to class K∞ if a = ∞ and a(r) → ∞ as → ∞

[72].
18 Cooperative Control of Multi-Agent Systems

existence of the solution u(t), and suppose u(t) ∈ J for all t ∈ [t0 , T ). Let v(t)
be a continuous function whose upper right-hand derivative D+ v(t) 2 satisfies
the differential inequality

D+ v(t) = f (t, v(t)), v(t0 ) = u0 ,

where v(t) ∈ J for all t ∈ [t0 , T ). Then, v(t) ≤ u(t) for all t ∈ [t0 , T ).

Lemma 17 (Young’s Inequality, [11]) If a and b are nonnegative real

numbers and p and q are positive real numbers such that 1/p + 1/q = 1,
p q
then ab ≤ ap + bq .

Lemma 18 ([58]) For any given vectors x, y ∈ Rn and positive-definite ma-

trix P ∈ Rn×n , the following inequality holds:

2xT y ≤ xT P x + y T P −1 y.

Lemma 19 (Schur h Complement

i Lemma, [15]) For any constant sym-
S11 S12
metric matrix S = S T S22 , the following statements are equivalent:
12

(1) S < 0,
T −1
(2) S11 < 0, S22 − S12 S11 S12 < 0,
−1 T
(3) S22 < 0, S11 − S12 S22 S12 < 0.

Lemma 20 (Finsler’s Lemma, [15, 64]) For P = P T ∈ Rn×n and H ∈

Rm×n , the following two statements are equivalent:
(1) P − σH T H < 0 holds for some scalar σ ∈ R,
(2) There exists a X ∈ Rn×m such that P + XH + H T X T < 0.

1.4 Notes
The materials of Section 1.1 are mainly based on [4, 8, 20, 74, 112, 120, 134,
138, 139]. Section 1.3.2 is mainly based on [48, 134, 138]. Section 1.3.3 is
mainly based on [15, 71, 72, 154].

2 D + v(t) v(t+h)−v(t)
is defined by D + v(t) = lim sup h
.
h→0+
2
Consensus Control of Linear Multi-Agent
Systems: Continuous-Time Case

CONTENTS
2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2 State Feedback Consensus Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.1 Consensus Condition and Consensus Value . . . . . . . . . . . . . . . . . 22
2.2.2 Consensus Region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.3 Consensus Protocol Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2.3.1 The Special Case with Neutrally Stable Agents . . 30
2.2.3.2 The General Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2.3.3 Consensus with a Prescribed Convergence Rate . 33
2.3 Observer-Type Consensus Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.3.1 Full-Order Observer-Type Protocol I . . . . . . . . . . . . . . . . . . . . . . . 35
2.3.2 Full-Order Observer-Type Protocol II . . . . . . . . . . . . . . . . . . . . . . 40
2.3.3 Reduced-Order Observer-Based Protocol . . . . . . . . . . . . . . . . . . . 41
2.4 Extensions to Switching Communication Graphs . . . . . . . . . . . . . . . . . . . . 43
2.5 Extension to Formation Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

In the area of cooperative control of multi-agent systems, consensus is an

important and fundamental problem, which is closely related to formation
control, flocking, and other problems. Consensus means that a team of agents
reaches an agreement on a common value by interacting with each other via
a sensing or communication network. For the consensus control problem, the
main task is to design appropriate distributed controllers, usually called con-
sensus protocols, based on local information of neighboring agents to achieve
consensus.
Two pioneer papers on consensus control are [65] and [121]. A theoreti-
cal explanation was provided in [65] for the alignment behavior observed in
the Vicsek model [174] and a general framework of the consensus problem
for networks of integrators was proposed in [121]. Since then, the consensus
problem has been extensively studied by numerous researchers from various
perspectives. Please refer to Section 1.1.1 of the previous chapter and the sur-
veys [4, 20, 120, 134] for recent works on consensus. In most existing works on
consensus, the agent dynamics are restricted to be first-order, second-order,
or high-order integrators, which might be restrictive in many circumstances.

19
20 Cooperative Control of Multi-Agent Systems

In this chapter, we consider the consensus problem for multi-agent systems

with each node being a general continuous-time linear system. The importance
of studying consensus for general linear multi-agent systems lies in at least two
aspects. First, the general linear agents are more representative, which contain
the first-order, second-order, and high-order integrators as special cases. Sec-
ond, consensus of general linear multi-agent systems can be regarded as local
consensus of nonlinear multi-agent systems, and it paves a way for studying
more complex multi-agents systems, e.g., those with linear nominal dynamics
but subject to different types of uncertainties.
In this chapter, we present a consensus region approach to systematically
designing distributed consensus protocols for multi-agent systems with general
continuous-time linear agents. In Section 2.2, a distributed static consensus
protocol based on the relative states of neighboring agents is proposed, under
which the condition for reaching consensus depends on three correlated factors:
the agent dynamics, the design parameters of the consensus protocol, and the
nonzero eigenvalues of the Laplacian matrix of the communication graph.
To facilitate the consensus protocol design, the notion of consensus region is
introduced and analyzed. In light of the consensus region methodology, the
design of the consensus protocol can be decoupled into two steps: 1) Determine
the feedback gain matrix in order to yield a desirable consensus region; 2)
adjust the coupling gain to deal with the effect of the communication graph
on consensus. It is also demonstrated that the consensus region should be large
enough in order to be robust with respect to variations of the communication
graph. Necessary and sufficient conditions are derived to ensure a desirable
unbounded consensus region and multi-step algorithms built on the consensus
region framework are presented to design the static consensus protocol. The
consensus problem with a prescribed convergence rate is further studied. It is
shown that for an arbitrarily large convergence rate of reaching consensus, a
consensus protocol can be designed if each agent is controllable.
When the relative outputs, rather than the relative states, of neighboring
agents are accessible, distributed full-order and reduced-order observer-type
consensus protocols are proposed in Section 2.3. The observer-type consen-
sus protocols can be viewed as an extension of the traditional observer-based
controller for a single system to one for multi-agent systems. The separation
principle of the traditional observer-based controllers still holds in the multi-
agent setting. Multi-step algorithms to design the observer-type protocols are
also given. It is worth mentioning that for the observer-type protocols the ex-
change of the protocol’s internal states or virtual outputs between neighboring
agents via the communication graph is required in order to maintain the sepa-
ration principle. Extensions to the case with switching communication graphs,
which are connected at every time instant, are discussed in Section 2.4. The
consensus protocols are modified to solve the formation control problem in
Section 2.5.
Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 21

2.1 Problem Statement

Consider a network of N identical agents with general continuous-time lin-
ear dynamics, which may also be regarded as the linearized model of some
nonlinear systems. The dynamics of the i-th agent are described by
ẋi = Axi + Bui ,
(2.1)
yi = Cxi , i = 1, · · · , N,
where xi ∈ Rn , ui ∈ Rp , and yi ∈ Rq are, respectively, the state, the control
input, and the output of the i-th agent, and A, B, C, are constant matrices
with compatible dimensions.
The communication topology among the agents is represented by a directed
graph G (knowledge of basic graph theory can be found in Chapter 1). An
edge (i, j) in G means that the j-th agent can obtain information from the i-
th agent, but not conversely. The information exchange between neighboring
agents can be done via sensing or/and communication networks. Note that
each agent can have access to only local information, i.e., the state or output
information from itself and its neighbors.
The objective of this chapter is to solve the consensus problem for the
N agents described by (2.1), i.e., to design distributed control laws using
only local information to ensure that the N agents described by (2.1) achieve
consensus in the sense of limt→∞ kxi (t) − xj (t)k = 0, ∀ i, j = 1, · · · , N . The
distributed control laws are usually called consensus protocols. We refer to
a consensus protocol dynamic if it depends on the protocol’s internal state,
otherwise we call it static.
Intuitively, if the communication graph G can be decomposed into some
disconnected components, consensus is impossible to reach. Thus, G should be
connected in certain sense in order to achieve consensus. As shown in Lemma
5, a necessary and sufficient condition for a network of single integrators to
achieve consensus is that G contains a directed spanning tree. Throughout
this chapter, the following assumption is needed.
Assumption 2.1 The communication graph G contains a directed spanning
tree.
In the following sections, we shall design several static and dynamic con-
sensus protocols to solve the consensus problem for the agents in (2.1).

2.2 State Feedback Consensus Protocols

In this section, we consider the case where each agent can have access to its
own state and the states from its neighbors. A distributed static consensus
22 Cooperative Control of Multi-Agent Systems

protocol in this case is proposed as

N
X
ui = cK aij (xi − xj ), i = 1, · · · , N, (2.2)
j=1

where c > 0 ∈ R is called the (constant) coupling gain or weight, K ∈ Rp×n

is the feedback gain matrix, and aij is the (i, j)-th entry of the adjacency
matrix associated with G. Both c and K in (2.2) are to be determined. A
distinct feature of the consensus protocol (2.2) is that it includes a positive
scalar c, which can be regarded as a scaling factor or a uniform weight on
the communication graph G. The usefulness of introducing c here (without
merging it into K) will be highlighted later.
Let x = [xT1 , · · · , xTN ]T . With (2.1) and (2.2), the closed-loop network
dynamics can be obtained as

ẋ = (IN ⊗ A + cL ⊗ BK)x, (2.3)

where L is the Laplacian matrix of G. The network (2.3) can be also regarded
as a large-scale interconnected system [151].
Let r = [r1 , · · · , rN ]T ∈ RN be the left eigenvector of L associated with
the eigenvalue 0 and satisfy rT 1 = 1. Introduce a new variable δ ∈ RN n×N n
as follows:
δ = [(IN − 1rT ) ⊗ In ]x. (2.4)
By the definition of r, it is easy to verify that δ satisfies

(rT ⊗ In )δ = 0. (2.5)

Also, by the definition of r, it is not difficult to see that 0 is a simple eigenvalue

of IN − 1rT with 1 as the corresponding right eigenvector and 1 is another
eigenvalue with multiplicity N − 1. Then, it follows from (2.4) that δ = 0 if
and only if x1 = · · · = xN . That is, the consensus problem is solved if and
only if limt→∞ δ(t) = 0. Hereafter, we refer to δ as the consensus error. By
using the facts that L1 = 0 and rT L = 0, it follows from (2.3) and (2.4) that
δ evolves according to the following dynamics:

δ̇ = (IN ⊗ A + cL ⊗ BK)δ. (2.6)

2.2.1 Consensus Condition and Consensus Value

The following presents a necessary and sufficient condition to achieve consen-
sus.

Theorem 1 Supposing Assumption 2.1 holds, the distributed consensus pro-

tocol (2.2) solves the consensus problem for the agents in (2.1) if and only if all
the matrices A + cλi BK, i = 2, · · · , N , are Hurwitz, where λi , i = 2, · · · , N ,
are the nonzero eigenvalues of the Laplacian matrix L.
Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 23

Proof 1 Since G contains a directed spanning, it follows from Lemma 1 that

0 is a simple eigenvalue of L and the other eigenvalues have positive real part.
It is well known that there exists a nonsingular matrix T such that T −1 LT is
in the Jordan canonical form [58]. Since the left and right eigenvectors of L
corresponding to the zero eigenvalue are r and 1, respectively, we can choose
 T
  −1 r
T = 1 Y1 and T = (2.7)
Y2
with Y1 ∈ CN ×(N −1) and Y2 ∈ C(N −1)×N such that
 
−1 0 0
T LT = J = , (2.8)
0 ∆
where ∆ ∈ C(N −1)×(N −1) is a upper-triangular matrix whose the diagonal
entries are the nonzero eigenvalues of L. Let δ̃ = (T −1 ⊗ In )δ with δ̃ =
T T
[δ̃1T , · · · , δ̃N ] . Then, (2.6) can be represented in terms of δ̃ as
˙ (2.9)
δ̃ = (IN ⊗ A + cJ ⊗ BK)δ̃.
From (2.5) and (2.6), it is easy to see that
δ̃1 = (rT ⊗ In )δ ≡ 0. (2.10)
Note that the elements of the state matrix of (2.9) are either block diagonal
or block upper-triangular. Hence, δ̃i , i = 2, · · · , N , converge asymptotically to
zero if and only if the N − 1 subsystems along the diagonal, i.e.,
˙
δ̃i = (A + cλi BK)δ̃i , i = 2, · · · , N, (2.11)
are asymptotically stable. Therefore, the stability of the matrices A + cλi BK,
i = 2, · · · , N , is equivalent to that the state δ of (2.6) converges asymptotically
to zero, i.e., the consensus problem is solved.
Remark 1 The importance of this theorem lies in that it converts the con-
sensus problem of a large-scale multi-agent network under the protocol (2.2)
to the simultaneous stabilization problem of a set of matrices with the same
dimension as a single agent, thereby significantly reducing the computational
complexity. The effects of the communication topology on consensus are char-
acterized by the nonzero eigenvalues of the corresponding Laplacian matrix L,
which may be complex, rendering the matrices in Theorem 1 complex-valued.
The following theorem presents the explicit form for the final consensus
value.
Theorem 2 Suppose Assumption 2.1 holds. If the consensus protocol (2.2)
satisfies Theorem 1, then
 
x1 (0)
 
xi (t) → ̟(t) , (rT ⊗ eAt )  ...  , as t → ∞, (2.12)
xN (0)
24 Cooperative Control of Multi-Agent Systems

where r ∈ RN is such that rT L = 0 and rT 1 = 1.

Proof 2 The solution of (2.3) can be obtained as

x(t) = e(IN ⊗A+cL⊗BK)t x(0)

= (T ⊗ In )e(IN ⊗A+cJ⊗BK)t (T −1 ⊗ In )x(0)
 At  (2.13)
e 0
= (T ⊗ In ) (T −1 ⊗ In )x(0),
0 e(IN −1 ⊗A+c∆⊗BK)t

where matrices T , J and ∆ are defined in (2.7) and (2.8). By Theorem 1,

IN −1 ⊗ A + c∆ ⊗ BK is Hurwitz. Thus,

e(IN ⊗A+cL⊗BK)t → (1 ⊗ In )eAt (rT ⊗ In )

= (1rT ) ⊗ eAt , as t → ∞.

It then follows from (2.13) that

x(t) → (1rT ) ⊗ eAt x(0), as t → ∞,

which directly leads to the assertion.

Remark 2 From (2.12), we can see that the final consensus value ̟(t) de-
pends on the state matrix A, the communication topology L, and the initial
states of the agents. Here we briefly discuss how the agent dynamics affect
̟(t). If A is Hurwitz, then ̟(t) → 0 as t → ∞, i.e., the consensus problem
in this case is trivial; If A has eigenvalues along the imaginary axis but no
eigenvalues with positive real parts, then ̟(t) is bounded; If A is unstable,
i.e., it has eigenvalues with positive real parts, then ̟(t) will tend to infin-
ity exponentially. The usefulness of studying the last case is that the linear
agents in (2.1) sometimes denote the linearized dynamics of nonlinear agents
(e.g., chaotic systems) whose solution, even though A is unstable, can still be
bounded, implying the final consensus value may also be bounded.

2.2.2 Consensus Region

From Theorem 1, it can be noticed that consensus of the agents (2.1) under
the protocol (2.2) depends on the feedback gain matrix K, the coupling gain
c, and the nonzero eigenvalues λi of L associated with the communication
graph G. These three factors are actually coupled with each other. Hence, it
is useful to analyze the correlated effects of c, K, and L on consensus. To this
end, the notion of consensus region is introduced.

Definition 8 The region S of the parameter σ belonging to the open right half
plane, such that the matrix A + σBK is Hurwitz, is called the (continuous-
time) consensus region of the network (2.3).
 Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 25

By definition, the shape of the consensus region relies on only the agent
dynamics and the feedback gain matrix K of (2.2). For directed communi-
cation graphs, the consensus regions can be roughly classified into five type-
s, namely, bounded consensus region, consensus region with unbounded real
part but bounded imaginary part, consensus region with unbounded imag-
inary part but bounded real part, consensus region with both unbounded
real and imaginary parts, and disconnected consensus regions. For undirected
communication graphs, there are only three different types of consensus re-
gions, namely, bounded consensus region, unbounded consensus region, and
disconnected consensus region.
It should be mentioned that the notion of consensus region is similar to
some extent to the synchronization region issue of complex dynamical net-
works, studied in [41, 42, 101, 105, 124]. The types of bounded, unbounded,
disconnected synchronization regions are discussed in [41, 101, 105] for undi-
rected graphs and in [42] for directed graphs. For the case where the inner
linking matrix, equivalently, the matrix BK in (2.3), is an identity matrix,
then an unbounded synchronization region naturally arises [101].
The key differences between the consensus region in this chapter and the
synchronization region in [41, 42, 101, 105, 124] are

• For the synchronization region in [41, 42, 101, 105, 124], the inner linking
matrix can be arbitrarily chosen. It is not the case here, since B is a priori
given, implying that the consensus region in this chapter is more complicated
to analyze.
• We are more interested in how the consensus region is related to the consen-
sus protocol and the communication topology, and more importantly, how
to design consensus protocols to yield desirable consensus regions.
The following result is a direct consequence of Theorem 1.

Corollary 1 The agents described by (2.1) reach consensus under the protocol
(2.2) if and only if cλi ∈ S, i = 2, · · · , N , where λi , i = 2, · · · , N , are the
nonzero eigenvalues of L 1 .

In light of the consensus region notion, the design of the consensus protocol
(2.2) can be divided into two steps:
1) Determine the feedback gain matrix K to yield a desirable consensus region;

2) Then adjust the coupling gain c such that cλi , i = 2, · · · , N , belong to the
consensus region.
The consensus protocol design based on the consensus region as above
has a desirable decoupling feature. Specifically, in step 1) the design of the
1 Actually, cλ ∈ S, i = 2, · · · , N , are the nonzero eigenvalues of the weighted Laplacian
i
matrix cL.
26 Cooperative Control of Multi-Agent Systems

feedback gain matrix K of the consensus protocol relies on only the agent
dynamics, independent of the communication topology, while the effect of the
communication topology on consensus is handled in step 2) by manipulating
the coupling gain c.
It is worth noting that the consensus region S can be seen as the stability
region of the matrix pencil A + σBK with respect to the complex parameter
σ. Thus, tools from the stability of matrix pencils will be utilized to analyze
the consensus region. Before moving on, the lemma below is needed.

Lemma 21 ([53]) Given a complex-coefficient polynomial

p(s) = s2 + (a + ιb)s + c + ιd,

√
where a, b, c, d ∈ R, and ι = −1 denotes the imaginary unit, p(s) is stable if
and only if a > 0 and abd + a2 c − d2 > 0.

In the above lemma, only second-order polynomials are considered. Similar

results for high-order complex-coefficient polynomials can also be given [53].
However, in the latter case, the analysis will be more complicated.
In what follows, we give three examples to depict different types of con-
sensus regions.

0.25

0.2

0.15

0.1

0.05
y

−0.05

−0.1

−0.15

−0.2

−0.25

0 0.2 0.4 0.6 0.8 1

x

FIGURE 2.1: Bounded consensus region.

Example 2 (Bounded consensus region)

The agent dynamics and the consensus protocol are given by (2.1) and
(2.2), respectively, with
   
−2 −1 1  
A= , B= , K = 1 −1 , σ = x + ιy.
2 1 0

The characteristic polynomial of A + σBK is

det(sI − (A + σBK)) = s2 + (1 − x − ιy)s + 3(x + ιy).

Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 27

By Lemma 21, A + σBK is Hurwitz if and only if

1 − x > 0, −3y 2 (1 − x) + 3x(1 − x)2 − 9y 2 > 0.

The consensus region S is depicted in FIGURE 2.1, which is bounded.

Example 3 (Consensus region with unbounded real part but bounded imagi-
nary part)
The agent dynamics and the consensus protocol are given by (2.1) and
(2.2), respectively, with
     
−2 −1 1 0 0 −1
A= , B= , K= , σ = x + ιy.
2 1 0 1 1 0

The characteristic polynomial of A + σBK in this case is

det(sI − (A + σBK)) = s2 + s − 2 + (x + 1)(x + 2) − y 2 + ι(2y + 3)y.

By Lemma 21, A+σBK is Hurwitz if and only if x(x+3)−y 2 −(2x+3)2 y 2 > 0.

The consensus region S in this case is depicted in FIGURE 2.2, which is
unbounded in the real coordinate but bounded in the imaginary coordinate.

0.8

0.6

0.4

0.2
y

−0.2

−0.4

−0.6

−0.8
0 2 4 6 8 10 12 14
x

FIGURE 2.2: Consensus region with unbounded real part.

Example 4 (Disconnected consensus region)

The agent dynamics and the consensus protocol are given by (2.1) and
(2.2), respectively, with
 
−1.4305 12.5142 3.3759
A= 1 −1 1 ,
−0.3911 −5.5845 −2.3369
 
0.8 −0.3  
  −0.8 0 0.3
B = 9.6 −5 , K = .
9.6 0 5
2.6 −1
28 Cooperative Control of Multi-Agent Systems

For simplicity, we consider here the case of undirected communication graphs.

In this case, σ ∈ R+ . In light of the well-known Hurwitz Criterion [23], we can
get that the consensus region S = (0, 1.4298) ∪ (2.2139, 7.386), which clearly
consists of two disjoint subregions.

Actually, it is theoretically proved in [41] that for undirected graphs and

any natural number n, there exists a network in the form of (2.3) with n
disconnected synchronized regions.
In the following, we will give a simple example to show that the consensus
region can serve in a certain sense as a measure for the robustness of the
consensus protocol to variations of the communication topology.

Example 5 The agent dynamics and the consensus protocol are given by
(2.1) and (2.2), respectively, with
   
−2.4188 1 0.0078 1 0
A = −0.3129 −1 1.1325  , B =  0 1 ,
−0.0576 1 −0.9778 3.7867 −0.0644
 
−0.85 0 0.35
K= .
−9.6 0 6.01

For simplicity, we here consider the case where the communication graph G is
undirected. The characteristic polynomial of A + σBK can be obtained as

det(sI − (A + σLC)) =s3 + (0.15σ + 4.3966)s2 + (0.06σ 2 − 1.8928σ

+ 4.9426)s + 4.33751σ 2 + 4.4978σ − 0.0001.

By Hurwitz criterion [23], A + σBK is Hurwitz if and only if

σ > 0, 0.06σ 2 − 1.8928σ + 4.9426 > 0, 4.33751σ 2 + 4.4978σ − 0.0001 > 0,

(0.15σ + 4.3966)(0.06σ 2 − 1.8928σ + 4.9426) > 4.33751σ 2 + 4.4978σ − 0.0001.

From the above inequalities, we can obtain that the consensus region S =
(1.037, 3.989), which is clearly bounded. Assume that the communication graph
G is given by FIGURE 2.3. The Laplacian matrix of G is equal to
 
4 −1 −1 −1 −1 0
−1 3 −1 0 0 −1
 
−1 −1 2 0 0 0
L=  ,
−1 0 0 2 −1 0  
−1 0 0 −1 3 −1
−1 0 0 0 −1 2

whose nonzero eigenvalues are 1.382, 1.6972, 3.618, 4, 5.3028. It follows from
Corollary 1 that consensus is achieved if and only if the coupling gain c satisfies
that 0.7504 < c < 0.7522.
Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 29
1 6

2 5

3 4

FIGURE 2.3: An undirected communication graph.

In many practical situations, due to the existence of obstacles, communi-

cation constraints, or communication link failures, the communication graph
may change during the process. Thus, the consensus protocol should maintain
certain degree of robustness with respect to the variations of the communica-
tion graph. Let us see how modifications of the communication graph affect
consensus by considering the following simple cases.
• An edge between node 3 and node 6 is added, i.e., more information ex-
changes exist inside the network. The minimal and maximal eigenvalues of
the resulting Laplacian matrix are 1.4384 and 5.5616, respectively. By Corol-
lary 1, it can be verified that the protocol (2.2) fails to solve the consensus
problem for any c ∈ R+ .
• The edge between node 4 and node 5 is removed, i.e., there are less infor-
mation exchanges in the network. The minimal and maximal eigenvalues of
the resulting Laplacian matrix become 0.8817 and 5.2688, respectively. The
protocol (2.2) in this case also fails to solve the consensus problem.
• An edge between node 2 and node 6 is added and meanwhile the edge between
node 4 and node 5 is removed. In this case, the number of information
exchanges remains the same. The minimal and maximal eigenvalues of the
resulting Laplacian matrix are 0.7639 and 5.2361, respectively. Once again,
the protocol (2.2) fails to solve the consensus problem.
For a bounded consensus region, it is required that both cλ2 and cλN , where
λ2 and λN denotes, respectively, the minimal and maximal nonzero eigenval-
ues of L, belong to the region. The reason for which the aforementioned three
cases fail to solve the consensus problem lies in that λλN2 does not grow mono-
tonically with the number of added or removed edges. The above sample cases
imply, for bounded consensus regions, that the consensus protocol (2.2), if not
well designed, can be quite fragile to variations of the communication topolo-
gy. In other words, it is desirable for the consensus region to be large enough
in order to ensure that the consensus protocol maintains a desired robustness
margin with respect to the communication topology.
 30 Cooperative Control of Multi-Agent Systems

2.2.3 Consensus Protocol Design

It was shown in the last subsection that the consensus protocol should have a
large enough consensus region so as to be robust with respect to the communi-
cation topology. One convenient and desirable choice is to design the protocol
yielding a consensus region which is unbounded in both real and imaginary
coordinates.

2.2.3.1 The Special Case with Neutrally Stable Agents

In this subsection, we consider a special case where the matrix A is neutrally
stable. It is shown that an unbounded consensus region in the form of the
open right-half plane can be achieved. A constructive design algorithm for the
protocol (2.2) is then presented.
First, the following lemmas are needed.

Lemma 22 ([117]) A complex matrix A ∈ Cn×n is Hurwitz if and only if

there exist a positive-definite matrix Q = QH and a matrix B ∈ Cn×m such
that (A, B) is controllable and AH Q + QA = −BB H .

Lemma 23 For matrices S ∈ Rn×n and H ∈ Rn×m , where S is skew-

symmetric (i.e., S + S T = 0) and (S, H) is controllable, the matrix S − (x +
ιy)HH T is Hurwitz for any x > 0 and y ∈ R.

Proof 3 Let S̃ = S − (x + ιy)HH T . Then,

S̃ + S̃ H = S − (x + ιy)HH T + S T − (x − ιy)HH T
(2.14)
= −2xHH T ≤ 0, ∀ x > 0.

Obviously, since (S, H) is controllable, then (S̃, H) is controllable. By Lemma

22, (2.14) directly leads to the assertion.

Next, an algorithm for constructing the protocol (2.2) is presented, which

will be used later.

Algorithm 1 Given that A is neutrally stable and that (A, B) is stabilizable,

the consensus protocol (2.2) can be constructed as follows:
1) Choose U ∈ Rn1 ×n and W ∈ R(n−n1 )×n such that 2
   −1  
U U S 0
A = , (2.15)
W W 0 X

where S ∈ Rn1 ×n1 is skew-symmetric and X ∈ R(n−n1 )×(n−n1 ) is Hurwitz.

2) Let K = −B T U T U .
2 The matrices U and W can be derived by rendering matrix A into the real Jordan

canonical form [58].

 Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 31

3) Select the coupling gain c > 0.

The following is the main result of this subsection.

Theorem 3 Given that A is neutrally stable and that Assumption 2.1 hold-
s, there exists a distributed protocol in the form of (2.2) that solves the
consensus problem and meanwhile yields an unbounded consensus region
(0, ∞) × (−∞, ∞) if and only if (A, B) is stabilizable. One such consensus
protocol is given by Algorithm 1.

Proof 4 (Necessity) It is trivial by Theorem 1.

(Sufficiency) Let the related variables be defined as in Algorithm 1. Con-
struct the protocol (2.2) by Algorithm 1 and let H = U B. Since (A, B) is sta-
bilizable, then it is easy to see that (S, H) is controllable. Let U † ∈ Rn×n1 and
W † ∈ Rn×(n−n1 ) be such that [ U † W † ] = [ W U ]−1 , where U U † = I, W W † = I,

W U † = 0, and U W † = 0. It can be verified by some algebraic manipulations

that
   −1
U U
(A + (x + ιy)BK)
W W
 
S + (x + ιy)U BKU † (x + ιy)U BKW †
= (2.16)
(x + ιy)W BKU † X + (x + ιy)W BKW †
 
S − (x + ιy)HH T 0
= ,
−(x + ιy)W BH T X

which implies that the matrix A + (x + iy)BK is Hurwitz for all x > 0 and
y ∈ R, because, by Lemma 22, S − (x + iy)HH T is Hurwitz for any x > 0
and y ∈ R. Hence, by Theorem 1, the protocol given by Algorithm 1 solves the
consensus problem with an unbounded consensus region (0, ∞) × (−∞, ∞).

Remark 3 The consensus region (0, ∞) × (−∞, ∞) achieved by the protocol

constructed by Algorithm 1 means that such a protocol can reach consensus for
any communication topology containing a directed spanning tree and for any
positive coupling gain. However, the communication topology and the coupling
gain do affect the performances of consensus, e.g., the convergence rate, which
will be discussed later.

Remark 4 It is worth mentioning that similar results of Theorem 3 are also

reported in [173]. Nevertheless, the method leading to Theorem 3 here is quite
different from that in [173] and is comparatively much simpler.

2.2.3.2 The General Case

In this subsubsection, we consider the general case without assuming that A
is neutrally stable. In this case, we will see that the consensus region we can
get is generally smaller than the case where A is neutrally stable.
32 Cooperative Control of Multi-Agent Systems

Proposition 1 For the agents described by (2.1), there exists a matrix K

such that A + (x + ιy)BK is Hurwitz for all x ∈ [1, ∞) and y ∈ (−∞, ∞), if
and only if (A, B) is stabilizable.

Proof 5 (Necessity) It is trivial by letting x = 1, y = 0.

(Sufficiency) Since (A, B) is stabilizable, there exists a matrix K such that
A + BK is Hurwitz, i.e., there exists a matrix P > 0 such that

(A + BK)P + P (A + BK)T < 0.

Let KP = Y . Then, the above inequality becomes

AP + P AT + BY + Y T B T < 0.

By Lemma 20 (Finsler’s Lemma), there exists a matrix Y satisfying the above

inequality if and only if there exists a scalar τ > 0 such that

AP + P AT − τ BB T < 0. (2.17)

Since the matrix P is to be determined, let τ = 2 without loss of generality.

Then,
AP + P AT − 2BB T < 0. (2.18)
Obviously, when (2.18) holds, for any τ ≥ 2, (2.17) holds. Take Y = −B T ,
i.e., K = −B T P −1 . Then in light of (2.18) we can get that

P [A + (x + ιy)BK]H + [A + (x + ιy)BK]P
= P [A + (x − ιy)BK]T + [A + (x + ιy)BK]P
= AP + P AT − 2xBB T < 0,

for all x ∈ [1, ∞) and y ∈ (−∞, ∞). That is, A + (x + ιy)BK is Hurwitz for
all x ∈ [1, ∞) and y ∈ (−∞, ∞).

Theorem 1 and the above proposition together lead to the following result.

Theorem 4 Assuming that Assumption 2.1 holds, there exists a protocol

(2.2) that solves the consensus problem for the agents described by (2.1) and
meanwhile yields an unbounded consensus region S , [1, ∞) × (−∞, ∞) if and
only if (A, B) is stabilizable.

Remark 5 Compared to the case when A is neutrally stable, where the con-
sensus region is the open right half plane, for the case where A is not neu-
trally stable, the consensus region can be achieved are generally smaller. This
is consistent with the intuition that unstable behaviors are more difficult to
synchronize than stable behaviors.

Next, a two-step consensus protocol design procedure based on the con-

sensus region notion is presented.
 Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 33

Algorithm 2 Given that (A, B) is stabilizable and Assumption 2.1 holds, a

protocol (2.2) solving the consensus problem can be constructed according to
the following steps.
1) Solve the LMI (2.18) to get one solution P > 0. Then, choose the feedback
gain matrix K = −B T P −1 .
2) Select the coupling gain c not less than the threshold value cth , given by
1
cth = , (2.19)
min Re(λi )
i=2,··,N

where λi , i = 2, · · · , N , are the nonzero eigenvalues of the Laplacian matrix

L.
Remark 6 This design algorithm has a favorable decoupling feature, as ex-
plained in the paragraph below Corollary 1. In step 2) the coupling gain c
generally has to be larger than a threshold value, which is related to the spe-
cific communication topology. For the case when the agent number N is large,
for which the eigenvalues of the corresponding Laplacian matrix are hard to
determine or even troublesome to estimate, we need to choose the coupling
gain c to be large enough, which of course involves conservatism.
Remark 7 The feedback gain matrix K of (2.2) can also be designed by solv-
ing the algebraic Riccati equation: AT Q + QA + I − QBB T Q = 0, as in
[171, 195]. In this case, K can be chosen as K = −B T Q. The solvability of
the above Ricatti equation is equal to that of the LMI (2.18).

2.2.3.3 Consensus with a Prescribed Convergence Rate

In this subsection, the protocol (2.2) is designed to achieve consensus with a
prescribed convergence rate.
From the proof of Theorem 1, it is easy to see that the convergence rate of
the N agents in (2.1) reaching consensus under the protocol (2.2) is equal to
the minimal decay rate of the N − 1 matrices A + cλi BK, i = 2, · · · , N . The
decay rate of the system ẋ = Ax is defined as the maximum of negative real
parts of the eigenvalues of A [15]. Thus, the convergence rate of the agents (2.1)
reaching consensus can be manipulated by properly assigning the eigenvalues
of the matrices A + cλi BK, i = 2, · · · , N .
Lemma 24 ([15]) The decay rate of the system ẋ = Ax is larger than α > 0
if and only if there exists a matrix Q > 0 such that AT Q + QA + 2αQ < 0.
Proposition 2 Given the agents in (2.1), there exists a matrix K such that
A + (x + ιy)BK is Hurwitz with a decay rate larger than α for all x ∈ [1, ∞)
and y ∈ (−∞, ∞), if and only if there exists a Q > 0 such that
AQ + QAT − 2BB T + 2αQ < 0. (2.20)
 34 Cooperative Control of Multi-Agent Systems

Proof 6 This proof can be completed by following similar steps as in the proof
of Proposition 1, which is omitted for conciseness.

The above proposition and Theorem 1 lead to the following result.

Theorem 5 Assume that Assumption 2.1 holds. There exists a protocol (2.2)
that solves the consensus problem for the agents in (2.1) with a convergence
rate larger than α and yields an unbounded consensus region [1, ∞)×(−∞, ∞)
if and only if there exists a K such that A + BK is Hurwitz with a decay rate
larger than α.

Then Algorithm 2 can be modified as follows.

Algorithm 3 For G satisfying Assumption 2.1, a protocol (2.2) solving the

consensus problem with a convergence rate larger than α can be constructed
as follows:
1) Choose the feedback gain matrix K = −B T Q−1 , where Q > 0 is one solution
to (2.20).
2) Select the coupling gain c ≥ cth , where cth is defined as in (2.19).

Remark 8 Note that a sufficient condition satisfying Theorem 5 is that

(A, B) is controllable. It is well known that the eigenvalues of A − BK can be
arbitrarily assigned provided that complex conjugate eigenvalues are assigned
in pairs if (A, B) is controllable [23]. However for the consensus problem with
a prescribed convergence rate of linear multi-agent systems in this subsection,
the eigenvalues of A + cλi BK, i = 2, · · · , N , cannot be arbitrarily assigned
due to the fact that the consensus protocol has to be distributed (depending
on local information), i.e., the protocol in this case is structural. This indi-
cates one constraint and difficulty brought by distributed control. Fortunately,
a stronger result is obtained that the real parts of the eigenvalues of A+cλi BK,
i = 2, · · · , N , which indicates the convergence rate of reaching consensus can
be arbitrarily assigned if (A, B) is controllable.

Remark 9 Under the condition that (A, B) is controllable, the protocol

achieving consensus with a arbitrarily large convergence rate α can be con-
structed by Algorithm 3. However, larger α implies higher feedback gains in
the consensus protocol (2.2). Thus, a tradeoff has to be made between the
convergence rate and the cost of the consensus protocol.

Remark 10 Similarly as in Remark 7, the feedback gain matrix K of (2.2)

solving the consensus problem with a convergence rate larger than α can also be
designed by solving the algebraic Riccati equation: AT Q̃ + Q̃A + I − Q̃BB T Q̃ +
2αQ̃ = 0. In this case, K can be chosen as K = −B T Q̃. A sufficient condition
for the solvability of the above Ricatti equation is that (A, B) is controllable.
Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 35

2.3 Observer-Type Consensus Protocols

In the previous section, the consensus protocol (2.2) rely on the relative states
of neighboring agents. However, in many circumstances the state information
might not be available. Instead, each agent can have access to the local output
information, i.e., its own output and the outputs from its neighbors. In this
section, we will design consensus protocols using local output information.

2.3.1 Full-Order Observer-Type Protocol I

In this subsection, we assume that each agent can get the relative output
information of itself with respect to its neighbors. Based on the relative output
information, we propose an observer-type consensus protocol as follows:
N
X
v̇i = (A + BF )vi + cL aij [C(vi − vj ) − (yi − yj )],
j=1 (2.21)
ui = F vi , i = 1, · · · , N,
where vi ∈ Rn , i = 1, · · · , N , are the protocol’s internal states, aij is the
(i, j)-th entry of the adjacency matrix associated with G, c > 0 denotes the
n×q
PNgain, L ∈ R
coupling and F ∈ Rp×n are the feedback gain matrices. The
term j=1 aij C(vi − vj ) in (2.21) means that the agents need to transmit the
virtual outputs of their corresponding controllers to their neighbors via the
communication topology G.
Let zi = [xTi , viT ]T and z = [z1T , · · · , zN
T T
] . Then, the closed-loop network
dynamics resulting from (2.1) and (2.21) can be obtained as
ż = (IN ⊗ A + cL ⊗ H)z, (2.22)

where L ∈ RN ×N is the Laplacian matrix of G, and

   
A BF 0 0
A= , H= .
0 A + BF −LC LC
The following presents a sufficient condition for the consensus problem
using the observer-type protocol (2.21).

Theorem 6 Suppose that Assumption 2.1 holds. The observer-type protocol

(2.21) solves the consensus problem for the agents described by (2.1) if all
the matrices A + BF and A + cλi LC, i = 2, · · · , N , are Hurwitz, where
λi , i = 2, · · · , N , are the nonzero eigenvalues of the L. Furthermore, the final
consensus value is given by
xi (t) → ̟(t), vi (t) → 0, i = 1, · · · , N, as t → ∞, (2.23)

where ̟(t) is defined as in (2.12).

36 Cooperative Control of Multi-Agent Systems

Proof 7 Similarly as in the previous section, let r ∈ RN be the left eigenvec-

tor of L associated with the eigenvalue 0, satisfying rT 1 = 1. Define

ξ(t) = [(IN − 1rT ) ⊗ I2n ]z(t), (2.24)

which satisfies (rT ⊗ I2n )ξ = 0. Following similar steps as in the proof of

Theorem 1, we can see that the consensus problem of (2.22) can be reduced to
the asymptotical stability problem of ξ, which evolves according to the following
dynamics:
ξ˙ = (IN ⊗ A + cL ⊗ H)ξ. (2.25)
Let T be defined as in (2.7) such that T −1 LT = J = [ 00 ∆
0 ], where the di-

agonal entries of ∆ are the nonzero eigenvalues of L. Let ξ = [ξ˜1T , · · · , ξ˜N

˜ T T
] =
−1
(T ⊗ I2n )ξ. Then, (2.24) can be rewritten as

˙
ξ˜ = (IN ⊗ A + cJ ⊗ H)ξ.
˜ (2.26)

Note that ξ˜1 = (rT ⊗ I2n )ξ ≡ 0 and that the elements of the state matrix
of (2.26) are either block diagonal or block upper-triangular. Hence, ξ˜i , i =
2, · · · , N , converge asymptotically to zero if and only if the N − 1 subsystems
along the diagonal, i.e.,
˙
ξ˜i = (A + cλi H)ξ˜i , i = 2, · · · , N, (2.27)

areasymptotically stable.
 It is easy to check that matrices A+cλi H are similar
0
to A+cλ i LC
−cλi LC A+BF , i = 2, · · · , N. Therefore, the stability of the matrices
A + BF , A + cλi LC, i = 2, · · · , N , is equivalent to that the state ξ of (2.25)
converges asymptotically to zero, i.e., the consensus problem is solved.
By following similar steps as in the proof of Theorem 2, we can obtain the
solution of (2.22) as

z(t) = e(IN ⊗A+cL⊗H)t z(0)

 At 
e 0 (2.28)
= (T ⊗ I2n ) (T −1 ⊗ I2n )z(0),
0 e(IN −1 ⊗A+c∆⊗H)t

Note that IN −1 ⊗ A + c∆ ⊗ H is Hurwitz. Thus, we can get from (2.28) that

z(t) → (1rT ) ⊗ eAt z(0), as t → ∞,

implying that  
z1 (0)
 
zi (t) → (rT ⊗ eAt )  ...  , as t → ∞, (2.29)
zN (0)
for i = 1, · · · , N. Since A + BK is Hurwitz, (2.29) directly leads to (2.23).
 Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 37

Remark 11 The observer-type consensus protocol (2.21) can be seen as an

extension of the traditional observer-based controller for a single system to one
for multi-agent systems. The Separation Principle of the traditional observer-
based controllers still holds in the multi-agent setting.

Remark 12 By comparing Theorems 2 and 6, it is observed that no matter

the consensus protocol takes the dynamic form (2.21) or the static form (2.2),
the final consensus value reached by the agents will be the same, which relies on
only the communication topology, the initial states, and the agent dynamics.

For the observer-type protocol (2.21), the consensus region can be similarly
defined. Assuming that A + BF is Hurwitz, the region of the parameter σ
belonging to the open right half plane, such that A + σLC is Hurwitz, is the
consensus region of the network (2.22).
In what follows, we will show how to design the observer-type protocol
(2.21). First, consider the special case where A is neutrally stable.

Algorithm 4 Given that A ∈ Rn×n is neutrally stable and that (A, B, C) is

stabilizable and detectable, the consensus protocol (2.21) can be constructed as
follows:
1) Let F be such that A + BF is Hurwitz.

2) Choose Û ∈ Rn×n1 and Ŵ ∈ Rn×(n−n1 ) such that

 
 −1   Ŝ 0
Û Ŵ A Û Ŵ = ,
0 X̂

where Ŝ ∈ Rn1 ×n1 is skew-symmetric and X̂ ∈ R(n−n1 )×(n−n1 ) is Hurwitz.

3) Let L = −Û Û T C T .
4) Select the coupling gain c > 0.

Theorem 7 Given that A is neutrally stable and that Assumption 2.1 holds,
there exists an observer-type protocol (2.21) that solves the consensus problem
and meanwhile yields an unbounded consensus region (0, ∞) × (−∞, ∞) if
and only if (A, B, C) is stabilizable and detectable. One such protocol can be
constructed by Algorithm 4.

Proof 8 (Necessity) It is trivial.

(Sufficiency) let Ĥ = C Û . Then, it follows from the detectability of (A, C)
† n1 ×n
h † (iŜ, Ĥ) is observable. Let Û ∈ R
that and Ŵ † ∈ R(n−n1 )×n be such that
−1
Û
Ŵ †
= [ Û Ŵ ] , where Û † Û = I, Ŵ † Ŵ = I, Û † Ŵ = 0, and Ŵ † Û = 0. It
 38 Cooperative Control of Multi-Agent Systems

can be verified by some algebraic manipulations that

 −1  
Û Ŵ (A + (x + ιy)LC) Û Ŵ
 
Ŝ + (x + ιy)Û † LC Û (x + ιy)Û † LC Ŵ
= †
(x + ιy)Ŵ LC Û X̂ + (x + ιy)Ŵ † LCW (2.30)
 
Ŝ − (x + ιy)Ĥ T Ĥ −(x + ιy)Ĥ T C Ŵ
= .
0 X̂

By Lemma 23, we can get that Ŝ − (x + ιy)Ĥ T Ĥ is Hurwitz for any x > 0 and
y ∈ R (which actually is dual to Lemma 23). Then, it follows from (2.30) that
A + (x + ιy)LC is Hurwitz for all x > 0 and y ∈ R. Hence, by Theorem 6, the
protocol given by Algorithm 4 solves the consensus problem with an unbounded
consensus region (0, ∞) × (−∞, ∞).

Next, consider the general case without requiring A neutrally stable.

The dual of Proposition 1 is presented as follows.

Proposition 3 Given the agent dynamics (2.1), there exists a matrix L such
that A + (x + ιy)LC is Hurwitz for all x ∈ [1, ∞) and y ∈ (−∞, ∞), if and
only if (A, C) is detectable.

Theorem 6 and the above proposition together lead to the following result.

Theorem 8 Assume that Assumption 2.1 holds. There exists an observer-

type protocol (2.21) that solves the consensus problem and meanwhile yields
an unbounded consensus region S , [1, ∞) × (−∞, ∞) if and only if (A, B, C)
is stabilizable and detectable.

The following algorithm is presented to construct an observer-type protocol

(2.21) satisfying Theorem 8.

Algorithm 5 Assuming that (A, B, C) is stabilizable and detectable, the

observer-type protocol (2.21) can be constructed according to the following
steps.
1) Choose the feedback gain matrix F such that A + BF is Hurwitz.
2) Solve the following LMI:

AQ + QAT − 2C T C < 0 (2.31)

to get one solution Q > 0. Then, choose the feedback gain matrix L =
−Q−1 C T .
3) Select the coupling gain c > cth , with cth given in (2.19).
 Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 39

1 6

2 5

3 4

FIGURE 2.4: A directed communication graph.

From the proof of Theorem 6, it is easy to see that the convergence rate of
the N agents in (2.1) reaching consensus under the protocol (2.21) is equal to
the minimal decay rate of the N − 1 systems in (2.27). Thus, the convergence
rate of the agents (2.1) reaching consensus can be manipulated by properly
assigning the eigenvalues of matrices A+BF and A+cλi LC, i = 2, · · · , N . The
following algorithm present a multi-step procedure to design (2.21) solving the
consensus problem with a prescribed convergence rate.

Algorithm 6 Assuming that (A, B, C) is controllable and observable, the

observer-type protocol (2.21) solving the consensus problem with a convergence
rate larger than α can be constructed as follows:
1) Get the feedback gain matrix F , e.g., by using the Ackermann’s formula [70],
such that the poles of matrix A + BF lie in the left-half plane of x = −α.
2) Choose the feedback gain matrix L = −Q−1 C T , where Q > 0 is one solution
to
AT Q + QA − 2C T C + 2αQ < 0. (2.32)

3) Select the coupling gain c > cth , with cth given in (2.19).

Example 6 The agent dynamics are given by (2.1), with

   
0 −1 0 1  
A = 1 0 0 , B = 0 , C = 0 0 1 .
0 1 0 0

Obviously, the matrix A is neutrally stable and (A, B, C) is controllable and

observable. A third-order consensus protocol is in the form of (2.2).
By the function place of MATLABr , the feedback gain matrix F of (2.21)
is given as F = [ −4.5 −5.5 −3 ] such that the poles of A + BF are −1, −1.5, −2.
The matrix T such that T −1 AT = J is of the real real Jordan canonical
40 Cooperative Control of Multi-Agent Systems
h0 0.5774 0
i h0 0 0i
form is T = 0 0 −0.5774 , with J = 0 0 1 . Then, by Algorithm 4,
1 −0.5774 0 0 −1 0
T
the feedback gain matrix L of (2.2) is obtained as L = [ 0.3333 0 −1.3333 ] . By
Theorem 7, the agents under this protocol can reach consensus with respect
to any communication graph containing a spanning tree and for any positive
coupling gain. One such graph with 6 nodes is shown in FIGURE 2.4. The
corresponding Laplacian matrix is
 
3 0 0 −1 −1 −1
−1 1 0 0 0 0
 
−1 −1 2 0 0 0
L= −1 0 0 1
,
 0 0
0 0 0 −1 1 0
0 0 0 0 −1 1

whose nonzero eigenvalues 1, 1.3376 ± ι0.5623, 2, 3.3247. Select c = 1 for sim-

plicity. It can be verified that the convergence rate in this case equals 0.0303.
Next, the protocol (2.21) is redesigned to achieve consensus with a specified
convergence rate larger than 1. The feedback gain F is chosen to be the same as
above. Solving LMI (2.32) with α = 1 by using SeDuMi toolbox [158] gives L =
T
[ −8.5763 −15.2128 −5.6107 ] . For the graph in FIGURE 2.4, the threshold value
for the coupling gain is cth = 1 by (2.19). Select c = 1, the same as before.
The consensus errors xi − x1 , i = 2, · · · , 6, for the graph in FIGURE 2.4,
under the protocols generated by Algorithms 4 and 6 with α = 1, are depicted
in FIGURE 2.5(a) and FIGURE 2.5(b), respectively. It can be observed that
the consensus process of the former case is indeed much slower than the latter.

10 8

6
5
4
Consensus errors

Consensus errors

2
0

−5
−2

−4
−10
−6

−15 −8
0 50 100 150 200 250 300 350 400 0 5 10 15 20 25 30 35 40
t t

(a) (b)

FIGURE 2.5: The consensus errors: (a) Algorithm 4, (b) Algorithm 6 with
α = 1.
Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 41

2.3.2 Full-Order Observer-Type Protocol II

In this subsection, we will propose another alternative dynamic consensus
protocol. Based on the relative output information of neighboring agents, we
propose the following full-order observer-type protocol:
N
X
v̄˙ i = (A + L̄C)v̄i + aij [cB F̄ (v̄i − v̄j ) + L̄(yi − yj )],
j=1 (2.33)
ui = cF̄ v̄i , i = 1, · · · , N,

where v̄i ∈ Rn , c > 0 denotes the coupling gain, L̄ ∈ Rn×q and F̄ ∈ Rp×n
are the feedback gain matrices.
Different from the dynamic protocol (2.21) where the protocol states vi
take roles of intermediate variables, the states v̄i of (2.33) can be the estimates
PN PN
of the relative states j=1 aij (xi − xj ). Let ei = v̄i − j=1 aij (xi − xj ),
i = 1, · · · , N . Then, we can get from (2.33) that ei , i = 1, · · · , N , satisfy the
following dynamics:

ėi = (A + L̄C)ei , i = 1, · · · , N.

Clearly, if L̄P
is chosen such that A + L̄C is Hurwtiz, then v̄i asymptotically
N
converge to j=1 aij (xi − xj ). Let e = [eT1 , · · · , eTN ]T and x = [xT1 , · · · , xTN ]T .
Then, we can get from (2.1) and (2.33) that the closed-loop network dynamics
can be written in terms of e and x as
    
ẋ A + cL ⊗ B F̄ cB F̄ x
= . (2.34)
ė 0 A + L̄C e

By following similar steps in showing Theorem 6, we obtain the following

result.

Theorem 9 Suppose that Assumption 2.1 holds. Then, the observer-type pro-
tocol (2.33) solves the consensus problem for the agents in (2.1) if all the
matrices A + L̄C and A + cλi B F̄ , i = 2, · · · , N , are Hurwitz.

Note that the design of F̄ and L̄ in Theorem 9 is actually dual to the design
of L and F in Theorem 6. Algorithms 4 and 5 in the previous subsection can
be modified to determine the parameters c, F̄ , and L̄ of the dynamic protocol
(2.33). The details are omitted here for conciseness.

2.3.3 Reduced-Order Observer-Based Protocol

The observer-type protocol (2.21) in the last subsection is of full order, i.e.,
whose dimension is equal to that of a single agent. Note that this full-order
protocol possesses a certain degree of redundancy, which stems from the fact
that while the observer constructs an “estimate” of the entire state, part of the
 42 Cooperative Control of Multi-Agent Systems

state information is already reflected in the system outputs. In this subsection,

we will design a reduced-order protocol so as to eliminate this redundancy and
thereby can considerably reduce the dimension of the protocol especially for
the case where the agents are multi-input multi-output (MIMO) systems.
Based on the local output information of neighboring agents, we introduce
here a reduced-order observer-based consensus protocol as
v̇i = F vi + Gyi + T Bui ,
N
X N
X (2.35)
ui = cK[Q1 aij (yi − yj ) + Q2 aij (vi − vj )], i = 1, · · · , N,
j=1 j=1

where vi ∈ Rn−q is the protocol state, c > 0 is the coupling gain, aij is the
(i, j)-th entry of the adjacency matrix of G, F ∈ R(n−q)×(n−q) is Hurwitz and
has no eigenvalues in common with those of A, G ∈ R(n−q)×q , T ∈ R(n−q)×n
is the unique solution to the following Sylvester equation:

T A − F T = GC, (2.36)
n×q
which further satisfies that [ CT ] is nonsingular, Q1 ∈ R and Q2 ∈ Rn×(n−q)
−1
are given by [ Q1 Q2 ] = [ C
T ] , and K ∈ Rp×n is the feedback gain matrix.

Remark 13 Different from the full-order observer-type protocols (2.21) and

(2.33) which rely on the relative output information of neighboring agents,
the above reduced-order protocol (2.35) requires the absolute measures of the
agents’ outputs, which might not be available in some circumstances, e.g., in
deep-space formation flying [155].
T T
Let ζi = [xTi , viT ]T and ζ = [ζ1T , · · · , ζN ] . Then, the closed-loop network
dynamics resulting from (2.1) and (2.35) can be written as

ζ̇ = (IN ⊗ M + cL ⊗ R) ζ, (2.37)

where L ∈ RN ×N is the Laplacian matrix of G, and

   
A 0 BKQ1 C BKQ2
M= , R= .
GC F T BKQ1 C T BKQ2

Next, an algorithm is presented to select the parameters in the consensus

protocol (2.35).

Algorithm 7 Supposing that (A, B, C) is stabilizable and observable and As-

sumption 2.1 holds, the protocol (2.35) can be constructed as follows:
1) Choose a Hurwitz matrix F having no eigenvalues in common with those of
A. Select G such that (F, G) is controllable.
2) Solve (2.36) to get a solution T , which satisfies that [ C T ] is nonsingular.
−1
Then, compute matrices Q1 and Q2 by [ Q1 Q2 ] = [ C T ] .
 Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 43

3) Solve the LMI (2.18) to get one solution P > 0. Then, choose the matrix
K = −B T P −1 .
4) Select the coupling gains c ≥ cth , with cth given as in (2.19).
Remark 14 By Theorem 8.M6 in [23], a necessary condition for the matrix T
to the unique solution to (2.36) and further to satisfy that [ CT ] is nonsingular is
that (F, G) is controllable, (A, C) is observable, and F and A have no common
eigenvalues. In the case where the agent in (2.1) is single-input single-output
(SISO), this condition is also sufficient. Under such a condition, it is shown
for the general MIMO case that the probability for [ C T ] to be nonsingular is 1
[23]. If [ C
T ] is singular in step 2), we need to go back to step 1) and repeat the
process. As shown in Proposition 1, a necessary and sufficient condition for
the existence of a positive-definite solution to the LMI (2.18) is that (A, B) is
stabilizable. Therefore, a sufficient condition for Algorithm 7 to successfully
construct a protocol (2.35) is that (A, B, C) is stabilizable and observable.
Theorem 10 Assuming that Assumption 2.1 holds, the reduced-order proto-
col (2.35) solves the consensus problem for the agents described by (2.1) if
the matrices F and A + cλi BK, i = 2, · · · , N are Hurwitz. One such protocol
solving the consensus problem is constructed by Algorithm 7. Then, the state
trajectories of (2.37) satisfy
 
x1 (0)
 
xi (t) → ̟(t) , (rT ⊗ eAt )  ...  ,
(2.38)
xN (0)
vi (t) → T ̟(t), i = 1, · · · , N, as t → ∞,

where r ∈ RN is a nonnegative vector such that rT L = 0 and rT 1 = 1.

Proof 9 By following similar steps as in the proof of Theorem 6, we can
reduce the consensus problem of (2.37) to the simultaneous asymptotically
stability problem of the following N − 1 systems:
˙
ζ̃i = M + cλi Rζ̃i , i = 2, · · · , N,
where λi , i = 2, · · · , N, are the nonzero eigenvalues
 I 0of L. Multiplying the left
−1
and right sides of the matrix M + cλi R by Q = −T I and Q , respectively,
and in virtue of (2.36) and step 2) of Algorithm 7, we get
 
A + cλi BK cλi BKQ2
Q(M + cλi R)Q−1 = .
0 F
Therefore, the consensus problem of (2.37) is reached if F and A+cλi BK, i =
2, · · · , N are Hurwitz. As shown in Proposition 1, A + cλi BK, i = 2, · · · , N ,
with c and K designed as in Algorithm 7, are Hurwitz. The rest of the proof
can be completed by following similar steps as in the proof of Theorem 2, which
is thus omitted here.
44 Cooperative Control of Multi-Agent Systems

For the case where the state matrix A is neutrally stable, Algorithm 7 can
be modified by refereing to Algorithm 1. Moreover, Algorithm 7 can also be
modified to construct the protocol (2.35) to reach consensus with a prescribed
convergence rate. We omit the details here for conciseness.

2.4 Extensions to Switching Communication Graphs

In the previous sections, the communication graph is assumed to be fixed
throughout the whole process. As mentioned in Example 5, the communica-
tion graph may change with time in many practical situations due to various
reasons, such as communication constraints and link variations. The objective
of this section is to extend the results in the previous sections to the case with
switching communication graphs. In this section, we restrict our attention to
the case with undirected communication graphs.
Denote by GN the set of all possible undirected connected graphs with
N nodes. Let σ(t) : [0, ∞) → P be a piecewise constant switching signal
with switching times t0 , t1 , · · · , and P be the index set associated with the
elements of GN , which is clearly finite. The communication graph at time t is
denoted by Gσ(t) .
For illustration and conciseness, here we consider only the static consensus
protocol (2.2). Extensions of the observer-type protocols can be similarly done.
For the case with switching undirected communication graphs, (2.2) becomes
N
X
ui = cK aij (t)(xi − xj ), (2.39)
j=1

where aij (t) is the (i, j)-th entry of the adjacency matrix associated with Gσ(t)
and the rest of the variables are the same as in (2.2). Using (2.39) for (2.1),
we can obtain the closed-loop network dynamics as

ẋ = (IN ⊗ A + cLσ(t) ⊗ BK)x, (2.40)

where x = [xT1 , · · · , xTN ]T and Lσ(t) is the Laplacian matrix associated with
Gσ(t) .
Let ς = [(IN − N1 11T ) ⊗ In ]x. Then, as shown in Section 2.2, the consensus
problem of (2.40) is reduced to the asymptotical stability problem of ς, which
satisfies
ς˙ = (IN ⊗ A + cLσ(t) ⊗ BK)ς. (2.41)
Algorithm 2 will be modified as below to design the consensus protocol
(2.39).

Algorithm 8 Assuming that (A, B, C) that is stabilizable and detectable, and

 Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 45

that Gσ(t) ∈ GN , a protocol (2.40) can be constructed according to the following

steps:

1) step 1) in Algorithm 2.
1
2) Select the coupling gain c ≥ λmin , where λmin
2 , minGσ(t) ∈GN {λ2 (Lσ(t) )}
2
denotes the minimum of the smallest nonzero eigenvalues of Lσ(t) for Gσ(t) ∈
GN .

The following theorem shows that the protocol (2.40) does solve the con-
sensus problem.

Theorem 11 For arbitrary switching communication graphs Gσ(t) belonging

to GN , the N agents described by (2.1) reach consensus under the protocol
(2.40) designed by Algorithm 8. Further, the final consensus value is given by
 
x1 (0)
1  
xi (t) → χ(t) , ( 1T ⊗ eAt )  ...  , as t → ∞. (2.42)
N
xN (0)

Proof 10 Consider a common Lyapunov function candidate as follows:

1 T
V1 = ς (IN ⊗ P −1 )ς,
2
where P > 0 is a solution to the LMI (2.18). The time derivative of V1 along
the trajectory of (2.41) can be obtained as

V̇1 = ς T (IN ⊗ P −1 A − cLσ(t) ⊗ P −1 BBP −1 )ς. (2.43)

Because Gσ(t) is connected and (1T ⊗ I)ς = 0, it follows from Lemma 2 that
ς T (Lσ(t) ⊗ I)ς ≥ λmin T
2 ς ς. Since P
−1
BBP −1 ≥ 0, we can further get that

ς T (Lσ(t) ⊗ P −1 BBP −1 )ς ≥ λmin T

2 ς (IN ⊗ P
−1
BBP −1 )ς. (2.44)

Then, it follows from (2.43) and (2.44) that

1 T
V̇1 ≤ ς (IN ⊗ (P −1 A + AT P −1 − cλmin
2 P
−1
BBP −1 )ς
2
< 0,

where we have used that fact that P −1 A + AT P −1 − cλmin2 P

−1
BBP −1 < 0,
which follows readily from (2.18). Therefore, we obtain that the system (2.41)
is asymptotically stable, i.e., the consensus problem is solved. The assertion
that xi (t) → ψ(t) as t → ∞ can be shown by following similar lines in the
proof of Theorem 2, which is omitted here.
46 Cooperative Control of Multi-Agent Systems

Remark 15 An interesting observation is that even though the undirected

communication graphs are switching according to some time sequence, the fi-
nal consensus value is still equal to the case with a fixed undirected graph.
This actually can be explained by noting the fact that χ(t) depends on the left
eigenvector 1 of the Laplacian matrix corresponding to the zero eigenvalue,
which remain the same for all undirected communication graphs.

2.5 Extension to Formation Control

In this section, the consensus protocols proposed in the preceding sections will
be modified to solve formation control problem of multi-agent systems.
Let He = (h1 , h2 , · · · , hN ) ∈ Rn×N describe a constant formation structure
of the agent network in a reference coordinate frame, where hi ∈ Rn is the
formation variable corresponding to agent i. Then, variable hi − hj can be
used to denote the relative formation vector between agents i and j, which
is independent of the reference coordinate. For the agents (2.1), the static
consensus protocol can be modified to get a distributed formation protocol,
described by
N
X
ui = cK aij (xi − xj − hi + hj ), (2.45)
j=1

where the variables are the same as those in (2.2). It should be noted that
(2.45) reduces to the consensus protocol (2.2), when hi − hj = 0, ∀ i, j =
1, · · · , N .

Definition 9 The agents (2.1) under the protocol (2.45) achieve a given for-
mation He = (h1 , h2 , · · · , hN ), if

lim k(xi (t) − hi ) − (xj (t) − hj )k → 0, ∀ i, j = 1, · · · , N. (2.46)

t→∞

Theorem 12 For graph G satisfying Assumption 2.1, the agents (2.1) reach
the formation H e under the protocol (2.45) if all the matrices A + cλi BK, i =
2, · · · , N , are Hurwitz, and Ahi = 0, ∀ i = 1, · · · , N , where λi , i = 2, · · · , N ,
are the nonzero eigenvalues of the Laplacian matrix L.

Proof 11 Let x̃i = xi − hi , i = 1, · · · , N . Then, it follows from (2.1) and

(2.45) that
N
X
x̃˙ i = Ax̃i + c Lij BK x̃j + Ahi , i = 1, · · · , N. (2.47)
j=1

e is achieved if the system (2.47) reaches consensus.

Note that the formation H
Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 47

By following similar steps in the proof of Theorem 1, it is easy to see that

the formation H e is achieved under the protocol (2.45) if all the matrices A +
cλi BK, i = 2, · · · , N , are Hurwitz, and Ahi = 0, ∀ i = 1, · · · , N .

Remark 16 Note that not all kinds of formation structure can be achieved for
the agents (2.1) by using protocol (2.45). The achievable formation structures
have to satisfy the constraints Ahi = 0, ∀ i = 1, · · · , N . Note that hi can be
replaced by hi − h1 , i = 2, · · · , N , in order to be independent of the reference
coordinate, by simply choosing h1 corresponding to agent 1 as the origin. The
formation protocol (2.45) satisfying Theorem 12 can be constructed by using
Algorithms 1 and 2.

The other consensus protocols discussed in the previous sections can also
be modified to solve the formation control problem. For instance, a distribut-
ed formation protocol corresponding to the observer-type consensus protocol
(2.1) can be described as follows:
N
X
v̇i = (A + BF )vi + cL aij [C(vi − vj ) − (yi − yj − C(hi − hj ))],
j=1 (2.48)
ui = F vi ,

where the variables are defined as in (2.21). Similar results can be accordingly
obtained. The only difference is that the constraints Ahi = 0, ∀ i = 1, · · · , N
need to be also satisfied.

Example 7 Consider a network of six double integrators, described by

ẋi = vi ,
ṽ˙ i = ui ,
yi = x i , i = 1, · · · , 6,

where xi ∈ R2 , ṽi ∈ R2 , yi ∈ R2 , and ui ∈ R2 are the position, the velocity,

the measured output, and the acceleration input of agent i, respectively.
The objective is to design a dynamic protocol (2.48) such that the agents
will evolve to form a regular hexagon with edge length 4. In this case, choose
T T √ T √ T
h1 = [ 0 0 0 0 ] , h2 = [ 4 0 0 0 ] , h3 = [ 6 2 3 0 0 ] , h4 = [ 4 4 3 0 0 ] , h5 =
√ T √ T
[ 0 4 3 0 0 ] , h6 = [ −2 2 3 0 0 ] . Take F = [ −1.5 −2.5 ] ⊗ I2 in (2.48) such that
matrix A + BF has eigenvalues −1 and −1.5. By solving LMI (2.32) with
T
α = 1, one obtains L = [ −3.6606 −4.8221 ] ⊗ I2 . The six agents under protocol
(2.48) with F , L given as above, and c = 1 will form a regular hexagon
with a convergence rate larger than 1 for the communication topology given in
FIGURE 2.4. The state trajectories of the six agents are depicted in FIGURE
2.6.

In the following, we will consider the formation keeping problem for satel-
lites moving in the low earth orbit. The translational dynamics of satellites
48 Cooperative Control of Multi-Agent Systems

10

−10

−20

y −30

−40

−50

−60

−70
−25 −20 −15 −10 −5 0 5
x

FIGURE 2.6: The six agents form a hexagon.

in the low earth orbit, different from those of deep-space satellites, cannot be
modeled as double integrators. In order to simplify the analysis, assume that
a virtual reference satellite is moving in a circular orbit of radius R0 . The
relative dynamics of the other satellites with respect to the virtual satellite
will be linearized in the following coordinate system, where the origin is on
the mass center of the virtual satellite, the x-axis is along the velocity vector,
the y-axis is aligned with the position vector, and the z-axis completes the
right-hand coordinate system. The linearized equations of the relative dynam-
ics of the i-th satellite with respect to the virtual satellite are given by the
Hill’s equations [69]:
¨i − 2ω0 ỹ˙ i = uxi ,
x̃
ỹ¨i + 2ω0 x̃˙ i − 3ω02 yi = uyi , (2.49)
z̃¨i + ω02 z̃i = uzi ,
where x̃i , ỹi , z̃i are the position components of the i-th satellite in the rotating
coordinate, uxi , uyi , uzi are control inputs, and ω0 denotes the angular rate of
the virtual satellite. The main assumption inherent in Hill’s equations is that
the distance between the i-th satellite and the virtual satellite is very small
in comparison to the orbital radius R0 .
Denote the position vector by ri = [x̃i , ỹi , z̃i ]T and the control vector by
ui = [uxi , uyi , uzi ]T . Then, (2.49) can be rewritten as
      
ṙi 0 I 3 ri 0
= + u, (2.50)
r̈i A1 A2 ṙi I3 i
where    
0 0 0 0 2ω0 0
A1 = 0 3ω02 0  , A2 = −2ω0 0 0 .
0 0 −ω02 0 0 0
Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 49

The satellites are said to achieve formation keeping, if their velocities con-
verge to the same and their positions maintain a prescribed separation, i.e.,
ri − hi → rj − hj , ṙi → ṙj , as t → ∞, where hi − hj ∈ R3 denotes the desired
constant separation between satellite i and satellite j.
Represent the communication topology among the N satellites by a direct-
ed graph G. Assume that measurements of both relative positions and relative
velocities between neighboring satellites are available. The control input to
satellite i is proposed here as
N
X
ui = −A1 hi + c aij [F1 (ri − hi − rj + hj ) + F2 (ṙi − ṙj )] , (2.51)
j=1

where c > 0, F1 and F2 ∈ R3×3 are the feedback gain matrices to be deter-
mined. If satellite k is a leader, i.e., it does not receive information from any
other satellite, then the term A1 hk is set to zero. With (2.51), the equation
(2.50) can be reformulated as
     N
X   
ṙi 0 I3 ri − hi 0 0 r i − h i − rj + h j
= +c aij . (2.52)
r̈i A1 A2 ṙi F1 F2 ṙi − ṙj
j=1

The following result is a direct consequence of Theorem 12.

Corollary 2 Assume that graph G has a directed spanning tree. Then, the
protocol
0 I3
 (2.51)solves
0 0
the formation keeping problem if and only if the matrices
A1 A 2 + cλ i F1 F2 are Hurwitz for i = 2, · · · , N , where λi , i = 2, · · · , N,
denote the nonzero eigenvalues of the Laplacian matrix of G.

The feedback gain matrices F1 and F2 satisfying Corollary 2 can be de-

signed by following Algorithm 2. Since the system (2.50) is stabilizable, they
always exist.

Example 8 Consider the formation keeping of four satellites with respect to

a virtual satellite which moves in a circular orbit at rate ω0 = 0.001. Assume
that the communication topology is given by FIGURE 2.7, from which it is
observed that satellite 1 plays the leader role. The nonzero eigenvalues of the
Laplacian matrix in this case are 1, 1, 2. Select c in (2.51) to be 1 for simplicity.
We can solve the formation keeping problem in the following steps: i) Select
properly the initial state of the leading satellite 1 such that it moves in a
spatial circular relative orbit with respect to the virtual satellite; ii) Design the
consensus protocol in the form of (2.51) such that the four satellites together
maintain a desired formation flying situation.
Select the initial state of satellite 1 as x̃0 = 0, ỹ0 = 500, z̃0 = 866, x̃˙ 0 = 1,
˙ỹ0 = 0, z̃˙0 = 0. In such a case, satellite 1 maintains a circular relative orbit
with respect to the virtual satellite, of radius r = 1000m, and with the tangent
angle to the orbital place α = 30 deg. Suppose that the four satellites will
50 Cooperative Control of Multi-Agent Systems

1 4

2 3

FIGURE 2.7: The communication topology.

maintain a square shape with the separation of 500m in a plane tangent to

the orbit of the virtual satellite by an angle θ = 60 deg. Let h1 = (100, 100, 0),
h2 = (−100, 100, 0), h3 = (100, 0, 173.21), and h4 = (−100, 10, 173.21). The
feedback gain matrices F1 and F2 in (2.51) can be obtained via Algorithm 2.
The simulation result is presented in FIGURE 2.8.

15
x−axis value 1500
y−axis value
10 z−axis value 1000

500
5
z

0
v

0
−500

−5 −1000
500
1000
0 500
−10
0
−500 −500
y −1000
0 5 10 15 20 25 30 −1000 −1500 x
t

(a) (b)

FIGURE 2.8: (a) The relative velocity; (b) the relative positions, t ∈ [0, 5000]s.
Consensus Control of Linear Multi-Agent Systems: Continuous-Time Case 51

2.6 Notes
The materials of this chapter are mainly based on [83, 86, 91, 198]. The full-
order observer-type consensus protocol (2.33) is adopted from [197], which is
also proposed in [169].
For more results about consensus of general continuous-time linear multi-
agent systems, please refer to [106, 116, 143, 146, 171, 173, 178, 186, 195]
and the references therein. In [106, 116, 171, 173], the authors considered the
case where the relative states of neighboring agents are available. Distributed
dynamic consensus protocols based on the local output information were pro-
posed in [143, 146, 195, 198]. Specially, [195] presented other types of observer-
type protocols except of those in this chapter. The observed-based consensus
protocol in [143] requires the absolute output of each agent but is applicable
to jointly connected switching communication graphs. A low gain technique
was used in [146, 198] to design an output feedback consensus protocol, which
are based on only the relative outputs of neighboring agents. It is worth men-
tioning that the convergence rate of reaching consensus under the protocols
using the low gain technique is generally much lower and a restriction on the
state matrix A is required in [146, 198]. Reduced-order consensus protocols in
[199] do not require the absolute output information, which however are based
on both the relative outputs and inputs of neighboring agents. Necessary and
sufficient conditions were derived in [186] for achieving continuous-time con-
sensus over Markovian switching topologies. The consensus problem of linear
multi-agent systems with uniform constant communication delay was con-
cerned in [178], where an upper bound for delay tolerance was obtained which
explicitly depends on the agent dynamics and the network topology.
This page intentionally left blank
3
Consensus Control of Linear Multi-Agent
Systems: Discrete-Time Case

CONTENTS
3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.2 State Feedback Consensus Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.2.1 Consensus Condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.2.2 Discrete-Time Consensus Region . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.2.3 Consensus Protocol Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.2.3.1 The Special Case with Neutrally Stable Agents . . 60
3.2.3.2 The General Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.3 Observer-Type Consensus Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.3.1 Full-Order Observer-Type Protocol I . . . . . . . . . . . . . . . . . . . . . . . 65
3.3.2 Full-Order Observer-Type Protocol II . . . . . . . . . . . . . . . . . . . . . . 67
3.3.3 Reduced-Order Observer-Based Protocol . . . . . . . . . . . . . . . . . . . 68
3.4 Application to Formation Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.5 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

In this chapter, we consider the consensus of discrete-time linear multi-agent

systems with directed communication topologies, which is the discrete-time
counterpart of the continuous-time consensus in the preceding chapter. The
consensus problem of discrete-time multi-agent systems is formulated in Sec-
tion 3.1. The communication graph among the agents is assumed to contain
a directed spanning tree throughout this chapter. Different from the previous
chapter where the Laplacian matrix is used to represent the communication
graph, the stochastic matrix will be used instead in this chapter.
Based on the relative states of neighboring agents, a distributed static
consensus protocol is proposed in Section 3.1. The consensus problem of the
discrete-time multi-agent system under the proposed static protocol is cast in-
to the stability of a set of matrices with the same dimension as a single agent.
The final consensus value reached by the agents is also derived. Inspired by
the continuous-time consensus region in Chapter 2, the notion of discrete-time
consensus region is introduced and analyzed. It is pointed out via numerical
examples that the consensus protocol should have a reasonably large bounded
consensus region so as to be robust to variations of the communication topolo-
gy. For the special case where the state matrix of each agent is neutrally stable,

53
54 Cooperative Control of Multi-Agent Systems

it is shown that there exists a static protocol with a bounded consensus region
in the form of an open unit disk if each agent is stabilizable, implying that
such static protocol can achieve consensus with respect to all communication
topologies containing a directed spanning tree. For the general case where the
state matrix might be unstable, an algorithm is proposed to construct a pro-
tocol with the origin-centered disk of radius δ (0 < δ < 1) as its consensus
region. It is worth noting that δ has to further satisfy a constraint relying on
the unstable eigenvalues of the state matrix for the case where each agent has
a least one eigenvalue outside the unit circle, which shows that the consensus
problem of the discrete-time multi-agent systems is generally more difficult to
solve, compared to the continuous-time case in the previous chapter.
Based on the relative outputs of neighboring agents, distributed full-order
and reduced-order observer-type consensus protocols are proposed in Section
3.3. The Separation Principle of the traditional observer-based controllers still
holds in the discrete-time multi-agent setting presented in this paper. Algo-
rithms are presented to design these observer-type protocols. In Section 3.4,
the consensus protocols are modified to solve formation control problem for
discrete-time multi-agent systems. The main differences of the discrete-time
consensus in this chapter and the continuous-time consensus in the previous
chapter are further highlighted in Section 3.5.

3.1 Problem Statement

Consider a network of N identical agents with general discrete-time linear
dynamics, where the dynamics of the i-th agent are described by
x+
i = Axi + Bui ,
(3.1)
yi = Cxi , i = 1, · · · , N,
where xi = xi (k) ∈ Rn is the state, x+i = xi (k + 1) is the state at the next
time instant, ui ∈ Rp is the control input, yi ∈ Rq is the measured output,
and A, B, C are constant matrices with compatible dimensions.
The communication topology among the N agents is represented by a di-
rected graph G, which is assumed to satisfy the following assumption through-
out this chapter.
Assumption 3.1 The communication graph G contains a directed spanning
tree.
The objective of this chapter is to solve the consensus problem for the
N agents described by (3.1), i.e., to design distributed consensus protocols
using only local information to ensure that the N agents described by (3.1)
can achieve consensus in the sense of limk→∞ kxi (k) − xj (k)k = 0, ∀ i, j =
1, · · · , N .
Consensus Control of Linear Multi-Agent Systems: Discrete-Time Case 55

3.2 State Feedback Consensus Protocols

In this section, we consider the case where each agent can have access to its
own state and the states from its neighbors. A distributed static consensus
protocol in this case can be proposed as
N
X
ui = K dij (xi − xj ), (3.2)
j=1

where K ∈ Rp×n is the feedback gain matrix and dij is the (i, j)-th entry of
the row-stochastic matrix D associated with the graph G. Note that different
from the Laplacian matrix used in the last chapter, the stochastic matrix will
be used to characterize the communication graph in this chapter.
Let x = [xT1 , · · · , xTN ]T . Using (3.2) for (3.1), it is not difficult to get that
the closed-loop network dynamics can be written as

x+ = [IN ⊗ A + (IN − D) ⊗ BK]x. (3.3)

By Lemma 7, 1 is an eigenvalue of D. Let r ∈ RN be the left eigenvector

of IN − D associated with the eigenvalue 0, satisfying rT 1 = 1. Introduce
ξ ∈ RN n×N n by
ξ = [(IN − 1rT ) ⊗ In ]x, (3.4)
which satisfies (rT ⊗ In )ξ = 0. By the definition of r, it is not difficult to see
that 0 is a simple eigenvalue of IN − 1rT with 1 as its right eigenvector and 1
is another eigenvalue with multiplicity N − 1. Thus, it follows from (3.4) that
ξ = 0 if and only if x1 = x2 = · · · = xN , i.e., the consensus problem of (3.3)
can be cast into the Schur stability of vector ξ, which evolves according to the
following dynamics:

ξ + = [IN ⊗ A + (I − D) ⊗ BK]ξ. (3.5)

3.2.1 Consensus Condition

The following theorem presents a necessary and sufficient condition for solving
the consensus problem of (3.3).

Theorem 13 Supposing Assumption 3.1 holds, the agents in (3.1) reach

consensus under the static protocol (3.2) if and only if all the matrices
A + (1 − λ̂i )BK, i = 2, · · · , N , are Schur stable, where λ̂i , i = 2, · · · , N ,
denote the eigenvalues of D located in the open unit disk.

Proof 12 Because the graph G satisfies Assumption 3.1, it follows from Lem-
ma 7 that 0 is a simple eigenvalue of IN − D and the other eigenvalues lie in
56 Cooperative Control of Multi-Agent Systems

the open unit disk centered at 1+ι0 in the complex plane. Let Y1 ∈ CN ×(N −1) ,
Y2 ∈ C(N −1)×N , T ∈ RN ×N , and upper-triangular ∆ ∈ C(N −1)×(N −1) be such
that
 T  
  r 0 0
T = 1 Y1 , T −1 = , T −1 (IN − D)T = J = , (3.6)
Y2 0 ∆

where the diagonal entries of ∆ are the nonzero eigenvalues of IN − D. Let

ζ = (T −1 ⊗In )ξ with ζ = [ζ1T , · · · , ζN
T T
] . Then, (3.5) can be rewritten in terms
of ζ as
ζ + = [IN ⊗ A + (I − J) ⊗ BK]ζ, (3.7)
where ζ1 = (rT ⊗ In )ξ ≡ 0, which follows directly from the definition of ξ
in (3.4). Note that the elements of the state matrix of (3.7) are either block
diagonal or block upper-triangular. Hence, ζi , i = 2, · · · , N , converge asymp-
totically to zero if and only if the N − 1 subsystems along the diagonal, i.e.,

ζi+ = [A + (1 − λ̂i )BK]ζi , i = 2, · · · , N, (3.8)

are Schur stable. Therefore, the Schur stability of the matrices A+(1− λ̂i )BK,
i = 2, · · · , N , is equivalent to that the state ζ of (3.7) converges asymptotically
to zero, implying that consensus is achieved.

Remark 17 The above theorem is a discrete-time counterpart of Theorem 1

in the last chapter. As mentioned earlier, the main difference is that the Lapla-
cian matrix is used in the last chapter while the stochastic matrix is used here
to characterize the communication topology. The effects of the communication
topology on consensus are characterized by the eigenvalues of the stochastic
matrix D except the eigenvalue one.

Theorem 14 Consider the multi-agent network (3.3) whose communication

topology G satisfying Assumption 3.1. If the protocol (3.2) satisfies Theorem
13, then
 
x1 (0)
 
xi (k + 1) → ̟(k) , (rT ⊗ Ak )  ...  , i = 1, · · · , N, as k → ∞,
xN (0)
(3.9)
where r ∈ RN satisfies rT (IN − D) = 0 and rT 1 = 1.

Proof 13 The solution of (3.3) can be obtained as

k
x(k + 1) = (IN ⊗ A + (IN − D) ⊗ BK) x(0)
= (T ⊗ I)(IN ⊗ A + J ⊗ BK)k (T −1 ⊗ I)x(0)
 k 
A 0
= (T ⊗ I) (T −1 ⊗ I)x(0),
0 (IN −1 ⊗ A + ∆ ⊗ BK)k
Consensus Control of Linear Multi-Agent Systems: Discrete-Time Case 57

where matrices T , J and ∆ are defined in (3.6). By Theorem 13, IN −1 ⊗ A +

∆ ⊗ BK is Schur stable. Thus,

x(k + 1) → (1 ⊗ I)Ak (rT ⊗ I)x(0)

= (1rT ) ⊗ Ak x(0) as k → ∞,

implying that
xi (k) → ̟(k) as k → ∞, i = 1, · · · , N.

Remark 18 Some observations on the final consensus value ̟(k) in (3.9)

can be concluded as follows. If the state matrix A in (3.1) is Schur stable,
then ̟(k) → 0 as k → ∞, i.e., the discrete-time consensus problem becomes
trivial. If A is unstable (having eigenvalues located outside the open unit cir-
cle), then ̟(k) will tend to infinity exponentially. On the other hand, if A
has eigenvalues in the closed unit circle, then the agents in (3.1) may reach
consensus nontrivially.

3.2.2 Discrete-Time Consensus Region

To facilitate the design of the consensus protocol and elucidate the robustness
of the protocol with respect to the communication topology, we introduce the
following discrete-time consensus region.

Definition 10 The region S of the parameter σ ∈ C, such that matrix A +

(1 − σ)BK is Schur stable, is called the discrete-time consensus region of the
network (3.3).

The above definition presents a discrete-time counterpart of the

continuous-time consensus region introduced in the previous chapter. The
following result is a direct consequence of Theorem 13.

Corollary 3 The agents in (3.1) reach consensus under the static protocol
(3.2) if λ̂i ∈ S, i = 2, · · · , N , where λ̂i , i = 2, · · · , N , are the eigenvalues of
D located in the open unit disk.

For an undirected communication graph, the consensus region of the net-

work (3.3) is a bounded interval or a union of several intervals on the real
axis. However, for a directed graph where the eigenvalues of D are generally
complex numbers, the consensus region S is either a bounded region or a set
of several disconnected regions in the complex plane. Due to the fact that the
eigenvalues of the row-stochastic matrix D lie in the unit disk, unbounded
consensus regions, desirable for consensus in the continuous-time setting as
in the previous chapter, generally do not exist for the discrete-time consensus
considered here.
The following example has a disconnected consensus region.
58 Cooperative Control of Multi-Agent Systems

Example 9 The agent dynamics and the consensus protocol are given by
(3.1) and (3.2), respectively, with
     
0 1 1 0 0 −1
A= , B= , K= .
−1 1.02 0 1 1 0
For simplicity in illustration, assume that the communication graph G is undi-
rected here. Then, the consensus region is a set of intervals on the real axis.
The characteristic equation of A + (1 − σ)BK is

det(zI − A − (1 − σ)BK) = z 2 − 1.02z + σ 2 = 0. (3.10)

s+1
Applying bilinear transformation z = s−1 to (3.10) gives

(σ 2 − 0.02)s2 + (1 − σ 2 )s + 2.02 + σ 2 = 0. (3.11)

It is well known that, under the bilinear transformation, (3.10) has all roots
within the unit disk if and only if the roots of (3.11) lie in the open left-half
plane (LHP). According to the Hurwitz criterion [23], (3.11) has all roots in
the open LHP if and only if 0.02 < σ 2 < 1. Therefore, the consensus region
in this case is S = (−1, −0.1414) ∪ (0.1414, 1), a union of two disconnected
intervals, which can be obtained from the plot of the eigenvalues of A + (1 −
σ)BK with respect to σ as depicted in FIGURE 3.1.

1.1
radius of A+(1−σ)BK

0.9

0.8

0.7

0.6

0.5
−1 −0.5 0 0.5 1
σ

FIGURE 3.1: A disconnected discrete-time consensus region.

For the communication graph shown in FIGURE 3.1, the corresponding

row-stochastic matrix is
 
0.3 0.2 0.2 0.2 0 0.1
0.2 0.6 0.2 0 0 0
 
0.2 0.2 0.6 0 0 0
D= 0.2 0
,
 0 0.4 0.4 0 
0 0 0 0.4 0.2 0.4
0.1 0 0 0 0.4 0.5
whose eigenvalues, other than 1, are −0.2935, 0.164, 0.4, 0.4624, 0.868, which
Consensus Control of Linear Multi-Agent Systems: Discrete-Time Case 59

1 6

2 5

3 4

FIGURE 3.2: An undirected communication topology.

all belong to S. Thus, it follows from Corollary 3 that the network (3.3) with
graph given in FIGURE 3.2 can achieve consensus.
Let us see how modifications of the communication topology affect the con-
sensus. Consider the following two simple cases:
1) An edge is added between nodes 1 and 5, thus more information exchange
will exist inside the network. Then, the row-stochastic matrix D becomes
 
0.2 0.2 0.2 0.2 0.1 0.1
0.2 0.6 0.2 0 0 0
 
0.2 0.2 0.6 0 0 0
 ,
0.2 0 0 0.4 0.4 0 
 
0.1 0 0 0.4 0.2 0.3
0.1 0 0 0 0.4 0.5
whose eigenvalues, in addition to 1, are −0.2346, 0.0352, 0.4, 0.4634, 0.836.
Clearly, the eigenvalue 0.0352 does not belong to S, i.e., consensus can not be
achieved in this case.
2) The edge between nodes 5 and 6 is removed. The row-stochastic matrix
D becomes  
0.3 0.2 0.2 0.2 0 0.1
0.2 0.6 0.2 0 0 0
 
0.2 0.2 0.6 0 0 0
 ,
0.2 0 0 0.4 0.4 0 
 
0.1 0 0 0.4 0.6 0 
0.1 0 0 0 0 0.9
whose eigenvalues, other than 1, are −0.0315, 0.2587, 0.4, 0.8676, 0.9052. In
this case, the eigenvalue −0.0315 does not belong to S, i.e., consensus can not
be achieved either.
These sample cases imply that, for disconnected consensus regions, con-
sensus can be quite fragile to the variations of the network’s communication
topology. Hence, the consensus protocol should be designed to have a suffi-
ciently large bounded consensus region in order to be robust with respect to the
communication topology, which is the topic of the following subsection.
 60 Cooperative Control of Multi-Agent Systems

3.2.3 Consensus Protocol Design

3.2.3.1 The Special Case with Neutrally Stable Agents
First we consider a special case where the matrix A is neutrally stable. Before
moving forward, we introduce the following lemma.

Lemma 25 ([200]) For a matrix Q = QH ∈ Cn×n , consider the following

Lyapunov equation:
AH XA − X + Q = 0.
If X > 0, Q ≥ 0, and (Q, V ) is observable, then the matrix A is Schur stable.

Proposition 4 For matrices Q ∈ Rn×n , V ∈ Rm×n , σ ∈ C, where Q is

orthogonal, V V T = I, and (Q, V ) is observable, if |σ| < 1, then the matrix
Q − (1 − σ)QV T V is Schur stable.

Proof 14 Observe that

[Q − (1 − σ)QV T V ]H [Q − (1 − σ)QV T V ] − I
= QT Q − (1 − σ)QT QV T V − (1 − σ̄)V T V QT Q
+ |1 − σ|2 V T V QT QV T V − I (3.12)
2 T
= (−2Re(1 − σ) + |1 − σ| )V V
= (|σ|2 − 1)V T V.

Since (Q, V ) is observable, it is easy to verify that (Q − (1 − σ)QV T V, V T V )

is also observable. Then, by Lemma 25, (3.12) implies that Q − (1 − σ)QV T V
is Schur stable for any |σ| < 1.

Next, an algorithm for protocol (3.1) is presented, which will be used later.

Algorithm 9 Given that A is neutrally stable and that (A, B) is stabilizable,

the protocol (3.1) can be constructed as follows:
1) Choose U ∈ Rn1 ×n and W ∈ R(n−n1 )×n such that 1

   −1  
U U M 0
A = (3.13)
W W 0 X

where M ∈ Rn1 ×n1 is orthogonal and X ∈ R(n−n1 )×(n−n1 ) is Schur stable

(assume that without loss of generality U T B has full column rank).

2) Choose V ∈ Rp×n1 such that V V T = Ip and range(V T ) = range(U B).

3) Define K = −(V U B)−1 V M U .
1 Matrices U and W can be derived by transforming matrix A into the real Jordan

canonical form [58].

Consensus Control of Linear Multi-Agent Systems: Discrete-Time Case 61

Theorem 15 Suppose that A is neutrally stable and that (A, B) is stabiliz-

able. The protocol (3.2) constructed via Algorithm 9 has the open unit disk
as its bounded consensus region. Thus, such a protocol solves the consensus
problem for (3.1) with respect to ΓN (which denotes the set of all the commu-
nication topologies containing a directed spanning tree).
Proof 15 Let the related variables be defined as in Algorithm 9. By step 2)
in Algorithm 9, it is easy to see that V T V is an orthogonal projection onto
range(V T ) = range(U B). By the definition of orthogonal projection (Defi-
nition 4 in Chapter 1), we can get that V T V U B = U B. By the fact that
range(V T ) = range(U B), it is not difficult to get that B T U T V T is invertible.
Thus, we have V = (B T U T V T )−1 B T U T and U BK = −V T V M U . Also, the
stablizability of (A, B) implies that (M, V ) is controllable. Let U † ∈ Rn×n1 and
W † ∈ Rn×(n−n1 ) be such that [ U † W † ] = [ WU ]−1 , where U U † = I, W W † = I,
† †
W U = 0, and U W = 0. It can be verified by some algebraic manipulations
that
   −1
U U
[A + (1 − σ)BK]
W W
 
M + (1 − σ)U BKU † (1 − σ)U BKW †
= (3.14)
(1 − σ)W BKU † X + (1 − σ)W BKW †
 
M − (1 − σ)V T V M −(1 − σ)U † BKW
= .
0 X
By Lemma 25, matrix M − (1 − σ)V T V M is Schur stable for any |σ| <
1. Hence, by Proposition 4, (3.14) implies that matrix A + (1 − σ)BK with
K given by Algorithm 9 is Schur stable for any |σ| < 1, i.e., the protocol
(3.2) constructed via Algorithm 9 has a bounded consensus region in the form
of the open unit disk. Since the eigenvalues of any communication topology
containing a spanning tree lie in the open unit disk, except eigenvalue 1, it
follows from Corollary 3 that this protocol solves the consensus problem with
respect to ΓN .
Remark 19 It should be mentioned that Algorithm 9 and the proof of Theo-
rem 15 are partly inspired by [172]. Here we use a different yet comparatively
much simpler method to derive Theorem 15.
Example 10 Consider a network of agents described by (3.1), with
   
0.2 0.6 0 0
A = −1.4 0.8 0  , B = 1 .
0.7 0.2 −0.5 0
The eigenvalues of matrix A are −0.5, 0.5 ± ι0.866, thus A is neutrally
stable. By transforming A into its real Jordan canonical form, we can get that
the matrices
 
0.1709 −0.4935 0  
U= , W = −0.0570 −0.2961 1
0.7977 0 0
 62 Cooperative Control of Multi-Agent Systems
 0.5 0.866 
satisfy (3.13) with M = −0.866 0.5 and X = −0.5. Thus, U B =
T
[ −0.4935 0 ] . Thus, we can take V = [ 1 0 ] such that V V T = 1 and
range(V T ) = range(U B). Then, by Algorithm 9, we obtain that K =
[ 1.5732 −0.5 0 ]. In light of Theorem 15, the agents considered in this exam-
ple will reach consensus under the protocol (3.2) with K given as above, with
respect to all the communication topologies containing a directed spanning tree.

3.2.3.2 The General Case

In this subsection, we consider the general case without assuming that A is
neutrally stable. Before moving forward, we introduce the following modified
algebraic Riccati equation (MARE) [152, 144]:
P = AT P A − (1 − δ 2 )AT P B(B T P B + I)−1 B T P A + Q, (3.15)
where P ≥ 0, Q > 0, and δ ∈ R. For δ = 0, the MARE (3.15) is reduced to
the commonly-used discrete-time Riccati equation discussed in, e.g., [200].
The following lemma concerns the existence of solutions for the MARE.
Lemma 26 ([152, 144]) Let (A, B) be stablizable. Then, the following state-
ments hold.
a) Suppose that the matrix A has no eigenvalues with magnitude larger than 1,
Then, the MARE (3.15) has a unique positive-definite solution P for any
0 < δ < 1.
b) For the case where A has a least one eigenvalue with magnitude larger than
1 and the rank of B is one, the MARE (3.15) has a unique positive-definite
solution P , if 0 < δ < Π|λu1(A)| , where λui (A) denote the unstable eigenvalues
i i

of A.
c) If the MARE (3.15) has a unique positive-definite solution P , then P =
limk→∞ Pk for any initial condition P0 ≥ 0, where Pk satisfies
P (k + 1) = AT P (k)A − δAT P (k)B(B T P (k)B + I)−1 B T P (k)A + Q.

Theorem 16 Let (A, B) be stabilizable. The feedback gain matrix K of (3.2)

is given by K = −(B T P B + I)−1 B T P A, where P > 0 is the unique solu-
tion of the MARE (3.15). Then, the protocol (3.2) designed as above has a
bounded consensus region in the form of an origin-centered disk of radius δ,
i.e., this protocol solves the consensus problem for networks with agents (3.1)
with respect to Γ≤δ (which denotes the set of all directed graphs containing a
directed spanning tree, whose non-one eigenvalues lie in the disk of radius δ
centered at the origin), where δ satisfies 0 < δ < 1 for the case where A has
no eigenvalues with magnitude larger than 1 and satisfies 0 < δ < Π|λu1(A)|
i i

for the case where A has a least one eigenvalue outside the unit circle and B
is of rank one.
Consensus Control of Linear Multi-Agent Systems: Discrete-Time Case 63

Proof 16 Observe that

[A + (1 − δ)BK]H P [A + (1 − δ)BK] − P
= AT P A − 2Re(1 − δ)AT P B(B T P B + I)−1 B T P A − P
+ |1 − δ|2 AT P B(B T P B + I)−1 B T P B(B T P B + I)−1 B T P A
= AT P A + [−2Re(1 − δ) + |1 − δ|2 ]AT P B(B T P B + I)−1 B T P A − P
+ |1 − δ|2 AT P B(B T P B + I)−1 [−I + B T P B(B T P B + I)−1 ]B T P A
= AT P A + (|δ|2 − 1)AT P B(B T P B + I)−1 B T P A − P
− |1 − δ|2 AT P B(B T P B + I)−2 B T P A
≤ AT P A − (1 − |δ|2 )AT P B(B T P B + I)−1 B T P A − P
= −Q < 0,
(3.16)
where the identity −I + B T P B(B T P B + I)−1 = −(B T P B + I)−1 has been
applied. Then, the assertion follows directly from (3.16), Lemma 26, and the
discrete-time Lyapunov inequality.

Remark 20 Note that Γ≤δ for the general case is a subset of ΓN for the
special case where A is neutrally stable. This is consistent with the intuition
that unstable behaviors are more difficult to synchronize than the neutrally
stable ones. By Lemma 26, it follows that a sufficient and necessary condition
for the existence of the consensus protocol (3.2) is that (A, B) is stabilizable for
the case where A has no eigenvalues with magnitude larger than 1. In contrast,
δ has to further satisfy δ < Π|λu1(A)| for the case where A has at least eigenvalue
i i

outside the unit circle and B is of rank one. This implies that contrary to
the continuous-time case in the previous chapter, both the eigenvalues of the
communication graph and the unstable eigenvalues of the agent dynamics are
critical for the design of the consensus protocol. In other words, the consensus
problem of discrete-time multi-agent systems in this chapter is generally more
challenging to solve.

Remark 21 For the case where B is of full column rank, the feedback
gain matrix K of (3.2) can be chosen as K = −(B T P B)−1 B T P A, where
P > 0 is the unique solution to this simplified MARE: P = AT P A − (1 −
δ 2 )AT P B(B T P B)−1 B T P A + Q.

Example 11 Let the agents in (3.1) be discrete-time double integrators, with

     
xi1 1 1 0
xi = , A= , B= .
xi2 0 1 1

Because the spectral radius of A is 1, we know from Lemma 26 that the

MARE 3.15 is solvable for any 0 < δ < 1. Solving the MARE (3.15) with
δ = 0.95 and Q = 3I gives P = 104 × [ 0.0062 0.0602
0.0602 1.1780 ]. Then we can obtain
64 Cooperative Control of Multi-Agent Systems

1 6

2 5

3 4

FIGURE 3.3: A directed communication topology.

that K = [ −0.0511 −1.0510 ]. It follows from Theorem 16 that the agents (3.1)
reach consensus under the protocol (3.2) with K given as above with respect
to Γ≤0.95 . Assume that the communication topology G is given as in FIGURE
3.3, and the corresponding row-stochastic matrix is
 
0.4 0 0 0.1 0.3 0.2
0.5 0.5 0 0 0 0
 
0.3 0.2 0.5 0 0 0
D=  ,
0.5 0 0 0.5 0 0
0 0 0 0.4 0.4 0.2
0 0 0 0 0.3 0.7
whose eigenvalues, other than 1, are λi = 0.5, 0.5565, 0.2217±ι0.2531. Clearly,
|λi | < 0.95, for i = 2, · · · , 6. FIGURE 3.4 depicts the state trajectories of
the discrete-time double integrators, which shows that consensus is actually
achieved.

90 1.4

80 1.2

70
1
60
0.8
50
x (k)

x (k)

0.6
40
i1

i2

0.4
30
0.2
20

10 0

0 −0.2

−10 −0.4
0 20 40 60 80 100 0 20 40 60 80 100
k k

FIGURE 3.4: The state trajectories of a double-integrator network.

Consensus Control of Linear Multi-Agent Systems: Discrete-Time Case 65

3.3 Observer-Type Consensus Protocols

In the previous section, the consensus protocol (3.2) rely on the relative states
of neighboring agents. However, in many circumstances the state information
might not be available. In this section, we will consider the consensus problem
of (3.1) for the case where each agent can have access to the local output
information, i.e., its own output and the outputs from its neighbors.

3.3.1 Full-Order Observer-Type Protocol I

Based on the relative output information of neighboring agents, we propose a
full-order observer-type consensus protocol as follows:
N
X
vi+ = (A + BF )vi + L dij [C(vi − vj ) − (yi − yj )],
j=1 (3.17)
ui = F v i ,

where vi ∈ Rn is the protocol state, i = 1, · · · , N , L ∈ Rq×n and F ∈

Rp×n are the feedback gain matrices to be determined. In (3.17), the term
PN
j=1 dij C(vi − vj ) denotes the information exchanges between the protocol
of agent i and those of its neighboring agents.
T T
Let zi = [xTi , viT ]T and z = [z1T , · · · , zN ] . Then, the closed-loop system
resulting from (3.1) and (3.17) can be written as

z + = [IN ⊗ A + (IN − D) ⊗ H]z, (3.18)

where    
A BF 0 0
A= , H= .
0 A + BF −LC LC

Theorem 17 For any G satisfying Assumption 3.1, the agents in (3.1) reach
consensus under the observer-type protocol (3.17) if all the matrices A + BK
and A + (1 − λ̂i )LC, i = 2, · · · , N , are Schur stable, where λ̂i , i = 2, · · · , N ,
denote the eigenvalues of D located in the open unit disk. Furthermore, the
final consensus value is given by

xi (k + 1) → ̟(k), vi (k + 1) → 0, i = 1, · · · , N, as k → ∞, (3.19)

where ̟(k) is defined as in (3.9).

Proof 17 By following similar steps in the proof of Theorem 13, we can derive
that the consensus problem of (3.18) can be reduced to the asymptotical stability
problem of the following systems:

ζi+ = [A + (1 − λ̂i )H]ζi , i = 2, · · · , N. (3.20)

 66 Cooperative Control of Multi-Agent Systems
h i
λ̂i )LC 0
It is easy to verify that matrices A + λ̂i H are similar to A+(1−
−(1−λ̂i )LC A+BF
,
i = 2, · · · , N. Therefore, the Schur stability of the matrices A + BK and
A + (1 − λ̂i )LC, i = 2, · · · , N , is equivalent to the asymptotical stability of
(3.20), implying the solvability of the consensus problem of (3.18). The final
consensus value of (3.18) can also be derived by following the steps in the proof
of Theorem 14, which is thus omitted here for brevity.

For the observer-type protocol (3.17), the consensus region can be similarly
defined as in Definition 10. Assuming that A + BF is Schur stable, the region
of the parameter σ ∈ C such that A + (1 − σ)LC is Schur stable, is the
discrete-time consensus region of the network (3.18).
In the following, we shall design the observer-type protocol (3.17). It is
noted that the design of A+(1− λ̂i )LC is dual to the design of A+(1− λ̂i )BK,
the latter of which has been discussed in the preceding section. First, for the
special case where A is neutrally stable, we can modify Algorithm 9 to get the
following.

Algorithm 10 Given that A is neutrally stable and that (A, B, C) is stabi-

lizable and detectable, the protocol (3.17) can be constructed as follows:
1) Select F be such that A + BF is Schur stable.
2) Choose Û ∈ Rn×n1 and Ŵ ∈ Rn×(n−n1 ) , satisfying
 
 −1   M̂ 0
U W A U W = ,
0 X̂

where M̂ ∈ Rn1 ×n1 is orthogonal and X̂ ∈ R(n−n1 )×(n−n1 ) is Schur stable.

3) Choose V̂ ∈ Rq×n1 such that V V T = Iq and range(V̂ T ) = range(Û T C T ).

4) Define L = −Û M̂ V̂ T (C Û V̂ T )−1 .

Theorem 18 Suppose that matrix A is neutrally stable and that (A, B, C) is

stabilizable and detectable. The protocol (3.17) constructed via Algorithm 10
has the open unit disk as its bounded consensus region. Thus, such a protocol
solves the consensus problem for (3.1) with respect to ΓN .

Proof 18 The proof can be completed by following similar steps in proving

Theorem 15.

Next, consider the general case without assumptions on A.

Theorem 19 Assume that (A, B, C) is stabilizable and detectable. The feed-

back gain matrices of (3.17) is designed as F satisfying that A + BF is Schur
stable and L = −AP C T (CP C T + I)−1 , where P > 0 is the unique solution of
the following MARE:

P = AP AT − (1 − δ 2 )AP C T (CP C T + I)−1 CP AT + Q,

Consensus Control of Linear Multi-Agent Systems: Discrete-Time Case 67

where Q > 0 and δ ∈ R. Then, the protocol (3.17) designed as above has a
bounded consensus region in the form of an origin-centered disk of radius δ,
i.e., this protocol solves the consensus problem for networks with agents (3.1)
with respect to Γ≤δ , where δ satisfies 0 < δ < 1 for the case where A has no
eigenvalues with magnitude larger than 1 and satisfies 0 < δ < Π|λu1(A)| for
i i

the case where A has a least one eigenvalue outside the unit circle and C is
of rank one.
Proof 19 The proof of this theorem can be completed by following similar
steps as in the proof of Theorem 16.

3.3.2 Full-Order Observer-Type Protocol II

In this subsection, we will propose another alternative dynamic consensus
protocol. Based on the relative output information of neighboring agents, we
propose the following full-order observer-type protocol:
N
X
v̄i+ = (A + L̄C)v̄i + dij [B F̄ (vi − vj ) + L̄(yi − yj )],
j=1 (3.21)
ui = F̄ vi , i = 1, · · · , N,

where v̄i ∈ Rn , L̄
P∈N R
n×q
and F̄ ∈ Rp×n are the feedback gain matrices.
Let ei = v̄i − j=1 aij (xi −xj ), i = 1, · · · , N . Then, we can get from (3.21)
that ei satisfy the following dynamics:
e+
i = (A + L̄C)ei , i = 1, · · · , N.
Clearly, if L̄ P
is chosen such that A+ L̄C is Schur stable, then v̄i asymptotically
N
converge to j=1 aij (xi − xj ). Let e = [eT1 , · · · , eTN ]T and x = [xT1 , · · · , xTN ]T .
Then, we can get from (3.1) and (3.21) that the closed-loop network dynamics
can be written in terms of e and x as
 +   
x A + cL ⊗ B F̄ cB F̄ x
= .
e+ 0 A + L̄C e
By following similar steps in showing Theorem 17, we obtain the following
result.
Theorem 20 For any G satisfying Assumption 3.1, the agents in (3.1) reach
consensus under the observer-type protocol (3.21) if all the matrices A + L̄C
and A + (1 − λ̂i )B F̄ , i = 2, · · · , N , are Schur stable, where λ̂i , i = 2, · · · , N ,
denote the eigenvalues of D located in the open unit disk.
Note that the design of F̄ and L̄ in Theorem 20 is actually dual to the
design of L and F in Theorem 17. The algorithms in the previous subsection
can be modified to determine the parameters F̄ and L̄ of the dynamic protocol
(3.21). The details are omitted here for conciseness.
68 Cooperative Control of Multi-Agent Systems

3.3.3 Reduced-Order Observer-Based Protocol

The observer-type protocol (3.17) in the last subsection is of full order, which
however possesses a certain degree of redundancy, which stems from the fact
that while the observer constructs an “estimate” of the entire state, part of the
state information is already reflected in the system outputs. In this subsection,
we will design a reduced-order protocol so as to eliminate this redundancy.
Based on the relative output information of neighboring agents, we intro-
duce here a reduced-order observer-based consensus protocol as

v̂i+ = F v̂i + Gyi + T Bui ,

N
X N
X (3.22)
ui = KQ1 dij (yi − yj ) + KQ2 dij (v̂i − v̂j ), i = 1, · · · , N,
j=1 j=1

where v̂i ∈ Rn−q is the protocol state, F ∈ R(n−q)×(n−q) is Schur stable and
has no eigenvalues in common with those of A, G ∈ R(n−q)×q , T ∈ R(n−q)×n
is the unique solution to (2.36), satisfying that [ C T ] is nonsingular, [ 1 2 ] =
Q Q
C −1 p×n
[T ] , K ∈ R is the feedback gain matrix to be designed, and dij is the
(i, j)-th entry of the row-stochastic matrix D associated with the graph G.
T T
Let ẑi = [xTi , v̂iT ]T and ẑ = [ẑ1T , · · · , ẑN ] . Then, the collective network
dynamics can be written as

ẑ + = [IN ⊗ M + (IN − D) ⊗ R]ẑ, (3.23)

where    
A 0 BKQ1 C BKQ2
M= , R= .
GC F T BKQ1 C T BKQ2

Theorem 21 For any G satisfying Assumption 3.1, the agents in (3.1) reach
consensus under the observer-type protocol (3.17) if the matrices F and A +
(1− λ̂i )BK, i = 2, · · · , N , are Schur stable, where λ̂i , i = 2, · · · , N , denote the
eigenvalues of D located in the open unit disk. Moreover, the final consensus
value is given by

xi (k + 1) → ̟(k), vi (k + 1) → T ̟(k), i = 1, · · · , N, as k → ∞,

where ̟(k) is defined as in (3.9).

Proof 20 By following similar steps as in the proof of Theorem 13, we can

reduce the consensus problem of (2.37) to the simultaneous asymptotically
stability problem of the following N − 1 systems:

ζ̃i+ = [M + (1 − λ̂i )R]ζ̃i , i = 2, · · · , N, (3.24)

where λ̂i , i = 2, · · · , N , denote the eigenvalues of D located in the open

unit disk. It is not difficult to verify that M + (1 − λ̂i )R is similar to
Consensus Control of Linear Multi-Agent Systems: Discrete-Time Case 69
 A+cλ BK cλ BKQ 
0
i i
F
2
. Therefore, the consensus problem of (2.37) is reached
if F and A + cλi BK, i = 2, · · · , N are Schur stable. The rest of the proof can
be completed by following similar steps as in the proof of Theorem 14. The
details are omitted here for brevity.

The feedback gain matrix K satisfying Theorem (21) can be constructed

by using Algorithms 9 or Theorem 19.

3.4 Application to Formation Control

Similarly to the continuous-time case in Chapter 2, the consensus protocols
proposed in the previous sections can also be modified to solve formation
control problem of discrete-time multi-agent systems. For illustration, we only
consider the observer-type protocol (3.17).
Let H e = (h1 , · · · , hN ) ∈ Rn×N describe a constant formation structure
of the agent network in a reference coordinate frame, where hi ∈ Rn is the
T
formation variable corresponding to agent i. For example, h1 = [ 0 0 ] , h2 =
T T T
[ 0 1 ] , h3 = [ 1 0 ] , and h4 = [ 1 1 ] represent a unit square. Variable hi −
hj denotes the relative formation vector between agents i and j, which is
independent of the reference coordinate.
Given a formation structure H, e the observer-type protocol (3.17) can be
modified into the following form:
N
X
vi+ = (A + BF )vi + dij L[C(vi − vj ) − (yi − yj − C(hi − hj ))],
j=1 (3.25)
ui = F v i .

It should be noted that (3.25) reduces to the consensus protocol (3.17), when
hi − hj = 0, ∀ i, j = 1, · · · , N .

Definition 11 The agents (3.1) under the protocol (3.25) achieve a given
e = (h1 , · · · , hN ) if
formation H

kxi (k) − hi − xj (k) + hj k → 0, as k → ∞, ∀ i, j = 1, · · · , N.

Theorem 22 For any G satisfying Assumption 3.1, the agents in (3.1) reach
the formation H e under the protocol (3.25) if all the matrices A + BF and
A + (1 − λ̂i )LC, i = 2, · · · , N , are Schur stable, and (A − I)hi = 0, ∀ i, j =
1, · · · , N , where λ̂i , i = 2, · · · , N , denote the eigenvalues of D located in the
open unit disk.
70 Cooperative Control of Multi-Agent Systems
x 
i −hi
Proof 21 Let z̃i = vi , i = 1, · · · , N . Then, it follows from (3.1) and
(3.25) that
 
+ A−I e
z̃ = [IN ⊗ A + (IN − D) ⊗ H]z̃ + ⊗ H, (3.26)
0

where the matrices A and H are defined in (3.18). Note that the formation
He is achieved if the system (3.26) reaches consensus. By following similar
steps in the proof of Theorem 17, it is easy to see that the formation H e is
achieved under the protocol (3.25) if the matrices A + BF and A + (1 − λ̂i )LC,
i = 2, · · · , N , are Schur stable, and (A − I)hi = 0, ∀ i = 1, · · · , N .

Remark 22 The achievable formation structures have to satisfy the con-

straints (A − I)hi = 0, ∀ i = 1, · · · , N . Note that hi can be replaced by hi − h1 ,
i = 2, · · · , N , in order to be independent of the reference coordinate, by sim-
ply choosing h1 corresponding to agent 1 as the origin. The formation protocol
(3.25) for a given achievable formation structure can be constructed by using
Algorithm 10 and Theorem 19.

80

60

40
k

20

0
200
20
100
0
0
−20
y −100 −40 x

FIGURE 3.5: The agents form a regular hexagon.

Example 12 Consider a network of six double integrators, described by

x+
i = xi + ṽi ,
ṽi+ = ṽi + ui ,
yi = x i , i = 1, · · · , 6,

where xi ∈ R2 , ṽi ∈ R2 , yi ∈ R2 , and ui ∈ R2 are the position, the velocity,

the measured output, and the acceleration input of agent i, respectively.
The objective is to design a protocol (3.25) such that the agents will evolve
T
to a regular hexagon with edge length 8. In this case, choose h1 = [ 0 0 0 0 ] ,
 Consensus Control of Linear Multi-Agent Systems: Discrete-Time Case 71
T √ T √ T √ T
h2 = [ 8 0 0 0 ] , h3 = [ 12 4 3 0 0 ] , h4 = [ 8 8 3 0 0 ] , h5 = [ 0 8 3 0 0 ] ,
√ T
h6 = [ −4 4 3 0 0 ] . Choose F = − [ 0.5 1.5 ] ⊗ I2 so that A + BF is Schur
stable. Solving equation (3.15) with δ = 0.95 gives P = 104 ×[ 1.1780 0.0602
0.0602 0.0062 ]⊗I2 .
T
By Theorem 19, we obtain L = − [ 1.051 0.051 ] ⊗ I2 . Then, the agents with
such a protocol (3.25) will form a regular hexagon with respect to Γ≤0.95 .
The state trajectories of the six agents are depicted in FIGURE 3.5 for the
communication topology given in FIGURE 3.3.

3.5 Discussions
This chapter has addressed the consensus control problem for multi-agent
systems with general discrete-time linear agent dynamics. The results obtained
in this chapter can be regarded as extensions of those results in the preceding
chapter in the discrete-time setting.
Regarding the continuous-time consensus in Chapter 2 and the discrete-
time consensus in this chapter, there exist some important differences, which
are summarized as follows.
i) For the continuous-time multi-agent systems in Chapter 2, the existence of
consensus protocol relies only on the stabilizability or controllability of each
agent if the given communication topology contains a directed spanning
tree. In contrast, for the discrete-time multi-agent systems in this chapter,
the existence condition for the consensus protocols additionally depends on
the unstable eigenvalues of the state matrix of each agent, implying that
the consensus problem of discrete-time multi-agent systems is usually much
harder to tackle than that of continuous-time multi-agent systems.
ii) The consensus region for the continuous-time multi-agent systems in Chap-
ter 2 is the open right half plane or a subset of the open right half plane.
On the contrary, the consensus region for the discrete-time multi-agent sys-
tems in this chapter is the unit disk centered in the origin or a subset of the
unit disk. The continuous-time consensus region is related to the discrete-
time consensus region by a bilinear transformation, similarly to the rela-
tionship between the continuous-time and discrete-time stability of linear
time-invariant systems.
iii) Laplacian matrices are utilized to depict the communication topology for
the continuous-time multi-agent systems in Chapter 2. Differently, stochas-
tic matrices are chosen for the discrete-time multi-agent systems in this
chapter. Because the discrete-time consensus region is the unit disk cen-
tered in the origin or a subset of the unit disk, both largest and smallest
nonzero eigenvalues of the Laplacian matrix (for undirected graphs) are re-
quired for the design of the consensus protocols when adopting Laplacian
72 Cooperative Control of Multi-Agent Systems

matrices as in related works [185, 50, 55]. The advantage of using stochastic
matrices in this chapter is that only the eigenvalue with the largest modulus
matters, which facilitates the consensus protocol design.

3.6 Notes
The materials of this chapter are mainly based on [82, 91]. For further re-
sults on consensus of general linear discrete-time multi-agent systems, see,
e.g., [50, 55, 164, 172, 185]. Specially, distributed consensus protocols were
designed in [172] for discrete-time multi-agent systems with each node being
neutrally stable. Reference [185], similar to this chapter, also used modified
algebraic Riccati equation to design consensus protocol and further consid-
ered the effect of a finite communication data rate. By introducing a properly
designed dynamic filter into the local control protocols, the consensusability
condition in [185] was further relaxed in [50]. Methods based on the H∞ and
H2 type Riccati inequalities were given in [55] to design the consensus pro-
tocols. The effect of random link failures and random switching topology on
discrete-time consensus was investigated in [186]. The consensus problem of
discrete-time multi-agent systems with a directed topology and communica-
tion delay was considered in [164], where a consensus protocol based on the
networked predictive control scheme was proposed to overcome the effect of
delay.
4
H∞ and H2 Consensus Control of Linear
Multi-Agent Systems

CONTENTS
4.1 H∞ Consensus on Undirected Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.1.1 Problem Formulation and Consensus Condition . . . . . . . . . . . . 74
4.1.2 H∞ Consensus Region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.1.3 H∞ Performance Limit and Protocol Synthesis . . . . . . . . . . . . 80
4.2 H2 Consensus on Undirected Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.3 H∞ Consensus on Directed Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.3.1 Leader-Follower Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.3.2 Strongly Connected Directed Graphs . . . . . . . . . . . . . . . . . . . . . . . 87
4.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

In the previous chapters, we have considered the consensus problems for multi-
agent systems with linear node dynamics. In many circumstances, the agent
dynamics might be subject to external disturbances, for which case the consen-
sus protocols should maintain certain required disturbance attenuation per-
formance. In this chapter, we consider the distributed H∞ and H2 consensus
control problems for linear multi-agent systems in order to examine the dis-
turbance attenuation performance of the consensus protocols with respect to
external disturbances.
In Section 4.1, the H∞ consensus problem for undirected communication
graphs is formulated. A distributed consensus protocol based on the relative
states of neighboring agents is proposed. The distributed H∞ consensus prob-
lem of a linear multi-agent network is converted to the H∞ control problem
of a set of independent systems of the same dimension as a single agent.
The notion of H∞ consensus region is then defined and discussed. The H∞
consensus region can be regarded as an extension of the notion of consensus
region introduced in Chapter 2 in order to evaluate the performance of the
multi-agent network subject to external disturbances. It is be pointed out via
several examples that the H∞ consensus regions can serve as a measure for the
robustness of the consensus protocol with respect to the variations of the com-
munication graph. A necessary and sufficient condition for the existence of a
protocol yielding an unbounded H∞ consensus region is derived. A multi-step
procedure for constructing such a protocol is further presented, which main-
tains a favorable decoupling property. Such a design procedure involves no

73
74 Cooperative Control of Multi-Agent Systems

conservatism and is convenient to implement, especially when the agent num-

ber is large. The exact H∞ performance limit of the consensus of the agents
under the protocol is derived in this section, which is equal to the minimal
H∞ norm of an isolated agent achieved by using a state feedback controller,
independent of the communication graph as long as it is connected.
The distributed H2 consensus problem for undirected communication
graphs is similarly addressed in Section 4.2. The consensus protocol is then
designed to achieve consensus with a given H2 performance and the H2 per-
formance limit of the consensus is also derived. It should be highlighted that
the H2 consensus condition and the H2 performance limit, different from the
H∞ consensus case, depend on the number of agents in the network.
The distributed H∞ consensus problems for leader-follower and strongly
connected communication graphs are further studied in Section 4.3, which
extend the results in Section 4.1. It is worth noting that for the H∞ consensus
problem with strongly connected graphs, the design of the consensus protocol
generally requires the knowledge of the left eigenvector of the Laplacian matrix
associated with the zero eigenvalue, implying the H∞ consensus problem for
the directed graph case is much harder than the undirected graph case.

4.1 H∞ Consensus on Undirected Graphs

4.1.1 Problem Formulation and Consensus Condition
Consider a network of N identical linear agents subject to external distur-
bances. The dynamics of the i-th agent are described by
ẋi = Axi + Bui + Dωi , i = 1, · · · , N, (4.1)
n p
where xi ∈ R is the state of the i-th agent, ui ∈ R is the control input,
ω i ∈ Lm m1
2 [0, ∞) is the external disturbance, L2 [0, ∞) denotes the space of
1

m1 -dimensional square integrable vector functions over [0, ∞), and A, B, and
D are constant matrices with compatible dimensions.
In this section, the communication graph among the N agents is repre-
sented by an undirected graph G. It is assumed that at each time instant each
agent knows the relative states of its neighboring agent with respect to itself.
Based on the relative states of neighboring agents, the following distributed
consensus protocol is proposed:
N
X
ui = cK aij (xi − xj ), i = 1, · · · , N, (4.2)
j=1

where c > 0 denotes the coupling gain, K ∈ Rp×n is the feedback gain matrix,
and aij denotes the (i, j)-th entry of the adjacency matrix associated with the
graph G.
 H∞ and H2 Consensus Control of Linear Multi-Agent Systems 75

The objective here is to find an appropriate protocol (4.2) for the agents
in (4.1) to reach consensus and meanwhile maintain a desirable performance
with respect to external disturbances ωi . To this end, define the performance
variable zi , i = 1, · · · , N , as the average of the weighted relative states of the
agents, described by
N
1 X
zi = C(xi − xj ), i = 1, · · · , N, (4.3)
N i=1

where zi ∈ Rm2 and C ∈ Rm2 ×n is a given constant matrix.

Let x = [xT1 , · · · , xTN ]T , ω = [ω1T , · · · , ωN
T T T T
] , and z = [z1T , · · · , zN ] . Then,
the agent network resulting from (4.1), (4.2), and (4.3) can be written as

ẋ = (IN ⊗ A + cL ⊗ BK)x + (IN ⊗ D)ω,

(4.4)
z = (M ⊗ C)x,

where L is the Laplacian matrix associated with the graph G and M , IN −

1 T
N 11 . Denote by Tωz the transfer function matrix from ω to z of the network
(4.4).
The distributed H∞ consensus problem for the network (4.4) is first de-
fined.

Definition 12 Given the agents in (4.1) and an allowable γ > 0, the protocol
(4.2) is said to solve the distributed suboptimal H∞ consensus problem if

i) the network (4.4) with wi = 0 can reach consensus in the sense of

limt→∞ kxi − xj k = 0, ∀ i, j = 1, · · · , N ;
ii) kTωz k∞ < γ, where kTωz k∞ is the H∞ norm of Tωz , defined by kTωz (s)k∞ =
supw∈R σ̄ (Tωz (jw)) [200].

The H∞ performance limit of the consensus of the network (4.4) is the mini-
mal kTωz (s)k∞ of the network (4.4) achieved by using the protocol (4.2).

The following presents a necessary and sufficient condition for solving the
H∞ consensus problem of (4.4).

Theorem 23 Assume that the communication graph G is connected. For a

given γ > 0, there exists a consensus protocol (4.2) solving the suboptimal H∞
consensus problem if and only if the following N −1 systems are simultaneously
asymptotically stable and the H∞ norms of their transfer function matrices
are all less than γ:
x̂˙ i = (A + cλi BK)x̂i + Dω̂i ,
(4.5)
ẑi = C x̂i , i = 2, · · · , N,
where λi , i = 2, · · · , N , are the nonzero eigenvalues of L.
76 Cooperative Control of Multi-Agent Systems

Proof 22 By using the fact that 1 is the right eigenvector and also the left
eigenvector of L associated with eigenvalue 0, we have

M L = L = LM. (4.6)

Let ξ = (M ⊗In )x. Then, by invoking (4.6), it follows from (4.4) that ξ evolves
according to the following dynamics:

ξ˙ = (IN ⊗ A + cL ⊗ BK)ξ + (M ⊗ D)ω,

(4.7)
z = (IN ⊗ C)ξ,

whose transfer function matrix from ω to z is denoted by T̃ωz . By the definition

of M , it is easy to see that 0 is a simple eigenvalue of matrix M with 1 as its
right eigenvector and 1 is another eigenvalue with multiplicity N − 1. Then
it follows that ξ = 0 if and only if x1 = x2 = · · · = xN . Because (4.6) holds,
there exists a unitary matrix U ∈ RN ×N such that both U T M U and U T LU are
diagonal [58]. Since L and M have 1 as the same right and left eigenvectors
h i 1T
associated with eigenvalue 0, we can choose U = [ √1N Y ] , UT = √
N , with
W
Y ∈ RN ×(N −1) and W ∈ R(N −1)×N , such that

U T M U = Ψ , diag(0, 1, · · · , 1),
(4.8)
U T LU = Λ , diag(0, λ2 , · · · , λN ).

In virtue of (4.8), we can verify that

Teωz = (IN ⊗ C)(sI − IN ⊗ A − cL ⊗ BK)−1 (M ⊗ D)

= (U ⊗ C)(sI − IN ⊗ A − cΛ ⊗ BK)−1 (ΨU T ⊗ D)
= (U ⊗ In )diag(0, T̃ω̂2 ẑ2 , · · · , T̃ω̂N ẑN )(U T ⊗ In )
(4.9)
= (U Ψ ⊗ C)(sI − IN ⊗ A − cΛ ⊗ BK)−1 (U T ⊗ D)
= (M ⊗ C)(sI − IN ⊗ A − cL ⊗ BK)−1 (IN ⊗ D)
= Tωz ,

where T̃ω̂i ẑi = C(sI − A − cλi BK)−1 D, i = 2, · · · , N , identical to the transfer

function matrices of (4.5). Then, it follows that the distributed H∞ consensus
problem is solved if and only if (4.7) is asymptotically stable and kT̃ωz k∞ < γ.
Introduce the following transformations:
˜
ξ = (U ⊗ In )ξ, ω = (U ⊗ Im1 )ω̂, z = (U ⊗ Im2 )ẑ, (4.10)

where ξ˜ = [ξ˜1T , · · · , ξ˜N

T T
] , ω̂ = [ω̂1T , · · · , ω̂N
T T
] , and ẑ = [ẑ1T , · · · , ẑN
T T
] . Then,
substituting (4.10) into (4.7) yields
˙
ξ˜ = (IN ⊗ A + cΛ ⊗ BK)ξ˜ + (Ψ ⊗ D)ω̂,
(4.11)
˜
ẑ = (IN ⊗ C)ξ,
H∞ and H2 Consensus Control of Linear Multi-Agent Systems 77

which clearly is composed of the following N independent systems:

˙
ξ˜1 = Aξ˜1 ,
ẑ1 = C ξ˜1 ,
(4.12)
˙
ξ˜i = (A + cλi BK)ξ˜i + Dω̂i ,
ẑi = C ξ˜i , i = 2, · · · , N.

Denote by T̃ω̂ẑ the transfer function matrices of (4.11). Then, it follows from
(4.12), (4.11), and (4.10) that
T̃ω̂ẑ = diag(0, T̃ω̂2 ẑ2 , · · · , T̃ω̂N ẑN ) = (U T ⊗ Im1 )T̃ωz (U ⊗ Im2 ), (4.13)
which implies that
kT̃ω̂ẑ k∞ = max kT̃ω̂i ẑi k∞ = kT̃ωz k∞ . (4.14)
i=2,··· ,N

As to ξ˜1 , we have
1T 1T
ξ˜1 = ( √ ⊗ In )ξ = ( √ ⊗ In )M x ≡ 0.
N N
Therefore, the suboptimal H∞ consensus problem for the network (4.4) is
solved if and only if the N − 1 systems in (4.5) are simultaneously asymp-
totically stable and kT̃ω̂i ẑi k∞ < γ, i = 2, · · · , N .
Remark 23 The usefulness of this theorem lies in that it converts the dis-
tributed H∞ consensus problem of the high-dimensional multi-agent network
(4.4) into the H∞ control problems of a set of independent systems having the
same dimensions as a single agent in (4.1), thereby significantly reducing the
computational complexity. A unique feature of protocol (4.2) proposed here is
that by introducing a constant scalar c > 0, called the coupling gain, the no-
tions of H∞ and H2 consensus regions can be brought forward, as detailed in
the following subsections.

4.1.2 H∞ Consensus Region

Given a protocol in the form of (4.2), the distributed H∞ consensus problem
of the network (4.4) can be recast into analyzing the following system:

ζ̇ = (A + σBK)ζ + Dωi ,
(4.15)
zi = Cζ,
where ζ ∈ Rn and σ ∈ R, with σ depending on c. The transfer function of
system (4.15) is denoted by Tbωi zi . Clearly, the stability and H∞ performance
of the system (4.15) depends on the scalar parameter σ.
The notion of H∞ consensus region is defined as follows.
78 Cooperative Control of Multi-Agent Systems

Definition 13 The region Sγ of the parameter σ ⊂ R+ , such that system

(4.15) is asymptotically stable and kTbωi zi k∞ < γ, is called the H∞ consensus
region with performance index γ of the network (4.4).

The H∞ consensus region can be regarded as an extension of the consensus

regions introduced in Chapter 2, used to evaluate the performance of a multi-
agent network subject to external disturbances. According to Theorem 23, we
have

Corollary 4 For a given γ > 0, the protocol (4.2) solves the suboptimal
H∞ consensus problem for the agents (4.1) if and only if cλi ∈ Sγ , for
i = 2, · · · , N.

For a protocol of the form (4.2), its H∞ consensus region with index γ, if it
exists, is an interval or a union of several intervals on the real axis, where the
intervals themselves can be either bounded or unbounded. The H∞ consensus
region can serve as a measure for the robustness of the consensus protocol (4.2)
with respect to the variations of the communication topology, as illustrated
by the following example.

Example 13 (bounded H∞ consensus region) The agent dynamics and the

protocol are given by (4.1) and (4.2), respectively, with
     
−2 2 1 1
A= , B= , D=
−1 1 −1 0.6
   
C = 1 0.8 , K = 1 0.2 .

The H∞ performance of (4.15) with respect to the parameter σ is depicted

in FIGURE 4.1 (a). It can be observed that Sγ>1.683 , i.e., the H∞ consensus
region with index γ larger than the minimal value 1.683, is a bounded interval
of σ in R; for example, Sγ=1.782 is [0.0526, 0.203].
For illustration, let the communication graph G be given as in FIGURE
4.1 (b), with Laplacian matrix
 
4 −1 −1 −1 −1 0
−1 3 −1 0 0 −1
 
−1 −1 2 0 0 0
L=  ,
−1 0 0 2 −1 0  
−1 0 0 −1 3 −1
−1 0 0 0 −1 2

whose nonzero eigenvalues are 1.382, 1.6972, 3.618, 4, 5.3028. Thus, the pro-
tocol (4.2) given as above solves the suboptimal H∞ consensus problem with
γ = 1.782 for the graph given in FIGURE 4.1(b) if and only if the coupling
gain c lies within the set [0.0381, 0.0383].
Let us see how modifications of the communication graph affect the H∞
consensus problem by considering the following simple cases.
H∞ and H2 Consensus Control of Linear Multi-Agent Systems 79

14

1 6
12

10
H norm of (4.15)

6 2 5
∞

0
−0.2 0 0.2 0.4 0.6 0.8 1 1.2
σ 3 4

(a) (b)

FIGURE 4.1: (a) The H∞ consensus region; (b) the communication graph.

• An edge between nodes 3 and 6 is added, i.e., more information exchanges

exist inside the network. The minimal and maximal eigenvalues of the re-
sulting Laplacian matrix are 1.4384 and 5.5616, respectively. Thus, it can
be verified that the protocol (4.2) fails to solve the suboptimal H∞ consensus
problem with γ = 1.782 for any c ∈ R+ .
• The edge between nodes 2 and 3 is removed, i.e., less information links in the
network. The minimal and maximal eigenvalues of the resulting Laplacian
matrix become 0.8817 and 5.2688, respectively. The protocol (4.2) in this
case also fails to solve the H∞ consensus problem with γ = 1.782.
• The edge between nodes 2 and 6 is removed and an edge between nodes
3 and 4 is added. In this case, the number of communication links in the
network remains unchanged. The minimal and maximal eigenvalues of the
resulting Laplacian matrix are 0.7639 and 5.2361, respectively. Once again,
the protocol fails to solve the suboptimal H∞ problem with γ = 1.782.
These sample cases imply that for a bounded H∞ consensus region, the dis-
tributed protocol (4.2), if not well designed, can be quite fragile to variations of
the network’s communication graph. In other words, it is desirable for the H∞
consensus region to be large enough in order to ensure the protocol maintain
a desired robustness margin with respect to the communication graph. This is
similar to the consensus region introduced in Chapter 2.

The second example has a disconnected H∞ consensus region.

Example 14 The agent dynamics and the protocol are given by (4.1) and
80 Cooperative Control of Multi-Agent Systems

(4.2), respectively, with

     
−0.4 −19.5998 1.5 0.3 1
A= , B= , D=
4.333 0.4 0 1 0
 
  0 2
C= 0 1 , K= .
−0.8 0

The H∞ consensus region of (4.15) with respect to parameter σ is depict-

ed in FIGURE 4.2, which is composed of two disjoint subregions, of which
one is bounded and the other is unbounded. It can be verified that Sγ=2 =
[0.962, 5.1845] ∪ [6.928, ∞). For the communication graph given in FIGURE
4.1(b), the protocol (4.2) solves the H∞ consensus problem with γ = 2 if and
only if c ∈ [0.6961, 0.9777] ∪ [5.013, ∞).

20

18

16

14
H norm of (4.15)

12

10

6
∞

−2
0 2 4 6 8 10 12
σ

FIGURE 4.2: The disconnected H∞ consensus region.

4.1.3 H∞ Performance Limit and Protocol Synthesis

It was shown in the last subsection that the distributed protocol (4.2) should
have a large enough H∞ consensus region to be robust with respect to the
communication graph. One convenient and desirable choice is to design the
protocol with an unbounded H∞ consensus region.

Lemma 27 (Bounded Real Lemma, [200]) Let γ > 0 and G(s) =

C(sI − A)−1 B. Then, the following two statements are equivalent:
1) A is stable and kG(s)k∞ < γ.
2) there exists an X > 0 such that
1
AX + XAT + XC T CX + BB T < 0.
γ2
In the following, a necessary and sufficient condition is derived for the
existence of a protocol (4.2) having an unbounded H∞ consensus region.
H∞ and H2 Consensus Control of Linear Multi-Agent Systems 81

Theorem 24 For a given γ > 0, there exists a protocol (4.2) having an un-
bounded H∞ consensus region Sγ , [τ, ∞) if and only if there exist a matrix
P > 0 and a scalar τ > 0 satisfying the following linear matrix inequality
(LMI):  
AP + P AT − τ BB T D P CT
 DT −γ 2 I 0  < 0. (4.16)
CP 0 −I

Proof 23 (Necessity) According to Definition 12, if the network (4.4) has an

unbounded H∞ consensus region, then A + σBK is Hurwitz and kC(sI − A −
σBK)−1 Dk∞ < γ for some matrix K and scalar σ. Since K is to be designed,
without loss of generality, choose σ = 1. By Lemma 27, there exists a matrix
K such that A + BK is Hurwitz and kC(sI − A − BK)−1 Dk∞ < γ if and only
if there exists a matrix P > 0 such that
1
(A + BK)P + P (A + BK)T + DDT + P C T CP < 0.
γ2
Let Y = KP . Then, the above inequality becomes
1
AP + P AT + BY + Y T B T + DDT + P C T CP < 0.
γ2
By Lemma 20 (Finsler’s Lemma), there exists a matrix Y satisfying the above
inequality if and only if there exists a scalar τ > 0 such that
1
AP + P AT − τ BB T + DDT + P C T CP < 0, (4.17)
γ2
which, in virtue of Lemma 19 (the Schur Complement Lemma), is equivalent
to  
AP + P AT − τ BB T D P CT
 DT −γ 2 I 0  < 0. (4.18)
CP 0 −I
(Sufficiency) If (4.18) holds for some matrix P > 0 and scalar τ > 0, then
(4.17) holds also. Take K = − 21 B T P −1 . Then, for cλi ≥ τ , i = 2, · · · , N , it
follows from (4.17) that
1
(A + cλi BK)P + P (A + cλi BK)T + DDT + P C T CP
γ2
(4.19)
1
= AP + P AT − cλi BB T + 2 DDT + P C T CP < 0,
γ

implying that kC(sI −A−cλi BK)−1 Dk∞ < γ, i = 2, · · · , N , i.e., the protocol
(4.2) with K given as above has an unbounded H∞ consensus region [τ, ∞).

The exact H∞ performance limit of the network (4.4) under the protocol
(4.2) is now obtained as a consequence.
 82 Cooperative Control of Multi-Agent Systems

Corollary 5 The H∞ performance limit γmin of the consensus of the network

(4.4) under the protocol (4.2) is given by the following optimization problem:

minimize γ
(4.20)
subject to LMI (4.16), with P > 0, τ > 0, γ > 0.

Remark 24 It should be noted that the H∞ performance limit γmin of the

network (4.4) under the consensus protocol (4.2) is actually equal to the min-
imal H∞ norm of a single agent (4.1) by using a state feedback controller of
the form ui = F xi , independent of the communication graph G as long as it is
connected. Moreover, the minimum γmin achieved by solving LMI (4.20) gen-
erally corresponds to a high-gain controller (4.2), as depicted in the example
below. Choosing a γ a little bigger than γmin would help keep the gain smaller.

A procedure for constructing the protocol (4.2) is now presented.

Algorithm 11 For any γ ≥ γmin , where γmin is given by (4.20), the proto-
col (4.2) solving the distributed H∞ consensus problem can be constructed as
follows:

1) Solve LMI (4.16) for a solution P > 0 and τ > 0. Then, choose the feedback
gain matrix K = − 21 B T P −1 .
τ
2) Select the coupling gain c not less than the threshold value cth = min λi ,
i=2,··,N
where λi , i = 2, · · · , N , are the nonzero eigenvalues of L.

Remark 25 Similar to the design algorithms based on the consensus region

in Chapter 2, the above protocol design procedure based on the H∞ consensus
region also has a favorable decoupling feature. Specifically, step 1) deals only
with the agent dynamics and the feedback gain matrix K of the protocol (4.2),
leaving the communication graph of the network to be handled in step 2) by
adjusting the coupling gain c. This feature is very desirable for the case where
the agent number N is large, for which the eigenvalues of the corresponding
Laplacian matrix L are hard to determine or even troublesome to estimate.
Here, we need to choose the coupling gain to be large enough, which of course
involves conservatism.

Now, Example 13 in the above subsection is revisited.

Example 15 The agent dynamics (4.1) remain the same as in Example 13,
while matrix K will be redesigned via Algorithm 11. Solving LMI (4.16) with
γ = 1 by using toolboxes YALMIP [103] and SeDuMi [158] gives feasible
solutions P = [ 1.3049 0.0369
0.0369 0.1384 ] and τ = 1.7106. Thus, the feedback gain matrix
of (4.2) is chosen as K = [ −0.4890 3.7443 ]. Different from Example 13, the
protocol (4.2) with this matrix K has an unbounded H∞ consensus region with
index γ = 1 in the form of [1.7106, ∞). For the graph in FIGURE 4.1(b),
 H∞ and H2 Consensus Control of Linear Multi-Agent Systems 83

the protocol (4.2) with K chosen here solves the suboptimal H∞ consensus
problem with γ = 1, if the coupling gain c is not less than the threshold value
cth = 1.2378.
By solving the optimization problem (4.20), the H∞ performance limit of
the consensus of the network (4.4) under the protocol (4.2) can be further
obtained as γmin = 0.0535. The corresponding optimal feedback gain matrix of
(4.2) is obtained as K = [ 48.6183 48.6450 ], and the scalar τ in (4.16) is τ =
9.5328×106 . For the graph in FIGURE 4.1 (b), the threshold cth corresponding
to γmin is cth = 6.8978 × 106 in this numerical example.

4.2 H2 Consensus on Undirected Graphs

In this section, the H2 performance of the network (4.4) will be discussed.
q aR stable transfer function F (s), its H2 norm is defined as kF k2 =
For
1 ∞ ∗
2π −∞ tr(F (jω)F (jω))dω [200].
Similar to the H∞ case, the distributed H2 consensus problem is stated as
follows.

Definition 14 Given the agents in (4.1) and an allowable γ > 0, the con-
sensus protocol (4.2) is said to solve the distributed suboptimal H2 consensus
problem if
i) the network (4.4) with wi = 0 can reach consensus in the sense of
limt→∞ kxi − xj k = 0, ∀ i, j = 1, · · · , N ;
ii) kTωz k2 < γ.
The H2 performance limit of the consensus of the network (4.4) is the minimal
kTωz (s)k2 of the network (4.4) achieved by using the consensus protocol (4.2).

Theorem 25 For a given γ̃ > 0, there exists a protocol (4.2) solving the
suboptimal H2 consensus problem for the agents in (4.1) if and only if
qPN − 1 systems in (4.5) are simultaneously asymptotically stable and
the
N
kTeω̂ ẑ k2 < γ̃, where Teω̂ ẑ , i = 2, · · · , N , are the transfer function
i=1 i i 2 i i

matrices of the systems in (4.5).

Proof 24 It can be shown by following similar steps in proving Theorem 23.

The H2 consensus region is defined as follows.

Definition 15 The region Seγ̃ of the parameter σ ⊂ R+ , such that system

(4.15) is asymptotically stable, with kTeωi zi k2 < √γ̃N , is called the H2 consensus
region with performance index γ̃ of the network (4.4).
84 Cooperative Control of Multi-Agent Systems

According to Theorem 25, we have

Corollary 6 The consensus protocol (4.2) solves the suboptimal H2 consen-
sus problem for the agents in (4.1), if cλi ∈ Seγ̃ , for i = 2, · · · , N.
Remark 26 Contrary to the H∞ case, the H2 consensus region is related
to the number of agents in the network and using H2 consensus region to
characterize the H2 performance of the network (4.4) involves certain conser-
vatism. This is essentially due to the inherent difference between the H2 and
H∞ norms, and due to the fact that, in the H2 case, the performance index γ̃
couples the N − 1 systems in (4.5), which thereby is more difficult to analyze.
The H2 consensus region analysis can be discussed similarly to Section 4.1,
therefore is omitted here for brevity. The synthesis issue is somewhat different,
hence is further discussed below.
Lemma 28 ([200]) Let γ̃ > 0 and G(s) = C(sI−A)−1 B. Then, the following
two statements are equivalent.
1) A is stable and kG(s)k2 < γ̃.
2) there exists an X > 0 such that
AX + XAT + BB T < 0, tr(CXC T ) < γ̃ 2 .
Theorem 26 For a given γ̃ > 0, there exists a distributed protocol (4.2)
having an unbounded H2 consensus region Seγ̃ , [τ̃ , ∞) if and only if there
exist a matrix Q > 0 and a scalar τ̃ such that
γ̃ 2
AQ + QAT − τ̃ BB T + DDT < 0, tr(CQC T ) < . (4.21)
N
Proof 25 (Necessity) Similar to the proof of Theorem 24, if the network (4.4)
has an unbounded H2 consensus region, then there exists a matrix K such that
A + BK is Hurwitz and kC(sI − A − BK)−1 Dk2 < √γ̃N , which is equivalent
to that there exists a matrix Q > 0 such that
γ̃ 2
(A + BK)Q + Q(A + BK)T + DDT < 0, tr(CQC T ) < .
N
Let V = KQ. Then, the above inequality becomes
γ̃ 2
AQ + QAT + BV + V T B T + DDT < 0, tr(CQC T ) < ,
N
which, by Lemma 20, is equivalent to that there exist a matrix Q > 0 and a
scalar τ̃ > 0 such that (4.21) holds.
(Sufficiency) Take K = − 21 B T Q−1 . For cλi ≥ τ̃ , i = 2, · · · , N , we have

(A + cλi BK)Q + Q(A + cλi BK)T + DDT

= AQ + QAT − cλi BB T + DDT < 0,
γ̃ 2
tr(CQC T ) < ,
N
 H∞ and H2 Consensus Control of Linear Multi-Agent Systems 85

which together imply that kC(sI−A−cλi BK)−1 Dk2 < √γ̃N , i = 2, · · · , N , i.e.,
the protocol (4.2) with K chosen as above has an unbounded H2 performance
region [τ̃ , ∞).

Corollary 7 The H2 performance limit γ̃min of the network (4.4) under the
consensus protocol (4.2) is given by the optimization problem:

minimize γ̃
(4.22)
subject to LMI (4.21), with Q > 0, τ̃ > 0, γ̃ > 0.

Proof 26 Solving the optimization problem (4.22) gives solutions γ̃min and
τ̃min . Select c such that cλi ≥ τ̃min , for i = 1, 2, · · · , N . Then, the protocol
(4.2) with K given as in the proof of Theorem 26 yields kTω̂i ẑi k2 = γ̃√min N
, for
i = 1, · · · , N , which by Theorem 25 implies that γ̃min is the H2 performance
limit of the network (4.4).

Remark 27 The H2 performance limit of the consensus of the network (4.4)

under the protocol (4.2) is related to two factors: the minimal H2 norm of a
single agent (4.1) by using the state feedback controller ui = F xi and the num-
ber of agents in the network. Contrary to the H∞ case, the H2 performance
limit of the network (4.4) scales with the size of the network.

Algorithm 12 For any γ̃ ≥ γ̃min , where γ̃min is given by (4.22), the proto-
col (4.2) solving the distributed H2 consensus problem can be constructed as
follows:

1) Solve LMI (4.21) to get solutions Q > 0 and τ̃ > 0. Then, choose the
feedback gain matrix K = − 12 B T Q−1 .
τ̃
2) Select the coupling gain c > c̃th , with c̃th = min λi , where λi , i = 2, · · · , N ,
i=2,··,N
are the nonzero eigenvalues of L.

4.3 H∞ Consensus on Directed Graphs

In the above sections, the communication graph is assumed to undirected.
In this section, we extend to consider the case where the multi-agent system
maintains a directed communication graph.

4.3.1 Leader-Follower Graphs

In this subsection, we consider the case where the communication graph G
is in the leader-follower form. Without loss of generality, assume that the
86 Cooperative Control of Multi-Agent Systems

agents indexed by 2 · · · , N , are the followers, whose dynamics are given by

(4.1), and the agent indexed by 1 is the leader, whose dynamics are described
by ẋ1 = Ax1 , where x1 ∈ Rn . It is assumed that the leader receives no
information from any follower and the state of the leader is available to only
a subset of the followers. The communication graph G satisfies the following
assumption.
Assumption 4.1 G contains a directed spanning tree with the leader as the
root and the subgraph associated with the N followers is undirected.
Denote by L the Laplacian matrix associated hwith G. Because
i the leader
0 0
has no neighbors, L can be partitioned as L = L2 1×(N L1
−1)
, where L2 ∈
R(N −1)×1 and L1 ∈ R(N −1)×(N −1) . By Lemma 1 and Assumption 4.1, it is
clear that L1 > 0.
Since the followers are desired to follow the leader, it is natural to define
the performance variables zi ∈ Rm2 as
zi = C2 (xi − x1 ), i = 2, · · · , N. (4.23)
In this case, the following distributed consensus protocol is proposed:
N
X
ui = cK aij (xi − xj ), i = 2, · · · , N, (4.24)
j=1

where c, aij , and K are defined as in (4.2).

T T
Let ε = [xT2 − xT1 , · · · , xTN − xT1 ]T , ω̄ = [ω2T , · · · , ωN ] , and z̄ =
T T T
[z2 , · · · , zN ] . Then, the agent network can be written as
ε̇ = (IN −1 ⊗ A + cL1 ⊗ BK)ε + (IN −1 ⊗ D)ω̄,
(4.25)
z̄ = (IN −1 ⊗ C)ε.
The transfer function matrix from ω̄ to z̄ of (4.25) is denoted by T ω̄z̄ .
The leader-follower H∞ consensus problem for the network (4.25) is stated
as follows.
Definition 16 Given the agents in (4.1) and an allowable γ > 0, the protocol
(4.24) is said to solve the distributed leader-follower H∞ consensus problem if
i) the states of the network (4.25) with wi = 0 satisfy limt→∞ kxi − x1 k = 0,
∀ i = 2, · · · , N ; ii) kT ω̄z̄ k∞ < γ.
Theorem 27 Assume that the graph G satisfies Assumption 4.1. For a given
γ > 0, there exists a protocol (4.24) solving the leader-follower H∞ consensus
problem if and only if the following N − 1 systems are simultaneously asymp-
totically stable and the H∞ norms of their transfer function matrices are all
less than γ:
x̃˙ i = (A + cλi BK)x̃i + Dω̃i ,
(4.26)
z̃i = C x̃i , i = 2, · · · , N,
where 0 < λ2 ≤ · · · ≤ λN are the nonzero eigenvalues of L.
H∞ and H2 Consensus Control of Linear Multi-Agent Systems 87

Proof 27 It can be completed by following similar steps in the proof of The-

orem 23.
The consensus protocol (4.24) can be designed by using Algorithm 11
to satisfy Theorem 27. The H2 consensus problem for leader-follower graphs
satisfying Assumption 4.1 can be discussed by following the steps in Section
4.1, which is omitted here.

4.3.2 Strongly Connected Directed Graphs

In this subsection, we extend the results in Section 4.1 to consider the H∞
consensus problem for the case of strongly connected directed communication
graphs. The H2 consensus problem for strongly connected directed graphs can
be similarly done.
Similarly as in Section 4.1, the objective in this subsection is to design
the consensus protocol (4.2) for the agents in (4.1) to reach consensus and
meanwhile maintain a desirable disturbance rejection performance. Different
from the performance variable in (4.3) for the case with undirected graphs,
we define a new performance variables for the directed communication graphs
in this subsection as
N
X
zi = C(xi − rj xj ), i = 1, · · · , N, (4.27)
j=1

where C ∈ Rm×n is a constant matrix, r = [r1 , · · · , rN ]T denotes the positive

left eigenvector of the Laplacian matrix L corresponding to the zero eigenvalue,
satisfying rT L = 0 and rT 1 = 1. Clearly, (4.27) will reduce to (4.3) for strongly
connected and balanced graphs.
Let x = [xT1 , · · · , xTN ]T , ω = [ω1T , · · · , ωN
T T
] , and z = [z1T , · · · , zN
T T
] . Then,
the agent network resulting from (4.1), (4.2), and (4.27) can be written as
ẋ = (IN ⊗ A + cL ⊗ BK)x + (IN ⊗ D)ω,
(4.28)
f ⊗ C)x,
z = (M

where M f , IN − 1N rT . Denote by Tωz the transfer function matrix of (4.28)

from ω to z.
The distributed H∞ consensus problem of the network (4.28) is defined in
Definition 12.
Define ζ = (Mf ⊗ I)x, where x = [xT , . . . , xT ]T . By the definition of rT , it
1 N
is easy to check that 0 is a simple eigenvalue of Mf with 1 as a right eigenvector
and 1 is the other eigenvalue with algebraic multiplicity N − 1. Thus, it is easy
to see that ζ = 0 if and only if x1 = x2 = . . . = xN . The network (4.28) can
be rewritten in terms of ζ as
f ⊗ D)ω,
ζ̇ = (IN ⊗ A + cL ⊗ BK)ζ + (M
(4.29)
z = (IN ⊗ C)ζ,
 88 Cooperative Control of Multi-Agent Systems

whose transfer function matrix from ω to z is denoted by Teωz . Since M fL =

f
LM , by following the steps in the proof of Theorem 23, it is not difficult to
show that Teωz = Tωz . Therefore, the distributed H∞ consensus problem with
γ > 0 is solved if and only if (4.29) is asymptotically stable and Teωz < γ.
The following algorithm is presented to construct the consensus protocol
(4.2).
Algorithm 13 For a scalar γ > 0, the protocol (4.2) solving the distributed
H∞ consensus problem can be constructed as follows:
1) Solve the following LMI:
 
XAT + AX − τ BB T XC T D
 CX −rmin In 0 <0 (4.30)
2
DT 0 − γrmax
rmin
In

to get a matrix X > 0 and a scalar τ > 0, where rmin = min ri and rmax =
i
max ri . Then, choose the feedback gain matrix K = − 21 B T X −1 .
i
τ
2) Select the coupling gain c > c̃th , with c̃th = a(L) , where a(L) denotes the
generalized algebraic connectivity of the communication graph G (see Lemma
4 in Chapter 1).
Remark 28 In the last step, the generalized algebraic connectivity a(L) is
used to determine the required coupling gain c. For a given graph, its a(L)
can be obtained by using Lemma 8 in [189], which might be not easy to com-
pute especially if the graph is of large scale. For the case where the com-
munication graph is balanced and strongly connected, then rT = 1T /N and
a(L) = λ2 ( 12 (L + LT )). In this case, the LMI (4.30) will reduce to the LMI
(4.16) in Section 4.1. Further, step 2) of Algorithm 13 will be identical to step
2) of Algorithm 11 if the communication graph is undirected and connected.
Theorem 28 Assume that the communication graph G is strongly connected.
For a given γ > 0, the consensus protocol (4.2) designed in Algorithm 13 can
solve the suboptimal H∞ consensus problem.
Proof 28 Consider the Lyapunov function candidate
1 T
V = ζ (R ⊗ X −1 )ζ,
2
where R = diag(r1 , r2 , . . . , rN ). Differentiating V with respect to t along the
trajectories of (4.29) and substituting K = 21 B T X −1 give
1 f ⊗ X −1 D)ω
V̇ = ζ T (R ⊗ X −1 A − cRL ⊗ X −1 BB T X −1 )ζ + ζ T (RM
2
c f ⊗ D)ω,
= ζ̃ T [R ⊗ AX − (RL + LT R) ⊗ BB T ]ζ̃ + ζ̃ T (RM
4
(4.31)
H∞ and H2 Consensus Control of Linear Multi-Agent Systems 89

where ζ̃ = (I ⊗ X −1 )ζ.
f ⊗ B T X −1 )x = 0, we can obtain from Lemma 4
Since (rT ⊗ B T )ζ̃ = (rT M
that
1 T
ζ̃ [(RL + LT R) ⊗ BB T ]ζ̃ ≥ a(L)ζ̃ T (IN ⊗ BB T )ζ̃, (4.32)
2
where a(L) > 0. In light of (4.32), it follows from (4.31) and step 2) of
Algorithm 13 that
1 T f ⊗ D)ω.
V̇ ≤ ζ̃ [R ⊗ (AX + XAT − τ BB T )]ζ̃ + ζ̃ T (RM (4.33)
2
For the case of ω(t) ≡ 0, it follows from the above analysis that
1 T
V̇ ≤ ζ̃ [R ⊗ (AX + XAT − τ BB T )]ζ̃ < 0,
2
where the last inequality follows from the LMI (4.30). This indicates that
consensus of (4.29) with ω(t) ≡ 0 is achieved. Thus, the first condition in
Definition 12 is satisfied.
Next, analyze the H∞ consensus performance of the multi-agent system
with ω(t) 6= 0. Observe that
1 T f ⊗ D)ω
V̇ ≤ ζ̃ [R ⊗ (AX + XAT − τ BB T )]ζ̃ + ζ̃ T (RM
2
1 1
+ ζ̃ T (R ⊗ XC T CX)ζ̃ − z T z
2rmin 2
γ2 γ 2 (4.34)
− ω T (R ⊗ In )ω + ω T ω
2rmax 2
 T  
1 ζ̃ ζ̃ 1 γ2
= Ω − z T z + ω T ω,
2 ω ω 2 2
where " #
R⊗Π RMf⊗D
Ω = fT γ2 , (4.35)
M R ⊗ DT − rmax R⊗I
1
and Π = AX + XAT − τ BB T + rmin XC T CX.
Noting that Ω < 0 if and only if the following inequality holds,
rmax fT f
R⊗Π+ M RM ⊗ DDT < 0, (4.36)
γ2
f=M
where we have used the fact that RM fT R.
Further, we have
rmax fT f rmax fT RMf)I ⊗ DDT
R⊗Π+ 2
M RM ⊗ DDT ≤ R ⊗ Π + 2 λmax (M
γ γ
rmax fT RM
f)DDT ].
≤ R ⊗ [Π + 2 λmax (M
γ rmin
(4.37)
90 Cooperative Control of Multi-Agent Systems

By using the Gershgorin’s disc theorem [58], it is not difficult to get that
fT RM
λmax (M f) ≤ 1. Then, it follows from (4.37) that

rmax fT f rmax
R⊗Π+ 2
M RM ⊗ DDT ≤ R ⊗ (Π + 2 DDT )
γ γ rmin (4.38)
< 0,

which implies that (4.36) holds, i.e., Ω < 0. Note that the last inequality in
(4.38) follows from the LMI (4.30).
Then, invoking (4.34) gives that

1 γ2
V̇ + y T y − ω T ω < 0, (4.39)
2 2
for all z and ω such that |z|2 + |ω|2 6= 0. Integrating inequality (4.39) over
infinite horizon yields

kyk2 − γ 2 kωk2 − z T (0)(R ⊗ P )z(0) < 0.

Recalling that z(0) = 0, we have kyk2 −γ 2 kωk2 < 0, i.e., kTeωz k∞ = kTωz k∞ <
γ. Thus, the second condition in Definition 12 is satisfied. Therefore, the H∞
consensus problem is solved.

Remark 29 Note that the Laplacian matrix L associated with a directed

communication graph is generally not positive semi-definite. Especially, for a
strongly connected graph without balanced conditions which is concerned here-
in, the matrix L + LT may not bet positive semi-definite either. Thus, this
unfavorable feature will not facilitate the decomposition method employed as
in Section 4.1. The key tools leading to Theorem 28 are the properly designed
Lyapunov function and the generalized algebraic connectivity of strongly con-
nected directed graphs. Compared to the H∞ consensus problem for undirected
communication graphs in Section 4.1, a distinct feature for the case with di-
rected graphs is that now the LMI condition in Algorithm 13 is related to the
positive left eigenvector of L associated with the zero eigenvalue.

3 3.9 4

4.2 4

3.8
1 2 4 5
4

FIGURE 4.3: A strongly connected directed communication graph.

H∞ and H2 Consensus Control of Linear Multi-Agent Systems 91

Example 16 Consider a network consisting of five agents whose communi-

cation graph as shown in FIGURE 4.3, where the weights are indicated on the
edges. The dynamics of the i-th agent is described by (4.1), with
       
x −0.4 −19.5998 1.5 0.3   1
xi = i1 , A = ,B = ,C = 1 0 ,D = .
xi2 4.333 0.4 0 1 1

The external disturbance ω = [2w, 5w, 4w, w, 1.5w]T , where w(t) is a ten-
period square wave starting at t = 0 with width 5 and height 1.
From FIGURE 4.3, it is easy to see that the network G is strongly connect-
ed. Some simple calculations give that both the minimum and maximum values
of the left eigenvector r are equal to 0.4472. By using Lemma 8 in [189], we
can get that a(L) = 2.2244. Choose the H∞ performance index γ = 1. Solving
LMI (4.30) by using the LMI toolbox of MATLAB gives a feasible solution
P = [ 0.21 0.08
0.08 0.03 ] and τ = 7.52. Thus, by Algorithm
 13, thefeedback gain ma-
trix of the protocol (4.2) is given as K = −23.52 57.73
33.78 −83.06 . Then, according
to Theorem 28 and Algorithm 13, the protocol (4.2) with K chosen as above
could solve H∞ consensus with performance index γ = 1, if the coupling gain
c ≥ 3.52. For the case of ω = 0, the state trajectories of the agents are re-
spectively shown in FIGURE 4.4, from which it can be seen that consensus is
indeed achieved. Furthermore, the trajectories of the performance variables zi ,
i = 1, · · · , 5, in the presence of disturbances under the zero initial conditions
are shown in FIGURE 4.5.

8 8

6
6

4
2
xi1

xi2

0 2

−2
0

−4

−2
−6

−8 −4
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
t t

FIGURE 4.4: The state trajectory of the agents under (4.2) constructed via
Algorithm 13.

4.4 Notes
The materials of Sections 4.1 and 4.2 are mainly adopted from [81, 85]. The
results in Section 4.3 are mainly based on [177]. For further results on dis-
tributed H∞ consensus and control problems of multi-agent systems, please
92 Cooperative Control of Multi-Agent Systems

0.15

0.1

0.05

zi
0

−0.05

−0.1
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
t

FIGURE 4.5: The performance variables under (4.2) constructed via Algo-
rithm 13.

refer to [87, 98, 99, 102, 108, 197]. Specifically, the H∞ consensus problem for
multi-agent systems of first-order and second-order integrators with external
disturbances and parameter uncertainties were considered in [98, 99]. A de-
composition approach was proposed in [108] to solve the distributed H2 and
H∞ control of identical coupled linear systems. An observer-type consensus
protocol was provided by using dynamic output feedback approach in [197]
to deal with the H∞ consensus problem for the case where the disturbances
satisfy the matching condition. Distributed H∞ consensus of linear multi-
agent systems with switching directed and balanced topologies was studied in
[180]. Dynamic controllers were proposed for distributed H∞ control problem
in [87, 102], whose design needs to solve a set of linear matrix inequalities
associated with all the nonzero eigenvalues of the Laplacian matrix.
5
Consensus Control of Linear Multi-Agent
Systems Using Distributed Adaptive
Protocols

CONTENTS
5.1 Distributed Relative-State Adaptive Consensus Protocols . . . . . . . . . . 94
5.1.1 Consensus Using Edge-Based Adaptive Protocols . . . . . . . . . . 96
5.1.2 Consensus Using Node-Based Adaptive Protocols . . . . . . . . . . 100
5.1.3 Extensions to Switching Communication Graphs . . . . . . . . . . . 101
5.2 Distributed Relative-Output Adaptive Consensus Protocols . . . . . . . . 103
5.2.1 Consensus Using Edge-Based Adaptive Protocols . . . . . . . . . . 104
5.2.2 Consensus Using Node-Based Adaptive Protocols . . . . . . . . . . 107
5.2.3 Simulation Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.3 Extensions to Leader-Follower Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.4 Robust Redesign of Distributed Adaptive Protocols . . . . . . . . . . . . . . . . . 114
5.4.1 Robust Edge-Based Adaptive Protocols . . . . . . . . . . . . . . . . . . . . 115
5.4.2 Robust Node-Based Adaptive Protocols . . . . . . . . . . . . . . . . . . . . 119
5.4.3 Simulation Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5.5 Distributed Adaptive Protocols for Graphs Containing Directed
Spanning Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.5.1 Distributed Adaptive Consensus Protocols . . . . . . . . . . . . . . . . . 123
5.5.2 Robust Redesign in the Presence of External Disturbances 129
5.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

In this chapter, we continue Chapter 2 to further consider the consensus con-

trol problem for multi-agent systems with general continuous-time linear agent
dynamics. The consensus protocols in Chapter 2 are proposed in a distributed
fashion, using only the local information of each agent and its neighbors. How-
ever, these consensus protocols involves some design issues. To be specific, for
the general case where the agents are not neutrally stable, the design of the
consensus protocols as in Chapter 2 generally requires the knowledge of some
eigenvalue information of the Laplacian matrix associated with the commu-
nication graph (the smallest nonzero eigenvalue of the Laplacian matrix for
undirected graphs and the smallest real part of the nonzero eigenvalues of the
Laplacian matrix for directed graphs). Similar situations take place in related
works [106, 146, 195]. Note that the smallest real part of the nonzero eigenval-
ues of the Laplacian matrix is global information in the sense that each agent

93
94 Cooperative Control of Multi-Agent Systems

has to know the entire communication graph G to compute it. Therefore, al-
though these consensus protocols are proposed and can be implemented in a
distributed fashion, they cannot be designed by each agent in a distributed
fashion. In other words, these consensus protocols are not fully distributed. As
explained in Chapter 1, due to the large number of agents, limited sensing ca-
pability of sensors, and short wireless communication ranges, fully distributed
consensus protocols, which can be designed and implemented by each agent
using only local information of its own and neighbors, are more desirable and
have been widely recognized. Fully distributed consensus protocols have the
advantages of high robustness, strong adaptivity, and flexible scalability.
In this paper, we intend to remove the limitation of requiring global in-
formation of the communication graph and introduce some fully distributed
consensus protocols. The main idea stems from the decoupling feature of the
consensus region approach proposed in Chapter 2 for consensus protocol de-
sign. As shown in Algorithm 2 in Chapter 2, the feedback gain matrix of the
consensus protocol (2.2) can be designed by using only the agent dynamics,
and the coupling gain c is used to deal with the effect of the communication
topology on consensus. The coupling gain c is essentially a uniform weight on
the edges in the communication graph. Since the knowledge of global informa-
tion of the communication graph is required for the selection of c, we intend
to implement some adaptive control tools to dynamically update the weights
on the communication graph in order to compensate for the lack of the global
information, and thereby to present fully distributed consensus protocols.
In Sections 5.1 and 5.2, we propose distributed consensus protocols based
on the relative state and output information, combined with an adaptive law
for adjusting the coupling weights between neighboring agents. Two types of
distributed adaptive consensus protocols are proposed, namely, the edge-based
adaptive protocol which assigns a time-varying coupling weight to each edge in
the communication graph and the node-based adaptive protocol which uses a
time-varying coupling weight for each node. For the case with undirected com-
munication graphs, these two classes of adaptive protocols are designed to en-
sure that consensus is reached in a fully distributed fashion for any undirected
connected communication graph without using any global information about
the communication graph. The case with switching communication graphs is
also studied. It is shown that the edge-based adaptive consensus protocol is ap-
plicable to arbitrary switching connected graphs. Extensions of the obtained
results to the case with a leader-follower communication graph are further
discussed in Section 5.3.
The robustness of the proposed adaptive protocols is discussed in Section
5.4. The σ-modification technique in [63] is implemented to present robust
adaptive protocols, which is shown to be able to guarantee the ultimate bound-
edness of both the consensus error and the adaptive weights in the presence
of bounded external disturbances. The upper bounds for the consensus error
are explicitly given.
Note that the aforementioned protocols are applicable to only undirected
Consensus Control Using Distributed Adaptive Protocols 95

communication graphs or leader-follower graphs where the subgraphs among

the followers are undirected. How to design fully distributed adaptive consen-
sus protocols for the case with general directed graphs is quite challenging.
The main difficulty lies in that the Laplacian matrices of directed graphs are
generally asymmetric, which renders the construction of adaptive consensus
protocols and the selection of appropriate Lyapunov function quite difficult.
Based on the relative states of neighboring agents, a novel distributed adap-
tive consensus protocol is constructed in Section 5.5. It is shown via a in-
tegral Lyapunov function that the proposed adaptive protocol can achieve
leader-follower consensus for any communication graph containing a directed
spanning tree with the leader as the root node. A distinct feature of the adap-
tive protocol for directed graphs is that monotonically increasing functions
are introduced to provide extra freedom for design. The adaptive protocol for
directed graphs is further redesigned in order to be robust in the presence of
matching external disturbances.

5.1 Distributed Relative-State Adaptive Consensus Pro-

tocols
Consider a group of N identical agents with general linear continuous-time
dynamics. The dynamics of the i-th agent are described by

ẋi = Axi + Bui ,

(5.1)
yi = Cxi , i = 1, · · · , N,

where xi ∈ Rn is the state, ui ∈ Rp is the control input, yi ∈ Rq is the mea-

sured output, and A, B, C are constant matrices with compatible dimensions.
In this section, the communication graph among the agents is represented
by an undirected graph G. It is assumed that each agent knows the relative
states, of its neighbors with respect to itself. Based on the relative information
of neighboring agents, we propose two types of distributed adaptive consensus
protocols, namely, the edge-based and node-based adaptive consensus proto-
cols.
The edge-based adaptive consensus protocol dynamically updates the cou-
pling weight for each edge (i.e., each communication link) and is given by
N
X
ui = K cij aij (xi − xj ),
j=1 (5.2)
T
ċij = κij aij (xi − xj ) Γ(xi − xj ), i = 1, · · · , N,

where aij denotes the (i, j)-th entry of the adjacency matrix associated with
G, cij (t) denotes the time-varying coupling weight for the edge (i, j) with
96 Cooperative Control of Multi-Agent Systems

cij (0) = cji (0), κij = κji are positive constants, and K ∈ Rp×n and Γ ∈ Rn×n
are the feedback gain matrices.
The node-based adaptive consensus protocol assigns a time-varying cou-
pling weight to each node (i.e., each agent) and is described by
N
X
ui = di K aij (xi − xj ),
j=1
(5.3)
N
X N
X
d˙i = ǫi [ aij (xi − xj )]T Γ[ aij (xi − xj )], i = 1, · · · , N,
j=1 j=1

where di (t) denotes the coupling weight for agent i, ǫi are positive constants,
and the rest of the variables are defined as in (5.2).
The adaptive protocols (5.2) and (5.3) are actually extensions of the stat-
ic protocol (2.2) in Chapter 2 by dynamically updating the coupling weights
of neighboring agents, which is equivalent to adaptively weighting the com-
munication graph. The edge-based adaptive protocol (5.2) assigns different
time-varying weights for different communication edges while the node-based
adaptive protocol (5.3) assigns the same time-varying weight for all the ingo-
ing edges of each node, as illustrated in FIGURE 5.1. For each time instant,
the adaptive protocols (5.2) and (5.3) are in more general forms of the stat-
ic protocol (2.2) with nonidentical constant weights on the communication
graph.

d6 (t)
1 6
1 c16 (t) 6 d1 (t)

d1 (t) d5 (t)
c12 (t) c15 (t) d2 (t) d1 (t)

d6 (t)
2 d (t) d4 (t) 5
2 c36 (t) c46 (t) 5 d3 (t) 6

d3 (t) d4 (t)
c23 (t) c24 (t) d2 (t) d2 (t)

3 4 3 4

(a) (b)

FIGURE 5.1: The adaptive consensus protocols (5.2) and (5.3) assign different
time-varying weights on a given graph: (a) the edge-based adaptive protocol;
(b) the node-based adaptive protocol.

In the adaptive protocols (5.2) and (5.3), it can be observed that the adap-
tive coupling weights cij and di are monotonically increasing and will converge
to constant values when consensus is reached. An intuitive explanation for the
Consensus Control Using Distributed Adaptive Protocols 97

reason why (5.2) and (5.3) may solve the consensus problem is that the cou-
pling weights cij and di may be adaptively tuned such that all the nonzero
eigenvalues of the Laplacian matrix of the weighted communication graph are
located within the unbounded consensus region as in Theorem 4 of Chapter 2
(since the coupling weights (5.2) and (5.3) vary very slowly when consensus is
nearly reached, the closed-loop network resulting from (2.1) and (5.2) or (5.3)
may be regarded as a linear time-invariant system). However, how to design
the feedback gain matrices of (5.2) and (5.3) and theoretically prove that they
can solve the consensus problem for the N agents in (2.1) is far from being
easy, which will be addressed in the following subsections.

5.1.1 Consensus Using Edge-Based Adaptive Protocols

In this subsection, we study the consensus problem of the agents in (5.1)
PN
under the edge-based adaptive protocol (5.2). Let ξi = xi − N1 j=1 xj and
ξ = [ξ1T , · · · , ξN
T T
] . Then, we get that
1 T
ξ = [(IN − 11 ) ⊗ In ]x. (5.4)
N
It is easy to see that 0 is a simple eigenvalue of IN − N1 11T with 1 as the
corresponding right eigenvector, and 1 is the other eigenvalue with multiplicity
N − 1. Then, it follows that ξ = 0 if and only if x1 = · · · = xN . Therefore,
the consensus problem under the adaptive protocol (5.2) is solved if and only
if ξ converges to zero. Hereafter we refer to ξ as the consensus error. Because
κij = κji and cij (0) = cji (0), it follows from (5.2) that cij (t) = cji (t), ∀ t ≥ 0.
Using (5.2) for (5.1), it follows that ξ and cij satisfy the following dynamics:
N
X
ξ˙i = Aξi + cij aij BK(ξi − ξj ),
j=1 (5.5)
ċij = κij aij (ξi − ξj )T Γ(ξi − ξj ), i = 1, · · · , N.
The following theorem presents a sufficient condition for designing (5.2) to
solve the consensus problem.
Theorem 29 For any given connected graph G, the N agents described by
(5.1) reach consensus under the edge-based adaptive protocol (5.2) with K =
−B T P −1 and Γ = P −1 BB T P −1 , where P > 0 is a solution to the following
linear matrix inequality (LMI):
AP + P AT − 2BB T < 0. (5.6)
Moreover, each coupling weight cij converges to some finite steady-state value.
Proof 29 Consider the Lyapunov function candidate
N N N
1 X T −1 X X (cij − α)2
V1 = ξi P ξi + , (5.7)
2 i=1 4κij
i=1 j=1,j6=i
98 Cooperative Control of Multi-Agent Systems

where α is a positive constant to be determined later. The time derivative of

V1 along the trajectory of (5.5) is given by
N
X N N
1X X cij − α
V̇1 = ξiT P −1 ξ˙i + ċij
2 i=1 κij
i=1 j=1,j6=i
N
X N
X
= ξiT P −1 [Aξi + cij aij BK(ξi − ξj )] (5.8)
i=1 j=1
N N
1X X
+ (cij − α)aij (ξi − ξj )T Γ(ξi − ξj ).
2 i=1
j=1,j6=i

Because cij (t) = cji (t), ∀ t ≥ 0, we have

N
X N
X
(cij − α)aij (ξi − ξj )T Γ(ξi − ξj )
i=1 j=1,j6=i
(5.9)
N X
X N
=2 (cij − α)aij ξiT Γ(ξi − ξj ).
i=1 j=1

Let ξ˜i = P −1 ξi and ξ˜ = [ξ˜1T , · · · , ξ˜N ] . Substituting K = −B T P −1 and Γ =

T T

P −1 BB T P −1 into (5.8), we can obtain

N
X N X
X N
V̇1 = ξiT P −1 Aξi − α aij ξiT P −1 BB T P −1 (ξi − ξj )
i=1 i=1 j=1
N
X N X
X N
1 (5.10)
= ξ˜iT (AP + P AT )ξ˜i − α Lij ξ˜iT BB T ξ˜j
2 i=1 i=1 j=1
1  
= ξ˜T IN ⊗ (AP + P AT ) − 2αL ⊗ BB T ξ,
˜
2
where Lij denotes the (i, j)-th entry of the Laplacian matrix L associated with
G.
˜ it is easy to see that (1T ⊗ I)ξ˜ = (1T ⊗
By the definitions of ξ and ξ,
−1
P )ξ = 0. Because G is connected, it then follows from Lemma 2 that
ξ˜T (L ⊗ I)ξ˜ ≥ λ2 (L)ξ˜T ξ,
˜ where λ2 (L) is the smallest nonzero eigenvalue of
L. Therefore, it follows from (5.10) that
1 ˜T  
˜
V̇1 ≤ ξ IN ⊗ (AP + P AT − 2αλ2 (L)BB T ξ, (5.11)
2
By choosing α sufficiently large such that αλ2 (L) ≥ 1, it follows from (5.6)
that
AP + P AT − 2αλ2 (L)BB T ≤ AP + P AT − 2BB T < 0.
Therefore, V̇1 ≤ 0.
Consensus Control Using Distributed Adaptive Protocols 99

Since V̇1 ≤ 0, V1 (t) is bounded, and so is each cij . By noting Γ ≥ 0, it can

be seen from (5.5) that cij is monotonically increasing. Then, it follows that
each coupling weight cij converges to some finite value. Let S = {ξ, ˜ cij |V̇1 =
0}. Note that V̇1 ≡ 0 implies that ξ˜i = 0, i = 1, · · · , N , which further implies
that ξ = 0. Hence, by Lemma 11 (LaSalle’s Invariance principle), it follows
that ξ(t) → 0 as t → ∞. That is, the consensus problem is solved.

Remark 30 Comparing the above theorem to Algorithm 2 in Chapter 2, it is

easy to see that the feedback gain matrix K of the adaptive protocol (5.2) is
identical to that of the static protocol (2.2). This is essentially thanks to the
decoupling feature of the consensus region approach, where the feedback gain
matrix is independent of the communication topology. As shown in Proposition
1 in Chapter 2, a sufficient condition for the existence of an adaptive proto-
col (5.2) satisfying Theorem 29 is that (A, B) is stabilizable. The consensus
protocol (5.2) can also be designed by solving the algebraic Riccati equation:
AT Q + QA + I − QBB T Q = 0, as in [171, 195]. In this case, the parameters
in (5.2) can be chosen as K = −B T Q and Γ = QBB T Q. The solvability of
the above Ricatti equation is equal to that of the LMI (5.6).

Remark 31 Equation (5.2) presents an adaptive protocol, under which the

agents in (5.1) can reach consensus for all connected communication topolo-
gies. In contrast to the static consensus protocol (2.2) in Chapter 2, the adap-
tive protocol (5.2) can be computed and implemented by each agent in a fully
distributed way without using any global information of the communication
graph. It should be mentioned that compared to the static protocol (2.2) in
Chapter 2, one disadvantage of the adaptive protocol (5.2) is that it is nonlin-
ear and has a higher dimension.

The following theorem presents the explicit expression for the final con-
sensus value.

Theorem 30 Suppose that the communication graph G is connected and the

adaptive protocol (5.2) satisfies Theorem 33. Then, the final consensus value
achieved by the agents in (5.1) is given by
N
1 X At
̟(t) , e xi (0). (5.12)
N i=1

Proof 30 Using (5.2) for (5.1), it follows that

ẋ = (IN ⊗ A + Lc ⊗ BK)x, (5.13)

where x = [xT1 , · · · , xTN ]T and Lc is defined as Lcij = −cij aij , i 6= j, and

PN
Lcii = j=1,j6=i cij aij . It is easy to see that Lc is also a symmetric Laplacian
matrix of the undirected graph G, with a time-varying weight cij aij for the
100 Cooperative Control of Multi-Agent Systems

edge (i, j). For a connected graph G, 0 is a simple eigenvalue of Lc with 1 as

the corresponding eigenvector, i.e., 1T Lc = 0.
Consider the quantity H = (1T ⊗e−At )x. By using (5.13), the time deriva-
tive of H is given by

Ḣ = −(1T ⊗ e−At A)x + (1T ⊗ e−At )(IN ⊗ A + Lc ⊗ BK)x

= (1T Lc ⊗ e−At BK)x ≡ 0.

which implies that H is an invariant quantity. Therefore,

(1T ⊗ e−At )(1 ⊗ ̟(t)) = (1T ⊗ In )x(0), (5.14)

from which we get the final consensus value ̟(t) in (5.12).

Remark 32 By comparing Theorem 30 to Theorem 1 in Chapter 2, it can be

found that under either the adaptive protocol (5.2) or the static linear proto-
col (2.2), the final consensus value achieved by the agents remains the same,
depending on the state matrix A, the Laplacian matrix L with unit weights,
and the initial states of the agents.

5.1.2 Consensus Using Node-Based Adaptive Protocols

This subsection considers the consensus problem of the agents in (5.1) under
the node-based adaptive protocol (5.3). Let the consensus error ξ be defined
as in (5.4). From (5.1) and (5.3), We can obtain that ξ and di satisfy the
following dynamics:
1
ξ˙ = [IN ⊗ A + ((IN − 11T )DL) ⊗ BK]ξ,
N
N
X N
X (5.15)
d˙i = ǫi [ Lij ξjT ]Γ[ Lij ξj ], i = 1, · · · , N,
j=1 j=1

where D(t) = diag(d1 (t), · · · , dN (t)).

The following result presents a sufficient condition for designing (5.3).

Theorem 31 Assume that the communication graph G is undirected and con-

nected. Then, the N agents in (5.1) reach consensus under the node-based
adaptive protocol (5.3) with K and Γ given as in Theorem 29. Moreover, each
coupling weight di converges to some finite steady-state value.

Proof 31 Consider the Lyapunov function candidate

XN
1 T (di − β)2
V2 = ξ (L ⊗ P −1 )ξ + , (5.16)
2 i=1
2ǫi

where β is a positive constant to be determined later. For a connected graph G,

Consensus Control Using Distributed Adaptive Protocols 101

it follows from Lemma 2 and the definition of ξ in (5.4) that ξ T (L ⊗ P −1 )ξ ≥

λ2 ξ T (IN ⊗ P −1 )ξ. Therefore, it is easy to see that Ωc = {ξ, di |V2 ≤ c} is
compact for any positive c.
Following similar steps to those in the proof of Theorem 29, we can obtain
the time derivative of V2 along the trajectory of (5.15) as
N
X di − β
V̇2 = ξ T (L ⊗ P −1 )ξ˙ + d˙i
i=1
τi
= ξ T [L ⊗ P −1 A + (LDL) ⊗ P −1 BK]ξ (5.17)
N
X N
X N
X
+ (di − β)( Lij ξjT )Γ( Lij ξj ),
i=1 j=1 j=1

where we have used the fact that L1 = 0. By noting that K = −B T P −1 , it is

easy to get that
N
X N
X N
X
ξ T (LDL ⊗ P −1 BK)ξ = − di ( Lij ξjT )P −1 BB T P −1 ( Lij ξj ). (5.18)
i=1 j=1 j=1

Substituting (5.18) into (5.17) yields

1 T
V̇2 = ξ [L ⊗ (P −1 A + AT P −1 ) − 2βL2 ⊗ P −1 BB T P −1 )ξ. (5.19)
2
Because G is connected, it follows from Lemma 2 that zero is a simple
eigenvalue of L and all the other eigenvalues are positive. Let U ∈ RN ×N
be such a unitary matrix that U T LU = Λ , diag(0, λ2 , · · · , λN ). Because the
right and left eigenvectors of L correspondingh to the
i zero eigenvalue are equal
1T
to 1, we can choose U = [ √1N Y1 ] and U T = √N , with Y1 ∈ RN ×(N −1) and
Y2
Y2 ∈ R(N −1)×N . Let ξ¯ , [ξ¯1T , · · · , ξ¯N ] = (U T ⊗ P −1 )ξ. By the definitions of
T T
¯ it is easy to see that
ξ and ξ,

1T
ξ¯1 = ( √ ⊗ P −1 )ξ = 0. (5.20)
N
Then, we have
1 ¯T
V̇2 = ξ [Λ ⊗ (AP + P AT ) − 2βΛ2 ⊗ BB T ]ξ¯
2
N (5.21)
1 X ¯T
= λi ξi (AP + P AT − 2βλi BB T )ξ¯i .
2 i=2

By choosing β sufficiently large such that βλi ≥ 1, i = 2, · · · , N , it follows

from (5.6) and (5.21) that V̇2 ≤ 0. Since V̇2 ≤ 0, V2 (t) is bounded and so is
each di . By noting Γ ≥ 0, it can be seen from (5.15) that di is monotonically
102 Cooperative Control of Multi-Agent Systems

increasing. Then, it follows that each coupling weight di converges to some

finite value. Note that V̇2 ≡ 0 implies that ξ¯i = 0, i = 2, · · · , N , which, by
noticing that ξ¯1 ≡ 0 in (5.20), further implies that ξ = 0. Hence, by Lemma 11
¯ → 0 and thereby ξ(t) → 0
(LaSalle’s Invariance principle), it follows that ξ(t)
as t → ∞.
Remark 33 Similar to the edge-based adaptive protocol (5.2), the node-based
adaptive protocol (5.3) can also be computed in a fully distributed fashion. In
the following, some comparisons between (5.2) and (5.3) are now briefly dis-
cussed. The dimension of the edge-based adaptive protocol (5.2) is proportional
to the number of edges in the communication graph. Since the number of edges
is usually larger than the number of nodes in a connected graph, the dimension
of the edge-based adaptive protocol (5.2) is generally higher than that of the
node-based protocol (5.3). On the other hand, the edge-based adaptive protocol
(5.2) is applicable to the case with switching communication graphs, which
will be shown in the following subsection. Moreover, the final consensus value
under the node-based adaptive protocol (5.3), different from the case of the
edge-based adaptive protocol (5.2), is not easy to derive, due to the asymmetry
of D(t)L.

5.1.3 Extensions to Switching Communication Graphs

In the last subsections, the communication graph is assumed to be fixed
throughout the whole process. However, the communication graph may change
with time in many practical situations due to various reasons, such as com-
munication constraints and link variations. In this subsection, the consensus
problem under the edge-based adaptive protocol (5.2) with switching commu-
nication graphs will be considered.
Denote by GN the set of all possible undirected connected graphs with
N nodes. Let σ(t) : [0, ∞) → P be a piecewise constant switching signal
with switching times t0 , t1 , · · · , and P be the index set associated with the
elements of GN , which is clearly finite. The communication graph at time t is
denoted by Gσ(t) . Accordingly, (5.2) becomes
N
X
ui = K cij aij (t)(xi − xj ),
j=1 (5.22)
ċij = κij aij (t)(xi − xj )T Γ(xi − xj ), i = 1, · · · , N,
where aij (t) is the (i, j)-th entry of the adjacency matrix associated with Gσ(t)
and the rest of the variables are the same as in (5.2).
Theorem 32 For arbitrary switching communication graphs Gσ(t) belonging
to GN , the N agents in (5.1) reach consensus under the edge-based protocol
(5.22) with K and Γ given as in Theorem 29. Besides, the coupling weights
cij converge to some finite values.
Consensus Control Using Distributed Adaptive Protocols 103

Proof 32 Let ξi and ξ be defined as in (5.4). By following similar steps to

those in the proof of Theorem 29, the consensus problem of the agents (5.1)
under the protocol (5.22) is solved if ξ converges to zero. Clearly, ξi and cij
satisfy the following dynamics:
N
X
ξ˙i = Aξi + cij aij (t)BK(ξi − ξj ),
j=1 (5.23)
T
ċij = κij aij (t)(ξi − ξj ) Γ(ξi − ξj ), i = 1, · · · , N.

Take a common Lyapunov function candidate

N N N
1 X T −1 X X c̃2ij
V3 = ξi P ξi + ,
2 i=1 4εij
i=1 j=1,j6=i

where c̃ij = cij − δ and δ is a positive scalar. Following similar steps as in the
proof of Theorem 29, we can obtain the time derivative of V3 along (5.23) as
1 ˜T  
˜
V̇3 = ξ IN ⊗ (AP + P AT ) − 2δLσ(t) ⊗ BB T ξ, (5.24)
2
where Lσ(t) is the Laplacian matrix associated with Gσ(t) and the rest of the
variables are the same as in (5.10). Since Gσ(t) is connected and (1T ⊗I)ξ˜ = 0,
it follows from Lemma 2 that ξ˜T (Lσ(t) ⊗ I)ξ˜ ≥ λmin ˜T ˜
2 ξ ξ, where λ2
min
,
minGσ(t) ∈GN {λ2 (Lσ(t) )} denotes the minimum of the smallest nonzero eigen-
values of Lσ(t) for all Gσ(t) ∈ GN . Therefore, we can get from (5.24) that

˜ , 1 ξ˜T [IN ⊗ (AP + P AT − 2δλmin BB T ]ξ.

V̇3 ≤ W (ξ) ˜
2
2
As shown in the proof of Theorem 29, by choosing δ > 0 sufficiently large such
that δλmin
2 ≥ 1, we have AP +P AT −2δλmin T
2 BB < 0. Therefore, V̇3 ≤ 0. Note
that V3 is positive definite and radically unbounded. By Lemma 14 (LaSalle-
Yoshizawa theorem), it follows that limt→∞ W (ξ)˜ = 0, implying that ξ(t)
˜ → 0,
as t → ∞, which further implies that ξ(t) → 0, as t → ∞. The convergence
of cij can be similarly shown as in the proof of Theorem 29.

Remark 34 Theorem 32 shows that the edge-based adaptive consensus pro-

tocol (5.2) given by Theorem 29 is applicable to arbitrary switching communi-
cation graphs which are connected at any time instant and the final consensus
value remains the same as in (5.12). Because the Lyapunov function in (5.16)
for the node-based adaptive protocol (5.3) is explicitly related with the commu-
nication graph, it cannot be taken as a feasible common Lyapunov function in
the case of switching topologies.
104 Cooperative Control of Multi-Agent Systems

5.2 Distributed Relative-Output Adaptive Consensus

Protocols
In the previous section, the adaptive consensus protocols (5.2) and (5.3) de-
pend on the relative state information of neighboring agents, which however
might not be available in some circumstances. In this section, we extend to
consider the case where that each agent knows the relative outputs, rather
than the relative states, of its neighbors with respect to itself. Based on the
relative output information of neighboring agents, distributed edge-based and
node-based adaptive dynamic consensus protocols are proposed in the follow-
ing.
The distributed edge-based adaptive observer-type consensus protocol is
given by
N
X
v̇i = (A + BF )vi + L cij aij [C(vi − vj ) − (yi − yj )],
j=1
 T   (5.25)
yi − y j yi − y j
ċij = εij aij Γ ,
C(vi − vj ) C(vi − vj )
ui = F v i , i = 1, · · · , N,

where vi ∈ Rn is the protocol state, i = 1, · · · , N , aij is the (i, j)-th entry

of the adjacency matrix associated with G, cij (t) denotes the time-varying
coupling weight for the edge (i, j) with cij (0) = cji (0), εij = εji are positive
constants, and F ∈ Rp×n , L ∈ Rn×q , and Γ ∈ R2q×2q are the feedback gain
matrices.
The node-based adaptive observer-type consensus protocol is described by
N
X
ṽ˙ i = (A + BF )ṽi + di L aij [C(ṽi − ṽj ) − (yi − yj )],
j=1
   
XN  T N
X   (5.26)
yi − yj yi − yj 
d˙i = τi  aij Γ aij ,
C(ṽi − ṽj ) C(ṽi − ṽj )
j=1 j=1

ui = F ṽi , i = 1, · · · , N,

where di (t) denotes the coupling weight for agent i, τi are positive constants,
and the rest of the variables are defined as in (5.25).
PN PN
Note that the terms j=1 cij aij C(vi −vj ) in (5.25) and j=1 aij C(ṽi − ṽj )
in (5.26) imply that the agents need to use the virtual outputs of the consensus
protocols from their neighbors via the communication topology G.
The adaptive dynamic protocols (5.25) and (5.26) are extensions of the
observer-type consensus protocol (2.21) in Chapter 2 by dynamically tun-
ing the coupling weights of the communication graph. Alternative distributed
Consensus Control Using Distributed Adaptive Protocols 105

adaptive consensus protocols based on the observer-type protocol (2.33) can

be similarly designed. The details are omitted here for conciseness.

5.2.1 Consensus Using Edge-Based Adaptive Protocols

In this subsection, we study the consensus problem of the agents in (5.1)
under the edge-based adaptive protocol (5.25). Let zi = [xTi , viT ]T , ζi = zi −
1
PN T T T T T T
N j=1 zj , z = [z1 , · · · , zN ] , and ζ = [ζ1 , · · · , ζN ] . Then, we get ζ =
[(IN − N1 11T ) ⊗ I2n ]z. Because 0 is a simple eigenvalue of IN − N1 11T with 1
as a corresponding eigenvector and 1 is the other eigenvalue with multiplicity
N − 1, it follows that ζ = 0 if and only if z1 = · · · = zN . Therefore, the
consensus problem of the agents in (5.1) under the protocol (5.25) is solved
if ζ converges to zero. Because εij = εji and cij (0) = cji (0), it follows from
(5.25) that cij (t) = cji (t), ∀ t ≥ 0. It is not difficult to obtain that ζi and cij
satisfy
X N
ζ̇i = Mζi + cij aij H(ζi − ζj ),
j=1 (5.27)
ċij = εij aij (ζi − ζj )T R(ζi − ζj ), i = 1, · · · , N,
where
   
A BF 0 0
M= , H= , R = (I2 ⊗ C T )Γ(I2 ⊗ C).
0 A + BF −LC LC

The following theorem designs the adaptive protocol (5.25) to achieve con-
sensus.

Theorem 33 Suppose that the communication graph G is undirected and

connected. Then, the N agents in (5.1) reach consensus under the edge-
based hadaptivei protocol (5.25) with F satisfying that A + BF is Hurwitz,
I −I
Γ = −Iqq Iqq , and L = −Q−1 C T , where Q > 0 is a solution to the fol-
lowing LMI:
AT Q + QA − 2C T C < 0. (5.28)
Moreover, the protocol states vi , i = 1, · · · , N , converge to zero and each
coupling weight cij converges to some finite steady-state value.

Proof 33 Consider the Lyapunov function candidate

N N N
1X T X X (cij − α)2
V4 = ζi Qζi + , (5.29)
2 i=1 4εij
i=1 j=1,j6=i
h i
−Q
where Q , ς Q̃+Q
−Q Q
, Q̃ > 0 satisfies that Q̃(A + BF ) + (A + BF )T Q̃ < 0,
and α and ς are positive constants to be determined later. In light of Lemma
19 (Schur Complement Lemma), it is easy to know that Q > 0.
106 Cooperative Control of Multi-Agent Systems

The time derivative of V4 along the trajectory of (5.27) can be obtained as

N
X N
X N
X cij − α
V̇4 = ζiT Qζ̇i + ċij
i=1 i=1 j=1,j6=i
2εij
N
X N
X
= ζiT Q[Mζi + cij aij H(ζi − ζj )] (5.30)
i=1 j=1
N N
1 XX
+ (cij − α)aij (ζi − ζj )T R(ζi − ζj ).
2 i=1 j=1
 I 0

Let ζ̃i = T ζi , i = 1, · · · , N, with T = −I I . Then, (5.30) can be rewritten as
N
X N X
X N
V̇4 = eM
ζ̃iT Q[ fζ̃i + e ζ̃i − ζ̃j )]
cij aij H(
i=1 i=1 j=1
(5.31)
N N
1 XX e ζ̃i − ζ̃j ),
+ (cij − α)aij (ζ̃i − ζ̃j )T R(
2 i=1 j=1

where
   
e = T −T QT −1 = ς Q̃ 0 , M
Q f = T MT −1 = A + BF BF ,
0 Q 0 A
   
e = T HT −1 = 0 0 , R
H e = T −T RT −1 = 0 0
.
0 LC 0 CT C

Because cij (t) = cji (t), ∀ t ≥ 0, we have

N X
X N
e ζ̃i − ζ̃j )
(cij − α)aij (ζ̃i − ζ̃j )T R(
i=1 j=1
(5.32)
N X
X N
=2 (cij − e ζ̃i
α)aij ζ̃iT R( − ζ̃j ).
i=1 j=1

eH
It is easy to see that Q e = −R.
e Then, by letting ζ̃ = [ζ̃ T , · · · , ζ̃ T ]T , it follows
1 N
from (5.31) and (5.32) that
N
X N X
X N
V̇4 = eM
ζ̃iT Q fζ̃i −α e ζ̃i − ζ̃j )
aij ζ̃iT R(
i=1 i=1 j=1 (5.33)
1 eMf+M
fT Q)
e − 2αL ⊗ R]
e ζ̃,
= ζ̃ T [IN ⊗ (Q
2
where L is the Laplacian matrix associated with G.
By the definitions of ζ and ζ̃, it is easy to see that (1T ⊗ I)ζ̃ =
Consensus Control Using Distributed Adaptive Protocols 107

(1T ⊗ T )ζ = 0. Because G is connected, it then follows from Lemma 2 that

ζ̃ T (L ⊗ I)ζ̃ ≥ λ2 (L)ζ̃ T ζ̃, where λ2 (L) is the smallest nonzero eigenvalue of L.
Therefore, we can get from (5.33) that
1 T eMf+M
fT Q
e − 2αλ2 (L)R)]
e ζ̃.
V̇4 ≤ ζ̃ [IN ⊗ (Q (5.34)
2
Note that
eM
Q f+MfT Q e − 2αλ2 (L)R
e
 
ς[Q̃(A + BF ) + (A + BF )T Q̃] ς Q̃BF (5.35)
= ,
ςF T B T Q̃ Π

where Π , QA + AT Q − 2αλ2 (L)C T C. By choosing α sufficiently large such

that αλ2 (L) ≥ 1, it follows from (5.28) that that Π < 0. Then, choosing ς > 0
sufficiently small and in virtue of Lemma 19, we can obtain from (5.35) and
(5.34) that Q eM
f+M fT Q e − 2αλ2 (L)R
e < 0 and V̇4 ≤ 0.
Since V̇4 ≤ 0, V1 (t) is bounded, implying that each cij is also bounded. By
noting that R ≥ 0, we can see from (5.27) that cij is monotonically increas-
ing. Then, it follows that each coupling weight cij converges to some finite
value. Note that V̇4 ≡ 0 implies that ζ̃ = 0 and ζ = 0. Hence, by Lemma 11
(LaSalle’s Invariance principle), it follows that ζ(t) → 0 as t → ∞. That is,
the consensus problem is solved. By (5.25) and noting the fact that A + BF
is Hurwitz, it is easy to see that the protocol states vi , i = 1, · · · , N , converge
to zero.

Remark 35 As shown in Proposition 3 of Chapter 2, a necessary and suffi-

cient condition for the existence of a Q > 0 to the LMI (5.28) is that (A, C)
is detectable. Therefore, a sufficient condition for the existence of an adap-
tive protocol (5.25) satisfying Theorem 33 is that (A, B, C) is stabilizable and
detectable. Because the distributed adaptive protocol (5.25) is essentially non-
linear, the separation principle which holds for linear multi-agent systems as in
Chapter 2 generally does not hold anymore. A favorable feature of the adaptive
protocol (5.25) is that its feedback gain matrices F , L, and Γ can be indepen-
dently designed. Similarly as shown in Theorem 32, it can be seen that the
edge-based adaptive protocol (5.25) designed in Theorem 33 is applicable to
arbitrary switching communication graphs which are connected at each time
instant.

5.2.2 Consensus Using Node-Based Adaptive Protocols

This subsection considers the consensus problem of the agents in (5.1) un-
der the node-based adaptive protocol (5.26). Let z̃i = [xTi , ṽiT ]T , ζ̄i =
PN
z̃i − N1 j=1 z̃j , and ζ̄ = [ζ̄1T , · · · , ζ̄N
T T
] . As shown in the last subsection,
the consensus problem of agents (5.1) under the protocol (5.26) is solved if ζ̄
108 Cooperative Control of Multi-Agent Systems

converges to zero. It is not difficult to obtain that ζ̄ and di satisfy the following
dynamics:
1
ζ̄˙ = [IN ⊗ M + ((IN − 11T )DL) ⊗ H]ζ̄,
N
N
X N
X (5.36)
d˙i = τi [ Lij ζ̄jT ]R[ Lij ζ̄j ], i = 1, · · · , N,
j=1 j=1

where D(t) = diag(d1 (t), · · · , dN (t)), M, H, and R are defined in (5.27).

Theorem 34 Assume that the communication graph G is undirected and con-

nected. Then, the N agents in (5.1) reach consensus under the node-based
adaptive protocol (5.26) with L, F , and Γ given as in Theorem 33. Moreover,
the protocol states ṽi , i = 1, · · · , N , converge to zero and each coupling weight
di converges to some finite steady-state value.

Proof 34 Consider the Lyapunov function candidate

XN
1 T (di − β)2
V5 = ζ̄ (L ⊗ Q)ζ̄ + , (5.37)
2 i=1
2τi

where Q is defined in (5.29) and β is a positive constant to be determined

later. For a connected graph G, it follows from Lemma 2 and the definition
of ζ̄ that ζ̄ T (L ⊗ Q)ζ̄ ≥ λ2 (L)ζ̄ T (IN ⊗ Q)ζ̄. Therefore, it is easy to see that
Ωc = {ζ̄, di |V5 ≤ c} is compact for any positive c.
Following similar steps to those in the proof of Theorem 33, we can obtain
the time derivative of V5 along the trajectory of (5.36) as

eM
V̇5 = ζ̂ T [L ⊗ Q f − LDL ⊗ R]
e ζ̂
N
X N
X N
X (5.38)
+ (di − β)( e
Lij ζ̂jT )R( Lij ζ̂j ),
i=1 j=1 j=1

T T
where ζ̂ , [ζ̂1T , · · · , ζ̂N e M,
] = (IN ⊗ T )ζ̄, T , Q, f and R
e are the same as in
(5.31). Observe that
N
X N
X N
X
e ζ̂ =
ζ̂ T (LDL ⊗ R) di ( e
Lij ζ̂jT )R( Lij ζ̂j ). (5.39)
i=1 j=1 j=1

Substituting (5.39) into (5.38) yields

1 T eMf+M
fT Q
eT ) − 2βL2 ⊗ R]
e ζ̂.
V̇5 = ζ̂ [L ⊗ (Q (5.40)
2
Because G is connected, let unitary matrices U and U T be defined as in the
proof of Theorem 31 such that U T LU = Λ , diag(0, λ2 , · · · , λN ), where λ2 ≤
· · · ≤ λN are the nonzero eigenvalues of L. Let ζ̆ , [ζ̆1T , · · · , ζ̆N
T T
] = (U T ⊗I)ζ̂.
Consensus Control Using Distributed Adaptive Protocols 109
T
By the definitions of ζ̆ and ζ̂, it is easy to see that ζ̆1 = ( √1N ⊗ T )ζ = 0. Then,
it follows from (5.40) that
1 T eMf+MfT Q)
e − 2βΛ2 ⊗ R] e ζ̆
V̇5 = ζ̆ [Λ ⊗ (Q
2
N (5.41)
1X eMf+MfT Q
e − 2βλi R)
e ζ̆i .
= λi ζ̆iT (Q
2 i=2

As shown in the proof of Theorem 33, by choosing ς > 0 sufficiently small

and β sufficiently large such that βλ2 (L) ≥ 1, we can obtain from (5.41) that
V̇5 ≤ 0. Note that V̇5 ≡ 0 implies that ζ̆i = 0, i = 2, · · · , N , which, together
with ζ̆1 = 0, further implies that ζ = 0. Therefore, it follows from Lemma 11
(Lasalle’s Invariance principle) that ζ → 0 as t → ∞. The convergence of di
and ṽi , i = 1, · · · , N , can be shown by following similar steps in the proof of
Theorem 33, which is omitted here for brevity.

Remark 36 The adaptive consensus protocols (5.2), (5.3), (5.25), and (5.26)
are extensions of their counterparts with constant coupling weights in Chapter
2 by using adaptive coupling weights. Compared to the consensus protocols
in Chapter 2 whose design generally requires the knowledge of the smallest
nonzero eigenvalue of the Laplacian matrix, one advantage of these adaptive
consensus protocols in the current chapter is that they depend on only local
information, which thereby are fully distributed and scalable in the sense that
new agents can be added or existing agents can be removed without redesigning
the protocol. As mentioned earlier, the reason why this can be done lies in the
decoupling property of the consensus region approach.

5.2.3 Simulation Examples

In the following, a numerical example is presented for illustration.

Example 17 Consider a network of third-order integrators, described by

(5.1), with    
0 1 0 0  
A = 0 0 1 , B = 0 , C = 1 0 0 .
0 0 0 1
Choose F = − [ 3 6.5 4.5 ] such that A + BF is Hurwitz. Solving the LMI (5.28)
by using the SeDuMi toolbox [158] gives the feedback gain matrix L in (5.25)
T
and (5.26) as L = − [ 2.1115 1.3528 0.6286 ] .
To illustrate Theorem 37, let the communication graph be G1 in FIGURE
5.2(a) and τi = 1, i = 1, · · · , 6, in (5.3). The consensus errors xi − x1 ,
i = 2, · · · , 6, of the third-order
 1 −1  integrators under the protocol (5.3) with F and
L as above and Γ = −1 1 are depicted in in FIGURE 5.3(a). The coupling
weights di associated with the nodes are drawn in FIGURE 5.3(b).
110 Cooperative Control of Multi-Agent Systems

1 6 1 6

2 5 2 5

3 4 3 4

(a) G1 (b) G2

FIGURE 5.2: Undirected communication graphs G1 and G2 .

To illustrate that Theorem 34 is applicable to switching graphs, let Gσ(t)

switch randomly every 0.1 second between G1 and G2 as shown in FIGURE 5.2.
Note that both G1 and G2 are connected. Let εij = 1, i, j = 1, · · · , 6, in (5.25),
and cij (0) = cji (0) be randomly chosen. The consensus errors xi − x1 , i =
2, · · · , 6, under the protocol (5.25) designed as above are depicted in FIGURE
5.4(a). The coupling weights cij associated with the edges in this case are
shown in FIGURES 5.4(b). FIGURES 5.3(a) and 5.4(a) state that consensus
is indeed achieved in both cases. From FIGURES 5.3(b) and 5.4(b), it can be
observed that the coupling weights converge to finite steady-state values.

10 3

8 2.5

6 2

4
1

1.5
x −x

i
d
i

2 1

0 0.5

−2 0

−4 −0.5
0 5 10 15 20 25 30 0 5 10 15 20 25 30
time (s) time (s)

(a) (b)

FIGURE 5.3: (a) The consensus errors xi −x1 of third-order integrators under
(5.26); (b) the coupling weights di in (5.26).
Consensus Control Using Distributed Adaptive Protocols 111

6 2.5

4 2

2
1.5

0
1
xi−x1

−2

ij
0.5

c
−4
0
−6

−0.5
−8

−10 −1

−12 −1.5
0 5 10 15 20 25 30 0 5 10 15 20 25 30
time (s) time (s)

(a) (b)

FIGURE 5.4: (a) The consensus errors xi −x1 of third-order integrators under
(5.25); (b) the coupling weights cij in (5.25).

5.3 Extensions to Leader-Follower Graphs

For undirected communication graphs in the previous sections, the final con-
sensus value reached by the agents under the adaptive protocols (5.2) and
(5.3), which depends on the initial values and the agent dynamics, might be
unknown a priori. In some cases, it might be desirable for the agents’ states
to converge onto a reference trajectory. This is actually the leader-follower
consensus problem.
In this section, we intend to consider the case where the N agents in (5.1)
maintain a leader-follower communication graph G. Without loss of generality,
assume that the agent indexed by 1 is the leader and the agents indexed by
2, · · · , N , are followers. The leader does not receive any information from
the followers, i.e., it has no neighbor. For simplicity, the control input of the
leader is assumed to be zero, i.e., u1 = 0. The case with nonzero u1 will be
investigated in the next chapter.
In the sequel, the following assumption is needed.

Assumption 5.1 The subgraph associated with the followers is undirected

and the graph G contains a directed spanning tree with the leader as the root.

Denote by L the Laplacian matrix associated hwith G. Because

i the leader
0 01×(N −1)
has no neighbors, L can be partitioned as L = L2 L1
, where L2 ∈
R(N −1)×1 and L1 ∈ R(N −1)×(N −1) is symmetric.
It is said that the leader-follower consensus problem is solved if the states of
the followers converge to the state of the leader, i.e., limt→∞ kxi (t) − x1 (t)k =
0, ∀ i = 2, · · · , N .
Similarly as in the previous sections, both the edge-based and node-based
112 Cooperative Control of Multi-Agent Systems

adaptive consensus protocols can be brought out to solve the leader-follower

consensus problem. For conciseness, only the case with relative state informa-
tion is discussed in this section. The case with relative output information can
be similarly done, which will be also briefly mentioned in the latter part of
the next chapter.
The edge-based adaptive consensus protocol is proposed for the followers
as
XN
ui = K cij aij (xi − xj ),
j=1 (5.42)
ċij = κij aij (xi − xj )T Γ(xi − xj ), i = 2, · · · , N,
where K ∈ Rp×n and Γ ∈ Rn×n are the feedback gain matrices, cij is the
coupling weight associated with the edge (i, j) with cij (0) = cji (0) for i =
2, · · · , N , j = 1, · · · , N , κij = κji > 0, and the rest of the variables are
defined in (5.3).

Theorem 35 For any graph G satisfying Assumption 5.1, the N agents in

(5.1) reach leader-follower consensus under the edge-based protocol (5.42) with
K = −B T P −1 and Γ = P −1 BB T P −1 , where P > 0 is a solution to (5.6) and
the coupling weights cij converge to finite steady-state values.

Proof 35 Let the consensus error υi = xi − x1 , i = 2, · · · , N . Then, we can

get from (5.1) and (5.42) that
N
X
υ̇i = Aυi + cij aij K(υi − υj ) + ci1 ai1 Kυi ,
j=2
(5.43)
ċi1 = ǫi1 ai1 υiT Γυi ,
ċij = ǫij aij (υi − υj )T Γ(υi − υj ), i = 2, · · · , N.

Clearly, the leader-follower consensus problem of (5.1) is solved by (5.42) if

the states υi of (5.43) converge to zero.
Consider the Lyapunov function candidate
N N N N
1 X T −1 X X (cij − β)2 X (ci1 − β)2
V6 = υi P υi + + ,
2 i=2 4κij 2κi1
i=2 j=2,j6=i i=2

where β is a positive constant. Following similar steps as in the proof of The-

Consensus Control Using Distributed Adaptive Protocols 113

orem 29, the time derivative of V6 along (5.43) can be obtained as

N
X N
X N
X N
X (ci1 − β)
cij − β
V̇6 = υiT P −1 υ̇i + ċij + ċi1
i=2 i=2 j=2,j6=i
2κij i=2
κi1
N
X N
X
= υiT P −1 [Aυi + cij aij K(υi − υj ) + ci1 ai1 Kυi ]
i=2 j=2
(5.44)
N N
1X X
+ (cij − β)aij (υi − υj )T Γ(υi − υj )
2 i=2
j=2,j6=i
N
X
+ (ci1 − β)ai1 υiT Γυi .
i=2

Similarly to (5.9), the following holds:

N
X N
X
(cij − β)aij (υi − υj )T Γ(υi − υj )
i=2 j=2,j6=i
N X
X N
=2 (cij − β)aij υiT Γ(υi − υj ).
i=2 j=2

Then, we can get from (5.44) that

1 T
V̇6 = υ [IN −1 ⊗ (P −1 A + AT P −1 ) − 2βL1 ⊗ Γ]υ,
2
T T
where υ = [υ2T , · · · , υN ] . For any graph G satisfying Assumption 5.2, it fol-
lows from Lemma 2 that L1 > 0. Thus υ T (L1 ⊗ Γ)υ ≥ λ2 (L)υ T (IN −1 ⊗ Γ)υ,
with λ2 (L) being the smallest eigenvalue of L1 . Then, we have
1 T
V̇6 ≤ υ [IN −1 ⊗ (P −1 A + AT P −1 − 2βλ2 (L)Γ]υ.
2
The rest of the proof is similar to that of Theorem 29, which is omitted for
brevity.

Besides, the node-based adaptive consensus protocol is proposed for each

follower as
N
X
ui = di K aij (xi − xj ),
j=1
(5.45)
N
X N
X
d˙i = ǫi [ T
aij (xi − xj ) ]Γ[ aij (xi − xj )], i = 2, · · · , N,
j=1 j=1

where d¯i denotes the coupling weight associated with follower i and ǫi > 0.
114 Cooperative Control of Multi-Agent Systems

Theorem 36 For any graph G satisfying Assumption 5.1, the N agents in

(5.1) reach leader-follower consensus under the node-based protocol (5.45) with
K and Γ given as in Theorem 35. Besides, each coupling weight di converges
to some finite steady-state value.

Proof 36 Clearly, the leader-follower consensus problem of (5.1) under

(5.45) is solved if the consensus error υi , i = 2, · · · , N , defined as in the
proof of Theorem 35, converge to zero. It is easy to see that υi and d¯i satisfy
N
X
υ̇i = Aυi + di K Lij υj ,
j=2
N
X N
X
d˙i = ǫi [ Lij υj ]T Γ[ Lij υj ], i = 2, · · · , N,
j=2 j=2

where Lij denotes the (i, j)-th entry of the Laplacian matrix L.
Consider the Lyapunov function candidate
N N
1 X T −1 X (di − σ)2
V7 = υi P υi + ,
2 i=2 i=2
2ǫi

where σ > 0 is to be determined. The rest of the proof can be completed by

following similar steps in proving Theorem 36.

5.4 Robust Redesign of Distributed Adaptive Protocols

In this section, we investigate the robustness of the proposed adaptive con-
sensus protocols (5.2) and (5.3) with respect to external disturbances. The
dynamics of the agents subject to external disturbances are described by

ẋi = Axi + Bui + ωi , i = 1, · · · , N, (5.46)

where xi ∈ Rn is the state, ui ∈ Rp is the control input, and ωi ∈ Rn

denotes external disturbances associated with the i-th agent, which satisfies
the following assumption.

Assumption 5.2 There exist positive constants θi such that kωi k ≤ θi , i =

1, · · · , N .

The communication graph among the N agents is represented by an undi-

rected graph G, which is assumed to be connected throughout this section.
It is worth noting that due to the existence of nonzero ωi , the relative
states under the adaptive protocols (5.2) and (5.3) generally will not converge
Consensus Control Using Distributed Adaptive Protocols 115

to zero any more but rather can only be expected to converge into some small
neighborhood of the origin. Since the coupling weights cij and di are integrals
of the nonnegative quadratic functions of the relative states, it is easy to see
that in this case cij and di will grow unbounded, which is called the parameter
drift phenomenon in the classic adaptive control literature [63]. Therefore, the
adaptive protocols (5.2) and (5.3) are fragile in the presence of the bounded
disturbances ωi .
The main objective of this section is to make modifications on (5.2) and
(5.3) in order to present some distributed robust adaptive protocols which can
guarantee the ultimate boundedness of the consensus error and the coupling
weights for the agents in (5.46) in the presence of bounded disturbances ωi .
Motivated by the σ-modification technique in the robust adaptive control
literature [63], we propose a modified edge-based adaptive protocol as follows:
N
X
ui = cij aij K(xi − xj ),
j=1 (5.47)
ċij = κij aij [−φij cij + (xi − xj )T Γ(xi − xj )], i = 1, · · · , N,

where φij , i, j = 1, · · · , N , are small positive constants and the rest of the
variables are defined as in (5.2).
Similarly, a modified node-based adaptive protocol can be described by
N
X
ui = di aij K(xi − xj ),
j=1
N
X N
X
d˙i = τi [−ϕi di + ( aij (xi − xj )T )Γ( aij (xi − xj ))], i = 1, · · · , N,
j=1 j=1
(5.48)
where ϕi , i = 1, · · · , N , are small positive constants and the rest of the vari-
ables are defined as in (5.3).

5.4.1 Robust Edge-Based Adaptive Protocols

In this subsection, we study the consensus problem of the agents in (5.46)
under the modified edge-based adaptive protocol (5.47). Let the consensus
error ξ be defined as in (5.4). From (5.46) and (5.47), it is not difficult to
obtain that ξi and cij satisfy
N
X N
1 X
ξ˙i = Aξi + cij aij BK(ξi − ξj ) + ωi − ωj ,
j=1
N j=1 (5.49)
T
ċij = κij aij [−φij cij + (ξi − ξj ) Γ(ξi − ξj )], i = 1, · · · , N.

The following theorem presents a sufficient condition to achieve consensus

for (5.49).
 116 Cooperative Control of Multi-Agent Systems

Theorem 37 Suppose that G is undirected and connected and Assumption 5.2

holds. The feedback gain matrices of the modified edge-based adaptive protocol
(5.47) are designed as K = −B T Q−1 and Γ = Q−1 BB T Q−1 , where Q > 0 is
a solution to the following LMI:

AQ + QAT + εQ − 2BB T < 0, (5.50)

where ε > 1. Then, both the consensus error ξ and the adaptive gains cij
i, j = 1, · · · , N , in (5.49) are uniformly ultimately bounded and the following
statements hold.

i) For any φij , the parameters ξ and cij exponentially converge to the residual
set
N
X N N
1 α2 X X
D1 , {ξ, cij : V8 < θi2 + φij aij }, (5.51)
2δλmax (Q) i=1
4δ i=1 j=1

where δ , mini,j=1,··· ,N {ε − 1, κij φij } and

N N N
1 X T −1 X X c̃2ij
V8 = ξi Q ξi + , (5.52)
2 i=1 4κij
i=1 j=1,j6=i

where c̃ij = cij − α, α is a positive scalar satisfying α ≥ λ12 , and λ2 is the

smallest nonzero eigenvalue of the Laplacian matrix L associated with G.

ii) If φij is chosen such that ̺ , mini,j=1,··· ,N {κij φij } < ε−1, then in addition
to i), ξ exponentially converges to the residual set

XN N N
λmax (Q) 1 α2 X X
D2 , {ξ : kξk2 ≤ [ θi2 + φij aij ]}. (5.53)
ε − 1 − ̺ λmin (Q) i=1 2 i=1 j=1

Proof 37 Choose the Lyapunov function candidate as in (5.52), which clearly

is positive definite. The time derivative of V8 along the trajectory of (5.49) is
given by
N
X N
X N
X c̃ij ˙
V̇8 = ξiT Q−1 ξ˙i + c̃ij
i=1 i=1 j=1,j6=i
2κij
N
X N
X N
1 X
= ξiT Q−1 [Aξi + (c̃ij + α)aij BF (ξi − ξj ) + ωi − ωj ] (5.54)
i=1 j=1
N j=1
N
X N
X
1
+ c̃ij aij [−φij (c̃ij + α) + (ξi − ξj )T Γ(ξi − ξj )].
2 i=1 j=1,j6=i
Consensus Control Using Distributed Adaptive Protocols 117

In light of (5.9), substituting K = −B T Q−1 and Γ = Q−1 BB T Q−1 into

(5.54) yields
N N
1 T 1 XX
V̇8 ≤ ξ X ξ + ξ T (M ⊗ Q−1 )ω + φij aij (−c̃2ij + α2 ), (5.55)
2 4 i=1 j=1

where ω = [ω1T , · · · , ωN ] , M = IN − N1 11T , X , IN ⊗ (Q−1 A + AT Q−1 ) −

T T

2αL⊗Q BB Q , and we have used the fact that −c̃2ij −c̃ij α ≤ − 21 c̃2ij + 12 α2 .
−1 T −1

Using the following fact:

1 1
− ξ T (M ⊗ Q−1 )ξ + ξ T (M ⊗ Q−1 )ω − ω T (M ⊗ Q−1 )ω
2 2 (5.56)
1 T −1
= − (ξ − ω) (M ⊗ Q )(ξ − ω) ≤ 0,
2
we can get from (5.55) that

X N
1 T 1
V̇8 ≤ ξ (X + M ⊗ Q−1 )ξ + θ2
2 2λmin (Q) i=1 i
N N
(5.57)
1 XX
+ φij aij (−c̃2ij + α2 ).
4 i=1 j=1

1
PN
where we have used the assertion that ω T (M ⊗ Q−1 )ω ≤ λmin (Q)
2
i=1 θi .
Note that (5.57) can be rewritten into
1
V̇8 ≤ −δV8 + δV8 + ξ T (X + M ⊗ Q−1 )ξ
2
XN N N
1 1 XX
+ θi2 + φij aij (−c̃2ij + α2 )
2λmin (Q) i=1 4 i=1 j=1

X N (5.58)
1 1
= −δV8 + ξ T [X + (δIN + M ) ⊗ Q−1 ]ξ + θ2
2 2λmin (Q) i=1 i
N N N N
1 XX δ α2 X X
− (φij − )aij c̃2ij + φij aij .
4 i=1 j=1 κij 4 i=1 j=1

By noting that δ ≤ mini,j=1,··· ,N {κij φij }, it follows from (5.58) that

1
V̇8 ≤ −δV8 + ξ T [X + (δIN + M ) ⊗ Q−1 ]ξ
2
1 XN N N
α2 X X (5.59)
+ θi2 + φij aij .
2λmin (Q) i=1 4 i=1 j=1

Because G is connected, it follows from Lemma 2 that zero is a simple

118 Cooperative Control of Multi-Agent Systems

eigenvalue of L and all the other eigenvalues are positive. By the definitions
of M and L, we have M L = L = LM. Thus there exists a unitary matrix U ∈
RN ×N such that U T M U and U T LU are both diagonal [58]. Since L and M
have the same right and left eigenvectors corresponding
h 1T i to the zero eigenvalue,
namely, 1, we can choose U = [ √1N Y ] , U T = √N , with Y ∈ RN ×(N −1) ,
W
W ∈ R(N −1)×N , satisfying
 
0 0
UT MU = Π = ,
0 IN −1 (5.60)
T
U LU = Λ = diag(0, λ2 , · · · , λN ),

where λi , i = 2, · · · , N , are the nonzero eigenvalues of L. Let ξ¯ ,

[ξ¯1T , · · · , ξ¯N ¯ it is easy to
] = (U T ⊗ Q−1 )ξ. By the definitions of ξ and ξ,
T T
T
see that ξ¯1 = ( √N ⊗ Q )ξ = 0. Then, by using (5.60), it follows that
1 −1

N
X
ξ T [X + (δIN + M ) ⊗ Q−1 ]ξ = ξ¯iT [AQ + QAT + (δ + 1)Q − 2αλi BB T ]ξ¯i
i=2
N
X
≤ ξ¯iT [AQ + QAT + εQ − 2BB T ]ξ¯i ≤ 0,
i=2
(5.61)
where we have used the fact that αλi ≥ 1, i = 2, · · · , N , and δ ≤ ε − 1. Then,
it follows from (5.59) and (5.61) that
N
X N N
1 α2 X X
V̇8 ≤ −δV8 + θi2 + φij aij . (5.62)
2λmin (Q) i=1
4 i=1 j=1

By using Lemma 16 (the Comparison lemma), we can obtain from (5.62) that
N
X N N
1 α2 X X
V8 ≤ [V8 (0) − θi2 − φij aij ]e−δt
2δλmin (Q) i=1
4δ i=1 j=1
(5.63)
N
X N N
1 α2 X X
+ θi2 + φij aij .
2δλmin (Q) i=1
4δ i=1 j=1

Therefore, V8 exponentially converges to the residual set D1 in (5.51) with

a convergence rate faster than e−δt , implying that ξ and cij are uniformly
ultimately bounded.
For the case where ̺ , mini,j=1,··· ,N {κij φij } < ε − 1, we can obtain a
Consensus Control Using Distributed Adaptive Protocols 119

smaller residual set for ξ by rewriting (5.57) into

N N
1 T −1 α2 X X
V̇8 ≤ −̺V8 + ξ [X + ((ε − 1)IN + M ) ⊗ Q ]ξ + φij aij
2 4 i=1 j=1

X N
ε−1−̺ T 1
− ξ (IN ⊗ Q−1 )ξ + θ2
2 2λmin (Q) i=1 i
N N
(5.64)
1 XX ̺
− (φij − )aij c̃2ij
4 i=1 j=1 κij

XN N N
ε−1−̺ 1 α2 X X
≤ −̺V8 − kξk2 + θi2 + φij aij .
2λmax (Q) 2λmin (Q) i=1 4 i=1 j=1

λmax (Q) 1
PN α2
PN PN
Obviously, if kξk2 > ε−1−̺ [ λmin (Q)
2
i=1 θi + 2 i=1 j=1 φij aij ], it fol-
lows from (5.64) that V̇8 < 0, which implies that ξ exponentially converges to
the residual set D2 in (5.53) with a convergence rate faster than e−̺t .

Remark 37 As shown in Proposition 2 in Chapter 2, there exists a Q >

0 satisfying (5.28) if and only if (A, B) is controllable. Thus, a sufficient
condition for the existence of (5.47) satisfying Theorem 37 is that (A, B) is
controllable, which, compared to the existence condition of (5.2) satisfying
Theorem 29, is more stringent. Theorem 37 shows that the modified edge-
based adaptive protocol (5.47) can ensure the boundedness of the consensus
error ξ and the adaptive gains cij for the agents described by (5.46). That is,
(5.47) is indeed robust in the presence of bounded external disturbances.

Remark 38 Note that the residual sets D1 and D2 in Theorem 37 depends

on the communication graph G, the upper bound of the external disturbances
ωi , the parameters Q, φij , and κij of the adaptive protocol (5.47). For small
φij satisfying min κij φij ≤ ε − 1, we can observed from (5.53) that smaller φij
implies a smaller bound for the consensus error ξ and from (5.51) that smaller
φij meanwhile yields a larger bound for coupling gains cij . For the case where
φij = 0, cij will tend to infinity. In real implementations, we should choose
relatively small φij in order to guarantee a small ξ and have tolerable cij .

5.4.2 Robust Node-Based Adaptive Protocols

In this subsection, we study the consensus problem of the agents in (5.46)
under the modified node-based adaptive protocol (5.48). Let the consensus
error ξ be defined as in (5.4). From (5.46) and (5.47), we can obtain that ξ
 120 Cooperative Control of Multi-Agent Systems

and di satisfy
ξ˙ = (IN ⊗ A + M DL ⊗ BK)ξ + (M ⊗ In )ω,
N
X N
X (5.65)
d˙i = τi [−ϕi di + ( Lij ξjT )Γ( Lij ξj )], i = 1, · · · , N,
j=1 j=1

1 T
where D = diag(d1 , · · · , dN ) and M = IN − N 11 .

Theorem 38 Suppose that G is connected and Assumption 5.2 holds. Both

the consensus error ξ and the adaptive gains di i = 1, · · · , N , in (5.65) are
uniformly ultimately bounded under the modified node-based adaptive protocol
(5.48) with K and Γ given as in Theorem 37. Moreover, the following two
statements hold.
i) For any ϕi , the parameters ξ and di exponentially converge to the residual
set
N N
λmax (L) X 2 α2 X
D3 , {ξ, di : V9 < θi + ϕi }, (5.66)
2δλmin (Q) i=1 2δ i=1
where δ , mini=1,··· ,N {ε − 1, τi ϕi } and
XN
1 T d˜2i
V9 = ξ (L ⊗ Q−1 )ξ + , (5.67)
2 i=1
2τi

where d˜i = di − α, and Q and α are the same as in Theorem 37.

ii) If small ϕi satisfies ϑ , mini=1,··· ,N {τi ϕi } < ε − 1, then in addition to i), ξ
exponentially converges to the residual set
N N
λmax (Q) λmax (L) X 2 X
D4 , {ξ : kξk2 ≤ [ θi + α 2 ϕi ]}. (5.68)
(ε − 1 − ϑ)λ2 λmin (Q) i=1 i=1

Proof 38 Choose the Lyapunov function candidate as in (5.67). The time

derivative of V9 along the trajectory of (5.65) can be obtained as
XN ˜
di ˜˙
V̇9 = ξ T (L ⊗ Q−1 )ξ˙ + di
τ
i=1 i
= ξ T (L ⊗ Q−1 A + LDL ⊗ Q−1 BK)ξ + ξ T (L ⊗ Q−1 )ω (5.69)
N
X N
X N
X
+ d˜i [−ϕi (d˜i + α) + ( Lij ξjT )Γ( Lij ξj )].
i=1 j=1 j=1

By using (5.18) and the assertion that −d˜2i − d˜i α ≤ − 12 d˜2i + 12 α2 , it follows
from (5.69) that
N
1 T 1X
V̇9 ≤ ξ Yξ + ξ T (L ⊗ Q−1 )ω − ϕi (d˜2i − α2 ), (5.70)
2 2 i=1
Consensus Control Using Distributed Adaptive Protocols 121

where Y , L⊗(Q−1 A+AT Q−1 )−2αL2 ⊗Q−1 BB T Q−1 . By using the technique
of completing the square as in (5.56), we can obtain from (5.70) that
N
1 1 1X
V̇9 ≤ ξ T (Y + L ⊗ Q−1 )ξ + ω T (L ⊗ Q−1 )ω − ϕi (d˜2i − α2 )
2 2 2 i=1
(5.71)
N N
1 λmax (L) X 2 1 X
≤ ξ T (Y + L ⊗ Q−1 )ξ + θ − ϕi (d˜2i − α2 ),
2 2λmin (Q) i=1 i 2 i=1

Note that (5.71) can be rewritten into

1
V̇9 ≤ −δV9 + ξ T [Y + (δ + 1)L ⊗ Q−1 ]ξ
2
N N N
λmax (L) X 2 1 X δ α2 X
+ θi − (ϕi − )d˜2i + ϕi
2λmin (Q) i=1 2 i=1 τi 2 i=1
(5.72)
1
≤ −δV9 + ξ T [Y + (δ + 1)L ⊗ Q−1 ]ξ
2
N N
λmax (L) X 2 α2 X
+ θi + ϕi ,
2λmin (Q) i=1 2 i=1

where we have used τi ϕi ≤ δ, i = 1, · · · , N , to get the last inequality. Letting

ξ¯ be defined as in the proof of Theorem 37, we have
N
X
ξ T [Y + (δ + 1)L ⊗ Q−1 ]ξ = λi ξ¯iT [AQ + QAT + (δ + 1)Q − 2αλi BB T ]ξ¯i
i=2
N
X
≤ λi ξ¯iT [AQ + QAT + εQ − 2BB T ]ξ¯i ≤ 0.
i=2
(5.73)
Then, it follows from (5.72) and (5.73) that
N N
λmax (L) X 2 α2 X
V̇9 ≤ −δV9 + θi + ϕi . (5.74)
2λmin (Q) i=1 2 i=1

Therefore, by using Lemma 16 (the Comparison lemma), we get from (5.74)

that V9 exponentially converges to the residual set D3 in (5.66) with a rate
λ2 PN d˜2i
faster than e−δt , which, in light of V9 ≥ 2λmax (Q) kξk +
2
i=1 2τi , further
implies that ξ and di are uniformly ultimately bounded.
Furthermore, for the case where ϑ , mini=1,··· ,N {τi ϕi } < ε − 1, we rewrite
122 Cooperative Control of Multi-Agent Systems

(5.71) into

V̇9 ≤ −ϑV9 + ξ T [Y + εL ⊗ Q−1 ]ξ

N N
ε−1−ϑ T λmax (L) X 2 α2 X
− ξ (L ⊗ Q−1 )ξ + θi + ϕi
2 2λmin (Q) i=1 2 i=1 (5.75)
N
X N
2 X
(ε − 1 − ϑ)λ2 λmax (L) α
≤ −ϑV2 − kξk2 + θi2 + ϕi .
2λmax (Q) 2λmin (Q) i=1
2 i=1

λmax (Q) λmax (L) PN PN

Obviously, if kξk2 > (ε−1−ϑ)λ2 [ λmin (Q)
2
i=1 θi + α2 i=1 ϕi ], it follows from
λ2
(5.75) that V̇9 ≤ −ϑV9 . Then, by noting V9 ≥ 2λmax (Q) kξk2 , we can get that
ξ exponentially converges to the residual set D4 in (5.68) with a rate faster
than e−ϑt .

Remark 39 Similar to the modified edge-based adaptive protocol (5.47), the

modified node-based adaptive protocol (5.48) can be also implemented in a fully
distributed fashion and, as shown in Theorem 37, is also robust with respect
to bounded external disturbances.

Remark 40 One weakness of the robust adaptive protocols (5.47) and (5.48)
is that in the absence of disturbances we can no longer ensure the asymptotical
convergence of the consensus error to the origin, which is actually an inherent
drawback of the σ-modification technique [63]. Other robust adaptive control
techniques, e.g., the classic dead-zone modification and the projection operator
techniques [62], or the new low-frequency learning method [191], might also be
able to be used to develop alternative robust adaptive consensus protocols.

5.4.3 Simulation Examples

Example 18 Consider a network of six agents whose the communication
graph is given as in FIGURE 5.2(b). The dynamics of the agents are giv-
en by (5.46), with
     
xi1 0 1 0 0
xi = xi2  , A = 0 0 1 , B =  0 .
xi3 0 0 0 1

For illustration, the disturbances

 associated
 with
 the agents are assumed
 to be

h 0
i 0 −0.1 sin(3t) 0.25 cos(2t)
ω1 = 0 , ω2 = −0.5 cos(5t) , ω3 = 0 , ω4 = 0 ,
0.1 sin(t) 0.3 sin(t) 0.3 sin(3t) 0.2 cos(2t)
h 0
i h −2t
i
e
ω5 = −0.2 sin(x51 ) , and ω6 = 0 .
0 0
First, we use the node-based adaptive protocol (5.3). Solving the LMI (5.6)
Consensus Control Using Distributed Adaptive Protocols 123
3.5

2.5

1.5

i
1

d
0.5

−0.5

−1

−1.5
0 5 10 15 20 25 30 35 40
t

FIGURE 5.5: The coupling weights di in (5.2).

gives the gain matrices K and Γ in (5.3) as

 
  0.3950 0.8500 1.3269
K = − 0.6285 1.3525 2.1113 , Γ = 0.8500 1.8292 2.8554 .
1.3269 2.8554 4.4574

Let τi = 5, i = 1, · · · , 6, in (5.3). The coupling weights di associated with the

nodes using (5.3) designed as above are drawn in FIGURE 5.5, from which
it can be observed that di , i = 1, · · · , 6, do keep increasing to infinity. Thus,
(5.3) is not robust in the presence of external disturbances. Next, we use the
modified node-based adaptive protocol (5.3) to achieve consensus. Solving the
LMI (5.50) with ε = 1.5 gives the gain matrices K and Γ in (5.48) as
 
  31.6151 50.9690 25.1200
K = − 5.6227 9.0648 4.4676 , Γ = 50.9690 82.1707 40.4978 .
25.1200 40.4978 19.9593

To illustrate Theorem 38, select ϕi = 0.005 and τi = 5, i = 1, · · · , 6, in (5.48).

The relative states xi − x1 , i = 2, · · · , 6, of the agents and the coupling weights
di under (5.48) designed as above are shown in FIGURE 5.6, both of which
are clearly bounded.

5.5 Distributed Adaptive Protocols for Graphs Contain-

ing Directed Spanning Trees
The distributed adaptive protocols proposed in the previous sections are ap-
plicable to only undirected communication graphs or leader-follower graphs
where the subgraph among the followers are undirected. How to design fully
distributed adaptive consensus protocols for the general case with directed
124 Cooperative Control of Multi-Agent Systems

8
18

16
6
14

4 12

10
1
x −x

di
8
i

6
0
4

2
−2
0

−4 −2
0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30 35 40
t t

(a) (b)

FIGURE 5.6: (a) xi − x1 , i = 2, · · · , 6, under (5.48); (b) the coupling weights

di in (5.48).

graphs is quite challenging. The main difficulty lies in that the Laplacian ma-
trices of directed graphs are generally asymmetric, which renders the construc-
tion of adaptive consensus protocol and the selection of appropriate Lyapunov
function far from being easy.

5.5.1 Distributed Adaptive Consensus Protocols

In this section, we extend the node-based adaptive protocol (5.3) in Section
5.3 to the case with general directed leader-follower graphs. The dynamics of
the agents are described by (5.1). As we did in Section 5.3, let the agent in
(5.1) indexed by 1 be the leader whose control input is assumed to be zero and
the agents indexed by 2, · · · , N , be the followers. The communication graph
G is supposed to satisfy the following assumption.

Assumption 5.3 The graph G contains a directed spanning tree with the lead-
er as the root node.

Denote by L the Laplacian matrix associated with G. Because the node

indexed by 1 is the leader which has no neighbors, L can be partitioned as
 
0 01×(N −1)
L= , (5.76)
L2 L1

where L2 ∈ R(N −1)×1 and L1 ∈ R(N −1)×(N −1) . Since G satisfies Assumption
5.3, it follows from Lemma 1 that all eigenvalues of L1 have positive real parts.
It then can be verified that L1 is a nonsingular M -matrix (see Definition 6 in
Chapter 1) and is diagonally dominant.
Consensus Control Using Distributed Adaptive Protocols 125

Based on the relative states of neighboring agents, we propose the following

distributed adaptive consensus protocol with time-varying coupling weights:

ui = di ρi (νiT P −1 νi )Kνi ,
(5.77)
d˙i = νiT Γνi , i = 2, · · · , N,
PN
where νi , j=1 aij (xi − xj ), di (t) denotes the time-varying coupling weight
associated with the i-th follower with di (0) ≥ 1, P > 0 is a solution to the
LMI (5.6), K ∈ Rp×n and Γ ∈ Rn×n are the feedback gain matrices, ρi (·)
are smooth and monotonically nondecreasing functions to be determined later
which satisfy that ρi (s) ≥ 1 for s > 0.
Let ν = [ν2T , · · · , νN
T T
] . Then,
 
x2 − x1
 .. 
ν = (L1 ⊗ In )  . , (5.78)
xN − x1

where L1 is defined in (5.76). Because L1 is nonsingular for G satisfying As-

sumption 5.3, it is easy to see that the leader-follower consensus problem is
solved if and only if ν converges to zero. Hereafter, we refer to ν as the con-
sensus error. In light of (5.1) and (5.77), it is not difficult to obtain that ν and
di satisfy the following dynamics:

ξ˙ = [IN −1 ⊗ A + L1 D
b ρ̂(ν) ⊗ BK]ν,
(5.79)
d˙i = ν T Γνi ,
i

where ρ̂(ν) , diag(ρ2 (ν2T P −1 ν2 ), · · · , ρN (νN

T −1 b , diag(d2 , · · · , dN ).
P νN )), D
Before moving on to present the main result of this subsection, we first
introduce a property of the nonsingular M -matrix L1 .

Lemma 29 There exists a positive diagonal matrix G such that GL1 +LT1 G >
0. One such G is given by diag(q2 , · · · , qN ), where q = [q2 , · · · , qN ]T =
(LT1 )−1 1.

Proof 39 The first assertion is well known; see Theorem 4.25 in [127] or
Theorem 2.3 in [10]. The second assertion is shown in the following. Note
that the specific form of G given here is different from that in [30, 127, 196].
Since L1 is a nonsingular M -matrix, it follows from Theorem 4.25 in [127]
that (LT1 )−1 exists, is nonnegative, and thereby cannot have a zero row. Then,
it is easy to verify that q > 0 and hence GL1 1 ≥ 0 1 . By noting that LT1 G1 =
LT1 q = 1, we can conclude that (GL1 + LT1 G)1 > 0, implying that GL1 +
LT1 G is strictly diagonally dominant. Since the diagonal entries of GL1 +LT1 G
are positive, it then follows from Gershgorin’s disc theorem [58] that every
eigenvalue of GL1 + LT1 G is positive, implying that GL1 + LT1 G > 0.
1 For a vector x, x > (≥)0 means that every entry of x is positive (nonnegative).
126 Cooperative Control of Multi-Agent Systems

The following result provides a sufficient condition to design the adaptive

consensus protocol (5.77).
Theorem 39 Suppose that the communication graph G satisfies Assumption
5.3. Then, the leader-follower consensus problem of the agents in (5.1) can be
solved by the adaptive protocol (5.77) with K = −B T P −1 , Γ = P −1 BB T P −1 ,
and ρi (νiT P −1 νi ) = (1 + νiT P −1 νi )3 , where P > 0 is a solution to the LMI
(5.6). Moreover, each coupling weight di converges to some finite steady-state
value.
Proof 40 Consider the following Lyapunov function candidate:
XN Z νiT P −1 νi N
di λ̂0 X ˜2
V10 = ρi (s)ds + d , (5.80)
i=2
2qi 0 24 i=2 i

where d˜i = di − α, α is a positive scalar to be determined later, λ̂0 denotes

the smallest eigenvalue of GL1 + LT1 G, and G and qi are given as in Lemma
29. Since G satisfies Assumption 5.3, it follows from Lemma 29 that G >
0 and λ̂0 > 0. Because di (0) ≥ 1, it follows from the second equation in
(5.79) that di (t) ≥ 1 for t > 0. Furthermore, by noting that ρi (·) are smooth
and monotonically increasing functions satisfying ρi (s) ≥ 1 for s > 0, it
is not difficult to see that V10 is positive definite with respect to νi and d˜i ,
i = 2, · · · , N .
The time derivative of V10 along the trajectory of (5.79) is given by
N
X di
V̇10 = ρi (νiT P −1 νi )νiT P −1 ν̇i
i=2
qi
XN Z νiT P −1 νi
d˙i
+ ρi (s)ds (5.81)
i=2
2qi 0
N
λ̂0 X
+ (di − α)νiT P −1 BB T P −1 νi .
12 i=2
In the sequel, for conciseness we shall use ρ̂ and ρi instead of ρ̂(ν) and
ρi (νiT P −1 νi ), respectively, whenever without causing any confusion.
By using (5.79) and after some mathematical manipulations, we can get
that
N
X di b ρ̂G ⊗ P −1 )ν̇
ρi νiT P −1 ν̇i = ν T (D
i=2
qi
1 T b
= ν [Dρ̂G ⊗ (P −1 A + AT P −1 )
2 (5.82)
−D b ρ̂(GL1 + LT1 G)D b ρ̂ ⊗ P −1 BB T P −1 ]ν
1 b ρ̂Q ⊗ (P −1 A + AT P −1 )
≤ ν T [D
2
b 2 ρ̂2 ⊗ P −1 BB T P −1 ]ν,
− λ̂0 D
Consensus Control Using Distributed Adaptive Protocols 127

where we have used the fact that QL1 + LT1 Q > 0 to get the first inequality.
Because ρi are monotonically increasing and satisfy ρ(s) ≥ 1 for s > 0, it
follows that
N ˙ Z
X νiT P −1 νi N ˙
X
di di
ρi (s)ds ≤ ρi νiT P −1 νi
i=2
qi 0 i=2
qi
N
X N
X
d˙i 2 3 3
≤ + λ̂0 d˙i ρi2 (νiT P −1 νi ) 2
3 2
i=2 3qi λ̂0 i=2
3
(5.83)
N
X N
X
d˙i 2 3 3
≤ + λ̂0 d˙i ρi (1 + νiT P −1 νi )
2 2

i=2 3qi3 λ̂20 i=1

3
N
X 1 2
= ( + λ̂0 ρ2i )νiT P −1 BB T P −1 νi ,
3 2
i=2 3qi λ̂0
3

where we have used the mean value theorem for integrals to get the first in-
equality and used Lemma 17 (Young’s inequality) to get the second inequality.
Substituting (5.82) and (5.83) into (5.81) yields
1 T b
V̇10 ≤ ν [DρG ⊗ (P −1 A + AT P −1 )]ν
2
XN
1 1 1
− [λ̂0 ( d2i ρ2i − di − ρ2i ) (5.84)
i=2
2 12 3
1 2
+ (λ̂0 α − )]νiT P −1 BB T P −1 νi .
12 qi λ̂20
3

2
Choose α ≥ α̂ +maxi=2,··· ,N qi3 λ̂30
, where α̂ > 0 will be determined later. Then,
by noting that ρi ≥ 1 and di ≥ 1, i = 2, · · · , N , it follows from (5.84) that
1 T b
V̇10 ≤ ν [DρG ⊗ (P −1 A + AT P −1 )]ν
2
N
λ̂0 X 2 2
− (d ρ + α̂)νiT P −1 BB T P −1 νi
12 i=2 i i (5.85)
1 b
≤ ν T [DρG ⊗ (P −1 A + AT P −1 )
2
1√ b ρ̂ ⊗ P −1 BB T P −1 ]ν.
− α̂λ̂0 D
3
q
Let ν̃ = ( D b ρ̂G ⊗ I)ν and choose α̂ to be sufficiently large such that
√ −1
α̂λ̂0 G ≥ 6I. Then, we can get from (5.85) that
1 T
V̇10 ≤ ν̃ [IN ⊗ (P −1 A + AT P −1 − 2P −1 BB T P −1 )]ν̃ ≤ 0, (5.86)
2
128 Cooperative Control of Multi-Agent Systems

where to get the last inequality we have used the assertion that P −1 A +
AT P −1 − 2P −1 BB T P −1 < 0, which follows readily form (5.6).
Since V̇10 ≤ 0, V10 (t) is bounded and so is each di . By noting d˙i ≥ 0, it
can be seen from (5.79) that di are monotonically increasing. Then, it follows
that each coupling weight di converges to some finite value. Note that V̇10 ≡ 0
implies that ν̃ = 0 and thereby ν = 0. Hence, by Lemma 11 (LaSalle’s Invari-
ance principle), it follows that the consensus error ν asymptotically converges
to zero. That is, the leader-follower consensus problem is solved.
Remark 41 In comparison to the adaptive protocols in the previous sections,
a distinct feature of (5.77) is that inspired by the changing supply functions
in [157], monotonically increasing functions ρi are introduced into (5.77) to
provide extra freedom for design. As the consensus error ν converges to zero,
the functions ρi will converge to 1, in which case the adaptive protocol (5.77)
will reduce to the adaptive protocol (5.45) for undirected graphs in Section
5.3. It should be mentioned that the Lyapunov function used in the proof of
Theorem 39 is partly inspired by [175].

1 2 7

3 6

4 5

FIGURE 5.7: The directed communication graph.

Example 19 Consider a network of double integrators, described by (5.1),

with    
0 1 0
A= , B= .
0 0 1
The communication graph is given as in FIGURE 5.7, where the node indexed
by 1 is the leader which is only accessible to the node indexed by 2. It is easy
to verify that the graph in FIGURE 5.7 satisfies Assumption 5.3.
Solving the LMI (5.6) by using the SeDuMi toolbox [158] gives a solution
 
2.7881 −0.8250
P = .
−0.8250 0.9886
Thus, the feedback gain matrices in (5.77) are obtained as
 
  0.1580 0.5339
K = − 0.3974 1.3432 , Γ = .
0.5339 1.8042
Consensus Control Using Distributed Adaptive Protocols 129
10

νi 0

−2

−4

−6

−8
0 5 10 15 20 25 30 35 40
t

FIGURE 5.8: νi , i = 2, · · · , 7 of double integrators under the protocol (5.77).

To illustrate Theorem 39, let the initial states di (0) and ρi (0) be randomly
chosen within the interval [1, 3]. The consensus errors νi , i = 2, · · · , 7, of the
double integrators, defined as in (5.78), under the adaptive protocol (5.77)
with K, Γ, and ρi chosen as above, are depicted in in FIGURE 5.8, which
states that leader-follower consensus is indeed achieved. The coupling weights
di associated with the nodes are drawn in FIGURE 5.9, from which it can be
observed that the coupling weights converge to finite steady-state values.

2.8

2.6

2.4

2.2

2
d (t)
i

1.8

1.6

1.4

1.2

1
0 5 10 15 20 25 30 35 40
t

FIGURE 5.9: The coupling weights di in (5.77).

5.5.2 Robust Redesign in the Presence of External Distur-

bances
In this subsection, we revisit the distributed adaptive protocol (5.77) for the
case where the agents are subject to external disturbances. The dynamics of
130 Cooperative Control of Multi-Agent Systems

the i-th agent are described by

ẋi = Axi + Bui + ω̃i , i = 1, · · · , N, (5.87)

where xi and ui are defined as in (5.1), and ωi ∈ Rn denotes external distur-

bances associated with the i-th agent, which is assumed to satisfy the following
matching condition [43, 183].

Assumption 5.4 There exist functions ωi such that ω̃i = Bωi , i = 1, · · · , N .

The disturbances satisfying Assumption 5.4 are called matching distur-

bances. Under Assumption 5.4, the dynamics of the i-th agent in (5.87) can
be rewritten into

ẋi = Axi + B[ui + ωi ], i = 1, · · · , N. (5.88)

In the following, we consider the case where the matching disturbances ωi

are bounded. That is, the following assumption holds.

Assumption 5.5 There exist positive constants θi such that kωi k ≤ θi , i =

1, · · · , N .

Similarly as discussed in Section 5.4, due to existences of ωi , the relative

states of the agents under the protocol (5.77) generally will not converge to
zero any more but rather can only be expected to converge into some small
neighborhood of the origin. Since the derivatives of the adaptive gains di are
of nonnegative quadratic forms in terms of the relative states, in this case it is
easy to see from (5.77) that the adaptive gains di will keep growing to infinity,
i.e., the parameter drift phenomenon arises. Therefore, the adaptive protocol
(5.77) is not robust in the presence of external disturbances.
The objective of this subsection is to make modifications on (5.77) in order
to present novel distributed robust adaptive protocols which can guarantee the
ultimate boundedness of the consensus error and the adaptive gains for the
agents in (5.88). Similarly as in Section 5.4, by utilizing the σ-modification
technique [62], we propose a new distributed adaptive consensus protocol as
follows:
ui = di ρi (νiT Qνi )Kνi ,
(5.89)
d˙i = −ϕi (di − 1) + ν T Γνi , i = 2, · · · , N,
i

where ϕi , i = 2, · · · , N , are small positive constants, Q > 0 is a solution to

the following algebraic Riccati equation (ARE):

AT Q + QA + I − QBB T Q = 0. (5.90)

and the rest of the variables are defined as in (5.77). Different from the previous
subsection where the LMI (5.6) is used to design the adaptive protocol (5.77),
here we use the algebraic Ricatti equation (5.90) to design the new adaptive
protocol (5.89). These two approaches are nearly equivalent.
Consensus Control Using Distributed Adaptive Protocols 131

The consensus error ν is defined as in (5.78). In light of (5.88) and (5.89),

it is not difficult to obtain that ν and di satisfy the following dynamics:

ν̇ = [IN −1 ⊗ A + L1 Db ρ̂(ν) ⊗ BK]ν + (L1 ⊗ B)ω,

(5.91)
d˙i = −ϕi (di − 1) + νiT Γνi ,

where ω , [ω2T − ω1T , · · · , ωN

T b are defined as in (5.79).
− ω1T ]T , ρ̂(ν) and D
In light of Assumption 5.5, it is easy to see that
v
uN
uX
kωk ≤ t (θi + θ1 )2 . (5.92)
i=2

By the second equation in (5.91), we have

Z t
−ϕi t
di (t) = di (0)e + e−ϕi (t−s) (ϕi + νiT Γνi )ds
0
Z t
−ϕi t (5.93)
= (di (0) − 1)e +1+ e−ϕi (t−s) νiT Γνi ds
0
≥ 1,

where we have used the fact that di (0) ≥ 1 to get the last inequality.

Theorem 40 Suppose that the communication graph G satisfies Assump-

tion 5.3. The parameters of the adaptive protocol (5.89) are designed as
K = −B T Q, Γ = QBB T Q, and ρi (νiT Qνi ) = (1 + νiT Qνi )3 . Then, both the
consensus error ν and the coupling gains di , i = 2, · · · , N , in (5.91), under
the adaptive protocol (5.89), are uniformly ultimately bounded. Furthermore,
if ϕi is chosen to be small enough such that δ , min ϕi < τ , λmax1 (Q) ,
i=2,··· ,N
then ν exponentially converges to the residual set
2Ξ
D5 , {ν : kνk2 ≤ 1 }, (5.94)
(τ − δ)λmin (Q) min
i=2,··· ,N qi

where
N N
λ̂0 X 12 X
Ξ, ϕi (α − 1)2 + σ̄ 2 (GL1 ) (θi + θ1 )2 , (5.95)
24 i=1 λ̂0 i=2
72 2
α= max q 2
λ̂20 i=2,··· ,N i
+ max 3 3, σ̄(GL1 ) denotes the largest singular value of
i=2,··· ,N qi λ̂0
GL1 , and qi , G, and λ̂0 are chosen as in the proof of Theorem 39.

Proof 41 Let the Lypapunov function candidate be V10 as in the proof of

Theorem 39. Because di (t) ≥ 1 for t > 0 in (5.93), by following the steps
in the proof of Theorem 39, it is not difficult to see that V10 is still positive
132 Cooperative Control of Multi-Agent Systems

definite with respect to νi and d˜i , i = 2, · · · , N . The time derivative of V10

along the trajectory of (5.91) is given by
N
X di
V̇10 = ρi (νiT Qνi )νiT Qν̇i
i=2
qi
XN Z νiT Qνi
d˙i
+ ρi (s)ds (5.96)
i=2
2qi 0
N
λ̂0 X
+ (di − α)[−ϕi (di − 1) + νiT Γνi ].
12 i=2

By following similar steps in the proof of Theorem 39, we can get the
following results:
N
X di 1 T b b 2 ρ̂2 ⊗ QBB T Q]ν
ρi νiT Qν̇i ≤ ν [Dρ̂G ⊗ (QA + AT Q) − λ̂0 D
i=2
qi 2 (5.97)
b ρ̂GL1 ⊗ QB)ω,
+ ν T (D
and
N ˙ Z Z
νiT Qνi
d˙i νiT Qνi
X N ¯
X
di
ρi (s)ds ≤ ρi (s)ds
i=2
qi 0 i=2
qi 0
N
X 1 2
≤ ( + λ̂0 ρ2i )d¯˙i (5.98)
3 2
i=2 3qi λ̂0
3
N
X 1 2
≤ ( + λ̂0 ρ2i )νiT QBB T Qνi
3 2
i=2 3qi λ̂0
3

we have used the fact that d˙i ≤ d¯˙i , νiT Γνi to get the first inequality. Substi-
tuting (5.97) and (5.98) into (5.96) yields
1 T b
V̇10 ≤ ν [Dρ̂G ⊗ (QA + AT Q)]ν
2
N
λ̂0 X 2 2
− (d ρ + 2α̂)νiT QBB T Qνi
12 i=2 i i (5.99)
N
λ̂0 X
+ ϕi [−d˜2i + (α − 1)2 ] + ν T (D
b ρ̂GL1 ⊗ QB)ω,
24 i=2

where we have used the facts that ρi ≥ 1 and di ≥ 1, i = 2, · · · , N , α =

2α̂ + max q32λ̂3 , where α̂ = λ̂362 max qi2 and used the following assertion:
i=2,··· ,N i 0 0 i=2,··· ,N

1 1
−(di − α)(di − 1) = −d˜i (d˜i + α − 1) ≤ − d˜2i + (α − 1)2 .
2 2
Consensus Control Using Distributed Adaptive Protocols 133

Note that
s s
b ρ̂GL1 ⊗ QB)ω = 2ν T ( λ̂0 b 12
2ν T (D Dρ̂ ⊗ QB)( GL1 ⊗ I)ω
12 λ̂0
λ̂0 T b 2 2 12 (5.100)
≤ ν (D ρ̂ ⊗ QBB T Q)ν + k(GL1 ⊗ I)ωk2
12 λ̂0
XN
λ̂0 T b 2 2 12
≤ ν (D ρ̂ ⊗ QBB T Q)ν + σ̄ 2 (GL1 ) (θi + θ1 )2 ,
12 λ̂0 i=2

where we have used (5.92) to get the last inequality. Then, substituting (5.100)
into (5.99) gives

1 T b λ̂0 b 2 2
V̇10 ≤ ν [Dρ̂G ⊗ (QA + AT Q) − (D ρ̂ + 4α̂I) ⊗ QBB T Q)]ν
2 24
N
λ̂0 X ˜2
− ϕi d i + Ξ (5.101)
24 i=2
N
1 λ̂0 X ˜2
≤ W (ν) − ϕi di + Ξ,
2 24 i=2
√
where we have used the assertion that λ̂120 (D b ρ̂ ≥ D
b 2 ρ̂2 + 4α̂I) ≥ λ̂0 α̂D b ρ̂G if
√ 6
6
α̂I ≥ λ̂ G to get the last inequality, Ξ is defined as in (5.95), and
0

b ρ̂G ⊗ (QA + AT Q − QBB T Q)]ν

W (ν) , ν T [D
= −ν T (D b ρ̂G ⊗ I)ν ≤ 0.

PN
Therefore, we can verify that 12 W (ν) − λ̂240 i=2 ϕi d˜2i is negative definite. In
virtue of Lemma 15, we get that both the consensus error ν and the adaptive
gains di are uniformly ultimately bounded.
Note that (5.99) can be rewritten into
N
1 λ̂0 X ˜2
V̇10 ≤ −δV10 + δV10 + W (ν) − ϕi di + Ξ. (5.102)
2 24 i=2

Because ρi are monotonically increasing and satisfy ρi (s) ≥ 1 for s > 0, we

have
XN Z T XN
di νi Qνi di
ρi (s)ds ≤ ρi νiT Qνi
i=2
q i 0 i=2
q i (5.103)
b ρ̂G ⊗ Q)ν.
= ν T (D
134 Cooperative Control of Multi-Agent Systems

Substituting (5.103) into (5.102) yields

N
1 λ̂0 X
V̇10 ≤ −δV10 + W̃ (ν) + (ϕi − δ)d˜2i
2 24 i=2
(5.104)
τ −δ T b
− ν (Dρ̂G ⊗ Q)ν + Ξ,
2
b ρ̂G ⊗ (−I + τ Q)]ν. Because τ =
where W̃ (ν) , ν T [D 1
λmax (Q) , we can obtain
that W̃ (ν) ≤ 0. Then, it follows from (5.104) that

τ −δ 1
V̇10 ≤ −δV10 − λmin (Q)( min )kνk2 + Ξ, (5.105)
2 i=2,··· ,N qi

where we have used the facts that ϕi ≥ δ, δ < τ , C b ≥ I, ρ̂ ≥ I, and

G > 0. Obviously, it follows from (5.104) that V̇10 ≤ −δV10 if kνk2 >
2Ξ
1 . Then, we can get that if δ ≤ τ then ν exponentially
(τ −δ)λ (Q) min
min
i=2,··· ,N qi

converges to the residual set D5 in (5.94) with a convergence rate faster than
e−δt .

Remark 42 Theorem 40 shows that the modified adaptive protocol (5.89) can
ensure the ultimate boundedness of the consensus error ν and the adaptive
gains di for the agents in (5.88), implying that (5.89) is indeed robust in
the presence of bounded external disturbances. As discussed in Remark 38, ϕi
should be chosen to be relatively small in order to ensure a smaller bound for
the consensus error ν.

Remark 43 In contrast to Section 5.4 with undirected graphs and general

bounded disturbances, the results in this subsection for directed graphs are
applicable to matching disturbances. Extending the results in this subsection
to the case with disturbances which does not necessarily satisfy the matching
condition is not an easy task. Note that the idea used in the proof of Theorem
40 can be derive similar results for the adaptive protocols (5.47) and (5.48) in
Section 5.4 for the special case with matching disturbances.

Example 20 Consider a network of double integrators, described by (5.88),

where the disturbances associated with the agents are assumed to be ω1 =
0, ω2 = 0.5 sin(t), ω3 = 0.1 sin(t), ω4 = 0.2 cos(2t), ω5 = −0.3e−2t , ω6 =
−0.2 sin(x51 ), and ω7 = 0. The communication graph is given as in FIGURE
5.7.
Solving the
 1.7321  ARE (5.90) by using MATLAB gives a solution Q =
1
−1 1.7321 . Thus, the feedback gain matrices in (5.89) are obtained as
 
  1 1.7321
K = − 1 1.7321 , Γ = .
1.7321 3
Consensus Control Using Distributed Adaptive Protocols 135
10

νi 0

−2

−4

−6

−8
0 10 20 30 40 50 60
t

FIGURE 5.10: The consensus error νi , i = 2, · · · , 7, of double integrators

under the protocol (5.89).

To illustrate Theorem 40, let let ϕi = 0.02 in (5.89) and the initial states
di (0) be randomly chosen within the interval [1, 3]. The consensus errors νi ,
i = 2, · · · , 7, of the double integrators, defined as in (5.78), and the coupling
weights di associated with the followers are depicted in in FIGURE 5.10 and
FIGURE 5.11, respectively, both of which are clearly bounded.

2.2

2.1

1.9

1.8
di

1.7

1.6

1.5

1.4

1.3

0 10 20 30 40 50 60
t

FIGURE 5.11: The coupling weights di in (5.89).

5.6 Notes
The results in this section are mainly based on [78, 80, 93, 95]. There exist
some related works which have proposed similar adaptive protocols to achieve
136 Cooperative Control of Multi-Agent Systems

consensus or synchronization. For instance, some edge-based adaptive strate-

gies were proposed for the synchronization of complex networks in [32, 33].
However, the inner coupling matrix is required to be given a priori and the
input matrix of each node is essentially assumed to be an identity matrix in
[32, 33], which is quite restrictive. Distributed adaptive protocols based on the
relative state information of neighboring agents were presented in [159, 190] to
achieve second-order consensus with nonlinear dynamics. In [175], distributed
adaptive consensus protocols were proposed for first-order multi-agent systems
with uncertainties and directed graphs.
6
Distributed Tracking of Linear Multi-Agent
Systems with a Leader of Possibly Nonzero
Input

CONTENTS
6.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
6.2 Distributed Discontinuous Tracking Controllers . . . . . . . . . . . . . . . . . . . . . 139
6.2.1 Discontinuous Static Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.2.2 Discontinuous Adaptive Controllers . . . . . . . . . . . . . . . . . . . . . . . . 142
6.3 Distributed Continuous Tracking Controllers . . . . . . . . . . . . . . . . . . . . . . . . 144
6.3.1 Continuous Static Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.3.2 Adaptive Continuous Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.4 Distributed Output-Feedback Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
6.5 Simulation Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
6.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

This chapter considers the distributed tracking (i.e., leader-follower consen-

sus) problem for multi-agent systems with general continuous-time linear a-
gent dynamics. Actually, the distributed tracking problem has been discussed
in Chapters 2 and 4. The distributed tracking problem is a consensus problem
for the special case where the communication graph contains a directed span-
ning tree and a root node. It is worth noting that one common assumption for
the results on tracking in Chapters 2 and 4 is that the leader’s control input
is either equal to zero or available to all the followers. In many circumstances,
nonzero control actions might be implemented on the leader in order to achieve
certain objectives, e.g., to reach a desirable consensus value or to avoid haz-
ardous obstacles. However, it is restrictive and impractical to assume that all
the followers know the leader’s control input, especially when the scale of the
network is large. Actually, the leader’s control input might not be available to
any follower, e.g., for the case where the leader is an uncooperative target.
In this chapter, we consider the distributed tracking control problem for
multi-agent systems with general linear dynamics with a leader whose control
input might be nonzero and time varying and available to at most a subset
of followers. Due to the nonzero control input, the dynamics of the leader
are different from those of the followers. Thus, contrary to the homogeneous
multi-agent systems in previous chapters, the resulting multi-agent system

137
138 Cooperative Control of Multi-Agent Systems

in this chapter is in essence heterogeneous, which makes the problem much

more challenging. The distributed controllers in the previous chapters are not
applicable any more.
The distributed tracking control problem is formulated in Section 6.1.
Without loss of generalization, it is assumed that the control input of the
leader is bounded. Such an assumption is not strong, since unbounded control
inputs are not implementable. In Section 6.2, based on the relative states of
neighboring agents, a distributed static tracking controller, including a dis-
continuous nonlinear term to deal with the effect of the nonzero control input
of the leader, is proposed and designed, under which the states of the follow-
ers converge to the state of the leader, if the subgraph among the followers is
undirected, the leader has directed paths to all followers. It is worthwhile men-
tioning that the design of the static controller requires the upper bound of the
leader’s control input and the nonzero eigenvalue information of the commu-
nication graph, which actually are global information. In order to remove this
limitation, a distributed discontinuous adaptive controller with time-varying
coupling gains is further designed, which is fully distributed and also solves
the distributed tracking problem.
Note that the proposed discontinuous controllers might cause the undesir-
able chattering phenomenon in real implementations. To eliminate the chat-
tering effect, by using the boundary layer concept and the σ-modification
technique, distributed continuous static and adaptive protocol are construct-
ed in Section 6.3, under which the tracking error and the coupling gains are
shown to be uniformly ultimately bounded. The upper bound of the tracking
error is also explicitly given, which can be made to be reasonably small by
properly selecting the design parameters of the proposed controllers. As an
extension, distributed tracking controllers based on the local output informa-
tion are further discussed in Section 6.4. A simulation example is presented
for illustration in Section 6.5.

6.1 Problem Statement

Consider a group of N identical agents with general linear dynamics, consisting
of N − 1 followers and a leader. The dynamics of the i-th agent are described
by
ẋi = Axi + Bui ,
(6.1)
yi = Cxi i = 1, · · · , N,
where xi ∈ Rn is the state, ui ∈ Rp is the control input, and yi ∈ Rq is the
output of the i-th agent, respectively.
Without loss of generality, let the agent in (6.1) indexed by 1 be the leader
and the agents indexed by 2, · · · , N , be the followers. It is assumed that the
leader receives no information from any follower and the state (but not the
Distributed Tracking with a Leader of Possibly Nonzero Input 139

control input) of the leader is available to only a subset of the followers. The
interaction graph among the N agents is represented by a directed graph G,
which satisfies the following assumption.

Assumption 6.1 The subgraph Gs associated with the followers is undirected

and in the graph G the leader has directed paths to all followers. (Equivalently,
G contains a directed spanning tree with the leader as the root)

Denote by L the Laplacian matrix associated with G. Because the leader

has no neighbors, L can be partitioned as
 
0 01×(N −1)
L= , (6.2)
L2 L1

where L2 ∈ R(N −1)×1 and L1 ∈ R(N −1)×(N −1) . Since Gs is undirected, L1 is

symmetric.
The objective of this chapter is to solve the distributed tracking problem
for the agents in (6.1), i.e., to design some distributed controllers under which
the states of the N followers converge to the state of the leader in the sense
of limt→∞ kxi (t) − x1 (t)k = 0, ∀ i = 2, · · · , N.
Here we consider the general case where u1 is possibly nonzero and time
varying and accessible to at most a subset of followers, under the following
mild assumption:

Assumption 6.2 The leader’s control input u1 is bounded, i.e., there exists
a positive constant γ such that ku1 k ≤ γ.

In the following sections, several distributed tracking controllers will be

presented.

6.2 Distributed Discontinuous Tracking Controllers

6.2.1 Discontinuous Static Controllers
In this section, it is assumed that each follower can have access to the local
state information of its own and its neighbors. Based on the relative states of
neighboring agents, the following distributed tracking controller is proposed
for each follower:
N
X N
X
ui = c 1 K aij (xi − xj ) + c2 g(K aij (xi − xj )), i = 2, · · · , N, (6.3)
j=1 j=1

where c1 > 0 and c2 > 0 ∈ R are constant coupling gains, K ∈ Rp×n is

the feedback gain matrix, aij is the (i, j)-th entry of the adjacency matrix
140 Cooperative Control of Multi-Agent Systems

associated with G, and g(·) is a nonlinear function defined such that for w ∈
Rn , (
w
if kwk =
6 0
g(w) = kwk (6.4)
0 if kwk = 0.
T T
Let ξi = xi − x1 , i = 2, · · · , N , and ξ = [ξ2T , · · · , ξN ] . Using (6.3) for
(6.1), we obtain the closed-loop network dynamics as

ξ˙ = (IN −1 ⊗ A + c1 L1 ⊗ BK)ξ
(6.5)
+ c2 (IN −1 ⊗ B)G(ξ) − (1 ⊗ B)u1 ,
 PN 
g(K j=2 L2j ξj )

where L1 is defined as in (6.2) and G(ξ) ,  .. . Since the

PN .
g(K j=2 LN j ξj )
function g(·) in the static controller (6.3) is nonsmooth, the well-posedness
and the existence of the solution to (6.5) can be understood in the Filippov
sense [150]. Clearly, the distributed tracking problem is solved by (6.3) if the
closed-loop system (6.5) is asymptotically stable. Hereafter, we refer to ξ as
the tracking error.

Theorem 41 Suppose that Assumptions 6.1 and 6.2 hold. The distributed
tracking control problem of the agents described by (6.1) is solved under the
controller (6.3) with c1 ≥ λmin1(L1 ) , c2 ≥ γ, and K = −B T P −1 , where
λmin (L1 ) denotes the smallest eigenvalue of L1 and P > 0 is a solution to
the following linear matrix inequality (LMI):

AP + P AT − 2BB T < 0. (6.6)

Proof 42 Consider the following Lyapunov function candidate:

1 T
V1 = ξ (L1 ⊗ P −1 )ξ. (6.7)
2
For a communication graph G satisfying Assumption 6.1, it follows from Lem-
ma 1 and (6.2) that L1 > 0. Clearly, V1 is positive definite and continuously
differentiable. The time derivative of V1 along the trajectory of (6.5) can be
obtained as

V̇1 = ξ T (L1 ⊗ P −1 )ξ˙

1 (6.8)
= ξ T X ξ + c2 ξ T (L1 ⊗ P −1 B)G(ξ) − ξ T (L1 1 ⊗ P −1 B)u1 ,
2
where
X = L1 ⊗ (P −1 A + AT P −1 ) − 2c1 L21 ⊗ P −1 BB T P −1 . (6.9)
Distributed Tracking with a Leader of Possibly Nonzero Input 141

In virtue of Assumption 6.2, we have

N X
X N
−ξ T (L1 1 ⊗ P −1 B)u1 = − Lij ξjT P −1 Bu1
i=2 j=2
N
X N
X
≤ kB T P −1 Lij ξj kku1 k (6.10)
i=2 j=2
N
X N
X
≤γ kB T P −1 Lij ξj k.
i=2 j=2

In this case, it follows from (6.4) that

hP PN i
N
ξ T (L1 ⊗ P −1 B)G(ξ) = T −1
j=2 L2j ξj P B ··· j=2 LN j ξjT P −1 B
 T −1 P N

−B P L2j ξj
P j=2
 kB T P −1 N j=1 L2j ξj k 
 . 

× .. 

 −B T P −1 PN LN j ξj 
PNj=2
kB T P −1 j=2 LN j ξj k

N
X N
X
=− kB T P −1 Lij ξj k.
i=2 j=2
(6.11)
Then, we can get from (6.8), (6.10), and (6.11) that

XN XN
1 T
V̇1 ≤ ξ X ξ − (c2 − γ) kB T P −1 Lij ξj k
2 i=2 j=2 (6.12)
1
≤ ξ T X ξ.
2
Because the graph L satisfies Assumption 6.1, by (6.6), we obtain that
− 21 − 21
(L1 ⊗ P )X (L1 ⊗ P ) = IN ⊗ (AP + P AT ) − 2c1 L1 ⊗ BB T
≤ IN ⊗ [AP + P AT − 2c1 λmin (L1 )BB T ] (6.13)
< 0,

which implies X < 0 and V̇1 < 0. Therefore, the state ξ of (6.5) is asymptoti-
cally stable, i.e., the tracking control problem is solved.

Remark 44 As shown in Proposition 1 in Chapter 2, a necessary and suffi-

cient condition for the existence of a P > 0 to the LMI (6.6) is that (A, B) is
stabilizable. Therefore, a sufficient condition for the existence of (6.3) satisfy-
ing Theorem 41 is that (A, B) is stabilizable. The design of the parameters c1 ,
c2 , and K of (6.3) in Theorem 41 is essentially also based on the consensus
142 Cooperative Control of Multi-Agent Systems

region approach. These three parameters can be independently designed. Specif-

ically, K relies on only the agent dynamics, c1 deals with the communication
graph G, and c2 is related to only the upper bound of u1 .
Remark 45 The distributed controller (6.3) consists of a linear term
PN PN
c1 K j=1 aij (xi − xj ) and a nonlinear term c2 g(K j=1 aij (xi − xj )), where
the nonlinear term is used to suppress the effect of the leader’s nonzero input.
Without the nonlinear term in (6.3), it can be seen from (6.5) that even though
K is designed such that IN −1 ⊗A+c1 L1 ⊗BK is Hurwitz, the tracking problem
cannot be solved due to the nonzero u1 . Furthermore, by letting c2 = 0, (6.3)
is reduced to the distributed controller in Chapter 2 for the special case where
u1 = 0.
Remark 46 To present the nonlinear controller (6.3), we borrow ideas from
the sliding mode control literature (e.g., [187, 43]) and treat the nonzero con-
trol input of the leader as an external disturbances satisfying the matching
condition [43]. The nonlinear function g(·) in (6.3) is a unit vector nonlin-
earity, i.e., kg(·)k = 1. Actually, g(·)in (6.3) can be replaced by the signum

1 if s > 0
function sgn(·), defined as sgn(s) = 0 if s = 0 . In this case, the design


−1 if s < 0
of the new controller and the derivation of the theorem can be accordingly
modified. Please refer to [92] for details.

6.2.2 Discontinuous Adaptive Controllers

In the last subsection, the design of the distributed controller (6.3) depends on
the minimal eigenvalue λmin (L1 ) of L1 and the upper bound γ of the leader’s
control input u1 . However, λmin (L1 ) is global information of the communi-
cation graph, which is not easy to compute especially when the multi-agent
network is of a large scale. Furthermore, the value of γ might not be explicitly
known for each follower in some applications. The objective of this subsection
is to solve the distributed tracking problem without requiring that λmin (L1 )
and γ be explicitly known. To this end, we propose the following distributed
tracking controller with an adaptive law for updating the coupling gain for
each follower:
N
X N
X
ui = di K aij (xi − xj ) + di g(K aij (xi − xj )),
j=1 j=1
N
X N
X
d˙i = τi [( aij (xi − xj )T )Γ( aij (xi − xj )) (6.14)
j=1 j=1
N
X
+ kK aij (xi − xj )k], i = 2, · · · , N,
j=1
Distributed Tracking with a Leader of Possibly Nonzero Input 143

where K ∈ Rp×n and aij are defined in (6.3), di (t) denotes the time-varying
coupling gain (weight) associated with the i-th follower, Γ ∈ Rn×n is the
feedback gain matrix, and τi are positive scalars.
Let ξ be defined as in (6.5) and D = diag(d2 , · · · , dN ). From (6.1) and
(6.14), we can get the closed-loop network dynamics as

ξ˙ = (IN −1 ⊗ A + DL1 ⊗ BK)ξ + (D ⊗ B)G(ξ) − (1 ⊗ B)u1 ,

N
X N
X N
X (6.15)
d˙i = τi [( Lij ξjT )Γ( Lij ξj ) + kK Lij ξj k], i = 2, · · · , N.
j=2 j=2 j=2

Theorem 42 Suppose that Assumptions 6.1 and 6.2 hold. Then, the dis-
tributed tracking control problem of the agents described by (6.1) is solved un-
der the adaptive controller (6.14) with K = −B T P −1 and Γ = P −1 BB T P −1 ,
where P > 0 is a solution to the LMI (6.6). Moreover, each coupling gain di
converges to some finite steady-state value.

Proof 43 Consider the following Lyapunov function candidate

XN
1 T −1 1
V2 = ξ (L1 ⊗ P )ξ + (di − β)2 , (6.16)
2 i=2
2τ i

where β is a positive constant. As shown in the proof of Theorem 41, V2 is

positive semi-definite. The time derivative of V2 along (6.15) can be obtained
as
N
X di − β
V̇2 = ξ T (L1 ⊗ P −1 )ξ˙ + d˙i
i=2
τi
= ξ T (L1 ⊗ P −1 A + L1 DL1 ⊗ P −1 BK)ξ
(6.17)
+ ξ T (L1 D ⊗ P −1 B)G(ξ) − ξ T (L1 1 ⊗ P −1 B)u1
N
X N
X N
X N
X
+ (di − β)[( Lij ξjT )Γ( Lij ξj ) + kK Lij ξj k].
i=2 j=2 j=2 j=2

By substituting K = −BP −1 , it is easy to get that

ξ T (L1 DL1 ⊗ P −1 BK)ξ

N
X N
X N
X (6.18)
=− di [ Lij ξj ]T P −1 BB T P −1 [ Lij ξj ].
i=2 j=2 j=2

By the definitions of g(·) and G(·), we can get that

N
X N
X
ξ T (L1 D ⊗ P −1 B)G(ξ) = − di kB T P −1 Lij ξj k. (6.19)
i=2 j=2
144 Cooperative Control of Multi-Agent Systems

Substituting (6.18), (6.19), and (6.10) into (6.17) yields

XN XN
1 T
V̇2 ≤ ξ Yξ − (β − γ) kB T P −1 Lij ξj k, (6.20)
2 i=2 j=2

where Y , L1 ⊗ (P −1 A + AT P −1 ) − 2βL21 ⊗ P −1 BB T P −1 .
As shown in the proof of Theorem 41, by selecting β sufficiently large such
that β ≥ γ and βλi ≥ 1, i = 2, · · · , N , we get from (6.20) that
1 T
V̇2 ≤ ξ Yξ, (6.21)
2
and from (6.13) that Y > 0. Then, it follows that V̇2 ≤ 0, implying that V2 (t)
is nonincreasing. Therefore, in view of (6.16), we know that di and ξ are
bounded. Since by Assumption 6.2, u1 is bounded, this implies from the first
equation in (6.15) that ξ˙ is bounded. As V2 (t) is nonincreasing and bounded
from Rbelow by zero, it has a finite limit V2∞ as
R ∞t → ∞. Integrating (6.21), we
∞
have 0 12 ξ T Yξdτ ≤ V2 (ζ(t1 )) − V2∞ . Thus, 0 21 ξ T Yξdτ exists and is finite.
Because ξ and ξ˙ are bounded, it is easy to see from (6.21) that ξ T Y ξ˙ is also
bounded, which in turn guarantees the uniform continuity of ξ T Yξ. Therefore,
by Lemma 13 (Barbalat’s Lemma), we get that ξ T Yξ → 0 as t → ∞, i.e.,
ξ → 0 as t → ∞. By noting that Γ ≥ 0 and τi > 0, it follows from (6.15) that
di is monotonically increasing. Thus, the boundedness of di implies that each
di converges to some finite value.
Remark 47 Compared to the static controller (6.3), the adaptive controller
(6.14) requires neither the minimal eigenvalue λmin (L1 ) of L1 nor the up-
per bound γ of u1 , as long as u1 is bounded. On the other hand, the cou-
pling gains need to be dynamically updated in (6.14), implying that the adap-
tive controller (6.14) is more complex than the static controller (6.3). The
PN
term di g( j=1 aij K(xi − xj )) in (6.14) is used to tackle the effect of the
leader’s nonzero control input u1 on consensus. For PNthe special case where
u1 = 0, we can accordingly remove the terms di g( j=1 aij K(xi − xj )) and
PN
τi k j=1 aij K(xi − xj )k from (6.14), which will reduce to the continuous pro-
tocol (5.45) in the previous chapter.

6.3 Distributed Continuous Tracking Controllers

6.3.1 Continuous Static Controllers
An inherent drawback of the discontinuous controller (6.3) is that it will result
in the undesirable chattering effect in real implementation, due to imperfec-
tions in switching devices [187, 43]. To avoid the chattering effect, one feasible
Distributed Tracking with a Leader of Possibly Nonzero Input 145

approach is to use the boundary layer technique [187, 43] to give a continuous
approximation of the discontinuous function g(·).
Using the boundary layer technique, we propose a distributed continuous
static controller as
N
X N
X
ui = c 1 K aij (xi − xj ) + c2 ĝi (K aij (xi − xj )), i = 2, · · · , N,
j=1 j=1
(6.22)
where the nonlinear functions ĝ(·) are defined such that for w ∈ Rn ,
(
w
if kwk > κi
ĝi (w) = kwk
w
(6.23)
κi if kwk ≤ κi

with κi being small positive scalars, denoting the widths of the boundary
layers, and the rest of the variables are the same as in (6.3).
Let the tracking error ξ be defined as in (6.5). From (6.1) and (6.22), we
can obtain that ξ in this case satisfies

ξ˙ = (IN −1 ⊗ A + c1 L1 ⊗ BK)ξ + c2 (IN −1 ⊗ B)G(ξ)

b − (1 ⊗ B)u1 , (6.24)

where  PN 
ĝ2 (K j=2 L2j ξj )
 .. 
b
G(ξ) , , (6.25)
 . 
PN
ĝN (K j=2 LN j ξj )
The following theorem states the ultimate boundedness of the tracking
error ξ.

Theorem 43 Assume that Assumptions 6.1 and 6.2 hold. Then, the tracking
error ξ of (6.24) under the continuous controller (6.22) with c1 , c2 , and K
chosen as in Theorem 41 is uniformly ultimately bounded and exponentially
converges to the residual set
N
2λmax (P )γ X
D1 , {ξ : kξk2 ≤ κi }, (6.26)
αλmin (L1 ) i=2

where
−λmax (AP + P AT − 2BB T )
α= . (6.27)
λmax (P )

Proof 44 Consider the Lyapunov function V1 as in the proof of Theorem 41.

The time derivative of V1 along the trajectory of (6.24) is given by

V̇1 = ξ T (L1 ⊗ P −1 )ξ˙

1 (6.28)
b
= ξ T X ξ + c2 ξ T (L1 ⊗ P −1 B)G(ξ) − ξ T (L1 1 ⊗ P −1 B)u1 ,
2
146 Cooperative Control of Multi-Agent Systems

where X is defined in (6.9).

Next, Pconsider the following three cases:P
M M
i) kK j=2 Lij ξj k > κi , i.e., kB T P −1 j=2 Lij ξj k > κi , i = 2, · · · , N .
In this case, it follows from (6.23) and (6.25) that
N
X N
X
b
ξ T (L1 ⊗ P −1 B)G(ξ) =− kB T P −1 Lij ξj k. (6.29)
i=2 j=2

Then, we can get from (6.28), (6.10), and (6.29) that

1 T
V̇1 ≤ ξ X ξ.
2
PM
ii) kB T P −1 j=2 Lij ξj k ≤ κi , i = 2, · · · , N .
From (6.10), we can obtain that
N
X N
X
−ξ T (L1 1 ⊗ P −1 B)u1 ≤ γ kB T P −1 Lij ξj k
i=2 j=2
(6.30)
N
X
≤γ κi .
i=2

Further, it follows from (6.23), (6.25), and (6.29) that

XN XN
b 1
ξ T (L1 ⊗ P −1 B)G(ξ) =− kB T P −1 Lij ξj k2 ≤ 0. (6.31)
κ
i=2 i j=2

Thus, we get from (6.28), (6.30), and (6.31) that

XN
1 T
V̇1 ≤ ξ Xξ + γ κi . (6.32)
2 i=2

iii) ξ satisfies neither Case i) nor Case ii).

PM
Without loss of generality, assume that kB T P −1 j=2 Lij ξj k > κi , i =
PM
2, · · · , l, and kB T P −1 j=2 Lij ξj k ≤ κi , i = l+1, · · · , N , where 3 ≤ l ≤ N −1.
It is easy to see from (6.10), (6.23), and (6.29) that
l
X N
X N
X −l
−ξ T (L1 1 ⊗ P −1 B)u1 ≤ γ kB T P −1 Lij ξj k + γ κi ,
i=2 j=2 i=2
l
X N
X
b
ξ T (L1 ⊗ P −1 B)G(ξ) ≤− kB T P −1 Lij ξj k.
i=2 j=2

Clearly, in this case we have

X N −l
1 T
V̇1 ≤ ξ Xξ + γ κi .
2 i=2
Distributed Tracking with a Leader of Possibly Nonzero Input 147

Therefore, by analyzing the above three cases, we get that V̇1 satisfies (6.32)
for all ξ ∈ RN n . Note that (6.32) can be rewritten as

XN
1
V̇1 ≤ −αV1 + αV1 + ξ T X ξ + γ κi
2 i=2
(6.33)
XN
1
= −αV1 + ξ T (X + αL1 ⊗ P −1 )ξ + γ κi .
2 i=2

−λmax (AP +P AT −2BB T )

Because α = λmax (P ) , in light of (6.6), we can obtain that

− 21 − 12
(L1 ⊗ P )(X + αL1 ⊗ P −1 )(L1 ⊗ P)
≤ IN ⊗ [AP + P A + αP − 2BB T ] < 0.
T

Then, it follows from (6.33) that

N
X
V̇1 ≤ −αV1 + γ κi . (6.34)
i=2

By using Lemma 16 (the Comparison lemma), we can obtain from (6.34) that
PN PN
γ i=2 κi −αt γ i=2 κi (6.35)
V1 (ξ) ≤ [V1 (ξ(0)) − ]e + ,
α α
which implies that ξ exponentially converges to the residual set D1 in (6.26)
with a convergence rate not less than e−αt .

Remark 48 Contrary to the discontinuous controller (6.3), the chattering ef-

fect can be avoided by using the continuous controller (6.22). The tradeoff is
that the continuous controller (6.22) does not guarantee asymptotic stability
of the tracking error ξ. Note that the upper bound of ξ depends on the com-
munication graph G, the number of followers, the upper bounds of the leader’s
control input and the widths κi of the boundary layers. By choosing sufficiently
small κi , the tracking error ξ under the continuous controller (6.22) can be
arbitrarily small, which is acceptable in most circumstances.

6.3.2 Adaptive Continuous Controllers

In this subsection, we will present a continuous approximation of the discon-
tinuous adaptive controller (6.14). It is worth noting that simply replacing
the discontinuous function g(·) by the continuous functions ĝi (·) in (6.23) as
did in the last subsection does not work. Using the adaptive controller with
g(·) replaced by ĝi (·), the tracking error ξ will not asymptotically converge to
zero but rather can only be expected to converge into some small neighbor-
hood of zero. In this case, it can be observed from the last equation in (6.14)
148 Cooperative Control of Multi-Agent Systems

that the adaptive weights di (t) will slowly grow unbounded. This argument
can also been easily verified by simulation results. To tackle this problem, we
propose to use the so-called σ-modification technique [63, 183] to modify the
discontinuous adaptive controller (6.14).
Using the boundary layer concept and the σ-modification technique, a
distributed continuous adaptive controller is proposed as
N
X N
X
ui = di K aij (xi − xj ) + di ri (K aij (xi − xj )),
j=1 j=1
N
X N
X
d˙i = τi [−ϕi di + ( aij (xi − xj )T )Γ( aij (xi − xj )) (6.36)
j=1 j=1
N
X
+ kK aij (xi − xj )k], i = 2, · · · , N,
j=1

where ϕi are small positive constants, the nonlinear functions ri (·) are defined
as follows: for w ∈ Rn ,
(
w
if di kwk > κi
ri (w) = kwk
w
(6.37)
κ i di if di kwk ≤ κi
and the rest of the variables are defined as in (6.14).
Let the tracking error ξ and the coupling gains D(t) be as defined in (6.15).
Then, it follows from (6.1) and (6.36) that ξ and D(t) satisfy the following
dynamics:
ξ˙ = (IN −1 ⊗ A + DL1 ⊗ BK)ξ + (D ⊗ B)R(ξ) − (1 ⊗ B)u1 ,
N
X N
X N
X
d˙i = τi [−ϕi di + ( Lij ξjT )Γ( Lij ξj ) + kK Lij ξj k], i = 2, · · · , N,
j=2 j=2 j=2

 PN  (6.38)
r2 (K j=2 L2j ξj )

where R(x) ,  .. .
PN.
rN (K j=2 LN j ξj )
The following theorem presents the ultimate boundedness of the states ξ
and di of (6.38).
Theorem 44 Suppose that Assumptions 6.1 and 6.2 hold. Then, both the
tracking error ξ and the coupling gains di , i = 2, · · · , N , in (6.38), under the
continuous adaptive controller (6.36) with K and Γ designed as in Theorem
42, are uniformly ultimately bounded . Moreover, the following two assertions
hold.
i) For any ϕi , both ξ and di exponentially converge to the residual set
N
1 X 2 κi
D2 , {ξ, di : V3 < (β ϕi + )}, (6.39)
2δ i=2 2
 Distributed Tracking with a Leader of Possibly Nonzero Input 149

where δ , mini=2,··· ,N {α, τi ϕi } and

XN
1 T d˜2i
V3 = ξ (L ⊗ P −1 )ξ + , (6.40)
2 i=2
2τi

where α is defined as in (6.27), d˜i = di − β, and β = max{ λmin1(L1 ) , γ}.

ii) If ϕi satisfies ̺ , mini=2,··· ,N {τi ϕi } < α, then in addition to i), ξ exponen-

tially converges to the residual set

X N
λmax (P )
2 1
D3 , {ξ : kξk ≤ (β 2 ϕi + κi )}. (6.41)
λmin (L1 )(α − ̺) i=2 2

Proof 45 Consider the Lyapunov function candidate V3 as in (6.40). The

time derivative of V3 along (6.38) can be obtained as

XN ˜
di ˜˙
V̇3 = ξ T (L1 ⊗ P −1 )ξ˙ + di
τ
i=2 i
= ξ T (L1 ⊗ P −1 A + L1 DL1 ⊗ P −1 BK)ξ
+ ξ T (L1 D ⊗ P −1 B)R(ξ) − ξ T (L1 1 ⊗ P −1 B)u1
N
X N
X N
X N
X
+ d˜i [−ϕi (d˜i + β) + ( Lij ξjT )Γ( Lij ξj ) + kK Lij ξj k],
i=2 j=2 j=2 j=2
(6.42)
where D = diag(d˜1 + β, · · · , d˜N + β).
Next, consider
PM the following three cases:
i) di kK j=2 Lij ξj k > κi , i = 2, · · · , N .
In this case, we can get from (6.37) that
N
X N
X
ξ T (L1 D ⊗ P −1 B)R(ξ) = − (d˜i + β)kB T P −1 Lij ξj k. (6.43)
i=2 j=2

In light of (6.10) and (6.18), substituting (6.43) into (6.42) yields

N N N
1 T 1X X X
V̇3 ≤ ξ Zξ + ϕi (β 2 − d˜2i ) − (β − γ) kB T P −1 Lij ξj k
2 2 i=2 i=2 j=2
N
1 T 1X
≤ ξ Zξ + ϕi (β 2 − d˜2i ),
2 2 i=2

where we have used the fact that −d˜2i − d˜i β ≤ − 21 d˜2i + 12 β 2 to get the first
inequality and Z , L1 ⊗ (P −1 A + AT P −1 ) − 2βL21 ⊗ P −1 BB T P −1 .
150 Cooperative Control of Multi-Agent Systems
PN
ii) di kK j=2 Lij ξj k ≤ κi , i = 2, · · · , N .
In this case, we can get from (6.37) that
N
X N
X
(d˜i + β)2
ξ T (L1 D ⊗ P −1 B)R(ξ) = − kB T P −1 Lij ξj k. (6.44)
i=2
κi j=2

Then, it follows from (6.43), (6.10), (6.18), and (6.42) that

N
1 T 1X
V̇3 ≤ ξ Zξ + ϕi (β 2 − d˜2i )
2 2 i=2
N
X N
X
(d˜i + β)2
− kB T P −1 Lij ξj k2
i=2
κi j=2
(6.45)
N
X N
X
+ (d˜i + β)kB T P −1 Lij ξj k
i=2 j=2
N N
1 T 1X 1X
≤ ξ Zξ + ϕi (β 2 − d˜2i ) + κi .
2 2 i=2 4 i=2

Note that to get the last inequality in (6.45), we have used the following fact:

XN XN
d2i T −1 2 T −1 1
− kB P Lij ξj k + di kB P Lij ξj k ≤ κi ,
κi j=2 j=2
4
PN
for di kK j=2 Lij ξj k ≤ κi , i = 2, · · · , N .
iii) ξ satisfies neither Case i) nor Case ii).
PN
Without loss of generality, assume that di kK j=2 Lij ξj k ≤ κi , i =
PN
2, · · · , l, and di kK j=2 Lij ξj k > κi , i = l + 1, · · · , N , where 3 ≤ l ≤ N − 1.
By following similar steps in the two cases above, it is easy to get that
N N −l
1 T 1X 1X
V̇3 ≤ ξ Zξ + ϕi (β 2 − d˜2i ) + κi .
2 2 i=2 4 i=2

Therefore, based on the above three cases, we can get that V̇3 satisfies (6.45)
for all ξ ∈ RN n . Note that (6.45) can be rewritten into
N N
1 1X 1X
V̇3 ≤ −δV3 + δV3 + ξ T Zξ + ϕi (β 2 − d˜2i ) + κi
2 2 i=2 4 i=2
N
1 1X δ
= −δV3 + ξ T (Z + δL ⊗ P −1 )ξ − (ϕi − )d˜2i (6.46)
2 2 i=2 τi
N
1X 2 1
+ (β ϕi + κi ).
2 i=2 2
Distributed Tracking with a Leader of Possibly Nonzero Input 151

Because βλmin (L1 ) ≥ 1 and 0 < δ ≤ α, by following similar steps in the proof
of Theorem 43, we can show that ξ T (Z + δL ⊗ P −1 )ξ ≤ 0. Further, by noting
that δ ≤ mini=2,··· ,N ϕi τi , it follows from (6.46) that
N
1X 2 1
V̇3 ≤ −δV3 + (β ϕi + κi ), (6.47)
2 i=2 2

which implies that

N
1 X 2 1
V3 ≤ [V3 (0) − ( β ϕi + κi )]e−δt
2δ i=2 2
(6.48)
N
1 X 2 1
+ (β ϕi + κi ).
2δ i=2 2

Therefore, V3 exponentially converges to the residual set D2 in (6.39) with

λmin (L1 )
a convergence rate faster than e−δt , which, in light of V3 ≥ 2λ max (P )
kξk2 +
PN d˜2i ˜
i=2 2τi , implies that ξ and di are uniformly ultimately bounded.
Next, if ̺ , mini=2,··· ,N ϕi τi < α, we can obtain a smaller residual set for
ξ by rewriting (6.46) into
N
1 1X ̺
V̇3 ≤ −̺V3 + ξ T (Z + αL ⊗ P −1 )ξ − (ϕi − )d˜2i
2 2 i=2 τi
N
α−̺ T 1X 2 1
− ξ (L ⊗ P −1 )ξ + (β ϕi + κi ) (6.49)
2 2 i=2 2
N
λmin (L1 )(α − ̺) 1X 2 1
≤ −̺V3 − kξk2 + (β ϕi + κi ).
2λmax (P ) 2 i=2 2

λmax (P ) PN
Obviously, it follows from (6.49) that V̇3 ≤ −̺V3 if kξk2 > λmin (L1 )(α−̺) i=2
λmin (L1 )
(β 2 ϕi + 21 κi ). Then, by noting V3 ≥ 2λ max (P )
kξk2 , we can get that if ̺ ≤ α then
ξ exponentially converges to the residual set D3 in (6.41) with a convergence
rate faster than e−̺t .

Remark 49 It is worth mentioning that adding −ϕi di into (6.36) is essen-

tially motivated by the so-called σ-modification technique in [63, 183], which
plays a vital role to guarantee the ultimate boundedness of the tracking error
ξ and the adaptive gains di . From (6.39) and (6.41), we can observe that the
residual sets D2 and D3 decrease as κi decrease. Given κi , smaller ϕi gives a
smaller bound for ξ and at the same time yields a larger bound for di . For the
case where ϕi = 0, di will tend to infinity. In real implementations, if large
di is acceptable, we can choose ϕi and κi to be relatively small in order to
guarantee a small tracking error ξ.
152 Cooperative Control of Multi-Agent Systems

Remark 50 Similarly as in Section 5.4 in the previous chapter, it is not

difficult to show that the proposed adaptive controller (6.36) can be redesigned
to be robust with respect to bounded external disturbances, i.e., for the agents
described by (5.46) in Section 5.4 of the previous chapter. The details are
omitted here for conciseness.

6.4 Distributed Output-Feedback Controllers

In this section, we intend to consider the case where each agent has only access
to the local output information, rather than the local state information, of
itself and its neighbors. In this case, based on the relative estimates of the
states of neighboring agents, the following distributed continuous adaptive
controller is proposed for each follower:

v̂˙ i = Av̂i + Bui + L̂(C v̂i − yi ),

N
X N
X
ui = di aij F̂ (v̂i − v̂j ) + di ri ( aij F̂ (v̂i − v̂j )),
j=1 j=1
N
X N
X (6.50)
d˙i = τi [−ϕi di + ( aij (v̂i − v̂j )T )Γ̂( aij (v̂i − v̂j ))
j=1 j=1
N
X
+k aij F̂ (v̂i − v̂j )k], i = 2, · · · , N,
j=1

where v̂i ∈ Rn is the estimate of the state of the i-th follower, v̂1 ∈ Rn denotes
the estimate of the state of the leader, given by v̂˙ 1 = Av̂1 + Bu1 + L(C v̂1 − y1 ),
ϕi are small positive constants, τi are positive scalars, di denotes the time-
varying coupling gain associated with the i-th follower, L̂ ∈ Rn×q , F̂ ∈ Rp×n ,
and Γ̂ ∈ Rn×n are the feedback gain matrices, and ri (·) are defined as in (6.37).
PN
Note that the term j=1 aij (v̂i − v̂j ) in (6.50) imply that the agents need to
get the estimates of the states of their neighbors, which can be transmitted
 −x1  graph G.
via the communication
Let ζi = xv̂ii −v̂ 1
, i = 2, · · · , N , and ζ = [ζ2T , · · · , ζN
T T
] . Using (6.50) for
(6.1), we obtain the closed-loop network dynamics as

ζ̇ = (I ⊗ S + DL1 ⊗ F)ζ + (D ⊗ B)R(ζ) − (1 ⊗ B)u1 ,

N
X N
X N
X
d˙i = τi [−ϕi di + ( Lij ζjT )I( Lij ζj ) + kO Lij ζj k], i = 2, · · · , N,
j=2 j=2 j=2
(6.51)
Distributed Tracking with a Leader of Possibly Nonzero Input 153

where D = diag(d2 , · · · , dN ), and

 PN 
r2 (O j=2 L2j ζj )  
 .  A 0
R(ζ) ,  . 
.  , S = −L̂C A + L̂C ,
PN
rN (O j=2 LN j ζj )
     
0 B F̂ 0 0 B  
F= , I= , B= , O = 0 F̂ .
0 B F̂ 0 Γ̂ B

Clearly, the distributed tracking problem is solved if the tracking error ζ of

(6.51) converges to zero.

Theorem 45 Suppose that Assumptions 6.1 and 6.2 hold. Then, both the
tracking error ζ and the coupling gains di , i = 2, · · · , N , in (6.51) are u-
niformly ultimately bounded under the distributed continuous adaptive pro-
tocol (6.50) with L̂ satisfying that A + L̂C is Hurwitz, F̂ = −B T P −1 , and
Γ̂ = P −1 BB T P −1 , where P > 0 is a solution to the LMI (6.6).

Proof 46 Consider the following Lyapunov function candidate

XN
1 T d˜2i
V4 = ζ (L1 ⊗ P)ζ + ,
2 i=2
2τi
h i
ϑP̂ −ϑP̂ T
where P , −ϑ P̂ ϑP̂ +P −1 , P̂ > 0 satisfies that (A+ L̂C)P̂ +(A+ L̂C) P̂ < 0,

d˜i = di − β, i = 2, · · · , N , β is sufficiently large such that βλmin (L1 ) ≥ 1 and

β ≥ γ, and ϑ is a positive constant to be determined later. In light of Lemma
19 (Schur Complement Lemma), it is easy to know that P > 0. Since P > 0,
it is easy to see that V4 is positive definite.
The time derivative of V4 along (6.51) can be obtained as

bSb + L1 DL1 ⊗ P
V̇4 = ζ̂ T [(L1 ⊗ P bF)
b ζ̂ − (L1 1 ⊗ P
bB)u
b 1]
N
X
bB)R(
+ ζ̂ T (L1 D ⊗ P b ζ̂) + d˜i [−ϕi (d˜i + β)
i=2 (6.52)
N
X N
X N
X
+( Lij ζ̂jT )I( Lij ζ̂j ) + kO Lij ζ̂j k],
j=2 j=2 j=2

where D = diag(hd˜2 + β, ·i· · , d˜N h+ β), ζ̂ , T T T

i [ζ̂1 , ·· · , ζ̂N] = (I ⊗ T̂ )ζ with
 I −I 
T̂ = 0 I , P b = ϑP̂ 0−1 , Sb = A+L̂C 0 , Fb = 0 0 , and Bb = [ 0 ].
0 P −L̂C A 0 B F̂ B
Note that
N
X N
X N
X
bF)
ζ̂ T (L1 DL1 ⊗ P b ζ̂ = (d˜i + β)[ bF[
Lij ζ̂jT ]P b Lij ζ̂j ]. (6.53)
i=2 j=2 j=2
154 Cooperative Control of Multi-Agent Systems

In virtue of Assumption 6.1, we have

N
X N
X
bB)u
−ζ̂ T (L1 1 ⊗ P b 1≤γ kBbT P
b Lij ζ̂j k. (6.54)
i=2 j=2

Next, consider
PN the following three cases.
i) di kO j=2 Lij ζ̂j k > κi , i = 2, · · · , N .
By observing that P bBb = −OT , it is not difficult to get from the definitions
of ri (·) and R(·) that
N
X N
X
ζbT (L1 D ⊗ P
bB)R(
b ζ̂) = − (d˜i + β)kBbT P
b Lij ζ̂j k. (6.55)
i=2 j=2

By noting that PbFb = −I and substituting (6.53), (6.54), and (6.55) into
(6.52), we can get that
N
X N
X N
X
V̇4 ≤ Z(ζ̂) − ϕi (d˜2i + β d˜i ) − (β − γ) kBbT P
b Lij ζ̂j k
i=2 i=2 j=2
N
1X
≤ Z(ζ̂) + ϕi (β 2 − d˜2i ),
2 i=2

where
1 T bSb + SbT P)
b − 2βL21 ⊗ I]ζ̂.
Z(ζ̂) , ζ̂ [L1 ⊗ (P (6.56)
2
PN
ii) di kO j=2 Lij ζ̂j k ≤ κi , i = 2, · · · , N .
From the definitions of ri (·) and R(·), in this case we get that
N
X N
X
(d˜i + β)2
ζbT (L1 D ⊗ P
bB)R(
b ζ̂) = − kBbT P
b Lij ζ̂j k2 . (6.57)
i=2
κi j=2

Then, it follows from (6.53), (6.54), (6.57), and (6.52) that

N
X N
X N
X
(d˜i + β)2
V̇4 ≤ Z(ζ̂) − ϕi (d˜2i + β d˜i ) − kBbT P
b Lij ζ̂j k2
i=2 i=2
κi j=2
N
X N
X
+ (d˜i + β)kBbT P
b Lij ζ̂j k (6.58)
i=2 j=2
N
X N
1X
≤ Z(ζ̂) − ϕi (d˜2i + β d˜i ) + κi .
i=2
4 i=2

Note that to get the last inequality in (6.58), we have used the fact that for
Distributed Tracking with a Leader of Possibly Nonzero Input 155

b PN Lij ζj k ≤ κi , − di kBbT Pb PN Lij ζ̂j k2 + di kBbT P b PN Lij ζ̂j k ≤

2
di kBbT P j=2 κi j=2 j=2
1
4 κi .
iii) ζ satisfies neither Case i) nor Case ii).
PN
In this case, without loss of generality, assume that di kO j=2 Lij ζ̂j k > κi ,
PN
i = 2, · · · , l, and di kO j=2 Lij ζ̂j k ≤ κi , i = l + 1, · · · , N , where 3 ≤ l ≤
N − 1. Combining (6.55) and (6.57), we have
l
X N
X
ζbT (L1 D ⊗ P
bB)R(
b ζ̂) = − (d˜i + β)kBbT P
b Lij ζ̂j k
i=2 j=2
N
X N
(d˜i + β)2 bT b X
− kB P Lij ζ̂j k2 .
κi j=2
i=l+1

Then, by following the steps in the two cases above, it is not difficult to get
that in this case
N
X N −l
1X
V̇4 ≤ Z(ζ̂) − ϕi (d˜2i + β d˜i ) + κi .
i=2
4 i=2

Analyzing the above three cases, we get that V̇4 satisfies (6.58) for all ζ ∈
R2N n . Note that
bSb + SbT P
diag(I, P )[P b − 2βλmin (L1 )I]diag(I, P )
 
Ξ −C T LT (6.59)
= ,
−LC AP + P AT − 2βλmin (L1 )BB T

where Ξ = ϑ[P̂ (A + LC) + (A + LC)T P̂ ]. Because βλmin (L1 ) ≥ 1, it follows

from (6.6) that AP + P AT − 2βλmin (L1 )BB T < 0. Then, choosing ϑ > 0
sufficiently large and in virtue of Lemma 19 (Schur Complement Lemma), we
can obtain from (6.59) that P bSb + SbT P
b − 2βλmin (L1 )I < 0. Observe that
− 21 b − 2βL2 ⊗ I](L− 2 ⊗ I)
bSb + SbT P)
1
(L1 ⊗ I)[L1 ⊗ (P 1 1
b b b T b
≤ I ⊗ [P S + S P − 2βλmin (L1 )I].

Thus, we know that L1 ⊗ (P bSb + SbT P)

b − 2βL2 ⊗ I < 0, implying that Z(ζ̂) −
P 1
1 N ˜2
2 i=2 ϕi di < 0, where Z(ζ̂) is defined in (6.56). In virtue of Lemma 15,
we get from (6.58) that the states ζ and di of (6.51) are uniformly ultimately
bounded.

Remark 51 As pointed out in Proposition 1 of Chapter 2, a sufficient con-

dition for the existence of a protocol (6.50) satisfying Theorem 45 is that
(A, B, C) is stabilizable and detectable. The adaptive protocol (6.50) can be im-
plemented by each agent in a fully distributed fashion requiring neither global
information of the communication topology nor the upper bound for the lead-
er’s control input.
156 Cooperative Control of Multi-Agent Systems
PN
Remark 52 Note that the term di ri ( j=1 aij F (v̂i − v̂j )) in (6.50) is used to
tackle the effect of the leader’s nonzero control input u1 on consensus. For the
special case where u1 = 0, we can simplify (6.50) into the following form:

v̂˙ i = Av̂i + Bui + L̂(C v̂i − yi ),

N
X N
X
d˙i = τi [ aij (v̂i − v̂j )T ]Γ̂[ aij (v̂i − v̂j )]
j=1 j=1 (6.60)
N
X
ui = di aij F̂ (v̂i − v̂j ), i = 2, · · · , N.
j=1

where v̂1 satisfies v̂˙ 1 = Av̂1 + L(C v̂1 − y1 ).

For the special case where u1 = 0, the adaptive consensus protocols (5.25)
and (5.26) based on the relative output information of neighboring agents in
the previous chapter can also be extended to solve the distributed tracking
problem. For instance, a node-based adaptive protocol can be proposed as
N
X
ṽ˙ i = (A + BF )ṽi + d˜i L aij [C(ṽi − ṽj ) − (yi − yj )] ,
j=1
   
XN  T XN   (6.61)
˙ y i − yj yi − y j 
d˜i = ǫi  aij Γ aij ,
C(ṽi − ṽj ) C(ṽi − ṽj )
j=1 j=1

ui = F ṽi , i = 2, · · · , N,

where ṽi ∈ Rn is the protocol state, i = 2, · · · , N , ṽ1 is generated by ṽ˙ 1 =

(A + BF )ṽ1 .

Remark 53 Comparing the protocols (6.60) and (6.61), to solve the distribut-
ed tracking problem with u1 , the latter one is more preferable due to two rea-
sons. First, the protocol (6.61) generally requires a lighter communication load
(i.e., a lower dimension of information to be transmitted) than (6.60). Note
that the protocol (6.61) exchanges yi and C ṽi between neighboring agents while
(6.60) exchanges v̂i . The sum of the dimensions of yi and C ṽi is generally low-
er than the dimension of v̂i , e.g., for the single output case with n > 2. Second,
the protocol (6.60) requires the absolute measures of the agents’ outputs, which
might not be available in some circumstances, e.g., the case where the agents
are equipped with only ultrasonic range sensors. On the other hand, an advan-
tage of (6.60) is that it can be modified (specifically, the protocol (6.50)) to
solve the distributed tracking problem for the case where the leader’s control
input is nonzero and time varying. In contrast, it is not an easy job to extend
(6.61) to achieve consensus for the case of a leader with nonzero input.
Distributed Tracking with a Leader of Possibly Nonzero Input 157

6.5 Simulation Examples

In this section, a numerical example is presented to illustrate the effectiveness
of the proposed theoretical results.

1 2 7

3 6

4 5

FIGURE 6.1: A leader-follower communication graph.

Example 21 The dynamics of the agents are given by (6.1), with

     
x 0 1 0  
xi = i1 , A = , B= , C= 1 0 .
xi2 −1 2 1
The communication graph is given as in FIGURE 6.1, where the node indexed
by 1 is the leader which is only accessible to the second node. Clearly all
the agents are unstable without control. For the leader, design a first-order
controller in the form of
υ̇ = −3υ − 2.5x11 ,
u1 = −8υ − 6x11 .
Then, the closed-loop dynamics of the leader have eigenvalues as −1 and ±ι.
In this case, u1 is clearly bounded. However, the bound γ, for which ku1 k ≤ γ,
depends on the initial state, which thereby might not be known to the followers.
Here we use the adaptive controller (6.50) to solve the distributed tracking
problem.  T
In (6.50), choose L̂ = − 3 6 such that A + L̂C is Hurwitz. Solving the
LMI (6.6) gives the gain matrices F̂ and Γ̂ in (6.50) as
 
  1.6857 4.3984
F̂ = − 1.2983 3.3878 , Γ̂ = .
4.3984 11.4769
To illustrate Theorem 45, select κi = 0.1, ϕi = 0.005, and τi = 2, i = 2, · · · , 7,
in (6.50). The tracking errors xi − x1 , i = 2, · · · , 7, of the agents under (6.50)
designed as above are shown in FIGURE 6.2(a), implying that distributed
tracking is indeed achieved. The coupling gains di associated with the followers
are drawn in FIGURE 6.2(b), which are clearly bounded.
158 Cooperative Control of Multi-Agent Systems

10
20

18
5
16

14

0 12
1
x −x

di
10
i

−5 8

−10 4

−15 0
0 10 20 30 40 50 0 10 20 30 40 50
t t

(a) (b)

FIGURE 6.2: (a) The tracking errors xi − x1 , i = 2, · · · , 7; (b) the coupling

gains di in (6.50).

6.6 Notes
The results in this chapter are mainly based on [92, 95]. In the literature,
there exist other works on distributed tracking control of multi-agent systems
with a leader of nonzero control input, e.g., [21, 109]. Specifically, in [21],
discontinuous controllers were studied for multi-agent systems with first- and
second-order integrator agent dynamics in the absence of velocity or accelera-
tion measurements. The authors in [109] addressed a distributed coordinated
tracking problem for multiple Euler-Lagrange systems with a dynamic leader.
It is worthwhile to mention that the distributed tracking controllers proposed
in [21, 109] are discontinuous, which will result in undesirable chattering phe-
nomenon in real applications. In this chapter, to deal with the effect of the
leader’s nonzero control input on consensus, we actually borrow ideas from the
sliding mode control literature and treat the leader’s control input as an exter-
nal disturbance satisfying the matching condition. Another possible approach
is to design certain learning algorithm using the relative state information to
estimate the leader’s nonzero control input and then add the estimate into
the consensus protocols. Related works along this line include [125, 192].
7
Containment Control of Linear Multi-Agent
Systems with Multiple Leaders

CONTENTS
7.1 Containment of Continuous-Time Multi-Agent Systems with Leaders
of Zero Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
7.1.1 Dynamic Containment Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . 161
7.1.2 Static Containment Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
7.2 Containment Control of Discrete-Time Multi-Agent Systems with
Leaders of Zero Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
7.2.1 Dynamic Containment Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . 165
7.2.2 Static Containment Controllers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.2.3 Simulation Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.3 Containment of Continuous-Time Multi-Agent Systems with Leaders
of Nonzero Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
7.3.1 Distributed Continuous Static Controllers . . . . . . . . . . . . . . . . . . 171
7.3.2 Adaptive Continuous Containment Controllers . . . . . . . . . . . . . 176
7.3.3 Simulation Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
7.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

Roughly speaking, existing consensus algorithms can be categorized into two

classes, namely, consensus without a leader and consensus with a leader. The
case of consensus with a leader is also called leader-follower consensus or dis-
tributed tracking. Both the leaderless and leader-follower consensus problems
have been addressed in the previous chapters. The distributed tracking prob-
lem deals with only one leader. However, in some practical applications, there
might exist more than one leader in agent networks. In the presence of multi-
ple leaders, the containment control problem arises where the followers are to
be driven into a given geometric space spanned by the leaders [67]. The study
of containment control has been motivated by many potential applications.
For instance, a group of autonomous vehicles (designated as leaders) equipped
with necessary sensors to detect the hazardous obstacles can be used to safely
maneuver another group of vehicles (designated as followers) from one target
to another, by ensuring that the followers are contained within the moving
safety area formed by the leaders [19, 67].
In this chapter, we extend the aforementioned papers to study the dis-
tributed containment control problem for multi-agent systems with general

159
160 Cooperative Control of Multi-Agent Systems

linear dynamics. Both the cases where the leaders have zero control inputs
and have nonzero unknown inputs are considered. For the simple case where
the leaders are of zero control inputs, the containment control problems for
both continuous-time and discrete-time multi-agent systems with general lin-
ear dynamics under directed communication topologies are considered in Sec-
tions 7.1 and 7.2, respectively. Distributed dynamic containment controllers
based on the relative outputs of neighboring agents are proposed for both the
continuous-time and discrete-time cases. In the continuous-time case, a multi-
step algorithm is presented to construct a dynamic containment controller,
under which the states of the followers will asymptotically converge to the
convex hull formed by those of the leaders, if for each follower there exists at
least one leader that has a directed path to that follower. In the discrete-time
case, in light of the modified algebraic Riccati equation, an algorithm is given
to design a dynamic containment controller that solves the containment con-
trol problem. Furthermore, as special cases of the dynamic controllers, static
containment controllers relying on the relative states of neighboring agents
are also discussed for both the continuous-time and discrete-time cases.
The containment control problem for the general case where the leaders
have nonzero, bounded, and time-varying control inputs is further investigated
in Section 7.3. Based on the relative states of neighboring agents, distributed
continuous static and adaptive controllers are designed, under which the con-
tainment error is uniformly ultimately bounded and the upper bound of the
containment error can be made arbitrarily small, if the subgraph associated
with the followers is undirected and for each follower there exists at least one
leader that has a directed path to that follower. Extensions to the case where
only local output information is available are discussed. Based on the rela-
tive estimates of the states of neighboring agents, distributed observer-based
containment controllers are proposed.

7.1 Containment of Continuous-Time Multi-Agent Sys-

tems with Leaders of Zero Inputs
Consider a group of N identical agents with general continuous-time linear
dynamics, described by
ẋi = Axi + Bui ,
(7.1)
yi = Cxi , i = 1, · · · , N,
where xi ∈ Rn , ui ∈ Rp , and yi ∈ Rq are, respectively, the state, the control
input, and the measured output of the i-th agent, and A, B, C, are constant
matrices with compatible dimensions.
We consider the case with multiple leaders. Suppose that there are M
(M < N ) followers and N − M leaders. An agent is called a leader if the
Containment Control of Multi-Agent Systems with Multiple Leaders 161

agent has no neighbor. An agent is called a follower if the agent has at least
one neighbor. Without loss of generality, we assume that the agents indexed
by 1, · · · , M , are followers, while the agents indexed by M + 1, · · · , N , are
leaders whose control inputs are set to be zero. We use R , {M + 1, · · · , N }
and F , {1, · · · , M } to denote, respectively, the leader set and the follower
set. In this section, we consider the case where the leaders have zero control
inputs, i.e., ui = 0, i ∈ R. The communication topology among the N agents
is represented by a directed graph G. Note that here the leaders do not receive
any information.

Assumption 7.1 Suppose that for each follower, there exists at least one
leader that has a directed path to that follower.

The objective is to design distributed controllers to solve the containment

control problem, defined as follows.

Definition 17 The containment control problem is solved for the agents in

(7.1) if the states of the followers under a certain distributed controller asymp-
totically converge to the convex hull formed by those of the leaders.

Clearly the containment control problem will reduce to the distributed

tracking problem discussed as in the previous chapters, when only one leader
exists.

7.1.1 Dynamic Containment Controllers

In this subsection, it is assumed that each agent has access to the relative
output measurements with respect to its neighbors. In this case, we propose
the following distributed dynamic containment controller for each follower as
X
v̇i = (A + BF )vi + cL aij [C(vi − vj ) − (yi − yj )],
j∈F ∪R (7.2)
ui = F vi , i ∈ F,

where vi ∈ Rn , i ∈ F, is the state of the controller corresponding to the

i-th follower, vj = 0 ∈ Rn for j ∈ R, aij is the (i, j)-th entry of the adja-
cency matrix associated with G, c > 0 ∈ R denotes the coupling gain, and
L ∈ Rn×q and F ∈ Rp×n are feedback gain matrices. In (7.2), the term
P
j∈F ∪R aij C(vi − vj ) means that the agents need to transmit the virtual
outputs of their corresponding controllers to their neighbors via the commu-
nication topology G.
The above distributed controller is an extension of the observer-type con-
sensus protocol (2.21) in Chapter 2 to the case with multiple leaders. The
other observer-type protocol (2.33) can be similarly extended. The details are
omitted here.
 162 Cooperative Control of Multi-Agent Systems

Denote by L the Laplacian matrix associated with G. Because the leaders

have no neighbors, L can be partitioned as
 
L1 L2
L= , (7.3)
0(N −M )×M 0(N −M )×(N −M )
where L1 ∈ RM ×M and L2 ∈ RM ×(N −M ) .
Lemma 30 ([111]) Under Assumption 7.1, all the eigenvalues of L1 has pos-
itive real parts, each entry of −L−1 −1
1 L2 is nonnegative, and each row of −L1 L2
has a sum equal to one.
Next, an algorithm is presented to select the control parameters in (7.2).
Algorithm 14 Under Assumption 7.1, the containment controller (7.2) can
be constructed as follows:
1) Choose the feedback gain matrix F such that A + BF is Hurwitz.
2) Take L = −Q−1 C T , where Q > 0 is a solution to the following linear matrix
inequality (LMI):
AT Q + QA − 2C T C < 0. (7.4)
3) Select the coupling gain c ≥ cth , with
1
cth = , (7.5)
min Re(λi )
i=1,··· ,M

where λi , i = 1, · · · , M , are the nonzero eigenvalues of L1 .

Remark 54 As shown in Proposition 3 in Chapter 2, a necessary and suffi-
cient condition on the existence of a positive-definite solution to the LMI (7.4)
is that (A, C) is detectable. Therefore, a necessary and sufficient condition on
the feasibility of Algorithm 14 is that (A, B, C) is stabilizable and detectable.
Remark 55 Similarly as in Chapter 2, the design method in Algorithm 14
essentially relies on the notion of consensus region. As discussed in Remark
6 in Chapter 2, Algorithm 14 has a favorable decoupling feature. Specifically,
the first two steps deal with only the agent dynamics and the feedback gain
matrices of (7.2), while the last step tackles the communication topology by
adjusting the coupling gain c.
Theorem 46 Assume that (A, B, C) be stabilizable and detectable. For a
directed communication graph G satisfying Assumption (7.1), the controller
(7.2) constructed by Algorithm 14 solves the containment control problem for
the agents described by (7.1). Specifically, limt→∞ (xf (t) − ̟x (t)) = 0 and
T T
limt→∞ vf (t) = 0, where xf = [xT1 , · · · , xTM ]T , vf = [v1T , · · · , vM ] ,
 
xM +1 (0)
−1 At  .. 
̟x (t) , (−L1 L2 ⊗ e )  . . (7.6)
xN (0)
Containment Control of Multi-Agent Systems with Multiple Leaders 163

Proof 47 Let zi = [xTi , viT ]T , zf = [z1T , · · · , zM

T T T
] , and zl = [zM T T
+1 , · · · , zN ] .
Then, the closed-loop network dynamics resulting from (7.1) and (7.2) can be
written as
żf = (IM ⊗ S + cL1 ⊗ H) zf + c(L2 ⊗ H)zl ,
(7.7)
żl = (IN −M ⊗ S) zl ,
where    
A BF 0 0
S= , H= .
0 A + BF −LC LC
P
Let ̺i = j∈F ∪R aij (zi − zj ), i ∈ F, and ̺ = [̺T1 , · · · , ̺TM ]T . Then, we have
̺ = (L1 ⊗ I2n )zf + (L2 ⊗ I2n )zl . (7.8)
From (7.8), it follows that ̺ = 0 if and only if zf (t) = −(L−1 1 L2 ⊗ I2n )zl (t),
which is equivalent to that xf = −(L−11 L 2 ⊗ I n )x l and v f = −(L −1
1 L2 ⊗ In )vl .
Then, in virtue of Lemma 30, we can get that the containment control problem
for the agents in (7.1) under the controller (7.2) is solved if ̺ asymptotically
converges to zero. Hereafter, we refer to ̺ as the containment error for (7.1)
under (7.2). Further, by noting that xj (t) = eAt xj (0) and limt→∞ vj (t) = 0,
j ∈ R, it then follows that if limt→∞ ̺(t) = 0, then limt→∞ (xf (t)−̟x (t)) = 0
and limt→∞ vf (t) = 0.
Considering the special structure of L as in (7.3), we can obtain from (7.7)
and (7.8) that the containment error ̺ satisfies the following dynamics:
̺˙ = (L1 ⊗ I2n )żf + (L2 ⊗ I2n )żl
= (L1 ⊗ I2n )[(IM ⊗ S)zf + c(L1 ⊗ H)zf + c(L2 ⊗ H)zl ]
+ (L2 ⊗ I2n )(IN −M ⊗ S)zl
(7.9)
= (L1 ⊗ S + cL21 ⊗ H)[(L−1 −1
1 ⊗ I2n )̺ − (L1 L2 ⊗ I2n )zl ]
+ (cL1 L2 ⊗ H + L2 ⊗ S)zl
= (IM ⊗ S + cL1 ⊗ H)̺.
Under Assumption 7.1, it follows from Lemma 30 that all the eigenvalues
of L1 have positive real parts. Let U ∈ CM ×M be such a unitary matrix that
U H L1 U = Λ, where Λ is an upper-triangular matrix with λi , i = 1, · · · , M ,
as its diagonal entries. Let ̺˜ , [˜̺T1 , · · · , ̺˜TN ]T = (U H ⊗ I2n )̺. Then, it follows
from (7.9) that
̺˜˙ = (IM ⊗ S + cΛ ⊗ H)˜ ̺. (7.10)
By noting that Λ is upper-triangular, it is clear that (7.10) is asymptotically
stable if and only if the following M systems:
̺˜˙ i = (S + cλi H)˜
̺i , i = 1, · · · , M, (7.11)
are simultaneously asymptotically
  stable. Multiplying the left and right sides
of S + cλi H by T = I0 −I
I and T −1
, respectively, we get
 
A + cλi LC 0
T (S + cλi H)T −1 = . (7.12)
−cλi LC A + BF
 164 Cooperative Control of Multi-Agent Systems

By steps 1) and 3) in Algorithm 14, it follows that there exists a Q > 0

satisfying
Q(A + cλi LC) + (A + cλi LC)H Q = QA + AT Q − 2cRe(λi )C T C
≤ QA + AT Q − 2C T C < 0.
That is, A + cλi LC, i = 1, · · · , M , are Hurwitz. Therefore, using (7.10),
(7.11), and (7.12), it follows that (7.9) is asymptotically stable, which implies
that the containment problem is solved.

7.1.2 Static Containment Controllers

In this subsection, a special case where the relative states between neigh-
boring agents are available is considered. For this case, a distributed static
containment controller is proposed as
X
ui = cK aij (xi − xj ), i ∈ F, (7.13)
j∈F ∪R

where c and aij are defined as in (7.2), and K ∈ Rp×n is the feedback gain
matrix to be designed. Let xf = [xT1 , · · · , xTM ]T and xl = [xTM +1 , · · · , xTN ]T .
Using (7.13) for (7.1) gives the closed-loop network dynamics as
ẋf = (IM ⊗ A + cL1 ⊗ BK)xf + c(L2 ⊗ BK)xl ,
ẋl = (IN −M ⊗ A)xl ,
where L1 and L2 are defined in (7.3).
The following algorithm is given for designing the control parameters in
(7.13).
Algorithm 15 Under Assumption 7.1, the controller (7.13) can be construct-
ed as follows:
1) Choose the feedback gain matrix K = −B T P −1 , where P > 0 is a solution
to the following LMI:
AP + P AT − 2BB T < 0. (7.14)

2) Select the coupling gain c ≥ cth , with cth given by (7.5).

Note that the LMI (7.14) is dual to the LMI (7.4). Thus, a sufficient and
necessary condition for the above algorithm is that (A, B) is stabilizable.
The following result can be derived by following similar steps as in the
proof of Theorem 46.
Corollary 8 Assume that (A, B) is stabilizable and the communication graph
G satisfies Assumption 7.1. Then, the static controller (7.13) constructed by
Algorithm 15 solves the containment control problem for the agents in (7.1).
Specifically, limt→∞ (xf (t) − ̟x (t)) = 0, where xf (t) and ̟x (t) are defined in
(7.6).
Containment Control of Multi-Agent Systems with Multiple Leaders 165

7.2 Containment Control of Discrete-Time Multi-Agent

Systems with Leaders of Zero Inputs
This section focuses on the discrete-time counterpart of the last section. Con-
sider a group of N identical agents with general discrete-time linear dynamics,
described by
x+i = Axi + Bui ,
(7.15)
yi = Cxi , i = 1, · · · , N,
where xi = xi (k) ∈ Rn , x+ p q
i = xi (k + 1), ui ∈ R , and yi ∈ R are, respective-
ly, the state at the current time instant, the state at the next time instant, the
control input, the measured output of the i-th agent. Without loss of gener-
ality, it is assumed throughout this section that B is of full column rank and
C has full row rank.
Similar to the last section, we assume that the agents indexed by 1, · · · , M ,
are followers, while the agents indexed by M +1, · · · , N , are leaders. The leader
set and the follower set are denoted, respectively, by R = {M + 1, · · · , N }
and F = {1, · · · , M }. We again let ui = 0, i ∈ R. The communication graph
G among the N agents is assumed to be directed and satisfy Assumption 7.1.

7.2.1 Dynamic Containment Controllers

Similar to the continuous-time case, we propose the following distributed dy-
namic containment controller for each follower as
X
v̂i+ = (A + BF )v̂i + L dij [C(v̂i − v̂j ) − (yi − yj )],
j∈F ∪R (7.16)
ui = F v̂i , i ∈ F,
where v̂i ∈ Rn , i ∈ F, is the state of the controller corresponding to the i-th
follower, v̂j = 0, j ∈ R, dij is the (i, j)-th entry of the row-stochastic matrix D
associated with G, and L ∈ Rn×q and F ∈ Rp×n are feedback gain matrices.
Because the last N − M agents are leaders that have no neighbors, D can
be partitioned as  
D1 D2
D= , (7.17)
0(N −M )×M IN −M
where D1 ∈ RM ×M and D2 ∈ RM ×(N −M ) .
Lemma 31 Under Assumption 7.1, all the eigenvalues of D1 lie in the open
unit disk, each entry of (IM − D1 )−1 D2 is nonnegative, and each row of (IM −
D1 )−1 D2 has a sum equal to one.
Proof 48 Consider the following new row-stochastic matrix:
 
D1 D2 1N −M
D= .
01×M 1
 166 Cooperative Control of Multi-Agent Systems

According to the definition of the directed spanning tree in Section 1.3.2 of

Chapter 1, the graph associated with D has a directed spanning tree if G sat-
isfies Assumption 7.1. Therefore, by Lemma 3.4 in [133], if Assumption 7.1
holds, then 1 is a simple eigenvalue of D and all the other eigenvalues of D
are in the open unit disk, which further implies that all the eigenvalues of D1
lie in the open unit disk.
Because every entry of D1 is nonnegative and the spectral radius of D1 is
less than 1, IM − D1 is a nonsingular M-matrix [3]. Therefore, each entry
of (IM − D1 )−1 is nonnegative [3]. Clearly, each entry of (IM − D1 )−1 D2 is
also nonnegative. Note that (IN − D)1N = 0, implying that (IM − D1 )1M −
D2 1N −M = 0. That is, (IM − D1 )−1 D2 1N −M = 1M , which shows that each
row of (IM − D1 )−1 D2 has a sum equal to 1.
Next, an algorithm for determining the control parameters in (7.16) is
presented.
Algorithm 16 Under Assumption 7.1, the containment controller (7.16) can
be constructed as follows:
1) The feedback gain matrix F is chosen such that A + BF is Schur stable.
2) Select L = −AP C T (CP C T )−1 , where P > 0 is the unique solution to the
following modified algebraic Riccati equation (MARE):
P = AP AT − (1 − max |λ̂i |2 )AP C T (CP C T )−1 CP AT + R, (7.18)
i=1,··· ,M

where R > 0 and λ̂i , i = 1, · · · , M are the eigenvalues of D1 .

Remark 56 The solvability of the MARE (7.18) can be found in Lemma 26 in
Chapter 3. As shown in Lemma 26, for the case where A has at least eigenvalue
outside the unit circle and C is of rank one, a sufficient condition for the
existence of the controller (7.16) by using Algorithm 16 is that (A, B, C) is
stabilizable and detectable, and max |λ̂i |2 < 1− Π|λu1(A)| . For the case where
i=1,··· ,M i i

A has no eigenvalues with magnitude larger than 1, the sufficient condition is

reduced to that (A, B, C) is stabilizable and detectable.
The following theorem shows that the controller (7.16) given by Algorithm
16 indeed solves the containment problem.
Theorem 47 Suppose that the directed communication graph G satisfies
Assumption 7.1. Then, the controller (7.16) given by Algorithm 16 solves
the containment control problem for the agents described by (7.15). Specif-
ically, limk→∞ (xf (k) − ψx (k)) = 0 and limk→∞ v̂f (k) = 0, where xf =
[xT1 , · · · , xTM ]T , and
 
xM +1 (0)
  .. 
ψx (k) , (IM − D1 )−1 D2 ⊗ Ak  . . (7.19)
xN (0)
Containment Control of Multi-Agent Systems with Multiple Leaders 167

Proof 49 Let ẑi = [xTi , v̂iT ]T , ẑf = [ẑ1T , · · · , ẑM

T T T
] , and ẑl = [ẑM T T
+1 , · · · , ẑN ] .
Then, we can obtain from (7.15) and (7.16) that the collective network dynam-
ics can be written as
ẑf+ = [IM ⊗ S + (IM − D1 ) ⊗ H]ẑf − (D2 ⊗ H)ẑl ,
(7.20)
ẑl+ = (IN −M ⊗ S)ẑl ,
P
where S and H are defined in (7.7). Let ζi = j∈F ∪R dij (ẑi − ẑj ), i ∈ F, and
ζ = [ζ1T , · · · , ζM
T T
] . Then, we have
ζ = [(IM − D1 ) ⊗ I2n ]ẑf − (D2 ⊗ I2n )ẑl . (7.21)
Similarly as shown in the proof of Theorem 46, ζ = 0 if and only if zf (k) =
[(IM − D1 )−1 D2 ⊗ I2n ]zl (k), which implies that limk→∞ (xf (k) − ψx (k)) = 0
and limk→∞ v̂f (k) = 0. In virtue of Lemma 31, we know that the contain-
ment problem of the agents in (7.15) under the controller (7.16) is solved if
limk→∞ ζ = 0. By following similar steps to those in the proof of Theorem 46,
we can obtain from (7.20) and (7.21) that ζ satisfies the following dynamics:
ζ + = [IM ⊗ S + (IM − D1 ) ⊗ H]ζ. (7.22)
Under Assumption 7.1, it follows from Lemma 31 that all the eigenvalues
of IM − D1 have positive real parts. Let Û ∈ CM ×M be such a unitary matrix
that Û H (IM − D1 )Û = Λ̂, where Λ̂ is an upper-triangular matrix with 1 − λ̂i ,
i = 1, · · · , M , on the diagonal. Let ζ̃ , [ζ̃1T , · · · , ζ̃N
T T
] = (Û H ⊗ I2n )ζ. Then,
(7.22) can be rewritten as

ζ̃ + = (IM ⊗ S + Λ̂ ⊗ H)ζ̃, (7.23)

Clearly, (7.23) is asymptotically stable if and if the following M systems:

ζ̃i+ = [S + (1 − λ̂i )H]ζ̃i , i = 1, · · · , M, (7.24)

are simultaneously asymptoticallyh stable. As shown
i in the proof of Theorem
A+(1−λ̂i )LC 0
46, S + (1 − λ̂i )H is similar to −(1−λ̂ )LC A+BF , i = 1, · · · , M . In light of
i
step 2) in Algorithm 16, we can obtain

[A + (1 − λ̂i )LC]P [A + (1 − λ̂i )LC]H − P

= AP AT − P + [−2Re(1 − λ̂i ) + |1 − λ̂i |2 ]AP C T (CP C T )−1 CP AT
= AP AT − P + (|λ̂i |2 − 1)AP C T (CP C T )−1 CP AT
≤ AP AT − P − (1 − max |λ̂i |2 )AP C T (CP C T )−1 CP AT
i=1,··· ,M

= −R < 0.
(7.25)
Then, (7.25) implies that A + (1 − λ̂i )LC, i = 1, · · · , M , are Schur stable.
Therefore, considering (7.23) and (7.24), we obtain that (7.22) is asymptoti-
cally stable, i.e., the containment control problem is solved.
168 Cooperative Control of Multi-Agent Systems

Remark 57 Theorem 47 gives the discrete-time counterpart of the results

in Theorem 46. In contrast to the continuous-time case where the Laplacian
matrix L is used to represent the communication graph, the row-stochastic
matrix D is utilized here in the discrete-time case. Different from Theorem 46
in the last subsection, Algorithm 16 for constructing discrete-time containment
controllers and the proof of Theorem 47 rely on the modified algebraic Riccati
equation. Furthermore, the eigenvalues of D in the discrete-time case have
to further satisfy a constraint related to the unstable eigenvalues of the state
matrix A, when A has eigenvalues outside the unit circle.

7.2.2 Static Containment Controllers

In this subsection, a special case where the relative states between neigh-
boring agents are available is considered. For this case, a distributed static
containment controller is proposed as
X
ui = K dij (xi − xj ), i ∈ F, (7.26)
j∈F ∪R

where dij is defined as in (7.16) and K ∈ Rp×n is the feedback gain matrix
to be designed.
Let xf = [xT1 , · · · , xTM ]T and xl = [xTM +1 , · · · , xTN ]T . Using (7.26) for (7.1)
gives the closed-loop network dynamics as

x+
f = [IN ⊗ A + (IM − D1 ) ⊗ BK]xf − (D2 ⊗ BK)xl ,
x+
l = (IN −M ⊗ A)xl ,

where D1 and D2 are defined in (7.17).

Corollary 9 Assume that the communication graph G satisfies Assumption

7.1. Then, the controller (7.26) with K = −(B T QB)−1 B T QA solve the con-
tainment control problem, where Q > 0 is the unique solution to the following
MARE:

Q = AT QA − (1 − max |λ̂i |2 )AT QB(B T QB)−1 B T QA + R, (7.27)

i=1,··· ,M

with R > 0 and λ̂i , i = 1, · · · , M , being the eigenvalues of D1 . Specifically,

limk→∞ (xf (k) − ψx (k)) = 0, where xf (k) and ψx (k) are defined in (7.19).

7.2.3 Simulation Examples

In this subsection, a simulation example is provided to validate the effective-
ness of the theoretical results.
Containment Control of Multi-Agent Systems with Multiple Leaders 169

1 6 7

2 5 8

3 4 9

FIGURE 7.1: The communication topology with multiple leaders.

Example 22 In this example, we take the agents in (7.1) as the Caltech

multi-vehicle wireless testbed vehicles, whose linearized dynamics can be de-
scribed by (7.1) [51], with
 T
xi = xi1 xi2 xi3 ẋi1 ẋi2 ẋi3 ,
 
0 0 0 1 0 0
0 0 0 0 1 0 
 
0 0 0 0 0 1 
A= 0
,

 0 −0.2003 −0.2003 0 0 
0 0 0.2003 0 −0.2003 0 
0 0 0 0 0 −1.6129
 
0 0
 0 0 
 
 0 0 

B= ,
 0.9441 0.9441 

 0.9441 0.9441 
−28.7097 28.7097

where xi1 and xi2 are, respectively, the positions of the i-th vehicle along the
x and y coordinates, xi3 is the orientation of the i-th vehicle. The objective is
to design a static containment controller in the form of (7.13) by using local
state information of neighboring vehicles.
Solving the LMI (7.14) by using the LMI toolbox of MATLAB gives the
feedback gain matrix of (7.13) as
 
−0.0089 0.0068 0.0389 −0.0329 0.0180 0.0538
K= .
0.0068 −0.0089 −0.0389 0.0180 −0.0329 −0.0538

For illustration, let the communication graph G be given by FIGURE 7.1,

where nodes 7, 8, 9 are three leaders and the others are followers. It can be
verified that G satisfies Assumption 7.1. Correspondingly, the matrix L1 in
170 Cooperative Control of Multi-Agent Systems
3
10 60

2.5
0 50
2
−10
40
1.5
−20
30 1

i3
xi2
i1

x
−30
x

0.5
20
−40
0
10
−50 −0.5

0
−60 −1

−70 −10 −1.5

0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50
t t t

FIGURE 7.2: The positions and orientations of the nine vehicles under (7.13).
The solid and dashdotted lines denote, respectively, the trajectories of the
leaders and the followers.

(7.3) is  
3 0 0 −1 −1 −1
−1 1 0 0 0 0
 
−1 −1 2 0 0 0
L1 = 
−1
,
 0 0 2 0 0 
0 0 0 −1 2 0
0 0 0 0 −1 2
whose eigenvalues are 0.8213, 1, 2, 2.3329 ± ι0.6708, 3.5129. By Algorithm
15, we choose the coupling gain c ≥ 1.2176. The positions and orientations of
the nine vehicles under the controller (7.13) with K as above and c = 2 are
depicted in FIGURE 7.2, from which it can be observed that the containment
control problem is indeed solved.

7.3 Containment of Continuous-Time Multi-Agent Sys-

tems with Leaders of Nonzero Inputs
In Section 7.1, the control inputs of the leaders are assumed to be zero, which
is restrictive in many circumstances. In real applications, the leaders might
need nonzero control actions to regulate their state trajectories, e.g., to avoid
obstacles or to form a desirable safety area.
The focus of this section is to extend Section 7.1 to consider the general
case where the leaders’ control inputs are possibly nonzero and time varying.
Note that due to the nonzero control inputs of the leaders which are available
to at most a subset of follower, the containment control problem is more
challenging.
Suppose that the following mild assumption holds.
Containment Control of Multi-Agent Systems with Multiple Leaders 171

Assumption 7.2 The leaders’ control inputs ui , i ∈ R, are bounded, i.e.,

kui k ≤ γi , i ∈ R, where γi are positive constants.

The communication graph G among the N agents in this case is assumed

to satisfy the following assumption, which is more stringent than Assumption
(7.1).

Assumption 7.3 The subgraph Gs associated with the M followers is undi-

rected. For each follower, there exists at least one leader that has a directed
path to that follower.

Similarly as in (7.3), in this case L can be partitioned as

 
L1 L2
L= , (7.28)
0(N −M )×M 0(N −M )×(N −M )

where L1 ∈ RM ×M is symmetric and L2 ∈ RM ×(N −M ) .

The following result is a special case of Lemma 31.

Lemma 32 ([111]) Under Assumption 7.3, all the eigenvalues of L1 are pos-
itive, each entry of −L−1 −1
1 L2 is nonnegative, and each row of −L1 L2 has a
sum equal to one.

7.3.1 Distributed Continuous Static Controllers

In order to solve the containment problem, based on the relative state in-
formation of neighboring agents, we propose a distributed continuous static
controller for each follower as
N
X N
X
ui = c 1 K aij (xi − xj ) + c2 gi (K aij (xi − xj )), i ∈ F, (7.29)
j=1 j=1

where c1 > 0 and c2 > 0 ∈ R are constant coupling gains, K ∈ Rp×n is

the feedback gain matrix, aij is the (i, j)-th entry of the adjacency matrix
associated with G, and the nonlinear functions gi (·) are defined such that for
w ∈ Rn , (
w
if kwk > κi
gi (w) = kwk
w
(7.30)
κi if kwk ≤ κi
with κi being small positive scalars.

Remark 58 The above containment controller (7.29) is a direct extension of

the tracking controller (6.22) in the previous chapter to the case with multiple
leaders. If only one leader exists, (7.29) clearly reduces to (6.22). In (7.29), the
information of the leaders’s control inputs ui , i ∈ R is actually
PNnot used. Com-
paring (7.29) to (7.13), it can observed that the term c2 gi (K j=1 aij (xi −xj ))
172 Cooperative Control of Multi-Agent Systems

in (7.29) is introduced to deal with the effect of the nonzero ui , i ∈ R. For the
special case where ui = 0, i ∈ R, (7.29) can be reduced to (7.13) by removing
PN
c2 gi (K j=1 aij (xi − xj )).

Remark 59 Note that the nonlinear functions gi (·) in (7.30) are continuous,
which are actually continuous approximations, via the boundary layer approach
[26, 71], of the discontinuous function:
(
w
if kwk =
6 0
ĝ(w) = kwk (7.31)
0 if kwk = 0.

The values of κi in (7.30) define the widths of the boundary layers. As κi → 0,

the continuous functions gi (·) approach the discontinuous function ĝ(·).

Let xf = [xT1 , · · · , xTM ]T , xl = [xTM +1 , · · · , xTN ]T , x = [xTf , xTl ]T , and ul =

[uTM +1 , · · · , uTN ]T . Then, it follows from (7.1) and (7.29) that the closed-loop
network dynamics can be written as

ẋf = (IM ⊗ A + c1 L1 ⊗ BK)xf + c1 (L2 ⊗ BK)xl + c2 (IM ⊗ B)G(x),

ẋl = (IN −M ⊗ A)xf + (IN −M ⊗ B)ul ,
(7.32)
where L1 and L2 are defined as in (7.28), and
 PN 
g1 (K j=1 a1j (x1 − xj ))
 .. 
G(x) ,  .
.

PN
gN (K j=1 aM j (xM − xj ))

Introduce the following variable:

ξ , xf + (L−1
1 L2 ⊗ In )xl , (7.33)
T T
where ξ = [ξ1T , · · · , ξM ] . From (7.33), it is easy to see that ξ = 0 if and
−1
only if xf = (−L1 L2 ⊗ In )xl . In virtue of Lemma 32, we can get that the
containment control problem of the agents in (7.1) under the controller (7.29)
is solved if ξ converges to zero. Hereafter, we refer to ξ as the containment
error for (7.1) under (7.29). By (7.33) and (7.32), it is not difficult to obtain
that ξ satisfies the following dynamics:

ξ˙ = (IM ⊗ A + c1 L1 ⊗ BK)ξ + c2 (IM ⊗ B)G(ξ)

b + (L−1
1 L2 ⊗ B)ul , (7.34)

where  PM 
g1 (K j=1 L1j ξj )
 .. 
b
G(ξ) , , (7.35)
 . 
PM
gN (K j=1 L M j ξj )
Containment Control of Multi-Agent Systems with Multiple Leaders 173
b
with Lij denoting the (i, j)-th entry of L1 . Actually, G(ξ) = G(x), which
PN PM
follows from the fact that j=1 aij (xi − xj ) = j=1 Lij ξj , i = 1, · · · , M .
The following theorem states the ultimate boundedness of the containment
error ξ.

Theorem 48 Assume that Assumptions 7.2 and 7.3 hold. The parameters in
the containment controller (7.29) are designed as c1 ≥ λmin1(L1 ) , c2 ≥ γmax ,
and K = −B T P −1 , where γmax , max γi and P > 0 is a solution to the
i∈R
LMI (7.14). Then, the containment error ξ of (7.34) is uniformly ultimately
bounded and exponentially converges to the residual set
M
2λmax (P )γmax X
D1 , {ξ : kξk2 ≤ κi }, (7.36)
αλmin (L1 ) i=1

where
−λmax (AP + P AT − 2BB T )
α= . (7.37)
λmax (P )

Proof 50 Consider the following Lyapunov function candidate:

1 T
V1 = ξ (L1 ⊗ P −1 )ξ. (7.38)
2
It follows from Lemma 32 that L1 > 0, so V1 is clearly positive definite. The
time derivative of V1 along the trajectory of (7.34) is given by

V̇1 = ξ T (L1 ⊗ P −1 )ξ˙

= ξ T (L1 ⊗ P −1 A + c1 L21 ⊗ P −1 BK)ξ
b (7.39)
+ c2 ξ T (L1 ⊗ P −1 B)G(ξ) + ξ T (L2 ⊗ P −1 B)ul
1 T b
= ξ X ξ + c2 ξ T (L1 ⊗ P −1 B)G(ξ) + ξ T (L2 ⊗ P −1 B)ul ,
2
where

X = L1 ⊗ (P −1 A + AT P −1 ) − 2c1 L21 ⊗ P −1 BB T P −1 . (7.40)

Let bij denote the (i, j)-th entry of −L−1

1 L2 , which by Lemma 32, satisfies
PN −M
that bij ≥ 0 and j=1 bij = 1 for i = 1, · · · , M . In virtue of Assumption
174 Cooperative Control of Multi-Agent Systems

7.2, we have

ξ T (L2 ⊗ P −1 B)ul
= ξ T (L1 ⊗ In )(L−1
1 L2 ⊗ P
−1
B)ul
 PN −M 
hP i k=1 b1k P −1 Buk
M PM  .. 
=− j=1 L1j ξjT ··· j=1 LM j ξjT  . 
PN −M
k=1 bM k P −1 Buk
M X
X M −M
NX
=− Lij ξjT P −1 B bjk uk
i=1 j=1 k=1
M
X M
X −M
NX
≤ kB T P −1 Lij ξj k bjk kuk k
i=1 j=1 k=1
M
X M
X
≤ γmax kB T P −1 Lij ξj k.
i=1 j=1
(7.41)
Next, Pconsider the following three cases:P
M M
i) kK j=1 Lij ξj k > κi , i.e., kB T P −1 j=1 Lij ξj k > κi , i = 1, · · · , M .
In this case, it follows from (7.30) and (7.35) that
M
X M
X
b
ξ T (L1 ⊗ P −1 B)G(ξ) =− kB T P −1 Lij ξj k. (7.42)
i=1 j=1

Then, we can get from (7.39), (7.41), and (7.42) that

XM XM
1 T
V̇1 ≤ ξ X ξ − (c2 − γmax ) kB T P −1 Lij ξj k
2 i=1 j=1
1 T
≤ ξ X ξ.
2
PM
ii) kB T P −1 j=1 Lij ξj k ≤ κi , i = 1, · · · , M .
From (7.41), we can obtain that
M
X
ξ T (L2 ⊗ P −1 B)ul ≤ γmax κi . (7.43)
i=1

Further, it follows from (7.30), (7.35), and (7.42) that

M M
b 1 X T −1 X
ξ T (L1 ⊗ P −1 B)G(ξ) =− kB P Lij ξj k2 ≤ 0. (7.44)
κi i=1 j=1
Containment Control of Multi-Agent Systems with Multiple Leaders 175

Thus, we get from (7.39), (7.43), and (7.44) that

XM
1 T
V̇1 ≤ ξ X ξ + γmax κi . (7.45)
2 i=1

iii) ξ satisfies neither Case i) nor Case ii).

PM
Without loss of generality, assume that kB T P −1 j=1 Lij ξj k > κi , i =
PM
1, · · · , l, and kB T P −1 j=1 Lij ξj k ≤ κi , i = l + 1, · · · , M . It is easy to see
from (7.41), (7.30), and (7.42) that
l
X M
X M
X −l
ξ T (L2 ⊗ P −1 B)ul ≤ γmax [ kB T P −1 Lij ξj k + κi ],
i=1 j=1 i=1
l
X M
X
b
ξ T (L1 ⊗ P −1 B)G(ξ) ≤− kB T P −1 Lij ξj k.
i=1 j=1

Clearly, in this case we have

M
X −l
1 T
V̇1 ≤ ξ X ξ + γmax κi .
2 i=1

Therefore, by analyzing the above three cases, we get that V̇1 satisfies (7.45)
for all ξ ∈ RM n . Note that (7.45) can be rewritten as
XM
1 T
V̇1 ≤ −αV1 + αV1 + ξ X ξ + γmax κi
2 i=1
(7.46)
XM
1
= −αV1 + ξ T (X + αL1 ⊗ P −1 )ξ + γmax κi .
2 i=1

−λmax (AP +P AT −2BB T )

Because α = λmax (P ) , in light of (7.14), we can obtain that
− 21 − 12
(L1 ⊗ P )(X + αL1 ⊗ P −1 )(L1 ⊗ P)
≤ IN ⊗ [AP + P A + αP − 2BB T ] < 0.
T

That is, X + αL1 ⊗ P −1 < 0. Then, it follows from (7.46) that

M
X
V̇1 ≤ −αV1 + γmax κi . (7.47)
i=1

By using Lemma 16 (the Comparison lemma), we can obtain from (7.47) that
PM PM
γmax i=1 κi −αt γmax i=1 κi (7.48)
V1 (ξ) ≤ [V1 (ξ(0)) − ]e + ,
α α
which implies that ξ exponentially converges to the residual set D1 in (7.36)
with a convergence rate not less than e−αt .
176 Cooperative Control of Multi-Agent Systems

Remark 60 Note that the residual set D1 of the containment error ξ depend-
s on the communication graph G, the number of followers, the upper bounds
of the leader’s control inputs, and the widths κi of the boundary layers. By
choosing sufficiently small κi , the containment error ξ under the continuous
controller (7.29) can be arbitrarily small, which is acceptable in most cir-
cumstances. It can be shown that the containment error ξ can asymptotically
converge to zero under the controller (7.29) with gi (·) replaced by ĝ(·). The
advantage of the continuous controller (7.29) is that it can avoid the unde-
sirable the chattering effect caused by (7.29) with gi (·) replaced by ĝ(·). The
tradeoff is that the continuous controller (7.29) does not guarantee asymptotic
stability.

7.3.2 Adaptive Continuous Containment Controllers

In the last subsection, to design the controller (7.29) we have to use the
minimal eigenvalue λmin (L1 ) of L1 and the upper bound γmax of the leaders’
control inputs. However, λmin (L1 ) is global information in the sense that each
follower has to know the entire communication graph to compute it, and it is
not practical to assume that the upper bound γmax are explicitly known to all
followers. In this subsection, we intend to design fully distributed controllers
to solve the containment problem without requiring λmin (L1 ) nor γmax .
Based on the relative states of neighboring agents, we propose the following
distributed controller with an adaptive law for updating the coupling gain
(weight) for each follower:
N
X N
X
ui = d i K aij (xi − xj ) + di ri (K aij (xi − xj )),
j=1 j=1
N
X N
X
d˙i = τi [−ϕi di + ( aij (xi − xj )T )Γ( aij (xi − xj )) (7.49)
j=1 j=1
N
X
+ kK aij (xi − xj )k], i = 1, · · · , M,
j=1

where di (t) denotes the time-varying coupling gain (weight) associated with
the i-th follower, ϕi are small positive constants, Γ ∈ Rn×n is the feedback
gain matrix, τi are positive scalars, the nonlinear functions ri (·) are defined
such that for w ∈ Rn ,
(
w
if di kwk > κi
ri (w) = kwk
w
(7.50)
κ i di if di kwk ≤ κi

and the rest of the variables are defined as in (7.29).

Let xf , xl , x, ul , and ξ be defined as in (7.34) and (7.33). Let D(t) =
Containment Control of Multi-Agent Systems with Multiple Leaders 177

diag(d1 (t), · · · , dM (t)). Then, it follows from (7.1) and (7.49) that the con-
tainment error ξ and the coupling gains D(t) satisfy the following dynamics:

ξ˙ = ẋf + (L−1
1 L2 ⊗ In )ẋl
= (I ⊗ A + DL1 ⊗ BK)xf + (DL2 ⊗ BK)xl + (D ⊗ B)R(x)
+ (L−1 −1
1 L2 ⊗ A)xl + (L1 L2 ⊗ B)ul
= (IM ⊗ A + DL1 ⊗ BK)ξ + (D ⊗ B)R(ξ) + (L−1
1 L2 ⊗ B)ul ,
M
X M
X M
X
d˙i = τi [−ϕi di + ( Lij ξjT )Γ( Lij ξj ) + kK Lij ξj k], i = 1, · · · , M,
j=1 j=1 j=1

 PM  (7.51)
r1 (K j=1 L1j ξj )

where R(ξ) =  .. .
PM.
rM (K j=1 LM j ξj )
The following theorem shows the ultimate boundedness of the states ξ and
di of (7.51).

Theorem 49 Suppose that Assumptions 7.2 and 7.3 hold. The feedback gain
matrices of the adaptive controller (7.49) are designed as K = −B T P −1 and
Γ = P −1 BB T P −1 , where P > 0 is a solution to the LMI (7.14). Then, both
the containment error ξ and the coupling gains di , i = 1, · · · , M , in (7.51)
are uniformly ultimately bounded. Furthermore, if ϕi are chosen such that
̺ , mini=1,··· ,M ϕi τi < α, where α is defined as in (7.37), then ξ exponentially
converges to the residual set

X M
λmax (P ) 1
D2 , {ξ : kξk2 ≤ (β 2 ϕi + κi )}, (7.52)
λmin (L1 )(α − ̺) i=1 2

where β = max{ γmax , λmin1(L1 ) }.

Proof 51 Let d˜i = di − β, i = 1, · · · , M . Then, (7.51) can be rewritten as

ξ˙ = (IM ⊗ A + DL1 ⊗ BK)ξ + (D ⊗ B)R(ξ) + (L−1

1 L2 ⊗ B)ul ,
X M X M X M
˙
d˜i = τi [−ϕi (d˜i + β) + ( Lij ξjT )Γ( Lij ξj ) + kK Lij ξj k], i = 1, · · · , M,
j=1 j=1 j=1
(7.53)
where D = diag(d˜1 + β, · · · , d˜M + β).
Consider the following Lyapunov function candidate

XM ˜2
1 T −1 di
V2 = ξ (L1 ⊗ P )ξ + ,
2 i=1
2τ i

As stated in the proof of Theorem 48, it is easy to see that V2 is positive

178 Cooperative Control of Multi-Agent Systems

definite. The time derivative of V2 along (7.53) can be obtained as

V̇2 = ξ T (L1 ⊗ P −1 A + L1 DL1 ⊗ P −1 BK)ξ

+ ξ T (L1 D ⊗ P −1 B)R(ξ) + ξ T (L2 ⊗ P −1 B)ul
M
X M
X M
X M
X
+ d˜i [−ϕi (d˜i + β) + ( Lij ξjT )Γ( Lij ξj ) + kK Lij ξj k].
i=1 j=1 j=1 j=1
(7.54)
By substituting K = −BP −1 , it is easy to get that
M
X M
X M
X
ξ T (L1 DL1 ⊗ P −1 BK)ξ = − (d˜i + β)[ Lij ξj ]T P −1 BB T P −1 [ Lij ξj ].
i=1 j=1 j=1
(7.55)
PM
For the case where di kK L ξ
j=1 ij j k > κi , i = 1, · · · , M , we can get from
(7.50) that
M
X M
X
ξ T (L1 D ⊗ P −1 B)R(ξ) = − (d˜i + β)kB T P −1 Lij ξj k. (7.56)
i=1 j=1

Substituting (7.55), (7.56), and (7.41) into (7.54) yields

M X
X M M
X
V̇2 ≤ ξ T (L1 ⊗ P −1 A)ξ − β [ Lij ξj ]T P −1 BB T P −1 [ Lij ξj ]
i=1 j=1 j=1
M
X M
X M
X
− (β − γmax ) kB T P −1 Lij ξj k − ϕi (d˜2i + d˜i β)
i=1 j=1 i=1
M
X
1 T 1
≤ ξ Zξ + ϕi (−d˜2i + β 2 ),
2 2 i=1

where we have used the fact that β ≥ max γi and −d˜2i − d˜i β ≤ − 21 d˜2i + 21 β 2 to
i∈R
get the last inequality and Z = L1 ⊗ (P −1 A + AT P −1 ) − 2βL21 ⊗ P −1 BBP −1 .
PM
For the case where di kK j=1 Lij ξj k ≤ κi , i = 1, · · · , M , we can get from
(7.50) that
M
X M
X
(d˜i + β)2
ξ T (L1 D ⊗ P −1 B)R(ξ) = − kB T P −1 Lij ξj k2 . (7.57)
i=1
κi j=1
Containment Control of Multi-Agent Systems with Multiple Leaders 179

Then, it follows from (7.55), (7.57), (7.41), and (7.54) that

XM XM M
1 T (d˜i + β)2 T −1 X
V̇2 ≤ ξ Zξ + ϕi (d˜2i + d˜i β) − kB P Lij ξj k2
2 i=1 i=1
κ i j=1
M
X M
X
+ (d˜i + β)kB T P −1 Lij ξj k (7.58)
i=1 j=1
M M
1 T 1X X 1
≤ ξ Zξ + ϕi (−d˜2i + β 2 ) + κi .
2 2 i=1 i=1
4

Note that to get the last inequality in (7.58), we have used the following fact:
M M
(d˜i + β)2 T −1 X X 1
− kB P Lij ξj k2 + (d˜i + β)kB T P −1 Lij ξj k ≤ κi ,
κi j=1 j=1
4
PM
for (d˜i + β)kK j=1 Lij ξj k ≤ κi , i = 1, · · · , M .
PM
For the case where di kK j=1 Lij ξj k ≤ κi , i = 1, · · · , l, and
PM
di kK j=1 Lij ξj k > κi , i = l + 1, · · · , M . By following the steps in the t-
wo cases above, it is easy to get that
M M −l
1 T 1X X 1
V̇2 ≤ ξ Zξ + ϕi (−d˜2i + β 2 ) + κi .
2 2 i=1 i=1
4

Therefore, V̇2 satisfies (7.58) for all ξ ∈ RN n . Because βλmin (L1 ) ≥ 1,

by following similar steps as in the proof of Theorem
PM 48, it is easy to shown
that Z < −αL1 ⊗ P −1 < 0 and thereby ξ T Zξ − i=1 ϕi d˜2i < 0. In virtue of
Lemma 15, we get that the states ξ and di of (7.51) are uniformly ultimately
bounded.
Next, we will derive the residual set for the containment error ξ. Rewrite
(7.58) into
M
1 1X ̺
V̇2 ≤ −̺V2 + ξ T (Z + αL1 ⊗ P −1 )ξ − (ϕi − )d˜2i
2 2 i=1 τi
M
α−̺ T 1X 2 1
− ξ (L1 ⊗ P −1 )ξ + (β ϕi + κi ) (7.59)
2 2 i=1 2
M
λmin (L1 )(α − ̺) 1X 2 1
≤ −̺V2 − kξk2 + (β ϕi + κi ).
2λmax (P ) 2 i=1 2
λmax (P ) PM
Obviously, it follows from (7.59) that V̇2 ≤ −̺V2 if kξk2 > λmin (L1 )(α−̺) i=1
λmin (L1 )
(β 2 ϕi + 21 κi ). Then, by noting V2 ≥ 2λ max (P )
kξk2 , we can get that if ̺ ≤ α then
ξ exponentially converges to the residual set D2 in (7.52) with a convergence
rate faster than e−̺t .
180 Cooperative Control of Multi-Agent Systems

Remark 61 As shown in Theorem 49, contrary to the static containment

controller (7.29), the design of the adaptive controller (7.49) relies on only
the agent dynamics and the relative state information, requiring neither the
minimal eigenvalue λmin (L1 ) nor the upper bounds of the leaders’ control in-
put. In (7.49), both the σ-modification technique in [63] and the boundary layer
concept are used to guarantee the ultimate boundedness of the containment er-
ror ξ and the adaptive gains di . It can be observed that apart from the facts in
(7.36), the residual set D2 in (7.52) also depends on the small constants ϕi .
We can choose ϕi and κi to be relatively small in order to guarantee a small
containment error ξ.

For the case where the relative state information is not available, the ideas
in Section 6.4 can be extended to construct dynamic containment controllers
using the relative estimates of the states of neighboring agents. The details
are omitted here. Interested readers can do it as an exercise.

7.3.3 Simulation Examples

In this subsection, a simulation example is provided to validate the effective-
ness of the theoretical results.

7 1 6

8 2 5

3 4

FIGURE 7.3: A communication graph with two leaders.

Consider a network of eight agents. For illustration, let the communication

graph among the agents be given as in FIGURE 7.3, where nodes 7 and 8 are
two leaders and the others are followers. The dynamics of the agents are given
by (7.1), with      
xi1 0 1 0
xi = , A= , B= ,
xi2 −1 1 1
Design the control inputs for the leaders
 as u7 = K7 x7 + 4 sin(2t) and u8 =
K8 x8 + 2 cos(t), with K7 = − 0 2 and K8 = − 1 3 . It is easy to see that
in this case u7 and u8 are bounded. Here we use the adaptive control (7.49)
to solve the containment control problem.
Containment Control of Multi-Agent Systems with Multiple Leaders 181
2 2

1.5 1.5

1 1

0.5 0.5

xi2
xi1

0 0

−0.5 −0.5

−1 −1

−1.5 −1.5

−2 −2
0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30 35 40

t t

FIGURE 7.4: The state trajectories of the agents. The solid and dashdotted
lines denote, respectively, the trajectories of the leaders and the followers.

Solving the LMI (7.14) by using the SeDuMi toolbox [158] gives the gain
matrices K and Γ in (7.49) as
 
  2.6255 7.7075
K = − 1.6203 4.7567 , Γ = .
7.7075 22.6266

To illustrate Theorem 49, select κi = 0.1, ϕi = 0.005, and τi = 5, i = 1, · · · , 6,

in (7.49). The state trajectories xi (t) of the agents under (7.49) designed as
above are depicted in FIGURE 7.4, implying that the containment control
problem is indeed solved. The coupling gains di associated with the followers
are drawn in FIGURE 7.5, which are clearly bounded.

20

15

10
i
d

−5
0 5 10 15 20 25 30 35 40
t

FIGURE 7.5: The coupling gains di in (7.49).

182 Cooperative Control of Multi-Agent Systems

7.4 Notes
The materials of this chapter are mainly based on [88, 94]. For further results
on containment control of multi-agent systems, see [16, 18, 19, 36, 46, 67, 104,
110, 111]. In particular, a hybrid containment control law was proposed in [67]
to drive the followers into the convex hull spanned by the leaders. Distribut-
ed containment control problems were studied in [16, 18, 19] for a group of
first-order and second-order integrator agents under fixed and switching di-
rected communication topologies. The containment control was considered in
[104] for second-order multi-agent systems with random switching topologies.
A hybrid model predictive control scheme was proposed in [46] to solve the
containment and distributed sensing problems in leader/follower multi-agent
systems. The authors in [36, 110, 111] study the containment control prob-
lem for a collection of Euler-Lagrange systems. In particular, [36] discussed
the case with multiple stationary leaders, [111] studied the case of dynam-
ic leaders with finite-time convergence, and [110] considered the case with
parametric uncertainties. In the above-mentioned works, the agent dynamics
are assumed to be single, double integrators, or second-order Euler-Lagrange
systems, which might be restrictive in some circumstances.
8
Distributed Robust Cooperative Control for
Multi-Agent Systems with Heterogeneous
Matching Uncertainties

CONTENTS
8.1 Distributed Robust Leaderless Consensus . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
8.1.1 Distributed Static Consensus Protocols . . . . . . . . . . . . . . . . . . . . . 185
8.1.2 Distributed Adaptive Consensus Protocols . . . . . . . . . . . . . . . . . 190
8.2 Distributed Robust Consensus with a Leader of Nonzero Control
Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
8.3 Robustness with Respect to Bounded Non-Matching Disturbances . 201
8.4 Distributed Robust Containment Control with Multiple Leaders . . . 205
8.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

The consensus problem of multi-agent systems with general discrete-time and

continuous-time linear dynamics has been studied in the previous chapters,
where various consensus protocols have been designed to achieve consensus.
A common assumption in the previous chapters is that the dynamics of the
agents are identical and precisely known, which might be restrictive and not
practical in many circumstances. In practical applications, the agents may be
subject to certain parameter uncertainties or unknown external disturbances.
In this chapter, we consider the distributed consensus problem of multi-
agent systems with identical nominal linear dynamics but subject to differ-
ent matching uncertainties. A typical example belonging to this scenario is
a network of mass-spring systems with different masses or unknown spring
constants. Due to the existence of the nonidentical uncertainties which may
be time-varying, nonlinear and unknown, the multi-agent systems discussed
in this chapter are essentially heterogeneous. The heterogeneous multi-agent
systems in this chapter contain the homogeneous linear multi-agent systems
studied in the previous chapters as a special case where the uncertainties do
not exist. Note that because of the existence of the uncertainties, the consen-
sus problem in this case becomes quite challenging to solve and the consensus
algorithms given in in the previous chapters are not applicable any more.
In this chapter, we present a systematic procedure to address the distribut-
ed robust consensus problem of multi-agent systems with matching uncertain-
ties for both the cases without and with a leader of possibly nonzero control

183
184 Cooperative Control of Multi-Agent Systems

input. The case where the communication graph is undirected and connected
is considered in Section 8.1. By assuming that there exists time-varying upper
bounds of the matching uncertainties, distributed continuous static consen-
sus protocols based on the relative states of neighboring agents are designed,
which includes a nonlinear term to deal with the effect of the uncertainties.
It is shown that the consensus error under this static protocol is uniformly
ultimately bounded and exponentially converges to a small residual set. Note
that the design of this static protocol relies on the eigenvalues of the Lapla-
cian matrix and the upper bounds of the matching uncertainties. In order
to remove these requirements, a fully distributed adaptive protocol is further
designed, under which the residual set of the consensus error is also given.
One desirable feature is that for both the static and adaptive protocols, the
residual sets of the consensus error can be made to be reasonably small by
properly selecting the design parameters of the protocols and the convergence
rates of the consensus error are explicitly given.
The robust consensus problem for the case where there exists a leader with
nonzero control input is investigated in Section 8.2. Here we study the general
case where the leader’s control input is not available to any follower, which
imposes additional difficulty. Distributed adaptive consensus protocols based
on the relative state information are proposed and designed to ensure that the
consensus error can converge to a small adjustable residual set.
The robustness issue of the proposed consensus protocols with respect to
external disturbances which do not satisfy the matching condition is discussed
in Section 8.3, where the proposed consensus protocols are redesigned to guar-
antee the boundedness of the consensus error and the adaptive gains in the
presence of bounded external disturbances. Extensions of the results to the
containment control problem with multiple leaders are discussed in Section
8.4.

8.1 Distributed Robust Leaderless Consensus

In this section, we consider a network of N autonomous agents with identi-
cal nominal linear dynamics but subject to heterogeneous uncertainties. The
dynamics of the i-th agent are described by

ẋi = Axi + Bui + Hi (xi , t) + νi (t), i = 1, · · · , N, (8.1)

where xi ∈ Rn is the state, ui ∈ Rp is the control input, A and B are

constant known matrices with compatible dimensions, and Hi (xi , t) ∈ Rn
and νi (t) ∈ Rn denote, respectively, the parameter uncertainties and external
disturbances associated with the i-th agent, which are assumed to satisfy the
following standard matching condition [43, 183].
Cooperative Control for Multi-Agent Systems with Matching Uncertainties 185

Assumption 8.1 There exist functions Ĥi (xi , t) and ν̂i (t) such that
Hi (xi , t) = B Ĥi (xi , t) and νi (t) = B ν̂i (t), i = 1, · · · , N .

By letting fi (xi , t) = Ĥi (xi , t) + ν̂i represent the lumped uncertainty of

the i-th agent, (8.1) can be rewritten into

ẋi = Axi + B[ui + fi (xi , t)], i = 1, · · · , N. (8.2)

In the previous chapters, the agents are identical linear systems and free
of uncertainties. In contrast, the agents (8.2) considered in this chapter are
subject to nonidentical uncertainties, which makes the resulting multi-agent
systems are essentially heterogeneous. The agents (8.2) can recover the nom-
inal linear agents in the previous chapters when the uncertainties fi (xi , t) do
not exist. Note that the existence of the uncertainties associated with the
agents makes the consensus problem quite challenging to solve, as detailed in
the sequel.
Regarding the bounds of the uncertainties fi (xi , t), we introduce the fol-
lowing assumption.

Assumption 8.2 There exist continuous scalar valued functions ρi (xi , t), i =
1, · · · , N , such that kfi (xi , t)k ≤ ρi (xi , t), i = 1, · · · , N , for all t ≥ 0 and
xi ∈ Rn .

The communication graph among the N agents is represented by a undi-

rected graph G, which is assumed to be connected throughout this sec-
tion. The objective of this section is to solve the consensus problem for
the agents in (8.2), i.e., to design distributed consensus protocols such that
limt→∞ kxi (t) − xj (t)k = 0, ∀ i, j = 1, · · · , N .

8.1.1 Distributed Static Consensus Protocols

Based on the relative states of neighboring agents, the following distributed
static consensus protocol is proposed:
N
X N
X
ui = cK aij (xi − xj ) + ρi (xi , t)gi (K aij (xi − xj )), i = 1, · · · , N,
j=0 j=0
(8.3)
where c > 0 is the constant coupling gain, K ∈ Rp×n is the feedback gain
matrix, aij is the (i, j)-th entry of the adjacency matrix associated with G,
and the nonlinear functions gi (·) are defined such that for w ∈ Rn ,
(
w
if ρi (xi , t)kwk > κi
gi (w) = kwkw
(8.4)
κi ρi (xi , t) if ρi (xi , t)kwk ≤ κi

where κ are small positive constants.

186 Cooperative Control of Multi-Agent Systems

Let x = [xT1 , · · · , xTN ]T and ρ(x, t) = diag(ρ1 (x1 , t), · · · , ρN (xN , t)). Using
(8.3) for (8.2), we can obtain the closed-loop network dynamics as

ẋ = (IN ⊗ A + cL ⊗ BK)x + (IN ⊗ B)F (x, t) + [ρ(x, t) ⊗ B]G(x), (8.5)

where L denotes the Laplacian matrix of G, and

   PN 
f1 (x1 , t) g1 (K j=1 L1j xj )
 ..   .. 
F (x, t) ,  .  , G(x) , 
 .
.
 (8.6)
PN
fN (xN , t) gN (K j=1 LN j xj )

Let ξ = (M ⊗ In )x, where M = IN − N1 11T and ξ = [ξ1T , · · · , ξN T T

] . It
is easy to see that 0 is a simple eigenvalue of M with 1 as a corresponding
right eigenvector and 1 is the other eigenvalue with multiplicity N − 1. Then,
it follows that ξ = 0 if and only if x1 = · · · = xN . Therefore, the consensus
problem under the protocol (8.3) is solved if and only if ξ asymptotically
converges to zero. Hereafter, we refer to ξ as the consensus error. By noting
that LM = L, it is not difficult to obtain from (8.5) that the consensus error
ξ satisfies
ξ˙ = (IN ⊗ A + cL ⊗ BK)ξ + (M ⊗ B)F (x, t)
(8.7)
+ [M ρ(x, t) ⊗ B]G(ξ).
The following result provides a sufficient condition to design the consensus
protocol (8.3).

Theorem 50 Suppose that the communication graph G is undirected and con-

nected and Assumption 8.2 holds. The parameters in the distributed protocol
(8.3) are designed as c ≥ λ12 and K = −B T P −1 , where λ2 is the smallest
nonzero eigenvalue of L and P > 0 is a solution to the following linear matrix
inequality (LMI):
AP + P AT − 2BB T < 0, (8.8)
Then, the consensus error ξ of (8.7) is uniformly ultimately bounded and ex-
ponentially converges to the residual set
N
2λmax (P ) X
D1 , {ξ : kξk2 ≤ κi }, (8.9)
αλ2 i=1

where
−λmax (AP + P AT − 2BB T )
α= . (8.10)
λmax (P )

Proof 52 Consider the following Lyapunov function candidate:

1 T
V1 = ξ (L ⊗ P −1 )ξ.
2
Cooperative Control for Multi-Agent Systems with Matching Uncertainties 187

By the definition of ξ, it is easy to see that (1T ⊗ I)ξ = 0. For a connected

graph G, it then follows from Lemma 2 that
1 λ2
V1 (ξ) ≥ λ2 ξ T (IN ⊗ P −1 )ξ ≥ kξk2 . (8.11)
2 2λmax (P )

The time derivative of V1 along the trajectory of (8.5) is given by

V̇1 = ξ T (L ⊗ P −1 A + cL2 ⊗ P −1 BK)ξ

(8.12)
+ ξ T (L ⊗ P −1 B)F (x, t) + ξ T [Lρ(x, t) ⊗ P −1 B]G(ξ).

By using Assumption 8.2, we can obtain that

N
X N
X
ξ T (L ⊗ P −1 B)F (x, t) ≤ kB T P −1 Lij ξj kkfi (xi , t)k
i=1 j=1
(8.13)
N
X N
X
≤ ρi (xi , t)kB T P −1 Lij ξj k.
i=1 j=1

Next, considerPthe following three cases.

N
i) ρi (xi , t)kK j=1 Lij ξj k > κi , i = 1, · · · , N .
In this case, it follows from (8.4) and (8.6) that
N
X N
X
ξ T [Lρ(x, t) ⊗ P −1 B]G(ξ) = − ρi (xi , t)kB T P −1 Lij ξj k. (8.14)
i=1 j=1

Substituting (8.14) and and (8.13) into (8.12) yields

1 T
V̇1 ≤ ξ X ξ,
2
where X = L ⊗ (P −1 A + AT P −1 ) − 2cL2 ⊗ P −1 BB T P −1 .
PN
ii) ρi (xi , t)kK j=1 Lij ξj k ≤ κi , i = 1, · · · , N .
In this case, we can get from (8.4) and (8.6) that
N
X N
X
ρi (xi , t)2
ξ T [Lρ(x, t) ⊗ P −1 B]G(ξ) = − kB T P −1 Lij ξj k2 ≤ 0.
i=1
κi j=1
(8.15)
Substituting (8.13) and (8.15) into (8.12) gives

XN XN
1 T
V̇1 ≤ ξ Xξ + ρi (xi , t)kB T P −1 Lij ξj k
2 i=1 j=1
(8.16)
XN
1
≤ ξT X ξ + κi .
2 i=1
188 Cooperative Control of Multi-Agent Systems

iii) ξ satisfies neither Case i) nor Case ii).

PN
Without loss of generality, assume that ρi (xi , t)kK j=1 Lij ξj k > κi , i =
PN
1, · · · , l, and ρi (xi , t)kK j=1 Lij ξj k ≤ κi , i = l + 1, · · · , N , where 2 ≤ l ≤
N − 1. By combining (8.14) and (8.15), in this case we can get that
l
X N
X
ξ T [Lρ(x, t) ⊗ P −1 B]G(ξ) = − ρi (xi , t)kB T P −1 Lij ξj k
i=1 j=1
N
X N
ρi (xi , t)2 T −1 X
− kB P Lij ξj k2 (8.17)
κi j=1
i=l+1
l
X N
X
≤− ρi (xi , t)kB T P −1 Lij ξj k.
i=1 j=1

Then, it follows from (8.12), (8.17), and (8.13) that

X N −l
1 T
V̇1 ≤ ξ Xξ + κi .
2 i=1

Therefore, by analyzing the above three cases, we get that V̇1 satisfies (8.16)
for all ξ ∈ RN n . Note that (8.16) can be rewritten as

XN
1
V̇1 ≤ −αV1 + αV1 + ξ T X ξ + κi
2 i=1
(8.18)
XN
1
= −αV1 + ξ T (X + αL ⊗ P −1 )ξ + κi ,
2 i=1

where α > 0.
Because G is connected, it follows from Lemma 1 that zero is a simple
i L and all the other eigenvalues are positive. Let U = [ N ] and
eigenvalue of √1 Y1
h 1T
UT = √
N , with Y1 ∈ RN ×(N −1) , Y2 ∈ R(N −1)×N , be such unitary matrices
Y2
that U T LU = Λ , diag(0, λ2 , · · · , λN ), where λ2 ≤ · · · ≤ λN are the nonzero
eigenvalues of L. Let ξ¯ , [ξ¯1T , · · · , ξ¯N ] = (U T ⊗ P −1 )ξ. By the definitions
T T

¯ it is easy to see that ξ¯1 = ( √1T ⊗ P −1 )ξ = ( √1T M ⊗ P −1 )x = 0.

of ξ and ξ, N N
Then, it follows that
N
X
ξ T (X + αL ⊗ P −1 )ξ = λi ξ¯iT (AP + P AT + αP − 2cλi BB T )ξ¯i
i=2
(8.19)
N
X
≤ λi ξ¯iT (AP + P AT + αP − 2BB T )ξ¯i .
i=2
Cooperative Control for Multi-Agent Systems with Matching Uncertainties 189
T T
Because α = −λmax (AP +P A −2BB )
λmax (P ) , we can see from (8.19) that ξ T (X + αL ⊗
P −1 )ξ ≤ 0. Then, we can get from (8.18) that
N
X
V̇1 ≤ −αV1 + κi . (8.20)
i=1

By using Lemma 16 (the Comparison lemma), we can obtain from (8.20) that
N
X N
X
V1 (ξ) ≤ [V1 (ξ(0)) − κi /α]e−αt + κi /α, (8.21)
i=1 i=1

which, by (8.11), implies that ξ exponentially converges to the residual set D1

in (8.9) with a convergence rate not less than e−αt .

Remark 62 The distributed consensus protocol (8.3) consists of a linear

PN
part and a nonlinear part, where the term ρi (xi , t)gi (K j=1 aij (xi − xj ))
is used to suppress the effect of the uncertainties fi (xi , t). For the case where
PN
fi (xi , t) = 0, we can accordingly remove ρi (xi , t)gi (K j=1 aij (xi − xj )) from
(8.3), which can recover the static consensus protocol (2.2) in Chapter 2. As
shown in Proposition 1 of Chapter 2, a necessary and sufficient condition for
the existence of a P > 0 to the LMI (8.8) is that (A, B) is stabilizable. There-
fore, a sufficient condition for the existence of (8.3) satisfying Theorem 50 is
that (A, B) is stabilizable. Note that in Theorem 50 the parameters c and K
of (8.3) are independently designed.

Note that the nonlinear components gi (·) in (8.4) are actually continuous
approximations, (via the boundary layer concept [43, 71], of the discontinuous
w
if kwk =6 0
function ĝ(w) = kwk . The values of κi in (8.4) define the widths
0 if kwk = 0
of the boundary layers. As κi → 0, the continuous functions gi (·) approach
the discontinuous function ĝ(·).

Corollary 10 Assume that G is connected and Assumption 8.2 holds. The

consensus error ξ converges to zero under the discontinuous consensus proto-
col:
N
X N
X
ui = cK aij (xi − xj ) + ρi (xi , t)ĝ(K aij (xi − xj )), i = 1, · · · , N,
j=1 j=1
(8.22)
where c and K are chosen as in Theorem 50.

Remark 63 An inherent drawback of the discontinuous protocol (8.22) is that

it will result in the undesirable chattering effect in real implementation, due to
imperfections in switching devices [43, 187]. The effect of chattering is avoided
190 Cooperative Control of Multi-Agent Systems

by using the continuous protocol (8.3). The cast is that the protocol (8.3) does
not guarantee asymptotic stability but rather uniform ultimate boundedness
of the consensus error ξ. Note that the residual set D1 of ξ depends on the
smallest nonzero eigenvalue of L, the number of agents, the largest eigenvalue
of P , and the widths κi of the boundary layers. By choosing sufficiently small
κi , the consensus error ξ under the protocol (8.3) can converge to an arbitrarily
small neighborhood of zero, which is acceptable in most applications.

8.1.2 Distributed Adaptive Consensus Protocols

In the last subsection, the design of the distributed protocol (8.3) relies on
the minimal nonzero eigenvalue λ2 of L and the upper bounds ρi (xi , t) of
the matching uncertainties fi (xi , t). However, λ2 is global information in the
sense that each agent has to know the entire communication graph to compute
it. Besides, the bounds ρi (xi , t) of the uncertainties fi (xi , t) might not be
easily obtained in some cases, e.g., fi (xi , t) contains certain unknown external
disturbances. In this subsection, we will implement some adaptive control
ideas to compensate the lack of λ2 and ρi (xi , t) and thereby to solve the
consensus problem using only the local information available to the agents.
Before moving forward, we introduce a modified assumption regarding the
bounds of the lumped uncertainties fi (xi , t), i = 1, · · · , N .

Assumption 8.3 There are positive constants di and ei such that

kfi (xi , t)k ≤ di + ei kxi k, i = 1, · · · , N .

Based on the local state information of neighboring agents, we propose the

following distributed adaptive protocol to each agent:
N
X N
X
ui = d¯i K aij (xi − xj ) + ri (K aij (xi − xj )),
j=1 j=1
N
X N
X N
X
d¯˙i = τi [−ϕi d¯i + ( aij (xi − xj )T )Γ( aij (xi − xj )) + kK aij (xi − xj )k],
j=1 j=1 j=1
N
X
ē˙ i = ǫi [−ψi ēi + kK aij (xi − xj )kkxi k], i = 1, · · · , N,
j=1
(8.23)
where d¯i (t) and ēi (t) are the adaptive coupling gains associated with the i-th
agent, Γ ∈ Rn×n is the feedback gain matrix, τi and ǫi are positive scalars,
ϕi and ψi are small positive constants chosen by the designer, the nonlinear
functions ri (·) are defined such that for w ∈ Rn ,
( w(d¯ +ē kx k)
i i
kwk
i
if (d¯i + ēi kxi k)kwk > κi
ri (w) = w(d¯i +ē 2 (8.24)
i kx
κi
i k)
if (d¯i + ēi kxi k)kwk ≤ κi
Cooperative Control for Multi-Agent Systems with Matching Uncertainties 191

and the rest of the variables are defined as in (8.3).

Let the consensus error ξ be defined as in (8.7) and D = diag(d¯1 , · · · , d¯N ).
Then, it is not difficult to get from (8.2) and (8.23) that the closed-loop
network dynamics can be written as

ξ˙ = (IN ⊗ A + M DL ⊗ BK)ξ + (M ⊗ B)[F (x, t) + R(ξ)],

N
X N
X N
X
d¯˙i = τi [−ϕi d¯i + ( Lij ξjT )Γ( Lij ξj ) + kK Lij ξj k],
j=1 j=1 j=1 (8.25)
N
X
ē˙ i = ǫi [−ψi ēi + kK Lij ξj kkxi k], i = 1, · · · , N,
j=1

where  PN 
r1 (K j=1 L1j ξj )
 .. 
R(ξ) , 
 .
,
 (8.26)
PN
rN (K j=1 L N j ξj )
and the rest of the variables are defined as in (8.5).
To establish the ultimate boundedness of the states ξ, d¯i , and ēi of (8.25),
we use the following Lyapunov function candidate

XN N
X
1 T d˜2i ẽ2i
V2 = ξ (L ⊗ P −1 )ξ + + , (8.27)
2 i=1
2τi i=1 2ǫi

where ẽi = ēi − ei , d˜i = d¯i − β, i = 1, · · · , N , and β = maxi=1,··· ,N {di , λ12 }.

The following theorem shows the ultimate boundedness of the states ξ, d¯i ,
and ēi of (8.25).

Theorem 51 Suppose that G is connected and Assumption 8.3 holds. The

feedback gain matrices of the distributed adaptive protocol (8.23) are designed
as K = −B T P −1 and Γ = P −1 BB T P −1 , where P > 0 is a solution to the
LMI (8.8). Then, both the consensus error ξ and the adaptive gains d¯i and
ēi , i = 1, · · · , N , in (8.25) are uniformly ultimately bounded and the following
statements hold.

i) For any ϕi and ψi , the parameters ξ, d˜i , and ẽi exponentially converge to
the residual set
N
1 X 2 κi
D2 , {ξ, d˜i , ẽi : V2 < (β ϕi + e2i ψi + )}, (8.28)
2δ i=1 2

where δ , mini=1,··· ,N {α, ϕi τi , ψi ǫi } and α is defined as in (8.10).

 192 Cooperative Control of Multi-Agent Systems

ii) If small ϕi and ψi satisfy ̺ , mini=1,··· ,N {ϕi τi , ψi ǫi } < α, then in addition

to i), ξ exponentially converges to the residual set
N
λmax (P ) X 2 1
D3 , {ξ : kξk2 ≤ (β ϕi + e2i ψi + κi )}. (8.29)
λ2 (α − ̺) i=1 2

Proof 53 The time derivative of V2 along (8.25) can be obtained as

V̇2 = ξ T [(L ⊗ P −1 A + LDL ⊗ P −1 BK)ξ
+ (L ⊗ P −1 B)F (x, t) + (L ⊗ P −1 B)R(ξ)]
N
X N
X N
X
+ d˜i [−ϕi (d˜i + β) + ( Lij ξjT )Γ( Lij ξj ) (8.30)
i=1 j=1 j=1
N
X N
X N
X
+ kK Lij ξj k] + ẽi [−ψi (ẽi + ei ) + kK Lij ξj kkxi k],
j=1 i=1 j=1

where D = diag(d˜1 + β, · · · , d˜N + β).

By noting that K = −BP −1 , it is easy to get that
ξ T (LDL ⊗ P −1 BK)ξ
N
X N
X N
X (8.31)
=− (d˜i + β)( Lij ξj )T P −1 BB T P −1 ( Lij ξj ).
i=1 j=1 j=1

In light of Assumption 8.3, we can obtain that

N
X N
X
ξ T (L ⊗ P −1 B)F (x, t) ≤ (di + ei kxi k)kB T P −1 Lij ξj k. (8.32)
j=1 j=1

In what follows, wePconsider three cases.

i) (d¯i + ēi kxi k)kK j=1 Lij ξj k > κi , i = 1, · · · , N .
N

In this case, we can get from (8.24) and (8.26) that

N
X N
X
ξ T (L ⊗ P −1 B)R(ξ) = − [d¯i + ēi kxi k]kB T P −1 Lij ξj k. (8.33)
i=1 j=1

Substituting (8.31), (8.32), and (8.33) into (8.30) yields

XN XN
1 T
V̇2 ≤ ξ Yξ − (β − di )kB T P −1 Lij ξj k
2 i=1 j=1
N N
1X 1X 2
− (ϕi d˜2i + ψi ẽ2i ) + (β ϕi + e2i ψi )
2 i=1 2 i=1
N N
1 T 1X 1X 2
≤ ξ Yξ − (ϕi d˜2i + ψi ẽ2i ) + (β ϕi + e2i ψi ),
2 2 i=1 2 i=1
Cooperative Control for Multi-Agent Systems with Matching Uncertainties 193

where Y , L ⊗ (P −1 A + AT P −1 ) − 2βL2 ⊗ P −1 BB T P −1 and we have used

the facts that β ≥ maxi=1,··· ,N di and −d˜2i − d˜i β ≤ − 21 d˜2i + 12 β 2 .
PN
ii) (d¯i + ēi kxi k)kK j=1 Lij ξj k ≤ κi , i = 1, · · · , N .
In this case, we can get from (8.24) and (8.26) that
N
X N
X
(d¯i + ēi kxi k)2
ξ T (L ⊗ P −1 B)R(ξ) = − kB T P −1 Lij ξj k2 . (8.34)
i=1
κi j=1

Then, it follows from (8.31), (8.32), (8.34), and (8.30) that

N
1 T 1X
V̇2 ≤ ξ Yξ − (ϕi d˜2i + ψi ẽ2i )
2 2 i=1
(8.35)
N
1X 2 1
+ (β ϕi + e2i ψi + κi ),
2 i=1 2
¯ 2 PN
where we have used the fact that − (di +ēκi kx i k)
kB T P −1 j=1 Lij ξj k2 + (d¯i +
PN i
PN
ēi kxi k)kB T P −1 j=1 Lij ξj k ≤ 41 κi , for (d¯i + ēi kxi k)kK j=1 Lij ξj k ≤ κi ,
i = 1, · · · , N . PN
iii) (d¯i + ēi kxi k)kK j=1 Lij ξj k > κi , i = 1, · · · , l, and (d¯i +
PN
ēi kxi k)kK j=1 Lij ξj k ≤ κi , i = l + 1, · · · , N , where 2 ≤ l ≤ N − 1.
By following similar steps in the two cases above, it is not difficult to get
that
N
1 1X
V̇2 ≤ ξ T Yξ − (ϕi d˜2i + ψi ẽ2i )
2 2 i=1
N N −l
1X 2 1X
+ (β ϕi + e2i ψi ) + κi .
2 i=1 4 i=1

Therefore, based on the above three cases, we can get that V̇2 satisfies (8.35)
for all ξ ∈ RN n . Note that (8.35) can be rewritten into
N
1 1X
V̇2 ≤ −δV2 + δV2 + ξ T Yξ − (ϕi d˜2i + ψi ẽ2i )
2 2 i=1
N
1X 2 1
+ (β ϕi + e2i ψi + κi )
2 i=1 2
(8.36)
N
1 1X δ
= −δV2 + ξ T (Y + δL ⊗ P −1 )ξ − [(ϕi − )d˜2i
2 2 i=1 τi
N
δ 2 1X 2 1
+ (ψi − )ẽi )] + (β ϕi + e2i ψi + κi ).
ǫi 2 i=1 2

Because βλ2 ≥ 1 and 0 < δ ≤ α, by following similar steps in the proof of

194 Cooperative Control of Multi-Agent Systems

Theorem 50, we can show that ξ T (Y + δL ⊗ P −1 )ξ ≤ 0. Further, by noting

that δ ≤ mini=1,··· ,N {ϕi τi , ψi ǫi }, it follows from (8.36) that
N
1X 2 1
V̇2 ≤ −δV2 + (β ϕi + e2i ψi + κi ), (8.37)
2 i=1 2

which implies that

N
1 X 2 1
V2 ≤ [V2 (0) − (β ϕi + e2i ψi + κi )]e−δt
2δ i=1 2
(8.38)
N
1 X 2 1
+ (β ϕi + e2i ψi + κi ).
2δ i=1 2

Therefore, V2 exponentially converges to the residual set D2 in (8.28) with

λ2
a convergence rate faster than e−δt , which, in light of V2 ≥ 2λmax 2
(P ) kξk +
PN d˜2i PN ẽ2i ¯
i=1 2τi + i=1 2ǫi , implies that ξ, di , and ēi are uniformly ultimately bound-
ed.
Next, if ̺ , mini=1,··· ,N {ϕi τi , ψi ǫi } < α, then we can choose δ = ̺ and
it is easy to see that D2 increase as min{ϕi τi , ψi ǫi } decreases. However, for
the case where ̺ < α, we can obtain a smaller residual set for ξ by rewriting
(8.36) into
1
V̇2 ≤ −̺V2 + ξ T (Y + αL ⊗ P −1 )ξ
2
N
α−̺ T 1X 2 1
− ξ (L ⊗ P −1 )ξ + (β ϕi + e2i ψi + κi )
2 2 i=1 2 (8.39)
N
λ2 (α − ̺) 1X 2 1
≤ −̺V2 − kξk2 + (β ϕi + e2i ψi + κi ).
2λmax (P ) 2 i=1 2

λmax (P ) PN
Obviously, it follows from (8.39) that V̇2 ≤ −̺V2 if kξk2 > λ2 (α−̺) i=1 (β
2
ϕi +
e2i ψi + 21 κi ). λ2 2
Then, by noting V2 ≥ 2λmax (P ) kξk , we can get that if ̺ ≤ α then
ξ exponentially converges to the residual set D3 in (8.29) with a convergence
rate faster than e−̺t .

Remark 64 It is worth mentioning that both the σ-modification technique in

[63, 183] and the boundary layer concept play a vital role to guarantee the
ultimate boundedness of the consensus error ξ and the adaptive gains d¯i and
ēi . From (8.28) and (8.29), we can observe that the residual sets D2 and D3
decrease as κi decrease. Given κi , smaller ϕi and ψi give a smaller bound for
ξ and at the same time yield a larger bound for d¯i and ēi . For the case where
ϕi = 0 or ψi = 0, d¯i and ēi will tend to infinity. In real implementations, if
large d¯i and ēi are acceptable, we can choose ϕi , ψi , and κi to be relatively
small in order to guarantee a small consensus error ξ.
Cooperative Control for Multi-Agent Systems with Matching Uncertainties 195

Remark 65 Contrary to the static protocol (8.3), the design of the adaptive
protocol (8.23) relies on only the agent dynamics and the local state informa-
tion of each agent and its neighbors, requiring neither the minimal nonzero
eigenvalue of L nor the upper bounds of of the uncertainties fi (xi , t). Thus,
the adaptive protocol (8.23) can be implemented by each agent in a fully dis-
tributed fashion without requiring any global information.

Remark 66 A special case of the uncertainties fi (xi , t) satisfying Assump-

tions 8.2 and 8.3 is that there exist positive constants di such that kfi (xi , t)k ≤
di , i = 1, · · · , N . For this case, the proposed protocols (8.3) and (8.23) can
be accordingly simplified. For (8.3), we can simply replace ρi (xi , t) by di . The
adaptive protocol (8.23) in this case can be modified into
N
X N
X
ui = d¯i K aij (xi − xj ) + r̄i (K aij (xi − xj )),
j=1 j=1
N
X N
X
d¯˙i = τi [−ϕi d¯i + ( aij (xi − xj )T )Γ( aij (xi − xj )) (8.40)
j=1 j=1
N
X
+ kK aij (xi − xj )k], i = 1, · · · , N,
j=1

where the nonlinear functions r̄i (·) are defined such that r̄i (w) =
(
wd¯i
kwk if d¯i kwk > κ
wd¯2i
and the rest of the variables are defined as in (8.23).
κ if d¯i kwk ≤ κ

1 6

2 5

3 4

FIGURE 8.1: The leaderless communication graph.

Example 23 Consider a network of mass-spring systems with a common

mass m but different unknown spring constants, described by

mÿi + ki yi = ui , i = 1, · · · , N, (8.41)
196 Cooperative Control of Multi-Agent Systems

where yi are the displacements from certain reference positions and ki , i =

T
1, · · · , N, are the bounded unknown spring constants. Denote by xi = [ yi ẏi ]
the state of the i-th agent. Then, (8.41) can be rewritten as

ẋi = Axi + B(ui + ki Exi ), i = 1, · · · , N, (8.42)

with    
0 1 0  
A= , B = 1 , E = −1 0 .
0 0 m

It is easy to see that ki Exi , i = 1, · · · , N , satisfy Assumption 8.3, i.e.,

kki Exi k ≤ ki kxi k, i = 1, · · · , N .
Because the spring constants ki are unknown, we will use the adaptive
protocol (8.23) to solve the consensus problem. Let m = 2.5kg and ki be ran-
domly chosen. Solving the LMI (8.8) by using the SeDuMi toolbox [158] gives
the feedback gain matrices of (8.23) as
 
  0.4480 1.6462
K = − 0.6693 2.4595 , Γ = .
1.6462 6.0489

Assume that the communication topology is given in FIGURE 8.1. In (8.23),

select κ = 0.5, ϕi = ψi = 0.05, and τi = ǫi = 10, i = 1, · · · , 6, in (8.23). The
state trajectories xi (t) of (8.42) under (8.23) designed as above are depicted in
FIGURE 8.2, which implies that consensus is indeed achieved. The adaptive
gains d¯i and ēi in (8.23) are shown in FIGURE 8.3, from which it can be
observed that d¯i and ēi tend to be quite small.

3 1.5

1
2
0.5

0
1
−0.5
xi1

xi2

0 −1

−1.5
−1
−2

−2.5
−2
−3

−3 −3.5
0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30 35 40
t t

FIGURE 8.2: The state trajectories of the mass-spring systems.

Cooperative Control for Multi-Agent Systems with Matching Uncertainties 197

25 8
0.03 0.04
7
0.03
20 0.02
6
0.02
15 0.01 5
0.01
4
0 0

i
i

10

e
d

35 36 37 38 39 40 35 36 37 38 39 40
3

5 2

1
0
0
−5 −1
0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30 35 40
t t

FIGURE 8.3: The adaptive gains d¯i and ēi in (8.23).

8.2 Distributed Robust Consensus with a Leader of

Nonzero Control Input
For the leaderless consensus problem in the previous section where the com-
munication graph is undirected, the final consensus values reached by the
agents under the protocols (8.3) and (8.23) are generally difficult to be ex-
plicitly obtained. The main difficulty lies in that the agents are subject to
uncertainties and the protocols (8.3) and (8.23) are essentially nonlinear. In
this section, we consider the leader-follower consensus (distributed tracking)
problem, for which case the agents’ states are required to converge onto a
reference trajectory.
Consider a network of N + 1 agents consisting of N followers and one
leader. Without loss of generality, let the agent indexed by 0 be the leader
and the agents indexed by 1, · · · , N , be the followers. The dynamics of the
followers are described by (8.2). For simplicity, we assume that the leader has
the nominal linear dynamics, given by

ẋ0 = Ax0 + Bu0 , (8.43)

where x0 ∈ Rn is the state and u0 ∈ Rp is the control input of the leader,

respectively. In some applications, the leader might need its own control action
to achieve certain objectives, e.g., to reach a desirable consensus value. In this
section, we consider the general case where u0 is possibly nonzero and time
varying and not accessible to any follower, which is much harder to solve than
the case with u0 = 0.
Before moving forward, the following mild assumption is needed.

Assumption 8.4 The leader’s control input u0 is bounded, i.e., there exists
a positive scalar γ such that ku0 k ≤ γ.

It is assumed that the leader receives no information from any follower

198 Cooperative Control of Multi-Agent Systems

and the state of the leader is available to only a subset of the followers. The
communication graph among the N + 1 agents is represented by a directed
b which satisfies the following assumption.
graph G,

Assumption 8.5 Gb contains a directed spanning tree with the leader as the
root and the subgraph associated with the N followers is undirected.

Denote by Lb the Laplacian matrix associated b

h with G.i Because the leader
0 0
has no neighbors, Lb can be partitioned as Lb = L2 L1 , where L2 ∈ RN ×1
1×N

and L1 ∈ RN ×N . By Lemma 1 and Assumption 8.5, it is clear that L1 > 0.

The objective of this section is to solve the leader-follower consensus prob-
lem for the agents in (8.2) and (8.43), i.e., to design distributed protocols
under which the states of the N followers converge to the state of the leader.
Based on the relative states of neighboring agents, we propose the following
distributed adaptive protocol to each follower as
N
X N
X
ui = dˆi K aij (xi − xj ) + ri (K aij (xi − xj )),
j=0 j=0

X N X N X N
˙
dˆi = τi [−ϕi dˆi + ( aij (xi − xj )T )Γ( aij (xi − xj )) + kK aij (xi − xj )k],
j=0 j=0 j=0
N
X
ê˙ i = ǫi [−ψi êi + kK aij (xi − xj )kkxi k], i = 1, · · · , N,
j=0
(8.44)
where dˆi (t) and êi (t) are the adaptive gains associated with the i-th follower,
aij is the (i, j)-th entry of the adjacency matrix associated with G, b and the
rest of the variables are defined as in (8.23).
T T
Let x = [xT1 , · · · , xTN ]T , ζi = xi − x0 , i = 1, · · · , N , and ζ = [ζ1T , · · · , ζN ] .
Then, it follows from (8.2), (8.43), and (8.44) that the closed-loop network
dynamics can be obtained as

b 1 ⊗ BK)ζ + (IN ⊗ B)F (x, t)

ζ̇ = (IN ⊗ A + DL
+ (IN ⊗ B)R(ζ) − (1 ⊗ B)u0 ,
X N X N X N
˙
dˆi = τi [−ϕi dˆi + ( Lij ζjT )Γ( Lij ζj ) + kK Lij ζj k], (8.45)
j=1 j=1 j=1
N
X
ê˙ i = ǫi [−ψi êi + kK Lij ζj kkxi k], i = 1, · · · , N,
j=1

where Db = diag(dˆ1 , · · · , dˆN ), R(·) remains the same as in (8.26), and the rest
of the variables are defined as in (8.25).
To present the following theorem, we use a Lyapunov function in the form
 Cooperative Control for Multi-Agent Systems with Matching Uncertainties 199

of
XN N
X
1 T d˘2i ĕ2i
V3 = ζ (L1 ⊗ P −1 )ζ + + ,
2 i=1
2τi i=1 2ǫi

where ĕi = êi − ei , d˘i = dˆi − β̂, i = 1, · · · , N , and β̂ = maxi=1,··· ,N {di +

γ, λmin1(L1 ) }.

Theorem 52 Supposing that Assumptions 8.3, 8.4, and 8.5 hold, the leader-
follower consensus error ζ and the adaptive gains dˆi and êi , i = 1, · · · , N , in
(8.45) are uniformly ultimately bounded under the distributed adaptive protocol
(8.44) with K and Γ designed as in Theorem 51. Moreover, the following two
assertions hold.
i) For any ϕi and ψi , the parameters ξ, d˘i , and ĕi exponentially converge to
the residual set
N
1 X 2 1
D4 , {ζ, d˘i , ĕi : V3 < (β̂ ϕi + e2i ψi + κi )}, (8.46)
2δ i=1 2

where δ is defined as in Theorem 51.

ii) If ̺ < α, where ̺ and α are defined as in Theorem 51 and (8.10), respec-
tively, then in addition to i), ζ exponentially converges to the residual set

X N
λmax (P ) 1
D5 , {ζ : kζk2 ≤ (β̂ 2 ϕi + e2i ψi + κi )}. (8.47)
λmin (L1 )(α − ̺) i=1 2

Proof 54 It can be completed by following similar steps as in the proof of

Theorem 51 and by using the following assertion:
N
X N
X
−ζ T (L1 1 ⊗ P B)u0 ≤ kB T P Lij ζj kku0 k
i=1 j=1
N
X N
X
T
≤γ kB P Lij ζj k.
i=1 j=1

The details are omitted here for brevity.

The following example is presented to illustrate Theorem 52.

Example 24 Consider a group of Chua’s circuits, whose dynamics in the

dimensionless form are given by [107]

ẋi1 = a[−xi1 + xi2 − h(xi1 )] + ui ,

ẋi2 = xi1 − xi2 + xi3 , (8.48)
ẋi2 = −bxi2 , i = 0, · · · , N,
200 Cooperative Control of Multi-Agent Systems

where a > 0, b > 0, and h(xi1 ) is a nonlinear function represented by h(xi1 ) =

m1i xi1 + 21 (m2i −m1i )(|xi1 +1|−|xi1 −1|), where m1i < 0 and m2i < 0. The circuit
indexed by 0 is the leader and the other circuits are the followers. It is assumed
that the Chua’s circuits have nonidentical nonlinear components, i.e., m1i and
m2i are different for different Chua’s circuits. By letting xi = [xi1 , xi2 , xi3 ]T ,
then (8.48) can be rewritten in a compact form as

ẋi = Axi + B[ui + fi (xi )], i = 0, · · · , N, (8.49)

where
   
−m10 (a + 1) a 0 1
A= 1 −1 1 , B = 0 ,
0 −b 0 0
a 1
f0 (x0 ) = (m0 − m20 )(|x01 + 1| − |x01 − 1|),
2
a
fi (xi ) = a(m10 − m1i )xi1 + (m1i − m2i )(|xi1 + 1| − |xi1 − 1|), i = 1, · · · , N.
2
For simplicity, we let u0 = 0 and take f0 (x0 ) as the virtual control input of
the leader, which clearly satisfies kf0 (x0 )k ≤ a2 |m10 − m20 |. Let a = 9, b =
18, m10 = − 43 , and m20 = − 34 . In this case, the leader displays a double-
scroll chaotic attractor [107]. The parameters m1i and m2i , i = 1, · · · , N , are
randomly chosen within the interval [−6, 0). It is easy to see that

kfi (xi )k ≤ a|m10 − m1i |kxi k + a|m1i − m2i |

189
= kxi k + 54, i = 1, · · · , N.
4
Note that m10 is a parameter of the leader, which might not be available to
the followers. Therefore, although fi (xi ) satisfy the above condition, the upper
bound of m1i − m10 might be not explicitly known for the followers. Hence,
we will use the adaptive protocol (8.44) to solve the leader-follower consensus
problem.

0 1 6

2 5

3 4

FIGURE 8.4: A leader-follower communication graph.

Cooperative Control for Multi-Agent Systems with Matching Uncertainties 201
6 1.5 8

5
1 6

4
4
0.5
3
2
2 0
0
i1

i3
xi2
1 −0.5
x

x
−2
0
−1
−4
−1
−1.5
−6
−2
−2 −8
−3

−4 −2.5 −10
0 5 10 15 20 25 30 0 5 10 15 20 25 30 0 5 10 15 20 25 30

t t t

FIGURE 8.5: The state trajectories of the Chua’s circuits.

The communication graph is given as in FIGURE 8.4, where the node in-
dexed by 0 is the leader. Solving the LMI (8.8) gives the feedback gain matrices
of (8.44) as
 
  285.8453 280.3016 30.9344
K = − 16.9070 16.5791 1.8297 , Γ = 280.3016 274.8654 30.3344 .
30.9344 30.3344 3.3477

To illustrate Theorem 52, select κi = 0.5, ϕi = ψi = 0.05, and τi = ǫi = 5,

i = 1, · · · , 6, in (8.44). The state trajectories xi (t) of the circuits under (8.44)
designed as above are depicted in FIGURE 8.5, implying that leader-follower
consensus is indeed achieved. The adaptive gains dˆi and êi in (8.44) are shown
in FIGURE 8.6, which are clearly bounded.

45
80

40
70
35
60
30
50
25
40
i

ei

20
d

30
15
20
10

5 10

0 0

−5 −10
0 5 10 15 20 25 30 0 5 10 15 20 25 30
t t

FIGURE 8.6: The adaptive gains dˆi and êi in (8.23).

202 Cooperative Control of Multi-Agent Systems

8.3 Robustness with Respect to Bounded Non-Matching

Disturbances
In the preceding sections, the external disturbances in (8.1) are assumed to
satisfy the matching condition, i.e., Assumption 8.1. In this section, we ex-
amine the case where the agents are subject to external disturbances which
do not necessarily satisfy the matching condition and investigate whether the
proposed protocols in the preceding sections still ensure the boundedness of
the consensus error. For conciseness, we consider here only the case of the lead-
erless consensus problem. The case of the leader-follower consensus problem
can also be similarly discussed.
Consider a network of N agents whose communication graph is represented
by an undirected graph G. The dynamics of the i-th agent is described by

ẋi = Axi + B[ui + fi (xi , t)] + ωi , (8.50)

where fi (xi , t) is the lumped matching uncertainty defined as in (8.2) and

ωi ∈ Rn is the non-matching external disturbance, satisfying

Assumption 8.6 There exist positive constants υi such that kωi k ≤ υi , i =

1, · · · , N .

Next, we will redesign the distributed adaptive protocol (8.23) to ensure

the boundedness of the consensus error for the agents in (8.50). Using (8.23)
for (8.50), we can obtain the closed-loop dynamics of the network as

ξ˙ = (IN ⊗ A + M DL ⊗ BK)ξ + (M ⊗ B)[F (x, t) + R(ξ)] + (M ⊗ I)ω,

N
X N
X N
X
d¯˙i = τi [−ϕi d¯i + ( Lij ξjT )Γ( Lij ξj ) + kK Lij ξj k],
j=1 j=1 j=1
N
X
ē˙ i = ǫi [−ψi ēi + kK Lij ξj kkxi k], i = 1, · · · , N,
j=1
(8.51)
T T
where ω = [ω1T , · · · , ωN ] and the variables are defined as in (8.25).

Theorem 53 Suppose that G is connected and Assumptions 8.3, 8.5, and

8.6 hold. Then, both the consensus error ξ and the adaptive gains d¯i and ēi ,
i = 1, · · · , N , in (8.25) are uniformly ultimately bounded under the distributed
adaptive protocol (8.23) with K = −B T Q−1 and Γ = Q−1 BB T Q−1 , where
Q > 0 is a solution to the following LMI:

AQ + QAT + εQ − 2BB T < 0, (8.52)

where ε > 1. Moreover, we have

 Cooperative Control for Multi-Agent Systems with Matching Uncertainties 203

i) For any ϕi and ψi , the parameters ξ, d˜i , and ẽi exponentially converge to
the residual set
N N
1 X 2 1 λmax (L) X 2
D6 , {ξ, d˜i , ẽi : V6 < (β ϕi + e2i ψi + κi ) + υ },
2σ i=1 2 2σλmin (Q) i=1 i
(8.53)
where σ , mini=1,··· ,N {ε − 1, ϕi τi , ψi ǫi } and

XN N
X
1 T d˜2i ẽ2i
V4 = ξ (L ⊗ Q−1 )ξ + + , (8.54)
2 i=1
2τi i=1 2ǫi

where the variables are defined as in (8.27).

ii) If ϕi and ψi satisfy ̺ < ε − 1, where ̺ is defined in Theorem 51, then in
addition to i), ξ exponentially converges to the residual set
N N
λmax (Q) X 2 1 λmax (L) X 2
D7 , {ξ : kξk2 ≤ [ (β ϕi + e2i ψi + κi ) + υ ]}.
λ2 (ε − 1 − ̺) i=1 2 λmin (Q) i=1 i
(8.55)

Proof 55 Choose the Lyapunov function candidate as in (8.54). By following

similar steps as in the proof of Theorem 51, it is not difficult to get that the
time derivative of V4 along the trajectory of (8.51) can be obtained as
1 T
V̇4 ≤ ξ Wξ + ξ T (L ⊗ Q−1 )ω
2
N N (8.56)
1X 1X 2 1
− (ϕi d˜2i + ψi ẽ2i ) + (β ϕi + e2i ψi + κi ),
2 i=1 2 i=1 2

where W , L ⊗ (Q−1 A + AT Q−1 ) − 2βL2 ⊗ Q−1 BB T Q−1 . Using the following

fact:
1 T 1
ξ (L ⊗ Q−1 )ξ − ξ T (L ⊗ Q−1 )ω + ω T (L ⊗ Q−1 )ω
2 2
1 T −1
= (ξ − ω) (L ⊗ Q )(ξ − ω) ≥ 0,
2
we can get from (8.56) that
1 T 1
V̇4 ≤ ξ (W + L ⊗ Q−1 )ξ + ω T (L ⊗ Q−1 )ω
2 2
N N (8.57)
1X 1 X 1
− (ϕi d˜2i + ψi ẽ2i ) + (β 2 ϕi + e2i ψi + κi ),
2 i=1 2 i=1 2
204 Cooperative Control of Multi-Agent Systems

Note that (8.57) can be rewritten into

1
V̇4 ≤ −σV4 + ξ T [W + (σ + 1)L ⊗ Q−1 ]ξ
2
N
1 1X σ
+ ω T (L ⊗ Q−1 )ω − [(ϕi − )d˜2i
2 2 i=1 τi
N
σ 2 1X 2 1
+ (ψi − )ẽi )] + (β ϕi + e2i ψi + κi ) (8.58)
ǫi 2 i=1 2
1
≤ −σV4 + ξ T [W + (σ + 1)L ⊗ Q−1 ]ξ
2
N N
λmax (L) X 2 1 X 2 1
+ υi + (β ϕi + e2i ψi + κi ),
2λmin (Q) i=1 2 i=1 2

where we have used the fact that σ ≤ mini=1,··· ,N {ϕi τi , ψi ǫi } to get that last
inequality. Let ξˆ , [ξˆ1T , · · · , ξˆN ] = (U T ⊗ Q−1 )ξ, where U is defined as in
T T

the proof of Theorem 50. By the definition of ξ, ˆ it is easy to see that ξˆ1 = 0.
Since σ ≤ ε − 1 and β ≥ λ12 , we then have
N
X
ξ T [W + (σ + 1)L ⊗ Q−1 ]ξ ≤ λi ξˆiT [AQ + QAT + εQ − 2βλi BB T ]ξˆi
i=2
N
X
≤ λi ξˆiT [AQ + QAT + εQ − 2BB T ]ξˆi ≤ 0.
i=2
(8.59)
Then, it follows from (8.58) and (8.59) that
N N
λmax (L) X 2 1 X 2 1
V̇4 ≤ −σV4 + υ + (β ϕi + e2i ψi + κi ), (8.60)
2λmin (Q) i=1 i 2 i=1 2

which implies that V4 exponentially converges to the residual set D6 in (8.53)

with a convergence rate faster than e−σt . By following similar steps as in the
last part of the proof of Theorem 51, it is not difficult to show that for the case
where ϕi and ψi satisfy ̺ < ε − 1, ξ exponentially converges to the residual
set D7 . The details are omitted here for conciseness.
Remark 67 As shown in Proposition 2 in Chapter 2, there exists a Q > 0 sat-
isfying (8.52) if and only if (A, B) is controllable. Thus, a sufficient condition
for the existence of (8.3) satisfying Theorem 52 is that (A, B) is controllable,
which, compared to the existence condition of (8.3) satisfying Theorem 51, is
more stringent. It is worth mentioning that large ε in (8.52) yields a faster
convergence rate of the consensus error ξ, but meanwhile generally implies a
high-gain K in the protocol (8.23). In implementation, a tradeoff has to be
made when choosing ε.
Cooperative Control for Multi-Agent Systems with Matching Uncertainties 205

8.4 Distributed Robust Containment Control with Mul-

tiple Leaders
For the case where there exist multiple leaders, the robust containment control
problem arises. The dynamics of the followers are described by (8.2) and the
leaders have identical nominal linear dynamics, given by

ẋi = Axi + Bui , i = 1, · · · , M, (8.61)

where xi ∈ Rn is the state and ui ∈ Rp is the control input of the i-th leader.
The communication graph of the agents consisting of the N followers and the
M leaders is assumed to satisfy Assumption 7.3 in Section 7.3.
Distributed static and adaptive containment controllers can be designed
by combing the ideas in the previous section and Section 7.3. Details of the
derivations are omitted here for brevity. Interested readers can complete the
derivations as an exercise.

8.5 Notes
The materials of this chapter are mainly based on [77]. The matching un-
certainties associated with the agents are assumed to satisfy certain upper
bounds in this chapter. Another kind of commonly encountered uncertainties
is the uncertainties which can be linearly parameterized. The distributed co-
operative control problems of multi-agent systems with linearly parameterized
uncertainties have been considered in, e.g., [30, 31, 126, 193, 196]. Distribut-
ed control of multi-agent systems with norm-bounded uncertainties which do
not necessarily satisfy the matching condition has been addressed in [89, 169],
where the distributed robust stabilization problem of multi-agent systems with
parameter uncertainties was concerned in [89] and the robust synchronization
problem was investigated in [169] for uncertainties in the form of additive
perturbations of the transfer matrices of the nominal dynamics.
This page intentionally left blank
9
Global Consensus of Multi-Agent Systems
with Lipschitz Nonlinear Dynamics

CONTENTS
9.1 Global Consensus of Nominal Lipschitz Nonlinear Multi-Agent
Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
9.1.1 Global Consensus without Disturbances . . . . . . . . . . . . . . . . . . . . 208
9.1.2 Global H∞ Consensus Subject to External Disturbances . . 211
9.1.3 Extensions to Leader-Follower Graphs . . . . . . . . . . . . . . . . . . . . . . 214
9.1.4 Simulation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
9.2 Robust Consensus of Lipschitz Nonlinear Multi-Agent Systems with
Matching Uncertainties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
9.2.1 Distributed Static Consensus Protocols . . . . . . . . . . . . . . . . . . . . . 219
9.2.2 Distributed Adaptive Consensus Protocols . . . . . . . . . . . . . . . . . 224
9.2.3 Adaptive Protocols for the Case without Uncertainties . . . . 230
9.2.4 Simulation Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
9.3 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

In the previous chapters, the agent dynamics or the nominal agent dynamics
are assumed to be linear. This assumption might be restrictive in some circum-
stances. In this chapter, we intend to address the global consensus problem for
a class of nonlinear multi-agent systems, the Lipschitz nonlinear multi-agent
systems where each agent contains a nonlinear function satisfying a global
Lipschitz condition. The Lipschitz nonlinear multi-agent systems will reduce
to general linear multi-agent systems when the nonlinearity does not exist.
In Section 9.1, the global consensus problem of high-order multi-agent sys-
tems with Lipschitz nonlinearity and directed communication graphs is for-
mulated. A distributed consensus protocol is proposed, based on the relative
states of neighboring agents. A two-step algorithm is presented to construct
one such protocol, under which a Lipschitz multi-agent system without distur-
bances can reach global consensus for a strongly connected directed commu-
nication graph. The existence condition of the proposed consensus protocol is
also discussed. For the case where the agents are subject to external distur-
bances, the global H∞ consensus problem is formulated and another algorithm
is then given to design the protocol which achieves global consensus with a
guaranteed H∞ performance for a strongly connected balanced communica-
tion graph. It is worth mentioning that in these two algorithms the feedback

207
208 Cooperative Control of Multi-Agent Systems

gain design of the consensus protocol is decoupled from the communication

graph. We further extend the results to the case with a leader-follower com-
munication graph which contains a directed spanning tree with the leader as
the root. In the final part of this section, a simulation example is presented
for illustration.
In Section 9.2, we intend to consider the distributed robust consensus prob-
lem of a class of Lipschitz nonlinear multi-agent systems subject to different
matching uncertainties. Contrary to Section 9.1 where the agents have identi-
cal nominal dynamics, the multi-agent systems considered in this section are
subject to nonidentical matching uncertainties, so the multi-agent systems
discussed in this section are essentially heterogeneous, for which case the con-
sensus problem becomes quite challenging to solve. For the case where the
communication graph is undirected and connected, a distributed continuous
static consensus protocol based on the relative states of neighboring agents
is designed, under which the consensus error is uniformly ultimately bounded
and exponentially converges to an adjustable residual set. In order to remove
the requirement of the nonzero eigenvalues of the Laplacian matrix and the
upper bounds of the matching uncertainties, a fully distributed adaptive pro-
tocol is further designed, under which the residual set of the consensus error
is also given. Note that the upper bound of the consensus error can be made
to be reasonably small by properly selecting the design parameters of the
proposed protocols.

9.1 Global Consensus of Nominal Lipschitz Nonlinear

Multi-Agent Systems
9.1.1 Global Consensus without Disturbances
Consider a group of N identical nonlinear agents, described by

ẋi = Axi + g(xi ) + Bui , i = 1, · · · , N, (9.1)

where xi ∈ Rn and ui ∈ Rp are the state and the control input of the i-th a-
gent, respectively, A and B are constant matrices with compatible dimensions,
and the nonlinear function g(xi ) is assumed to satisfy the Lipschitz condition
with a Lipschitz constant γ > 0, i.e.,

kg(x) − g(y)k ≤ γkx − yk, ∀ x, y ∈ Rn . (9.2)

The communication graph among the N agents is represented by a directed

graph G. It is assumed that G is strongly connected throughout this subsection
and each agent has access to the relative states with respect to its neighbors.
In order to achieve consensus, the following distributed consensus protocol is
 Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 209

proposed:
N
X
ui = cK aij (xi − xj ), i = 1, · · · , N, (9.3)
j=1

where c > 0 ∈ R denotes the coupling gain, K ∈ Rp×n is the feedback gain
matrix, and aij is (i, j)-th entry of the adjacency matrix associated with G.
The objective is to design a consensus protocol (9.3) such that the N agents
in (9.1) can achieve global consensus in the sense of limt→∞ kxi (t)−xj (t)k = 0,
∀ i, j = 1, · · · , N.
Let r = [r1 , · · · , rN ]T ∈ RN be the left eigenvector of L associated with
the zero eigenvalue, satisfying rT 1 = 1 and ri > 0, i = 1, · · · , N . Define
e = (W ⊗ In )x, where W = IN − 1rT and e = [eT1 , · · · , eTN ]T . It is easy to
verify that (rT ⊗ In )e = 0. By the definition of rT , it is easy to see that 0
is a simple eigenvalue of W with 1 as a right eigenvector and 1 is the other
eigenvalue with multiplicity N − 1. Then, it follows that e = 0 if and only
if x1 = · · · = xN . Therefore, the consensus problem under the protocol (9.3)
can be reduced to the asymptotical stability of e. Using (9.3) for (9.1), it can
be verified that e satisfies the following dynamics:
N
X N
X
ėi = Aei + g(xi ) − rj g(xj ) + c Lij BKej , i = 1, · · · , N, (9.4)
j=1 j=1

where Lij is the (i, j)-th entry of the Laplacian matrix associated with G.
Next, an algorithm is presented to select the control parameters in (9.3).

Algorithm 17 For the agents in (9.1), a consensus protocol (9.3) can be

constructed as follows:

1) Solve the following linear matrix inequality (LMI):

 
AP + P AT − τ BB T + µI P
<0 (9.5)
P −(γ −1 )2 µI

to get a matrix P > 0 and scalars τ > 0 and µ > 0. Then, choose K =
− 21 B T P −1 .
τ
2) Select the coupling gain c ≥ a(L) , where a(L) is the generalized algebraic
connectivity of G (defined in Lemma 4 in Chapter 1).

Remark 68 It is worth noting that Algorithm 17 has a favorable decoupling

feature. Specifically, the first step deals only with the agent dynamics in (9.1)
while the second step tackles the communication graph by adjusting the cou-
pling gain c. The generalized algebraic connectivity a(L) for a given graph can
be obtained by using Lemma 8 in [189]. By Lemma 4, a(L) can be replaced by
T
λ2 ( L+L
2 ) in step 2), if the directed graph L is balanced and strongly connected.
210 Cooperative Control of Multi-Agent Systems

Remark 69 By using using Lemma 20 (Finsler’s Lemma), it is not difficult

to get that there exist a P > 0, a τ > 0 and a µ > 0 such that (9.5) holds if and
only if there exists a K such that (A+BK)P +P (A+BK)T +µI +α2 µ−1 P 2 <
0, which, with µ = 1, is the existence condition for the observer design problem
of a single Lipschitz system in [128, 129]. According to Theorem 2 in [129],
the LMI (9.5) is feasible if the distance to unobservability of (AT , B T ) is larger
than γ and A is stable. Besides, a scalar µ > 0 is introduced into (9.5) so as
to reduce conservatism.

The following theorem presents a sufficient condition for the global con-
sensus of (9.4).

Theorem 54 Assume that the directed graph G is strongly connected and

there exists a solution to (9.5). Then, the N agents in (9.1) can reach global
consensus under the protocol (9.3) constructed by Algorithm 17.

Proof 56 Consider the Lyapunov function candidate

N
X
V1 = ri eTi P −1 ei . (9.6)
i=1

Clearly, V1 is positive definite. The time derivative of V1 along the trajectory

of (9.4) is given by
N
X N
X N
X
V̇1 = 2 ri eTi P −1 [Aei + g(xi ) − ri g(xj ) + c Lij BKej ]
i=1 j=1 j=1
N
X N
X
=2 ri eTi P −1 [Aei + c Lij BKej ] (9.7)
i=1 j=1
N
X N
X
+2 ri eTi P −1 [g(xi ) − g(x̄) + g(x̄) − ri g(xj )],
i=1 j=1

PN
where x̄ = j=1 rj xj .
Using the Lipschitz condition (9.2) gives

2eTi P −1 (g(xi ) − g(x̄)) ≤ 2αkP −1 ei kkei k

(9.8)
≤ eTi [µ(P −1 )2 + µ−1 γ 2 I]ei ,
P
where µ is a positive scalar. Since i=1 ri ei = 0, we have

X N
X
ri eTi P −1 [g(x̄) − rj g(xj )] = 0. (9.9)
i=1 j=1
Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 211

Let ξi = P −1 ei , ξ = [ξ1T , · · · , ξN ] . Substituting K = −B T P −1 in (9.7), it

T T

follows from (9.7) by using (9.8) and (9.9) that

N
X N
X
−1 2 2
V̇1 ≤ ri ξiT [(AP T
+ P A + µI + µ γ P )ξi + 2c Lij BKP ξj ]
i=1 j=1
c
= ξ T [R ⊗ (AP + P AT + µI + µ−1 γ 2 P 2 ) − (RL + LT R) ⊗ BB T ]ξ,
2
(9.10)
where R = diag(r1 , · · · , rN ).
Since (rT ⊗ In )e = 0, i.e., (rT ⊗ In )ξ = 0, we can get from Lemma 4 that

ξ T [(RL + LT R) ⊗ In ]ξ ≥ 2a(L)ξ T (R ⊗ In )ξ, (9.11)

where a(L) > 0. In light of (9.11), it then follows from (9.10) that

V̇1 ≤ ξ T [R ⊗ (AP + P AT + µI + µ−1 γ 2 P 2 − c a(L)BB T )]ξ. (9.12)

By steps 1) and 2) in Algorithm 17, we can obtain that

AP + P AT + µI + µ−1 γ 2 P 2 − c a(L)BB T
(9.13)
≤ AP + P AT + µI + µ−1 γ 2 P 2 − τ BB T < 0,

where the last inequality follows from (9.5) by using Lemma 19 (the Schur
complement lemma). Therefore, it follows from (9.12) that V̇1 < 0, implying
that e(t) → 0, as t → ∞. That is, the global consensus of the network (9.4) is
achieved.

Remark 70 Theorem 54 converts the global consensus of the N Lipschitz

agents in (9.1) under the protocol (9.3) to the feasibility of a low-dimensional
linear matrix inequality. The effect of the communication graph on consensus
is characterized by the generalized algebraic connectivity of the communication
graph.

9.1.2 Global H∞ Consensus Subject to External Distur-

bances
This subsection continues to consider a network of N identical nonlinear agents
subject to external disturbances, described by

ẋi = Axi + g(xi ) + Bui + Dωi , i = 1, · · · , N, (9.14)

where xi ∈ Rn is the state of the i-th agent, ui ∈ Rp is the control input,

ω i ∈ Lm
2 [0, ∞) is the external disturbance, and g(xi ) is a nonlinear function
1

satisfying (9.2). The communication graph G is assumed to balanced and

strongly connected in this subsection.
 212 Cooperative Control of Multi-Agent Systems

The objective in this subsection is to design a protocol (9.3) for the agents
in (9.14) to reach global consensus and meanwhile maintain a desirable dis-
turbance rejection performance. To this end, define the performance variable
zi , i = 1, · · · , N , as the average of the weighted relative states of the agents:
N
1 X
zi = C(xi − xj ), i = 1, · · · , N, (9.15)
N j=1

where zi ∈ Rm2 and C ∈ Rm2 ×n is a constant matrix.

With (9.14), (9.3), and (9.15), the closed-loop network dynamics can be
obtained as
N
X
ẋi = Axi + g(xi ) + c Lij BKxj + Dωi ,
j=1
(9.16)
N
X
1
zi = C(xi − xj ), i = 1, · · · , N,
N j=1

The global H∞ consensus problem for (9.14) under the protocol (9.3) is
first defined.

Definition 18 Given a positive scalar η, the protocol (9.3) is said to achieve

global consensus with a guaranteed H∞ performance η for the agents in (9.14),
if the following two requirements hold:
(1) The network (9.16) with ωi = 0 can reach global consensus in the sense of
limt→∞ kxi − xj k = 0, ∀ i, j = 1, · · · , N .
(2) Under the zero-initial condition, the performance variable z satisfies
Z ∞
J= [z T (t)z(t) − η 2 ω T (t)ω(t)]dt < 0, (9.17)
0

where z = [z1T , · · · , zN
T T
] , ω = [ω1T , · · · , ωN
T T
] .

Next, an algorithm for the protocol (9.3) is presented.

Algorithm 18 For a given scalar γ > 0 and the agents in (9.14), a consensus
protocol (9.3) can be constructed as follows:
1) Solve the following LMI:
 
AQ + QAT − ςBB T + µI Q QC T D
 Q −(γ −1 )2 µ 0 0 
 <0 (9.18)
 CQ 0 −I 0 
DT 0 0 η2 I

to get a matrix Q > 0 and scalars ς > 0 and µ > 0. Then, choose K =
− 21 B T Q−1 .
 Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 213
T T
2) Select the coupling gain c ≥ ς/λ2 ( L+L
2 ), where λ2 ( L+L
2 ) denotes the s-
L+LT
mallest nonzero eigenvalue of 2 .

Theorem 55 Assume that G is balanced and strongly connected, and there

exists a solution to (9.18). Then, the protocol (9.3) constructed by Algorithm
18 can achieve global consensus with a guaranteed H∞ performance η > 0 for
the N agents in (9.14).
PN
Proof 57 Let ei = xi − N1 j=1 xj , and e = [eT1 , · · · , eTN ]T . As shown in the
last subsection, we know that e = 0 if and only if x1 = · · · = xN . From (9.16),
it is easy to get that e satisfies the following dynamics:
N
1 X
ėi = Aei + g(xi ) − g(xj )
N j=1
N
X N (9.19)
1 X
+c Lij BKej + D(ωi − ωj ),
j=1
N j=1
zi = Cei , i = 1, · · · , N.

Therefore, the protocol (9.3) solves the global H∞ consensus problem, if the
system (9.19) is asymptotically stable and satisfies (9.17).
Consider the Lyapunov function candidate
N
X
V2 = eTi Q−1 ei .
i=1

By following similar steps to those in Theorem 54, we can obtain the time
derivative of V2 along the trajectory of (9.19) as
N
X N
X
V̇2 ≤ eTi [(2Q−1 A + γ 2 µ−1 I + µ(Q−1 )2 )ei + 2c Lij Q−1 BKej
i=1 j=1
N
X
2
+ Q−1 D(ωi − ωj )] (9.20)
N j=1
c
= êT [IN ⊗ (AQ + QAT + γ 2 µ−1 Q2 + µI) − (L + LT ) ⊗ BB T ]ê
2
+ 2êT (M ⊗ D)ω,
1
where êi = Q−1 ei , ê = [êT1 , · · · , êTN ]T , and M = IN − T
N 11 .
214 Cooperative Control of Multi-Agent Systems

Next, for any nonzero ω, we have

Z ∞
J= [z T (t)z(t) − η 2 ω T (t)ω(t) + V̇2 ]dt − V2 (∞) + V2 (0)
0
Z ∞
c
≤ êT [IN ⊗ (AQ + QAT + γ 2 µ−1 Q2 + µI) − (L + LT ) ⊗ BB T
0 2
+ IN ⊗ QC T CQ + η −2 M 2 ⊗ DDT ]ê dt
Z ∞
− η 2 [ω − η −2 (M ⊗ DT )ê]T [ω − η −2 (M ⊗ DT )ê]dt
0
− V2 (∞) + V2 (0).
(9.21)
By noting that 1 is the right and also the left eigenvectors of L + LT and
M associated with the zero eigenvalue, respectively, we have

M (L + LT ) = L + LT = (L + LT )M.

Thus there exists a unitary matrix Û ∈ RN ×N such that Û T M Û and Û T (L +

T T
hL )Ûi are both diagonal [58]. Moreover, we can choose Û = [ N ] , Û =
√1 Y
1T
√
N , with Y ∈ RN ×(N −1) , W ∈ R(N −1)×N , satisfying
W
 
0 0
Û T M Û = Π = ,
0 IN −1
(9.22)
1 T
Û (L + LT )Û = Λ = diag(0, λ2 , · · · , λN ),
2
T
where λi , i = 2, · · · , N , are the nonzero eigenvalues of L+L 2 . Let ζ ,
T
[ζ1T , · · · , ζN ] = (Û T ⊗ In )ê. Clearly, ζ1 = ( √1N ⊗ Q−1 )e = 0. By using (9.22),
T T

we can obtain that

c
êT [IN ⊗ (AQ + QAT + γ 2 µ−1 Q2 + µI) − (L + LT ) ⊗ BB T
2
+ IN ⊗ QC T CQ + η −2 M 2 ⊗ DDT ]ê
= ζ T [IN ⊗ (AQ + QAT + γ 2 µ−1 Q2 + µI) − cΛ ⊗ BB T
+ IN ⊗ QC T CQ + η −2 Π2 ⊗ DDT ]ζ (9.23)
N
X
= ζiT [AQ + QAT + γ 2 µ−1 Q2 + µI
i=2
− cλi BB T + η −2 DDT + QC T CQ]ζi .

In light of steps 1) and 2) in Algorithm 18, we have

AQ + QAT + γ 2 µ−1 Q2 + µI − cλi BB T + η −2 DDT + QC T CQ

≤ AQ + QAT + γ 2 µ−1 Q2 + µI − ςBB T + η −2 DDT + QC T CQ < 0.
(9.24)
Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 215

By comparing Algorithm 18 with Algorithm 17, it follows from Theorem

54 that the first condition in Definition 18 holds. Since x(0) = 0, it is clear
that V2 (0) = 0. Considering (9.23) and (9.24), we can obtain from (9.21) that
J < 0. Therefore, the global H∞ consensus problem is solved.
Remark 71 Theorem 55 and Algorithm 18 extend Theorem 54 and Algorithm
17 to evaluate the performance of a multi-agent network subject to external
disturbances. The decoupling property of Algorithm 17 still holds for Algorithm
18.

9.1.3 Extensions to Leader-Follower Graphs

In the above subsections, the communication graph is assumed to be strongly
connected, where the final consensus value reached by the agents is generally
not explicitly known. In many practical cases, it is desirable that the agents’
states asymptotically approach a reference state. In this subsection, we intend
to consider the case where a network of N agents in (9.1) maintains a leader-
follower communication graph G. An agent is called a leader if the agent has
no neighbor, i.e., it does not receive any information. An agent is called a
follower if the agent has at least one neighbor. Without loss of generality,
assume that the agent indexed by 1 is the leader with u1 = 0 and the rest
N − 1 agents are followers. The following distributed consensus protocol is
proposed for each follower:
N
X
ui = cK aij (xi − xj ), i = 2, · · · , N, (9.25)
j=1

where c > 0, K, and aij are the same as defined in (9.3).

The objective in this subsection is to solve the global leader-follower con-
sensus problem, i.e., to design a consensus protocol (9.25) under which the
states of the followers asymptotically approach the state of the leader in the
sense of limt→∞ kxi (t) − x1 (t)k = 0, ∀ i = 2, · · · , N .
In the sequel, we make the following assumption.
Assumption 9.1 The communication graph G contains a directed spanning
tree with the leader as the root.
Because the leader has no neighbors,
h the Laplacian
i matrix L associated
0 01×(N −1)
with G can be partitioned as L = L2 L1
, where L2 ∈ R(N −1)×1 and
L1 ∈ R(N −1)×(N −1) . Under Assumption 9.1, it follows from Lemma 1 that
all the eigenvalues of L1 have positive real parts and L1 is a nonsingular
M -matrix. By Lemma 29, we have
1
G , diag(q2 , · · · , qN ) > 0 and H, (GL1 + LT1 G) > 0, (9.26)
2
where q = [q2 , · · · , qN ]T = (LT1 )−1 1.
 216 Cooperative Control of Multi-Agent Systems

Algorithm 19 For the agents in (9.1) satisfying Assumption 9.1, a consen-

sus protocol (9.25) can be constructed as follows:

1) Solve the LMI (9.5) to get a matrix P > 0 and scalars τ > 0 and µ > 0.
Then, choose K = − 12 B T P −1 .
τ
2) Select the coupling gain c ≥ λmin (H) min qi , where H and qi are defined in
i=2,··· ,N

(9.26).

Remark 72 For the case where the subgraph associated with the followers is
balanced and strongly connected, L1 + LT1 > 0 [86]. Then, by letting G = I,
L1 +LT
step 2) can be simplified to c ≥ τ /λmin ( 2
1
).

Theorem 56 Suppose that Assumption 9.1 holds and there exists a solution
to (9.5). Then, the consensus protocol (9.25) given by Algorithm 19 solves the
leader-follower global consensus problem for the N agents described by (9.1).

Proof 58 Denote the consensus errors by υi = xi − x1 , i = 2, · · · , N . Then,

we can obtain from (9.1) and (9.25) the closed-loop network dynamics as
N
X
υ̇i = Aυi + g(xi ) − g(x1 ) + cBK aij (υi − υj ), i = 2, · · · , N, (9.27)
j=1

Clearly, the leader-follower global consensus problem can be reduced to the

asymptotical stability of (9.27).
Consider the Lyapunov function candidate
N
X
V3 = qi υiT P −1 υi ,
i=2

where qi is defined as in (9.26). Clearly V3 is positive definite. Following simi-

lar steps in proving Theorem 54, the time derivative of V3 along the trajectory
of (9.27) is obtained as
N
X N
X
V̇3 = 2 qi υiT P −1 [Aυi + g(xi ) − g(x1 ) + cBK aij (υi − υj )]
i=2 j=1
N
X N
X
≤ qi υ̃iT [(AP + P AT + µI + γ 2 µ−1 P 2 )υ̃i − cBB T aij (υ̃i − υ̃j )]
i=2 j=1

= υ̃ T [G ⊗ (AP + P AT + µI + γ 2 µ−1 P 2 ) − cH ⊗ BB T ]υ̃

= υ̃ T [G ⊗ (AP + P AT + µI + γ 2 µ−1 P 2 − τ BB T )]υ̃
+ υ̃ T [(τ G − cH) ⊗ BB T ]υ̃,
(9.28)
Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 217

where υ̃i = P −1 υi , υ̃ = [υ̃2T , · · · , υ̃N

T T
] , G and H are defined in (9.26), and
we have used (9.8) to get the first inequality.
Since G and H are positive definite, we can get that τ G − cH < 0 if
τ G < cλmin (H)I, which is true in light of step 2) in Algorithm 19. Thus,
(τ G − cH) ⊗ BB T ≤ 0. Then, it follows from (9.13) and (9.28) that V̇3 < 0,
implying that (9.27) is asymptotically stable. This completes the proof.

The global H∞ consensus for the agents in (9.14) with a leader-follower

communication graph can be discussed similarly, which is omitted here for
brevity.

9.1.4 Simulation Example

In this subsection, a simulation example is provided for illustration.

1 6

2 5

3 4

FIGURE 9.1: A directed balanced communication graph.

Example 25 Consider a network of six single-link manipulators with revo-

lute joints actuated by a DC motor. The dynamics of the i-th manipulator is
described by (9.14), with [129, 201]
     
xi1 0 1 0 0 0
xi2  −48.6 −1.26 48.6 0  
xi =     , B = 21.6 ,
xi3  , A =  0 0 0 10   0 
xi4 1.95 0 −1.95 0 0
 T  
D = 0 1 0.4 0 , C = 1 0 0 0 ,
 T
g(xi ) = 0 0 0 −0.333sin(xi1 ) .

Clearly, g(xi ) here satisfies (9.2) with a Lipschitz constant α = 0.333. The
external disturbance here is ω = [w, −w, 1.5w, 3w, −0.6w, 2w]T , where w(t) is
a one-period square wave starting at t = 0 with width 2 and height 1.
Choose the H∞ performance index η = 2. Solving the LMI (9.18) by using
218 Cooperative Control of Multi-Agent Systems
5 100

80

0 60

40

−5 20

i2
i1

x
x

−10 −20

−40

−15 −60

−80

−20 −100
0 1 2 3 4 5 0 1 2 3 4 5
t t
5 3

2
0

−5
0
i3

xi4
x

−1
−10

−2

−15
−3

−20 −4
0 1 2 3 4 5 0 1 2 3 4 5
t t

FIGURE 9.2: The six manipulators reach global consensus.

the LMI toolbox of MATLAB gives a feasible solution:

 
0.4060 −0.9667 0.3547 −0.0842
−0.9667 67.6536 0.0162 −0.0024
Q=  0.3547
, ς = 29.6636.
0.0162 0.4941 −0.0496
−0.0842 −0.0024 −0.0496 0.0367

Thus, by Algorithm 18, the feedback gain matrix of (9.3) is chosen as

 
K = −2.4920 −0.1957 1.4115 −3.8216 .

For illustration, let the communication graph G be given as in FIGURE 9.1. It

is easy to verify that G is balanced and strongly connected. The corresponding
Laplacian matrix is
 
2 0 −1 −1 0 0
−1 2 0 0 0 −1
 
 0 −1 1 0 0 0
L=  .
−1 0 0 2 −1 0  
0 0 0 −1 2 −1
0 −1 0 0 −1 2
T
The smallest nonzero eigenvalue of L+L2 is equal to 0.8139. By Theorem 55
and Algorithm 18, the protocol (9.3) with K chosen as above achieves global
Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 219

consensus with a H∞ performance γ = 2, if the coupling gain c ≥ 36.4462.

For the case without disturbances, the state trajectories of the six manipula-
tors under the protocol (9.3) with K given as above and c = 37 are depicted
in FIGURE 9.2, from which it can be observed that the global consensus is
indeed achieved. With the zero-initial condition and external disturbances ω,
the trajectories of the performance variables zi , i = 1, · · · , 6, are shown in
FIGURE 9.3.

0.3

0.2

0.1

0
i
z

−0.1

−0.2

−0.3

−0.4
0 1 2 3 4 5 6 7 8
t
FIGURE 9.3: The performance variables zi , i = 1, · · · , 6.

9.2 Robust Consensus of Lipschitz Nonlinear Multi-

Agent Systems with Matching Uncertainties
In this section, we consider a network of N autonomous agents with Lipschitz
nonlinear dynamics subject to heterogeneous uncertainties. The dynamics of
the i-th agent are described by
ẋi = Axi + g(xi ) + B[ui + fi (xi , t)], i = 1, · · · , N, (9.29)
where xi ∈ Rn is the state, ui ∈ Rp is the control input, A and B are con-
stant known matrices with compatible dimensions, and fi (xi , t) ∈ Rn denotes
the lumped matching uncertainty (including the parameter uncertainties and
external disturbances) associated with the i-th agent, and the nonlinear func-
tion g(xi ) is assumed to satisfy the Lipschitz condition (9.2) with a Lipschitz
constant γ > 0 .
Regarding the bounds of the uncertainties fi (xi , t), we introduce the fol-
lowing assumption.
Assumption 9.2 There exist continuous scalar valued functions ρi (xi , t), i =
1, · · · , N , such that kfi (xi , t)k ≤ ρi (xi , t), i = 1, · · · , N , for all t ≥ 0 and
xi ∈ Rn .
220 Cooperative Control of Multi-Agent Systems

The communication graph among the N agents is represented by a undi-

rected graph G, which is assumed to be connected throughout this section.
The objective of this section is to solve the consensus problem, i.e., to design
distributed protocols such that the agents in (9.29) achieve consensus in the
sense of limt→∞ kxi (t) − xj (t)k = 0, ∀ i, j = 1, · · · , N.

9.2.1 Distributed Static Consensus Protocols

Based on the relative states of neighboring agents, the following distributed
static consensus protocol is proposed:
N
X N
X
ui = cK aij (xi − xj ) + ρi (xi , t)ri (K aij (xi − xj )), i = 1, · · · , N,
j=0 j=0
(9.30)
where c > 0 is the constant coupling gain, K ∈ Rp×n is the feedback gain
matrix, aij is the (i, j)-th entry of the adjacency matrix associated with G,
and the nonlinear functions ri (·) are defined such that for w ∈ Rn ,
(
w
if ρi (xi , t)kwk > κi
ri (w) = kwkw
(9.31)
κi ρi (xi , t) if ρi (xi , t)kwk ≤ κi

where κi are small positive values

Define ξ = (M ⊗ In )x, where M = IN − N1 11T and ξ = [ξ1T , · · · , ξN T T
] . It
is easy to see that 0 is a simple eigenvalue of M with 1 as a corresponding
eigenvector and 1 is the other eigenvalue with multiplicity N − 1. Then, it
follows that ξ = 0 if and only if x1 = · · · = xN . Therefore, the states of (9.29)
under (9.49) reach agreement if and only if ξ asymptotically converges to zero.
Hereafter, we refer to ξ as the consensus error. By noting that LM = L, it is
not difficult to obtain that the closed-loop network dynamics can be written
in terms of ξ as

ξ˙ = (IN ⊗ A + cL ⊗ BK)ξ + (M ⊗ I)G(x)

(9.32)
+ (M ⊗ B)F (x, t) + [M ρ(x, t) ⊗ B]R(ξ),

where L denotes the Laplacian matrix of G, x = [xT1 , · · · , xTN ]T , ρ(x, t) =

diag(ρ1 (x1 , t), · · · , ρN (xN , t)), F (x, t) = [f1 (x1 , t)T , · · · , fN (xN , t)T ]T , G(x) =
[g(x1 )T , · · · , g(xN )T ]T , and
 PN 
r1 (K j=1 L1j ξj )
 .. 
R(ξ) =   .
.
 (9.33)
PN
rN (K j=1 LN j ξj )

The following result provides a sufficient condition to design the consensus

protocol (9.30).
Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 221

Theorem 57 Suppose that the communication graph G is undirected and con-

nected and Assumption 9.2 holds. Solve the LMI (9.5) to get a matrix P > 0,
scalars τ > 0 and µ > 0. The parameters in the distributed protocol (9.30)
are designed as c ≥ λτ2 and K = −B T P −1 , where λ2 is the smallest nonzero
eigenvalue of L. Then, the consensus error ξ of (9.32) is uniformly ultimately
bounded and exponentially converges to the residual set
N
2λmax (P ) X
D1 , {ξ : kξk2 ≤ κi }, (9.34)
αλ2 i=1

where

α = −λmax (AP + P AT − 2τ BB T + µI + µ−1 γ 2 P 2 )/λmax (P ). (9.35)

Proof 59 Consider the following Lyapunov function candidate:

1 T
V5 = ξ (L ⊗ P −1 )ξ.
2
By the definition of ξ, it is easy to see that (1T ⊗ I)ξ = 0. For a connected
graph G, it then follows from Lemma 2 that
1 λ2
V5 (ξ) ≥ λ2 ξ T (I ⊗ P −1 )ξ ≥ kξk2 . (9.36)
2 2λmax (P )
The time derivative of V5 along the trajectory of (9.32) is given by

V̇5 = ξ T (L ⊗ P −1 )ξ˙
= ξ T (L ⊗ P −1 A + cL2 ⊗ P −1 BK)ξ
(9.37)
+ ξ T (L ⊗ P −1 )G(x) + ξ T (L ⊗ P −1 B)F (x, t)
+ ξ T [Lρ(x, t) ⊗ P −1 B]R(ξ).
By using Assumption 9.2, we can obtain that
N
X N
X
ξ T (L ⊗ P −1 B)F (x, t) ≤ kB T P −1 Lij ξj kkfi (xi , t)k
i=1 j=1
(9.38)
N
X N
X
T −1
≤ ρi (xi , t)kB P Lij ξj k.
i=1 j=1

Observe that
ξ T (L ⊗ P −1 )G(x) = ξ T (L ⊗ P −1 )[(M ⊗ I)G(x)]
= ξ T (L ⊗ P −1 )[G(x) − 1 ⊗ g(x̄)
1 (9.39)
+ 1 ⊗ g(x̄) − ( 11T ⊗ I)G(x)]
N
= ξ T (L ⊗ P −1 )[G(x) − 1 ⊗ g(x̄)],
222 Cooperative Control of Multi-Agent Systems

where x̄ = N1 (1T ⊗ I)x and we have used L1 = 0 to get the last equation.
Next, considerPthe following three cases.
N
i) ρi (xi , t)kK j=1 Lij ξj k > κi , i = 1, · · · , N .
In this case, it follows from (9.31) and (9.33) that
N
X N
X
T −1 T −1
ξ [Lρ(x, t) ⊗ P B]R(ξ) = − ρi (xi , t)kB P Lij ξj k. (9.40)
i=1 j=1

Substituting (9.38), (9.39), and (9.40) into (9.37) yields

1 T
V̇5 ≤ ξ [X ξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))],
2
where X = L ⊗ (P −1 A + AT P −1 ) − 2cL2 ⊗ P −1 BB T P −1 .
PN
ii) ρi (xi , t)kK j=1 Lij ξj k ≤ κi , i = 1, · · · , N .
In this case, we can get from (9.31) and (9.33) that

ξ T [Lρ(x, t) ⊗ P −1 B]R(ξ)
N
X N
X
ρi (xi , t)2 (9.41)
=− kB T P −1 Lij ξj k2 ≤ 0.
i=1
κi j=1

Substituting (9.38), (9.39), and (9.41) into (9.37) gives

XN
1 T
V̇5 ≤ ξ [X ξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))] + κi . (9.42)
2 i=1

iii) ξ satisfies neither Case i) nor Case ii).

PN
Without loss of generality, assume that ρi (xi , t)kK j=1 Lij ξj k > κi , i =
PN
1, · · · , l, and ρi (xi , t)kK j=1 Lij ξj k ≤ κi , i = l + 1, · · · , N , where 2 ≤ l ≤
N − 1. By combing (9.40) and (9.41), in this case we can get that
l
X N
X
ξ T [Lρ(x, t) ⊗ P −1 B]R(ξ) ≤ − ρi (xi , t)kB T P −1 Lij ξj k. (9.43)
i=1 j=1

Then, it follows from (9.37), (9.40), (9.43), (9.38), and (9.39) that

X N −l
1 T
V̇5 ≤ ξ [X ξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))] + κi .
2 i=1

Therefore, by analyzing the above three cases, we get that V̇5 satisfies (9.42)
Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 223

for all ξ ∈ RN n . Note that (9.42) can be rewritten as

XN
1
V̇5 ≤ −αV5 + αV5 + ξ T [X ξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))] + κi
2 i=1
1
= −αV5 + ξ T [X + αL ⊗ P −1 + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))]
2
N
X
+ κi ,
i=1
(9.44)
where α > 0.
Because G is connected, it follows from Lemma 1 that zero is a simple
h L and
eigenvalue of √1
i all the other eigenvalues are positive. Let U = [ N ]
Y1
1T
and U T = √
N , with Y1 ∈ RN ×(N −1) , Y2 ∈ R(N −1)×N , be such unitary
Y2
matrices that U T LU = Λ , diag(0, λ2 , · · · , λN ), where λ2 ≤ · · · ≤ λN are
the nonzero eigenvalues of L. Let ξ¯ , [ξ¯1T , · · · , ξ¯N
T T
] = (U T ⊗ I)ξ. By the
¯ it is easy to see that ξ¯1 = ( √T ⊗ I)ξ = 0. In light of
definitions of ξ and ξ, 1
N
Lemma 18, it then follows that

2ξ T (L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))
= 2ξ¯T (Λ ⊗ P −1 )(U T ⊗ I)(G(x) − 1 ⊗ g(x̄))
1
≤ µξ¯T (Λ ⊗ P −1 )(Π−1 ⊗ I)(Λ ⊗ P −1 )ξ¯ + [G(x) − 1 ⊗ g(x̄)]T
µ
× (U ⊗ I)(Π ⊗ I)(U T ⊗ I)[G(x) − 1 ⊗ g(x̄)] (9.45)
N
X
1
= µξ¯T (ΛΠ−1 Λ ⊗ (P −1 )2 )ξ¯ + πi kg(xi ) − g(x̄)k2
µ i=1

¯T −1 −1 2 1 ¯
= µξ [ΛΠ Λ ⊗ (P ) + γ 2 Π]ξ,
µ
where Π = diag(π1 , · · · , πN ) is a positive-definite diagonal matrix and µ > 0,
and we have used the Lipschitz condition (9.2) to get the last inequality. By
choosing Π = diag(1, λ2 , · · · , λN ) and letting ξˆ = (I ⊗ P −1 )ξ¯ = (U T ⊗ P −1 )ξ,
it then follows from (9.45) and (9.44) that

ξ T [X + αL ⊗ P −1 + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))]

N
X
= λi ξˆiT (AP + P AT − 2cλi BB T + αP + µI + µ−1 γ 2 P 2 )ξˆi
i=2 (9.46)
N
X
≤ λi ξˆiT (AP + P AT − 2τ BB T + αP + µI + µ−1 γ 2 P 2 )ξˆi .
i=2

By using Lemma 19 (Schur complement lemma), it follows from (9.5) that

224 Cooperative Control of Multi-Agent Systems

AP + P AT − 2τ BB T + µI + µ−1 γ 2 P 2 < 0. Because α = −λmax (AP + P AT −

2τ BB T + µI + µ−1 γ 2 P 2 )/λmax (P ), we can see from (9.46) that

ξ T [X + αL ⊗ P −1 + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))] ≤ 0.

Then, we can get from (9.44) that

N
X
V̇5 ≤ −αV5 + κi . (9.47)
i=1

By using Lemma 16 (the Comparison lemma), we can obtain from (9.47) that
N N
1X 1X
V5 (ξ) ≤ [V5 (ξ(0)) − κi ]e−αt + κi , (9.48)
α i=1 α i=1

which, by (9.36), implies that ξ exponentially converges to the residual set D1

in (9.34) with a convergence rate not less than e−αt .

Remark 73 Note that the nonlinear functions ri (·) in (9.31) are actually
continuous approximations, via(the boundary layer concept [43, 71], of the
w
if kwk =
6 0
discontinuous function ĝ(w) = kwk . The values of κi in (9.31)
0 if kwk = 0
define the widths of the boundary layers. From (9.34) we can observe that
the residual set D1 of the consensus error ξ depends on the smallest nonzero
eigenvalue of L, the number of agents, the largest eigenvalue of P , and the
values of κi . By choosing sufficiently small κi , the consensus error ξ under
the protocol (9.30) can converge to an arbitrarily small neighborhood of zero.

9.2.2 Distributed Adaptive Consensus Protocols

In the last subsection, the design of the protocol (9.30) relies on the minimal
nonzero eigenvalue λ2 of L which is global information in the sense that each
agent has to know the entire communication graph to compute it and the
upper bounds ρi (xi , t) of the matching uncertainties fi (xi , t), which might
not be easily obtained in some cases, In this subsection, we will develop some
fully distributed consensus protocols without using either λ2 or ρi (xi , t).
Before moving forward, we introduce a modified assumption regarding the
bounds of the lumped uncertainties fi (xi , t), i = 1, · · · , N .

Assumption 9.3 There are positive constants ai and bi such that

kfi (xi , t)k ≤ ai + bi kxi k, i = 1, · · · , N .

Based on the local state information of neighboring agents, we propose the

Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 225

following distributed adaptive protocol to each agent:

N
X N
X
ui = di K aij (xi − xj ) + r̄i (K aij (xi − xj )),
j=1 j=1
N
X N
X
d˙i = νi [−ϕi di + ( aij (xi − xj )T )Γ( aij (xi − xj ))
j=1 j=1
(9.49)
N
X
+ kK aij (xi − xj )k],
j=1
N
X
ėi = ǫi [−ψi ei + kK aij (xi − xj )kkxi k], i = 1, · · · , N,
j=1

where di (t) and ei (t) are the adaptive gains associated with the i-th agent,
K ∈ Rp×n and Γ ∈ Rn×n are the feedback gain matrices to be designed, νi
and ǫi are positive scalars, ϕi and ψi are small positive constants, the nonlinear
functions r̄i (·) are defined such that for w ∈ Rn ,
( w(d +e kx k)
i i
kwk
i
if (di + ei kxi k)kwk > κi
r̄i (w) = w(di +e i kxi k)
2 (9.50)
κi if (di + ei kxi k)kwk ≤ κi
and the rest of the variables are defined as in (9.30).
Let the consensus error ξ be defined as in (9.32) and D(t) =
diag(d1 (t), · · · , dN (t)). Then, it is not difficult to get from (9.29) and (9.49)
that the closed-loop network dynamics can be written as
ξ˙ = (IN ⊗ A + M DL ⊗ BK)ξ + (M ⊗ I)G(x)
+ (M ⊗ B)[F (x, t) + R̄(ξ)],
N
X N
X N
X
d˙i = νi [−ϕi di + ( Lij ξjT )Γ( Lij ξj ) + kK Lij ξj k], (9.51)
j=1 j=1 j=1
N
X
ėi = ǫi [−ψi ei + kK Lij ξj kkxi k], i = 1, · · · , N,
j=1

where  PN 
r̄i (K j=1 L1j ξj )
 .. 
R̄(ξ) = 
 .
,
 (9.52)
PN
r̄i (K j=1 LN j ξj )
and the rest of the variables are defined as in (9.32).
In order to establish the ultimate boundedness of the states ξ, di , and ei
of (9.51), we use the following Lyapunov function candidate
XN N
X
1 T d˜2i ẽ2i
V6 = ξ (L ⊗ P −1 )ξ + + , (9.53)
2 i=1
2νi i=1 2ǫi
 226 Cooperative Control of Multi-Agent Systems

where ẽi = ei − bi , d˜i = di − β, i = 1, · · · , N , and β = maxi=1,··· ,N {ai , λτ2 },

and P > 0 and τ satisfy (9.5).
The following result provides a sufficient condition to design the consensus
protocol (9.49).
Theorem 58 Suppose that G is connected and Assumption 9.3 holds. Then,
both the consensus error ξ and the adaptive gains di and ei , i = 1, · · · , N ,
in (9.51) are uniformly ultimately bounded, under the distributed adaptive
protocol (9.49) with K = −B T P −1 and Γ = P −1 BB T P −1 , where P > 0 is a
solution to the LMI (9.5). Moreover, the following statements hold.
i) For any ϕi and ψi , the parameters ξ, d˜i , and ẽi exponentially converge to
the residual set
N
1 X 2 1
D2 , {ξ, d˜i , ẽi : V6 < (β ϕi + b2i ψi + κi )}, (9.54)
2δ i=1 2

where δ , mini=1,··· ,N {α, ϕi νi , ψi ǫi } and α is defined as in (9.35).

ii) If ϕi and ψi satisfy ̺ , mini=1,··· ,N {ϕi νi , ψi ǫi } < α, then in addition to i),

ξ exponentially converges to the residual set
N
λmax (P ) X 2 1
D3 , {ξ : kξk2 ≤ (β ϕi + b2i ψi + κi )}. (9.55)
λ2 (α − ̺) i=1 2

Proof 60 Let V6 in (9.53) be the Lyapunov function candidate. The time

derivative of V6 along (9.51) can be obtained as

XN ˜ XN
di ˜˙ ẽi ˙
V̇6 = ξ T (L ⊗ P −1 )ξ˙ + di + ẽi
ν
i=1 i
ǫ
i=1 i
= ξ T [(L ⊗ P −1 A + LDL ⊗ P −1 BK)ξ + (L ⊗ P −1 )]G(x)
+ ξ T (L ⊗ P −1 B)[F (x, t) + R̄(ξ)]
N N N (9.56)
X X X
+ d˜i [−ϕi (d˜i + β) + ( Lij ξjT )Γ( Lij ξj )
i=1 j=1 j=1
N
X N
X N
X
+ kK Lij ξj k] + ẽi [−ψi (ẽi + bi ) + kK Lij ξj kkxi k],
j=1 i=1 j=1

where D = diag(d˜1 + β, · · · , c̃N + β).

By noting that K = −BP −1 , it is easy to get that
ξ T (LDL ⊗ P −1 BK)ξ
N
X N
X N
X (9.57)
=− (d˜i + β)( Lij ξj )T P −1 BB T P −1 ( Lij ξj ).
i=1 j=1 j=1
Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 227

In light of Assumption 9.3, we can obtain that

N
X N
X
ξ T (L ⊗ P −1 B)F (x, t) ≤ (ai + bi kxi k)kB T P −1 Lij ξj k. (9.58)
j=1 j=1

In what follows, wePconsider three cases.

N
i) (di + ei kxi k)kK j=1 Lij ξj k > κi , i = 1, · · · , N .
In this case, we can get from (9.50) and (9.33) that
N
X N
X
ξ T (L ⊗ P −1 B)R̄(ξ) = − [di + ei kxi k]kB T P −1 Lij ξj k. (9.59)
i=1 j=1

Substituting (9.57), (9.58), and (9.59) into (9.56) yields

1 T
V̇6 ≤ ξ [Yξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))]
2
XN N
X
− (β − ai )kB T P −1 Lij ξj k
i=1 j=1
N
X N
1 1X 2
− (ϕi d˜2i + ψi ẽ2i ) + (β ϕi + b2i ψi )
2 i=1
2 i=1
1
≤ ξ T [Yξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))]
2
N N
1X 1X 2
− (ϕi d˜2i + ψi ẽ2i ) + (β ϕi + b2i ψi ),
2 i=1 2 i=1

where Y , L ⊗ (P −1 A + AT P −1 ) − 2βL2 ⊗ P −1 BB T P −1 and we have used

the facts that β ≥ maxi=1,··· ,N ai and −d˜2i − d˜i β ≤ − 21 d˜2i + 21 β 2 .
PN
ii) (di + ei kxi k)kK j=1 Lij ξj k ≤ κi , i = 1, · · · , N .
In this case, we can get from (9.50) and (9.33) that
N
X N
X
(di + ei kxi k)2
ξ T (L ⊗ P −1 B)R̄(ξ) = − kB T P −1 Lij ξj k2 . (9.60)
i=1
κi j=1

Then, it follows from (9.57), (9.58), (9.60), and (9.56) that

1 T
V̇6 ≤ ξ [Yξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))]
2
N N (9.61)
1X 1X 2 1
− (ϕi d˜2i + ψi ẽ2i ) + (β ϕi + b2i ψi + κi ),
2 i=1 2 i=1 2

2 PN
where we have used the fact that − (di +eκi kx
i
i k)
kB T P −1 j=1 Lij ξj k2 + (di +
228 Cooperative Control of Multi-Agent Systems
PN PN
ei kxi k)kB T P −1 j=1 Lij ξj k ≤ 41 κi , for (di + ei kxi k)kK j=1 Lij ξj k ≤ κi ,
i = 1, · · · , N . PN
iii) (di + ei kxi k)kK j=1 Lij ξj k > κi , i = 1, · · · , l, and (di +
PN
ei kxi k)kK j=1 Lij ξj k ≤ κi , i = l + 1, · · · , N , where 2 ≤ l ≤ N − 1.
By following similar steps in the two cases above, it is not difficult to get
that
N −l
1 1X
V̇6 ≤ ξ T [Yξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))] + κi
2 4 i=1
N N
1X 1X 2
− (ϕi d˜2i + ψi ẽ2i ) + (β ϕi + b2i ψi ).
2 i=1 2 i=1

Therefore, based on the above three cases, we can get that V̇6 satisfies (9.61)
for all ξ ∈ RN n . Note that (9.61) can be rewritten into
1
V̇6 ≤ −δV6 + δV6 + ξ T [Yξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))]
2
N N
1X 1X 2 1
− (ϕi d˜2i + ψi ẽ2i ) + (β ϕi + b2i ψi + κi )
2 i=1 2 i=1 2
1
= −δV6 + ξ T [Yξ + δ(L ⊗ P −1 )ξ + 2(L ⊗ P −1 ) (9.62)
2
N
1X δ
× (G(x) − 1 ⊗ g(x̄))] − [(ϕi − )d˜2i
2 i=1 νi
N
δ 1X 2 1
+ (ψi − )ẽ2i )] + (β ϕi + b2i ψi + κi ).
ǫi 2 i=1 2

Because βλ2 ≥ 1 and 0 < δ ≤ α, by following similar steps in the proof of

Theorem 57, we can show that

ξ T [Yξ + δ(L ⊗ P −1 )ξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))] ≤ 0.

Further, by noting that δ ≤ mini=1,··· ,N {ϕi νi , ψi ǫi }, it follows from (9.62) that

N
1X 2 1
V̇6 ≤ −δV6 + (β ϕi + b2i ψi + κi ), (9.63)
2 i=1 2

which implies that

N
1 X 2 1
V6 ≤ [V6 (0) − (β ϕi + b2i ψi + κi )]e−δt
2δ i=1 2
(9.64)
N
1 X 2 1
+ (β ϕi + b2i ψi + κi ).
2δ i=1 2
Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 229

Therefore, V6 exponentially converges to the residual set D2 in (9.54) with

λ2
a convergence rate faster than e−δt , which, in light of V6 ≥ 2λmax 2
(P ) kξk +
PN d˜2i PN ẽ2i
i=1 2νi + i=1 2ǫi , implies that ξ, di , and ei are uniformly ultimately bound-
ed.
Next, if ̺ , mini=1,··· ,N {ϕi νi , ψi ǫi } < α, we can obtain a smaller residual
set for ξ by rewriting (9.62) into
1
V̇6 ≤ −̺V6 + ξ T [Yξ + α(L ⊗ P −1 )ξ + 2(L ⊗ P −1 )(G(x) − 1 ⊗ g(x̄))]
2
N
α−̺ T 1X 2 1
− ξ (L ⊗ P −1 )ξ + (β ϕi + b2i ψi + κi )
2 2 i=1 2
N
λ2 (α − ̺) 1X 2 1
≤ −̺V6 − kξk2 + (β ϕi + b2i ψi + κi ).
2λmax (P ) 2 i=1 2
(9.65)
(P ) PN
Obviously, it follows from (9.65) that V̇6 ≤ −̺V6 if kξk2 > λλ2max
(α−̺) i=1 (β 2
ϕi +
2 1
bi ψi + 2 κi ). Then, we can get that if ̺ ≤ α then ξ exponentially converges to
the residual set D3 in (9.55) with a convergence rate faster than e−̺t .

Remark 74 In Theorem 58, the design of the adaptive protocol (9.49) relies
on only the agent dynamics, requiring neither the minimal nonzero eigenvalue
of L nor the upper bounds of of the uncertainties fi (xi , t). Thus, the adaptive
controller (9.49), contrary to the static protocol (9.30), can be computed imple-
mented in a fully distributed fashion without requiring any global information.
As explained in Section 8.1 of Chapter 8, we should choose reasonably small
ϕi , ψi , and κi in practical implementations such that the consensus error ξ
is satisfactorily small and at the same time the adaptive gains di and ei are
tolerable.

Remark 75 A special case of the uncertainties fi (xi , t) satisfying Assump-

tions 9.2 and 9.3 is that there exist positive constants di such that kfi (xi , t)k ≤
di , i = 1, · · · , N . For this case, the proposed protocols (9.30) and (9.49) can be
accordingly simplified. The adaptive protocol (9.49) in this case can be modified
into
N
X N
X
ui = d¯i K aij (xi − xj ) + r̂i (K aij (xi − xj )),
j=1 j=1
N
X
d¯˙i = τi [−ϕi d¯i + ( aij (xi − xj )T )Γ (9.66)
j=1
N
X N
X
×( aij (xi − xj )) + kK aij (xi − xj )k],
j=1 j=1
230 Cooperative Control of Multi-Agent Systems

where the nonlinear functions r̂i (·) are defined such that r̂i (w) =
(
wd¯i
kwk if d¯i kwk > κi
¯2 and the rest of the variables are defined as in (9.49).
if d¯i kwk ≤ κi
w di
κ

Remark 76 The case where there exists a leader whose control input is
bounded and unknown to any follower can be also investigated. Distributed
controllers based on the relative state information can be similarly designed by
following the steps in Section 8.3 of Chapter 8 and in this section, when the
communication graph contains a directed spanning tree with the leader as the
root and the subgraph associated with the followers is undirected. The details
are omitted here for conciseness.

9.2.3 Adaptive Protocols for the Case without Uncertainties

Theorem 58 shows that the distributed adaptive protocol (9.49) can solve the
consensus problem for the agents in (9.29) subject to matching uncertainties
fi (xi , t). For the special case where the agents do not contain any uncertainty,
i.e., fi (xi , t) = 0, i = 1, · · · , N , the adaptive protocol (9.49) can be simplified
into the following form:
N
X
ui = di K aij (xi − xj ),
j=1
(9.67)
N
X N
X
d˙i = νi [ aij (xi − xj )T ]Γ[ aij (xi − xj )], i = 1, · · · , N.
j=1 j=1

Accordingly, Theorem 58 can be reduced as follows.

Theorem 59 Suppose that G is connected. Then, for the agents in (9.29)

with fi (xi , t) = 0 under the adaptive protocol (9.66) with K and Γ chosen as
in Theorem 58, the consensus error ξ asymptotically converges to zero and the
adaptive gains di , i = 1, · · · , N , converges to some finite steady-state values.

The proof of Theorem 59 can be completed by following the steps in the

proof of Theorem 59. The protocol (9.66) is actually a node-based adaptive
protocol, using the terms in Chapter 5. Similarly, distributed edge-based adap-
tive protocol can be constructed and the consensus results can be accordingly
derived. The details are omitted here.
Consider the case where the agents in (9.29) with fi (xi , t) = 0 but subject
to external disturbances. The dynamics of the agents are described by

ẋi = Axi + Bui + g(xi ) + ωi , i = 1, · · · , N, (9.68)

where ωi ∈ Rn denotes bounded external disturbances. By the analysis in

Section 4.3, we know that the distributed adaptive protocol (9.66) can cause
Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 231

parameter drift phenomenon (specifically, the adaptive gains di , i = 1, · · · , N ,

grows unbounded), i.e., it is not robust in the presence of external distur-
bances. The σ-modification technique can be utilized to present a distributed
robust adaptive protocol, described by
N
X
ui = di K aij (xi − xj ),
j=1
N
X N
X
d˙i = νi [−ϕi di + ( aij (xi − xj )T )Γ( aij (xi − xj ))], i = 1, · · · , N.
j=1 j=1
(9.69)
Following the steps in the proof of Theorem 58, it is not difficult to show that
the adaptive protocol (9.69) can guarantee the ultimate boundedness of the
consensus error ξ and adaptive gains di , i = 1, · · · , N , i.e., (9.69) is robust
with respect to bounded disturbances.

9.2.4 Simulation Examples

In this subsection, a simulation example is provided for illustration.

1 6

2 5

3 4

FIGURE 9.4: A leaderless communication graph.

Example 26
 xConsider
 a network of uncertain agents described by (9.29),
i1 0 1 0 0
xi2 −48.6 −1.26 48.6 0 T
with xi = xi3 , A = 0 0 0 10 , B = [ 0 21.6 0 0 ] , g(xi ) =
xi4 1.95 0 −1.95 0
T
[ 0 0 0 −0.333sin(xi1 ) ]
[129]. Clearly, g(xi ) here satisfies (9.2) with a Lipschitz
constant α = 0.333. For illustration, the heterogeneous uncertainties are
chosen as f1 = sin(t), f2 = sin(x25 )/2, f3 = sin(t/2), f4 = cos(t) + 1,
f5 = sin(t + 1) and f6 = sin(x63 − 4)/2.
Here we use the adaptive protocol (9.66) to solve the global consensus prob-
lem. The communication graph is given as in FIGURE 9.4, which is connect-
ed. Solving the LMI (9.5) gives the feedback gain matrices of (9.66) as K =
232 Cooperative Control of Multi-Agent Systems
20

10

−10

xi−x1
−20

−30

−40

−50

−60
0 2 4 6 8 10
t

FIGURE 9.5: The consensus error xi − x1 , i = 2, · · · , 6, under (9.66).

 7.9464 0.4979 −5.5061 15.5372 

0.4979 0.0312 −0.3450 0.9735
− [ −2.8189 −0.1766 1.9532 −5.5117 ] and Γ = −5.5061 −0.3450 3.8151 −10.7657 . S-
15.5372 0.9735 −10.7657 30.3791
elect κi = 0.5, ϕi = 0.05, and τi = 1, i = 1, · · · , 6, in (9.66). The consensus
error xi − x1 , i = 2, · · · , 5, under (9.66) designed as above are depicted in
FIGURE 9.5, implying that consensus is achieved. The adaptive gains d¯i in
(9.66) are shown in FIGURE 9.6, which are bounded.

4
di

−1
0 2 4 6 8 10
t

FIGURE 9.6: The adaptive gains d¯i in (9.66).

9.3 Notes
The materials of this section are mainly based on [90, 96]. For more results on
consensus of Lipschitz-type nonlinear multi-agent systems, please refer to [60,
Global Consensus of Lipschitz Nonlinear Multi-Agent Systems 233

84, 156, 181, 189]. In [181], the agents are characterized by first-order Lipschitz
nonlinear systems. In [156, 189], the agents were assumed to be second-order
systems with Lipschitz nonlinearity. [84] studied the global leader-follower
consensus of coupled Lur’e systems with certain sector-bound nonlinearity,
where the subgraph associated with the followers is required to be undirected.
[60] was concerned with the second-order consensus problem of multi-agent
systems with a virtual leader, where all agents and the virtual leader share the
same intrinsic dynamics with a locally Lipschitz condition. Consensus tracking
for multi-agent systems with Lipschitz-type nonlinear dynamics and switching
directed topology was studied in [182].
There exists a quite large body of research papers on cooperative control
of other types of nonlinear multi-agent systems. For instance, consensus of
a network of Euler-Lagrange systems was studied in [1, 25, 79, 109, 176], a
passivity-based design framework was proposed to deal with the consensus
problem and other group coordination problems in [5, 8], and cooperative
adaptive output regulation problems were addressed in [37, 163] for several
classes of nonlinear multi-agent systems.
This page intentionally left blank
Bibliography

[1] A. Abdessameud and A. Tayebi. Attitude synchronization of a group

of spacecraft without velocity measurements. IEEE Transactions on
Automatic Control, 54(11):2642–2648, 2009.
[2] R. Agaev and P. Chebotarev. On the spectra of nonsymmetric Laplacian
matrices. Linear Algebra and its Applications, 399(1):157–178, 2005.
[3] G. Alefeld and N. Schneider. On square roots of M-matrices. Linear
Algebra and its Applications, 42:119–132, 1982.
[4] G. Antonelli. Interconnected dynamic systems: An overview on dis-
tributed control. IEEE Control Systems Magazine, 33(1):76–88, 2013.
[5] M. Arcak. Passivity as a design tool for group coordination. IEEE
Transactions on Automatic Control, 52(8):1380–1390, 2007.
[6] M. Aung, A. Ahmed, M. Wette, D. Scharf, J. Tien, G. Purcell,
M. Regehr, and B. Landin. An overview of formation flying technol-
ogy development for the terrestrial planet finder mission. Proceedings of
the 2004 Aerospace Conference, pp. 2667–2679, 2004.
[7] H. Bai, M. Arcak, and J.T. Wen. Rigid body attitude coordination
without inertial frame information. Automatica, 44(12):3170–3175, 2008.
[8] H. Bai, M. Arcak, and J.T. Wen. Cooperative Control Design: A System-
atic, Passivity-based Approach. Springer-Verlag, New York, NY, 2011.
[9] R. W Beard, J. Lawton, F.Y Hadaegh, A coordination architecture
for spacecraft formation control. IEEE Transactions on control systems
technology, 9(6):777–790, 2001.
[10] A. Berman and R.J. Plemmons. Nonnegative Matrices in the Mathe-
matical Sciences. Academic Press, New York, NY, 1979.
[11] D.S. Bernstein. Matrix Mathematics: Theory, Facts, and Formulas.
Princeton University Press, Princeton, NJ, 2009.
[12] A. Bidram, A. Davoudi, F.L. Lewis, and Z.H. Qu. Secondary control
of microgrids based on distributed cooperative control of multi-agent
systems. IET Generation, Transmission and Distribution, 7(8):822–831,
2013.

235
236 Cooperative Control of Multi-Agent Systems

[13] V.S. Bokharaie, O. Mason, and M. Verwoerd. D-stability and delay-

independent stability of homogeneous cooperative systems. IEEE Trans-
actions on Automatic Control, 55(12):2882–2885, 2010.
[14] V. Borkar and P. Varaiya. Asymptotic agreement in distributed estima-
tion. IEEE Transactions on Automatic Control, 27(3):650–655, 1982.
[15] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix
Inequalities in System and Control Theory. SIAM, Philadelphia, PA,
1994.
[16] Y.C. Cao and W. Ren. Containment control with multiple stationary or
dynamic leaders under a directed interaction graph. Proceedings of the
48th IEEE Conference on Decision and Control and the 28th Chinese
Control Conference, pp. 3014–3019, 2009.

[17] Y.C. Cao and W. Ren. Sampled-data discrete-time coordination algo-

rithms for double-integrator dynamics under dynamic directed interac-
tion. International Journal of Control, 83(3):506–515, 2010.
[18] Y.C. Cao, W. Ren, and M. Egerstedt. Distributed containment con-
trol with multiple stationary or dynamic leaders in fixed and switching
directed networks. Automatica, 48(8):1586–1597, 2012.
[19] Y.C. Cao, D. Stuart, W. Ren, and Z.Y. Meng. Distributed contain-
ment control for multiple autonomous vehicles with double-integrator
dynamics: Algorithms and experiments. IEEE Transactions on Control
Systems Technology, 19(4):929–938, 2011.
[20] Y.C. Cao, W.W. Yu, W. Ren, and G.R. Chen. An overview of recent
progress in the study of distributed multi-agent coordination. IEEE
Transactions on Industrial Informatics, 9(1):427–438, 2013.
[21] Y.C. Cao and W. Ren. Distributed coordinated tracking with reduced
interaction via a variable structure approach. IEEE Transactions on
Automatic Control, 57(1):33–48, 2012.
[22] R. Carli, F. Bullo, and S. Zampieri. Quantized average consensus via
dynamic coding/decoding schemes. International Journal of Robust and
Nonlinear Control, 20(2):156–175, 2009.

[23] C.T. Chen. Linear System Theory and Design. Oxford University Press,
New York, NY, 1999.
[24] G. Chen and F.L. Lewis. Distributed adaptive tracking control for
synchronization of unknown networked lagrangian systems. IEEE
Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics,
41(3):805–816, 2011.
Bibliography 237

[25] S.J. Chung, U. Ahsun, and J.J.E. Slotine. Application of synchroniza-

tion to formation flying spacecraft: Lagrangian approach. Journal of
Guidance, Control and Dynamics, 32(2):512–526, 2009.
[26] M. Corless and G. Leitmann. Continuous state feedback guaranteeing
uniform ultimate boundedness for uncertain dynamic systems. IEEE
Transactions on Automatic Control, 26(5):1139–1144, 1981.
[27] J. Cortes, S. Martinez, T. Karatas, and F. Bullo. Coverage control for
mobile sensing networks. IEEE Transactions on Robotics and Automa-
tion, 20(2):243–255, 2004.
[28] F. Cucker and J.G. Dong. Avoiding collisions in flocks. IEEE Transac-
tions on Automatic Control, 55(5):1238–1243, 2010.
[29] F. Cucker and S. Smale. Emergent behavior in flocks. IEEE Transac-
tions on Automatic Control, 52(5):852–862, 2007.
[30] A. Das and F.L. Lewis. Distributed adaptive control for synchronization
of unknown nonlinear networked systems. Automatica, 46(12):2014–
2021, 2010.
[31] A. Das and F.L. Lewis. Cooperative adaptive control for synchroniza-
tion of second-order systems with unknown nonlinearities. International
Journal of Robust and Nonlinear Control, 21(13): 1509-1524, 2011.
[32] P. DeLellis, M. diBernardo, and F. Garofalo. Novel decentralized adap-
tive strategies for the synchronization of complex networks. Automatica,
45(5):1312–1318, 2009.
[33] P. DeLellis, M.D. diBernardo, T.E. Gorochowski, and G. Russo. Syn-
chronization and control of complex networks via contraction, adapta-
tion and evolution. IEEE Circuits and Systems Magazine, 10(3):64–82,
2010.
[34] D.V. Dimarogonas, E. Frazzoli, and K.H. Johansson. Distributed event-
triggered control for multi-agent systems. IEEE Transactions on Auto-
matic Control, 57(5):1291–1297, 2012.
[35] D.V. Dimarogonas and K.J. Kyriakopoulos. Inverse agreement protocols
with application to distributed multi-agent dispersion. IEEE Transac-
tions on Automatic Control, 54(3):657–663, 2009.
[36] D.V. Dimarogonas, P. Tsiotras, and K.J. Kyriakopoulos. Leader–
follower cooperative attitude control of multiple rigid bodies. Systems
and Control Letters, 58(6):429–435, 2009.
[37] Z.T. Ding. Consensus output regulation of a class of heterogeneous non-
linear systems. IEEE Transactions on Automatic Control, 58(10):2648–
2653, 2013.
238 Cooperative Control of Multi-Agent Systems

[38] K.D. Do. Formation tracking control of unicycle-type mobile robots

with limited sensing ranges. IEEE Transactions on Control Systems
Technology, 16(3):527–538, 2008.
[39] W.J. Dong. Flocking of multiple mobile robots based on backstep-
ping. IEEE Transactions on Systems, Man, and Cybernetics, Part B:
Cybernetics, 41(2):414–424, 2011.

[40] W.J. Dong and J.A. Farrell. Cooperative control of multiple non-
holonomic mobile agents. IEEE Transactions on Automatic Control,
53(6):1434–1448, 2008.
[41] Z.S. Duan, G.R. Chen, and L. Huang. Disconnected synchronized re-
gions of complex dynamical networks. IEEE Transactions on Automatic
Control, 54(4):845–849, 2009.

[42] Z.S. Duan, G.R. Chen, and L. Huang. Synchronization of weight-

ed networks and complex synchronized regions. Physics Letters A,
372(21):3741–3751, 2008.
[43] C. Edwards and S.K. Spurgeon. Sliding mode control: Theory and ap-
plications. Taylor & Francis, London, 1998.
[44] J.A. Fax and R.M. Murray. Information flow and cooperative con-
trol of vehicle formations. IEEE Transactions on Automatic Control,
49(9):1465–1476, 2004.
[45] B. Fidan, J.M. Hendrickx, and B.D.O. Anderson. Closing ranks in rigid
multi-agent formations using edge contraction. International Journal of
Robust and Nonlinear Control, 20(18):2077–2092, 2010.
[46] L. Galbusera, G. Ferrari-Trecate, and R. Scattolini. A hybrid model
predictive control scheme for containment and distributed sensing in
multi-agent systems. Systems and Control Letters, 62(5):413–419, 2013.

[47] Y. Gao, L. Wang, G. Xie, and B. Wu. Consensus of multi-agent sys-

tems based on sampled-data control. International Journal of Control,
82(12):2193–2205, 2009.

[48] C. Godsil and G. Royle. Algebraic Graph Theory. Springer-Verlag, New

York, NY, 2001.
[49] B. Grocholsky, J. Keller, V. Kumar, and G.J. Pappas. Cooperative
air and ground surveillance. IEEE Robotics and Automation Magazine,
13(3):16–25, 2006.
[50] G.X. Gu, L. Marinovici, and F.L. Lewis. Consensusability of discrete-
time dynamic multiagent systems. IEEE Transactions on Automatic
Control, 57(8):2085–2089, 2012.
Bibliography 239

[51] V. Gupta, B. Hassibi, and R.M. Murray. A sub-optimal algorithm to

synthesize control laws for a network of dynamic agents. International
Journal of Control, 78(16):1302–1313, 2005.
[52] S.Y. Ha, T. Ha, and J.H. Kim. Emergent behavior of a cucker-smale type
particle model with nonlinear velocity couplings. IEEE Transactions on
Automatic Control, 55(7):1679–1683, 2010.

[53] V. Hahn and P.C. Parks. Stability Theory. Prentice-Hall, Englewood

Cliffs, NJ, 1993.
[54] J.M. Hendrickx, B.D.O. Anderson, J. Delvenne, and V.D. Blondel. Di-
rected graphs for the analysis of rigidity and persistence in autonomous
agent systems. International Journal of Robust and Nonlinear Control,
17(10-11):960–981, 2007.

[55] K. Hengster-Movric, K.Y. You, F.L. Lewis, and L.H. Xie. Synchroniza-
tion of discrete-time multi-agent systems on graphs using Riccati design.
Automatica, 49(2):414–423, 2013.
[56] Y.G. Hong, J.P. Hu, and L. Gao. Tracking control for multi-agent
consensus with an active leader and variable topology. Automatica,
42(7):1177–1182, 2006.
[57] Y.G. Hong, G.R. Chen, and L. Bushnell. Distributed observers design for
leader-following control of multi-agent networks. Automatica, 44(3):846–
850, 2008.

[58] R.A. Horn and C.R. Johnson. Matrix Analysis. Cambridge University
Press, New York, NY, 1990.
[59] A. Howard, M.J. Matarić, and G.S. Sukhatme. Mobile sensor network
deployment using potential fields: A distributed, scalable solution to the
area coverage problem. Proceedings of the 6th International Symposium
on Distributed Autonomous Robotics Systems, pp. 299–308, 2002.
[60] Y.B. Hu, H.S. Su, and J. Lam. Adaptive consensus with a virtual leader
of multiple agents governed by locally Lipschitz nonlinearity. Interna-
tional Journal of Robust and Nonlinear Control, 23(9):978–990, 2013.

[61] I.I. Hussein and D.M. Stipanovic. Effective coverage control for mobile
sensor networks with guaranteed collision avoidance. IEEE Transactions
on Control Systems Technology, 15(4):642–657, 2007.
[62] P. A. Ioannou and J. Sun. Robust Adaptive Control. Prentice-Hall, New
York, NY, 1996.

[63] P.A. Ioannou and P.V. Kokotovic. Instability analysis and improvement
of robustness of adaptive control. Automatica, 20(5):583–594, 1984.
240 Cooperative Control of Multi-Agent Systems

[64] T. Iwasaki and R.E. Skelton. All controllers for the general H∞ control
problem: LMI existence conditions and state space formulas. Automat-
ica, 30(8):1307–1317, 1994.
[65] A. Jadbabaie, J. Lin, and A.S. Morse. Coordination of groups of mobile
autonomous agents using nearest neighbor rules. IEEE Transactions on
Automatic Control, 48(6):988–1001, 2003.
[66] I.S. Jeon, J.I. Lee, and M.J. Tahk. Homing guidance law for cooper-
ative attack of multiple missiles. Journal of Guidance, Control, and
Dynamics, 33(1):275–280, 2010.
[67] M. Ji, G. Ferrari-Trecate, M. Egerstedt, and A. Buffa. Containment
control in mobile networks. IEEE Transactions on Automatic Control,
53(8):1972–1975, 2008.
[68] F.C. Jiang and L. Wang. Consensus seeking of high-order dynamic
multi-agent systems with fixed and switching topologies. International
Journal of Control, 85(2):404–420, 2010.
[69] V. Kapila, A.G. Sparks, J.M. Buffington, and Q. Yan. Spacecraft forma-
tion flying: Dynamics and control. Journal of Guidance, Control, and
Dynamics, 23(3):561–564, 2000.
[70] O. Katsuhiko. Modern Control Engineering. Prentice Hall, Upper Saddle
River, NJ, 1996.
[71] H.K. Khalil. Nonlinear Systems. Prentice Hall, Englewood Cliffs, NJ,
2002.
[72] M. Krstić, I. Kanellakopoulos, and P.V. Kokotovic. Nonlinear and Adap-
tive Control Design. John Wiley & Sons, New York, 1995.
[73] J. Larson, C. Kammer, K.Y. Liang, and K.H. Johansson. Coordinated
route optimization for heavy-duty vehicle platoons. Proceedings of the
16th International IEEE Annual Conference on Intelligent Transporta-
tion Systems, pp. 1196–1202, 2013.
[74] F.L. Lewis, H.W. Zhang, K. Hengster-Movric, and A. Das. Coopera-
tive Control of Multi-Agent Systems: Optimal and Adaptive Design Ap-
proaches. Springer-Verlag, London, 2014.
[75] T. Li, M.Y. Fu, L.H. Xie, and J.F. Zhang. Distributed consensus with
limited communication data rate. IEEE Transactions on Automatic
Control, 56(2):279–292, 2011.
[76] W. Li and C.G. Cassandras. Distributed cooperative coverage control of
sensor networks. Proceedings of the 44th IEEE Conference on Decision
and Control and 2005 European Control Conference, pp. 2542–2547,
2005.
Bibliography 241

[77] Z.K. Li, Z.S. Duan, and F.L. Lewis. Distributed robust consensus con-
trol of multi-agent systems with heterogeneous matching uncertainties.
Automatica, 50(3):883–889, 2014.
[78] Z.K. Li, G.H. Wen, Z.S. Duan, and W. Ren. Designing fully distributed
consensus protocols for linear multi-agent systems with directed com-
munication graphs. IEEE Transactions on Automatic Control, in press,
2014.

[79] Z.K. Li and Z.S. Duan. Distributed adaptive attitude synchronization of

multiple spacecraft. Science China Technological Sciences, 54(8):1992–
1998, 2011.
[80] Z.K. Li and Z.S. Duan. Distributed adaptive consensus protocols for lin-
ear multi-agent systems with directed graphs in the presence of external
disturbances. Proceedings of the 33th Chinese Control Conference, pp.
1632–1637, 2014.
[81] Z.K. Li, Z.S. Duan, and G.R. Chen. H∞ consensus regions of multi-
agent systems. Proceedings of the 29th Chinese Control Conference, pp.
4601–4606, 2010.

[82] Z.K. Li, Z.S. Duan, and G.R. Chen. Consensus of discrete-time linear
multi-agent systems with observer-type protocols. Discrete and Contin-
uous Dynamical Systems-Series B, 16(2):489–505, 2011.
[83] Z.K. Li, Z.S. Duan, and G.R. Chen. Dynamic consensus of linear multi-
agent systems. IET Control Theory and Applications, 5(1):19–28, 2011.
[84] Z.K. Li, Z.S. Duan, and G.R. Chen. Global synchronised regions
of linearly coupled Lur’e systems. International Journal of Control,
84(2):216–227, 2011.
[85] Z.K. Li, Z.S. Duan, and G.R. Chen. On H∞ and H2 performance regions
of multi-agent systems. Automatica, 47(4):797–803, 2011.
[86] Z.K. Li, Z.S. Duan, G.R. Chen, and L. Huang. Consensus of multia-
gent systems and synchronization of complex networks: A unified view-
point. IEEE Transactions on Circuits and Systems I: Regular Papers,
57(1):213–224, 2010.

[87] Z.K. Li, Z.S. Duan, and L. Huang. H∞ control of networked multi-agent
systems. Journal of Systems Science and Complexity, 22(1):35–48, 2009.
[88] Z.K. Li, Z.S. Duan, W. Ren, and G. Feng. Containment control of
linear multi-agent systems with multiple leaders of bounded inputs using
distributed continuous controllers. International Journal of Robust and
Nonlinear Control, in press, 2014.
242 Cooperative Control of Multi-Agent Systems

[89] Z.K. Li, Z.S. Duan, L.H. Xie, and X.D. Liu. Distributed robust control of
linear multi-agent systems with parameter uncertainties. International
Journal of Control, 85(8):1039–1050, 2012.
[90] Z.K. Li, X.D. Liu, M.Y. Fu, and L.H. Xie. Global H∞ consensus of
multi-agent systems with Lipschitz non-linear dynamics. IET Control
Theory and Applications, 6(13):2041–2048, 2012.

[91] Z.K. Li, X.D. Liu, P. Lin, and W. Ren. Consensus of linear multi-
agent systems with reduced-order observer-based protocols. Systems
and Control Letters, 60(7):510–516, 2011.
[92] Z.K. Li, X.D. Liu, W. Ren, and L.H. Xie. Distributed tracking control
for linear multi-agent systems with a leader of bounded unknown input.
IEEE Transactions on Automatic Control, 58(2):518–523, 2013.

[93] Z.K. Li, W. Ren, X.D. Liu, and M.Y. Fu. Consensus of multi-agent
systems with general linear and Lipschitz nonlinear dynamics using dis-
tributed adaptive protocols. IEEE Transactions on Automatic Control,
58(7):1786–1791, 2013.

[94] Z.K. Li, W. Ren, X.D. Liu, and M.Y. Fu. Distributed containment con-
trol of multi-agent systems with general linear dynamics in the presence
of multiple leaders. International Journal of Robust and Nonlinear Con-
trol, 23(5):534–547, 2013.
[95] Z.K. Li, W. Ren, X.D. Liu, and L.H. Xie. Distributed consensus of lin-
ear multi-agent systems with adaptive dynamic protocols. Automatica,
49(7):1986–1995, 2013.
[96] Z.K. Li, Y. Zhao, and Z.S. Duan. Distributed robust global consen-
sus of a class of Lipschitz nonlinear multi-agent systems with matching
uncertainties. Asian Journal of Control, in press, 2014.

[97] P. Lin and Y.M. Jia. Consensus of second-order discrete-time multi-

agent systems with nonuniform time-delays and dynamically changing
topologies. Automatica, 45(9):2154–2158, 2009.
[98] P. Lin and Y.M. Jia. Robust H∞ consensus analysis of a class of second-
order multi-agent systems with uncertainty. IET Control Theory and
Applications, 4(3):487–498, 2010.
[99] P. Lin and Y.M. Jia. Distributed robust H∞ consensus control in di-
rected networks of agents with time-delay. Systems and Control Letters,
57(8):643–653, 2008.

[100] P. Lin, K.Y. Qin, Z.K. Li, and W. Ren. Collective rotating motions of
second-order multi-agent systems in three-dimensional space. Systems
and Control Letters, 60(6):365–372, 2011.
Bibliography 243

[101] C. Liu, Z.S. Duan, G.R. Chen, and L. Huang. Analyzing and controlling
the network synchronization regions. Physica A, 386(1):531–542, 2007.

[102] Y. Liu and Y.M. Jia. H∞ consensus control of multi-agent systems with
switching topology: A dynamic output feedback protocol. International
Journal of Control, 83(3):527–537, 2010.
[103] J. Löfberg. YALMIP: A toolbox for modeling and optimization in MAT-
LAB. Proceedings of 2004 IEEE International Symposium on Computer
Aided Control Systems Design, pp. 284–289, 2004.
[104] Y. Lou and Y.G. Hong. Target containment control of multi-agent sys-
tems with random switching interconnection topologies. Automatica,
48(5):879–885, 2012.

[105] J.H. Lv, X.H. Yu, G.R. Chen, and D.Z. Cheng. Characterizing the syn-
chronizability of small-world dynamical networks. IEEE Transactions
on Circuits and Systems I: Regular Papers, 51(4):787–796, 2004.
[106] C.Q. Ma and J.F. Zhang. Necessary and sufficient conditions for con-
sensusability of linear multi-sgent systems. IEEE Transactions on Au-
tomatic Control, 55(5):1263–1268, 2010.
[107] R.N. Madan. Chua’s Circuit: A Paradigm for Chaos. World Scientific,
Singapore, 1993.
[108] P. Massioni and M. Verhaegen. Distributed control for identical dynam-
ically coupled systems: A decomposition approach. IEEE Transactions
on Automatic Control, 54(1):124–135, 2009.
[109] J. Mei, W. Ren, and G. Ma. Distributed coordinated tracking with a
dynamic leader for multiple Euler-Lagrange systems. IEEE Transactions
on Automatic Control, 56(6):1415–1421, 2011.

[110] J. Mei, W. Ren, and G. Ma. Distributed containment control for La-
grangian networks with parametric uncertainties under a directed graph.
Automatica, 48(4):653–659, 2012.
[111] Z.Y. Meng, W. Ren, and Z. You. Distributed finite-time attitude con-
tainment control for multiple rigid bodies. Automatica, 46(12):2092–
2099, 2010.
[112] M. Mesbahi and M. Egerstedt. Graph Theoretic Methods in Multiagent
Networks. Princeton University Press, Princeton, NJ, 2010.
[113] N. Michael, J. Fink, and V. Kumar. Cooperative manipulation and
transportation with aerial robots. Autonomous Robots, 30(1):73–86,
2011.
244 Cooperative Control of Multi-Agent Systems

[114] U. Munz, A. Papachristodoulou, and F. Allgower. Robust consensus con-

troller design for nonlinear relative degree two multi-agent systems with
communication constraints. IEEE Transactions on Automatic Control,
56(1):145–151, 2011.
[115] R.M. Murray. Recent research in cooperative control of multivehi-
cle systems. Journal of Dynamic Systems, Measurement, and Control,
129:571–583, 2007.
[116] W. Ni and D.Z. Cheng. Leader-following consensus of multi-agent sys-
tems under fixed and switching topologies. Systems and Control Letters,
59(3-4):209–217, 2010.
[117] K. Ogata. Modern Control Engineering. Prentice Hall, Englewood Cliffs,
NJ, 1996.
[118] A. Okubo. Dynamical aspects of animal grouping: Swarms, schools,
flocks, and herds. Advances in Biophysics, 22:1–94, 1986.
[119] R. Olfati-Saber. Flocking for multi-agent dynamic systems: Algorithms
and theory. IEEE Transactions on Automatic Control, 51(3):401–420,
2006.
[120] R. Olfati-Saber, J.A. Fax, and R.M. Murray. Consensus and cooperation
in networked multi-agent systems. Proceedings of the IEEE, 95(1):215–
233, 2007.
[121] R. Olfati-Saber and R.M. Murray. Consensus problems in networks of
agents with switching topology and time-delays. IEEE Transactions on
Automatic Control, 49(9):1520–1533, 2004.
[122] A. Papachristodoulou, A. Jadbabaie, and U. Munz. Effects of delay in
multi-agent consensus and oscillator synchronization. IEEE Transac-
tions on Automatic Control, 55(6):1471–1477, 2010.
[123] M. Pavone and E. Frazzoli. Decentralized policies for geometric pattern
formation and path coverage. Journal of Dynamic Systems, Measure-
ment, and Control, 129(5):633–643, 2007.
[124] L.M. Pecora and T.L. Carroll. Master stability functions for synchro-
nized coupled systems. Physical Review Letters, 80(10):2109–2112, 1998.
[125] Z.H. Peng, D. Wang, and H.W. Zhang. Cooperative tracking and esti-
mation of linear multi-agent systems with a dynamic leader via iterative
learning. International Journal of Control, 87(6):1163–1171, 2014.
[126] Z.H. Peng, D. Wang, H.W. Zhang, G. Sun, and H. Wang. Distributed
model reference adaptive control for cooperative tracking of uncertain
dynamical multi-agent systems. IET Control Theory and Applications,
7(8):1079–1087, 2013.
Bibliography 245

[127] Z.H. Qu. Cooperative Control of Dynamical Systems: Applications to

Autonomous Vehicles. Springer-Verlag, London, UK, 2009.

[128] R. Rajamani. Observers for Lipschitz nonlinear systems. IEEE Trans-

actions on Automatic Control, 43(3):397–401, 1998.
[129] R. Rajamani and Y.M. Cho. Existence and design of observers for
nonlinear systems: Relation to distance to unobservability. International
Journal of Control, 69(5):717–731, 1998.
[130] W. Ren. On consensus algorithms for double-integrator dynamics. IEEE
Transactions on Automatic Control, 53(6):1503–1509, 2008.
[131] W. Ren. Consensus tracking under directed interaction topologies: Algo-
rithms and experiments. IEEE Transactions on Control Systems Tech-
nology, 18(1):230–237, 2010.
[132] W. Ren and E.M. Atkins. Distributed multi-vehicle coordinated control
via local information exchange. International Journal of Robust and
Nonlinear Control, 17(10-11):1002–1033, 2007.

[133] W. Ren and R.W. Beard. Consensus seeking in multiagent systems

under dynamically changing interaction topologies. IEEE Transactions
on Automatic Control, 50(5):655–661, 2005.
[134] W. Ren, R.W. Beard, and E.M. Atkins. Information consensus in multi-
vehicle cooperative control. IEEE Control Systems Magazine, 27(2):71–
82, 2007.

[135] W. Ren, K.L. Moore, and Y. Chen. High-order and model reference con-
sensus algorithms in cooperative control of multivehicle systems. Journal
of Dynamic Systems, Measurement, and Control, 129(5):678–688, 2007.
[136] W. Ren. Consensus strategies for cooperative control of vehicle forma-
tions. IET Control Theory and Applications, 1(2):505–512, 2007.
[137] W. Ren. Collective motion from consensus with Cartesian coordinate
coupling. IEEE Transactions on Automatic Control, 54(6):1330–1335,
2009.
[138] W. Ren and R.W. Beard. Distributed Consensus in Multi-Vehicle Co-
operative Control. Springer-Verlag, London, 2008.
[139] W. Ren and Y.C. Cao. Distributed Coordination of Multi-Agent Net-
works: Emergent Problems, Models, and Issues. Springer-Verlag, Lon-
don, 2010.

[140] C.W. Reynolds. Flocks, herds, and schools: A distributed behavioral

model. Computer Graphics, 21(4):25–34, 1987.
246 Cooperative Control of Multi-Agent Systems

[141] E.J. Rodrı́guez-Seda, J.J. Troy, C.A. Erignac, P. Murray, D.M. Sti-
panovic, and M.W. Spong. Bilateral teleoperation of multiple mobile
agents: coordinated motion and collision avoidance. IEEE Transactions
on Control Systems Technology, 18(4):984–992, 2010.
[142] A. Sarlette, R. Sepulchre, and N.E. Leonard. Autonomous rigid body
attitude synchronization. Automatica, 45(2):572–577, 2009.
[143] L. Scardovi and R. Sepulchre. Synchronization in networks of identical
linear systems. Automatica, 45(11):2557–2562, 2009.
[144] L. Schenato, B. Sinopoli, M. Franceschetti, K. Poolla, and S.S. Sastry.
Foundations of control and estimation over lossy networks. Proceedings
of the IEEE, 95(1):163–187, 2007.
[145] L. Schenato and F. Fiorentin. Average timesynch: A consensus-based
protocol for clock synchronization in wireless sensor networks. Auto-
matica, 47(9):1878–1886, 2011.
[146] J.H. Seo, H. Shim, and J. Back. Consensus of high-order linear sys-
tems using dynamic output feedback compensator: Low gain approach.
Automatica, 45(11):2659–2664, 2009.
[147] R. Sepulchre, D.A. Paley, and N.E. Leonard. Stabilization of planar
collective motion: All-to-all communication. IEEE Transactions on Au-
tomatic Control, 52(5):811–824, 2007.
[148] R. Sepulchre, D.A. Paley, and N.E. Leonard. Stabilization of planar
collective motion with limited communication. IEEE Transactions on
Automatic Control, 53(3):706–719, 2008.
[149] G.S. Seyboth, D.V. Dimarogonas, and K.H. Johansson. Event-based
broadcasting for multi-agent average consensus. Automatica, 49(1):245–
252, 2013.
[150] D. Shevitz and B. Paden. Lyapunov stability theory of nonsmooth sys-
tems. IEEE Transactions on Automatic Control, 39(9):1910–1914, 1994.
[151] D.D. Šiljak. Decentralized Control of Complex Systems. Academic Press,
New York, NY, 1991.
[152] B. Sinopoli, L. Schenato, M. Franceschetti, K. Poolla, M.I. Jordan, and
S.S. Sastry. Kalman filtering with intermittent observations. IEEE
Transactions on Automatic Control, 49(9):1453–1464, 2004.
[153] F. Sivrikaya and B. Yener. Time synchronization in sensor networks: A
survey. IEEE Network, 18(4):45–50, 2004.
[154] J.J.E. Slotine and W. Li. Applied Nonlinear Control. Prentice Hall,
Englewood Cliffs, NJ, 1991.
Bibliography 247

[155] R.S. Smith and F.Y. Hadaegh. Control of deep-space formation-flying

spacecraft; Relative sensing and switched information. Journal of Guid-
ance, Control and Dynamics, 28(1):106–114, 2005.
[156] Q. Song, J.D. Cao, and W.W. Yu. Second-order leader-following con-
sensus of nonlinear multi-agent systems via pinning control. Systems
and Control Letters, 59(9):553-562, 2010.

[157] E. Sontag and A. Teel. Changing supply functions in input/state stable

systems. IEEE Transactions on Automatic Control, 40(8):1476–1478,
1995.
[158] J.F. Sturm. Using SeDuMi 1.02, a MATLAB toolbox for optimization
over symmetric cones. Optimization Methods and Software, 11(1):625–
653, 1999.

[159] H.S. Su, G.R. Chen, X.F. Wang, and Z.L. Lin. Adaptive second-order
consensus of networked mobile agents with nonlinear dynamics. Auto-
matica, 47(2):368–375, 2011.
[160] H.S. Su, X.F. Wang, and G.R. Chen. A connectivity-preserving flocking
algorithm for multi-agent systems based only on position measurements.
International Journal of Control, 82(7):1334–1343, 2009.
[161] H.S. Su, X.F. Wang, and W. Yang. Flocking in multi-agent systems
with multiple virtual leaders. Asian Journal of control, 10(2):238–245,
2008.

[162] H.S. Su, X.F. Wang, and Z.L. Lin. Flocking of multi-agents with a
virtual leader. IEEE Transactions on Automatic Control, 54(2):293–
307, 2009.
[163] Y.F. Su and J. Huang. Cooperative adaptive output regulation for a
class of nonlinear uncertain multi-agent systems with unknown leader.
Systems and Control Letters, 62(6):461–467, 2013.
[164] C. Tan and G. P. Liu. Consensus of networked multi-agent systems via
the networked predictive control and relative outputs. Journal of the
Franklin Institute, 349(7):2343–2356, 2012.

[165] H.G. Tanner, A. Jadbabaie, and G.J. Pappas. Flocking in fixed

and switching networks. IEEE Transactions on Automatic Control,
52(5):863–868, 2007.
[166] Y.P. Tian and C.L. Liu. Robust consensus of multi-agent systems with
diverse input delays and asymmetric interconnection perturbations. Au-
tomatica, 45(5):1347–1353, 2009.
248 Cooperative Control of Multi-Agent Systems

[167] M. Tillerson, G. Inalhan, and J.P. How. Co-ordination and control of

distributed spacecraft systems using convex optimization techniques.
International Journal of Robust and Nonlinear Control, 12(2-3):207–
242, 2002.
[168] L.N. Trefethen and D. Bau. Numerical Linear Algebra. SIAM, Philadel-
phia, PA, 1997.
[169] H.L. Trentelman, K. Takaba, and N. Monshizadeh. Robust synchro-
nization of uncertain linear multi-agent systems. IEEE Transactions on
Automatic Control, 58(6):1511–1523, 2013.
[170] J.N. Tsitsiklis, D.P. Bertsekas, M. Athans, Distributed asynchronous
deterministic and stochastic gradient optimization algorithms. IEEE
Transactions on Automatic Control, 31(9):803–812, 1986.
[171] S. E. Tuna. LQR-based coupling gain for synchronization of linear sys-
tems. arXiv preprint arXiv:0801.3390, 2008.
[172] S.E. Tuna. Synchronizing linear systems via partial-state coupling. Au-
tomatica, 44(8):2179–2184, 2008.
[173] S.E. Tuna. Conditions for synchronizability in arrays of coupled linear
systems. IEEE Transactions on Automatic Control, 54(10):2416–2420,
2009.
[174] T. Vicsek, A. Czirók, E. Ben-Jacob, I. Cohen, and O. Shochet. Novel
type of phase transition in a system of self-driven particles. Physical
Review Letters, 75(6):1226–1229, 1995.
[175] C. Wang, X. Wang, and H. Ji. Leader-following consensus for an
integrator-type nonlinear multi-agent systems using distributed adap-
tive protocol. Proceedings of the 10th IEEE International Conference
on Control and Automation, pp. 1166–1171, 2011.
[176] H.L. Wang. Passivity based synchronization for networked robotic sys-
tems with uncertain kinematics and dynamics. Automatica, 49(3):755–
761, 2013.
[177] J.Y. Wang, Z.S. Duan, Z.K. Li, and G.H. Wen. Distributed H∞ and
H2 consensus control in directed networks. IET Control Theory and
Applications, 8(3):193–201, 2014.
[178] X. Wang, A. Saberi, A.A. Stoorvogel, H.F. Grip, and T. Yang. Consen-
sus in the network with uniform constant communication delay. Auto-
matica, 49(8):2461–2467, 2013.
[179] G.H. Wen, Z.S. Duan, Z.K. Li, and G.R. Chen. Flocking of multi-agent
dynamical systems with intermittent nonlinear velocity measurements.
International Journal of Robust and Nonlinear Control, 22(16):1790–
1805, 2012.
Bibliography 249

[180] G.H. Wen, G.Q. Hu, W.W. Yu, and G.R. Chen. Distributed H∞ consen-
sus of higher order multiagent systems with switching topologies. IEEE
Transactions on Circuits and Systems II: Express Briefs, 61(5):359-363,
2014.
[181] G.H. Wen, Z.S. Duan, Z.K. Li, G.R. Chen. Consensus and its L2 -gain
performance of multi-agent systems with intermittent information trans-
missions. International Journal of Control, 85(4):384-396, 2012.

[182] G.H. Wen, Z.S. Duan, G.R. Chen, W.W. Yu. Consensus tracking of
multi-agent systems with Lipschitz-type node dynamics and switching
topologies. IEEE Transactions on Circuits and Systems I: Regular Pa-
pers, 61(2):499-511, 2014.
[183] G. Wheeler, C.Y. Su, and Y. Stepanenko. A sliding mode controller
with improved adaptation laws for the upper bounds on the norm of
uncertainties. Automatica, 34(12):1657–1661, 1998.
[184] E. Yong. Autonomous drones flock like birds. Nature News,
doi:10.1038/nature.2014.14776, 2014.

[185] K.Y. You and L.H. Xie. Network topology and communication data
rate for consensusability of discrete-time multi-agent systems. IEEE
transactions on automatic control, 56(10):2262–2275, 2011.
[186] K.Y. You, Z.K. Li, and L.H. Xie. Consensus condition for linear
multi-agent systems over randomly switching topologies. Automatica,
49(10):3125–3132, 2013.
[187] K.D. Young, V.I. Utkin, and U. Ozguner. A control engineer’s guide to
sliding mode control. IEEE Transactions on Control Systems Technol-
ogy, 7(3):328–342, 1999.
[188] C.B. Yu, B.D.O. Anderson, S. Dasgupta, and B. Fidan. Control of
minimally persistent formations in the plane. SIAM Journal on Control
and Optimization, 48(1):206–233, 2009.
[189] W.W. Yu, G.R. Chen, M. Cao, and J. Kurths. Second-order consensus
for multiagent systems with directed topologies and nonlinear dynam-
ics. IEEE Transactions on Systems, Man, and Cybernetics, Part B:
Cybernetics, 40(3):881–891, 2010.

[190] W.W. Yu, W. Ren, W.X. Zheng, G.R. Chen, and J.H. Lv. Distributed
control gains design for consensus in multi-agent systems with second-
order nonlinear dynamics. Automatica, 49(7):2107–2115, 2013.
[191] T. Yucelen and W.M. Haddad. Low-frequency learning and fast adap-
tation in model reference adaptive control. IEEE Transactions on Au-
tomatic Control, 58(4):1080–1085, 2013.
250 Cooperative Control of Multi-Agent Systems

[192] T. Yucelen and M. Egerstedt. Control of multiagent systems under per-

sistent disturbances. Proceedings of the 2012 American Control Confer-
ence, pp. 5264–5269, 2012.
[193] T. Yucelen and E.N. Johnson. Control of multivehicle systems in
the presence of uncertain dynamics. International Journal of Control,
86(9):1540–1553, 2013.

[194] M.M. Zavlanos, H.G. Tanner, A. Jadbabaie, and G.J. Pappas. Hybrid
control for connectivity preserving flocking. IEEE Transactions on Au-
tomatic Control, 54(12):2869–2875, 2009.
[195] H.W. Zhang, F.L. Lewis, and A. Das. Optimal design for synchro-
nization of cooperative systems: State feedback, observer, and output
feedback. IEEE Transactions on Automatic Control, 56(8):1948–1952,
2011.
[196] H.W. Zhang and F.L. Lewis. Adaptive cooperative tracking control of
higher-order nonlinear systems with unknown dynamics. Automatica,
48(7):1432–1439, 2012.

[197] Y. Zhao, Z.S. Duan, G.H. Wen, and G.R. Chen. Distributed H∞ con-
sensus of multi-agent systems: A performance region-based approach.
International Journal of Control, 85(3):332–341, 2012.
[198] Y. Zhao, G.H. Wen, Z.S. Duan, X. Xu, and G.R. Chen. A new observer-
type consensus protocol for linear multi-agent dynamical systems. Asian
Journal of Control, 15(2):571–582, 2013.
[199] B. Zhou, C.C. Xu, and G.R. Duan. Distributed and truncated reduced-
order observer based output feedback consensus of multi-agent systems.
IEEE Transactions on Automatic Control, 59(8): 2264–2270, 2014
[200] K.M. Zhou and J.C. Doyle. Essentials of Robust Control. Prentice Hall,
Upper Saddle River, NJ, 1998.
[201] F. Zhu and Z. Han. A note on observers for Lipschitz nonlinear systems.
IEEE Transactions on automatic control, 47(10):1751–1754, 2002.

[202] W. Zhu and D.Z. Cheng. Leader-following consensus of second-order

agents with multiple time-varying delays. Automatica, 46(12):1994–
1999, 2010.