Systems & Control: Foundations & Applications

Series Editor

Christopher I. Byrnes, Washington University

Associate Editors

S.- I. Amari, University of Tokyo

B.D.O. Anderson, Australian National University, Canberra
Karl Johan Astrom, Lund Institute of Technology, Sweden
Jean-Pierre Aubin, EDOMADE, Paris
H.T. Banks, North Carolina State University, Raleigh
John S. Baras, University of Maryland, College Park
A. Bensoussan, INRIA, Paris
John Burns, Virginia Polytechnic Institute, Blacksburg
Han-Fu Chen, Academia Sinica, Beijing
M.H.A. Davis, Imperial College of Science and Technology, London
Wendell Fleming, Brown University, Providence, Rhode Island
Michel F1iess, CNRS-ESE, Gif-sur-Yvette, France
Keith Glover, University of Cambridge, England
Diederich Hinrichsen, University of Bremen, Germany
Alberto Isidori, University of Rome
B. Jakubczyk, Polish Academy of Sciences, Warsaw
Hidenori Kimura, University of Osaka
Arthur J. Krener, University of California, Davis
H. Kunita, Kyushu University, Japan
Alexander Kurzhanski, Russian Academy of Sciences, Moscow
Harold J. Kushner, Brown University, Providence, Rhode Island
Anders Lindquist, Royal Institute of Technology, Stockholm
Andrzej Manitius, George Mason University, Fairfax, Virginia
Clyde F. Martin, Texas Tech University, Lubbock, Texas
Sanjoy K. Mitter, Massachusetts Institute of Technology, Cambridge
Giorgio Picci, University of Padova, Italy
Boris Pshenichnyj, G1ushkov Institute of Cybernetics, Kiev
H.J. Sussman, Rutgers University, New Brunswick, New Jersey
T.J. Tam, Washington University, St. Louis, Missouri
V.M. Tikhomirov, Institute for Problems in Mechanics, Moscow
Pravin P. Varaiya, University of California, Berkeley
Jan C. Willems, University of Groningen, The Netherlands
W.M. Wonham, University of Toronto
Xunjing Li
Jiongmin Yong

Optimal Control Theory

for Infinite Dimensional Systems

Birkhauser
Boston • Basel • Berlin
Xunjing Li Jiongmin Y ong
Department of Mathematics Department of Mathematics
Fudan University Fudan University
Shanghai 200433 Shanghai 200433
China China

Library of Congress Cataloging-in-Publication Data

Li, HSiin-ching
Optimal control theory for infinite dimensional systems / Xunjing
Li, Jiongmin Yong.
p. cm. - (Systems & control)
Includes bibliographical references and index.
TSBN-I3: 978-1-4612-8712-4 e-TSBN-13: 978-1-4612-8712-4
DOT: 10.1007/978-1-4612-4260-4
I. Control theory. 2. Mathematical optimization. 3. Linear
systems. I. Yong, 1. (Jiongmin), 1958- . II. Title.
III. Series.
QA402.3.IA878 1994 94-37168
515'.64--dc20 CIP

Printed on acid-free paper ~®

Birkhauser ~
© Birkhauser Boston 1995
Softcover reprint of the hardcover 1st edition 1995
Copyright is not claimed for works of U.S. Government employees.
All rights reserved. No partofthis publication may be reproduced, stored in a retrieval system,
or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording,
or otherwise, without prior permission of the copyright owner.

Permission to photocopy for internal or personal use of specific clients is granted by

Birkhauser Boston for libraries and other users registered with the Copyright Clearance
Center (CCC), provided thatthe base fee of$6.00 per copy, plus $0.20 per page is paid directly
to CCC, 222 Rosewood Drive, Danvers, MA 01923, U.S.A. Special requests should be
addressed directly to Birkhauser Boston, 675 Massachusetts Avenue, Cambridge, MA 02139,
U.S.A.

Reformatted by Texniques, Inc. from authors' disks.

Printed and bound by Quinn-Woodbine, Woodbine, NJ.

9 8 7 6 5 432 1
Contents

Preface .............................................................. ix

Chapter 1. Control Problems in Infinite Dimensions ............ 1

§1. Diffusion Problems ............................................. 1
§2. Vibration Problems ............................................ 5
§3. Population Dynamics ........................................... 8
§4. Fluid Dynamics ............................................... 12
§5. Free Boundary Problems ...................................... 15
Remarks .......................................................... 22

Chapter 2. Mathematical Preliminaries ......................... 24

§1. Elements in Functional Analysis ............................... 24
§1.1. Spaces ................................................... 24
§1.2. Linear operators .......................................... 27
§1.3. Linear functionals and dual spaces ......................... 28
§1.4. Adjoint operators ......................................... 31
§1.5. Spectral theory ........................................... 32
§1.6. Compact operators ........................................ 33
§2. Some Geometric Aspects of Banach Spaces .................... 36
§2.1. Convex sets ............................................... 36
§2.2. Convexity of Banach spaces ............................... 41
§3. Banach Space Valued Functions ............................... 45
§3.1. Measurability and integrability ............................ 45
§3.2. Continuity and differentiability ............................ 47
§4. Theory of Co Semigroups ...................................... 49
§4.1. Unbounded operators ..................................... 49
§4.2. Co semigroups ............................................ 52
§4.3. Special types of Co semigroups ............................ 55
§4.4. Examples ................................................. 57
§5. Evolution Equations .......................................... 63
§5.1. Solutions .................................................. 63
§5.2. Semilinear equations ...................................... 66
§5.3. Variation of constants formula ............................. 68
§6. Elliptic Partial Differential Equations .......................... 71
§6.1. Sobolev spaces ............................................ 71
§6.2. Linear elliptic equations ................................... 75
§6.3. Semilinear elliptic equations ............................... 78
Remarks .......................................................... 80

Chapter 3. Existence Theory of Optimal Controls ............. 81

§1. Souslin Space ................................................. 81
vi Contents

§1.1. Polish space ............................................... 81

§1.2. Souslin space .............................................. 84
§1.3. Capacity and capacitability ............................... 86
§2. Multifunctions and Selection Theorems ........................ 89
§2.1. Continuity ................................................ 89
§2.2. Measurability ............................................. 94
§2.3. Measurable selection theorems ......... " ................. 100
§3. Evolution Systems with Compact Semigroups ................ 109
§4. Existence of Feasible Pairs and Optimal Pairs ................ 106
§4.1. Cesari property .......................................... 106
§4.2. Existence theorems ...................................... 110
§5. Second Order Evolution Systems ............................. 113
§5.1. Formulation of the problem ......... " ................... 113
§5.2. Existence of optimal controls ............................. 118
§6. Elliptic Partial Differential Equations
and Variational Inequalities ................................. 121
Remarks ......................................................... 129

Chapter 4. Necessary Conditions for Optimal Controls

- Abstract Evolution Equations ................. 130
§1. Formulation of the Problem .. " ...................... , ....... 130
§2. Ekeland Variational Principle ................................ 135
§3. Other Preliminary Results ................................... 137
§3.1. Finite co dimensionality .................................. 137
§3.2. Preliminaries for spike perturbation ...................... 143
§3.3. The distance function .................................... 146
§4. Proof of the Maximum Principle ............................. 150
§5. Applications ................................................. 159
Remarks ......................................................... 165

Chapter 5. Necessary Conditions for Optimal Controls

- Elliptic Partial Differential Equations ......... 168
§1. Semilinear Elliptic Equations ....................... " ........ 168
§1.1. Optimal control problem and
the maximum principle .................................. 168
§1.2. The state constraints ..................................... 171
§2. Variation along Feasible Pairs ................................ 175
§3. Proof of the Maximum Principle ............................. 179
§4. Variational Inequalities ....................................... 183
§4.1. Stability of the optimal cost .............................. 184
§4.2. Approximate control problems ........................... 185
§4.3. Maximum principle and its proof ......................... 188
§5. Quasilinear Equations ........................................ 191
§5.1. The state equation and the optimal control problem ...... 191
Contents vii

§5.2. The maximum principle .................................. 196

§6. Minimax Control Problem .................................... 197
§6.1. Statement of the problem ............. " ................. 197
§6.2. Regularization of the cost functional. ..................... 199
§6.3. Necessary conditions for optimal controls ................. 200
§7. Boundary Control Problems .................................. 207
§7.1. Formulation of the problem .............................. 207
§7.2. Strong stability and the qualified maximum principle ..... 209
§7.3. Neumann problem with measure data .................... 212
§7.4. Exact penalization and a proof of
the maximum principle .................................. 214
Remarks ......................................................... 220

Chapter 6. Dynamic Programming Method for

Evolution Systems ................................. 223
§l. Optimality Principle and Hamilton-Jacobi-
Bellman Equations .......................................... 223
§2. Properties of the Value Functions ............................ 227
§2.1. Continuity ............................................... 228
§2.2. B-continuity ............................................. 231
§2.3. Semi-concavity ........................................... 234
§3. Viscosity Solutions ........................................... 239
§4. Uniqueness of Viscosity Solutions ............................. 244
§4.1. A perturbed optimization lemma ...... " ................. 244
§4.2. The Hilbert space Xu .................................... 248
§4.3. A uniqueness theorem .................................... 250
§5. Relation to Maximum Principle and Optimal Synthesis ....... 256
§6. Infinite Horizon Problems .................................... 264
Remarks ......................................................... 272

Chapter 7. Controllability and Time Optimal Control . ....... 274

§l. Definitions of Controllability ................................. 274
§2. Controllability for linear systems ............................. 278
§2.1. Approximate controllability .............................. 279
§2.2. Exact controllability ..................................... 282
§3. Approximate controllability for semilinear systems ............ 286
§4. Time Optimal Control - Semilinear Systems ................. 294
§4.1. Necessary conditions for time optimal pairs ............... 294
§4.2. The minimum time function .............................. 299
§5. Time Optimal Control- Linear Systems ..................... 302
§5.1. Convexity of the reachable set ............................ 303
§5.2. Encounter of moving sets ................................. 308
§5.3. Time optimal control ..................................... 315
Remarks ......................................................... 317
viii Contents

Chapter 8. Optimal Switching and Impulse Controls ......... 319

§1. Switching and Impulse Controls .............................. 319
§2. Preliminary Results .......................................... 322
§3. Properties of the Value Function ............................. 328
§4. Optimality Principle and the HJB Equation .................. 331
§5. Construction of an Optimal Control .......................... 334
§6. Approximation of the Control Problem ....................... 338
§7. Viscosity Solutions ........................................... 344
§8. Problem in Finite Horizon .................................... 352
Remarks ......................................................... 359

Chapter 9. Linear Quadratic Optimal Control Problems ..... 361

§1. Formulation of the Problem .................................. 361
§1.1. Examples of unbounded control problems ................. 361
§1.2. The LQ problem ......................................... 366
§2. Well-posedness and Solvability ............................... 371
§3. State Feedback Control ...................................... 379
§3.1. Two-point boundary value problem ....................... 379
§3.2. The Problem (LQ)t ...................................... 382
§3.3. A Fredholm integral equation ............................ 386
§3.4. State feedback representation of optimal controls ......... 391
§4. Riccati Integral Equation .................................... 395
§5. Problem in Infinite Horizon .................................. 401
§5.1. Reduction of the problem ................................ 401
§5.2. Well-posedness and solvability ............................ 405
§5.3. Algebraic Riccati equation ............................... 407
§5.4. The positive real lemma ........... , .................. " . .408
§5.5. Feedback stabilization .................................... 412
§5.6. Fredholm integral equation and
Riccati integral equation ................................. 414
Remarks ......................................................... 415

References ......................................................... 419

Index .............................................................. 443

Preface

Infinite dimensional systems can be used to describe many phenomena in

the real world. As is well known, heat conduction, properties of elastic-
plastic material, fluid dynamics, diffusion-reaction processes, etc., all lie
within this area. The object that we are studying (temperature, displace-
ment, concentration, velocity, etc.) is usually referred to as the state. We
are interested in the case where the state satisfies proper differential equa-
tions that are derived from certain physical laws, such as Newton's law,
Fourier's law etc. The space in which the state exists is called the state
space, and the equation that the state satisfies is called the state equation.
By an infinite dimensional system we mean one whose corresponding state
space is infinite dimensional. In particular, we are interested in the case
where the state equation is one of the following types: partial differential
equation, functional differential equation, integro-differential equation, or
abstract evolution equation. The case in which the state equation is being
a stochastic differential equation is also an infinite dimensional problem,
but we will not discuss such a case in this book.
It is known by now that the Pontryagin maximum principle, the Bell-
man dynamic programming method, and the Kalman optimal linear reg-
ulator theory are three milestones of modern (finite dimensional) optimal
control theory (see Fleming [1]). The study of optimal control theory for
infinite dimensional systems can be traced back to the beginning of the
1960's. A main goal of such a theory is to establish the infinite dimen-
sional version of the above-mentioned three fundamental theories. In the
past 30 years, many mathematicians and control theorists have made great
contributions in this research area.
In recent years we and some of our colleagues have been involved in
the study of optimal control theory for infinite dimensional systems. Com-
pared with the works of other mathematicians, we find that ours have their
own flavor, and some of the methods might interest other people working
in this area or in some related fields. Also, in the past few years, we have
taught several courses entitled "Optimal Control for Distributed Parameter
Systems" to graduate students at the Institute of Mathematics, Fudan Uni-
versity. The materials were taken from our recent works; many new results
developed by other mathematicians in recent years were also adopted. We
decided that it would be better to write a book to give a unified presentation
of these theories.
The main feature of this book is the following. First of all, we have
given a unified presentation of optimal control theory for infinite dimen-
sional systems. This includes the existence theory, the necessary conditions
(Pontryagin type maximum principle), the dynamic programming method
(involving the viscosity solution of infinite dimensional Hamilton-Jacobi-
x Pr'eJace

Bellman equations), and the linear-quadratic optimal control problems.

Second, we have made efforts to provide self-contained proofs for many pre-
liminary results that were not contained in previous control theory books.
This will help many graduate students or scholars from other fields under-
stand the theory. Among these proofs, let us mention two such efforts: (i)
We have spent a very reasonable number of pages introducing the theory of
Souslin space and general selection theorems for measurable multifunctions,
which are essential in the presentation of existence theory. (ii) We present
a perturbed optimization lemma resulting from the work of Ekeland and
Lebourg, which is closely related to the well-known Ekeland's variational
principle and plays an important role in the proof of the uniqueness of vis-
cosity solutions. Third, instead of just making some negative statements,
we have presented or cited a number of counterexamples, such as nonclos-
able operators, continuous functions not attaining an infimum on the closed
unit ball in infinite dimensional spaces, nonconvexity of the reachable set
in infinite dimensional spaces, etc. This will help the readers understand
some basic features of infinite dimensional spaces. Finally, to keep the book
at a reasonable length, we had to leave out much important material. To
compensate for this, we have made some brief bibliographic remarks at the
end of each chapter to survey some of the related works.
From the above, we see that this book is something between a mono-
graph and a textbook. It is our desire that the book be useful for graduate
students and researchers in the areas of control theory and applied mathe-
matics. People from other fields such as engineering and economics might
also find the book valuable.
There is a vast amount of literature devoted to the study of optimal
control theory for infinite dimensional systems and related problems. We
have not attempted to give a complete list of references. What we have cited
at the end of the book are those that we find closely related to our presen-
tation. We admit that very many important works might be overlooked.
Fortunately, while preparing the book, we were informed that Professor
H.O. Fattorini (of UCLA) was writing a book relevant to ours. The readers
are suggested to consult that book for some related works on the topic and
a possibly better list of references.
The book is organized as follows. We begin with many examples of con-
trol problems in infinite dimensions in Chapter 1. Chapter 2 discusses some
very basic elements of preliminaries such as functional analysis, semigroups,
evolution equations, and elliptic partial differential equations. The rest of
the book is divided into five " parts." The first part is Chapter 3, in which the
existence theory for optimal controls is presented. The systems discussed
contain elliptic, parabolic, and hyperbolic partial differential equations. Re-
sults for elliptic variational inequalities are also presented. The second part
consists of Chapters 4 and 5. In these two chapters, we present necessary
conditions of the Pontryagin maximum principle type. The results cover
abstract evolution equations, elliptic partial differential equations, and vari-
Preface xi

ational inequalities. Distributed control, boundary control, and minimax

problems, etc. are discussed. The main tools are the Ekeland variational
principle and the spike variation technique for different systems. The third
part consists of Chapters 6 and 8. In this part, we present the Bellman
dynamic programming method for optimal control of abstract evolution
equations. A viscous solution in infinite dimensions is the main topic in
these two chapters. The relation between the Pontryagin maximum princi-
ple and the viscosity solution to the Hamilton-Jacobi-Bellman equation is
studied. The optimal switching and the impulse control problem are also
discussed. The fourth part is Chapter 7, in which we discuss the control-
lability and the time optimal control problem. Linear as well as semilinear
systems evolutions are treated. In the last part, Chapter 9, we discuss the
linear-quadratic (LQ for short) optimal control problem. We concentrate
on the parabolic problem with unbounded control. The general problem
(not necessarily parabolic) with bounded control is also covered. The op-
erators appearing in the cost functional are allowed to be indefinite. For
the finite horizon case, it is shown that, in some sense, the solvabilities
of the LQ problem, an operator Riccati equation and a Fredholm integral
equation are equivalent. Then the infinite horizon case is briefly discussed.
The stabilization of the system is naturally contained in this part.
Some words about the numbering convention. All the heads (by this
we mean definitions, examples, lemmas, propositions, remarks, theorems,
etc.) are numbered consecutively within each section of each chapter with
the first number being the section number. For example, Theorem 3.2 is
the second head in Section 3. When a head of another chapter is cited, the
number of that chapter will be indicated each time. The equations are also
numbered consecutively within each chapter with the first number being
the section number. Thus, (4.5) refers to the fifth equation in Section 4.
The authors would like to acknowledge support from the following: the
NSF of China, the Chinese State Education Commission NSF, and the Fok
Ying Tung Education Foundation. For many years the authors benefited
from their colleagues at Fudan University, to whom we would like to extend
our thanks. Also, the following colleagues deserve special acknowledgment:
Shuping Chen and Kangsheng Liu (Zhejiang University); Zuhao Chen and
Shige Peng (Shandong University); Fulin Jin, Liping Pan, Shanjian Tang,
and Yunlong Yao (Fudan University); and Jingyuan Yu (Beijing Institute
of Information and Control). During the preparation of the book, the
second author spent a year participating in the Control Year held at the
Institute for Mathematics and Its Applications, University of Minnesota,
U.S.A. He would like to thank Professor Avner Friedman for his kind invi-
tation and partial support. During that period the second author benefited
from many stimulating conversations, discussions and collaborations with
other visitors. Among them, the following deserve special acknowledgment:
Gang Bao, Eduardo Casas, Hector O. Fattorini, Scott W. Hansen, Suzanne
M. Lenhart, Walter Littman, Wensheng Liu, Zhuangyi Liu, Jin Ma, Srdjan
xii Preface

Stojanovic, and Bing Yu Zhang. In his mathematical career, the second

author has been deeply influenced mathematically by Professor Leonard
D. Berkovitz, to whom he owes sincere gratitude and would like to dedicate
the book. Finally, the authors would like to thank their family members for
their understanding and patience during the long and tedious preparation
of the book.

Xunjing Li and Jiongmin Yong

Fudan University, China
August 1994