EECE 574 - Adaptive Control

Overview Guy Dumont

Department of Electrical and Computer Engineering University of British Columbia

Lectures: Thursday 09h30-12h00 Location: MCML 158

Guy Dumont (UBC)

EECE 574 Overview

1 / 49

Introduction

Objectives

Objectives

Course Objective: To give an overview of the theory and practice of the mainstream adaptive control techniques Four assignments: 15% each Project: 40% Textbook: K.J. strm and B. Wittenmark, Adaptive Control, Addison-Wesley Publishing Co., Inc., Reading, Massachusetts, 1995. (This book is out of print, but is downloadable from the internet)

Guy Dumont (UBC)

EECE 574 Overview

2 / 49

Introduction

Books

Related Books

N. Hovakimyan, C. Cao, L1 Adaptive Control theory, SIAM Press, Philadelphia, 2010. P. Ioannou and B. Fidan, Adaptive Control Tutorial, SIAM Press, Philadelphia, 2006. V. Bobal, J. Bohm, J. Fessl and J. Macacek, Digital Self-Tuning Controllers, Springer-Verlag, Berlin, 2005. Landau, Lozano and MSaad, Adaptive Control, Springer-Verlag, Berlin, 1998. Isermann, Lachmann and Matko, Adaptive Control Systems, Prentice-Hall, Englewood Cliffs, NJ, 1992. Wellstead and Zarrop, Self-Tuning Systems Control and Signal Processing, J. Wiley and Sons, NY, 1991. Bitmead, Gevers and Wertz, Adaptive Optimal Control, Prentice-Hall, Englewood Cliffs, NJ, 1990. Goodwin and Sin, Adaptive Filtering, Prediction, and Control, Prentice-Hall, Englewood Cliffs, NJ, 1984. Ljung, and Sderstrm, Theory and Practice of Recursive Identication, MIT Press, Cambridge, MA, 1983.

Guy Dumont (UBC)

EECE 574 Overview

3 / 49

Introduction

Books

Course Outline

1 2 3 4 5 6 7 8 9

Introduction Identication Control Design Self-Tuning Control Model-Reference Adaptive Control Properties of Adaptive Controllers Auto-Tuning and Gain Scheduling Implementation and Practical Considerations Extensions

Guy Dumont (UBC)

EECE 574 Overview

4 / 49

Background

Denitions

What Is Adaptive Control?

According to the Websters dictionary, to adapt means:

to adjust oneself to particular conditions to bring oneself in harmony with a particular environment to bring ones acts, behaviour in harmony with a particular environment

According to the Websters dictionary, adaptation means:

adjustment to environmental conditions alteration or change in form or structure to better t the environment

Guy Dumont (UBC)

EECE 574 Overview

5 / 49

Background

Denitions

When is a Controller Adaptive?

Linear feedback can cope with parameter changes (within some limits) According to G. Zames1 :
A non-adaptive controller is based solely on a-priori information An adaptive controller is based on a posteriori information as well

35th CDC, Kobe, Dec 1996

Guy Dumont (UBC) EECE 574 Overview 6 / 49

Background

Denitions

A Narrow Denition of Adaptive Control

An adaptive controller is a xed-structure controller with adjustable parameters and a mechanism for automatically adjusting those parameters In this sense, an adaptive controller is one way of dealing with parametric uncertainty Adaptive control theory essentially deals with nding parameter ajustment algorithms that guarantee global stability and convergence

Guy Dumont (UBC)

EECE 574 Overview

7 / 49

Background

Denitions

Why Use Adaptive Control?

Control of systems with time-varying dynamics If dynamics change with operating conditions in a known, predictable fashion, use gain scheduling If the use of a xed controller cannot achieve a satisfactory compromise between robustness and performance, then and only then, should adaptive control be used Use the simplest technique that meets the specications 2

. . . or as A. Einstein apparently once said: make things as simple as possible, but no simpler
Guy Dumont (UBC) EECE 574 Overview 8 / 49

Background

Process Variations

Feedback and Process Variations

Consider the feedback loop:
Controller ysp + C Process

u
P

The closed-loop transfer function is T = Differentiating T with respect to P: dT T = 1 dP 1 + PC P =S dP P PC 1 + PC

T and S are respectively known as the complementary sensitivity and the sensitivity functions. Note that S+T =1

Guy Dumont (UBC)

EECE 574 Overview

9 / 49

Background

Process Variations

Feedback and Process Variations

The closed-loop transfer function is NOT sensitive to process variations at those frequencies where the loop transfer function L = PC is large Generally L >> 1 at low frequencies, and L << 1 at high frequencies However, L >> 1 can only be achieved in a limited bandwidth, particularly when unstable zeros are present

Guy Dumont (UBC)

EECE 574 Overview

10 / 49

Background

Process Variations

Judging the Severity of Process Variations

Difcult to judge impact of process variations on closed-loop behaviour from open-loop time responses
Signicant changes in open-loop responses may have little effect on closed-loop response Small changes in open-loop responses may have signicant effect on closed-loop response

Effect depends on the desired closed-loop bandwidth Better to use frequency responses

Guy Dumont (UBC)

EECE 574 Overview

11 / 49

Background

Example 1

Effect of Process Variations

Consider the system given by G(s) = 1 (s + 1)(s + a)

Open loop step responses for a = 0.01, 0, 0.01:

Step Response 350 a=0.01 a=0 a=0.01

300

250

Amplitude

200

150

100

50

50 Time (sec)

100

150

Figure: Open-loop responses

Guy Dumont (UBC) EECE 574 Overview 12 / 49

Background

Example 1

Effect of Process Variations

Step Response 1.4 a=0.01 a=0 a=0.01

1.2

Amplitude

0.8

0.6

0.4

0.2

5 Time (sec)

10

15

Figure: Closed-loop responses for unit feedback

Guy Dumont (UBC)

EECE 574 Overview

13 / 49

Background

Example 1

Effect of Process Variations

Bode Diagram 150 100 Magnitude (dB) 50 0 50 100 0 45 Phase (deg) 90 135 180 10
4

a=0.01 a=0 a=0.01

10

10

10 Frequency (rad/sec)

10

10

10

Figure: Open-loop Bode plots

Guy Dumont (UBC)

EECE 574 Overview

14 / 49

Background

Example 1

Effect of Process Variations

Bode Diagram 20 0 Magnitude (dB) 20 40 60 80 0 45 Phase (deg) 90 135 180 10
2

a=0.01 a=0 a=0.01

10

10 Frequency (rad/sec)

10

10

Figure: Closed-loop Bode plots

Guy Dumont (UBC)

EECE 574 Overview

15 / 49

Background

Example 2

Effect of Process Variations

Consider now the system G(s) = 400(1 sT ) (s + 1)(s + 20)(1 + sT )

Open-loop responses for T = 0, 0.015, 0.03:

Step Response 20 T=0 T=0.015 T=0.03 15

Amplitude

10

0.5

1.5

2.5 Time (sec)

3.5

4.5

Figure: Open-loop responses

Guy Dumont (UBC) EECE 574 Overview 16 / 49

Background

Example 2

Effect of Process Variations

Step Response 2 T=0 T=0.015 T=0.03 1.5

Amplitude

0.5

0.5

0.1

0.2

0.3

0.4

0.5 Time (sec)

0.6

0.7

0.8

0.9

Figure: Unit-feedback closed-loop responses

Guy Dumont (UBC)

EECE 574 Overview

17 / 49

Background

Example 2

Effect of Process Variations

Bode Diagram 50

Magnitude (dB)

50

100

150 360

Phase (deg)

180

T=0 T=0.015 T=0.03

180 10
2

10

10

10 Frequency (rad/sec)

10

10

10

Figure: Open-loop Bode plots

Guy Dumont (UBC)

EECE 574 Overview

18 / 49

Background

Example 2

Effect of Process Variations

Bode Diagram 50

Magnitude (dB)

50

100

150 360

Phase (deg)

180

T=0 T=0.015 T=0.03

180 10
0

10

10 Frequency (rad/sec)

10

10

Figure: Unit-feedback closed-loop Bode plots

Guy Dumont (UBC)

EECE 574 Overview

19 / 49

Background

Example 2

Effect of Process Variations

Consider now the same system but with a controller C(s) = 0.075/(s + 1):
Step Response 12 T=0 T=0.015 T=0.03

10

Amplitude

0.5

1.5

2.5 Time (sec)

3.5

4.5

Figure: New closed-loop responses

Guy Dumont (UBC) EECE 574 Overview 20 / 49

Background

Example 2

Effect of Process Variations

Bode Diagram 50

Magnitude (dB)

50

100

150 540 360 Phase (deg) 180 0 180 10

1

T=0 T=0.015 T=0.03

10

10 10 Frequency (rad/sec)

10

10

Figure: New closed-loop Bode plots

Guy Dumont (UBC)

EECE 574 Overview

21 / 49

Background

Example 2

Mechanisms for Process Dynamics Changes

Nonlinear actuators or sensors

Nonlinear valves pH probes

Flow and speed variations

Concentration control Steel rolling mills Paper machines Rotary kilns

Guy Dumont (UBC)

EECE 574 Overview

22 / 49

Background

Example 2

Mechanisms for Process Dynamics Changes

Wide operating range with a nonlinear system

Flight control

Variations in Disturbance Dynamics

Wave characteristics in ship steering Raw materials in process industries

Guy Dumont (UBC)

EECE 574 Overview

23 / 49

Adaptive Schemes

Gain Scheduling

Gain Scheduling

In many cases, process dynamics change with operating conditions in a known fashion
Flight control systems Compensation for production rate changes Compensation for paper machine speed

Controller parameters change in a predetermined fashion with the operating conditions Is gain scheduling adaptive?

Guy Dumont (UBC)

EECE 574 Overview

24 / 49

Adaptive Schemes

Gain Scheduling

Gain Scheduling

Controller parameters

Gain schedule

Setpoint Controller

Input

Process

Output

Guy Dumont (UBC)

EECE 574 Overview

25 / 49

Adaptive Schemes

Development of Adaptive Control

Development of Adaptive Control

Mid 1950s: Flight control systems (eventually solved by gain scheduling) 1957: Bellman develops dynamic programming 1958: Kalman develops the self-optimizing controller which adjusts itself automatically to control an arbitrary dynamic process 1960: Feldbaum develops the dual controller in which the control action serves a dual purpose as it is directing as well as investigating

Guy Dumont (UBC)

EECE 574 Overview

26 / 49

Adaptive Schemes

Development of Adaptive Control

Development of Adaptive Control

Mid 60s-early 70s: Model reference adaptive systems

But now came a technical problem that spelled the end. The Honeywell adaptive ight control system began a limit-cycle oscillation just as the plane came out of the spin, preventing the systems gain changer from reducing pitch as dynamic pressure increased. The X-15 began a rapid pitching motion of increasing severity. All the while, the plane shot downward at 160,000 feet per minute, dynamic pressure increasing intolerably. . . . As the X-15 neared 65,000 feet, it was speeding downward at Mach 3.93 and experiencing over 15 g vertically, both positive and negative, and 8 g laterally. It broke up into many pieces amid loud sonic rumblings, . . . Then an Air Force pilot, . . . , spotted the main wreckage . . . . Mike Adams was dead and the X-15 destroyed.

Guy Dumont (UBC)

EECE 574 Overview

27 / 49

Adaptive Schemes

Development of Adaptive Control

Development of Adaptive Control

Late 60s-early 70s: System identication approach with recursive least-squares Early 1980s: Convergence and stability analysis Mid 1980s: Robustness analysis 1990s: Multimodel adaptive control 1990s: Iterative control 2000s: L1 adaptive control: fast adaptation with guaranteed robustness.

Guy Dumont (UBC)

EECE 574 Overview

28 / 49

Adaptive Schemes

Model Reference Adaptive Control

Model Reference Adaptive Control

Performance specications given in terms of reference model Originally introduced for ight control systems (MIT rule) Nontrivial adjusment mechanism

Guy Dumont (UBC)

EECE 574 Overview

29 / 49

Adaptive Schemes

Model Reference Adaptive Control

Model Reference Adaptive Control

Model

Model output

Controller parameters Setpoint Controller Input

Adjustment mechanism

Process Output

Guy Dumont (UBC)

EECE 574 Overview

30 / 49

Adaptive Schemes

Self-Tuning Control

Self-Tuning Controller

Model-based tuning consists of two operations:

Model building via identication Controller design using the identied model

Self-tuning control can be thought of as an automation of this procedure when these two operations are performed on-line

Guy Dumont (UBC)

EECE 574 Overview

31 / 49

Adaptive Schemes

Self-Tuning Control

Self-Tuning Controller

Process parameters Control design Controller parameters Setpoint Controller Input Process Output

Recursive estimation

Guy Dumont (UBC)

EECE 574 Overview

32 / 49

Adaptive Schemes

Self-Tuning Control

Self-Tuning vs. Auto-Tuning

Self-tuning
Continuous updating of controller parameters Used for truly time-varying plants

Auto-tuning
Once controller parameters near convergence, adaptation is stopped Used for time invariant or very slowly varying processes Used for periodic, usually on-demand tuning

Guy Dumont (UBC)

EECE 574 Overview

33 / 49

Adaptive Schemes

Self-Tuning Control

Final Motivation...

Guy Dumont (UBC)

EECE 574 Overview

34 / 49

Adaptive Schemes

Self-Tuning Control

Final Motivation...

Guy Dumont (UBC)

EECE 574 Overview

35 / 49

Dual Control

Dual Control: A Rigorous Approach to Adaptive Control

Use of nonlinear stochastic control theory to derive an adaptive controller No distinction between parameters and state variables Hyperstate The controller is a nonlinear mapping from the hyperstate to the control variable
Hyperstate

Hyperstate estimation Setpoint Nonlinear mapping Process Input Output

Guy Dumont (UBC)

EECE 574 Overview

36 / 49

Dual Control

A Rigorous Approach to Adaptive Control

Can handle very rapid parameter changes Resulting controller has very interesting features:
Regulation Caution Probing

Unfortunately solution is untractable for most systems

Guy Dumont (UBC)

EECE 574 Overview

37 / 49

Dual Control

Illustration of Dual Control

Consider the simple process y (t + 1) = y (t) + bu(t) + e(t + 1) where e(t) is zero-mean white noise N(0, ), y (t)and u(t) are the output and the input signals. One-stage control Find u(t) that minimizes I1 = E[y 2 (t + 1)|y (t), u(t)]

Guy Dumont (UBC)

EECE 574 Overview

38 / 49

Dual Control

Certainty equivalence controller

Certainty Equivalence Controller

In case b is known, the solution is trivial: min I1 = min [y (t 1) + bu(t 1)]2 + 2 = 2 since e(t) is independent of y (t 1), u(t 1) and b. u(t) = y (t) b I1opt = 2

Guy Dumont (UBC)

EECE 574 Overview

39 / 49

Dual Control

Certainty equivalence controller

Certainty Equivalence Controller

Now, assume that b is unknown. We now have an estimate b with covariance pb If least-squares is used:
t t

b=
s=1

[y (s) y (s 1)]u(s 1) /
s=1 t

u 2 (s 1)

pb = 2 /
s=1

u 2 (s 1)

Guy Dumont (UBC)

EECE 574 Overview

40 / 49

Dual Control

Certainty equivalence controller

Certainty Equivalence Controller

The most direct way to control the system is simply to replace b by b in the controller above, thus ignoring the uncertainty: uce (t) = then I1ce = 2 + y (t) b

pb 2 y (t 1) b2

Guy Dumont (UBC)

EECE 574 Overview

41 / 49

Dual Control

Cautious Controller

Cautious Controller

Performing the minimization of I1 actually gives: u(t) = b b 2 + pb y (t)

and the minimum performance index I1caut = 2 + pb y 2 (t 1) 2 + pb b

Guy Dumont (UBC)

EECE 574 Overview

42 / 49

Dual Control

Cautious Controller

Cautious Controller

Because pb is positive, the cautious controller has a smaller gain than the certainty equivalence one, which by ignoring uncertainty may be at times too bold Turn-off phenomenon:
When the uncertainty pb is large, controller gain is small and so does the control action So, unless an external perturbation is added to the input, no learning can take place and the uncertainty pb cannot be reduced

This highlights the importance of probing signals

Guy Dumont (UBC)

EECE 574 Overview

43 / 49

Dual Control

Dual Controller

Dual Controller
N-stage control Find u(t) that minimizes
N

IN = E[
1

y 2 (t + i)|y (t), u(t)]

By using the N-stage control problem with N > 1, it can be shown that the effect of present inputs on the future values of b and pb enters the minimization of IN Indeed it is sometimes benecial to sacrice short term performance by sending a probing signal to reduce the uncertainty, and thus improve performance in the long term

Guy Dumont (UBC)

EECE 574 Overview

44 / 49

Dual Control

Dual Controller

Dual Controller

Using dynamic programming, a functional equation (Bellman equation) can be derived However, this equation can only be solved numerically and for very simple cases For large N, the control tends towards a steady-state control law

Guy Dumont (UBC)

EECE 574 Overview

45 / 49

Dual Control

Dual Controller

Dual Controller

Dene = y =

b pb

bu y

= 1 corresponds to the certainty-equivalence controller = 2 /(1 + 2 ) corresponds to the cautious controller Dual controller for large N is: = 2 0.56 + 2 + 0.08 + 2.2 1.9 + 1.7 1

Guy Dumont (UBC)

EECE 574 Overview

46 / 49

Dual Control

Dual Controller

Dual Controller
Dual Control Map

2 1.5 1 0.5 0 0.5 0

0 0.2 0.4 0.6 0.5 0.8 1 1 Increasing Control Error

Increasing Estimate Accuracy

Figure: Dual control map

Guy Dumont (UBC) EECE 574 Overview 47 / 49

Dual Control

Dual Controller

Properties of the Dual Controller

Dual control nds the best compromise between

boldness caution probing

Low uncertainty boldness prevails Large uncertainty + large control error caution prevails Large uncertainty + small control error probing prevails

Guy Dumont (UBC)

EECE 574 Overview

48 / 49

Implications forAdaptive Control

Implications for Adaptive Control

The dual controller is in general impossible to compute Most current adaptive control methods enforce certainty equivalence Thus, learning is passive rather than active Passive learning is a shortcoming of current adaptive control methods Practical methods of active learning attractive for
Commissioning of adaptive controllers Adaptive control of processes with rapidly time-varying dynamics

Guy Dumont (UBC)

EECE 574 Overview

49 / 49