EE5103/ME5403 Computer Control Systems

Part I

Xiang Cheng
Dept. of Electrical and Computer Engineering
National University of Singapore
Contents

1 Systems and Control —Background Overview 4

1.1 What is Control Theory? . . . . . . . . . . . . . . . . . . . . 4
1.1.1 Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.2 Block Diagrams . . . . . . . . . . . . . . . . . . . . . . 6
1.1.3 Mathematical Systems . . . . . . . . . . . . . . . . . . 9
1.1.4 The behavior of systems . . . . . . . . . . . . . . . . . 9
1.1.5 Improvement of the behavior of systems . . . . . . . . 10
1.2 A simple example of feedback controller . . . . . . . . . . . . 12
1.2.1 Model of the robotic single-link rotational joint . . . . 12
1.2.2 Mathematical analysis of the model . . . . . . . . . . 14
1.2.3 Open loop and feedback controllers . . . . . . . . . . . 22
1.3 State-Space approach . . . . . . . . . . . . . . . . . . . . . . . 27
1.3.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.3.2 Properties . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.3.3 State-feedback controller . . . . . . . . . . . . . . . . . 32
1.4 Computer Control System . . . . . . . . . . . . . . . . . . . . 33
1.5 Notes and References . . . . . . . . . . . . . . . . . . . . . . . 34

2 Computer Control and Discrete-Time Systems 35

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.1.1 A Naive Approach to Computer-Controlled Systems . 37
2.1.2 Deadbeat Control . . . . . . . . . . . . . . . . . . . . 40
2.2 Sampling a Continuous-Time State-Space System . . . . . . . 42
2.3 State-space Model of Discrete-Time Systems . . . . . . . . . . 44
2.3.1 Solution of the System Equation . . . . . . . . . . . . 44
2.4 Input-Output Models . . . . . . . . . . . . . . . . . . . . . . 45
2.4.1 Shift-Operator calculus . . . . . . . . . . . . . . . . . 45
2.5 The z-Transform . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.6 Frequency Response for Discrete Systems . . . . . . . . . . . 52

1
 2.7 Transfer Function from Continuous Time to Discrete Systems 53
2.8 Poles and Zeros . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.9 Notes and References . . . . . . . . . . . . . . . . . . . . . . . 59

3 Analysis of Discrete - Time Systems 60

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . 61
3.2.2 Stability of Linear Discrete-Time Systems . . . . . . . 62
3.2.3 Input-Output Stability . . . . . . . . . . . . . . . . . . 63
3.2.4 Stability Tests . . . . . . . . . . . . . . . . . . . . . . 66
3.2.5 Jury’s Stability Criterion . . . . . . . . . . . . . . . . 67
3.3 Controllability, Observability and Realizability . . . . . . . . 69
3.3.1 Controllability . . . . . . . . . . . . . . . . . . . . . . 69
3.3.2 Controllable Canonical Form . . . . . . . . . . . . . . 72
3.3.3 Trajectory Following . . . . . . . . . . . . . . . . . . . 73
3.3.4 Observability . . . . . . . . . . . . . . . . . . . . . . . 74
3.3.5 Observable Canonical Form . . . . . . . . . . . . . . . 76
3.3.6 Loss of Controllabilty and Observability Through Sam-
pling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.3.7 Realizability . . . . . . . . . . . . . . . . . . . . . . . 78
3.4 Analysis of Simple Feedback Loops . . . . . . . . . . . . . . . 82
3.4.1 Character of Disturbances . . . . . . . . . . . . . . . . 83
3.4.2 Simple Disturbance Models . . . . . . . . . . . . . . . 84
3.4.3 Generation of disturbances . . . . . . . . . . . . . . . 85
3.4.4 Steady-State Values . . . . . . . . . . . . . . . . . . . 85
3.4.5 Simulation . . . . . . . . . . . . . . . . . . . . . . . . 86

4 Pole-Placement Problem — State-Space Approach 96

4.1 Pole placement problem . . . . . . . . . . . . . . . . . . . . . 97
4.1.1 Dead beat control . . . . . . . . . . . . . . . . . . . . 103
4.1.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 104
4.2 Observer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4.2.1 The Naive Observer . . . . . . . . . . . . . . . . . . . 104
4.2.2 Observer . . . . . . . . . . . . . . . . . . . . . . . . . 105
4.2.3 Dead-beat observer . . . . . . . . . . . . . . . . . . . . 106
4.3 Output-Feedback Controller . . . . . . . . . . . . . . . . . . . 107
4.4 Disturbance Rejection Problem . . . . . . . . . . . . . . . . . 108
4.5 The Tracking Problem — Zero Placement . . . . . . . . . . . 112
4.5.1 A Naive Approach . . . . . . . . . . . . . . . . . . . . 112

2
 4.5.2 Two-Degree-of-Freedom Controller . . . . . . . . . . . 113
4.5.3 Putting It All Together . . . . . . . . . . . . . . . . . 115

5 Pole-Placement — Polynomial Approach 124

5.1 Review: Pole placement problem - State-space approach . . . 124
5.2 Relation between State-Space Model and Input-Output Model 126
5.3 Pole-Placement Problem – Polynomial Approach . . . . . . . 128
5.3.1 Error-Feedback Control . . . . . . . . . . . . . . . . . 128
5.3.2 Two-degree-of-freedom controller . . . . . . . . . . . . 130
5.4 Pole-Placement Problem— More Realistic Situations . . . . . 138
5.4.1 Pole-Zero Cancelation . . . . . . . . . . . . . . . . . . 139
5.4.2 Disturbance Rejection Problem . . . . . . . . . . . . 142
5.4.3 Tracking Problem — Command Signal Following . . . 145

6 Predictive Control — Solution to Tracking Problem 149

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6.2 Minimum Prediction Error Controllers . . . . . . . . . . . . . 151
6.2.1 One-Step-Ahead Control: The Case of d = 1 . . . . . 151
6.2.2 Prediction with Known Models . . . . . . . . . . . . . 152
6.2.3 One-Step-Ahead Control: General Case . . . . . . . . 154
6.2.4 Weighted One-Step-Ahead Control . . . . . . . . . . . 155
6.3 Further Beyond . . . . . . . . . . . . . . . . . . . . . . . . . . 158
6.4 Notes and References . . . . . . . . . . . . . . . . . . . . . . . 160

3
Chapter 1

Systems and Control

—Background Overview

Questions of control have been of great interest since ancient times, and are
assuming major importance in modern society. By the “control of a system”
we mean the ability to influence its behavior so as to achieve a desired goal.
Control mechanisms are ubiquitous in living organisms, and the design of
self-regulating systems by human beings also has a long history. These sys-
tems range from water clocks in antiquity to aqueducts in early Rome, from
Watt’s steam engine governor in 1769 which ushered in the Industrial Revo-
lution in England, to the sophisticated automobiles, robots, and unmanned
aircraft in the modern times.

The main purpose of this module is to teach you the most fundamental
concepts and tools in designing a control system for real world problems.
Both computer control theory and applications will be discussed. For the
first half of this module, our focus will be only upon the control theory, and
I will leave the practical applications and design issues to the later stage.
Therefore, the first question I would like to raise is: what is control theory
all about?

1.1 What is Control Theory?

If the answer is given in one short sentence, then control theory centers
about the study of systems. Indeed, one might describe control theory as
the care and feeding of systems. But, what is a system?

4
1.1.1 Systems
Intuitively, we can consider a system to be a set of interacting components
subject to various inputs and producing various outputs. In fact system is
such a general concept that there is no universally accepted definition. Ev-
eryone has his/her own view of what a system should look like. It is like the
concept of set in mathematics, everyone knows what a set is, but we cannot
define a set! Likewise, we cannot define a system precisely!

The student may object, “I do not like this imprecision, I should like to have
everything defined exactly; in fact, it says in some books that any science
is an exact subject, in which everything is defined precisely.” But the truth
is, if you insist upon a precise definition of system, you will never get it!
For example, philosophers are always saying, “Well, just take a chair for ex-
ample.” The moment they say that, you know that they do not know what
they are talking about any more. What is a chair? Well, a chair is a certain
thing over there ... certain? how certain? The atoms are evaporating from
it from time to time – not many atoms, but a few — dirt falls on it and gets
dissolved in the paint; so to define a chair precisely, to say exactly which
atoms are paint that belongs to the chair is impossible.

A mathematical definition will be good for mathematics, in which all the

logic can be followed out completely, but the physical world is too complex.
In mathematics, almost everything can be defined precisely, and then we
do not know what we are talking about. In fact, the glory of mathematics
is that we do not have to say what we are talking about. The glory is that
the laws, the arguments, and the logic are independent of what “it” is. If
we have any other set of objects that obey the same system of axioms as
Euclid’s geometry, then if we make new definitions and follow them out with
correct logic, all the consequences will be correct, and it makes no difference
what the subject is. In nature, however, when we draw a line or establish
a line by using a light beam, as we do in surveying, are we measuring a
line in the sense of Euclid? No, we are making an approximation; the light
beam has some width, but a geometrical line has no width, and so, whether
Euclidean geometry can be used for surveying or not is a physical question,
not a mathematical question. However, from an experimental standpoint,
not a mathematical standpoint, we need to know whether the laws of Euclid
apply to the kind of geometry that we use in measuring land; so we make
a hypothesis that it does, and it works pretty well; but it is not precise,
because our surveying lines are not really geometrical lines.

5
Let’s return to the concept of “system”. Some people would insist that
system must contain certain internal components, while others may argue
that any object in the world can be considered as a system. Can we say that
a single electron is a system? That is definitely debatable! How about your
mind? do you consider it as a system? Well, we don’t have to quarrel over
the definition of the system, which we will happily leave to the philosophers.
We can attack a problem without knowing the precise definition! Let’s just
look at the various types of systems around us:

• Mechanical systems: Clocks, Pistons

• Electrical systems: Radios, TVs

• Electrical-mechanical-chemical systems: Automobiles, Hard-disk-drives

• Industrial systems: Factories

• Medical systems: Hospitals

• Educational systems: Universities

• Biological systems: Human beings

• Information processing systems: Digital computers

This brief list, however, is sufficient to emphasize the fact that one of the
most profound concepts in current culture is that of a “system”.

1.1.2 Block Diagrams

From the standpoint of control engineering, there is always something going
on in the system. If you are particularly interested in one of the variables,
then you may want to measure it with some sensors. And we call these vari-
ables, outputs. Sometimes, you can even affect the output of the system by
tuning some other signals, which we call inputs. And it is often a convenient
guide to conceive of a system in terms of a block diagram.
The first attempt might be the one shown in Figure 1.1.

Frequently, Fig1.1 is much too crude and one prefers the following diagram
to indicate the fact that a system usually has a variety of inputs and outputs:

6
 Figure 1.1: The simplest block diagram of a system

Figure 1.2: A block diagram of a system by variety of inputs and outputs

But wait a minute, we have not discussed the meaning of input. What is
input?
Input: the attributes which can be varied by the controller. (Car driving
example: brake, gas paddle, steering wheel)
Hey, you may wonder: can we consider noise, or disturbance (which cannot
be varied by the man-made controller easily) as input? Yes and No. It all
depends upon your point of view. If you adopt a very broad view of the
input as any factors that can affect the behavior of the system, then the
answer is yes. But if you limit it to the things that can be manipulated,
then all those uncontrollable factors such as noise have to be distinguished
from a normal input. Once again, don’t try to find a precise definition of
“input”, as you may get into trouble again. But you never bother about it,
as in reality, the meaning of the input is usually self evident for any specific
control system.

What is output?
Output: the attributes of a system which are of interest to you, and are
observed carefully, but not varied directly. (Car driving example: position
and velocity of the car)

7
But there is no clear cut between input and output, it all depends upon
which aspect of the system you are looking at! In multi-stage production
process, or chemical separation process, we may employ a block diagram as
shown in Fig.1.3. Fig 1.3 indicates that the inputs u1 , u2 to subsystems S1
produce outputs y1 , y2 , that are themselves inputs to subsystem S2 , and so
on until the final output.

Figure 1.3: A block diagram of a multi-stage system

It is better to think of inputs and outputs in terms of cause and effect. For
a certain system, inputs are the signals to drive the dynamics of the system
and the outputs are the effects of the inputs. However, we cannot define
precisely what inputs and outputs are!

In the study of atmospheric physics, geological exploration, and cardiology,

we frequently think in terms of diagrams such as:

Figure 1.4: A block diagram of systems in atmospheric physics

Fig1.4 brings clearly to the fore that there are internal subsystems which we
can neither directly examine nor influence. The associated question of iden-
tification and control of S3 are quite challenging. And it depends upon how
much you know about the internal system. In particular, it depends upon
whether any mathematical models are available for the various subsystems.

8
1.1.3 Mathematical Systems
It cannot be too strongly emphasized that real systems possess many differ-
ent conceptual and mathematical realizations. Each mathematical descrip-
tion serves its own purpose and possesses certain advantages and disadvan-
tages. There is no such universal mathematical model that can serve every
purpose.

For example, consider an airplane which can be analyzed by many mathe-

matical models from different viewpoints:

• Electrical engineers: autopilot design; sensor systems (signal process-

ing models);

• Mechanical engineers: air dynamic models; structure optimization;

material analysis.

Different models serve different purposes. Once the mathematical die has
been cast, equations assume a life of their own and can easily end by be-
coming master rather than servant. But as an engineer, you must always
check the physical meanings of the various variables you are studying.

In any case, it should be constantly kept in mind that the mathematical

system is never more than a projection of the real system on a conceptual
axis. Mathematical model is only an approximation of the real world. It
does not make sense to talk about the “true” model, you can only argue
that a model is good or bad based upon the experimental evidence. But you
will never be 100% sure that a model is “true”. This is one of the lessons
we have learned from the history of science. Take an example of Newton’s
law, almost everyone believed it was a “true” law governing the nature, until
Einstein showed the world his Relativity. If Newton’s law is not accurate,
or is not “true”, every model can be “wrong”!

1.1.4 The behavior of systems

In all parts of contemporary society, we observe systems that are not oper-
ating in a completely desired fashion.

• Economical systems: inflation, depression and recession;

• Human systems: cancer, heart diseases;

• Industrial systems: unproductivity and non-profitability;

9
 • Ecological systems: pests and drought; green house effect.

What we can conclude from this sorry recital is that systems do indeed re-
quire care. They do not operate in a completely satisfactory manner by
themselves. That’s why control theory can play very important role in the
real world. On the other hand, as we shall see, the cost of supervision must
be kept clearly in mind. This cost may be spelled out in terms of resources,
money, manpower, time, or complexity.

To control or not to control, that is a difficult question under certain cir-

cumstances. For instance, there is this huge debate on what is the role
of government in the economy. The advocates of the free economy theory
strongly oppose any interference of the government into the market. They
believe that the “invisible hand” can handle everything. The opposite view
is that the government should play active role in regulating the market.
Consider the extreme example of the Hong Kong’s Financial Crisis 1997-
1998, would it be better that the Hong Kong government did not react to
the attack of the international hedge funds? You can cast your own vote.

Fortunately, for engineers, we don’t have to make this hard decision. Your
boss makes the call! (but what if you become the boss in the future?) It
is our primary concern to improve the behavior of any specific engineering
system.

So in the next section, we are going to discuss how to improve the perfor-
mance of a system.

1.1.5 Improvement of the behavior of systems

There are several immediate ways of improving the performance of a system.

1. Build a new system;

2. Replace or change the subsystems;

3. Change the inputs.

In some cases, such as, say fixing an old car, a new system may be the ideal
solution. In other cases, say the human body or an economical system, re-
placement is not a feasible procedure. We shall think in terms of the more
feasible program of altering the design of the system or in terms of modifying
the inputs, or both. In reality, there are always multiple solutions to fix the

10
problems. The choice of the solutions would depend upon your requirement
and your resource. For instance, if an old car keeps giving you headaches,
how are you going to fix that? If you are a rich man, then the solution is
simple, just buy a new one. If you are a poor student, probably you will
send it to a garage and have the engine or other parts replaced. But before
we go into further details, let’s try to give a rough definition of control.
Control: influence the behavior of the system to achieve a desired goal.

Control theory mostly concerns about the third way: how to change the
inputs such that the outputs behave in a desired fashion.
Usually the goal of automatic control for industrial systems is maintaining
one or more outputs around some desired constant or time-varying values.

(a) Constant—Regulation, or Set (b) Time-varying—Tracking: Au-

point control: Air-conditioning sys- topilot; Guided missile.
tem; Cruise system.

Figure 1.5: The goal of automatic control

How do we achieve such goals? Is there any fundamental principle to guide

us to design self-regulating systems?
Let’s try to learn from our own experiences. There are many things for us
to control in our daily life. How do we do that? For instance, consider a
driving example – steering the car within one lane. There are three processes
involved:
Observe the heading—compare with the lane lines —adjust the wheel

Trio: Measurement — comparison — adjustment

By measuring the quantities of interest, comparing it with the desired value,

and using the error to correct the process, the familiar chain of cause and
effect in the process is converted into a closed loop of interdependent events
as shown in Fig 1.6. That is the

11
 Figure 1.6: A closed-loop system

Fundamental Concept of Control: Feedback

How to apply this idea of feedback to build self-regulating system is the

central problem of automatic control.

Once again we have to emphasize that control theory is about applying the
idea of feedback to build a self-regulating system. Since the systems can be
represented in terms of differential or difference equations, it is sometimes
misleading to think that the control theory is about solving or analyzing the
differential equations, which is totally wrong.

In the next, we are going to study a simple example to show why feedback
is playing the central role in controller design.

1.2 A simple example of feedback controller

Following we will use a very simple physical example to give an intuitive
presentation of some of the goals, terminology and methodology of automatic
control. The emphasis of the discussion will be put on motivation and
understanding of the basic concepts rather than mathematical rigidity.

1.2.1 Model of the robotic single-link rotational joint

One of the simplest problems in robotics is that of controlling the position
of a single-link rotational joint using a motor placed at the pivot. Mathe-
matically, this is just a pendulum to which one can apply a torque as an
external force.

12
 Figure 1.7: A single-link rotational joint

m — mass
l — length of rod
g — acceleration due to gravity
u(t) — external torque at time t — input
θ(t) — counterclockwise angle of the pendulum at time t — output

Once the system is chosen, the first step is building a mathematical model.
But how to build a model? It depends upon how much knowledge you have.
There are basically three ways:

• White Box —First principles.

• Black Box —- Assume a function or equation form, then estimate the

parameters by data fitting.

• Grey Box

White-box is always the first option you should consider for modeling if
possible. We assume the friction is negligible, and all of the mass is con-
centrated at the end. From Newton’s second law, the change rate of the
angular momentum is proportional to the external torque, we have

ml2 θ̈(t) + mgl sin θ(t) = u(t) (1.1)

To avoid keeping track of constants, let’s assume that the units of time and
distance have been chosen such that

ml2 = mgl = 1

So we are looking at a system describing by a second order nonlinear differ-

ential equation,
θ̈(t) + sin θ(t) = u(t) (1.2)

13
1.2.2 Mathematical analysis of the model
What are the equilibrium positions when there is no torque?
The equilibrium positions are the positions where θ̈ = 0, and θ̇ = 0.
Equilibrium positions:
1. θ̇ = 0, θ = 0 : Stable position
2. θ̇ = 0, θ = π : Unstable position
Stable:
If you change the initial condition a little bit, the dynamics of the system
will change a little bit, but not too much.
Unstable:
If the initial conditions are just a bit different, the dynamic behaviors of the
system are dramatically different.
Asymptotic stable:
Not only stable but also the response of the system will converge to the
equilibrium point.
Before we design the controller, we have to be clear about the objective we
try to achieve.

Control objective:
Apply torque u(t) to stabilize the inverted pendulum such that the pendu-
lum will maintain the upright position (θ = π).

Now please try to put a pen or pencil in your finger tip, and keep it upright.
Can you do that? Is it simple? Then try it again, but this time with your
eyes closed! Can you do that? Why can’t you do it? Because there is little
feedback information when your eyes are closed! By now I hope you would
appreciate the importance of the feedback principle. Now let’s see if we can
make a machine to imitate what we can do.

Stabilization or Set-point Control :

In most of the control problem in industry, the objective is to make the
overall system asymptotic stable by proper design of the controller, such
that any deviation from the desired goal will be corrected. This will also be
the central topic of this module.

14
Since equation (1.2) is just a second-order differential equation, why don’t
we try to solve it directly? Would it be wonderful that we can work out the
solution to this ODE? If the solution is available to us, perhaps it can help
us in designing the control system.

Is there analytical solution to this problem? Unfortunately, the answer is

No. This is a nonlinear system, and normally, it is very hard to find analyt-
ical solution to nonlinear differential equations.

Although we don’t know how to solve nonlinear equations in general, we do

know how to solve linear equations. And if we can turn the nonlinear system
into linear one by some magic, then the problem is finished! Now here is the
trick. Remember, you are free to choose any control signal in any way you
like (as long as feasible), right? Is it possible to design a feedback controller
such that the nonlinear term sin(θ(t)) disappears? From the equation (1.2),
we may simply let
u(t) = sin(θ(t))
and the closed loop system becomes

θ̈(t) = 0.

Although it becomes a linear system now, it is not stable! To further stabilize

it, we need to add extra terms like

u(t) = sin(θ(t)) − 1.4θ̇(t) − θ(t).

And we have

θ̈(t) + 1.4θ̇(t) + θ(t) = 0

But still it is not exactly what we want. We can easily find out that the
angle θ(t) would eventually converge to 0! But we want it to converge to π
(upright position)! Well, that is simple once you are at this stage. You just
add a feed-forward term, uf f (t) in your controller as

u(t) = sin(θ(t)) − 1.4θ̇(t) − θ(t) + uf f (t)

Plug it into equation (1.2), we have

θ̈(t) + 1.4θ̇(t) + θ(t) = uf f (t)

Now the nonlinear system becomes a well-behaved second order linear sys-
tem. Remember you still have freedom to use uf f (t). Simply choose a

15
constant uf f (t) = π. This trick used the idea of converting a nonlinear
system into linear ones, which actually is the most widely used technique to
study nonlinear systems. Of course, we don’t have time to study all those
tricks for nonlinear systems, we will limit our interests to linear systems in
our module.

How to implement a nonlinear controller is not an easy issue, especially for

analog controller. In many cases, we may have to use simple linear con-
troller rather than a nonlinear one to solve this problem. There is another
approach, which is also very commonly used —- Linearization! Through
linearization, we are still trying to turn the nonlinear systems into linear
systems. But this time, we are going to do it by approximation, which does
not involve manipulating the control input u(t) with the nonlinear terms at
all!

If only small deviations are of interest, we can linearize the system around
the equilibrium point.
For small θ − π,

sin θ = − sin(θ − π) = −(θ − π) + o(θ − π)

Define y = θ − π, we replace the nonlinear equation by the linear equation

as our object of study,
ÿ(t) − y(t) = u(t). (1.3)
Why linearization?

1. There is no general method to solve nonlinear equations, and linear

system is much easier to analyze.

2. The local behavior of the nonlinear system can be well approximated

by linear system. Just like any curve looks like a line locally. But does
it work? The good news is that we have the following linearization
principal.

Linearization principal:
Design based on linearizations works locally for the original nonlinear sys-
tem. Local means that satisfactory behavior only can be expected for those
initial conditions that are close to the point about which linearization was
made. The bad news is that the design may not work if the deviations from
the operating points are large.

16
 Figure 1.8: Superposition Principle

Why is linear system easier to analyze? What’s the fundamental difference

between linear and nonlinear systems?

Fundamental property of Linear System —-Superposition Princi-

ple
More specifically, suppose that the input signal u(t) is resolved into a set of
component time functions φ1 , φ2 ... by expressing u as an infinite series

u = a1 φ1 + a2 φ2 + ....
where the coefficients aλ , λ = 1, 2, ... , are constants representing the “weights”
of the component time functions φ1 , φ2 ... in u. Now, if the system is linear,
then the response of the system to input u, A(u), can be expressed as an
infinite series

A(u) = a1 A(φ1 ) + a2 A(φ2 ) + ....

Where A(φλ ) represents the response of the system to the component φλ , λ =
1, 2, ... . Thus, if φλ are chosen in such a way that the determination of
the response of system to input φλ is a significantly simpler problem than
the direct calculation of A(u), then it may be advantageous to determine
A(u) indirectly by

1. resolving u into the component functions φλ

2. calculating A(φλ ), λ = 1, 2, .

3. obtaining A(u) by summing the series

This basic procedure appears in various guises in many of the methods used
to analyze the behavior of linear systems. In general, the set of component

17
time functions φλ is a continuum rather than a countable set, and the sum-
mations are expressed as integrals in which λ plays the role of variable of
integration.
Remark: It is important to think of the integration as approximation by
summation for understanding many of the concepts and properties of the
system.
Example of application of superposition principle:
Choose component functions as est , where s = σ+jω is the complex variable.

Figure 1.9: The response function for exponential input est

Try the response function as H(s)est , and plug it into the pendulum Eqn.
(1.3), we have
s2 H(s)est − H(s)est = est (1.4)
we obtain
1
H(s) = . (1.5)
s2
−1
Hence H(s)est is indeed the response to est , which can be easily determined.
Consider the special case of unit step input, where s = 0, the response is
H(0). Therefore, the static (or steady state) gain is simply H(0).
Question: Can we decompose u(t) by est ?
Yes! The input u(t) can be decomposed by est
Z a+j∞
1
u(t) = F (s)est ds (1.6)
2πj a−j∞
But what is F (s)?
F (s) is the Laplace transform of u(t)!
Z ∞
L[u(t)] = F (s) = u(t)e−st dt (1.7)
0
Did you ever wonder why Laplace transform is such a wonderful tool for
analyzing linear system? It is rooted in the superposition principle!

There are many nice properties of Laplace transform, among which two of
them are the most often used.

18
 1. L[f (n) (t)] = sn L[f (t)] − sn−1 f (0) − sn−2 f 0 (0) − ... − f (n−1) (0)
Remark : if the initial conditions are zero, we simply replace the
differential operator d/dt with s:
d
⇔s
dt

2. L[f1 (t) ∗ f2 (t)] = L[f1 (t)] · L[f2 (t)]

where
Z∞
f1 (t) ∗ f2 (t) = f1 (τ )f2 (t − τ )dτ −−−− convolution
−∞

If f 1 and f 2 are one sided, ie. vanish for all t < 0, then
Zt
f1 (t) ∗ f2 (t) = f1 (τ )f2 (t − τ )dτ
0

Now let’s apply the Laplace transform to our example, denote

L[y(t)] = Y (s)
L[u(t)] = U (s)

Then
L[ÿ(t)] = s2 Y (s) − sy(0) − y 0 (0) (1.8)
Substitute them into Eqn. (1.3), we have

s2 Y (s) − sy(0) − y 0 (0) − Y (s) = U (s)

(s2 − 1)Y (s) = sy(0) + y 0 (0) + U (s)
1
Y (s) = 2 (sy(0) + y 0 (0) + U (s))
s −1
Remark : Note that Laplace transform magically turns the original differ-
ential Eq (1.3) into an algebraic equation that can be easily solved even by
middle school students!

Y (s) = H(s)(sy(0) + y 0 (0)) + H(s)U (s) (1.9)

But hold on, you may argue that this solution Y (s) does not look like any
other familiar solutions in time domain. In order to get the time-domain

19
solution, you may just transform it back by partial fraction expansion and
look them up in the Laplace transform table.
1 1 1 1 1
L−1 [H(s)] = L−1 [ ] = L−1 [ ( − )] = (et − e−t )
s2
−1 2 s−1 s+1 2
−1 −1 s −1 1 1 1 1
L [sH(s)] = L [ 2 ]=L [ ( + )] = (et + e−t )
s −1 2 s−1 s+1 2
Z t
1 t−τ
L−1 [H(s)U (s)] = h(t) ∗ u(t) = (e − e−(t−τ ) )u(τ )dτ
2
0

Overall, we obtain the solution of y(t) as

Zt
1 1 1 t−τ
y(t) = y 0 (0)(et − e−t ) + y(0)(et + e−t ) + (e − e−(t−τ ) )u(τ )dτ
2 2 2
0
Zt
1 1 1 t−τ
= (y 0 (0) + y(0))et + (y(0) − y 0 (0))e−t + (e − e−(t−τ ) )u(τ )dτ
2 2 2
0
(1.10)
If the initial conditions are zero, we have a very simple relationship between
the output and input,
Y (s) = H(s)U (s) (1.11)
It seems that the output Y(s) is simply transformation of the input U(s),
and we have a special name for this transformation H(s).

Transfer function — H(s)

Figure 1.10: Transfer Function

The transfer function is an alternative and convenient mathematical descrip-

tion of the system. But it is only applicable to linear systems since it is based
upon superposition principle. Given any differential equation, the transfer
function can be easily obtained by replacing the differential operator “d/dt”
with “s”. And given any transfer function, the corresponding differential
equation can be found by simply replacing “s” with “d/dt”. In addition to

20
this, transfer function also has other meanings.

What’s the relation between impulse response and transfer function? Im-
pulse function is a mathematical description of a signal which takes a huge
value for a very short period and then vanish elsewhere. For instance, im-
pulse function can be used to describe the force resulting from a hammer
hitting the table.

Now let the input be the impulse, u(t) = δ(t), then U (s) = 1, and Y (s) =
U (s)H(s) = H(s)
Hence the transfer function is the Laplace transform of the impulse response.
In other words, transfer function is the impulse response in the frequency
domain.
In general, consider

dn y dn−1 y dm u dm−1 u
an + a n−1 + ... + a 0 y = bm + bm−1 + ... + b0 u
dtn dtn−1 dtm dtm−1
(an sn + an−1 sn−1 + ... + a0 )Y (s) = (bm sm + bm−1 sm−1 + ... + b0 )U (s)
Y (s) = H(s)U (s)

P (s) bm sm + bm−1 sm−1 + ... + b0

H(s) = = (1.12)
Q(s) an sn + an−1 sn−1 + ... + a0
The roots of the denominator and the numerator of the transfer function
H(s) turn out to be very important for mathematical analysis of the system.

Poles: roots of Q(s)

The poles are important because the stability of the system can be deter-
mined by the following well known criterion (assuming you still remember
what you learned in undergraduate course on systems and control).

Stability criterion:
If all poles have negative real parts then the system is stable!

Why?
If λ is the pole, then eλt is one component of the impulse response (verified
by partial fraction expansion of the transfer function). So if all the poles
have the negative real parts, the impulse response will decrease to zero! But
what does impulse response have to do with stability?

21
Stability concerns the behavior of the system without external input when
the initial condition is perturbed from the equilibrium point, say zero. What
is the catch here?
If an impulse input is applied to a system in equilibrium position, this im-
pulse will certainly excite the states of the system from zero conditions (equi-
librium positions) into some non-zero values (non-equilibrium positions),
and then all the inputs are zero immediately after t = 0. Therefore, impulse
response can describe the behavior of the system with deviations from the
equilibrium points. Then the stability will depend upon whether the devia-
tions will be magnified or attenuated. If all the poles contain negative real
parts, then the impulse response will converge to zero since each component
eλt goes to zero. Hence the system is stable.

Zeros: roots of P (s)

The concept of zeros is useful for stability analysis for the inverse system
described by transfer function
1 Q(s)
= (1.13)
H(s) P (s)
There is also another reason for why they are called “zero”. Remember
that H(λ)eλt is the output of the system for input signal eλt . If λ is the
zero, input eλt will have no effect on the system, just like zero input. So
“zero” implies zero output for certain inputs. When do we want to use this
property? It can be used for disturbance rejection, i.e. the effect of the
disturbance on the output is minimized.

1.2.3 Open loop and feedback controllers

Let’s return to our control problem of the inverted pendulum. Since we
already obtained the solution of the approximated linear system, it makes
it easier for us to design controller.
What would happen without any control action? Would the pendulum
maintain its upright position without any control?

Control Goal
Our objective is to bring y and y 0 to zero, for any small nonzero initial con-
ditions, and preferably to do so as fast as possible, with few oscillations, and
without ever letting the angle and velocity become too large. Although this
is a highly simplified system, this kind of “servo” problem illustrates what

22
is done in engineering practice. One typically wants to achieve a desired
value for certain variables, such as the correct idling speed in an automo-
bile’s electronic ignition system or the position of the read/write head in a
disk drive controller.
But why do we require that the initial values are small? Because the lin-
earized model only works locally for nonlinear systems!

First attempt: open loop control – control without feedback

Since we have already obtained the solution, we can now try to solve the
control problem. How should we choose the control function u(t)? There is
no difficulty in choosing u(t) to dispose of any disturbance corresponding to
a particular initial conditions, y(0) and y 0 (0).

For example, assume we are only interested in the problem of controlling

the pendulum when starting from the initial position y(0) = 1 and velocity
y 0 (0) = −2. In this case, from solution (1.9), we have

s−2 U (s) s − 2 + U (s)

Y (s) = 2
+ 2 =
s −1 s −1 (s + 1)(s − 1)

If we choose U (s) = 1, then y(t) will go to zero. But the corresponding

signal is impulse function, hard to implement precisely. Let’s instead try

u(t) = 3e−2t

or
3
U (s) =
s+2
Then
1 3
Y (s) = (s − 2) + 2
s2 − 1 (s − 1)(s + 2)
(s − 2)(s + 2) + 3
=
(s2 − 1)(s + 2) (1.14)
1
= ⇒ e−2t
s+2

It is certainly true that y(t) and its derivative approach zero, actually rather
quickly. It looks like the control problem is solved satisfactorily. However,

23
there is one serious problem with this control method. If we made any mis-
takes in estimating the initial velocity, the control result will be disastrous.

For instance, if the differential equation is again solved with the same control
input, but now using instead the initial conditions:

y(0) = 1, y 0 (0) = −2 + ε

Y (s) = H(s)(sy(0) + y 0 (0)) + H(s)U (s)

1 3
= 2 (s − 2 + ε) + 2
s −1 (s − 1)(s + 2)
ε 1 (1.15)
= 2 +
s −1 s+2
ε 1 1 1 ε ε
= ( − )+ ⇒ et − e−t + e−2t
2 s−1 s+1 s+2 2 2
It is obvious that no matter how small ε is, the solution will diverge to
infinity, namely,
lim y(t) = ∞
t→∞

What about the other types of open-loop control?

Figure 1.11: Open Loop Control

No matter what type of C(s) is used, the unstable pole at s = 1 cannot be

eliminated. Then you may argue how about choosing C(s) = (s−1)/(s+2)?
That is an interesting solution! This raises the following question: Can un-
stable poles be canceled out by the zeros?
1
Let’s just examine a simple example with transfer functions H1 (s) = s+1
s−1
and H2 (s) = (s+1)(s−1) . Are these two transfer functions equivalent?
The simplest way to test it out is to write down the corresponding differ-
ential equations and set all the inputs to be zero, and examine whether the
solutions to the two differential equations are the same or not.
1
Then from H1 (s) = s+1 , we have

dy
+y =0
dt

24
 s−1
and from H2 (s) = (s+1)(s−1) , we have

d2 y
−y =0
dt2
Obviously one solution will converge to zero, and the other solution will
blow up to infinity!
I hope at this moment you are certain that the unstable poles cannot be
simply canceled out by zeros. And we can conclude that the open loop
control can not achieve the goal very well. What we need is a controller
such that the system returns to equilibrium position rapidly regardless of
the nature of the small perturbations, or the time at which it occurs.
One way to accomplish this is to take u(t) not to depend upon the time t,
but rather upon the state of the system, y and y 0 . Thus, we write

u = g(y, y 0 ) (1.16)

This is exactly a feedback controller: the control inputs depend upon the
output, not upon time! This also how you keep your pen upright in your
palm. You watch the pen (measure and compare) and move your hand
accordingly (control). Again, we can see control theory is not about how to
solve the differential equations, but on how to apply the idea of feedback.

Second attempt: Proportional control – the simplest feedback con-

trol
A naive first attempt using the idea of feedback would be as follows: If we
are to the left of the vertical, that is, if y > 0, then we wish to move to the
right, and therefore we apply a negative torque. If instead we are to the
right, we apply a positive torque. In other words, we apply proportional
feedback
u(t) = −Kp y(t) (1.17)
where Kp is some positive real number, the feedback gain.
What is the transfer function of the closed loop system?

Kp s21−1 Kp
Hcl (s) = Kp
=
1+ s2 − 1 + Kp
s2 −1
Where are the poles? p
s = ± 1 − Kp (1.18)

25
 Figure 1.12: Proportional Control

1. If Kp < 1, then all of the solutions diverge to infinity.

2. If Kp = 1, then it becomes a double integrator, which is unstable.

3. If Kp > 1, then the solutions are all oscillatory.

Therefore, in none of the cases is the system guaranteed to approach the

desired position. We need to add damping to the system. We arrive then
at a PD, or

Third attempt: Proportional and Derivative Control

Consider the resulting transfer function of the closed-loop system,

(Kp + Kd s) s21−1 Kp + Kd s
Hcl (s) = = (1.19)
1+
(Kp +Kd s) s2 + Kd s + Kp − 1
s2 −1

Then is it possible to choose the gains Kp and Kd such that the overall
system is asymptotically stable?
It is evident that as long as Kp > 1 and Kd > 0, the closed loop will be
stable. Further more, the poles of the closed loop can be placed at any
desired position by choosing proper gains.
We conclude from the above discussion that through a suitable choice of the
gains Kp and Kd it is possible to attain the desired behavior, at least for

26
 Figure 1.13: Proportional and Derivative Control

the linearized model. That this same design will still work for the original
nonlinear model is due to what is perhaps the most important fact in control
theory – and for the matter in much of mathematics – namely that first or-
der approximations are sufficient to characterize local behavior. Informally,
we have the following linearization principle:
Design based on linearizations works locally for the original nonlinear sys-
tem.
In the next section, we will see how to solve the control problem using a
different approach.

1.3 State-Space approach

For many of you, I believe that today is not the first time you hear of the
concept of state for dynamic system. What is the state of a system?
Historical background: the concept of state has been around since the
end of 19th century, (Poincare introduced it to study dynamical systems)
but it was only till the late 1950s that it was introduced into the control
theory. And it marks the new era of control theory and modern control
theory usually means the control theory formulated in the state space. And
classical control theory means the transfer function approach.

State at time t (present):

27
 Figure 1.14: State at time t

state at time t0 (present) separates the future from the past by providing
all the information about the past of the system that is relevant to the de-
termination of the response of the system to any input starting from t0 . In
other words, the future of the system can be completely determined by the
knowledge of current state combining with the present and future inputs.
To find the right state variable, basically we have to answer the following
question: What information is really needed for predicting the future as-
suming we have the complete information about the past and the present of
the system combining with the inputs information?

1.3.1 Examples
Pendulum problem
ÿ(t) − y(t) = u(t) (1.20)
At present time t0 , what do we have to specify in order to determine y(t)
in the future? Obviously the knowledge of the initial condition y(0), y 0 (0)
would suffice. So the state variable of the pendulum at any time t can be
defined as
x(t) = [y(t), y 0 (t)] (1.21)
Order: The dimension of the state. In the pendulum case, it is second
order system.

Perfect delay problem

y(t) = y(t − d) (1.22)
At time t0 , state x(t0 ) = y[t0 −d,t0 ] , since the value of y(t) after t0 is
completely determined by state x(t0 ). The order of the system is infinity!
Since the state of the system is specified by the output signal for the period
of [t0 − d, t0 ]. This is one of the reasons for the difficulty of dealing with

28
time-delay in continuous time system.
Question: what is the state for the pendulum with delayed input as shown
below?
ÿ(t) − y(t) = u(t − d)

1.3.2 Properties
By definition of state we have following map:

x(t) = x(x(t0 ); u[t0 ,t] )

(1.23)
y(t) = y(x(t0 ); u[t0 ,t] )

There are many properties associate with the concept of state, and the most
important is the

Separation property

Figure 1.15: Separation property

The present state separates the past from the future, and the future only
depends upon the present state. In other words, all the past information
relevant for determining the future of the system is compacted into the cur-
rent state.

More specifically, for any input consisting of a segment u[t0 ,t1 ] followed by a
segment u[t1 ,t] , the response of the system to the whole segment u[t0 ,t] start-
ing from initial state x(t0 ) consists of the response segment Y (x(t0 ); u[t0 ,t1 ] )
followed by the response segment Y (x(t1 ); u[t1 ,t] ), where x(t1 ) is the state
of the system at time t1 . x(t1 ) in a sense separates the past response
Y (x(t0 ); u[t0 ,t1 ] ) from the future response Y (x(t1 ); u[t1 ,t] ) at time t1 . And

x(t) = x(x(t0 ); u[t0 ,t] ) = x(x(t1 ); u[t1 ,t] ) = x(x(x(t0 ); u[t0 ,t1 ] ); u[t1 ,t] ) (1.24)

29
Decomposition property of linear system

Figure 1.16: System Response= zero-input response + zero-state response

x(t) = x(x(t0 ); u[t0 ,t] ) = x(x(t0 ); 0) + x(0; u[t0 ,t] )

(1.25)
y(t) = y(x(t0 ); u[t0 ,t] ) = y(x(t0 ); 0) + y(0; u[t0 ,t] )
If the system is stable, then the zero-input response can be safely ignored
for steady state behavior because this component will decay to zero in the
long run.

Let’s return to our pendulum example.

Using the state vector x(t) = [y(t), ẏ(t)], we can derive the state-space rep-
resentation of the pendulum.

ẋ1 (t) = x2 (t)

ẋ2 (t) = x1 (t) + u(t)
y(t) = x1 (t)

or in matrix form
ẋ = Ax + bu
(1.26)
y = cx
where " #
0
 
0 1  
A= , b= , c= 1 0
1 0 1
Note: the original second order differential equation has been transformed
into a first order differential equation of state vectors.

In general, most of the linear systems can be described by equation (1.26)

If x is a scalar, then the solution to above equation can be easily obtained

30
by Laplace transform as follows.
sX(s) − x(0) = AX(s) + bU (s)
1 bU (s)
X(s) = x(0) +
s−A s−A
Zt
At
x(t) = e x(0) + eA(t−τ ) bu(τ )dτ
0
(1.27)
Zt
y = cx = ceAt x(0) + c eA(t−τ ) bu(τ )dτ
0
Now if x is a vector, can we still write the solution in such forms?
Fortunately, even if x is a vector, the solution still takes above form, where
tk
eAt = I + At + ... + Ak + ... (1.28)
k!

How to calculate eAt ?

Using the infinite series obviously is not an option.
To calculate eAt , let’s see if Laplace transform can do the trick again.
sX(s) − X(0) = AX(s) + bU (s)
Y (s) = CX(s)
we have
(sI − A)X(s) = X(0) + bU (s)
X(s) = (sI − A)−1 X(0) + (sI − A)−1 bU (s)
Y (s) = C(sI − A)−1 X(0) + C(sI − A)−1 bU (s)
Compared to
Zt
At
x(t) = e x(0) + eA(t−τ ) bu(τ )dτ
0
Zt
y = cx = ceAt x(0) + c eA(t−τ ) bu(τ )dτ
0
it is obvious that (sI − A)−1 is eAt
in the s-domain. Therefore eAt can be
easily obtained by inverse Laplace transform of (sI − A)−1 .
There are some other ways to compute eAt . Using Laplace transform is the
easiest way.

31
What’s the corresponding transfer function?
Setting X(0) = 0,

We have
Y (s) = H(s)U (s)
And
CB(s)b
H(s) = C(sI − A)−1 b = (1.29)
d(s)

1.3.3 State-feedback controller

Input depends upon the states, not on time.

u = g(x) (1.30)

If we use the linear state-feedback controller:

u = −kx = −k1 x1 − k2 x2 + ... (1.31)

The closed loop equations are:

ẋ = Ax + bu = Ax − bkx = (A − bk)x
(1.32)
y = cx
The poles of the system can be determined by solving

det(sI − (A − bk)) = 0 (1.33)

Now the question becomes an algebraic one: given pair (A, b), can we choose
proper k such that the eigenvalues of (A − bk) can match any set of n eigen-
values?

We will examine this question in greater detail later in this module. Let’s
just take a look at the pendulum example.

The pendulum example:

" #
0
 
0 1
A= , b=
1 0 1

If we choose to use the state feedback controller as

u = −k1 x1 − k2 x2 = −kx (1.34)

32
where  
k= k1 k2
Remark: the previous PD (proportional and derivative) controller is the
same as this, just with a different notation.
Plug this control input into the equation, we have

ẋ = Ax + bu = (A − bk)x (1.35)

where  
0 1
A − bk =
1 − k1 −k2
The characteristic equation of the closed loop systems

det(sI − (A − bk)) = s2 + k2 s + (k1 − 1) (1.36)

So if the desired characteristic equation is

s2 + 2ζωn s + ωn2 (1.37)

where ζ and ωn are the desired damping ratio and natural frequency of the
closed loop system.
Just let
k2 = 2ζωn
(1.38)
k1 − 1 = ωn2

1.4 Computer Control System

In all the previous discussion, the system is continuous-time. All the signals
are usually handled by analog technology. If we want to implement the con-
troller using the digital computers which are discrete in nature, we have to
sample the signals of the continuous-time system.

There are many issues involved such as:

• How do we sample the continuous-time system? what are the re-
lated issues? After sampling, the continuous-time system will become
discrete-time system.

• What are the analytical tools for discrete-time system?

• Is there a similar transform for discrete-time system just as the Laplace

transform for continuous time?

33
 • What is the transfer function for discrete-time system?

• What is the meaning of poles and zeros?

• How to determine the stability?

All the issues will be discussed in the future lectures.

1.5 Notes and References

Most of the materials covered in this chapter can be found in following
references:

1. R. Bellman, Introduction to the Mathematical Theory of Control Pro-

cesses, vol. 1, Academic Press, 1967.

2. L. A. Zadeh and C. A. Desoer, Linear System Theory, McGraw-Hill,

1963.

3. E. D. Sontag, Mathematical Control Theory, second edition, Springer-

Verlag, 1998.

34
Chapter 2

Computer Control and

Discrete-Time Systems

2.1 Introduction
What is Computer Control? Literally, the answer is simple. The controller
is implemented by a computer and the input is calculated by the controller!

Practically all control systems that are implemented today are based on com-
puter control. It is therefore important to understand computer-controlled
systems well. Such systems can be viewed as approximations of analog-
control systems, but this is a poor approach because the full potential of
computer control is not used. At best the results are only as good as
those obtained with analog control. It is much better to master computer-
controlled systems, so that the full potential of computer control can be
used. There are also phenomena that occur in computer-controlled sys-
tems that have no correspondence in analog systems. It is important for
an engineer to understand this. The main goal of this module is to provide
a solid background for understanding, analyzing, and designing computer-
controlled systems.

A computer-controlled system can be described schematically as in Fig. 2.1.

The output from the process y(t) is a continuous-time signal. Can you di-
rectly feed a continuous-time signal into a computer? Of course not! The
computer can only deal with digits. That’s why you need the A-D con-
verter. The output is converted into digital form by the analog-to-digital
(A − D) converter. The A − D converter can be included in the computer or

35
 Figure 2.1: Schematic diagram of a computer-controlled system.

regarded as a separate unit, according to one’s preference. The conversion

is done at the sampling instants, tk . The time between successive samplings
is called the sampling period and is denoted by h. Periodic sampling is nor-
mally used, but there are, of course, many other possibilities. For example,
it is possible to sample when the output signals have changed by a certain
amount. It is also possible to use different sampling periods for different
loops in a system. This is called multirate sampling. How to choose the
sampling period properly is a big issue, which you can find out more in the
textbooks.

The computer interprets the converted signal, y(tk ), as a sequence of num-

bers, processes the measurements using an algorithm, and gives a new se-
quence of numbers, u(tk ). This sequence is converted to an analog signal by
a digital-to-analog (D − A) converter. The events are synchronized by the
realtime clock in the computer. The digital computer operates sequentially
in time and each operation takes some time. The D-A converter must, how-
ever, produce a continuous-time signal. This is normally done by keeping
the control signal constant between the conversions, which is called Zero-
order Hold.

Between the sampling instants, the input would be a constant due to zero-
order hold. Now an interesting question is: is the controller open-loop,
or feedback control between the sampling instants? Since the input is a
constant regardless of what happens in the output during this period, the
system runs open loop in the time interval between the sampling instants.

The computer-controlled system contains both continuous-time signals and

sampled, or discrete-time, signals. The mixture of different types of signals

36
Figure 2.2: A system for controlling the position of the arm of a disk drive.

sometimes causes difficulties. In most cases it is, however, sufficient to de-

scribe the behavior of the system at the sampling instants. The signals are
then of interest only at discrete times. Such systems will be called discrete-
time systems. Discrete-time systems deal with sequences of numbers, so a
natural way to represent these systems is to use difference equations.

Using computers to implement controllers has substantial advantages. Many

of the difficulties with analog implementation can be avoided. For example,
there are no problems with accuracy or drift of the components. It is very
easy to have sophisticated calculations in the control law, and it is easy to
include logic and nonlinear functions. Tables can be used to store data in
order to accumulate knowledge about the properties of the system. It is also
possible to have effective user interfaces.

We will give examples that illustrate the differences and the similarities of
analog and computer-controlled systems. It will be shown that essential new
phenomena that require theoretical attention do indeed occur.

2.1.1 A Naive Approach to Computer-Controlled Systems

We may expect that a computer-controlled system behaves as a continuous-
time system if the sampling period is sufficiently small. This is true under
very reasonable assumptions. We will illustrate this with an example.

Example 2.1 Controlling the arm of a disk drive

A schematic diagram of a disk-drive assembly is shown in Fig. 2.2.

Let J be the moment of inertia of the arm assembly. The dynamics re-
lating the position y of the arm to the voltage u of the drive amplifier is
approximately described by the Newton’s law,
J ÿ(t) = u(t)

37
the corresponding transfer function is
k
G(s) = (2.1)
Js2
where k is a constant. The purpose of the control system is to control the
position of the arm so that the head follows a given track and that it can be
rapidly moved to a different track. It is easy to find the benefits of improved
control. Better trackkeeping allows narrower tracks and higher packing den-
sity. A faster control system reduces the search time. In this example we
will focus on the search problem, which is a typical servo problem. Let uc ,
be the command signal and denote Laplace transforms with capital letters.
A simple servo controller can be described by
bK s+b
U (s) = Uc (s) − K Y (s) (2.2)
a s+a
Here we ignore the details on the procedure of how to design above analog
controller due to time limitation. If the controller parameters are chosen as

a = 2ω0

b = ω0 /2
Jω02
K=2
k
a closed system with the characteristic polynomial

F (s) = s3 + 2ω0 s2 + 2ω02 s + ω03

is obtained. This system has a reasonable behavior with a settling time to

5% of 5.52/ω0 . See Fig. 2.3.

Now the question is how to convert this analog controller to computer con-
trol system? To obtain an algorithm for a computer-controlled system, the
control law given by (2.2) is first written as

bK a−b b
U (s) = Uc (s) − KY (s) + K Y (s) = K( Uc (s) − Y (s) + X(s))
a s+a a
This control law can be written as
b
u(t) = K( uc (t) − y(t) + x(t)) (2.3)
a

38
Figure 2.3: Simulation of the disk arm servo with analog (dashed) and
computer control (solid). The sampling period is h = 0.2/ω0 .

where
dx
= −ax + (a − b)y
dt
How to calculate x(t)? The derivative dx/dt may be simply approximated
with a difference. This gives

x(t + h) − x(t)
= −ax(t) + (a − b)y(t)
h
The following approximation of the continuous algorithm (2.3) is then ob-
tained:
b
u(tk ) = K( uc (tk ) − y(tk ) + x(tk ))
a
x(tk + h) = x(tk ) + h((a − b)y(tk ) − ax(tk ))
Arm position y is read from an analog input. Its desired value is assumed
to be given digitally. The algorithm has one state, variable x, which is up-
dated at each sampling instant. The control law is computed and the value
is converted to an analog signal. The program is executed periodically with
period h by a scheduling program, as illustrated in Fig. 2.2.

Because the approximation of the derivative by a difference is good if the

interval h is small, we can expect the behavior of the computer-controlled
system to be close to the continuous-time system. This is illustrated in Fig.
2.3, which shows the arm positions and the control signals for the systems
with h = 0.2/ω0 . Notice that the control signal for the computer-controlled

39
Figure 2.4: Simulation of the disk arm servo with computer control having
sampling rates (a)h = 0.5/ω0 and (b)h = 1.08/ω0 . For comparison, the
signals for analog control are shown with dashed lines.

system is constant between the sampling instants. Also notice that the
difference between the outputs of the systems is very small. The computer-
controlled system has slightly higher overshoot and the settling time to 5%
is a little longer, 5.7/ω0 instead of 5.5/ω0 . The difference between the sys-
tems decreases when the sampling period decreases. When the sampling
period increases the computer-controlled system will, however, deteriorate.
This is illustrated in Fig. 2.4, which shows the behavior of the system for
the sampling periods h = 0.5/ω0 and h = 1.08/ω0 . The response is quite
reasonable for short sampling periods, but the system becomes unstable for
long sampling periods.

We have thus shown that it is straightforward to obtain an algorithm for

computer control simply by writing the continuous-time control law as a
differential equation and approximating the derivatives by differences. The
example indicated that the procedure seemed to work well if the sampling
period was sufficiently small. The overshoot and the settling time are, how-
ever, a little larger for the computer-controlled system.

2.1.2 Deadbeat Control

Example 2.1 seems to indicate that a computer-controlled system will be
inferior to a continuous-time system. We will now show that this is not
necessarily the case. The periodic nature of the control actions can be

40
Figure 2.5: Simulation of the disk arm servo with deadbeat control (solid).
The sampling period is h = 1.4/ω0 . The analog controller from Example 1.2
is also shown (dashed).

actually used to obtain control strategies with superior performance.

Example 2.2 Disk drive with deadbeat control

Consider the disk drive in the previous example. Fig. 2.5 shows the be-
havior of a computer-controlled system with a very long sampling interval
h = 1.4/ω0 . For comparison we have also shown the arm position, its veloc-
ity, and the control signal for the continuous controller used in Example 2.1.
Notice the excellent behavior of the computer-controlled system. It settles
much quicker than the continuous-time system. The 5% settling time is
2.34/ω0 , which is much shorter than the settling time 5.5/ω0 of the continu-
ous system. The output also reaches the desired position without overshoot
and it remains constant when it has achieved its desired value, which hap-
pens in finite time. This behavior cannot be obtained with continuous-time
systems because the solutions to such systems are sums of functions that
are products of polynomial and exponential functions. The behavior ob-
tained can be also described in the following way: The arm accelerates
with constant acceleration until it is halfway to the desired position and it
then decelerates with constant retardation. The example shows that control
strategies with different behavior can be obtained with computer control.
In the particular example the response time can be reduced by a factor of

41
Figure 2.6: Block diagram of a continuous-time system connected to A − D
and D − A converters.

2. The control strategy in Example 2.2 is called deadbeat control because

the system is at rest when the desired position is reached. Such a control
scheme cannot be obtained with a continuous-time controller. We will dis-
cuss how to design deadbeat controller in greater details later in this course.
Therefore, we cannot simply regard computer-control system as an approx-
imation of the analog controller. It is more powerful than analog controller,
and that’s why we need to study it as a separate subject for one semester!

2.2 Sampling a Continuous-Time State-Space Sys-

tem
The computer receives measurements from the process at discrete times and
transmits new control signals at discrete times. The goal then is to describe
the change in the signals from sample to sample and disregard the behavior
between the samples. The use of difference equations then becomes a natural
tool. It should be emphasized that computer-oriented mathematical models
only give the behavior at the sampling points, and the physical process is
still a continuous-time system. Looking at the problem this way, however,
will greatly simplify the treatment.
Assume that the continuous-time system is given in the following state-space
form:
dx
= Ax(t) + Bu(t)
dt (2.4)
y(t) = Cx(t) + Du(t)
The system has r inputs, p outputs, and is of order n.
A common situation in computer control is that the D − A converter is so
constructed that it holds the analog signal constant until a new conversion
is commanded. This is often called a zero-order-hold circuit. Given the

42
state at the sampling time tk , the state at some future time t is obtained by
solving (2.4). The state at time t, where tk ≤ t ≤ tk+1 , is thus given by
Z t
0
x(t) = e A(t−tk )
x(tk ) + eA(t−s ) Bu(s0 )ds0
tk
Z t
0
= eA(t−tk ) x(tk ) + eA(t−s ) ds0 Bu(tk )
tk (2.5)
Z t−tk
= eA(t−tk ) x(tk ) + eAs dsBu(tk )
0
= Φ(t, tk )x(tk ) + Γ(t, tk )u(tk )
The second equality follows because u is constant between the sampling
instants. The state vector at time t is thus a linear function of x(tk ) and
u(tk ). The system equation of the sampled system at the sampling instants
is then

x(tk+1 ) = Φ(tk+1 , tk )x(tk ) + Γ(tk+1 , tk )u(tk )

(2.6)
y(tk ) = Cx(tk ) + Du(tk )
where
Φ(tk+1 , tk ) = eA(tk+1 −tk )
Z tk+1 −tk
Γ(tk+1 , tk ) = eAs dsB
0
The relationship between the sampled signals thus can be expressed by the
linear difference equation (2.6). The model in (2.6) is therefore called a
zero-order-hold sampling of the system in (2.4). The system in (2.6) can
also be called the zero-order-hold equivalent of (2.4).
The state vector at times between sampling points is given by (2.5). This
makes it possible to investigate the intersample behavior of the system.
Notice that the responses between the sampling points are parts of step re-
sponses, with initial conditions, for the system. This implies that the system
is running in open loop between the sampling points.

For periodic sampling with period h, we have tk = kh and the model of (2.6)
simplifies to the time-invariant system

x(kh + h) = Φx(kh) + Γu(kh)

(2.7)
y(kh) = Cx(kh) + Du(kh)

43
where
Φ = eAh
Z h (2.8)
Γ= eAs dsB
0

By the way, how to get eAh ?

Use Laplace transform to get eAt first! Check
out section 1.3.2 in your lecture notes!

2.3 State-space Model of Discrete-Time Systems

In most of the remaining part of this course we will disregard how the
difference equation representing the discrete-time system has been obtained.
Instead we will concentrate on the properties of difference equations. Time-
invariant discrete-time systems can be described by the difference equation

x(k + 1) = Φx(k) + Γu(k)

(2.9)
y(k) = Cx(k) + Du(k)

For simplicity the sampling period is used as the time unit, i.e., h = 1.

2.3.1 Solution of the System Equation

To analyze discrete-time systems it is necessary to solve the system Eq (2.9).
Assume that the initial condition x(k0 ) and the input signals u(k0 ), u(k0 +
1), ... are given. How is the state then evolving? It is possible to solve (2.9)
by simple iterations.

x(k0 + 1) = Φx(k0 ) + Γu(k0 )

x(k0 + 2) = Φx(k0 + 1) + Γu(k0 + 1)
= Φ2 x(k0 ) + ΦΓu(k0 ) + Γu(k0 + 1)
..
. (2.10)
k−k0 k−k0 −1
x(k) = Φ x(k0 ) + Φ Γu(k0 ) + ... + Γu(k − 1)
k−1
X
= Φk−k0 x(k0 ) + Φk−j−1 Γu(j)
j=k0

The solution consists of two parts: One depends on the initial condition,
and the other is a weighted sum of the input signals.

44
2.4 Input-Output Models
A dynamic system can be described using either internal models or external
models. Internal models – for instance, the state-space models – describe
all internal couplings among the system variables. The external models give
only the relationship between the input and the output of the system. For
instance the ARMA (Auto-Regressive Moving Average) model,
n−1
X
y(k + 1) = ai y(k − i) + bi u(k − i) (2.11)
i=0

In order to describe such difference equation, it is convenient to introduce

the time shift-operator.

2.4.1 Shift-Operator calculus

The forward-shift operator is denoted by q. It has the property

qf (k) = f (k + 1) (2.12)

The inverse of the forward-shift operator is called the backward-shift oper-

ator or the delay operator and is denoted by q −1 . Hence

q −1 f (k) = f (k − 1) (2.13)

Operator calculus gives compact descriptions of systems and makes it easy

to derive relationships among system variables, because manipulation of dif-
ference equations is reduced to a purely algebraic problem.

The shift operator is used to simplify the manipulation of higher-order dif-

ference equations. Consider the equation

y(k+na )+a1 y(k+na −1)+· · ·+ana y(k) = b0 u(k+nb )+· · ·+bnb u(k) (2.14)

where na > nb . Use of the shift operator gives

(q na + a1 q na −1 + ... + ana )y(k) = (b0 q nb + ... + bnb )u(k) (2.15)

With the introduction of the polynomials

A(z) = z na + a1 z na −1 + ... + ana (2.16)

and
B(z) = b0 z nb + b1 z nb −1 + ... + bnb (2.17)

45
the difference equation can be written as

A(q)y(k) = B(q)u(k) (2.18)

Or in a more compact form

B(q)
y(k) = u(k) = H(q)u(k)
A(q)
and we have a name for H(q)—pulse-transfer operator. Table 2.1. lists
the pulse-transfer operators for various discrete-time systems obtained by
Zero-order hold sampling of continuous-time systems, G(s).

Eq (2.14) can be also expressed in terms of the backward-shift operator.

Notice that (2.14) can be written as

y(k)+a1 y(k−1)+...+ana y(k−na ) = b0 u(k−d)+...+bnb u(k−d−nb ) (2.19)

where d = na − nb is the pole excess of the system. The system in (2.14)

can be written as

A∗ (q −1 )y(k) = B ∗ (q −1 )u(k − d) (2.20)

where
A∗ (z −1 ) = A(z)/z na = 1 + a1 z −1 + · · · + ana z −na (2.21)
∗ −1 nb −1 −nb
B (z ) = B(z)/z = b0 + b1 z + · · · + bnb z (2.22)

2.5 The z-Transform

In the analysis of continuous-time systems the Laplace transform plays an
important role. The Laplace transform turns the differential equations into
algebraic ones, and makes it possible to introduce the transfer function and
the frequency interpretation of a system. The discrete-time analogy of the
Laplace transform is the z-transform — a convenient tool to study linear
difference equations.

Definition 2.1 z-Transform

Consider the discrete-time signal { f (k) : k = 0, 1, ... }. The z-transform of
f (k) is defined as
X∞
z{f (k)} = F (z) = f (k)z −k (2.23)
k=0

46
Table 2.1 Zero-order hold sampling of a continuous-time system, G(s). The
table gives the zero-order-hold equivalent of the continuous-time system,
G(s), preceded by a zero-order hold. The sampled system is described by
its pulse-transfer operator. The pulse-transfer operator is given in terms of
the coefficients of
b1 q n−1 + b2 q n−2 + ... + bn
H(q) =
q n + a1 q n−1 + ... + an

47
Table 2.1 continued

48
Table 2.2 Table of Laplace and Z Transforms

49
where z is a complex variable. It is interesting to note that the z-transform
bears a form similar to Laplace transform, in which the integration is re-
placed by summation now. The z-transform of f is denoted by Zf or F (z).

The inverse transform is given by

I
1
f (k) = F (z)z k−1 dz (2.24)
2πj
where the contour of integration encloses all singularities of F (z). How we
derive this inverse transform is not our concern here, and this inverse z-
transform is rarely used in practise just as the case that the inverse Laplace
transform is seldom used in the calculation of the time-domain signal. In-
stead, a more convenient way is simply by looking up the transform table.

Example 2.3 Transform of a ramp

Consider a ramp signal defined by y(k) = k for k ≥ 0. Then
z
Y (z) = 0 + z −1 + 2z −2 + ... = z −1 + 2z −2 + ...) =
(z − 1)2
The most important property of z-transform is,

Z{x(k + 1)} = zX(z) − zx(0).

Compare with the forward shift operator q{x(k)} = x(k + 1), we have fol-
lowing correspondence with zero initial condition,

Z −−−q
just as the correspondence in the continuous-time case:

s − − − d/dt

Other properties of the z-transform are collected in Table 2.3.

Table 2.3 Some properties of the z-transform.

1. Definition.
∞
f (k)z −k
P
F (z) =
k=0

2. Inversion.
1
F (z)z k−1 dz
H
f (k) = 2πj

50
 3. Linearity.
Z{αf + βg} = αZf + βZg

4. Time shift.
Z{q −n f } = z −n F
n−1
X
Z{q n f } = z n (F − F1 ) where F1 (z) = f (j)z −j
j=0

5. Initial-value theorem.
f (0) = lim F (z)
z→∞

6. Final-value theorem.
If (1 − z −1 )F (z) does not have any poles on or outside the unit circle,
then
lim f (k) = lim (1 − z −1 )F (z)
k→∞ z→1

7. Convolution. 
k

P
Z{f ∗ g} = Z f (n)g(k − n) = (Zf )(Zg)
n=0

Shift-Operator Calculus and z-transforms

There are strong formal relations between shift-operator calculus and calcu-
lations with z-transforms. When manipulating difference equations we can
use either. The expressions obtained look formally very similar. In many
textbooks the same notation is in fact used for both. The situation is very
similar to the difference between the differential operator p = d/dt and the
Laplace transform s for continuous-time systems.
The z-transform can be used to solve difference equations; for instance,

x(k + 1) = Φx(k) + Γu(k)

y(k) = Cx(k) + Du(k)

If the z-transform of both sides is taken,

z(X(z) − x(0)) = ΦX(z) + ΓU (z)

X(z) = (zI − Φ)−1 (zx(0) + ΓU (z))

and
Y (z) = C(zI − Φ)−1 zx(0) + (C(zI − Φ)−1 Γ + D)U (z)

51
 Figure 2.7: Steady state response of system H(z) to input ak

The transfer function can now be introduced.

H(z) = C(zI − Φ)−1 Γ + D (2.25)

The time sequence y(k) can now be obtained using the partial fraction ex-
pansion and looking up the z-transform table as shown in table 2.2.

Just as in the continuous-time case, the transfer function is an alternative

and convenient mathematical description of the system. Given any differ-
ence equation, the transfer function can be easily obtained by replacing the
time shift operator “q” with “z”. And given any transfer function, the cor-
responding difference equation can be found by simply replacing “z” with
“q”. For instance, The transfer functions of the systems listed in table 2.1.
can be obtained by simply replacing “q” with “z”.

It can be easily verified that H(a)ak is the steady state response of the
system H(z) subject to input ak as shown in Figure 2.7.

In the special case of unit step input (a = 1), the response is simply
H(1), which is the steady state gain, or, the static gain. Note that in
the continuous-time case, the static gain is given by H(0), which is different
from that of the discrete-time system.

2.6 Frequency Response for Discrete Systems

If a sinusoid at frequency ω, ejωt , is applied to a stable, linear time-invariant
continuous system described by the transfer function H(s), it can be shown
that the response is a transient (which decreases to zero quickly) plus a si-
nusoidal steady state response, H(jω)ejωt , as shown in Fig. 2.8.

52
Figure 2.8: Block Diagram of Frequency Response for Continuous System.

We can say almost exactly the same respecting the frequency response of
a stable, linear time-invariant discrete system. If the system has a transfer
function H(z), then its frequency response is H(ejωT ), where T is the sam-
pling period. If a sinusoid, ejωT k , is applied to the system, H(z), then in
the steady state, the response can be represented as H(ejωT )ejωT k as shown
in Fig. 2.9.

The frequency response can be easily derived using the knowledge that the
steady state response of the system, H(z), to the input signal, ak , is H(a)ak .
Therefore, consider the special case of sinusoid input ejωT k , where a = ejωT ,
the response is simply

H(a)ak = H(ejωT )ejωT k

We are going to use this knowledge when we deal with disturbances later in
this module.

Figure 2.9: Block Diagram of Frequency Response for Discrete System.

2.7 Transfer Function from Continuous Time to

Discrete Systems
The transfer function can be obtained from the state space model using the
formula (2.25). It can also be determined directly from the continuous-time
transfer function, G(s). We know that the transfer function is the impulse
response. Therefore what we need to do is just to figure out the impulse
response for the sampled system.
The unit impulse signal in discrete-time is δ(k). Due to zero-order hold,
the corresponding input signal in the continuous time is just a square pulse
signal as follows, (
1, t ∈ [0, h]
u(t) =
0, elsewhere

53
which can easily expressed as combination of two step input signals as

u(t) = U (t) − U (t − h)

The step response in continuous-time is simply G(s)

s (where G(s) is the trans-
fer function of the continuous-time process.) in s-domain. In time domain,
the step response can be obtained by inverse Laplace Transform as

G(s)
L−1 ( )
s
We need to get the step response in z-domain by z-transform as

G(s)
Z[L−1 ( )]
s
Therefore, the formula to convert continuous-time TF to discrete-time TF
is
G(s) G(s) G(s)
H(z) = Z[L−1 ( )]−z −1 Z[L−1 ( )] = (1−z −1 )Z[L−1 ( )] (2.26)
s s s
The table 2.1 lists the transformation formulae for the commonly used trans-
fer functions.

2.8 Poles and Zeros

For single-input-single-output finite-dimensional systems, poles and zeros
can be conveniently obtained from the denominator and numerator of the
transfer function. Poles and zeros have good system-theoretic interpreta-
tion. A pole z = a corresponds to a free mode of the system associated
1
with the time function z(k) = ak (corresponding to the component z−a in
the transfer function). Poles are also the eigenvalues of the system matrix Φ.

The zeros are related to how the inputs and outputs are coupled to the
states. Zeros can also be characterized by their signal blocking properties.
A zero z = a means that the transmission of the input signal u(k) = ak is
blocked by the system. In other words, the system does not respond to the
input signal ak at all if a is one of the zeros of the system.

54
Poles
As discussed in the first lecture, the stability concerns the behavior of the
system under initial deviations from the equilibrium point (zero states), and
can be examined from the impulse response. For the discrete-time system,
the unit impulse signal is δ(k), where δ(0) = 1, and δ(k) = 0, for k ≥ 1.
The corresponding z-transform is 1, hence the unit impulse response of the
system is Y (z) = H(z)U (z) = H(z). A pole z = a = ρejθ corresponds to a
component of the impulse response of the system associated with the time
function ak = ρk ejθk . Obviously, if the magnitude of the pole, |a| = ρ > 1,
this component will blow up, and the system is unstable. Otherwise, the
system will be stable.

When would the output be oscillatory? In continuous-time, we know that

the system would exhibit oscillation if the imaginary part of the pole is non-
zero. Similarly, since ak = ρk ejθk , the oscillation is definitely related to the
imaginary part. However, there is an extra twist here. Even if the pole is
purely negative real, the resulting output is also oscillatory! For instance, if
the pole is -1, then the corresponding free mode is {1, -1,1,-1, ...}.

Remark: Please note the difference between the stability criterion of the
discrete-time and continuous-time systems. In the continuous-time system,
since the corresponding component function is eat = eσt ejωt corresponding
to the pole of a = σ + ωj, the stability is determined by the real part of the
pole, σ. While in the discrete-time, the stability is decided by the magnitude
of the poles as shown above.

In the following we will study how the poles of the continuous-time systems
are transferred into poles of the discrete-time system under zero-order held.
Consider a continuous-time system described by the nth-order state-space
model
dx
= Ax + Bu
dt (2.27)
y = Cx
The poles of the system are the eigenvalues of A, which we denote by
λi (A), i = 1, ... n. The zero-order-hold sampling of (2.27) gives the discrete-
time system
x(k + 1) = Φx(k) + Γu(k)
y(k) = Cx(k)

55
 Figure 2.10: The conformal map z = exp(sh).

Its poles are the eigenvalues of Φ, λi (Φ), i = 1...n. Because Φ = exp(Ah) it

follows (please try to verify by yourself) that

λi (Φ) = eλi (A)h (2.28)

Eq (2.28) gives the mapping from the continuous-time poles to the discrete-
time poles. Fig. 2.10 illustrates a mapping of the complex s-plane into the
z-plane, under the map
z = esh
For instance, the left half of the s-plane is mapped into the unit disc of
the z-plane. The map is not bijective (one to one) — several points in the
s-plane are mapped into the same point in the z-plane (see Fig. 2.10).
Remark: please note that the map z = esh describes the relationship be-
tween the poles of the continuous-time system and its sampled system. It is
NOT the map between the transfer function of the continuous-time system
and its counter-part in the discrete-time!

56
Example 2.4
Consider the continuous-time system

ω02
(2.29)
s2 + 2ζω0 s + ω02

The poles of the corresponding discrete-time system are given by the char-
acteristic equation by looking up Table 2.1.

z 2 + a1 z + a2 = 0

where p
a1 = −2e−ζω0 h cos( 1 − ζ 2 ω0 h)
a2 = e−2ζω0 h
It can be easily verified that the poles of the continuous-time system is
p
s = −ζω0 ± j 1 − ζ 2 ω0

and the poles of the discrete-time system can be found by applying the map
z = esh , √
2
z = e−ζω0 h±j 1−ζ ω0 h

Zeros
Can we also find a straightforward map between the zeros of the continuous-
time and discrete-time systems? The answer is unfortunately NO. It is not
possible to give a simple formula for the mapping of zeros. The following ex-
ample shows that a continuous time system without any zeros may introduce
zeros to the corresponding discrete-time system by sampling.

Example 2.5 Second-order system

Consider the continuous-time transfer function
2
(s + 1)(s + 2)

Using Table 2.1 gives the zero of the transfer function

(1 − e−2h )e−h − 2(1 − e−h )e−2h

z=−
2(1 − e−h ) − (1 − e−2h )

57
Table 2.4 Numerator polynomials, Zd , when sampling the continuous-time
system s1d with sampling period h → 0.

Definition 2.3 Systems with stable and unstable inverses

A discrete-time system has a stable inverse if all the zeros are stable (inside
the unit disc, or on the unit disc for single ones). Otherwise, the system has
an unstable inverse.

We will show later in this course that the condition of stable inverse plays
a vital role in designing perfect tracking system.

A continuous-time with stable inverse may become a discrete-time system

with unstable inverse when it is sampled. For example, it follows from Table
2.4 that the sampled system always has an unstable inverse if the pole excess
of the continuous- time system is larger than 2, and if the sampling period
is sufficiently short.

Further, a continuous-time system with unstable inverse will not always be-
come a discrete-time system with unstable inverse, as shown in the following
example.

Example 2.6 stable and unstable inverse changes with sampling

The transfer function
6(1 − s)
G(s) =
(s + 2)(s + 3)
has an unstable zero s = 1, and it has an unstable inverse. Sampling the
system gives a discrete-time transfer function with a zero:
8e−2h − 9e−3h + e−5h
z1 = −
1 − 9e−2h + 8e−3h
For h ≈ 1.25, z1 = −1; for larger h, the zero is always inside the unit circle
and the sampled system has a stable inverse.

58
2.9 Notes and References
Most of the materials covered in Chapters 2 to 5, are extracted from the
main text book of this module:

• K. J. Astrom and B. Wittenmark, Computer-controlled Systems, The-

ory and Design, Prentice Hall, 1997.

59
Chapter 3

Analysis of Discrete - Time

Systems

3.1 Introduction
Chapter Two has shown how continuous-time systems are transformed into
discrete-time systems when sampled. Z-transform has also been introduced
to deal with the discrete-time system, which can easily transform the differ-
ence equations into algebraic equations. And transfer function can be readily
derived using z-transform. In this lecture we will continue to develop the
key tools for analyzing discrete-time systems. Stability is introduced first
in Sec 3.2. The concepts of controllability, observability and realizability,
which are useful for understanding discrete-time systems, are discussed in
Sec 3.3. Simple feedback loops and their properties are treated in Sec 3.4.

3.2 Stability
The stability of the systems concerns about the fundamental question: what
would happen if there is a small disturbance to the system?
The concept of stability is of primary importance for control system design.
The reason is very simple. For any control system design, stability has
to be guaranteed first before its practical implementation. Disaster would
follow if the system is unstable as any small deviations from the set point
or equilibrium position will lead to divergence.

60
3.2.1 Definitions
In the previous lectures, we have touched upon the concept of stability sev-
eral times without giving any precise definition. Now it is time to give a
mathematical definition for stability.
There are many types of stability defined in the literature. It is usually de-
fined with respect to changes in the initial conditions. Consider the discrete-
time state-space equation (possibly nonlinear and time-varying)

x(k + 1) = f (x(k), k) (3.1)

Let xo (k) and x(k) be solutions of (3.1) when the initial conditions are xo (0)
and x(0), respectively. Further, let denote k.k a vector norm.

Definition 3.1 Stability

The solution xo (k) of (3.1) is stable if for any given ε > 0, there exists a
δ(ε) > 0, such that all solutions with ||x(0) − xo (0)|| < δ are such that
||x(k) − xo (k)|| < ε for all k ≥ 0.

Definition 3.2 Asymptotic Stability

The solution xo (k) of (3.1) is asymptotically stable if it is stable and if δ
can be chosen such that ||x(0) − xo (0)|| < δ implies that ||x(k) − xo (k)|| → 0
when k → ∞.

From the definitions, it follows that stability in general is defined for a

particular solution and not for the system. The definitions also imply that
stability, in general, is a local concept. When we talk about stability, it
normally refers to stability around certain solutions such as set points and
equilibrium positions etc. Sometimes, the reference solution can also be
time-varying function. The interpretation of Definitions 3.1 and 3.2 is that
the system is (asymptotically) stable if the trajectories do not change much
if the initial condition is changed by a small amount, which is consistent
with our previous discussions.
In the following, we are going to apply this definition to linear systems,
which leads to something very special about the stability of linear systems.
It is no longer a local concept! It is global for linear systems!

61
3.2.2 Stability of Linear Discrete-Time Systems
Consider the linear system

xo (k + 1) = Φxo (k)
(3.2)
xo (0) = ao

To investigate the stability of the solution of (3.2), xo (k), the initial value
is perturbed. Hence
x(k + 1) = Φx(k)
x(0) = a
The deviation of the solutions, x̃ = x − xo , satisfies the equation

x̃(k + 1) = Φx̃(k)
(3.3)
x̃(0) = a − ao

Because the form of above equation (3.3) is independent of the reference

solution xo (k), it implies that if the solution xo (k) is stable, then every
other solution is also stable. For linear, time-invariant systems, stability is
thus a property of the system and not of a special solution.
The system (3.3) has the solution

x̃(k) = Φk x̃(0)

If it is possible to diagonalize Φ, then obviously the solution is a linear com-

bination of terms λki , where λi , i = 1, ... , n are the eigenvalues of Φ. In the
general case, when Φ cannot be diagonalized, the solution is instead a linear
combination of the terms λki , kλki ,...,k m−1 λki , where m is the multiplicity
of the corresponding eigenvalue λi (i.e, the eigenvalue is repeated roots of
the characteristic equation). If you are interested in the theoretical proof of
this, you may try to use the knowledge from Linear algebra regarding the
Jordan forms of matrices. For the time being, we will simply accept this
and move on. To get asymptotic stability, the deviations x̃(k) must go to
zero as k increases to infinity. The eigenvalues of Φ then have the property

|λi | < 1, i = 1, ... , n

If one of the eigenvalues |λi | > 1, then the term λki will blow up, and the
system is unstable.
What would happen if some of the eigenvalues lie on the unit disc, i.e.
|λi | = 1? Things will be more complicated and it depends upon whether

62
the corresponding eigenvalue is simple or multiple. If the eigenvalue is sim-
ple, then the corresponding component solution is λki with a magnitude of
one. In this case, the system is still stable (verify it by yourself!) , but
not asymptotic stable as it does not converge to zero. If the eigenvalue is
multiple (repeated m times) , there exist components like kλki ,...,k m−1 λki ,
which will increase to infinity.

Another way of expressing the solution of x̃(k) = Φk x̃(0) is using Z-transform.

It can be easily shown that the z-transform of Φk is z(zI − Φ)−1 . Then by
decomposing the solution through partial fraction expansion, we can reach
the same conclusion regarding the relationship between poles (eigenvalues)
and the stability as discussed above.

All the above discussions might be summarized into the following theorem.

Theorem 3.1 Stability of Linear Systems

Discrete-time linear time-invariant system (3.2) is stable if all eigenvalues of
Φ, i.e. poles, are either inside or on the unit disc, and all the eigenvalues
with unit magnitude are simple (not repeated poles). The system is asymp-
totically stable if all eigenvalues of Φ are strictly inside the unit disk.

It is noted that input signal is assumed to be zero or a constant in above

discussions on stability. As mentioned earlier, there are many types of sta-
bility. Stability with respect to disturbances in the initial value has already
been defined. Sometimes, we might be keen in knowing whether the inputs
and outputs are bounded or not.

3.2.3 Input-Output Stability

Definition 3.3 Bounded-Input Bounded-Output Stability
A linear time-invariant system is defined as bounded-input-bounded-output
(BIBO) stable if a bounded input gives a bounded output for every initial
value.

Since BIBO stability is not defined with respect to any particular refer-
ence solution, it is not a local concept. From the definition it follows that
asymptotic stability is the strongest concept. The following theorem is a
result.

63
Theorem 3.2 Relation between Stability Concepts
Asymptotic stability implies stability and BIBO stability for linear systems.
When the word stable is used without further qualification in this module,
it normally means asymptotic stability.

It is noted that for nonlinear systems, asymptotic stability might not imply
BIBO stability. For instance, consider the following example,

0.5x(k)
x(k + 1) =
(1 − u(k))

Obviously the system is asymptotic stable when the input is zero. But it is
unstable if input u(k)=0.8. Therefore the system is not BIBO stable.

It is easy to give examples showing that stability does not imply BIBO
stability even for linear systems, and vice versa.

Example 3.1 Harmonic oscillator

Consider the sampled harmonic oscillator
   
cos ωh sin ωh 1 − cosωh
x(k + 1) = x(k) + u(k)
− sin ωh cos ωh sin ωh


y(k) = 1 0 x(k)
The magnitude of the eigenvalues is one. The system is stable because
kx(k)k = kx(0)k if u(k) = 0, although not asymptotic stable. Let the input
be a square wave with the frequency ω rad/s. By using the z-transform, it
is easily seen that the output contains a sinusoidal function with growing
amplitude and the system is not BIBO stable. Fig. 3.1 shows the input
and output of the system. The input signal is exciting the system at its
undamped frequency and the output amplitude is growing, which is the
well-known phenomenon of resonant oscillation. Similar phenomenon exists
for the continuous -time system. What would happen if feeding a sinusoid
to the undamped system where the frequency of the input signal matches
the natural frequency? The system becomes unstable!

There are also simple examples that the system is BIBO stable, but not
stable. A good example is a ball rolling in the surface shown in Fig. 3.2.

64
Figure 3.1: Input and output of the system in Example 3.1 when ω = 1,
h = 0.5, and the initial state is zero.

65
 Figure 3.2: A system is BIBO stable, but not stable.

Obviously, the ball is unstable around the point A and B, but overall, it is
BIBO stable.
Since above example is a nonlinear system, you may still wonder whether
BIBO stability might imply stability for linear system. Consider the follow-
ing example    
0.5 0 1
x(k + 1) = x(k) + u(k)
1 2 1


y(k) = 1 0 x(k)
It can be easily verified that the system is unstable since it has a pole at 2.
But it is BIBO stable since the observation of the output y(k) cannot reveal
the dynamics of the second state variable x2 (k). Such kind of system is
unobservable (the information of the state cannot be recovered by observing
the inputs and outputs), which will be discussed in details later in this
module.

3.2.4 Stability Tests

It follows from Theorem 3.1 that a straightforward way to test the stability
of a given system is to calculate the eigenvalues of the matrix Φ. There are
good numerical algorithms for doing this. The routines are also included
in packages like MATLABr. The eigenvalues of a matrix then can be
calculated with a single command (eig in MATLAB).
It is, however, also important to have algebraic or graphical methods for
investigating stability. These methods make it possible to understand how
parameters in the system or the controller will influence the stability. The
following are some of the ways of determining the stability of a discrete-time
system:

• Direct numerical or algebraic computation of the eigenvalues of Φ

66
 • Methods based on properties of characteristic polynomials

• The root locus method

• The Nyquist criterion

• Lyapunov’s method
Explicit calculation of the eigenvalues of a matrix cannot be done conve-
niently by hand for systems of order higher than 2. In some cases it is easy
to calculate the characteristic equation

A(z) = a0 z n + a1 z n−1 + ... + an = 0 (3.4)

and investigate its roots. Stability tests can be obtained by investigating

conditions for the zeros of a polynomial to be inside the unit disc.
It is also useful to have algebraic or graphical conditions that tell directly if
a polynomial has all its zeros inside the unit disc. Such a criterion, which
is the equivalent of the Routh-Hurwitz criterion for the continuous-time
case, was developed by Schur, Cohn, and Jury. This test will be described
in detail in the following section.

3.2.5 Jury’s Stability Criterion

The following test is useful for determining if (3.4) has all its zeros inside
the unit disc. Form the table

where
ak−1
i = aki − αk akk−i
αk = akk /ak0
The first and second rows are the coefficients in Eq (3.4) in forward and
reverse order, respectively. The third row is obtained by multiplying the
second row by αn = an /a0 and subtracting this from the first row. The last
element in the third row is thus zero. The fourth row is the third row in
reverse order. The scheme is then repeated until there are 2n + 1 rows. The
last row consists of only one element. The following theorem results.

Theorem 3.3 Jury’s Stability Test

If a0 > 0, then Eq (3.4) has all roots inside the unit disc if and only if all
ak0 , k = 0, 1, ... , n − 1 are positive. If no ak0 is zero, then the number of
negative ak0 is equal to the number of roots outside the unit disc.

67
Example 3.2 Stability of a second-order system
Let the characteristic equation be

A(a) = z 2 + a1 z + a2 = 0 (3.5)

Jury’s scheme is
All the roots of Eq (3.5) are inside the unit circle if

1 − a22 > 0
1 − a2  
(1 + a2 )2 − a21 > 0
1 + a2
This gives the conditions 
a2 < 1

a2 > −1 + a1

a2 > −1 − a1


The stability area for the second-order equation is a triangle shown in Fig.
3.3.

68
Figure 3.3: The stability area for the second-order Eq (3.5) of the coefficients
a1 and a2 .

3.3 Controllability, Observability and Realizabil-

ity
In this section, two fundamental questions for dynamic systems are discussed
first. The first is whether it is possible to steer a system from a given initial
state to any other state. The second is how to determine the state of a
dynamic system from observations of inputs and outputs. Toward the end
of the section, we will also discuss how to construct a state space model for
a system whose behavior is described by input-output model.

3.3.1 Controllability
Definition 3.4 Controllability
The system is controllable if it is possible to find a control sequence such
that an arbitrary state can be reached from any initial state in finite time.
For nonlinear system, it is difficult to check whether the system is control-

69
lable or not. But for linear system, we have a neat result.

Consider the system

x(k + 1) = Φx(k) + Γu(k)
(3.6)
y(k) = Cx(k)

Assume that the initial state x(0) is given. The state at time n, where n is
the order of the system, is given by
x(n) = Φn x(0) + Φn−1 Γu(0) + ... + Γu(n − 1)
(3.7)
= Φn x(0) + Wc U
where
Φn−1 Γ


Wc = Γ ΦΓ · · ·

T
U= u(n − 1) · · · u(0)
Now the question regarding controllability is following: starting from arbi-
trary initial state x(0) is it possible to find control sequence U such that any

70
final state x(n) can be reached?

If Wc has rank n, then it is possible to obtain n equations from which the

control signals can be found such that the initial state is transferred to the
desired final state x(n).

But what would happen if Wc is not of full rank? Can we apply more inputs
to drive the state into the target state? If we try it in one more step by
applying input u(n), then we have

x(n + 1) = Φn+1 x(0) + Φn Γu(0) + Φn−1 Γu(1)... + Γu(n − 1)

= Φn+1 x(0) + W U

where
Φn Γ


W = Γ ΦΓ · · ·

T
U= u(n) · · · u(0)
If the rank of W is of full rank, then it is possible to find out the control
sequence. Unfortunately, the rank of W will be the same as that of Wc ! This
fact can be shown by the following famous Cayley-Hamilton theorem.

Cayley-Hamilton theorem
Given any n × n matrix A, and its characteristic equation

det[zI − A] = z n + a1 z n−1 + . . . + an

Then
An + a1 An−1 + . . . + an I = 0
In other words, An is a linear combination of An−1 , An−2 , . . . , A, I.
Applying Cayley-Hamilton theorem to our problem, it follows that Φn Γ
is a linear combination of Φn−1 Γ, Φn−2 Γ, . . . , Γ. Hence the rank of W cannot
be increased by applying more inputs!
The following theorem follows from the preceding definition and calculations.

Theorem 3.4 Controllability

The system (3.6) is controllable if and only if the matrix Wc has rank n.
Remark. The matrix Wc is referred to as the controllability matrix.

71
3.3.2 Controllable Canonical Form
Assume that Φ has the characteristic equation

det(λI − Φ) = λn + a1 λn−1 + ... + an = 0 (3.8)

and that Wc , is nonsingular. Then there exists a transformation, T , such

that the transformed system, z(k) = T x(k), is described by
   
−a1 −a2 · · · −an−1 −an 1
 1
 0 · · · 0 0 

 0 
 
z(k + 1) =  0
 1 ··· 0 0   z(k) +  0  u(k)
 
 .. .. .. .. ..   ..  (3.9)
 . . . . .   . 
0 0 ··· 1 0 0
y(k) = (b1 ... bn )z(k)

which is called the controllable canonical form. There are a number of

advantages in expressing the model in this form. First, it is easy to compute
the input-output model. It can be verified directly by z-transform that the
corresponding input-output model, i.e. transfer function, is determined by
the first row of elements in Φ̃ and C̃ as follows.
b1 z n−1 + ... + bn
H(z) =
zn + a1 z n−1 + ... + an
The second advantage is that the state feedback-control law can be easily
derived from controllable canonical form. We will elaborate this point later
when we discuss the control system design in the future.

But how to find out the transformation T , such that it can transform any
controllable system into its canonical form?

Assume that new coordinates are introduced by a nonsingular transforma-

tion matrix T

z(k) = T x(k) (3.10)

then

z(k + 1) = T x(k + 1) = T (Φx(k) + Γu(k)) = T ΦT −1 z(k) + T Γu(k) (3.11)

72
In the new coordinates,


W̃C = Γ̃ Φ̃Γ̃ · · · Φ̃n−1 Γ̃
T Γ T ΦT −1 T Γ · · · T Φn−1 T −1 T Γ


= (3.12)
= T Wc

For a single-input system the transformation matrix to the controllable

canonical form is
T = W̃c Wc−1
where W̃c is the controllability matrix for the representation (3.9).

3.3.3 Trajectory Following

From the preceding definitions and calculations, it is possible to determine
a control sequence such that a desired state can be reached after at most n
steps of time. Does controllability also imply that it is possible to follow a
given trajectory in the state space at every step? The answer is usually no.
Note that for a single-input-single-output system it is, in general, possible
to reach desired states only at each n-th sample point (not every step!),
provided that the desired points are known n steps ahead.

It is easier to make the output follow a given trajectory. Assume that the
desired trajectory is given by uc (k).
A naive approach would be the following. The control signal u then should
satisfy
B(q)
y(k) = u(k) = uc (k)
A(q)
or
A(q)
u(k) = uc (k) (3.13)
B(q)
Assume that there are d steps of delay in the system. The generation of u(k)
is then causal only if the desired trajectory is known d steps ahead. The
control signal then can be generated in real time. The control signal thus
is obtained by sending the desired output trajectory through the inverse
system A/B. The signal u is bounded if uc is bounded and if the system
has a stable inverse, i.e., B(z) is stable. Now you should realize that the
condition of stable inverse is important in tracking. But does y(k) really
track the command signal uc (k)? We have to check out the transfer function

73
 Figure 3.4: Trajectory Following

from the command signal to the output. It can be shown from the open loop
transfer function and the inverse controller (3.13), that

A(q)
y(k) = uc (k)
A(q)

Therefore we can conclude that another important condition for perfect

tracking is that the original system has to be stable, in which case the poles
and zeros can be perfectly canceled out.

Overall, if the system and its inverse are stable, it is possible to use very
simple open-loop control to follow any desired trajectory.

3.3.4 Observability
The concept of observability is regarding whether the state information can
be extracted from the observations of the inputs and output. This is an
important issue since only the inputs and outputs are measurable in many
practical situations. The system in (3.6) is observable if there is a finite
k such that knowledge of the inputs u(0), ... , u(k − 1) and the outputs
y(0), ... , y(k − 1) is sufficient to determine the initial state of the system.
Once the initial state information is obtained, the states at every step can
then be calculated.

Consider the system in (3.6). The effect of the known input signal always
can be determined, and there is no loss of generality to assume that u(k) = 0.
Assume that y(0), y(l), ... , y(n − 1) are given. This gives the following set

74
of equations:
y(0) = Cx(0)
y(1) = Cx(1) = CΦx(0)
..
.
y(n − 1) = CΦn−1 x(0)
Using vector notation gives
   
C y(0)
 CΦ   y(1) 
 x(0) =  (3.14)
   
 .. .. 
 .   . 
CΦn−1 y(n − 1)

The state x(0) can be obtained from (3.14) if and only if the observability
matrix  
C
 CΦ 
Wo =  (3.15)
 
.. 
 . 
CΦn−1
is of full rank. The state x(0) is unobservable if x(0) is in the null space
of Wo . If two states are unobservable, then any linear combination is also
unobservable; that is, the unobservable states form a linear subspace.

Theorem 3.5 Observability

The system (3.6) is observable if and only if Wo is of full rank.

Definition 3.5 Detectability

A system is detectable if the only unobservable states are such that they
decay to the origin. That is, the corresponding eigenvalues are stable.
Question: consider the example given at the end of section 3.2.3, which
shows that BIBO stability might not imply stability. Is this system de-
tectable?
In general, if the linear system is observable or detectable, then BIBO sta-
bility implies stability. If the system is undetectable, then the unobservable
state variables will blow up, which will not be revealed by the observation of
the inputs and outputs. Hence BIBO stability is not sufficient to guarantee
the stability of the system.

75
Example 3.3 A system with unobservable states
Consider the system
 
1.1 −0.3
x(k + 1) = x(k)
1 0
y(k) = (1 − 0.5)x(k)
The observability matrix is
   
C 1 −0.5
Wo = =
CΦ 0.6 −0.3
The rank of Wo is 1, and the unobservable states belong to the null space
of Wo , that is, [0.5 1]. Fig. 3.5 shows the output for four different initial
states. All initial states that lie on a line parallel to [0.5 1] give the same
output (see Fig. 3.5 (b) and (d)). Therefore, the initial state cannot be
determined uniquely by observing the inputs and outputs.

3.3.5 Observable Canonical Form

Similar to the controllable canonical form, there is also an observable canon-
ical form, which would make the control system design more convenient.
Assume that the characteristic equation of Φ is (3.8) and that the observ-
ability matrix Wo is nonsingular. Then there exists a transformation matrix
such that
the transformed system is
   
−a1 1 0 ··· 0 b1
 −a2 0 1 ··· 0   b2 
   
z(k + 1) =  .. .. .. . . .. ..
 z(k) +   u(k)
   
. . . . . .

 −an−1
   (3.16)
0 0 ··· 1   bn−1 
−an 0 0 ··· 0 bn


y(k) = 1 0 ··· 0 z(k)
which is called the observable canonical form. This form has the advantage
that it is easy to find the input-output model and to determine a suitable
observer. Following the similar argument as that in the section on control-
lability, the transformation in this case can be found as
T = W̃o−1 Wo
where Wo is the observability matrix for the representation (3.16).
Remark. The observable and controllable canonical forms are also called
companion forms.

76
3.3.6 Loss of Controllabilty and Observability Through Sam-
pling
Sampling of a continuous-time system gives a discrete-time system with sys-
tem matrices that depend on the sampling period. How will that influence
the controllability and observability of the sampled system? To get control-
lable discrete-time system, it is necessary that the continuous-time system
also be controllable, because the allowable control signals for the sampled
system-piecewise — constant signals — are a subset of the allowable control
signals for the continuous-time system.

However, it may happen that the controllability and/or observability is lost

for some sampling periods. The harmonic oscillator can be used to illustrate
the preceding discussion.

Example 3.4 Loss of controllability and observability

The discrete-time model of the harmonic oscillator is given by
   
cos ωh sin ωh 1 − cosωh
x(k + 1) = x(k) + u(k)
− sin ωh cos ωh sin ωh


y(k) = 1 0 x(k)
The determinants of the controllability and observability matrices are

det Wc = −2 sin ωh(1 − cos ωh)

and
det Wo = sin ωh
Both controllability and observability are lost for ωh = nπ, although the
corresponding continuous-time system is both controllable and observable.

The example shows one obvious way to lose observability and/or controlla-
bility. If the sampling period is half the period time (or a multiple thereof)
of the natural frequency of the system, then this frequency will not be seen
in the output. This happens only for isolated values of the sampling pe-
riod. A change in sampling period will make the system observable and/or
controllable again.

77
3.3.7 Realizability
The concepts of controllability and observability are defined only for state-
space model of the system. An interesting question would arise: how to
determine whether the system is controllable or observable if only the input-
output model, i.e. transfer function, is available? In order to address this
question, another important concept in control theory has to be introduced
— the realizability.
Given a sequence of observations of inputs and outputs, what possible sys-
tems could give rise to it? In other words, if we start with an input-output
model which describes how the inputs affect the outputs, how does one de-
duce the underlying state-space model that is responsible for this behavior?
Let’s first start with the simplest case — the first order system. Assume the
input-output model is

y(k + 1) = −a1 y(k) + b1 u(k)

then the natural choice of the state variable is

x(k) = y(k)

which leads to the state-space model

x(k + 1) = −a1 x(k) + b1 u(k)

This procedure is trivial. However, it has one important trick: one can
always choose the output y(k) as the state variable! Let’s follow this idea
to deal with the second-order system

y(k + 1) = −a1 y(k) − a2 y(k − 1) + b1 u(k) + b2 u(k − 1)

If we still let one of the state be the output,

x1 (k) = y(k)

then the input-output model becomes

x1 (k+1) = −a1 x1 (k)−a2 y(k−1)+b1 u(k)+b2 u(k−1) = −a1 x1 (k)+b1 u(k)−a2 y(k−1)+b2 u(k−1)

Note that above equation contains terms in the past step k-1, −a2 y(k − 1) +
b2 u(k − 1), which are not supposed to appear in the right hand of the stan-
dard form of the state-space model like equation (3.6), which only consists of
present state x(k), and input u(k). How to deal with the past terms? This

78
is the most important step! Once you understand this trick, then realization
problem would be trivial to you.

This trick is very simple: you just introduce another state variable which
summarizes all the past information!

Let
x2 (k) = −a2 y(k − 1) + b2 u(k − 1)
then we have
x1 (k + 1) = −a1 x1 (k) + b1 u(k) + x2 (k) = −a1 x1 (k) + x2 (k) + b1 u(k)
x2 (k + 1) = −a2 y(k) + b2 u(k) = −a2 x1 (k) + b2 u(k)
Now let’s try above procedure for the third order system
y(k +1) = −a1 y(k)−a2 y(k −1)−a3 y(k −2)+b1 u(k)+b2 u(k −1)+b3 u(k −2)
First step: let
x1 (k) = y(k)
and obtain the first equation
x1 (k+1) = −a1 x1 (k)+b1 u(k)−a2 y(k−1)−a3 y(k−2)+b2 u(k−1)+b3 u(k−2)
Second step: introduce one state variable to summarize the past information
x2 (k) = −a2 y(k − 1) − a3 y(k − 2) + b2 u(k − 1) + b3 u(k − 2)
then we have obtained two equations
x1 (k + 1) = −a1 x1 (k) + b1 u(k) + x2 (k) = −a1 x1 (k) + x2 (k) + b1 u(k)
x2 (k + 1) = −a2 x1 (k) + b2 u(k) − a3 y(k − 1) + b3 u(k − 1)
How to deal with the past information in the second equation? I hope ev-
eryone here in the class would give me the right answer: introduce another
state variable!

x3 (k) = −a3 y(k − 1) + b3 u(k − 1)

That would be the end of the story. Overall, we have obtained the following
state-space equation,
x1 (k + 1) = −a1 x1 (k) + b1 u(k) + x2 (k) = −a1 x1 (k) + x2 (k) + b1 u(k)
x2 (k + 1) = −a2 x1 (k) + b2 u(k) + x3 (k) = −a2 x1 (k) + x3 (k) + b2 u(k)
x3 (k + 1) = −a3 x1 (k) + b3 u(k)

79
where the state variables are defined as combinations of the present and past
outputs and inputs:
x1 (k) = y(k)
x2 (k) = −a2 y(k − 1) − a3 y(k − 2) + b2 u(k − 1) + b3 u(k − 2)
x3 (k) = −a3 y(k − 1) + b3 u(k − 1)
Note that the state variables are completely determined by the observations
of the inputs and outputs. Therefore, the system must be observable. If you
still have any doubt regarding this, let’s put the above state-space model in
the matrix form
   
−a1 1 0 b1
x(k + 1) =  −a2 0 1  x(k) +  b2  u(k)
−a3 0 0 b3 (3.17)


y(k) = 1 0 0 x(k)
Is there anything special about this form? Does it look familiar to you?
This is the observable canonical form!
Now it is time to state the following realizability result for realizability.

Theorem 3.6 Realizability

Given a system which is described by an input-output model
b1 z n−1 + ... + bn
H(z) = (3.18)
zn + a1 z n−1 + ... + an
If the system is observable, then the system can be realized by the observable
canonical form (3.16), and the corresponding state variables are defined as
x1 (k) = y(k)
x2 (k) = −a2 y(k − 1) + ... − an y(k − n + 1) + b2 u(k − 1) + ... + bn u(k − n + 1)
x3 (k) = −a3 y(k − 1) + ... − an y(k − n + 2) + b3 u(k − 1) + ... + bn u(k − n + 2)
..
.
xn (k) = −an y(k − 1) + bn u(k − 1)
Once one set of state-space model is deduced, other state-space model can
be obtained by various linear transformations.

The following example shows that the controllability and observability some-
times cannot be decided by checking the input-output model only.

80
Example 3.5 Realization of uncontrollable and/or unobservable
system
Consider the following input-output model
z+1 z+1
H(z) = = 2 (3.19)
(z + 1)(z + 2) z + 3z + 2
If the system is observable, applying above realizability theorem results in
following state-space model
   
−3 1 1
x(k + 1) = x(k) + u(k)
−2 0 1 (3.20)


y(k) = 1 0 x(k)
and the corresponding state variables are defined as

x1 (k) = y(k)
x2 (k) = −2y(k − 1) + u(k − 1)

Now let’s check if the system is controllable. The controllability matrix is

 
1 −2
Wc =
1 −2
Obviously, this system is not controllable as Wc is singular.

Notice that the above realization is under the assumption that the system
is observable. You may wonder whether this system can be realized using
the following state-space model in controllable canonical form
   
−3 −2 1
z(k + 1) = z(k) + u(k)
1 0 0 (3.21)


y(k) = 1 1 z(k)
You may also wonder whether this system is observable or not. Check it
out by calculating Wo ! It is unobservable! But is it really the realization of
(3.19)?

In the following, we will try to show that the above controllable canonical
form (3.21) is NOT the realization of the input-output model (3.19). It
follows immediately from equation (3.21) by adding the two state equations
that
z1 (k + 1) + z2 (k + 1) = −2(z1 (k) + z2 (k)) + u(k)

81
Since y(k) = z1 (k) + z2 (k), it follows that

y(k + 1) = −2y(k) + u(k)

and hence the corresponding transfer function is

1
G(z) =
z+2
Therefore, the state-space model (3.21) is in fact the realization of the trans-
1 z+1
fer function z+2 instead of (z+1)(z+2) . However, we also know that a more
1
natural realization of the transfer function, z+2 is simply the first-order
system
x(k + 1) = −2x(k) + u(k)
Now we end up in an interesting situation where we have two realizations
1
of the same input-output model z+2 . But one system is a first order system
which is both controllable and observable, and the other one is a second-
order system which is controllable but unobservable. I guess everyone in
this class is eager to know: which realization is correct?

The answer is we don’t know based upon the transfer function only. The
first order system is called minimal-realization of the system, in which the
order of the state-space model is equal to the order of the input-output
model. There are other infinite possible realizations of the system which
are of higher-order. But those higher order realizations are all unobservable
since the additional state variables cannot be determined by the observation
of the outputs.

In general, given a system which is described by an input-output model

without common poles and zeros, the minimal-realization of the system is
both controllable and observable. If there are common poles and zeros,
the minimal-realization of the system is observable but uncontrollable. The
controllability and observability can not be determined by transfer function
only as there are infinite number of non-minimal-realizations of the same
transfer function.

3.4 Analysis of Simple Feedback Loops

Several advantages are obtained by using feedback in continuous-time as well
as in discrete-time systems. Feedback, for instance, can do the following:

82
 • Stabilize the system if the open-loop is unstable.

• Improve the transient behavior of the system

• Increase the robustness to parameter changes in the open-loop system

• Eliminate steady-state errors if there are enough integrators in the

open-loop system

• Decrease the influence of load disturbances and measurement errors

3.4.1 Character of Disturbances

It is customary to distinguish among different types of disturbances, such
as load disturbances, measurement errors, and parameter variations.

Load disturbances
Load disturbances influence the process variables. They may represent dis-
turbance forces in a mechanical system – for example, wind gusts on a
stabilized antenna, waves on a ship, load on a motor. In process control,
load disturbances may be quality variations in a feed flow or variations in
demanded flow. In thermal systems, the load disturbances may be varia-
tions in surrounding temperature. Load disturbances typically vary slowly.
They may also be periodic – for example, waves in ship-control systems.

Measurement errors
Measurement errors enter in the sensors. There may be a steady-state error
in some sensors due to errors in calibration. However, measurement errors
typically have high-frequency components. There may also be dynamic er-
rors because of sensor dynamics. There may also be complicated dynamic
interaction between sensors and the process. Typical examples are gyro-
scopic measurements and measurement of liquid level in nuclear reactors.
The character of the measurement errors often depends on the filtering in
the instruments. It is often a good idea to look at the instrument and modify
the filtering so that it fits the particular problem.

Parameter variations
Linear theory is used throughout this module. The load disturbance and the
measurement noise then appear additively. Real systems are, however, often

83
nonlinear. This means that disturbances can enter in a more complicated
way. Because the linear models are obtained by linearizing the nonlinear
models, some disturbances then also appear as variations in the parameters
of the linear model.

3.4.2 Simple Disturbance Models

There are four different types of disturbances-impulse, step, ramp, and
sinusoid-that are commonly used in analyzing control systems. These distur-
bances are illustrated in Fig. 3.6 and a discussion of their properties follows.

The impulse and the pulse

The impulse and the pulse are simple idealizations of sudden disturbances
of short duration. They can represent load disturbances as well as measure-
ment errors. For continuous systems, the disturbance is an impulse (a delta
function); for sampled systems, the disturbance is modeled as a pulse with
unit amplitude and a duration of one sampling period.

The pulse and the impulse are also important for theoretical reasons because
the response of a linear continuous-time system is completely specified by
its impulse response and a linear discrete-time system by its pulse response.

The step
The step signal is another prototype for a disturbance (see Fig. 3.6). It is
typically used to represent a load disturbance or an offset in a measurement.

The ramp
The ramp is a signal that is zero for negative time and increases linearly for
positive time (see Fig. 3.6). It is used to represent drifting measurement
errors and disturbances that suddenly start to drift away. In practice, the
disturbances are often bounded; however, the ramp is a useful idealization.

The sinusoid
The sine wave is the prototype for a periodic disturbance. Choice of the
frequency makes it possible to represent low-frequency load disturbances, as
well as high-frequency measurement noise.

84
3.4.3 Generation of disturbances
It is convenient to view disturbances as being generated by dynamic systems
(see Fig. 3.7). It is assumed that the input to the dynamic system is a unit
pulse δk that is,
uc (k) = Hr (q)δk (3.22)
In order to generate a step, use Hr (q) = q/(q − 1); to generate a ramp,
use Hr (q) = q/(q − 1)2 , and a sinusoid from a harmonic oscillator. From
an input-output viewpoint, disturbances may be described as impulse re-
sponses. Disturbances also may be regarded as the responses of dynamic
systems with zero inputs but nonzero initial conditions. In both cases
the major characteristics of the disturbances are described by the dynamic
systems that generate them. The approach, of course, can be applied to
continuous-time, as well as discrete-time systems.

3.4.4 Steady-State Values

When analyzing control systems, it is important to calculate steady-state
values of the output and of the error of the system. To get the steady state
value of the step response, it is very simple. Just put z = 1 of the transfer
function involved, which was discussed in the previous lecture.

Example 3.5 Steady-state errors for step inputs

Consider the system described by
z − 0.5
y(k) = H(z)u(k) = u(k)
(z − 0.8)(z − 1)
Closing the system with unit feedback, as in Fig. 3.7, gives
H(z) z − 0.5
y(k) = uc (k) = uc (k)
1 + H(z) (z − 0.8)(z − 1) + (z − 0.5)
and
1 (z − 0.8)(z − 1)
e(k) = uc (k) = uc (k)
1 + H(z) (z − 0.8)(z − 1) + (z − 0.5)
Assume that uc is a unit step, the steady state gain can be obtained simply
by putting z = 1 in the corresponding transfer functions. Notice that the
open-loop system contains one integrator, that is, a pole in +1, which makes
H(1)
H(1) = ∞. Therefore the steady state gain for the output is 1+H(1) = 1,
1
and the steady state gain for the error is 1+H(1) = 0, which means the steady
state error of the step response is zero.

85
3.4.5 Simulation
Simulation is a good way to investigate the behavior of dynamic systems
– for example, the intersample behavior of computer-controlled systems.
Computer simulation is a very good tool, but it should be remembered that
simulation and analysis have to be used together. When making simulations,
it is not always possible to investigate all combinations that are unfavorable,
for instance, from the point of view of stability, observability, or reachability.
These cases can be found through analysis.

It is important that the simulation program be so simple to use that the

person primarily interested in the results can be involved in the simulation
and in the evaluation of the simulation results.

In the beginning of the 1960s, several digital simulation packages were de-
veloped. These packages were basically a digital implementation of analog
simulation. The programming was done using block diagrams and fixed-
operation modules. Later programs were developed in which the models
were given directly as equations. It is important to have good user-machine
communication for simulations; the user should be able to change parame-
ters and modify the model easily. Most simulation programs are interactive,
which means that the user interacts with the computer and decides the next
step based on the results obtained so far. One way to implement interac-
tion is to let the computer ask questions and the user select from prede-
fined answers. This is called menu-driven interaction. Another possibility
is command-driven interaction, which is like a high-level problem-solving
language in which the user can choose freely from all commands available
in the system. This is also a more flexible way of communicating with the
computer, and it is very efficient for the experienced user, whereas a menu-
driven program is easier to use for an inexperienced user. In a simulation
package, it is also important to have a flexible way of presenting the results,
which are often curves. Finally, to be able to solve the type of problems of
interest in this module, it is important to be able to mix continuous- and
discrete-time systems.

Examples of simulation packages are MATLABr with SIMULINKr Be-

cause these packages are readily available we will not describe any of them
in detail. However, we urge the students to use simulation to get a good feel
for the behavior of the computer-controlled systems that are described in
the module. For the figures in the lectures we have used MATLABr with

86
SIMULINKr

Control of the Double Integrator

The double integrator will be used as the main example to show how the
closed-loop behavior is changed with different controllers. The transfer func-
tion of the double integrator for the sampling period h = 1 is
0.5(z + 1)
H0 (z) = (3.23)
(z − 1)2
Assume that the purpose of the control is to make the output follow changes
in the reference value. Let’s start with the simplest feedback controller,
proportional feedback, that is,

u(k) = K(uc (k) − y(k)) = Ke(k) K > 0

where uc is the reference value. The characteristic equation of the closed-

loop system is

(q − 1)2 + 0.5K(q + 1) = q 2 + (0.5K − 2)q + 1 + 0.5K = 0 (3.24)

Jury’s stability test gives the following conditions for stability:

1 + 0.5K < 1
1 + 0.5K > −1 + 0.5K − 2
1 + 0.5K > −1 − 0.5K + 2

The closed-loop system is unstable for all values of the gain K. The root
locus (the graphic plot of the poles against different gains) is shown in Fig.
3.8.

To get a stable system, the controller must be modified. It is known from

continuous-time synthesis that derivative action improves stability, so pro-
portional and derivative feedback can be tried also for the discrete-time sys-
tem. We now assume that it is possible to measure and sample the velocity
ẏ and use that for feedback; that is,

u(k) = K(e(k) − Td ẏ(k)) (3.25)

To find the input-output model of the closed-loop system with controller

(3.25), observe that
dẏ
ÿ = =u
dt

87
Because u is constant over the sampling intervals,

ẏ(k + 1) − ẏ(k) = u(k)

or
1
ẏ(k) = u(k) (3.26)
q−1
Equations (3.23), (3.25), and (3.26) give the closed-loop system

0.5K(q + 1)
y(k) = uc (k) (3.27)
(q − 1)(q − 1 + Td K) + 0.5K(q + 1)

The system is of second order, and there are two free parameters, K and Td ,
that can be used to select the closed-loop poles. The closed-loop system is
stable if K > 0, Td > 0.5, and Td K < 2. The root locus with respect to K
of the characteristic equation of (3.27) is shown in Fig. 3.10 when Td = 1.5.

Let the reference signal be a step. Fig. 3.11 shows the continuous-time
output for four different values of K. The behavior of the closed-loop system
varies from an oscillatory to a well-damped response. When K = 1, the poles
are in the origin and the output is equal to the reference value after two
samples. This is called deadbeat control and will be discussed further later
in this module. When K > 1, the output and the control signal oscillate
because of the discrete-time pole on the negative real axis. Remember that
in addition to complex poles, negative poles can also generate oscillation for
discrete-time system, as discussed in section 2.6. The poles are inside the
unit circle if K < 4/3. When K > 4/3, the system becomes unstable.

88
Figure 3.5: The output of the system in Example 3.3 for the initial states
(a) [0.5 1], (b) [1.5 0.5], (c) [2.5 0], and (d) [1 − 0.5] .

89
Figure 3.6: Idealized models of simple disturbances.

90
Figure 3.7: Generation of the reference value using a dynamic system with
a pulse input.

91
Figure 3.8: The root locus of (3.24) when K > 0.

92
Figure 3.9: Discrete-time controller with feedback from position and velocity
of the double integrator.

93
Figure 3.10: The root locus of the characteristic equation of the system in
(3.27) with respect to the parameter K when Td = 1.5.

94
Figure 3.11: The continuous-time output of the system in Fig. 3.8 when
Td = 1.5 and (a) K = 0.5, (b) K = 0.75, (c) K = 1, and (d) K = 1.25.

95
Chapter 4

Pole-Placement Problem —
State-Space Approach

In the previous lectures, we have discussed some key tools and concepts for
analyzing discrete-time systems. Now it is time to apply these tools to com-
puter control system design, which is supposed to be the core of this course.
What is the ultimate goal of the control system? To make the output of
the system follow some desired trajectory! But how to specify the desired
output? There are normally two ways. The first one is just to specify the
desired output arbitrarily without any constraint of the behavior. You can
imagine that it might be extremely hard to make the system behave in any
way you want in practical problems. We will discuss this type of problem
later under certain conditions.
Usually, we have to be more modest and try to make the desired output
more attainable. One way of doing this is to generate the desired output
through a well defined reference model to meet various system requirements.
And then the next step is to design the controller such that the behavior of
the overall system approaches that of the reference model. In other words,
we will try to make the transfer function of the closed loop system as close
to the reference model as possible. This approach is called model reference
control. In practice, perfect matching of these two transfer functions is often
impossible. Which task is more important then, matching the zeros or the
poles? As discussed in the previous lectures, since stability is of fundamen-
tal importance for control system design, which is decided by the location
of the poles, everyone would agree what we should try to match the poles
first. This is so called the pole placement problem: design the controller to
place the poles of the closed loop at the desired positions that are given by

96
the reference model.

4.1 Pole placement problem

We will try to solve the pole placement problem for the simplest case first:
both the state-space model and the state variables are available. When you
do research, it is always a good strategy to attack the simplest case first,
then build it up.

Consider a linear system described by

ẋ = Ax + bu
(4.1)
y = cx

Discretize the system using sampling period h, and we have

x(k + 1) = Φx(k) + Γu(k)

(4.2)
y(k) = cx(k)

where
Φ = eAh
Zh
Γ= eAs bds
0

What’s the simplest feedback-controller in the robotic manipulator example

discussed in the first class? Proportional control.

Similarly, we will try the simplest one first. Let the state feedback controller
be
u(k) = −Lx(k) (4.3)
and the desired characteristic polynomial (C.P.), i.e., the denominator of
the reference model, Hm (z) = B m (z)
Am (z) , is

Am (z) = z n + p1 z n−1 + · · · + pn (4.4)

Question: Can we choose L such that the poles of the closed loop system
match desired poles given by above characteristic polynomial?

97
Example: Double-integrator:
ÿ(t) = u(t) (4.5)
Discretize it into
  h2 
1 h 2
x(k + 1) = x(k) + u(k)
0 1 h (4.6)
 
y(k) = 1 0 x(k)

Let
u(k) = −Lx(k) = −l1 x1 (k) − l2 x2 (k) (4.7)
and the closed-loop system becomes
2 2
1 − l12h h − l22h
 
x(k + 1) = x(k) (4.8)
−l1 h 1 − l2 h

The corresponding characteristic polynomial is

l1 h2 l1 h2
z2 + ( + l2 h − 2)z + ( − l2 h + 1)
2 2
Assume that the desired polynomial is

Am (z) = z 2 + p1 z + p2 (4.9)

Then just let (

l1 h2
2 + l2 h − 2 = p1
l1 h2
2 − l2 h + 1 = p2
and we obtain
1

l1 = h2
(1 + p1 + p2 )
1 (4.10)
l2 = 2h (3 + p1 − p2 )
Conclusion:
The state-feedback control can not only stabilize the system, but also place
the poles at any desired locations!

Question:
Is this true for any linear system?
The answer to this question turns out to be directly connected with another
fundamental concept in control theory: Controllability

98
Now let’s take a look at another example.

x(k + 1) = Φx(k) + Γu(k) (4.11)

where    
−a1 −a2 1
Φ= ,Γ =
1 0 0
What’s the C.P. of this system?
 
z + a1 +a2
det{zI − Φ} = det = z 2 + a1 z + a2 (4.12)
−1 z

Note that the first row of Φ determines the C.P.!

Let’s check the controllability matrix,

 
  1 −a1
Wc = Γ ΦΓ = (4.13)
0 1

which is nonsingular. Therefore, the system is controllable.

Let’s then try to design a state-feedback controller to place the poles at

some desired places.
 
u(k) = −Lx(k) = − l1 l2 x(k) (4.14)

The closed loop system then becomes

x(k + 1) = Φx(k) + Γu(k) = (Φ − ΓL)x(k)

where  
l1 l2
ΓL =
0 0
and  
−a1 − l1 −a2 − l2
Φ − ΓL =
1 0
whose C.P. is
z 2 + (a1 + l1 )z + (a2 + l2 )
If the desired C.P. is
z 2 + p1 z + p2

99
We simply choose the control gains L as

l1 = p1 − a1
l2 = p2 − a2

and hence  
u(k) = −Lx(k) = − p1 − a1 p2 − a2 x(k)
Note that the control parameters are the difference between the coefficients
of the C.P. of the system and the desired polynomial. The pole-placement
problem can be easily solved for such special kind of system. And the above
result can be extended to general case as follows.

z(k + 1) = Φ̃z(k) + Γ̃u(k)

where  
−a1 −a2 · · · −an
 1 0 ··· 0 
Φ̃ = 
 
.. .. .. .. 
 . . . . 
0 ··· 1 0
 
1
 0 
Γ̃ =  . 
 
 .. 
0
and a1 , a2 , ... , an are the coefficients of the C.P.

z n + a1 z n−1 + · · · + an

If the desired C.P. is

Am (z) = z n + p1 z n−1 + · · · + pn

then
 
u(k) = −L̃z(k) = − p1 − a1 p2 − a2 · · · p n − an z(k) (4.15)

Remark: The pole-placement problem can be easily solved because of its

special form. If you look at this form carefully, can you recognize what form
it is? It is controllable companion /canonical form.
Question: How about systems in other forms?

x(k + 1) = Φx(k) + Γu(k)

100
If the system is controllable, then it is similar to its corresponding con-
trollable form. Or in other words, it can be transformed into controllable
companion form by

z = Tx
x(k + 1) = Φx(k) + Γu(k)
z(k + 1) = T x(k + 1) = T Φx(k) + T Γu(k) = T ΦT −1 z(k) + T Γu(k) = Φ̃z(k) + Γ̃u(k)

where
Φ̃ = T ΦT −1
and
Γ̃ = T Γ
and the state-feedback controller is then
 
u(k) = − p1 − a1 p2 − a2 · · · pn − an z(k)
 
= − p1 − a1 p2 − a2 · · · p n − an T x(k)

Question: what is T ?

T = W̃c Wc−1 (4.16)

Please refer to section 3.4.2 for further details about equation (4.16).

Overall, we have

u(k) = − p1 − a1 p2 − a2 · · · pn − an W̃c Wc−1 x(k) = −Lx(k)

 

L = p1 − a1 p2 − a2 · · · pn − an W̃c Wc−1
 

Using Cayley-Hamilton theorem, it can be shown that

L = 0 · · · 0 1 Wc−1 Am (Φ)
 
(4.17)

where the polynomial Am (z) is the desired C.P.

This is called the Ackermann’s formula

Remark: The pole-placement problem can be formulated as following al-

gebraic problem:
Given pair {Φ, Γ}, find L, such that {Φ − ΓL} has any desired C.P. Am (z).

101
Example: double-integrator
   h2 
1 h 2
x(k + 1) = x(k) + u(k)
0 1 h
the controllability matrix
h2 3h2
 
Wc = 2 2
h h
is nonsingular. Therefore, it is controllable. We will use Ackermann’s
formula to calculate the feedback gains L.
− h12 1.5
 
−1 h
Wc = 1 −0.5
h2 h
 
2 1 + p1 + p2 2h + p1 h
Am (Φ) = Φ + p1 Φ + p2 I =
0 1 + p1 + p2
Then applying Ackermann’s formula gives
1 + p1 + p2 3 + p1 − p2
h i
L = 0 1 Wc−1 Am (Φ) =
 
h2 2h
which is the same result as what we obtained before.

But what if the system is not controllable?

Example:
   
a 1 1
x(k + 1) = x(k) + u(k)
0 b 0

The controllability matrix

 
1 a
Wc =
0 0
is singular. Can we still use Ackermann’s Formula? No!

Let
u(k) = −l1 x1 (k) − l2 x2 (k)
then  
a − l1 1 − l2
x(k + 1) = x(k)
0 b

102
The C.P. is
(z − a + l1 )(z − b)
Hence, the pole z = a can be changed, but the pole z = b can not be
changed!

• If |b| < 1, this uncontrollable mode is stable, we call such kind of

system stabilizable.

• If |b| > 1, since this mode is unstable, there’s nothing we can do to

control this system.

4.1.1 Dead beat control

Let the desired poles are all zero. i.e.

Am (z) = z n (4.18)

By Cayley-Hamilton theorem

Φnc = (Φ − ΓL)n = 0

immediately we have

x(n) = (Φ − ΓL)n x(0) = 0

The state can be driven into zero at most n steps.

Example: Double-integrator
Let
p1 = p2 = 0
L = h12 2h3
 

h2 1 3h
 h i  
ΓL = 2 1 3 = 2 4
h2 2h 1 3
h h 2
1 h
 
(Φ − ΓL) = 2 4
− h1 − 12
(Φ − ΓL)2 = 0
and
x(2) = (Φ − ΓL)2 x(0) = 0

103
4.1.2 Summary
The state-feedback controller
u(k) = −Lx(k)
can place the poles of the system at any desired location if the system is
controllable. Note that the state vector x(k) is involved in the calculation
of control input u(k) at every instant k. But in many practical problems,
only the output y(k) is available, how to solve this problem?

We have to estimate the state x by the observations of the output y and

input u.

4.2 Observer
Consider the system
x(k + 1) = Φx(k) + Γu(k)
y(k) = cx(k)
where x(k) is not accessible. Since the state feedback controller, u(k) =
−Lx(k), needs the state information to calculate the input, it is necessary
to estimate the state. This state estimator is usually called observer. Let’s
first try the simplest state estimator you might have in mind.

4.2.1 The Naive Observer

The naive observer takes the same form as the state-space model by replacing
the state vector x(k) (not available) with its estimate x̂(k).
x̂(k + 1) = Φx̂(k) + Γu(k)
ŷ(k) = cx̂(k)
Since the observer has the same dynamic equation as that of the state-space
model, does that mean that the response of the observer will be the same as
that of the original system subject to the same input sequence u(k)? The
answer is no! Different initial conditions will lead to different solutions!

Our main concern is: would the state estimation error decrease to zero? If
yes, then this will be a good observer.
Let the estimation error be
e(k) = x(k) − x̂(k)

104
it easily follows that
e(k + 1) = Φ(e(k))
Therefore, the estimation error will blow up if the system is unstable. It
might work if the system is stable.
What is missing in the naive observer? The output y! It did not use any
information of the output! Let’s see next if we can improve the performance
of the observer by feeding back the output estimation error, y(k) − ŷ(k).

4.2.2 Observer
Construct an observer by feeding back the output estimation error,
x̂(k + 1) = Φx̂(k) + Γu(k) + K(y(k) − ŷ(k))
(4.19)
ŷ(k) = cx̂(k)
Then
x(k + 1) − x̂(k + 1) = Φ(x(k) − x̂(k)) − K(y(k) − ŷ(k))
Let the estimation error be defined by
e(k) = x(k) − x̂(k)
then
e(k + 1) = Φe(k) − Kce(k) = (Φ − Kc)e(k)
Question: Can we chose K such that Φ − Kc has any desired C.P., Ao (z)?
Recall that we have already solved a very similar problem: Given pair {Φ, Γ},
find L, such that {Φ − ΓL} has any desired C.P. And the solution is given
by the Ackermann’s formula:
L = 0 · · · 0 1 Wc−1 Am (Φ)
 

We cannot directly apply this formula here since matrix multiplication is

not commutable. But we know that the C.P. of a matrix in invariant to the
operation of transpose. Let
(Φ − Kc)T = ΦT − cT K T
Then the problem becomes: given pair of {ΦT , cT }, find K T , such that
Φ − Kc has any desired C.P.. Now we can apply the previous Ackermann’s
formula by identifying following
Φ ⇒ ΦT
Γ ⇒ cT (4.20)
T
L⇒K

105
The corresponding Wc in the Ackermann’s formula is then
 T
c
   cΦ 

Wc = cT ΦT cT · · · (Φn−1 )T cT =   = WoT

.. (4.21)
 . 
cΦn−1

Then applying the Ackermann’s formula gives

K T = 0 · · · 0 1 (WoT )−1 Ao (ΦT )

 

which can be rewritten as

T
K = Ao (Φ)(Wo )−1

0 ··· 0 1 (4.22)

Example: Double-Integrator
   h2 
1 h 2
x(k + 1) = x(k) + u(k)
0 1 h
 
y(k) = 1 0 x(k)
 
c= 1 0
 
1 h
Φ=
0 1
   
c 1 0
Wo = =
cΦ 1 h
   
−1 1 h 0 1 0
Wo = =
h −1 1 − h1 h1
 
2 1 + λ1 + λ2 2h + λ1 h
Ao (Φ) = Φ + λ1 Φ + λ2 =
0 1 + λ1 + λ2
   
−1 0 2 + λ1
K = Ao (Φ)(Wo ) = 1+λ1 +λ2
1 h

4.2.3 Dead-beat observer

Choose Ao (z) = z n . Since Ao (z) is the C.P. of (Φ − Kc), it follows that

e(k + n) = (Φ − Kc)n e(k) = 0

Then the observation error goes to zero at most n steps.

106
4.3 Output-Feedback Controller
Summarizing the previous results, we can now give the complete solution to
pole-placement problem when only outputs are available. Consider system

x(k + 1) = Φx(k) + Γu(k)

y(k) = cx(k)

Step 1: Assume the state variables x(k) is available, design a state-feedback

controller
u(k) = −Lx(k) (4.23)
such that (Φ − ΓL) has desired C.P.

Am (z) = z n + p1 z n−1 + · · · + pn

L can be calculated by Ackermann’s formula if the system is controllable.

L = 0 · · · 0 1 Wc−1 Am (Φ)
 
(4.24)

Step 2: Build an observer using output feedback

x̂(k + 1) = Φx̂(k) + Γu(k) + K(y(k) − ŷ(k))

(4.25)
ŷ(k) = cx̂(k)

e(k + 1) = (Φ − Kc)e(k)
Choose K such that (Φ − Kc) has desired C.P.

Ao (z) = z n + λ1 z n−1 + · · · + λn

K can be calculated by
T
K = Ao (Φ)(Wo )−1

0 ··· 1 (4.26)

if the system is observable.

Step 3: Replace the state x(k) in step 1 by the estimate x̂(k)

u(k) = −Lx̂(k) (4.27)

The overall closed loop system becomes

(
x(k + 1) = Φx(k) − ΓLx̂(k)
e(k + 1) = (Φ − Kc)e(k)

107
(
x(k + 1) = Φx(k) − ΓLx(k) + ΓL(x(k) − x̂(k)) = (Φ − ΓL)x(k) + ΓLe(k)
e(k + 1) = (Φ − Kc)e(k)
If both (Φ − ΓL) and (Φ − Kc) is stable, the overall system is stable, and

e(k) → 0, x(k) → 0

The above design has a very nice property:

Separation property: the poles for the controller and observer can be
designed separately.

Example: Double-integrator
   h2 
1 h 2
x(k + 1) = x(k) + u(k)
0 1 h
 
y(k) = 1 0 x(k)
Step 1:
u(k) = −Lx(k)
 1+p1 +p2 3+p1 −p2 
L= h2 2h

Step 2:
   h2 
1 h
x̂(k + 1) = x̂(k) + 2 u(k) + K(y(k) − ŷ(k))
0 1 h
 
ŷ(k) = 1 0 x̂(k)

where  
2 + λ1
K= 1+λ1 +λ2
h
Step 3: Output feedback controller:

u(k) = −Lx̂(k)

4.4 Disturbance Rejection Problem

The results we have derived are only useful for impulse disturbances such as

when each impulse disturbance occurs, the states will be driven away from
equilibrium position, and the feedback controller would drive them back to

108
equilibrium position as shown in Fig. 4.6.

In general if there are disturbances acting upon the system, the controller
has to cancel out the effects of the disturbances. This is called disturbance
rejection problem.

Let’s consider a system where disturbance ω(k) affects the system as follows.

x(k + 1) = Φx(k) + Φxω ω(k) + Γu(k)

ω(k + 1) = Φω ω(k) (4.28)
y(k) = cx(k)

What would happen simply ignoring the disturbance term? Let the con-
troller still be the state feedback controller

u(k) = −Lx(k)

then
x(k + 1) = (Φ − ΓL)x(k) + Φxω ω(k)
Obviously, the response will be heavily affected by the disturbance. Hence
the disturbance cannot be ignored.

What’s the state of the system? We’d better consider the disturbance ω(k)
as part of the state instead of the input, since we cannot change the ω(k)
itself.

Introduce the augmented state vector,

 
x(k)
z(k) = (4.29)
ω(k)

then
     
x(k + 1) Φ Φxω Γ
z(k + 1) = = z(k) + u(k)
ω(k + 1) 0 Φω 0 (4.30)
 
y(k) = c 0 z(k)

Is this system controllable?

Of course not. How can you control the disturbance! But we can control
x(k), which is the original state vector by estimating the disturbance as

109
shown below.

Step 1: Assume the disturbances and state variables are measurable, design
the controller to stabilize the system and cancel out the disturbances. Let

u(k) = −Lx(k) − Lω ω(k) (4.31)

The closed loop system becomes

        
x(k + 1) Φ Φxω x(k) Γ   x(k)
= − L Lω
ω(k + 1) 0 Φω ω(k) 0 ω(k)
  
Φ − ΓL Φxω − ΓLω x(k)
=
0 Φω ω(k)
 
y(k) = c 0 z(k)

The we can choose L such that (Φ − ΓL) has desired poles as before.

But how to choose Lω ?

Choose Lω to reduce the effect of the disturbance ωk . This control action is
particularly effective if Φxω − ΓLω = 0 , which implies that the disturbance
can be completely canceled out. But this can be hardly achieved for most
cases, and selecting proper Lω remains a difficult problem. If the distur-
bance ω is constant, then it is possible to choose Lω such that the steady
state gain from the disturbance to the output is zero, which implies that the
effect of the disturbance can be completely filtered out in steady state.

Step 2: If the disturbances are not accessible, which is usually the case in
practice, then build an observer to estimate the disturbances, as well as the
states.
   
Φ Φxω Γ
ẑ(k + 1) = ẑ(k) + u(k) + K(y(k) − ŷ(k))
0 Φω 0 (4.32)
 
ŷ(k) = c 0 ẑ(k)
Is the system observable? It is indeed observable in many cases.
Then K can be chosen such that the C.P. of
  
Φ Φxω  
−K c 0
0 Φω
match any desired one. Then we can make sure that the estimation error
of the augmented state vector z(k) converge to zero. It can be made to be

110
zero after finite number of steps if dead-beat observer is used.

Step 3: Replace the disturbance variable and state variables by their cor-
responding estimates in the controller designed in Step 1. i.e.

u(k) = −Lx̂(k) − Lω ω̂(k) (4.33)

The block diagram of the overall system is shown in Fig. 4.7.

Example: Double-Integrator
A constant but unknown disturbance ω(k) acting on the input as follows,
   h2 
1 h 2
x(k + 1) = x(k) + (u(k) + ω(k))
0 1 h
 
y(k) = 1 0 x(k)
ω(k + 1) = ω(k)

Step 1: If both x(k) and ω(k) are known, design a state-feedback controller

u(k) = −Lx(k) − ω(k)

Step 2: Add ω(k) into the state vector

 
x(k)
z(k) =
ω(k)

then 2   h2 
1 h h2

   
x(k + 1) x(k) 2
= 0 1 h  +  h  u(k)
ω(k + 1) ω(k)
0 0 1 0
 
  x(k)
y(k) = 1 0 0
ω(k)
The observability matrix for the augmented system is
   
c 1 0 0
Wo =  cΦ  =  1 h h2 
2
cΦ2 1 2h 2h2

which is nonsingular, hence the system is observable! Therefore, the aug-

mented state vector including the state x(k) and disturbance v(k) can be

111
estimated.

Build the observer as follows,

h2 h2
   
  1 h 2
 
2
x̂(k + 1) x̂(k)
= 0 1 h  +  h  u(k) + K(y(k) − ŷ(k))
ω̂(k + 1) ω̂(k)
0 0 1 0
 
  x̂(k)
ŷ(k) = 1 0 0
ω̂(k)
K can be calculated based on the desired C.P. for the observer
Ao (z) = z 3 + λ1 z 2 + λ2 z + λ3
then  
0
K = λ(Φ)(Wo )−1  0 
1
Step 3: Replace the true values by the estimated in step 1 :
u(k) = −Lx̂(k) − ω̂(k)

4.5 The Tracking Problem — Zero Placement

So far only the pole placement problem has been completely solved. In
the following we will try to match the zeros too. This can be called zero
placement problem.

4.5.1 A Naive Approach

A simple way to obtain the desired response to command signals is to replace
the regular state feedback u(k) = −Lx̂(k) by
u(k) = −Lx̂(k) + Lc uc (k) (4.34)
where uc (k) is the command signal. To investigate the response of such a
controller, we consider the closed-loop system that is described by
x(k + 1) = Φx(k) + Γu(k)
y(k) = cx(k)
x̂(k + 1) = Φx̂(k) + Γu(k) + K(y(k) − ŷ(k))
ŷ(k) = cx̂(k)
u(k) = −Lx̂(k) + Lc uc (k)

112
A block diagram of the system is shown in the Fig. 4.8. Eliminating u and
introducing the estimation error e we find that the closed-loop system can
be described by

x(k + 1) = (Φ − ΓL)x(k) + ΓLe(k) + ΓLc uc (k)

e(k + 1) = (Φ − Kc)e(k)
y(k) = cx(k)
It follows from above equations that the transfer function from the command
signal uc (k) to the process output y is given by
B(z)
Hcl (z) = c(zI − Φ + ΓL)−1 ΓLc = Lc (4.35)
Am (z)
This can be compared with the transfer function of the process
B(z)
H(z) = c(zI − Φ)−1 Γ = (4.36)
A(z)
The fact that the polynomial B(z) appears in the numerator of both transfer
functions can be seen by transforming both systems to controllable canoni-
cal form.

The closed-loop system obtained with this naive control law has the same
zeros as the plant and its poles are the eigenvalues of the matrix (Φ − ΓL),
which can be placed at any positions by properly choosing L is the system
is controllable.

Bm (z)
Let the transfer function of the reference model be Hm (z) = , then the
Am (z)
tracking problem is to design the controller such that the transfer function of
the closed loop Hcl (z) match the reference model Hm (z) as close as possible.

B(z) choose Lc Bm (z)

Lc −−−−−−−−−−→
Am (z) Am (z)
It is impossible to achieve this if Lc is limited to some constant vector as in
the naive controller. However, what if Lc is another transfer function?

4.5.2 Two-Degree-of-Freedom Controller

Practical control systems often have specifications that involve both servo
and regulation properties. This is traditionally solved using a two-degree-
of-freedom structure, as shown in Fig. 4.9. This configuration has the ad-
vantage that the servo and regulation problems are separated. The feedback

113
controller Hf b , is designed to obtain a closed-loop system that is insensitive
to process disturbances, measurement noise, and process uncertainties. The
feed-forward compensator Hf f is then designed to obtain the desired servo
properties. Obviously, the naive controller discussed before is just the sim-
ple case of two-degree-of-freedom controller with Hf f = Lc . If instead, we
choose
Bm (z)
Hf f = Lc = (4.37)
B(z)
Then we have the closed loop transfer function from the command signal uc
to the output,

Y (z) B(z) Bm (z) B(z) Bm (z)

= Lc = = = Hm (z) (4.38)
Uc (z) Am (z) B(z) Am (z) Am (z)

and mission accomplished!

However, this is only true if all the zeros of B(z) are stable or all the unstable
zeros of B(z) are also the zeros for Bm (z)! This is due to the fact that the
closed-loop transfer function from the command signal uc to the input signal
is
U (z) A(z)Bm (z)
= (4.39)
Uc (z) B(z)Am (z)
which requires that B(z) has to be stable. Otherwise, the input signal may
blow up!

The condition of all the zeros of B(z) to be stable is called the

Stable inverse condition which was discussed in section 2.6.

In general, only system with stable inverse can be controlled to perfectly

track arbitral command signals. If the system has an unstable inverse, how
to design Hf f would depend upon the nature of the specific command signal,
which is outside the scope of this module.

114
4.5.3 Putting It All Together
By combining the solutions to the regulation and servo problems we have a
powerful controller, which is described by

x(k + 1) = Φx(k) + Φxω ω(k) + Γu(k)

ω(k + 1) = Φω ω(k)
y(k) = cx(k)
x̂(k + 1) = Φx̂(k) + Φxω ω̂(k) + Γu(k) + K(y(k) − ŷ(k))
ω̂(k + 1) = Φω ω̂(k) + Kω (y(k) − ŷ(k)) (4.40)
ŷ(k) = cx̂(k)
u(k) = uf f (k) + uf b (k)
uf b (k) = −Lx̂(k) − Lω ω̂(k)
uf f (k) = Hf f uc (k)

This controller captures many aspects of a control problem such as load-

disturbance attenuation, reduction of effects of measurement noise, and
command signal following. The responses to load disturbances, command
signals, and measurement noise are completely separated. The command
signal response is determined by the reference model. The response to load
disturbances and measurement noise is influenced by the observer and the
state feedback. A block diagram of the closed-loop system is shown in Fig.
4.10.

115
Figure 4.1: Model reference control system

116
 Figure 4.2: State feedback controller

Figure 4.3: observer by output error feedback

117
Figure 4.4: output feedback controller by combining state feedback with
observer.

Figure 4.5: Impulse Disturbance

118
Figure 4.6: Response of the system to impulse disturbance

119
Figure 4.7: output feedback controller with estimated disturbance and
states.

120
Figure 4.8: Naive approach for tracking problem

121
Figure 4.9: Two-degree-of-freedom Controller

122
Figure 4.10: The state-space approach to control system design

123
Chapter 5

Pole-Placement —
Polynomial Approach

5.1 Review: Pole placement problem - State-space

approach
Consider system
x(k + 1) = Φx(k) + Γu(k)
y(k) = cx(k)
Step 1: Assume that the state variables x(k) are available, design a state-
feedback controller
u(k) = −Lx(k)
such that (Φ−ΓL) has desired C.P. and L can be calculated by Ackermann’s
formula.

What if the system is not controllable, i.e., Wc is singular? Can we still

assign the poles to any desired locations?
When the system is not controllable, it implies there are some poles that
cannot be changed by the state-feedback controller. The situation is not
completely hopeless provided these uncontrollable poles are stable! Such
systems are called stabilizable.

Step 2: In many cases, only output y(k) is available, we need to build an

124
observer using output feedback

x̂(k + 1) = Φx̂(k) + Γu(k) + K(y(k) − ŷ(k))

ŷ(k) = cx̂(k)

e(k + 1) = (Φ − Kc)e(k)
Choose K such that (Φ − Kc) has desired C.P. K can be calculated by
Ackermann’s formula.
Again, what if the system is unobservable? It implies that some poles for the
observer cannot be changed by output feedback. But if these unobservable
poles are all stable, then the estimation error can be made to converge to
zero asymptotically. We call such kind of system detectable.
When a system is detectable but unobservable, the state variables can still
be estimated. But unobservability implies that the state variables cannot
be determined uniquely by the observation of the outputs, do we have a
contradiction here?
No. Convergence of estimation error to zero does not imply that the state
variables can be determined exactly. For instance, consider two convergent
series, {sin(t)/t} and {1/t}. Both of them converge to zero, but they are
not equal to each other.

Step 3: Output-feedback controller–Replacing the state x(k) by the esti-

mate
u(k) = −Lx̂(k)

Disturbance Rejection
The standard way of dealing with disturbance is adding them into the state
vector, and building an observer to estimate the whole augmented state vec-
tor.

Question: If x(k) available, we have u(k) = −Lx(k). If x(k) not available,

we use u(k) = −Lx̂(k). What if x(k) is partially available?

x1 (k), · · · , xl (k), xl+1 (k), · · · , xn (k)

| {z }| {z }
Y es No

125
Should we still use u(k) = −Lx̂(k), or try the combination.
 
x1 (k)
 .. 

 . 

 xl (k) 
u(k) = −L  
 x̂l+1 (k) 

 .. 
 . 
x̂n (k)
Rule of Thumb: always use the more accurate information.

5.2 Relation between State-Space Model and Input-

Output Model
As discussed in the last lecture, the pole-placement problem can be com-
pletely solved by state-space approach. Then why do we need another ap-
proach? Because there are a number of difficulties associated with this
approach: first of all, choosing proper Lω for disturbance rejection is quite
a headache. To make things worse, the state-space model may not be avail-
able in real world problems! In real problems, many of the systems are
directly described by input-output models. It is natural to explore if the
pole-placement problem can be done directly by polynomial calculation.

But before we address this problem, let’s discuss the relationship between
the state-space model and the input-output model.

Given a state-space model

x(k + 1) = Φx(k) + Γu(k)
y(k) = cx(k)
How do we deduce the input-output model or transfer function? What’s
the tool? The z-transform! Apply z-transform and assume zero initial
conditions, we have
zX(z) = ΦX(z) + ΓU (z)
Y (z) = cX(z)
From the above equation,
(zI − Φ)X(z) = ΓU (z)
X(z) = (zI − Φ)−1 ΓU (z)

126
Finally, we have

Y (z) = cX(z) = c(zI − Φ)−1 ΓU (z)

Transfer function:
B(z)
c(zI − Φ)−1 Γ =
A(z)
Difference Equation:

A(q)y(k) = B(q)u(k)

Now the inverse problem: let a system be described by difference equation

A(q)y(k) = B(q)u(k)

Or equivalently a transfer function

B(z)
Y (z) = U (z)
A(z)

Can we find the corresponding state-space representation? This is the so

called
Realization Problem:

Let
A(z) = z n + a1 z n−1 + · · · + an
B(z) = b1 z n−1 + · · · + bn
How many state-space models can be derived? There are infinite number of
realizations. For instance, the most famous one is
the Controllable Companion/Canonical Form: {Φc , Γc , cc }
   
−a1 −a2 · · · −an 1
 1 0 · · · 0   0 
Φc =  . ..  , Γc =  . 
   
 .. .. ..  .. 
. . . 
0 ··· 1 0 0
 
cc = b1 · · · bn
Or

127
The Observable Companion/Canonical form: {Φo , Γo , co }, which is
simply the dual system of the controllable companion form {ΦTc , ΓTc , cTc }
 
−a1 1 · · · 0  
 ..  b1
 −a2 0 · · · .  . 
Φo = ΦTc =  .. . .
 , Γo = cTc = 
 .. 
 . .. .. 1 
bn
−an · · · 0 0

co = ΓTc =
 
1 ··· 0
Are these two representations similar to each other? Not always!
If the system is both controllable and observable, then these two companion
forms are similar to each other.

5.3 Pole-Placement Problem – Polynomial Approach

In many problems, only input-output representation is available

A(q)y(k) = B(q)u(k) (5.1)

The poles of the system is decided by the roots of A(z). If the poles of
A(q) are not at ideal places, then we may want to place the poles at proper
positions. — The same objective as before.

One obvious approach: realize the system with a state-space representation,

and then design the output-feedback controller.

Or is there a direct way?

Yes.

5.3.1 Error-Feedback Control

Let’s start with the simple design first — the one degree of freedom con-
troller. The control input will depend upon the error signal only, so called
error feedback controller. The block diagram for error-feedback controller is
shown in Fig. 5.1.

128
 Figure 5.1: Pure Error-Feedback Controller

The pure error feedback control is in the form of

R(q)u(k) = S(q)e(k) = S(q)(uc (k) − y(k)) (5.2)

The closed loop transfer function from the command signal uc (k) to the
output y(k) can be easily obtained as
B(z)S(z)
Y (z) = Uc (z) (5.3)
A(z)R(z) + B(z)S(z)
Suppose we can design the R(z) and S(z) such that A(z)R(z) + B(z)S(z)

129
matches the desired C.P., then there is no room left for us to match the ze-
ros since there is no freedom to manipulate the numerator in above transfer
function.
Therefore, error feedback controller is sufficient only for stabilization of the
process. In order to match the zeros of the reference model, we have to
increase the degree of freedom in the controller, just as we did for the state-
space approach.

5.3.2 Two-degree-of-freedom controller

The two-degree-of-freedom controller can be designed in following form

R(q)u(k) = T (q)uc (k) − S(q)y(k) (5.4)

The block diagram is shown in the Fig. 5.2.

The closed loop transfer function from the command signal uc (k) to the
output y(k) can be readily obtained as
B(z)T (z)
Y (z) = Uc (z) (5.5)
A(z)R(z) + B(z)S(z)
If the desired C.P. is Acl , then let

Acl (z) = A(z)R(z) + B(z)S(z) (5.6)

Question: Can we solve R(z) and S(z) from above polynomial equation?
Yes, under certain conditions.
B(z)T (z)
Y (z) = Uc (z)
Acl (z)
In the state-space approach, there are two desired polynomials, one for the
state-feedback controller Ac , and one for the output-feedback observer, Ao ,
and the C.P. for the overall system is

Acl (z) = Ac (z)Ao (z) (5.7)

So
B(z)T (z)
Y (z) = Uc (z) (5.8)
Ac (z)Ao (z)
How to choose T (z)?
T (z) can be chosen to satisfy other design requirements.

130
 Figure 5.2: Two-Degree-of-Freedom Controller

If perfect match of the reference model is not required, one simple way to
choose T (z) is
T (z) = to Ao (z) (5.9)
Such that
to B(z)
Y (z) = Uc (z) (5.10)
Ac (z)
Note that the dynamics of the closed-loop system is low-order now by can-
celing out the stable poles and zeros.

131
to is chosen to obtain the desired static gain of the system. Note that the
static gain of the closed loop is tAo B(1)
c (1)
. For example, to have unit static gain,
we simply let
Ac (1)
to = (5.11)
B(1)
But you may not be satisfied with above choice of T (z) as the zeros of the
closed loop do not match the zeros of the reference model. If you want to
achieve perfect match, it is possible under certain conditions which will be
discussed later in this class.

Keep in mind that there are always multiple ways to control a system, once
the poles are properly assigned, T (z) can be chosen to satisfy other design
specifications.

Rules to follow:

1. The order of R, S and T should be as low as possible! This is so called

Occam’s razor , the simpler, the better.

2. The controller has to be causal: The input cannot depend upon the
future values of the output.

Example:
R = z + 1, S = z 2 + 1
then we have
Ru = −Sy ⇒
u(k + 1) + u(k) = −y(k + 2) − y(k)
What is the problem with such design? Such kind of controllers can not be
implemented, although mathematically it is fine.

Causality condition:
DegR ≥ DegS (5.12)

Example: Double integrator

A(z)y = B(z)u
A(z) = (z − 1)2
h2
B(z) = (z + 1)
2

132
Let’s try the simplest controller, the proportional control,

R = 1 S = s0

So
h2
AR + BS = (z − 1)2 + s0 (z + 1)
2
If the desired C.P. is
Acl = z 2 + p1 z + p2
Can we match the desired C.P.?
Two equations with one parameter:—-mission impossible!

Therefore let’s try the first-order controller,

R = z + r1
S = s0 z + s1

Then
h2
AR + BS = (z − 1)2 (z + r1 ) + (z + 1)(s0 z + s1 )
2
h2 h2 h2
= z 3 + (r1 + s0 − 2)z 2 + (1 − 2r1 + (s0 + s1 ))z + r1 + s1
2 2 2
If the desired C.P. is

Acl = z 3 + p1 z 2 + p2 z + p3

Compare the coefficients

h2
r1 + s0 − 2 = p1
2
h2
1 − 2r1 + (s0 + s1 ) = p2
2
h2
r1 + s1 = p3
2
We have
3 + p1 + p2 − p3
r1 =
4
5 + 3p1 + p2 − p3
s0 = (5.13)
2h2
3 + p1 − p2 − 3p3
s1 = −
2h2

133
Next question is then how to choose T (z)?

We notice that Acl is of third order, let

Acl (z) = Ac (z)Ao (z)

Since the third order polynomial always has a real root, we can choose this
as Ao (z), Let
T (z) = to Ao (z)
where
Ac (1)
to =
B(1)
Remark: The controller must have sufficient degree of freedom. In this
example, first order dynamical controller is used instead of the proportional
controller. Increasing the order of the controller by one will give an increase
of two parameters. But the degree of the desired C.P. is increased by one,
and hence the total number of equations for solving the design parameters
is increased by one. For instance, if a second order controller is used in the
double integrator example, then there may be infinite number of solutions
to achieve the same goal.

Now it is time to ask the following question:

Given polynomials A(z) and B(z) whose maximum degree is n, and a third
polynomial Acl (z), under what conditions does the solution exist for

A(z)R(z) + B(z)S(z) = Acl (z) (5.14)

Diophantine equation

First of all, let’s see under what conditions the solution may not exist?
Hint: In the state space approach, if the system is uncontrollable, then the
problem cannot be solved. Translating that into the input-output model,
we may state that:
If A(z) and B(z) have a common factor (z − c), then Acl (z) must contain
the same factor (z − c), otherwise, the solution does not exist.
And we know, if A(z) and B(z) have a common factor, then this pole cannot
be changed by linear state-feedback controller, and the minimal realization
of the system is uncontrollable.

134
Do we need other conditions to guarantee the existence of solution?
Hint: if A(z) and B(z) have no common factors, then the minimal real-
ization of the system is both controllable and observable. Therefore, the
solution exists at least for the minimal realization of the system. So our
intuition tells us that we don’t need extra conditions.

Let’s see if our intuition is true. Let’s examine the simple case of second
order system.
A(z) = a0 z 2 + a1 z + a2
B(z) = b0 z 2 + b1 z + b2
Acl (z) = p0 z 3 + p1 z 2 + p2 z + p3
Design a first order controller

R(z) = r0 z + r1
S(z) = s0 z + s1

From
AR + BS = Acl
and compare the coefficients, we have

z3 : p0 a0 r0 + b0 s0
z2 : p1 a1 r0 + a0 r1 + b1 s0 + b0 s1
z: p2 a2 r0 + a1 r1 + b2 s0 + b1 s1
1: p3 a2 r1 + b2 s1

Rewrite it in a compact matrix form, we have

    
a0 0 b0 0 r0 p0
 a1 a0 b1 b0   r1   p1 
  = 
 a2 a1 b2 b1   s0   p2 
0 a2 0 b2 s1 p3

Sylvester matrix

If Sylvester matrix is nonsingular, then the solution exists.

What is the condition for nonsingularity of Sylvester matrix?

Sylvester Theorem:

135
Two polynomials A(z) and B(z) have maximum order of n. They are rel-
atively prime, i.e. they have no common factors, if and only if the corre-
sponding Sylvester matrix Me is nonsingular, where Me is defined to be the
following 2n × 2n matrix:

a0 0 ··· 0 b0 0 ··· 0
 
.. .. .. .. 
a1 a0 . . b1 b0 . . 


 .. .. .. ..


 . a1 . 0 . b1 . 0 

 .. .. .. .. .. .. 

. . . a0 . . . b0 

Me =  (5.15)

.. .. .. .. 

 an . . a1 bn . . b1  
 .. .. .. .. .. .. 

 0 . . . 0 . . . 
 .. .. .. .. .. .. .. .. 
 . . . . . . . . 
0 ··· 0 an 0 ··· 0 bn
| {z }
<−−−−−−n−−−−−−><−−−−−−n−−−−−−>

where
A(z) = a0 z n + a1 z n−1 + · · · + an
(5.16)
B(z) = b0 z n + b1 z n−1 + · · · + bn
The proof is not included here due to time constraint. I hope you can figure
it out after the lecture.

Using Sylvester Theorem, we can solve the simplest pole-place problem

as follows, which may provide a solid basis for tackling more complicated
design problem.

Given a system described by

A(z)y(k) = B(z)u(k)

Let
Deg A = n, Deg B ≤ n
(5.17)
Deg Acl = 2n − 1
Design a controller of degree n − 1,

Deg R = n − 1
Deg S = n − 1

136
Then if the minimal realization of the system is both controllable and ob-
servable (i.e., A and B have no common factors), R and S can be uniquely
determined from the Diophantine equation
A(z)R(z) + B(z)S(z) = Acl (z)
by  
p0
 
r0
 ..   p1 
 .  
 ..


.
 
 rn−1  
 = Ms−1 

(5.18)


 s0  .. 
.
  
 
 ..  
..

 .  
 .


sn−1 p2n−1
You may notice that the controllable and observable condition assures the
existence of inverse of the Sylvester matrix, Ms−1 , in above solution. But
what about the condition of the degree of the desired polynomial, Acl (z)?
Why is it required to be 2n-1? Or in other words, the first coefficient, p0
cannot be zero? What would happen if we let p0 = 0?

If p0 = 0, the solution in above equation (5.18) is still available. But it may

not be what you want. We will use the double-integrator to illustrate this
point.

If the order of the controller is allowed to be larger than n − 1, then the

number of solution to the Diophantine equation may be infinity. The extra
freedom can be used to satisfy other design specifications.

Example: Double Integrator

A(z) = (z − 1)2 = z 2 − 2z + 1
h2
B(z) = (z + 1)
2
n = 2, hence the degree of Acl should be 2n − 1 = 3,
Acl = z 3 + p1 z 2 + p2 z + p3
Then  
1 0 0 0
  
r0 1
h2
 −2 1 2 0  r1  
  p1 
h2   s  =  (5.19)

h2

1 −2 p2 

 2 2  0
h2 s1 p3
0 1 0 2

137
and we can obtain
r0 = 1
3 + p1 + p2 − p3
r1 =
4
5 + 3p1 + p2 − p3
s0 =
2h2
3 + p1 − p2 − 3p3
s1 = −
2h2
which is the same answer as those obtained by direct calculation.
As mentioned before, the degree of the desired polynomial, 2n-1=3, is im-
portant. Let’s see what would happen if the degree of Acl is 2, and hence in
the form of
Acl = z 2 + p2 z + p3 = 0z 3 + z 2 + p2 z + p3
A simple check of the equation (5.19), by replacing 1 with 0, and p1 = 1 in
the right hand would lead to
r0 = 0
and the degree of R(z) will be lower than that of S(z), which cannot be
implemented in reality (violation of causality condition) as discussed before.
That’s why if the degree of the denominator in the reference model, Am (z)
is lower than 2n-1, we have to increase its order by multiplying with Ao (z)
rather than setting higher order terms to be zero. This extra term of Ao (z)
can be canceled out later by properly choosing T (z), and the goal of match-
ing Am (z) can still be achieved.

5.4 Pole-Placement Problem— More Realistic Sit-

uations
We have already given the solution to pole-placement problem for the input-
output model (5.1). In practical problems, we have to consider other realistic
factors. For instance, we may wish to decrease the order of the controller
and hence the overall system in order to simplify the design. Sometimes, we
also have to consider how to deal with the external disturbances. To achieve
the goal of lowering the order of the system, pole-zero cancelation can be
used.

138
5.4.1 Pole-Zero Cancelation
Consider the system described by

A(q)y(k) = B(q)u(k)

Let the output feedback controller be

R(q)u(k) = T (q)uc (k) − S(q)y(k)

Then we have
B(z)T (z)
Y (z) = Uc (z)
Acl
where
Acl (z) = A(z)R(z) + B(z)S(z)
The idea is to reduce the order of the closed-loop system as low as possible.
To achieve this, we may properly choose T (z) to cancel out Ao (z). Besides
this, is there any other way to further reduce the degree?
Another way is to cancel out the stable zeros in B(z)! To achieve this, Acl (z)
must have the stable zeros as the factors.

If A(z) has stable poles, to simplify the controller, we may even leave those
stable poles of A(z) alone, which means Acl (z) have those stable poles as
factors too.

More precisely, let

A = A+ A−

(5.20)
B = B+B−
Where A+ and B + are the stable factors, and A− and B − are the unstable
factors. Then let
Acl (z) = A+ B + Ācl (5.21)
and
T (z) = A+ T̄ (5.22)
Then the closed loop transfer function becomes
B(z)T (z) B + B − A+ T̄ B − T̄
= = (5.23)
Acl (z) A+ B + Ācl Ācl
The corresponding Diophantine equation

A(z)R(z) + B(z)S(z) = Acl (z)

139
becomes
A+ A− R + B + B − S = A+ B + Ācl (5.24)
Obviously
R = B + R̄
(5.25)
S = A+ S̄
And we have another Diophantine equation of lower order polynomials which
is easier to solve.

ĀR̄ + B̄ S̄ = Ācl (5.26)

The canceled factors must correspond to stable modes. In practice, it is use-
ful to have more stringent requirements on allowable cancelations. Some-
times cancelation may not be desirable at all. In other cases, it may be
reasonable to cancel zeros that are sufficiently well damped. As we dis-
cussed before, the canceled mode still affect the transient response. If it is
well-damped, then the effects will decay to zero very fast any way. Oth-
erwise, the performance of the system may be affected by these canceled
modes. For instance, if the canceled modes are not well-damped, and are
negative or complex, then undesirable oscillations may be introduced to the
system.

Example: Cancelation of zeros

Let the open loop transfer function be

k(z − b)
H(z) =
(z − 1)(z − a)

where b < 1 and a > 1

Assume the desired close-loop system is characterized by the transfer func-

tion
z(1 + p1 + p2 )
Hm (z) = 2
z + p1 z + p2
z = b is a stable zero, which may be canceled out.

The order of the system is 2, therefore,

Deg R = 1, Deg S = 1
Deg Acl = 2n − 1 = 3

140
But the desired closed-loop is of second order. One possible way to reduce
the order of the system is to cancel out the stable zero z − b,
Then let
R=z−b
And
Acl = (z − b)(z 2 + p1 z + p2 )
From the Diophantine equation we have
AR + BS = A(z − b) + k(z − b)S = Acl = (z − b)(z 2 + p1 z + p2 )
we have
A + kS = z 2 + p1 z + p2
(z − 1)(z − a) + k(s0 z + s1 ) = z 2 + (ks0 − a − 1)z + a + ks1 = z 2 + p1 z + p2
Therefore
ks0 − a − 1 = p1
a + ks1 = p2
and
a + 1 + p1
s0 =
k
p2 − a
s1 =
k
Then the transfer function for the closed loop is
k(z − b)T kT
2
= 2
(z − b)(z + p1 z + p2 ) (z + p1 z + p2 )
We still have the freedom to choose proper T .

Compare the above form to the desired transfer function Hm , we simply let
kT = z(1 + p1 + p2 )
or
z(1 + p1 + p2 )
T = = t0 z
k
and the resulting controller is then
R(q)u(k) = T (q)uc (k) − S(q)y(k)
(z − b)u = t0 zuc − (s0 z + s1 )y
or
u(k) = bu(k − 1) + t0 uc (k) − s0 y(k) − s1 y(k − 1)

141
5.4.2 Disturbance Rejection Problem
Consider a system described by
B(z)
Y (z) = (U (z) + D(z))
A(z)
where d(k), i.e. D(z), is the load disturbance. Let the two-degree-of-freedom
controller be designed as
T (z) S(z)
U (z) = Uc (z) − Y (z)
R(z) R(z)
The closed-loop transfer function from the disturbance to the output, Dcl (z),
can be derived as
Y (z) R(z)B(z)
Dcl (z) = = (5.27)
D(z) A(z)R(z) + B(z)S(z)
The block diagram is shown in Fig. 5.3.
Let’s first consider the case when the disturbance is an unknown constant.
To eliminate the effect of the constant disturbance, we should design the con-
troller such that the static gain of the closed-loop transfer function, Dcl (z),
relating the disturbance to the output, is zero. Since the static gain is simply
R(1)B(1)
Dcl (1) =
A(1)R(1) + B(1)S(1)
all we need to do is letting R(z) have (z − 1) as a factor,

R(z) = R0 (z)(z − 1) (5.28)

Then
R(1) = 0 (5.29)
This design is essentially putting the integrator in the controller.

Example:
y(k + 1) = 3y(k) + u(k) + d(k)
where d(k) is a constant disturbance. In order to cancel out the disturbance
R must contain the factor (z − 1).

A(z) = z − 3
B(z) = 1

142
 Figure 5.3: Two-Degree-of-Freedom Controller with load disturbance.

The simplest controller is the first order controller

R(z) = z − 1
S(z) = s0 z + s1

AR + BS = (z − 3)(z − 1) + s0 z + s1 = z 2 + (s0 − 4)z + s1 + 3

If the desired C.P. is
Acl (z) = z 2 + p1 z + p2

143
we have
s0 − 4 = p1
s1 + 3 = p2
and
s0 = 4 + p1
s1 = p2 − 3
The closed loop transfer function is then

T (z)
z2 + p1 z + p2

where T (z) can be chosen to meet other requirements.

Other disturbances
What if the disturbance is not a constant?
If the disturbance d(k) is deterministic rather than purely random, then it
is still possible to reject the effect of the disturbance. If the frequency of the
disturbance, ωd , can be estimated, then we can use the frequency response
of the transfer function, Dcl (z), to design the controller.

If d(k) is a sinusoid with frequency ωd , then the frequency response due to

this sinusoid is expressed by

Dcl (ejωd h ) (5.30)

From the expression of Dcl (z), if we can design R(z) such that

R(ejωd h ) = 0

then the frequency response due to this sinusoid will be zero at steady state.
In order to achieve this, let’s construct a polynomial as follows

(z − ejωd h )(z − e−jωd h ) = z 2 − 2 cos ωd hz + 1

Then let R(z) contain above polynomial as a factor, the frequency response
due to the sinusoid disturbance will be zero. If there are multiple frequencies
in the disturbance, we just add more factors into R(z). It is as simple as
that.

144
5.4.3 Tracking Problem — Command Signal Following
If the desired transfer function is
Bm (z)
Am (z)

Can we make the transfer function of the closed-loop match this?

We know by output feedback control

R(q)u(k) = T (q)uc (k) − S(q)y(k)

We can have
B(z)T (z)
Y (z) = Uc (z)
Acl (z)
Question: is it possible to make

B(z)T (z) Bm (z)

=
Acl (z) Am (z)

Obviously, there is no problem to make the denominators match each other.

The problem is how to deal with B(z)?

• If the zeros of B(z) are all stable, then we can make Acl contains B
such that B will be canceled out, and then let T = Bm .

• If the some zeros of B(z) are unstable, it is impossible to make the

transfer function match the ideal one unless Bm contains all the un-
stable zeros of B as factors. That’s why tracking an arbitrary signal is
possible only for system with stable inverse (all zeros are stable). This
is the same conclusion as we found out for the state space approach.

Is it really impossible to follow a reference model perfectly for system with

unstable inverse? Can we overcome this problem by increasing the freedom
of the controller? For instance, in the two-degree-of-freedom controller as
shown in Fig. 5.2, the control input is the summation of the feedforward
control signal, uf f and uf b ,

u(k) = uf f (k) + uf b (k)

where
T (q)
uf f (k) = uc (k)
R(q)

145
and
S(q)
uf b (k) = − y(k)
R(q)
Note that the transfer functions for generating the two inputs share the
same denominator R(q). In order to gain extra freedom, it is also possible
to relax this constraint, and let

uf f (k) = Hf f (q)uc (k)

where the feedforward transfer function Hf f (q) gives you additional freedom.

Figure 5.4: Another type of two-degree-of-freedom controller

What would be the advantages by doing this?

Easy calculation will give us the following transfer functions if T /R is re-

placed by Hf f (z),
Y (z) BR
= Hf f
Uc (z) AR + BS
U (z) AR
= Hf f
Uc (z) AR + BS

146
In order to match the reference model, design the feedback controller S(z)/R(z)
such that
AR + BS = Am
and choose the feedforward controller,
Bm
Hf f (z) =
RB
we will have
Y (z) Bm
=
Uc (z) Am
So the transfer function indeed matches the reference model perfectly. But
how about the input signal? We have to make sure the input signals are all
bounded. For this purpose, we have to check the transfer function from the
command signal to the input,

U (z) ABm
=
Uc (z) BAm
Still, B(z) is required to be stable to keep the input signal bounded. This
shows that the stable inverse condition is essential for perfect tracking, which
cannot be simply got rid of by adding more design freedoms.

In summary, we have following rules of thumb for polynomial design,

R(q)u = T (q)uc − S(q)y

1. Occam’s razor — the simpler, the better.
2. Based on the design specifications, determine the constraints on R.
For instance,
• If zero cancelation is needed, then R must contain the stable zero
factor B +
• If Disturbance rejection is required, then R must contain certain
factors to reject the disturbance.
3. Causal conditions: DegR ≥ DegS
4. Design R and S first by solving the Diophantine equation AR + BS =
Acl
5. Choose T or Hf f at the final stage, to satisfy other design require-
ments, such as zero placement.

147
6. For the simplest case, the order of the controller is limited to n − 1.
However, the order of the controller can be increased in order to meet
other design requirements.

148
Chapter 6

Predictive Control —
Solution to Tracking
Problem

6.1 Introduction
In any control system design problem one can distinguish five important con-
siderations: stability, transient response, tracking performance, constraints,
and robustness.
1. Stability. This is concerned with stability of the system, including
boundedness of inputs, outputs, and states. This is clearly a prime
consideration in any control system.
2. Transient response. Roughly this is concerned with how fast the sys-
tem responds. For linear systems, it can be specified in the time do-
main in terms of rise time, settling time, percent overshoot, and so
on, and in the frequency domain in terms of bandwidth, damping,
resonance, and so on.
3. Tracking performance. This is concerned with the ability of the system
to reproduce desired output values. A special case is when the desired
outputs are constant, in which case we often use the term output
regulation, in lieu of tracking.
4. Constraints. It turns out that in the linear case if the system model is
known exactly and there is no noise, then arbitrarily good performance
can be achieved by use of a sufficiently complex controller. However

149
 one must bear in mind that there are usually physical constraints that
must be adhered to, such as limits in the magnitude of the allowable
control effort, limits in the rate of change of control signals, limits on
internal variables such as temperatures and pressures, and limits on
controller complexity. These factors ultimately place an upper limit
on the achievable performance.
5. Robustness. This is concerned with the degradation of the performance
of the system depending on certain contingencies, such as unmodeled
dynamics, including disturbances, parameter variations, component
failure, and so on.
In this module, we primarily treat sampled data control systems so that our
inputs and outputs are sequences of numbers. Typically, the actual process
will be continuous in nature. Suitable data models can then be obtained by
the techniques discussed in previous lectures. It must be borne in mind that
a design based on discrete-time models refers only to the sampled values of
the input and output. Usually, the corresponding continuous response will
also be satisfactory, but in certain cases it may be important to check that
the response between samples is acceptable.

We shall begin our discussion by emphasizing tracking performance and in-

troduce a very simple form of control law which we call a one-step-ahead
controller. The basic principle behind this controller is that the control
input is determined at each point in time so as to bring the system out-
put at a future time instant to a desired value. This is a very simple idea
but, nonetheless, has considerable intuitive appeal. We shall show that the
scheme works well not only for linear systems but can also be applied in a
straightforward fashion to a large class of nonlinear systems. However, as
with all design techniques, one must be aware of its limitations as well as its
applications. Potential difficulties with this approach are that an excessively
large effort may be called for to bring the output to the desired value in one
step.

We are therefore motivated to look at alternative control schemes which

allow us to limit the control effort needed to achieve the objective. This
will then lead us to consider weighted one-step-ahead controllers wherein a
penalty is placed on excessive control effort. We shall subsequently show
that the one-step-ahead controller is a special case of the weighted one-step-
ahead controller. Again, we shall point to its limitations and show that it
may still lead to difficulty in some cases.

150
6.2 Minimum Prediction Error Controllers
We can think of a model as providing a way of predicting the future out-
puts of a system based on past outputs and on past and present inputs. It
therefore seems reasonable to turn this around and ask what control action
at the present instant of time would bring the future output to a desired
value. This will be especially simple if the future outputs are a linear func-
tion of the present control, since then the determination of the control effort
involves the solution of a set of linear equations.

We consider first the case of linear dynamical system. We have seen in previ-
ous lectures that the input-output properties of these systems (for arbitrary
initial states) can be described by either state-space model or transfer func-
tion.

Let a system be described by difference equation

y(k)+a1 y(k−1)+· · ·+an y(k−n) = b0 u(k−d)+b1 u(k−d−1)+· · ·+bn1 u(k−d−n1 )
(6.1)
which can be expressed as
A(q −1 )y(k) = B(q −1 )u(k) (6.2)
where
A(q −1 ) = 1 + a1 q −1 + · · · an q −n
B(q −1 ) = q −d (b0 + b1 q −1 + · · · bn1 q −n1 ), b0 6= 0 (6.3)
= q −d B 0 (q −1 )

6.2.1 One-Step-Ahead Control: The Case of d = 1

We first consider the case of time delay d = 1. Here we will consider one-
step-ahead control, which brings y(k + 1) to a desired value r(k + 1) in one
step. Let’s rewrite the model (6.1) in a predictor form as (6.4).
y(k+1) = −a1 y(k)+· · ·−an y(k−n+1)+b0 u(k)+b1 u(k−1)+· · ·+bn1 u(k−n1 )
(6.4)
or equivalently,
y(k + 1) = α(q −1 )y(k) + β(q −1 )u(k) (6.5)
where
α(q −1 ) = −a1 + · · · − an q −(n−1)
(6.6)
β(q −1 ) = B 0 (q −1 ) = (b0 + b1 q −1 + · · · bn1 q −n1 ), b0 6= 0

151
Obviously, the simple solution of tracking problem is to replace y(k + 1),
the future output at the next step, in model (6.4) with r(k + 1), the desired
output at the next step,

r(k+1) = −a1 y(k)+· · ·−an y(k−n+1)+b0 u(k)+b1 u(k−1)+· · ·+bn1 u(k−n1 )

(6.7)
and then solve for u(k) as

1
u(k) = [r(k+1)+a1 y(k)+· · ·+an y(k−n+1)−b1 u(k−1)+· · ·−bn1 u(k−n1 )]
b0
(6.8)
If control law (6.8) is used, it follows immediately by comparing (6.4) and
(6.7) that
y(k + 1) = r(k + 1)
It appears that the above controller design can be directly extended to the
case of d > 1, by writing the model (6.1) in the form of

y(k+d) = −a1 y(k+d−1)+· · ·−an y(k+d−n)+b0 u(k)+b1 u(k−1)+· · ·+bn1 u(k−n1 )

(6.9)
and solve for u(k) from the equation below

r(k+d) = −a1 y(k+d−1)+· · ·−an y(k+d−n)+b0 u(k)+b1 u(k−1)+· · ·+bn1 u(k−n1 )

Unfortunately, this type of controller violates the law of causality since the
future inputs y(k + d − 1) is not available at time k!

Therefore, we need to put the input-output model (6.1) in a d-step-ahead

prediction form which is more suitable for design of controllers.

6.2.2 Prediction with Known Models

If a model for the evolution of the time series of interest is known, it is
possible to construct suitable predictors from the model. In fact, in the case
that the model is linear and finite-dimensional, suitable predictors can be
obtained by simple algebraic manipulations of the model. We show how this
is achieved below.

Consider a linear discrete-time system described by model (6.2) as

A(q −1 )y(k) = B(q −1 )u(k) = q −d B 0 (q −1 )u(k) (6.10)

152
Then the output of the system at time t+d can be expressed in the following
predictor form:

y(k + d) = α(q −1 )y(k) + β(q −1 )u(k) (6.11)

where
α(q −1 ) = G(q −1 )
(6.12)
β(q −1 ) = F (q −1 )B 0 (q −1 )
and F (q −1 ) and G(q −1 ) are the unique polynomials satisfying

1 = F (q −1 )A(q −1 ) + q −d G(q −1 )
F (q −1 ) = 1 + f1 q −1 + · · · + f d−1 q −d+1 (6.13)
G(q −1 ) = g0 + g1 q −1 + · · · + g n−1 q −n+1

Proof
Multiplying (6.10) by F (q −1 ) gives

F (q −1 )A(q −1 )y(k) = F (q −1 )B(q −1 )u(k) = q −d F (q −1 )B 0 (q −1 )u(k)

Using (6.13), we have

(1 − q −d G(q −1 ))y(k) = q −d F (q −1 )B 0 (q −1 )u(k)

which can be rewritten as

y(k) = q −d G(q −1 )y(k) + q −d F (q −1 )B 0 (q −1 )u(k)

or
y(k + d) = α(q −1 )y(k) + β(q −1 )u(k)
where
α(q −1 ) = G(q −1 )
β(q −1 ) = F (q −1 )B 0 (q −1 )
How to solve equation (6.13)? Does this equation look familiar to you? If
you compare it with the equation (5.14), you may realize that it is also a
Diophantine equation, and you know how to solve it.

There is also another natural way to convert the original input-output model
into predictor form as shown in the following example.

153
Example 6.1
Consider the system

y(k + 2) = y(k + 1) + u(k)

as mentioned before, since y(k+1) appears in the right hand, it is not in the
proper form of predictor, in which all the terms in the right hand have to
be from the present or the past.
So how to deal with the “future term”, y(k+1)?
A simple answer is just expressing y(k+1) in terms of information in the
past!
y(k + 1) = y(k) + u(k − 1)
Then plug it into the original equation, we have

y(k + 2) = y(k) + u(k) + u(k − 1)

What if the delay d > 2? like

y(k + 3) = y(k + 2) + y(k + 1) + u(k)

You just repeat above procedure! First express y(k+2)in terms of y(k+1),
y(k), u(k) etc. Then express y(k+1) in terms of y(k), y(k-1), u(k),u(k-1), etc.

From this example, you can see how easy it is to convert the equation into
predictor form! Personally, I would rely on this natural way to do the job
instead of solving the Diophantine equation.

6.2.3 One-Step-Ahead Control: General Case

Now we are ready to use the predictor model (6.11) to design the feedback
controller, which brings the output at time k + d, y(k + d), to some desired
bounded value r(k + d) as follows. Set

r(k + d) = α(q −1 )y(k) + β(q −1 )u(k) (6.14)

in which u(k) can be easily solved.

The resulting closed-loop system is described by

y(k) = r(k); k>d (6.15)

154
 B(q −1 )u(k) = A(q −1 )r(k); k >d+n (6.16)
The resulting closed-loop system has bounded inputs and outputs provided
that the system has a stable inverse, i.e., the zeros of the polynomial lie
inside the closed unit disk.

Example 6.2
Consider a simple system with disturbance described by the following state-
space equations:

ω(k + 1) = ω(k); k > 0 (disturbance model)

x(k + 1) = u(k); k > 0 (system model)
y(k) = x(k) + ω(k) k > 0 (output)

Let’s first derive the input-output model

y(k + 1) = x(k + 1) + ω(k + 1)

= u(k) + ω(k)
= u(k) + (y(k) − x(k))
= y(k) + u(k) − u(k − 1)

The one-step-ahead control law (6.14) is then

u(k) = r(k + 1) − y(k) + u(k − 1)

and this gives a closed-loop system characterized by

y(k) = r(k)
u(k) − u(k − 1) = r(k + 1) − r(k)

Note that although the zero is not strictly inside the unit disk, the inputs
will still be bounded, which implies that the uncontrollable pole (z=1 in this
example) can lie on the unit disk.

6.2.4 Weighted One-Step-Ahead Control

We note in passing that the one-step-ahead feedback control law (6.14) min-
imizes the following cost function comprising the squared prediction error:
1
J1 (k + d) = [y(k + d) − r(k + d)]2 (6.17)
2

155
As noted in Section 6.1, excessive control effort may be called for to bring
y(k + d) to r(k + d) in one step. We therefore consider a slight generalization
of the cost function (6.17) to the form (6.18) given below, which aims to
achieve a compromise between bringing y(k + d) to r(k + d) and the amount
of effort expended.
1 λ
J2 (k + d) = [y(k + d) − r(k + d)]2 + u(k)2 (6.18)
2 2
We shall call the control law minimizing (6.18) at each time instant a
weighted one-step-ahead controller. The term “one-step-ahead” is still used
because we aim to minimize (6.18) on a one-step basis rather than in an
average sense over an extended time horizon.

Substituting (6.11) into (6.18) gives

1 λ
J2 (k + d) = [α(q −1 )y(k) + β(q −1 )u(k) − r(k + d)]2 + u(k)2 (6.19)
2 2
Differentiating with respect to u(k) and setting the result to zero gives

β0 [α(q −1 )y(k) + β(q −1 )u(k) − r(k + d)] + λu(k) = 0 (6.20)

the control input u(k) can be easily obtained as

β0 [r(k + d) − α(q −1 )y(k) − β 0 (q −1 )u(k − 1)]

u(k) = (6.21)
β02 + λ

where
β 0 (q −1 ) = q(β(q −1 ) − β0 ) = β1 + β2 q −1 + · · ·
The control law is (6.20). This equation can also be written using the
predictor representation (6.11) as

β0 [y(k + d) − r(k + d)] + λu(k) = 0 (6.22)

Multiplying by A(q −1 ) gives

λ
β0 [A(q −1 )y(k + d) − A(q −1 )r(k + d) + A(q −1 )u(k)] = 0
β0
and using (6.10) results in
λ
β0 [B 0 (q −1 )u(k) − A(q −1 )r(k + d) + A(q −1 )u(k)] = 0
β0

156
It follows that
λ
[B 0 (q −1 ) + A(q −1 )]u(k) = A(q −1 )r(k + d) (6.23)
β0
Similarly, multiplying (6.22) by B 0 (q −1 ) gives
λ
[B 0 (q −1 ) + A(q −1 )]y(k + d) = B 0 (q −1 )r(k + d) (6.24)
β0
Obviously, the resulting closed-loop system has bounded inputs and outputs
provided that the zeros of the polynomial [B 0 (z −1 ) + λ A(z −1 )] lie inside
β0
the unit disk.

It is important to note that the closed-loop system is stable if [B 0 (z −1 ) +

λ A(z −1 )] is stable. This represents a slight relaxation of the requirement
β0
for the one-step-ahead controller. We now have an additional degree of
freedom. If λ = 0, the result reduces to the one-step-ahead result.
But there is no free lunch, in one-step-ahead control, we can achieve perfect
tracking, i.e., y(k + 1) = r(k + 1). If we want to keep lower control efforts,
the weighted one-step-ahead controller will in general lead to a steady-state
tracking error. This is because a compromise is made between bringing
y(k + d) to r(k + d) and keeping u(k) small.

Example 6.3
Consider a system in the form of (6.10) with d = 1 and

A(q −1 ) = 1 − 2q −1
B(q −1 ) = 1 + 3q −1

This system does not satisfy the condition of stable inverse, and thus u(k)
will not be bounded for a one-step-ahead control law. The weighted one-
step-ahead controller gives a closed-loop system whose modes are the zeros
of
1 + 3z −1 + λ(1 − 2z −1 ) = 0
And the pole of the closed loop system is z = 2λ − 3 . Therefore the system
λ+1
2
is asymptotically stable for 3 < λ < 4.

The idea of bringing a predicted system output to a desired value is not

restricted to linear models. The idea can be used with nonlinear models as

157
well. This is particularly straightforward in the case of bilinear systems. For
example, in the first-order case, the bilinear model has the simple form

y(k + 1) = ay(k) + bu(k) + nu(k)y(k) (6.25)

The corresponding one-step-ahead bilinear control law is simply

r(k + 1) = ay(k) + bu(k) + nu(k)y(k) (6.26)

which results in

r(k + 1) − ay(k)
u(k) =
b + ny(k)
The one-step-ahead control principle is very simple and may not be as gen-
eral as other control laws, such as those based on pole assignment or linear
optimal control. However, as we have seen above, the one-step-ahead control
principle does have the advantage of being able to utilize the true nonlinear
dynamical model of the process. Thus it may be better able to capture the
intrinsic features of the control problem than a control law based on a linear
approximation. And the strongest point of this type of controller is that it
can track arbitrary desired output r(k), whereas other controllers may not
be able to accomplish.

As discussed earlier, the motivation for introducing the weighted one-step-

ahead control is that we need to achieve a compromise between bringing
y(k + d) to r(k + d) and the cost of the control effort in the real world
control system applications. This actually is the starting point of the so
called “Model Predictive Control”, which will be covered in greater details
in part II of this module.

6.3 Further Beyond

Now I hope you have mastered some of the fundamental knowledge in con-
troller designs for improving the performance of the system. Of course, we
have made many assumptions in order to construct the controller properly.
In reality, these assumptions may be invalid, which stimulate us to explore
more in coping with the complex real world.

We have made three critical assumptions through out the module:

158
 1. The system is linear.

2. The mathematical model is completely known.

3. The dynamic equations governing the system do not change with time.
In other words, the system is time-invariant.

Regarding the first assumption, it is not a big issue in many of the practical
problems, where we are only interested in controlling small deviations from
certain operating point. If the deviation is big, and the linearization does
not work any more, you may have to resort to nonlinear control technology,
which still has a lot of open questions. The biggest challenge of nonlinear
system is that we don’t have superposition principle to guide us any more!
The most often used trick in solving the nonlinear control problem is to
choose the control input properly such that the closed loop system is linear
just as I showed you in the introduction of this module. If you are inter-
ested in exploring more in this topic, you can study our module: EE6105
Non-linear Dynamics and Control.

If the model is unknown, or partially known, the first option is to use model-
free controller such as the classical PID control. If PID controller does not
work well, then you may have to estimate the model parameters and use
the adaptive control scheme to solve the problem, which is covered by our
modules: EE5104/EE6104 Adaptive control systems.

We only examine the single-input, single-output system in this module. If

there are multiple inputs and multiple outputs, there are other interesting
phenomena arising, which are covered by our module: EE5102 Multivari-
able Control Systems.

We have formulated our control problem in the framework of model refer-

ence control, in which the objective is to track a reference signal. If there
are many other constraints to consider in the problem, like the cost of the
control inputs, it is better to formulate it as an optimization problem. And
we have another module devoted to this topic: EE5105 Optimal Control
Systems.

How to deal with time-varying system? That is a very challenging problem,

and currently we do not have a systematic way in dealing with this. In the
past decade or so, a new field called hybrid systems, which involves multiple
models, switching, and tuning, has emerged as a promising tool in coping

159
with time-varying system. We will try to offer this module to you in the
future.

6.4 Notes and References

Most of the materials covered in this chapter can be found in the following
book

• G. C. Goodwin and K. S. Sin, Adaptive Filtering Prediction and Con-

trol, Prentice Hall, 1984.

160