Lectures on Multivariable Feedback Control

Ali Karimpour
Department of Electrical Engineering, Faculty of Engineering, Ferdowsi
University of Mashhad (Fall 2008)

Chapter1: Linear Algebra

To start the multivariable control course, learning the tools of linear algebra is
essential. We now present a checklist of important notions from linear algebra for
you to review. Use your favorite linear algebra text for further investigation.

1-1 Vector Spaces

Definition 1-1 (Vector space)

A set of vectors and a field of scalars with vector addition (which must be
associative and commutative), scalar multiplication (with its own associativity and
distributivity properties), the existence of a zero vector 0 such that x + 0 = x for
every vector x, and the normalization conditions 0x = 0, 1x = x is a vector space.
Chapter 1 Lecture Notes of Multivariable Control

1-2 Norms

To meter the lengths of vectors in a vector space we need the idea of a norm. A
norm for a vector space F over the field of real numbers R or complex numbers C
is defined to be a function that maps vectors x to nonnegative real numbers x ,

and that satisfies the following properties:

1 Positivity: x > 0 for x ≠ 0

2 Homogeneity: αx = α x for any scalar α .

3 Triangle inequality: x + y ≤ x + y ∀ x, y ∈ F :

1-2-1 Norms of Vectors

One of the most important norms for vectors is p-norm. It is defined as:
1
⎛ p ⎞p
x = ⎜ ∑ ai ⎟ p ≥1 1-1
⎝ i ⎠
p

- For p=1 we have 1-norm or sum norm i.e. x 1 = ∑ ai

i

∑a
2
- For p=2 we have 2-norm or familiar Euclidian norm i.e. x 2 = i
i

- For p → ∞ we have ∞ -norm or max norm x ∞

= max ai
i

1-2-2 Norms of Real Functions

The space of continuous functions on the interval [0,1] clearly forms a vector
space. One possible norm defined on this space is the 1-norm defined as:
f (t ) ∞
= sup f (t ) 1-2
t∈[ 0 ,1]

This measures the peak value of the function in the interval [0,1]. Another norm is
the 2-norm defined as:

2
Chapter 1 Lecture Notes of Multivariable Control

= ⎛⎜ ∫ f (t ) dt ⎞⎟
1
f (t )
2 2
1-3
2
⎝ 0 ⎠

1-2-3 Norms of Matrices

We can extend norm of vectors to matrices. Consider A as an n × n matrix.
- Sum matrix norm(extension of 1-norm of vectors) is:
A sum
= ∑ a ij 1-4
i, j

- Frobenius norm(extension of 2-norm of vectors) is:

∑a
2
A F
= ij 1-5
i, j

- Max element norm(extension of max norm of vectors) is:

A max
= max aij 1-6
i, j

1-2-4 Matrix Norms

A norm of a matrix is called matrix norm if it satisfy
AB ≤ A . B 1-7
An m × n complex matrix may be viewed as an operator on the (finite dimensional)
normed vector space C n that maps a vector of the vector space C n to the vector
space C m . Define the induced-norm of A as follows:
A ip = max Ax p 1-8
x p
=1

where . p
is any vector norm. From this definition, it follows that the induced

norm measures the amount of the amplification of the matrix A provides to vectors
n
on the unit sphere in C , i.e. it measures the gain of the matrix.
If we put p=1 so we have
A i1 = max Ax 1 = max ∑ aij Maximum column sum 1-9
x 1 =1 j
i

For p → ∞ we have

3
Chapter 1 Lecture Notes of Multivariable Control

A i∞ = max Ax ∞
= max ∑ aij Maximum row sum 1-10
x ∞
=1 i
j

1-3 Unitary Matrices

A matrix U ∈ C n×n is unitary if U H U = I which U H denotes the complex

conjugate of the transpose of matrix U, which is also called the Hermitian
transpose or conjugate transpose.
A matrix U ∈ R n×n is orthogonal if U T U = I , where U T denotes the transpose.
Property:
If U is unitary, then Ux 2 = x 2 .

1-4 Hermitian Matrices and Positive (Negative) Definite Matrices

A complex matrix Q is termed Hermitian if Q H = Q . If Q is real, then this

condition simply states that Q is symmetric.
A matrix is termed positive (negative) definite if x H Qx is real and positive
(negative) for any x ≠ 0 .
Property of Hermitian matrix:
• If S is Hermitian ( S = S H ), then there exists a unitary matrix such that
U H SU = Λ where Λ is diagonal matrix.
• For any matrix A , the both A H A and AA H are Hermitian, and thus can
always be diagonalized by unitary matrices.
• For any matrix A , the eigenvalues of A H A and AA H are always real and
non-negative (proved easily by use of A H Ax = λx and multiplies it by x H
from left).

4
Chapter 1 Lecture Notes of Multivariable Control

1-5 Inner Product

The most useful vector spaces in practice are those on which one can define a
notion of inner product. An inner product is a function of two vectors, usually
denoted by < x, y > where x and y are vectors, with the following properties:
Symmetry: < x, y >=< y, x > ∗ . (superscript ∗
means complex conjugate)
Linearity: < ax + by, z >= a < x, z > +b < y, z > for all scalars a and b.
Positivity: < x, x > is positive for all x ≠ 0.
Two vectors x, y are said to be orthogonal if < x, y > =0. Two sets of vectors X and
Y are called orthogonal if every vector in one is orthogonal to every vector in the
other. The orthogonal complement of a set of vectors X is the set of vectors
orthogonal to X, and is denoted by X┴.

1-6 Singular Value Decomposition (SVD)

The singular value decomposition or SVD of a matrix is presented in this section.

The SVD exposes the 2-norm of a matrix, but its value to us goes much further: it
enables the solution of a class of matrix perturbation problems that form the basis
for the stability robustness concepts introduced later, it solves the so-called total
least squares problem, which is a generalization of the least squares estimation
problem considered earlier, and it allows us to clarify the notion of conditioning,
in the context of matrix inversion. These applications of the SVD are presented at
greater length in the following.
Example 1-1
To provide some immediate motivation for the study and application of SVD, we
begin with an example that clearly brings out the issue of matrix conditioning with

5
Chapter 1 Lecture Notes of Multivariable Control

respect to inversion. An interesting question is how the inverse of a matrix is

sensitive to perturbations.
Consider the matrix
⎡100 100⎤
A=⎢
⎣100.2 100⎥⎦

A quick calculation shows that

⎡− 5 5⎤
A −1 = ⎢
⎣5.01 − 5⎥⎦

Now suppose the perturbed matrix as

⎡100 100⎤
A + ∆A = ⎢
⎣100.1 100 ⎥⎦

Inverse of the perturbed system is

⎡− 10 10 ⎤
( A + ∆A) −1 = ⎢ ≅ A −1 + ∆( A −1 )
⎣10.01 − 10⎥⎦

Here ∆ ( A) denotes the perturbation in A and ∆ ( A −1 ) denotes the resulting per-

turbation in A −1 . Evidently a 0.1% change in one entry of A has resulted in a 100%
⎡1⎤
change in all entries of A −1 . If we want to solve the problem Ax=b where b = ⎢ ⎥,
⎣− 1⎦
⎡ − 10 ⎤
then x = A −1b = ⎢ ⎥, while after perturbation of A we get
⎣10.01⎦
⎡ − 20 ⎤
x + ∆x = ( A + ∆A) −1 b = ⎢ ⎥ . Again, we see a 100% change in the entries of the
⎣20.01⎦
solution with only a 0.1% change in the data.
What is seen in the above example, is a purely matrix phenomenon. It would seem
to be related to the fact that A is nearly singular in the sense that its columns are
nearly dependent; its determinant is much smaller than its largest element. In what
follows, we shall develop a way to measure nearness to singularity, and show how
this measure relates to sensitivity under inversion.

6
Chapter 1 Lecture Notes of Multivariable Control

Theorem 1-1
Let M ∈ C l×m . Then there exist Σ ∈ R l×m and unitary matrices Y ∈ C l×l and U ∈ C m×m
such that
M = YΣU H 1-11
⎡S 0 ⎤
where Σ=⎢ ⎥, S = diag{σ 1 , σ 2 ,......, σ r } with σ 1 ≥ σ 2 ≥ ........ ≥ σ r and
⎣0 0 ⎦
r ≤ min{l , m} , Y = [ y1 , y 2 ,......, y l ], U = [u1 , u 2 ,......, u m ] and U H is complex conjugate

transpose of U. The column vectors of Y, identified by y i , are orthogonal and of

unit length, and represent the output directions. The set {y1 , y 2 ,...., y r }
corresponding to nonzero singular values is the basis of the range space of M. The
column vectors of U, identified by u i , are orthogonal and of unit length, and
represent the input directions. The set {u1 , u 2 ,...., u r } corresponding to nonzero
singular values is the basis of the domain space of M. These input and output
directions are related through singular values. So one can write:
Mu i = σ i y i 1-12
It means that if an input is applied in the direction u i , then the output is in the
direction y i and has a gain of σ i . Input direction u i for i>r corresponds to inputs
that do not have any influence on the outputs and similarly y i for i>r corresponds
to the outputs that can not be accessed by any input. Since the diagonal elements
of S are arranged in a descending order, it can be shown that the largest gain for
any input direction is equal to the maximum singular value σ 1 and so one can
write:
Md Mu1
σ 1 = max 2
= 2
1-13
d ≠0 d u1
2 2

where d is any input signal and . 2 is the Euclidian norm.

Note that u i 2
= yi 2
= 1 since Y and U are unitary. Expansion of (1-11) leads to

the following equation.

7
Chapter 1 Lecture Notes of Multivariable Control

r
M = ∑σ k yk uk
H
1-14
k =1

Proof: See the Multivariable course of MIT.

Example 1-2
Consider
⎡1 2 − 1⎤
M = ⎢⎢3 4 1 ⎥⎥
⎢⎣4 2 8 ⎥⎦

then singular value decomposition of M is:

⎡0.04 − 0.53 − 0.85 ⎤ ⎡9.77 0 0 ⎤ ⎡0.50 − 0.33 − 0.80 ⎤

H

M = ⎢⎢0.38 − 0.77 0.51 ⎥⎥ . ⎢⎢ 0 4.53 0 ⎥⎥ . ⎢⎢0.35 − 0.77 0.53 ⎥⎥

⎢⎣0.92 0.34 − 0.17 ⎥⎦ ⎢⎣ 0 0 0⎥⎦ ⎢⎣0.79 0.55 0.27 ⎥⎦
The largest gain of 9.77 is for input in the direction u1 = [0.50 0.35 0.79]T and

corresponding output is y1 = [0.04 0.38 0.92]T . The gain 4.53 is for input in the

direction u 2 = [− 0.33 − 0.77 0.55]T and the corresponding output direction is

y 2 = [− 0.53 − 0.77 0.34] . The input direction u 3 = [− 0.80 0.53 0.27] has no
T T

influence on outputs and the output direction y 3 = [− 0.85 0.51 − 0.17 ] is not
T

accessible with any input. In this example, input in the direction v1 will excite the
largest gain, or, in other words, will be the most effective input.

Theorem 1-2

The induced 2-norm of a matrix A is:

Ax
A 2 ≅ sup 2
= σ 1 = σ max ( A) 1-15
x≠0 x 2

Proof: It is clear by 1-12 and 1-13.

8
Chapter 1 Lecture Notes of Multivariable Control

1-7 Relative Gain Array (RGA)

The relative gain array (RGA), was introduced by Bristol (1966). The RGA of a
complex non-singular m × m matrix A, denoted by RGA(A) or Λ ( A) , is a complex
m × m matrix defined by
RGA( A) = Λ( A) = A × ( A −1 ) T 1-16
Where the operator × denotes element by element multiplication or Schur product.
If A is real then Λ ( A) is also real.
The RGA may be generalized to a non-square matrix A by use of the pseudo-
inverse A † as
RGA( A) = Λ ( A) = A × ( A † ) T 1-17

1-8 Matrix Perturbation

The following question arises frequently in the matrix theory: What is the smallest
possible perturbation of a matrix that causes it to lose rank? Following we discuss
two cases, one with perturbations measured in the 2-norm, and then discuss the
perturbations in the one element of a matrix.

1-8-1 Additive Perturbation

Here we suppose that matrix A perturbed by a matrix ∆ as additive form. The
smallest ∆ that makes the perturbed system singular is measured.

Theorem 1-3
Suppose A ∈ C m×n has full column rank (n). Then

∆∈C m × n
{ }
min ∆ 2 | rank ( A + ∆) < n = σ n ( A) = σ ( A)
−
1-18

Proof: See “Multivariable feedback design”, S. Skogestad and I. Postlethwaite

9
Chapter 1 Lecture Notes of Multivariable Control

1-8-2 Multiplicative Perturbation

Here we suppose that matrix A perturbed by a matrix ∆ as multiplicative form.
The smallest ∆ that makes the perturbed system singular is measured.
Theorem 1-4(Small gain theorem)
Given A ∈ C m×n then

{ }
min ∆ 2 | rank ( I − A∆) < n =
∆∈C n× m
1
σ ( A)
1-19

Proof: See “Multivariable feedback design”, S. Skogestad and I. Postlethwaite

1-8-3 Element by Element Perturbation

Element by element perturbation assume independent perturbation in the
individual elements of a matrix. This kind of perturbation is not happen from
physical point of view. The RGA can be used to address this kind of uncertainty.
Following theorem shows that matrices with large RGA values are very sensitive
to element by element perturbation.
Theorem 1-5
Suppose A ∈ C n×n is non-singular and suppose λij is the ijth element of the RGA
1
of A. The matrix A will be singular if ijth element of A perturbed by aijp = aij (1 − )
λij

Proof: See “Multivariable feedback design”, S. Skogestad and I. Postlethwaite

Example 1-3
Consider the matrix
⎡100 100⎤
A=⎢
⎣100.2 100⎥⎦

Find the smallest perturbation on the a12 element of A that makes the matrix
singular.
Solution: The RGA of the matrix A is:
⎡− 500 501 ⎤
Λ ( A) = ⎢ ⎥
⎣ 501 − 500⎦

10
Chapter 1 Lecture Notes of Multivariable Control

1
Now according to theorem 1-5 if a12 multiplied by (1 − ) = 1.002 then the
λ12
perturbed A is singular or
⎡100 *1.002 100⎤ ⎡100.2 100⎤
AP = ⎢ =
⎣ 100.2 100⎥⎦ ⎢⎣100.2 100⎥⎦

Clearly AP is singular.

Exercises
1-1 Which of the following set are vector spaces?
n n
a) R and C .
b) Real continuous functions f(t) on the real line, with obvious definitions of
vector addition (add the functions point wise, f(t)+g(t)) and scalar multiplication
(scale the function by a constant, a f (t)).
c) The set of m × n matrices.
d) The set of solutions y(t) of the LTI ordinary differential equation
y ′(t ) + 3 y (t ) = 0 .

e) The set of points [ x1 x 2 x3 ] in R 3 satisfying x1 2 + x2 2 + x3 2 = 1 , i.e. “vectors from

the origin to the unit sphere.”
f) The set of solutions y(t) of the LTI ordinary differential equation
y ′(t ) + 3 y (t ) = sin(t ) .

Answer: a, b, c and d are vector spaces but e and f are not.

1-2 Show that any induced norm is a norm. (Check the 3 properties of the norm
definition)

1-3 Show that any induced norm is a matrix norm.

11
Chapter 1 Lecture Notes of Multivariable Control

1-4 The spectral radius of a matrix is:

ρ ( A) = max λi
i

where λi is the eigenvalue of A. Show that the spectral radius is not a norm.

1-5 Suppose A is Hermitian. Find the exact relation between the eigenvalues and
singular values of A. Does this hold if A is not Hermitian?

1-6 Verify that x H Qx is always real, if Q is Hermitian.

1-7 Verify that if Q is Hermitian then its eigenvalues are real.

1-8 Verify that < x, x > defines a norm.

1-9 Verify that x T Qy constitutes an inner product if Q is Hermitian and positive

definite.

1-10 Show that

1

∫ x(t ) y(t )dt

0

defines an inner product on the space of continuous functions. In this case, the
norm generated from this inner product is the same as the 2-norm defined earlier.

1-11 Show that Frobenius norm can be derived by tr ( A H A) .

1-12 Show that if rank(A)=1, then, A F

= A 2
.

12
Chapter 1 Lecture Notes of Multivariable Control

1-13 Suppose
⎡3 4 7⎤
⎢
A = ⎢1 2 3⎥⎥
⎢⎣− 2 7 5⎥⎦

by use of SVD:
a) Find the null space of A..
b) Find the range space of A.
c) If x 2 = 2.75 what is the maximum and minimum of Ax 2 .

1-14 Suppose the SVD of the m × n m matrix A is:

⎡Σ 0⎤ H
A =U⎢
0⎥⎦
V
⎣0
And Σ is an invertible r × r matrix. Then the “Moore-Penrose inverse”, or pseudo-
inverse of A, denoted by A † , can be defined as the n × m matrix

† ⎡ Σ −1 0 ⎤ H
A =V⎢ ⎥U
⎣0 0⎦

a) Show that A †A and A A † are symmetric, and that A A †A=A and A † A A †= A † .

(These four conditions actually constitute an alternative definition of the pseudo-
inverse.)
b) Show that when A has full column rank then A † = ( A' A) −1 A' , and that when A
has full row rank then A † = A' ( AA' ) −1 .
c) Show that, of all x that minimize y − Ax 2 (and there will be many, if A does not

have full column rank), the one with smallest length x 2

is given by x = A †y.

References
Skogestad Sigurd and Postlethwaite Ian. (2005) Multivariable Feedback Control:
England, John Wiley & Sons, Ltd.
Maciejowski J.M. (1989). Multivariable Feedback Design: Adison-Wesley.

13