2
Linear Algebra
MATLAB Linear Algebra Functions
The MATLAB matfun directory contains linear algebra functions. For a
complete list, brief descriptions, and links to reference pages, type:
help matfun
The following table lists the MATLAB linear algebra functions by category.
Function Summary
2-2
Category
Function
Description
Matrix analysis
norm
Matrix or vector norm
normest
Estimate the matrix
2-norm
rank
Matrix rank
det
Determinant
trace
Sum of diagonal
elements
null
Null space
orth
Orthogonalization
rref
Reduced row echelon
form
subspace
Angle between two
subspaces
MATLAB Linear Algebra Functions
Function Summary (Continued)
Category
Function
Description
Linear equations
\ and /
Linear equation
solution
inv
Matrix inverse
cond
Condition number for
inversion
condest
1-norm condition
number estimate
chol
Cholesky factorization
ichol
Incomplete Cholesky
factorization
linsolve
Solve a system of linear
equations
lu
LU factorization
ilu
Incomplete LU
factorization
qr
Orthogonal-triangular
decomposition
lsqnonneg
Nonnegative
least-squares
pinv
Pseudoinverse
lscov
Least squares with
known covariance
2-3
Linear Algebra
Function Summary (Continued)
Category
Function
Description
Eigenvalues and
singular values
eig
Eigenvalues and
eigenvectors
svd
Singular value
decomposition
eigs
A few eigenvalues
svds
A few singular values
poly
Characteristic
polynomial
polyeig
Polynomial eigenvalue
problem
condeig
Condition number for
eigenvalues
hess
Hessenberg form
qz
QZ factorization
schur
Schur decomposition
expm
Matrix exponential
logm
Matrix logarithm
sqrtm
Matrix square root
funm
Evaluate general
matrix function
Matrix functions
2-4
Matrices in the MATLAB Environment
Adding and Subtracting Matrices
Addition and subtraction of matrices is defined just as it is for arrays, element
by element. Adding A to B and then subtracting A from the result recovers B:
A = pascal(3);
B = magic(3);
X = A + B
X =
9
4
5
2
7
12
7
10
8
1
5
9
6
7
2
Y = X - A
Y =
8
3
4
Addition and subtraction require both matrices to have the same dimension, or
one of them be a scalar. If the dimensions are incompatible, an error results:
C = fix(10*rand(3,2))
X = A + C
Error using plus
Matrix dimensions must agree.
w = v + s
w =
9
Vector Products and Transpose
A row vector and a column vector of the same length can be multiplied in
either order. The result is either a scalar, the inner product, or a matrix,
the outer product :
u = [3; 1; 4];
v = [2 0 -1];
x = v*u
2-7
Linear Algebra
x =
2
X = u*v
X =
6
2
8
0
0
0
-3
-1
-4
For real matrices, the transpose operation interchanges aij and aji. MATLAB
uses the apostrophe operator (') to perform a complex conjugate transpose,
and uses the dot-apostrophe operator (.') to transpose without conjugation.
For matrices containing all real elements, the two operators return the same
result.
The example matrix A is symmetric, so A' is equal to A. But B is not symmetric:
B = magic(3);
X = B'
X =
8
1
6
3
5
7
4
9
2
Transposition turns a row vector into a column vector:
x = v'
x =
2
0
-1
If x and y are both real column vectors, the product x*y is not defined, but
the two products
x'*y
2-8
Matrices in the MATLAB Environment
and
y'*x
are the same scalar. This quantity is used so frequently, it has three different
names: inner product, scalar product, or dot product.
For a complex vector or matrix, z, the quantity z' not only transposes the
vector or matrix, but also converts each complex element to its complex
conjugate. That is, the sign of the imaginary part of each complex element
changes. So if
z = [1+2i 7-3i 3+4i; 6-2i 9i 4+7i]
z =
1.0000 + 2.0000i
7.0000 - 3.0000i
6.0000 - 2.0000i
0 + 9.0000i
3.0000 + 4.0000i
4.0000 + 7.0000i
then
z'
ans =
1.0000 - 2.0000i
7.0000 + 3.0000i
3.0000 - 4.0000i
6.0000 + 2.0000i
0 - 9.0000i
4.0000 - 7.0000i
The unconjugated complex transpose, where the complex part of each element
retains its sign, is denoted by z.':
z.'
ans =
1.0000 + 2.0000i
7.0000 - 3.0000i
3.0000 + 4.0000i
6.0000 - 2.0000i
0 + 9.0000i
4.0000 + 7.0000i
For complex vectors, the two scalar products x'*y and y'*x are complex
conjugates of each other, and the scalar product x'*x of a complex vector
with itself is real.
Multiplying Matrices
Multiplication of matrices is defined in a way that reflects composition of
the underlying linear transformations and allows compact representation of
2-9
Linear Algebra
systems of simultaneous linear equations. The matrix product C = AB is
defined when the column dimension of A is equal to the row dimension of B,
or when one of them is a scalar. If A is m-by-p and B is p-by-n, their product
C is m-by-n. The product can actually be defined using MATLAB for loops,
colon notation, and vector dot products:
A =
B =
m =
for
pascal(3);
magic(3);
3; n = 3;
i = 1:m
for j = 1:n
C(i,j) = A(i,:)*B(:,j);
end
end
MATLAB uses a single asterisk to denote matrix multiplication. The next two
examples illustrate the fact that matrix multiplication is not commutative;
AB is usually not equal to BA:
X = A*B
X =
15
26
41
15
38
70
15
26
39
28
34
28
47
60
43
Y = B*A
Y =
15
15
15
A matrix can be multiplied on the right by a column vector and on the left
by a row vector:
u = [3; 1; 4];
x = A*u
x =
8
2-10
Matrices in the MATLAB Environment
17
30
v = [2 0 -1];
y = v*B
y =
12
-7
10
Rectangular matrix multiplications must satisfy the dimension compatibility
conditions:
C = fix(10*rand(3,2));
X = A*C
X =
17
31
51
19
41
70
Y = C*A
Error using mtimes
Inner matrix dimensions must agree.
Anything can be multiplied by a scalar:
s = 7;
w = s*v
w =
14
-7
Identity Matrix
Generally accepted mathematical notation uses the capital letter I to denote
identity matrices, matrices of various sizes with ones on the main diagonal
and zeros elsewhere. These matrices have the property that AI = A and IA = A
whenever the dimensions are compatible. The original version of MATLAB
could not use I for this purpose because it did not distinguish between
2-11
Systems of Linear Equations
Systems of Linear Equations
In this section...
Computational Considerations on page 2-15
The mldivide Algorithm on page 2-17
General Solution on page 2-18
Square Systems on page 2-18
Overdetermined Systems on page 2-21
Using Multithreaded Computation with Systems of Linear Equations
on page 2-24
Iterative Methods for Solving Systems of Linear Equations on page 2-25
Computational Considerations
One of the most important problems in technical computing is the solution of
systems of simultaneous linear equations.
In matrix notation, the general problem takes the following form: Given two
matrices A and B, does there exist a unique matrix X so that AX = B or XA = B?
It is instructive to consider a 1-by-1 example. For example, does the equation
7x = 21
have a unique solution?
The answer, of course, is yes. The equation has the unique solution x = 3. The
solution is easily obtained by division:
x = 21/7 = 3.
The solution is not ordinarily obtained by computing the inverse of 7, that is
71 = 0.142857..., and then multiplying 71 by 21. This would be more work
and, if 71 is represented to a finite number of digits, less accurate. Similar
considerations apply to sets of linear equations with more than one unknown;
2-15
Linear Algebra
the MATLAB software solves such equations without computing the inverse
of the matrix.
Although it is not standard mathematical notation, MATLAB uses the
division terminology familiar in the scalar case to describe the solution of a
general system of simultaneous equations. The two division symbols, slash, /,
and backslash, \, correspond to the two MATLAB functions mldivide and
mrdivide. mldivide and mrdivide are used for the two situations where the
unknown matrix appears on the left or right of the coefficient matrix:
X = B/A
Denotes the solution to the matrix equation
XA = B.
X = A\B
Denotes the solution to the matrix equation
AX = B.
Think of dividing both sides of the equation AX = B or XA = B by A. The
coefficient matrix A is always in the denominator.
The dimension compatibility conditions for X = A\B require the two matrices
A and B to have the same number of rows. The solution X then has the
same number of columns as B and its row dimension is equal to the column
dimension of A. For X = B/A, the roles of rows and columns are interchanged.
In practice, linear equations of the form AX = B occur more frequently than
those of the form XA = B. Consequently, the backslash is used far more
frequently than the slash. The remainder of this section concentrates on the
backslash operator; the corresponding properties of the slash operator can
be inferred from the identity:
(B/A)' = (A'\B')
The coefficient matrix A need not be square. If A is m-by-n, there are three
cases:
2-16
Systems of Linear Equations
m=n
Square system. Seek an exact solution.
m>n
Overdetermined system. Find a least squares
solution.
m<n
Underdetermined system. Find a basic solution
with at most m nonzero components.
The mldivide Algorithm
The mldivide operator employs different algorithms to handle different kinds
of coefficient matrices. The various cases are diagnosed automatically by
examining the coefficient matrix.
Permutations of Triangular Matrices
mldivide checks for triangularity by testing for zero elements. If a matrix A
is triangular, MATLAB software uses a substitution to compute the solution
vector x. If A is a permutation of a triangular matrix, MATLAB software uses
a permuted substitution algorithm.
Square Matrices
If A is symmetric and has real, positive diagonal elements, MATLAB attempts
a Cholesky factorization. If the Cholesky factorization fails, MATLAB
performs a symmetric, indefinite factorization. If A is upper Hessenberg,
MATLAB uses Gaussian elimination to reduce the system to a triangular
matrix. If A is square but is neither permuted triangular, symmetric and
positive definite, or Hessenberg, MATLAB performs a general triangular
factorization using LU factorization with partial pivoting (see lu).
Rectangular Matrices
If A is rectangular, mldivide returns a least-squares solution. MATLAB
solves overdetermined systems with QR factorization (see qr). For an
underdetermined system, MATLAB returns the solution with the maximum
number of zero elements.
The mldivide function reference page contains a more detailed description
of the algorithm.
2-17
Linear Algebra
General Solution
The general solution to a system of linear equations AX = b describes all
possible solutions. You can find the general solution by:
1 Solving the corresponding homogeneous system AX = 0. Do this using the
null command, by typing null(A). This returns a basis for the solution
space to AX = 0. Any solution is a linear combination of basis vectors.
2 Finding a particular solution to the nonhomogeneous system AX = b.
You can then write any solution to AX = b as the sum of the particular
solution to AX = b, from step 2, plus a linear combination of the basis vectors
from step 1.
The rest of this section describes how to use MATLAB to find a particular
solution to AX = b, as in step 2.
Square Systems
The most common situation involves a square coefficient matrix A and a single
right-hand side column vector b.
Nonsingular Coefficient Matrix
If the matrix A is nonsingular, the solution, x = A\b, is then the same size as
b. For example:
A = pascal(3);
u = [3; 1; 4];
x = A\u
x =
10
-12
5
It can be confirmed that A*x is exactly equal to u.
If A and B are square and the same size, X = A\B is also that size:
B = magic(3);
2-18
Systems of Linear Equations
X = A\B
X =
19
-17
6
-3
4
0
-1
13
-6
It can be confirmed that A*X is exactly equal to B.
Both of these examples have exact, integer solutions. This is because the
coefficient matrix was chosen to be pascal(3), which has a determinant
equal to 1.
Singular Coefficient Matrix
A square matrix A is singular if it does not have linearly independent
columns. If A is singular, the solution to AX = B either does not exist, or is not
unique. The backslash operator, A\B, issues a warning if A is nearly singular
and raises an error condition if it detects exact singularity.
If A is singular and AX = b has a solution, you can find a particular solution
that is not unique, by typing
P = pinv(A)*b
P is a pseudoinverse of A. If AX = b does not have an exact solution, pinv(A)
returns a least-squares solution.
For example:
A = [ 1
-1
1
3
4
10
7
4
18 ]
is singular, as you can verify by typing
det(A)
ans =
0
2-19
Linear Algebra
Note For information about using pinv to solve systems with rectangular
coefficient matrices, see Pseudoinverses on page 2-28.
Exact Solutions. For b =[5;2;12], the equation AX = b has an exact
solution, given by
pinv(A)*b
ans =
0.3850
-0.1103
0.7066
Verify that pinv(A)*b is an exact solution by typing
A*pinv(A)*b
ans =
5.0000
2.0000
12.0000
Least-Squares Solutions. On the other hand, if b = [3;6;0], AX = b does
not have an exact solution. In this case, pinv(A)*b returns a least squares
solution. If you type
A*pinv(A)*b
ans =
-1.0000
4.0000
2.0000
you do not get back the original vector b.
You can determine whether AX = b has an exact solution by finding the
row reduced echelon form of the augmented matrix [A b]. To do so for this
example, enter
2-20
Systems of Linear Equations
rref([A b])
ans =
1.0000
0
0
0
1.0000
0
2.2857
1.5714
0
0
0
1.0000
Since the bottom row contains all zeros except for the last entry, the equation
does not have a solution. In this case, pinv(A) returns a least-squares
solution.
Overdetermined Systems
Overdetermined systems of simultaneous linear equations are often
encountered in various kinds of curve fitting to experimental data. Here is a
hypothetical example. A quantity y is measured at several different values
of time, t, to produce the following observations:
t
0.0
0.82
0.3
0.72
0.8
0.63
1.1
0.60
1.6
0.55
2.3
0.50
Enter the data into MATLAB with the statements
t = [0 .3 .8 1.1 1.6 2.3]';
y = [.82 .72 .63 .60 .55 .50]';
Try modeling the data with a decaying exponential function:
y(t) = c1 + c2et.
The preceding equation says that the vector y should be approximated by a
linear combination of two other vectors, one the constant vector containing all
ones and the other the vector with components et. The unknown coefficients,
2-21