Partt 3

P
Chapter 10

LU Factorization
F t i ti

PowerPoints organized by Dr. Michael R. Gustafson II, Duke University

Revised by Prof. Jang, CAU
All images copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
 Chapter Objectives
• Understanding that LU factorization involves decomposing
the coefficient matrix into two triangular matrices that can
then be used to efficiently evaluate different right-hand-
side vectors
vectors.
• Knowing how to express Gauss elimination as an LU
factorization.
• Given
Gi an LU factorization,
f t i ti k
knowing
i how
h to
t evaluate
l t multiple
lti l
right-hand-side vectors.
• Recognizing that Cholesky’s method provides an efficient
way to
t decompose
d a symmetric
t i matrix
t i and
d that
th t the
th
resulting triangular matrix and its transpose can be used to
evaluate right-hand-side vectors efficiently.
• Understanding
U d di in
i generall terms what
h happens
h when
h
MATLAB’s backslash operator is used to solve linear systems.
 LU Factorization
• R
Recallll th
thatt the
th forward-elimination
f d li i ti step
t off
Gauss elimination comprises the bulk of the
computational effort.
effort
• LU factorization methods separate the time-
consuming elimination of the matrix [A]
from the manipulations of the right-hand-
side [b].
• Once [A] has been factored (or
decomposed), multiple right-hand-side
vectors can be b evaluated
l d in
i an efficient
ffi i
manner.
 LU Factorization
• LU factorization involves
two steps:
– Factorization to decompose
p
the [A] matrix into a
product of a lower
triangular matrix [L] and an
upper triangular matrix [U].
[L] has 1 for each entry on
the diagonal
diagonal.
– Substitution to solve for {x}
• Gauss elimination can be  1 
implemented using LU
[ L]   f 21 1 

factorization
 f 31 f 32 1
 Gauss Elimination as
LU Factorization (cont)
• T l [A]{x}={
To solve } {b},
} first
fi d decompose [A] to get
[L][U]{x}={b}
• Set
S t up and l [L]{d}={
d solve } {b},
} where
h {d} can be
b
found using forward substitution.
• Set
S t up and l [U]{x}={
d solve } {d},
} where
h {x} can be
b
found using backward substitution.
• In
I MATLAB
MATLAB:
[L, U] = lu(A)
d = L\b
x = U\d
 Gauss Elimination as
LU Factorization
i i (cont)
 a11 a12 a13   x1   b1 
a   x   b 
 21 a 22 a 23  2   2 
 a 31 a 32 a 33   x3  b3 
Th first
The fi t step
t in
i Gauss
G Eli i ti n is
Eliminatio i to
t multiply
lti l row 1 by
b the
th factor
f t
a
f 21  21
a11
and subtract t he result from the second row to eliminate a 21 . Similarly row 1 is multiplied by
a 31
f 31 
a11
and the result subtracted from the third row to eliminate a 31 .
The final step is to multiply t he modified second row by the factor
a
f 32  32

a 22
and
d the
th result
lt subtracted
bt t d from
f th third
the thi d row to
t eliminate
li i t a 32 .
 Gauss Elimination as
LU Factorization (cont)

 a11 a12 a13   1  a11 a12 a13 

a a 23    f 21  ' 
 21
 a31 Aa 22
a32 a33   f 31 L1
f 32

1  U '
a 22 a 23
"
a33



[M
[MATLAB 함수] f 21 
a21 a
f 31  31
'
a32
f 32  '
>> [L,U] = lu(X) a11 a11 a22

Check : f 21a12  a 22
'
 f 21a12  f 21a12  a 22  a 22
 Gauss Elimination as
LU Factorization
• [A]{x}={
} {b} can be
b rewritten
i as [L][U]{x}={
} {b}
using LU factorization.
Th LU factorization
• The f t i ti algorithm
l ith requiresi the
th
same total flops as for Gauss elimination.
• The
Th main i advantage
d t i once [A] is
is i
decomposed, the same [L] and [U] can be
used for multiple {b} vectors.
vectors
• MATLAB’s lu function can be used to generate
the [L] and [U] matrices:
[L, U] = lu(A)
 Example
p 10.1+10.2 ((1/4))
Q. To use LU decomposition 3x1  0.1x2  0.2 x3  7.85
to solve this problem. 0.1x1  7 x2  0.3 x3   19.3
0.3x1  0.2 x2  10 x3  71.4

 3  0.1  0.2  x1   7.85 

0.1    x    19.3
 7 0.3 2   
0.3  0.2 10   x3   71.4 

3  0.1  0.2 
 
Forward Elimination: [U ]  0 7.00333  0.293333
0 0 10.0120 

3  0.1  0.2   x1   7.85 

0 7.00333  0.293333  x    19.5617
  2   
0 0 10.0120   x3   70.0843 
 Example 10.1+10.2
10 1+10 2 (2/4)
0.1 0.3
f 21   0.0333333 f 31   0.1000000
3 3
0.19
f 32   0.0271300
7.00333

 1  3  0 .1  0 .2 
[ A]  [ L][U ]  0.0333333 1  0 7.00333  0.293333
 
 0.100000  0.0271300 1 0 0 10.0120 

 3  0.1  0.2 
  0.3   [ A]
Confirm: [ L][U ]  0.0999999 7
 0.3  0.2 9.99996
 Example 10.1+10.2
10 1+10 2 (3/4)
 1 0 0  d1   7.85 
Substitution: 0.0333333  d    19.3
 1 0  2   
 0.100000  0.0271300 1 d 3   71.4 

d1  7.85
-> 0.0333333d1  d2   19.3
0.100000d1  0.027300d 2  d3  71.4

d 2  19.3  0.0333333(7.85)  19.5617

d 3  71.4  0.1(7.85)  0.02713(19.5617)  70.0843

 Example 10.1+10.2
10 1+10 2 (4/4)
 7.85 
d    19.5617 
 70.0843 
  3  0 .1  0.2   x1   7.85 
0 7.00333  0.293333  x    19.5617 
-> To solve for x,   2   
0 0 10.0120   x3   70.0843 
 3 
x    2.5 
7.00003
 
 Cholesky Factorization
• S
Symmetric
t i systems
t occur commonly l in
i both
b th
mathematical and engineering/science
problem contexts, and there are special
solution techniques available for such systems.
• The Choleskyy factorization is one of the most
popular off these techniques, and is based on
the fact that a symmetric matrix can be
decomposed as [A]= [U]T[U], ] where T stands
for transpose.
• The rest of the process is similar to LU
decomposition and Gauss elimination, except
only one matrix, [U], needs to be stored.
 MATLAB
• MATLAB can perform f a Cholesky
Ch l k factorization
f i i
with the built-in chol command:
U = chol(A)
• MATLAB’s left division operator \ examines
the system to see which method will most
efficiently solve the problem. This includes
trying banded solvers,
solvers back and forward
substitutions, Cholesky factorization for
symmetric
y systems.
y If these do not work and
the system is square, Gauss elimination with
partial pivoting is used.