0% found this document useful (0 votes)
46 views8 pages

The LU Factorization: Matrix Factorizations

This document discusses LU factorization, which decomposes a matrix A into the product of a lower triangular matrix L and an upper triangular matrix U. It covers LU factorization without pivoting, which may not always be possible, and with row pivoting, which is guaranteed to work for any nonsingular matrix. It provides examples of performing LU factorization by hand and discusses the effect of rounding errors. The key advantages of LU factorization with pivoting are that it works for any nonsingular matrix and can improve numerical stability by selecting a permutation that places large matrix elements in strategic positions.

Uploaded by

Mz K
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
46 views8 pages

The LU Factorization: Matrix Factorizations

This document discusses LU factorization, which decomposes a matrix A into the product of a lower triangular matrix L and an upper triangular matrix U. It covers LU factorization without pivoting, which may not always be possible, and with row pivoting, which is guaranteed to work for any nonsingular matrix. It provides examples of performing LU factorization by hand and discusses the effect of rounding errors. The key advantages of LU factorization with pivoting are that it works for any nonsingular matrix and can improve numerical stability by selecting a permutation that places large matrix elements in strategic positions.

Uploaded by

Mz K
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

EE103 (Spring 2004-05)

6. The LU factorization

• LU factorization without pivoting

• LU factorization with row pivoting

• effect of rounding error

• sparse LU factorization

6–1

Matrix factorizations

LU factorization without pivoting

A = LU

• L unit lower triangular, U upper triangular


• does not always exist (even if A is nonsingular)

LU factorization (with row pivoting)

A = P LU

• P permutation matrix, L unit lower triangular, U upper triangular


• exists for all nonsingular A

The LU factorization 6–2


Example

LU factorization (without pivoting) of


 
8 2 9
A= 4 9 4 
6 7 9

write as A = LU with L unit lower triangular, U upper triangular


    
8 2 9 1 0 0 u11 u12 u13
A =  4 9 4  =  l21 1 0   0 u22 u23 
6 7 9 l31 l32 1 0 0 u33

The LU factorization 6–3

• first row of U , first column of L:


    
8 2 9 1 0 0 8 2 9
 4 9 4  =  1/2 1 0   0 u22 u23 
6 7 9 3/4 l32 1 0 0 u33

• second row of U , second column of L:


· ¸ · ¸ · ¸· ¸
9 4 1/2 £ ¤ 1 0 u22 u23
− 2 9 =
7 9 3/4 l32 1 0 u33
· ¸ · ¸· ¸
8 −1/2 1 0 8 −1/2
=
11/2 9/4 11/16 1 0 u33

• third row of U : u33 = 9/4 + 11/32 = 83/32

conclusion:
    
8 2 9 1 0 0 8 2 9
A =  4 9 4  =  1/2 1 0   0 8 −1/2 
6 7 9 3/4 11/16 1 0 0 83/32

The LU factorization 6–4


Algorithm for LU factorization without pivoting

partition A, L, U as block matrices:


· ¸ · ¸ · ¸
a11 A12 1 0 u11 U12
A= , L= , U=
A21 A22 L21 L22 0 U22

• a11 ∈ R, u11 ∈ R (other dimensions follow from block matrix notation)


• L22 unit lower-triangular, U22 upper triangular

determine L and U from A = LU , i.e.,


· ¸ · ¸· ¸
a11 A12 1 0 u11 U12
=
A21 A22 L21 L22 0 U22
· ¸
u11 U12
=
u11L21 L21U12 + L22U22

The LU factorization 6–5

recursive algorithm:

• determine first row of U and first column of L

u11 = a11, U12 = A12, L21 = (1/a11)A21

• factor the (n − 1) × (n − 1)-matrix

L22U22 = A22 − L21U12 (= A22 − (1/a11)A21A12)

cost: (2/3)n3 flops (no proof)

The LU factorization 6–6


Not every nonsingular A can be factored as A = LU

 
  
1 0 0 1 0 0 u11 u12 u13
A= 0 0 2  =  l21 1 0   0 u22 u23 
0 1 −1 l31 l32 1 0 0 u33

• first row of U , first column of L:


    
1 0 0 1 0 0 1 0 0
 0 0 2  =  0 1 0   0 u22 u23 
0 1 −1 0 l32 1 0 0 u33

• second row of U , second column of L:


· ¸ · ¸· ¸
0 2 1 0 u22 u23
=
1 −1 l32 1 0 u33

u22 = 0, u23 = 2, l32 · 0 = 1 ?

The LU factorization 6–7

LU factorization (with row pivoting)

if A ∈ Rn×n is nonsingular, then it can be factored as

A = P LU

P is a permutation matrix, L is unit lower triangular, U is upper triangular

• not unique; in general there are several possible choices for P , L, U


• interpretation: applying P T re-orders or permutes the rows of A so that
the re-ordered matrix has a factorization P T A = LU
• also known as Gaussian elimination with partial pivoting (GEPP)
• cost: 2n3/3 flops
• basis of standard method for solving Ax = b

we’ll skip the details of calculating P , L, U

The LU factorization 6–8


example
     
0 5 5 0 0 1 1 0 0 6 8 8
 2 9 0  =  0 1 0   1/3 1 0   0 19/3 −8/3 
6 8 8 1 0 0 0 15/19 1 0 0 135/19

the factorization is not unique; we can factor the same matrix as


     
0 5 5 0 1 0 1 0 0 2 9 0
 2 9 0  =  1 0 0  0 1 0  0 5 5 
6 8 8 0 0 1 3 −19/5 1 0 0 27

The LU factorization 6–9

Effect of rounding error

example · ¸· ¸ · ¸
10−5 1 x1 1
=
1 1 x2 0

• exact solution: x1 = −1/(1 − 10−5), x2 = 1/(1 − 10−5)


• A is a well-conditioned matrix (κ(A) = 2.6)

let us solve using LU factorization, rounding intermediate results to 4


significant decimal digits

we will do this for the two possible permutation matrices:


· ¸ · ¸
1 0 0 1
P = or P =
0 1 1 0

The LU factorization 6–10


first choice of P : P = I (no pivoting)
· −5 ¸ · ¸ · −5 ¸
10 1 1 0 10 1
=
1 1 105 1 0 1 − 105

L, U rounded to 4 decimal significant digits


· ¸ · −5 ¸
1 0 10 1
L= , U=
105 1 0 −105

forward substitution
· ¸· ¸ · ¸
1 0 z1 1
5 = =⇒ z1 = 1, z2 = −105
10 1 z2 0

backward substitution
· −5 ¸· ¸ · ¸
10 1 x1 1
= =⇒ x1 = 0, x2 = 1
0 −105 x2 −105

error in x1 is 100%

The LU factorization 6–11

second choice of P : interchange rows


· ¸ · ¸· ¸
1 1 1 0 1 1
=
10−5 1 10−5 1 0 1 − 10−5

L, U rounded to 4 decimal significant digits


· ¸ · ¸
1 0 1 1
L= , U=
10−5 1 0 1

forward substitution
· ¸· ¸ · ¸
1 0 z1 0
= =⇒ z1 = 0, z2 = 1
10 −5
1 z2 1

backward substitution
· ¸· ¸ · ¸
1 1 x1 0
= =⇒ x1 = −1, x2 = 1
0 1 x2 1

error in x1, x2 is about 10−5

The LU factorization 6–12


conclusion:

• for some choices of P , small rounding errors in the algorithm cause very
large errors in the solution

• this is called numerical instability: for the first choice of P , the


algorithm is unstable; for the second choice of P , it is stable

• from numerical analysis: there is a simple rule for selecting a good


(stable) permutation (we’ll skip the details, since we skipped the details
of the factorization algorithm)

• in the example, the second permutation is better because it permutes


the largest element (in absolute value) of the first column of A to the
(1,1)-position

The LU factorization 6–13

Sparse linear equations

if A is sparse (most elements are zero), it is usually factored as

A = P1LU P2

P1 and P2 are permutation matrices

• interpretation: we permute the rows and columns of A so that


à = P1T AP2T has a factorization à = LU
• usually L and U are sparse: factorization and forward/backward
substitution can be carried more efficiently
• choice of P1 and P2 greatly affects the sparsity of L and U : many
heuristic methods for selecting good permutation matrices P1 and P2
• in practice: #flops ¿ (2/3)n3; exact value is a complicated function of
n, number of nonzero elements, sparsity pattern

The LU factorization 6–14


Summary

different levels of understanding how linear equation solvers work

1. highest level: x=A\b costs (2/3)n3; more efficient than x=inv(A)*b


2. intermediate level: factorization step A = P LU ((2/3)n3 flops)
followed by solve step (2n2 flops)
3. lowest level: details of factorization A = P LU

• for most applications level 1 is sufficient


• in some important applications (for example, multiple righthand sides)
level 2 is useful
• level 3 is important for experts who write numerical libraries

The LU factorization 6–15

You might also like