LU Decomposition

1
Chapter 10

• LU Decomposition

2
L and U Matrices

• Lower Triangular Matrix
• Upper Triangular Matrix

3
LU Decomposition

• Another method for solving matrix equations
• Idea behind the LU Decomposition - start with
• We know (because we did it in Gauss Elimination)
  we can write

4
LU Decomposition

• Assume there exists L
• Such that
• This implies

5
The Steps of LU Decomposition
6
LU Decomposition

• LU Decomposition
• Based on Gauss elimination
• More efficient
• Decomposition Methods (not unique)
• Doolittle decomposition lii 1
• Crout decomposition uii 1
  (omitted)
• Cholesky decomposition (for symmetric
  matrices) uii lii

7
LU Decomposition

• Three Basic Steps
• (1) Factor (decompose) A into L and U
• (2) given b, determine d from Ld b
• (3) using Ux d and back-substitution,
  solve for x
• Advantage Once we have L and U, we can use
  many different bs without repeating the
  decomposition process

8
LU Decomposition

• LU decomposition / factorization
• A x L U x b
• Forward substitution
• L d b
• Back substitution
• U x d
• Forward substitutions are more efficient than
  elimination

9
Simple Truss
F45
4
5
F35
F14
F24
F25
1
3
H1
F23
F12
2
V1
V2
W
10
Exampe Forces in a Simple Truss
A depends on geometry only but b varies with
applied load
11
LU Factorization

• Gauss elimination Axb
• elimination steps need to be repeated for
  each different b
• LU factorization / decomposition
• A L U
• does not involve the RHS b !!!

12
Doolittle LU Decomposition

• Doolittle Algorithm
• Get 1s on diagonal of L (lii 1)
• Operate on rows and columns sequentially,
  narrowing down to single element
• Identical to LU decomposition based on Gauss
  elimination (see pp238-239), but different
  approach

13
Doolittle LU Decomposition
14
Doolittle LU Decomposition
15
Doolittle LU Decomposition
16
Algorithm of Doolittle Decomposition
17
Forward Substitution

• Once L is formed, we can use forward
  substitution instead of forward elimination for
  different bs

Very efficient for large matrices !
18
Back Substitution
Identical to Gauss elimination
19
Forward Substitution
Example
20
Back-Substitution
21
Forward and Back Substitutions

• Forward-substitution
• Back-substitution (identical to Gauss elimination)

Error in textbook (p. 165)
22
Forward and Back Substitutions
23
Example Forward and Back Substitutions
A1 0 2 3 -1 2 2 -3 0 1 1 4 6 2 2 4
b1 -1 2 1' L,U LU_factor(A) L
1.0000 0 0 0 -1.0000
1.0000 0 0 0 0.5000
1.0000 0 6.0000 1.0000 14.0000
1.0000 U 1 0 2 3 0
2 4 0 0 0 -1 4 0
0 0 -70 xLU_solve(L,U,b) x
-0.1857 0.2286 -0.1143 0.4714
(without pivoting)
MATLAB M-file LU_factor
Forward and back substitution
24
Pivoting in LU Decomposition

• Still need pivoting in LU decomposition
• Messes up order of L
• What to do?
• Need to pivot both L and a permutation matrix
  P
• Initialize P as identity matrix and pivot when
  A is pivoted ? Also pivot L

25
LU Decomposition with Pivoting

• Permutation matrix P
• - permutation of identity matrix I
• Permutation matrix performs bookkeeping
  associated with the row exchanges
• Permuted matrix P A
• LU factorization of the permuted matrix
• P A L U
• Solution
• L U x P b

26
Permutation Matrix

• Bookkeeping for row exchanges
• Example P1 interchanges row 1 and 3
• Multiple permutations P

27
LU Decomposition with Pivoting
Start with
No need to consider b in decomposition
28
Forward Elimination
Interchange rows 1 4
Gauss elimination of first column
Save fi1 in the first column of L
29
Forward Elimination
No interchange required
Gauss elimination of second column
Save fi2 in the second column of L
30
Forward Elimination
Interchange rows 3 4
Partial pivoting for L and P
31
Forward Elimination
Gauss elimination of third column
Save fi3 in third column of L
32
LU Decomposition with Pivoting
Gauss elimination with partial pivoting
33
LU Decomposition with Pivoting
Forward substitution
Back substitution
34
LU Decomposition with Pivoting
partial pivoting
35
LU Decomposition with Pivoting
A1 0 2 3 -1 2 2 -3 0 1 1 4 6 2 2 4
b1 -1 2 1' L,U,PLU_pivot(A) L
1.0000 0 0 0 -0.1667
1.0000 0 0 0.1667 -0.1429
1.0000 0 0 0.4286 0
1.0000 U 6.0000 2.0000 2.0000
4.0000 0 2.3333 2.3333 -2.3333
0 0 2.0000 2.0000 0
0 0 5.0000 T1 6 2
2 4 -1 2 2 -3 1 0
2 3 0 1 1 4 T2 6
2 2 4 -1 2 2 -3 1
0 2 3 0 1 1 4
T1 LU T2 PA
Verify T1 T2
36
LU Decomposition with Pivoting
x 0 0 -0.1143 0.4714 x
0 0.2286 -0.1143 0.4714 x
-0.1857 0.2286 -0.1143 0.4714 x
-0.1857 0.2286 -0.1143 0.4714
A1 0 2 3 -1 2 2 -3 0 1 1 4 6 2 2 4
b1 -1 2 1 L,U,PLU_pivot(A)
PbPb Pb 1 -1 1 2
xLU_Solve(L,U,Pb) d 1.0000 -0.8333
0 0 d 1.0000 -0.8333
0.7143 0 d 1.0000 -0.8333
0.7143 2.3571
LU Decomposition LUx Pb
Back Substitution
Forward Substitution
37
MATLABs Methods

• LU factorization L,U lu (A)
• -- returns L and U
• L, U, P lu (A)
• -- returns also the permutation matrix P
• Cholesky factorization R chol(A)
• -- A RR?
• Determinant det (A)

38
MATLAB function lu
A1 0 2 3 -1 2 2 -3 0 1 1 4 6 2 2 4
b1 -1 2 1 L,Ulu(A) L 0.1667
-0.1429 1.0000 0 -0.1667 1.0000
0 0 0 0.4286 0
1.0000 1.0000 0 0
0 U 6.0000 2.0000 2.0000 4.0000
0 2.3333 2.3333 -2.3333 0
0 2.0000 2.0000 0 0
0 5.0000 LU ans 1 0
2 3 -1 2 2 -3 0 1
1 4 6 2 2 4
dL\b d 1.0000 -0.8333 0.7143
2.3571 xU\d x -0.1857 0.2286
-0.1143 0.4714
Forward and Back Substitutions
LU Decomposition without Pivoting
39
Cholesky LU Factorization

• If A is symmetric and positive definite, it is
  convenient to use Cholesky decomposition.
• A LLT UTU
• Cholesky factorization can be used for either
  symmetric or non-symmetric matrices
• No pivoting or scaling needed if A is symmetric
  and positive definite (all eigenvalues are
  positive)
• If A is not positive definite, the procedure
  may encounter the square root of a negative number

40
Cholesky LU Factorization

• For a general non-symmetric matrix
• For symmetric and positive definite matrices

41
Cholesky LU Decomposition
42
Example Cholesky LU
43
Cholesky LU Factorization

• A UTU U? U
• Recurrence relations

44
Script file for Cholesky Decomposition
Symmetric Matrix L U? Compute only the upper
triangular elements
45
A9 -6 12 -3 -6 5 -9 2 12 -9 21 0 -3 2 0
6A 9 -6 12 -3 -6 5
-9 2 12 -9 21 0 -3 2
0 6 L,U Cholesky(A)L 3 0
0 0 -2 1 0 0 4 -1
2 0 -1 0 2 1 U 3
-2 4 -1 0 1 -1 0 0
0 2 2 0 0 0 1
Symmetric L U?
46
MATLAB Function chol
A9 -6 12 -3 -6 5 -9 2 12 -9 21 0 -3 2 0
6 b24 -19 51 11 U chol(A) U
3 -2 4 -1 0 1 -1 0
0 0 2 2 0 0 0 1
U'U ans 9 -6 12 -3 -6 5
-9 2 12 -9 21 0 -3 2
0 6 d U'\b d 8 -3 8
3 x U\d x 1 -2 1 3
Forward substitution
Back substitution