Math 308K (Badger, Winter 2010)

LU Factorization
Selected Notes January 13, 2010

Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix multiplication).

What is an elementary matrix? Our goal in these notes is to rewrite Gaussian elimination using matrix multiplication. The starting point for this project is a special kind of matrix: An elementary matrix E is a square matrix with 1s on the main diagonal and at most one non-zero entry o of the main diagonal. For example, 1 0 0 E21 = 3 1 0 0 0 1 is a 3 3 elementary matrix. We call this matrix E21 to signify that the non-zero entry in a non-diagonal position appears in the (2, 1) entry. Lets nd the inverse of E21 . The general procedure states that we should form the 1 matrix [E21 |I ] and apply elimination [I |E21 ] until the identity matrix is on the left: 1 0 0 1 0 0 1 0 0 1 0 0 R23R1 3 1 0 0 1 0 0 1 0 3 1 0 . 0 0 1 0 0 1 0 0 1 0 0 1 We are done after one step (there was only one non-zero entry in a non-diagonal entry, so we only needed to clear one entry to obtain the identity); the inverse of E21 is given by 1 0 0 1 E21 = 3 1 0 . 0 0 1 This pattern is easily shown to hold in general. Let Eij (i = j ) be an elementary matrix with non-zero (i, j ) entry. Then 1 inverse Eij is the elementary matrix formed by negating the (i, j ) of Eij . Why do we care about elementary matrices? We care because multiplication by an elementary matrix on the left is equivalent to an performing an elementary row operation. For example, if A is any 3 3 matrix and E21 is the matrix from above, then multiplying A by E21 on the left is the equivalent to adding 3 Row 1 of A to Row 2 of A: 1 0 0 Row 1 Row 1 E21 A = 3 1 0 Row 2 = 3 Row 1 + Row 2 . 0 0 1 Row 3 Row 3 This is the type of operation we do in Gaussian elimination!
1

Rewriting Gaussian Elimination in Terms of Multiplication As usual we will illustrate the general process with an example. Assign A be the matrix 5 1 2 A = 10 3 7 . 15 17 19 Forward elimination will convert A to an upper triangular matrix U . After each step of elimination, we will record the same step using multiplication by an elementary matrix. Step 1: We need to zero out the (2,1) entry. In elimination notation, 5 1 2 5 1 2 R22R1 10 3 7 0 5 3 . 15 17 19 15 17 19 In terms of multiplication, 5 1 2 5 1 2 1 0 0 2 1 0 10 3 7 = 0 5 3 . 15 17 19 0 0 1 15 17 19 E21 A Notice the (2,1) entry of E21 came Step 2: We need to zero out the 5 1 0 5 15 17 In terms of multiplication, 1 1 0 0 0 1 0 2 0 3 0 1 E31 from the multiple of row 1 added to row 2. (3,1) entry. In elimination notation, 5 1 2 2 R33R1 0 5 3 . 3 0 20 13 19

5 1 2 5 1 2 0 0 1 0 10 3 7 = 0 5 3 . 15 17 19 0 20 13 0 1 E21 A from the multiple of row 1 added to row 3. (3,2) entry. In elimination notation, 2 5 1 2 R34R2 0 5 3 . 3 0 0 1 13 0 5 1 2 5 1 2 0 10 3 7 = 0 5 3 . 1 15 17 19 0 0 1 A = U

Notice the (3,1) entry of E31 came Step 3: We need to zero out the 5 1 0 5 0 20

In terms of multiplication, 1 0 0 1 0 0 1 0 0 1 0 0 1 0 2 1 0 4 1 3 0 1 0 0 E32 E31 E21

Notice the (3,2) entry of E21 came from the multiple of row 2 added to row 3.

What have we done so far? We have used the process of Gaussian elimination to nd elementary matrices E21 , E31 , E32 so that E32 E31 E21 A = U is upper triangular. This is what we wanted to do (write elimination using multiplication), but lets probe further. What happens if we move the matrices E21 , E31 , E32 to the other side of the equation 1 1 1 E32 E31 E21 A = U ? To do this, we just multiply the equation by on the left E21 E31 E32 :
1 1 1 A = (E21 E31 E32 )U.

Fortunately, it is easy to compute the inverse of an elementary matrix (remember the 1 1 1 example on page 1just negate the (i, j ) entry of Eij ). Lets calculate E21 E31 E32 : 1 0 0 1 0 0 1 0 0 1 1 1 E31 E32 = 2 1 0 0 1 0 0 1 0 E21 0 0 1 3 0 1 0 4 1 1 0 0 1 0 0 1 0 0 = 2 1 0 0 1 0 = 2 1 0 = L. 0 0 1 3 4 1 3 4 1
1 1 1 The matrix E21 E31 E32 is just a 5 1 A = 10 3 15 17

lower triangular matrix 1 0 0 5 1 2 7 = 2 1 0 0 5 3 4 1 0 0 19

L! Thus we have shown that 2 3 = LU, 1

A = LU is a lower triangular matrix times an upper triangular matrix!!! LU Factorization Theorem. Suppose that A is n n matrix that can be reduced to an upper triangular matrix U by Gaussian elimination without using row exchanges. Then A = LU where L is lower triangular and U is upper triangular. Each entry on the diagonal of L is 1. For each i > j , the (i, j ) entry of L is the multiple m of row j that was subtracted from row i during the elimination process. The diagonal entries of U are the pivots from the elimination process. Example 1: Reading Off Information from an LU Factorizatoin Suppose we are given a matrix A already written as 1 0 0 3 2 0 4 A= 5 1 0 2 3 1 0 0 A = LU : 1 1 . 2

Question: What multiple of row 1 was subtracted from row 3 during elimination? Answer: By looking at the (3,1) entry of L, we know that 2 times row 2 was subtracted from row 3 during elimination. Question: What were the pivots? Answer: The pivots were 3, 4 and 2 (the diagonal entries of U ).

Example 2: Finding an LU Factorization Suppose we are asked to nd the the factors L and U for the matrix 6 0 2 24 1 8 . 12 1 3 To do this, we perform Gaussian elimination and then read o the information we need from the elimination steps. 6 0 2 6 0 2 6 0 2 6 0 2 R24R1 R3+2R1 R3R2 24 1 8 0 1 0 0 1 0 0 1 0 . 12 1 3 12 1 3 0 1 1 0 0 1 The matrix U is just the upper triangular matrix we end with doing elimination, 6 0 2 U = 0 1 0 . 0 0 1 To record L, we just start with a lower triangular matrix with 1s on the diagonal and then for each i > j put the multiple m of row j which was subtracted from row i in the (i, j ) entry of L: 1 0 0 L = 4 1 0 . 2 1 1 (Note: Since we added 2 Row 1 to Row 3, we subtracted 2 Row 1 from Row 3.)