Chapter 1

1.1

How to study Applied Mathematics

After reading Chapter 1 Notes,

What problems are you solving??
What conditions are being given??
Does the theory guarantee a solution exist??
When will it fail??
How to solve the problem??
How much time to solve this problem by the given method??
Is there any other method performs better/faster??
In order to answer these questions, you have to understand:
1. Theory
2. Pseudo code (Algorithm)
3. Program (Independent to this course)
The Key step is after reading a lot of things, do you know how to implement the algorithm?? (Pseudo code can help)
This kind of learning curve should be happened in all the following Chapters.

CHAPTER 1.

1.2

LU Factorization

Algorithm 1.2.1 (LU Factorization, with the main diagonal of L of all ones, and without pivoting).
l11 = 1, u11 = a11 , if a11 = 0, Output Factorization Impossible, STOP
for j = 2 : n
a1j
u1j =
l11
aj1
lj1 =
u11
end for
for i = 2 : n 1
lii = 1
P
uii = aii i1
k=1 lik uki
if uii = 0, then Output Factorization Impossible, STOP
for j = i + 1 : n
P
1
uij = [aij i1
k=1 lik ukj ]
lii
P
1
lji =
[aji i1
k=1 ljk uki ]
uii
end for
end for
lnn = 1
P
unn = ann n1
k=1 lnk ukn
STOP
Definition 1.2.2. A permutation matrix P is a row-permuted identity matrix.
Proposition 1.2.3. Let P be a permutation matrix, then
1. P X is the same as X with its rows permuted. XP is the same as X with its column
permuted.
2. P 1 = P T .
3. det(P ) = 1.
4. P1 P2 is a permutation matrix.
Remark 1.2.4 (Elimination = Factorization). The direct method for solving Ax = b using
Gaussian elimination is (sometimes implicitly) carried out by the factorization procedure
(Algorithm 1.2.5).
Algorithm 1.2.5. Suppose Ax = b with A nonsingular.
1. Factorize A = P LU where
P = permutation matrix,
L = unit lower triangular matrix,
U = nonsingular upper triangular matrix.

1.3. SYMMETRIC POSITIVE DEFINITE MATRICES

2. Solve x = U 1 [L1 (P 1 b)] step by step by forward/backward substitution.

Definition 1.2.6. A(1 : j, 1 : j) is the leading j-by-j principal submatrix of A.
Theorem 1.2.7. The following two statements are equivalent:
1. There exists a unique unit lower triangular L and nonsingular upper triangular U
s.t. A = LU .
2. All leading principal submatrices of A are nonsingular.
Remark 1.2.8. The LU factorization without pivoting can fail on particular examples
such as the permutation matrix. So inevitably we need to introduce P into the factorization.
Algorithm 1.2.9 (Gaussian elimination with pivoting). Permute A by P1 s.t. a 6= 0 is
the largest entry in absolute value in its column. More generally, at step i of Gaussian
elimination, where we are computing the i-th column of L, we reorder rows i to n s.t. the
largest entry in the column is on the diagonal. Consequently, all entries of L are bounded
by 1 in absolute value.
Remark 1.2.10. Besides Remark 1.2.8, another important reason for pivoting is concerned with numerical stability.
4

10
1
Example 1.2.11. Consider LU factorization of A =
.
1
1

1.3

Symmetric Positive Definite Matrices

Definition 1.3.1. Some matrix A is symmetric positive definite (spd) if AT = A and

xT Ax > 0 for any x 6= 0.
Proposition 1.3.2. If X is nonsingular, then A is spd iff X T AX is spd.
Proposition 1.3.3. If A is spd and H = A(j : k, j : k) for some j k, then H is also
spd.
Proposition 1.3.4. A is spd iff AT = A and all its eigenvalues are positive.
Proposition 1.3.5. If A is spd, then all Aii > 0 and maxi,j |Aij | = maxi Aii .
Proposition 1.3.6. A is spd iff A = LLT where L is a unique lower triangular nonsingular matrix with positive diagonal entries. A = LLT is known as Cholesky factorization.
Algorithm 1.3.7 (Cholesky factorization).
for j = 1 : n
P
2 21
ljj = (ajj j1
k=1 ljk )
for i = j + 1 : n
P
lij = (aij j1
k=1 lik ljk )/ljj
end for
end for