Numerical Computing

CSCI-UA.0421-001
New York University, Spring 2020
Instructor: Margaret Wright
Homework Assignment 3
Assigned February 28, 2020; due midnight, Tuesday, March 10, 2020

Pay attention to the choices of norm in each problem!

Exercise 3.1. Assume that U and V are nonsingular n × n upper-triangular matrices whose diagonal
elements are, respectively, uii and vii .
(a) Show that the matrix U V is upper triangular;
(b) Show that U −1 (the inverse of U ) is upper triangular;
(c) Show that the i-th diagonal element of U −1 is equal to 1/uii .
(d) Recalling that the matrix infinity norm is the maximum absolute row sum, show that
1
kU k∞ ≥ max |uii | and kU −1 k∞ ≥ . (3.1)
i min|uii |
i

This property provides a lower bound on the condition number of an upper-triangular matrix: mea-
sured in the infinity norm,
maxi |uii |
cond(U ) ≥ . (3.2)
∞ mini |uii |
In practice, this easily computable lower bound is frequently used as an estimate of the condition of an
upper-triangular matrix. It also shows that U must be ill-conditioned if its diagonals vary greatly in
magnitude.

Exercise 3.2.
(a) Assuming that the square matrices B and C are nonsingular, show that

condα (BC) ≤ condα (B) condα (C),

where α = 1, 2, and ∞, i.e., the condition number is measured in the “standard” matrix norms (one,
two, and infinity).
(b) Show that cond2 (AT ) = cond2 (A).
(c) Is cond1 (AT ) always equal to cond1 (A)? If your answer is “yes”, explain why. If your answer is “no”,
find a specific matrix A such that cond1 (AT ) 6= cond1 (A), and give A, A−1 , kAk1 and kAT k1 .
Exercise 3.3. Let ! !
0.671 −0.273 0.398
A= and b = .
−0.335 0.136 −0.199
(a) Show using exact arithmetic (e.g., try hand calculation!) that the exact solution xstar (denoted
by x∗ ) to Ax = b is x∗ = (1, 1)T .
(b) Let xtil (denoted by x
e) be the solution computed by Matlab with the command xtil = A\b.
Compute and print the following vectors and their two-norms:
e − x∗ ;
(i) d = x
(ii) r∗ = b − Ax∗ ; and
 2

(iii) re = b − Ae
x.
Comment on the relative sizes of these three norms. Are these results what you expected? Explain.
(c) Given any nonzero vector x
b and its associated exact residual rb = b − Ab
x, define E as the rank-one
matrix
1
E = T rbx bT . (3.3)
x
b x b
Show mathematically that the exact matrix E satisfies (A + E)b x = b.
(d) Consider the specific vector xhat (denoted by x
b)
!
−45
x
b= .
22

b is close to x∗ ? Or to x
Would you say that x e? Explain, including your interpretation of the meaning
of “close to”.
(e) Using the vector x
b from (d), compute and print
• rb = b − Ab
x,
• the matrix E from (3.3),
• kEk2 ,
• b − (A + E)x̂ and
• kb − (A + E)x̂k2 .
Comment on the sizes of the quantities listed in (e). How would you describe the “size” of kEk2 ?
(f) Use Matlab to compute xbar (denoted by x̄) with the command xbar = (A+E)\b. With exact
arithmetic, what equation would x̄ satisfy? Compute and print
• xbar,
• (A+E)*xbar,
• b-(A+E)*xbar, and
• kx̄ − x
bk2 .
Based on kx̄ − x
bk2 , do you feel that x̄ is “close to” x
b? Explain why or why not.
(g) Consider the statement: x
b is close to the exact solution of a system that is close to the original
system. Do you agree? Explain the significance of such a statement in terms of backward error
analysis.
(h) Please explain the results of this problem, using the SVD of A to analyze the relationships.

Exercise 3.4. Let P be an n × n permutation matrix that interchanges only two rows (or two columns)
of a nonsingular matrix.
(1) Show that P must be symmetric.
(2) Is the product of two such permutations necessarily symmetric? If your answer is “yes”, give a
short proof. If your answer is “no”, give a counterexample.

Exercise 3.5.
(a) Without using determinants, prove that a square upper-triangular matrix R is singular if and only
if at least one diagonal element is zero. (The same result is true for a lower-triangular matrix.)
(b) Assume that (i) R is an n × n triangular matrix, (ii) k (k < n) of its diagonal elements are zero, and
(iii) the remaining diagonal elements are nonzero. Must the rank of R be exactly n − k? Explain
your answer. If your answer is “yes”, give a proof. If your answer is “no”, give a counterexample.
 3

Exercise 3.6. Consider an n × n upper-triangular matrix U such that

u11 u22 · · · un−1,n−1 6= 0 but unn = 0.
(This implies that U is singular.)
(a) Assuming that U has the above form, give a general algorithm for computing a nonzero vector x
such that U x = 0.
(b) Given the matrix  
2 −1 1
 
U = 2 1 , (3.4)
0
define a one-parameter family of nonzero vectors x that satisfy U x = 0. In addition, give a nonzero
x that corresponds to a particular value of the parameter.
Exercise 3.7.
(a) If A has linearly independent columns and Ax = Ay for nonzero vectors x and y, show that x = y.
(This result implies that we can “cancel” a matrix with linearly independent columns appearing on
the left of both sides of an equation.)
(b) Give a specific numerical example where A has linearly independent rows and Ax = Ay, but x 6= y.
(The contrast between parts (a) and (b) emphasizes the different roles of rows and columns in
matrix multiplication.)
Exercise 3.8. [Matrix norms.] Consider the matrix
!
1 −2
A= .
1 −1
(a) Give the value of kAk1 and the value of kAk∞ .
(b) Find a specific 2-vector u such that kuk1 = 1 and kAuk1 = kAk1 . Explain how you found u.
(c) Find a specific 2-vector v such that kvk∞ = 1 and kAvk∞ = kAk∞ . Explain how you found v.
Exercise 3.9. Consider the n × n lower-triangular matrix Ln of the form
 
1
 −1 1 
 
 .. 
 . 
Ln =  −1 −1 .
 
 .. 
 . 1 
−1 −1 · · · −1 1
(a) What is the value of kLn k∞ ? (Note: the infinity-norm.)
(b) Find a vector x of unit infinity-norm such that kLn xk∞ = kLn k∞ .
(c) Consider the n-vector y whose ith component is ( 12 )n−i , i.e.,
 1 
 2n−1 
 1 
 
 n−2 
 2 
 .. 
y= .
 . 
 
 1 
 
 2 
1
What is the value of kyk∞ ?
 4

(d) Display the vector Ln y. What is kLn yk∞ ?

(e) Assume that A is nonsingular, x 6= 0, and Ax = b. For compatible vector and matrix norms, show
that
kAxk kxk
kAk ≥ and kA−1 k ≥ .
kxk kbk
(f) With (e) in mind, what do the results of (b) and (c) tell you about cond(Ln ) measured in the
infinity norm, i.e., kLn k∞ kLn−1 k∞ ?
(g) Let Un = LTn denote the transpose of Ln . If Gaussian elimination with partial pivoting is applied
to Un , what are the LU factors?
(h) In your opinion, is the lower bound from (3.2) a good estimate of cond∞ (Un ) for this Un ? Explain.

Exercise 3.10. Consider the symmetric matrix

!
1.01 × 10−2 1.0046
A= .
1.0046 99.9229

(a) Is A positive definite? Explain how you deduced your answer.

(b) Compute cond∞ (A). (The Matlab command is cond(A,inf).)
(c) Compute the Cholesky factor R from the Matlab command R = chol(A). Give R. What is
cond∞ (R)?
(d) What is the lower bound from (3.2) on cond∞ (R) (which is, after all, an upper-triangular matrix)?
Is this estimate “reasonable” in this case? Explain.
(e) Use the Matlab command [L,U] = lu(A) to obtain the LU factorization of A derived from Gaussian
elimination with partial pivoting. (Note that Matlab incorporates any row interchanges directly
into the L factor, so that the computed “L” is really P T L.) Give L and U .
(f) Did any interchanges happen when Matlab computed the LU factors? Explain how you can tell.
(g) Compute
• cond∞ (L),
• cond∞ (U ), and
• the lower bound on cond∞ (U ) from (3.2).
Explain how you can determine by looking at U that A is ill-conditioned.