Chapter 3 Matrices

Basic operations:
Given ⎡1 2 0⎤ A = [1 2 0;
⎡ a11 " a1n ⎤
a12 A = ⎢⎢0 − 1 2⎥⎥ 0 - 1 2;
⎢a a22 " a2 n ⎥⎥ ⎢⎣1 0 1 ⎥⎦
A = ⎢ 21 ∈ R n×n det( A) = 3
1 0 1]
⎢ # # # ⎥
⎢ ⎥ ⎡1 0 1 ⎤
det(A)
⎣an1 an 2 " ann ⎦
A = ⎢⎢2 − 1 0⎥⎥
T
A'
• Determinant det( A) (or A ) ⎢⎣0 2 1⎥⎦
T
• Transpose A n tr ( A) = 1 trace( A)
• Trace tr ( A) = ∑i =1 aii ⎡− 1 − 2 4 ⎤
• Inverse A−1 inv( A)
A−1 = ⎢⎢ 2 1 − 2⎥⎥
1
• Matrix functions f ( A) 3
⎢⎣ 1 2 − 1 ⎥⎦
rank ( A)
• Rank rank ( A) = 3
steven.ding@uni-due.de (V-SS-2009) 21

Computation of determinant
Given
⎡ a11 a12 " a1n ⎤
⎢a a22 " a2 n ⎥⎥
A = ⎢ 21 ∈ R n×n
⎢ # # # ⎥
⎢ ⎥
⎣ an1 an 2 " ann ⎦
then
n
det( A) = ∑ aij γ ij , for any i = 1, 2," , n
j
where

γ ij denotes the cofactor corresponding to aij and is equal to

γ ij = (−1) j +i det( M ij )
det( M ij ) is called the ijth minor of the matrix, and
M ij is obtained by delecting the ith row and jth column of A

steven.ding@uni-due.de (V-SS-2009) 22

1
Properties of the determinant:
(1) det( AB ) = det( A) det( B), if A, B are square matrices of same dimensions
(2) det( A−1 ) = (det( A)) −1 (Proof : det (I) = det (AA-1 ) = det (A) det (A-1 ) = I))
(3) det( AT ) = det( A)
⎛ ⎡a11 a12 " a1n ⎤ ⎞
⎜⎢ ⎥⎟
⎜ 0 a22 ⎥ ⎟ = a a "a
(4) det ⎜ ⎢
⎢ % % # ⎥ ⎟⎟
11 22 nn
⎜⎢ # ⎥
⎜ 0 " 0 ann ⎦ ⎟⎠
⎝⎣
⎛ ⎡A B⎤⎞
(5) det⎜⎜ ⎢ ⎥ ⎟⎟ = det( A) det( D)
⎝ ⎣O D ⎦ ⎠
(6) If A is singular, then det( A) = 0.
If A is invertible, then det( A) ≠ 0.
steven.ding@uni-due.de (V-SS-2009) 23

Properties of the determinant (cont.):

⎛⎡A A12 ⎤ ⎞
(6) det⎜⎜ ⎢ 11 ⎥ ⎟ = det( A11 ) det( A22 − A21 A11−1 A12 ), if A11 invertible
⎝ ⎣ A21 A22 ⎦ ⎟⎠
= det( A22 ) det( A11 − A12 A22−1 A21 ), if A22 invertible
Proof: If A11 invertible, then
⎡ A11 A12 ⎤ ⎡ I O ⎤ ⎡ A11 O ⎤ ⎡ I A11−1 A12 ⎤
⎢A =⎢ ⎢ ⎥
⎣ 21 A22 ⎦ ⎣ A21 A11 I ⎥⎦ ⎢⎣ O
⎥ −1
A22 − A21 A11−1 A12 ⎥⎦ ⎣O I ⎦
⎛⎡ I O⎤ ⎞ ⎛ ⎡ I A11−1 A12 ⎤ ⎞
As det ⎜⎜ ⎢ ⎟ = 1, det ⎜ ⎢ ⎥ ⎟⎟ = 1, there is
−1
⎝ ⎣ A21 A11 I ⎥⎦ ⎟⎠ ⎜ O
⎝⎣ I ⎦⎠
⎛⎡A A12 ⎤ ⎞ ⎛⎡A O ⎤⎞
det ⎜⎜ ⎢ 11 ⎥ ⎟ = det ⎜ ⎢ 11 ⎟
⎝ ⎣ A21 A22 ⎦ ⎟⎠ ⎜ O
⎝⎣ A22 − A21 A11 A12 ⎥⎦ ⎟⎠
−1

steven.ding@uni-due.de (V-SS-2009) 24

2
Properties of the determinant (cont.):

Example 8: True or false, with reason:

⎛⎡A A12 ⎤ ⎞
det ⎜⎜ ⎢ 11 ⎟ = det( A11 A22 − A11 A21 A11−1 A12 ),
⎝ ⎣ A21 A22 ⎥⎦ ⎟⎠
⎛⎡A A12 ⎤ ⎞
det ⎜⎜ ⎢ 11 ⎟ = det( A11 A22 − A21 A12 )
⎝ ⎣ A21 A22 ⎥⎦ ⎟⎠

steven.ding@uni-due.de (V-SS-2009) 25

Properties of the determinant (cont.):

⎛⎡I B⎤ ⎞
(7) det ⎜⎜ ⎢ ⎟ = det( I − CB ) = det( I − BC )
⎝ ⎣C I ⎥⎦ ⎟⎠
(8) Given b ∈ R1×n , c ∈ R n×1. Then det( I − cb) = det( I − bc) = 1 − bc
(9) Influence of elementary operations on the determinant:
• Interchange of rows (*(-1))
• multiplication of a row by a scalar (*c)
• addition of a scalar multiple of one row to another row (*1)

steven.ding@uni-due.de (V-SS-2009) 26

3
Definition of inverse:
For the n by n matrix A, if there exists a matrix B, such that BA = AB = I ,
Then B is called the inverse of matrix A, i.e. A is invertible/regular/
nonsingular.
For the m by n matrix A, if there exists a matrix B, such that BA=I, then
B is called the left inverse of A, i.e. A is left invertible. If there exists a
matrix C, such that AC =I, then C is called the right inverse of A, A is
right invertible.
Note 1: A is left invertible, if and only if A is of full column rank.
Proof: BA = I n ⇒ rank ( BA) = rank ( I n ) = n . Because
rank ( BA) ≤ rank (A) and rank (A) ≤ n , we know rank (A) = n .
Note 2: A is right invertible, if and only if A is of full row rank.

steven.ding@uni-due.de (V-SS-2009) 27

Note 3: If A has both a left inverse and a right inverse, then A is

invertible and the inverse is unique.
Proof: Suppose that BA=I and AC=I. Then B=B(AC)=(BA)C=C.

Properties of the inverse

(1) ( A−1 ) −1 = A
(2) ( AT ) −1 = ( A−1 )T , if A is square and invertible
(3) ( AB) −1 = B −1 A−1 , if A and B are invertible
−1
⎡A O⎤ ⎡ A1−1 O ⎤
(4) ⎢ 1 = ⎢ −1 ⎥
A2 ⎥⎦
, if A1 , A2 are invertible
⎣O ⎣O A2 ⎦

steven.ding@uni-due.de (V-SS-2009) 28

4
 Properties of the inverse (cont.)
(5) If A11 is invertible, then
−1
⎡ A11 A12 ⎤ ⎡ A11−1 + A11−1 A12 ∆−1 A21 A11−1 − A11−1 A12 ∆−1 ⎤
⎢A =⎢ ⎥
⎣ 21 A22 ⎥⎦ ⎣ − ∆−1 A21 A11−1 ∆−1 ⎦
with ∆ = A22 − A21 A11−1 A12 .
If A22 is invertible, then
−1
⎡ A11 A12 ⎤ ⎡ ∆−1 − ∆−1 A12 A22−1 ⎤
⎢A = ⎢ −1 ⎥
⎣ 21 A22 ⎥⎦ ⎣− A22 A21∆
−1
A22−1 + A22−1 A21∆−1 A12 A22−1 ⎦
with ∆ = A11 − A12 A22−1 A21.

steven.ding@uni-due.de (V-SS-2009) 29

Proof: If A11 invertible, then

⎡ A11 A12 ⎤ ⎡ I O ⎤ ⎡ A11 O ⎤⎡ I A11−1 A12 ⎤
⎢A = ⎢ ⎥
⎣ 21 A22 ⎥⎦ ⎢⎣ A21 A11−1 I ⎥⎦ ⎢⎣ O A22 − A21 A11−1 A12 ⎥⎦ ⎣O I ⎦

−1 −1 −1 −1
⎡ A11 A12 ⎤ ⎡I A11−1 A12 ⎤ ⎡ A11 O ⎤ ⎡ I O⎤
⎢A ⎥ =⎢ ⎥ ⎢
⎣ 21 A22 ⎦ ⎣O I ⎦ ⎣O
−1 ⎥ ⎢
A22 − A21 A11 A12 ⎦ ⎣ A21 A11 I ⎥⎦
−1

⎡ A−1 O ⎤
⎡ I − A11−1 A12 ⎤ ⎢ 11 ⎥⎡ I O⎤
=⎢ − −
⎥ ⎢ O ( A22 − A21 A11 A12 ) ⎥ ⎢

⎣− A21 A11 I ⎥⎦
1 1
−1
⎣O I ⎦⎢   
⎣ = ∆−1 ⎦⎥
⎡ A + A11 A12 ∆ A21 A11 − A11 A12 ∆ ⎤
−1 −1 −1 −1 −1 −1
= ⎢ 11 ⎥
⎣ − ∆−1 A21 A11−1 ∆−1 ⎦

steven.ding@uni-due.de (V-SS-2009) 30

5
 Properties of the inverse (cont.)
(6)
−1
⎡ A11 O⎤ ⎡ A11−1 O⎤
⎢A = ⎢ −1 ⎥.
⎣ 21 A22 ⎥⎦ −1
⎣ − A22 A21 A11 A22−1 ⎦

−1
⎡ A11 A12 ⎤ ⎡ A11−1 − A11−1 A12 A22−1 ⎤
⎢O = ⎢ ⎥.
⎣ A22 ⎥⎦ ⎣O A22−1 ⎦

steven.ding@uni-due.de (V-SS-2009) 31

Properties of the inverse (cont.)

(7) ( A + BCD ) −1 = A−1 − A−1 B(C −1 + DA−1 B) −1 DA−1

Proof: Let E = A + BCD

E inversible ⇒ I = E −1 E = E −1 A + E −1 BCD
Multiply on both sides A−1 B ⇒
A−1 B = E −1 B + E −1 BCDA−1 B = E −1 BC (C −1 + DA−1 B )
A−1 B(C −1 + DA−1 B) −1 = E −1 BC

A−1 B(C −1 + DA−1 B) −1 DA−1 = E −1 BCDA−1

= E −1 ( E − A) A−1 = A−1 − E −1 = A−1 − ( A + BCD ) −1
Pseudo-inverse?
steven.ding@uni-due.de (V-SS-2009) 32

6
Properties of the trace
tr ( A) = ∑i =1 aii , for A ∈ R n×n
n

(1) tr (α A) = α trA
(2) tr ( A + B) = trA + trB
(3) tr ( AB ) = tr ( BA) (AB,BA are square, but A,B may not be square)
(4) xT Ax = tr ( xT Ax) = tr ( AxxT ) = tr ( xxT A), for x ∈ R n , A ∈ R n×n

steven.ding@uni-due.de (V-SS-2009) 33

Rank of matrix
Given a matrix A of dimension m by n. Then rank ( A) = r
(1) A has at most r independent rows (i.e. row rank = r )
(2) A has at most r independent columns (i.e. column rank = r )
(3) ∃ submatrix Asub,r of r by r , s.t. Asub,r ≠ 0,
while ∀submatrix Asub,r +1 of r + 1 by r + 1, s.t. Asub,r +1 = 0.
(4) Define the left null space N A = {v vA = 0} . Then rankN A = m − r.
⎡σ 1 ⎤
⎢ % O ⎥⎥
(5) ∃ orthogonal matrices U ,V , s.t. A = U ΣV , Σ = ⎢
T
⎢ σr ⎥
⎢ ⎥
⎣ O O⎦
steven.ding@uni-due.de (V-SS-2009) 34

7
Properties of rank
(1) rank ( AT ) = rank ( A)
(2) rank ( AT A) = rank ( AAT ) = rank ( A)
(3) Given matrix A(m × n) and nonsingular matrices P(m × m), Q(n × n).
Then rank ( A) = rank ( PA) = rank ( AQ) = rank ( PAQ)
(4) Given matrix A(m × n), rank ( A) = r. Then ∃ matrices P(m × r ), Q(r × n)
and a nonsingular matrix B(r × r ), s.t. A = PBQ.
Given matrix A(m × n), rank ( A) = 1. Then ∃ vectors p(m ×1), q( 1× n),
s.t. A = pq.

Given matrix A(m × n), rank ( A) = r. Then ∃ matrices P(m × r ), Q(r × n),
s.t. A = PQ, rank ( P) = rank (Q) = r.

steven.ding@uni-due.de (V-SS-2009) 35

Several inequalities of matrix rank

(1) ∀ A(m × n),
rank ( A) ≤ min{m, n}
(2) ∀ A(m × k ), B(k × n),
rank ( AB) ≤ min{rank ( A), rank ( B )}
Proof : If B has linearly dependent columns , i.e., ∃x ≠ 0, s.t. Bx = 0,
then ABx = A(Bx) = 0, AB has linearly dependent columns.
Thus, rankAB ≤ rankB. Similarly, rankAB ≤ rankA.

steven.ding@uni-due.de (V-SS-2009) 36

8
 Several inequalities of matrix rank (cont.)

(3) ∀ A(m × k ), B(k × n), rank ( AB ) ≥ rank ( A) + rank ( B ) − k

Proof:
Do column elementary transformation on B, s.t.
BP = [b1 " brB Ok ×( n − rB ) ]. Then ABP = [ Ab1 " AbrB Ok ×( n − rB ) ]
Transform again columns, ABPQ = [c1 " crABP Ok ×( n − rABP ) ]
Note that rank(ABPQ) = rank(ABP) = r(AB). Note that b1 , " , brB
are linear independent, and rB − rABP = rB − rAB are the number of
linearly dependent combination of columns of matrix A
⇒ rB − rAB ≤ k − rA .
(4) rank(A + B) ≤ rank(A) + rank(B)

steven.ding@uni-due.de (V-SS-2009) 37

Example 20 How about the rank, the number of independent rows

and the number of independent columns in the following matrix?
Please factorize A as the product of full rank matrices.

⎡1 2 0⎤
⎡1 2 0 0 ⎤ ⎢2
⎢ ⎥ 0 1⎥⎥
(i ) A = ⎢2 0 4 0⎥, (ii ) A = ⎢
⎢0 4 0⎥
⎢⎣0 1 0 0⎥⎦ ⎢ ⎥
⎣0 0 0⎦

steven.ding@uni-due.de (V-SS-2009) 38