Matrix Theory

Lecture Notes in Applied Mathematics Dr. K.C.
Raajasekar
Advanced Matrix Theory
Vector Space A vector space over a Field F (real or complex) is a set V with two
operations called addition and scalar multiplication described as follows:
1. Given any two elements v1 and v2 of V there exists a unique element of V,

denoted v1+v2, and called the sum of v1 and v2.
2. Given any element v of V, and any element λ of F there exists a unique element
of V denoted λv, and called the scalar multiple of v by λ.
These two operations must satisfy the following axioms:

i. v1 + v2 = v2 + v1 for all v1, v2 in V.
ii. (v1 + v2) + v3 = v1 + (v2 + v3) for all v1, v2, v3 in V.
iii. There exists a unique element of V called the zero vector and denoted 0 with
the property that v + 0 = 0 + v = v for all element v in V.
iv. For each element v of V there exists an element denoted (-v) and called the
additive inverse of v with the property that v + (-v) = (-v) + v = 0 for all
element v in V.
v. λ1(λ2v) = (λ1λ2 )v for all λ1, λ2 in F, and for all v in V.
vi. (λ1+λ2) v) = λ1v + λ2 v for all λ1, λ2 in F, and for all v in V.
vii. λ(v1 + v2) = λv1 + λv2 for all λ in F and for all v1, v2 in V.
viii. 1v = v for all elements v in V, where 1 denotes the unit element of the field F.
Note: The elements of V are usually referred to as vectors while the elements of F are
referred as scalars.
Example 1.
Let V = Rn = (v1, v2, ….., vn) each vi ∈ R, the set of all ordered n-tuples of real numbers.
Define addition on V by
(v1, v2, ….., vn) + (w1, w2, . . . . , wn) = (v1 + w1, v2 + w2, ….., vn + wn)
for all vi, wi ∈ R (i = 1, 2, … , n).
Define scalar multiplication on V by
λ (v1, v2, ….., vn) = (λv1, λv2, …..,λvn) for all λ ∈ F, and vi ∈ R.
Then it is easy to check that V = Rn is a vector space over F = R.
1
Lecture Notes in Applied Mathematics Dr. K.C. Raajasekar
Inner Product
The Inner product on a real vector space V is a mapping V × V → R, which is symmetric,
bilinear and positive definite.
Length or Norm of a vector

The length or norm of a vector v with respect to an inner product < > on the vector
space V over F, denoted by || v ||, and is defined by ||v|| 2 = <v, v>.
Orthogonal basis
A basis {v1, v2, . . . . , vn} of a finite dimensional vector space V is called an orthogonal
basis with respect to the inner product < > on V provided <vi , vj> = 0 for all i ≠ j.
Orthonormal basis
An orthogonal basis {v1, v2, . . . . , vn}of finite dimensional vector space V is called an
orthonormal basis with respect to the inner product < > on V provided || vi || = 1 for each i
= 1, 2, ……, n.
Gram-Schmidt orthonormalization process
Let {v1, v2, . . . . , vn} be a basis of a finite-dimensional vector space V over F.
Let < > be an inner product on V. An orthnormal basis {w 1, w2, .. . . . . , wn} of V is
constructed as follows:
v1
Let w1 =
|| v1 ||
y2
Let y2 = v2 - <w1, v2> w1, and then w2 =
|| y2 ||
k 1
Assuming w1, w2, . . . . , wk-1 have been defined, we first let yk = vk -  w ,v
i 1
i k  wi , then
yk
define wk = .
|| yk ||
Then {w1, w2, . . . ., wn} is an orthonormal basis of V.
2
Example 2.
Use Gram-Schmidth Orthonormalization process to find orthonormal basis for the
following orthogonal basis {1, 2, 3}.
Soln.
Here v1 = i; v2 = 2j; and v3 = 3k.
||v1|| = ||i|| = 1.
v1 i
Let w1 = = = i.
|| v1 || 1
Let y2 = v2 - <w1, v2> w1 = 2j - <i, j>i = 2j – (0)i = 2j.
y2 2j
w2 = = = j.
|| y2 || 2
Let y3 = v3 - <w1, v3> w1 - <w2, v3>w2 = 3k - <i, 3k>i - <2j, 3k>2j
= 3k – (0)i – (0)(2j) = 3k
y3 3k
w3 = = = k.
|| y3 || 3
Clearly, {w1, w2, w3} forms an orthonormal basis vector for V.
Note: The above question may be seen as follows:
 1 0 0
 
V =  0 2 0
 0 0 3
 
Each of the vectors vi is orthogonal to vj for i ≠ j.
The orthonormal vector is
 1 0 0
 
W =  0 1 0 .
 0 0 1
 
Characteristic Polynomial
Let A = (aij) be a square matrix. The characteristic polynomial PA() of A is defined as
determinant of (A - I).
i.e. PA() = | A - I |.
3
Characteristic Root (Eigenvalue)

The roots of the characteristic equation PA(λ) = 0 is known as characteristic root of the
square matrix A.
Example 3.
Find the eigenvalues for the matrix
 1 0 0
 
A =  0 2 0
 0 0 3
 
Soln.
Characteristic Equation is obtained by equating |A - λI| to 0.
Thus,
| A - λI | = 0
1  0 0
⇒ 0 2 0 =0
0 0 3
 (1  )(2  )(3  ) = 0
The eigenvalues are  = 1, 2, 3.
Algebraic Multiplicity of an eigenvalue

The algebraic multiplicity of the eigenvalue 0 of an n × n matrix A is its multiplicity as a
root of the characteristic polynomial PA().
Example 4.
Find the eigenvalue of the following matrix and find the algebraic multiplicity for
each of the eigenvalues:
 2 1 0
 
A =  0 2 1
 0 0 2
 
4
Soln.
The characteristic polynomial = | A - λI |.
The characteristic equation is | A - λI | = 0.
i.e.
2 1 0
0 2 1 =0
0 0 2
 (2 - )(2 - )(2 - ) = 0
  = 2, 2, 2.
Thus, the only eigenvalue is  = 2 whose algebraic multiplicity is 3.
Eigenspace for the eigenvalue
The eigenspace for the eigenvalue 0 of the n × n matrix A, denoted by Wλ0, is defined as
Wλ0= { v ∈ Rn : Av = λ0v }
Example 5.
Find the eigenspace for the eigenvalue(s) obtained for the matrix given in example 4.
Soln.
We obtained the eigenvalue of the given matrix in the previous example as λ = 2 with
algebraic multiplicity 3.
Now, we first find the eigenvector corresponding to eigenvalue λ = 2.
i.e. we have to solve the equation Av = 2v.
Let v = (x, y, z)t.
Then we get
 2 1 0  x   x
   = 2 
 0 2 1 y  y
 0 0 2 z   z
     
 2x  y   2x 
   
i.e.  2 y  z  =  2y
 2z   2z 
   
5
i.e. 2x + y – 2x = 0 ⇒ y=0
2y + z – 2y = 0 ⇒ z=0
2z – 2z = 0 ⇒ 0 = 0.
Thus,
 x  1
   
 y = x  0
 z  z
   
Thus, if we choose x = 1, we get the eigenvector as (1, 0, 0)t corresponding to  = 2.
Thus, the eigenspace corresponding to  = 2 is
{ ( k, 0, 0); k  R}.
Note
1. A vector v = (v1, v2, …. , vn) is said to be linearly independent if for scalars 1, 
2, …. n  R, 1v1 + 2v2 + 3v3 + ….. + nvn = 0  i = 0 for all i = 1, 2, 3, … ,

n.
2. Two vectors v and w are dependant if one is a constant multiple of another vector.
3. The linearly independent vectors of the eigenspace are its dimension.
4. The number of linearly independent vectors in the eigenspace shown in example 5
is one. Thus, its dimension is one.
Geometric Multiplicity of an eigenvalue

The geometric multiplicity of the eigenvalue 0 of an n × n matrix A is the dimension of
the eigenspace Wλ0.
The geometric multiplicity of the eigenvalue λ = 2 for the matrix A discussed in
example 5 above is one.
6
Example 6. Find the eigenvalue, its algebraic multiplicity, eigenvector for each of
the eigenvalues, and their geometric multiplicities for the matrix
 2 1 0
 
A =  0 2 0 .
 0 0 2
 
Soln.
The characteristic polynomial = | A - I |.
The characteristic equation is | A - I | = 0.
i.e.
2 1 0
0 2 0 =0
0 0 2
 (2 - )(2 - )(2 - ) = 0
  = 2, 2, 2.
We find the eigenvector corresponding to eigenvalue  = 2.
Let v = (x, y, z)t.
Then we get
 2 1 0  x   x
   = 2 
 0 2 0 y  y
 0 0 2 z   z
     
 2x  y   2x 
   
i.e.  2 y  =  2y
 2z   2z 
   
i.e. 2x + y – 2x = 0 ⇒ y=0
2y – 2y = 0 ⇒ 0=0
2z – 2z = 0 ⇒ 0 = 0.
7
Thus,
 x  x  1  0
       
 y =  0   x  0  z  0 
 z  z   0  1
       
Thus, if we choose x = 1, and z = 0, we get the eigenvector as (1, 0, 0) t, and if we choose
x = 0 and z = 1, we get another eigenvector as (0, 0 , 1) corresponding to  = 2. Thus, we
have two linearly independent eigenvectors corresponding to the eigenvalue  = 2.
Thus, the geometric multiplicity corresponding to  = 2 is two.
Example 7. Find the eigenvalue, its algebraic multiplicity, eigenvector for each of
the eigenvalues, and their geometric multiplicities for the matrix
 2 0 0
 
A =  0 2 0 .
 0 0 2
 
Soln.
The characteristic polynomial = | A - I |.
The characteristic equation is | A - I | = 0.
i.e.
2 0 0
0 2 0 =0
0 0 2
 (2 - )(2 - )(2 - ) = 0
  = 2, 2, 2.
We find the eigenvector corresponding to eigenvalue  = 2.
8
Let v = (x, y, z)t.

Then we get
 2 0 0  x   x
   = 2 
 0 2 0 y  y
 0 0 2 z   z
     
 2x   2x 
   
i.e.  2y =  2y
 2z   2z 
   
i.e. 2x – 2x = 0 ⇒ 0=0
2y – 2y = 0 ⇒ 0=0
2z – 2z = 0 ⇒ 0 = 0.
Thus,
 x  1  0  0
       
 y  = x  0  y  1  z  0 
 z  0  0  1
       
Thus, if we choose x = 1, y = 0, z = 0, we get the eigenvector as (1, 0, 0)t, and if we
choose x = 0, y = 1 and z = 0, we get another eigenvector as (0, 1 , 0). Further, If we
choose x = 0, y= 0 and z = 1, we get yet another eigenvector (0, 0, 1) corresponding to 
= 2. Thus, we have three linearly independent eigenvectors corresponding to the
eigenvalue  = 2.
Thus, the geometric multiplicity corresponding to  = 2 is three.
9
Example 8. Find the eigenvalues, and their algebraic and geometric multiplicities
for the matrix:
 2 6 1 
 
A=  0 1 3 
 0 3 1
 
Soln.
The given matrix is a 3 × 3 matrix. Its characteristic equation may be found as
follows:
λ 3 – aλ 2 + bλ - c = 0
Where
a = sum of the diagonal elements of A
b = sum of the minors of the main diagonal elements
c = | A |.
Thus,
a = a11 + a22 + a33 = 2 + 1 + 1 = 4
1 3 2 1 2 6
b=   = (1 – 9) + ( 2 – 0) + (2 – 0) = - 8 + 2 + 2 = - 4.
3 1 0 1 0 1
2 6 1
c = 0 1 3 = 2(1 – 9) = - 16.
0 3 1
Thus, the characteristic equation is

λ 3 – 4 λ 2 + (-4)λ - (-16) = 0
i.e. λ 3 – 4 λ 2 – 4 λ + 16 = 0 --------------(1)
We shall find one root of equation (1) by trial and error method.
It is not difficult to check that λ = 2 is one of the root.
We reduce the equation by synthetic division method as follows:
1 4 4 16
2
0 2 4 16
1 -2 -8 0
The reduced equation is (λ 2 – 2λ - 8) = 0
10
i.e. (λ - 4) (λ + 2) = 0
i.e. λ = 4, -2.
Thus, the eigenvalue of A are
λ1 = 2 with algebraic multiplicity 1;
λ2 = 4 with algebraic multiplicity 1;
and λ3 = -2 with algebraic multiplicity 1.
Now, we find the geometric multiplicities for each of the eigenvalues obtained.
First, we find eigenvector corresponding to the eigenvalue λ1 = 2.
Thus, we have to solve the equation Av = 2v.
Let v = (x1, y1, z1)t.
Then we get
 2 6 1  x1   x1 
    
 0 1 3   y1  = 2  y1 
 0 3 1  z   z 
   1  1
 2 x1  6 y1  z1   2 x1 
   
i.e.  y1  3 z1 =  2 y1 
 3 y1  z1   2z 
   1
i.e. 2x1 + 6y1 – z1 – 2x1 = 0  6y1 – z1 = 0 -----(i)
y1 + 3z1 – 2y1 = 0  - y1 + 3z1 = 0 -----(ii)
3y1 + z1 – 2z1 = 0  3y1 – z1 = 0. -----(iii)
(i) - (iii)  3y1 = 0  y1 = 0

Using (i), z1 = 6y1 = 6(0) = 0.
Thus,
 x1   x1   1
     
 y1  =  0  = x1  0 
 z   0  0
 1    
For x1 = 1, the eigenvector corresponding to the eigenvalue  = 2 is (1, 0, 0)t.
11
It has only one (independent vector). Thus, the geometric multiplicity for the
eigenvalue λ = 2 is one. (note that only one variable remain in the vector).
Now, we find eigenvector corresponding to λ2 = 4.
Thus, we have to solve the equation Av = 4v.
Let v = (x2, y2, z2)t.
Then we get
 2 6 1  x2   x2 
    
 0 1 3   y2  = 4  y2 
 0 3 1  z   z 
   2  2
 2 x2  6 y2  z2   4 x2 
   
i.e.  y2  3 z 2  =  4 y2 
 3 y2  z 2   4z 
   2 
i.e. 2x2 + 6y2 – z2 – 4x2 = 0  - 2x2 + 6y2 – z2 = 0 -----(i)

y2 + 3z2 – 4y2 = 0  - 3y2 + 3z2 = 0 -----(ii)
3y2 + z2 – 4z2 = 0  3y2 – 3z2 = 0 -----(iii)
(iii)  y2 = z2.
2
(i)  - 2x2 + 6z2 – z2 = 0  - 2x2 + 5z2 = 0  2x2 = 5z2  x2 = z2
5
 
 x2   1 
 x2   
   
 y2  =  x2 = x2  1 
  
 z2  
 2   2 
 x2  5
 5 
Thus, for x2 = 1, the eigenvector is (1, 1, 2/5) corresponding to 2 = 4. The geometric
multiplicity is one.
Finally, we consider the eigenvector 3 = -2. The eigenvector may be calculated as
follows:
i.e. we have to solve the equation Av = - 2v.
Let v = (x3, y3, z3)t.
Then we get
12
 2 6 1  x3   x3 
    
 0 1 3   y3 = -2  y3 
 0 3 1  z   z 
   3  3
 2 x3  6 y3  z3   2 x3 
   
i.e.  y3  3 z3  =  2 y3 
 3 y3  z3   2 z 
   3 
i.e. 2x3 + 6y3 – z3 + 2x3 = 0  4x3 + 6y3 – z3 = 0 -----(i)

y3 + 3z3 + 2y3 = 0  3y3 + 3z3 = 0 -----(ii)
3y3 + z3 + 2z3 = 0  3y3 + 3z3 = 0 -----(iii)
(iii)  z3 = - y3.
(i)  4x3 + 6y3 – (-y3) = 0
4x3 + 7y3 = 0
4
 7y3 = - 4x3  y3 = x3
7
4
z3 = x3
7
 
 x3   1 
 x3     
  4 
 y3  =
 x3  x3 4 

 7   7
 z     4 
 3 4  
 x3   7
 7 
For x3 = 1, the eigenvector is (1, -4/7, 4/7)t corresponding to the eigenvalue 3 = -2. The
geometric multiplicity is one.
Example 9.
Find the eigenvalues, and their algebraic and geometric multiplicities for the
matrix:
 1 0 1 0
 
0 1 0 1
A=  .
 1 0 1 0
 
 0 1 0 1 
Soln.
13
First, we find the characteristic equation.

The characteristic equation is
| A - λI | = 0.
1  0 1 0
0 1  0 1
=0
1 0 1  0
0 1 0 1 
1  0 1 0 1  1
(1-) 0 1   0 +11 0 0 =0
1 0 1  0 1 1 
(1 - ) (1 - )[(1 - )2 – 1] - 1 [(1- )2 – 1 ] = 0 [ when expanded along row 2

[ (1- )2 – 1 ] [ ( 1 -  )2 - 1 ] = 0.
[1 - 2 + 2 -1]2 = 0
[2 – 2]2 = 0
[ ( - 2)]2 = 0
  = 0, 2 each with algebraic multiplicity 2.
Now, we find the eigenvector corresponding to  = 0.
i.e. Av = 0.
Let v = (x1, y1, z1, w1).
 1 0 1 0  x1 
  
 0 1 0 1  y1 
 =0
 1 0 1 0  z1 
   
 0 1 0 1  w1  
 x1 + z1 = 0; y1 + w1 = 0
 z1 = -x1; w1 = - y1.
Thus,
14
 x1   x1   1  0
       
 y1  =  y1  = x1  0  + y1  1
 z1    x1   1   0
     
  
 w1 

  y1 

 0  1 

For x1 = 1 and y1 = 0, we get the eigenvector as ( 1, 0 , -1, 0), and for x1 = 0 and y1 = 1,
we get another independent vector (0, 1, 0, -1).
This shows that the eigenvalue  = 0 has geometric multiplicity 2.
Now, we find the eigenvector corresponding to λ = 2.

i.e. Av = 2v.
Let v = (x2, y2, z2, w2).
 1 0 1 0  x2   x2 
    
 0 1 0 1  y2  y2 
⇒ = 2
 1 0 1 0  z2   z2 
 0     
 1 0 1  w2  
 w2  
 x2 + z2 – 2x2 = 0;
y2 + w2 – 2y2 = 0
x2 + z2 – 2z2 = 0
y2 + w2 – 2w2 = 0
 z2 = x2 ; w2 = y2;
 x2   x2   1  0
       
 y2  =  y2  = x2  0 + y2  1
 z2   x2   1  0
 w    y    0   1

 2  2    
For x2 = 1 and y2 = 0, we get the eigenvector as ( 1, 0 , 1, 0), and for x2 = 0 and y2 = 1, we
get another independent vector (0, 1, 0, 1).
This shows that the eigenvalue λ = 2 also has geometric multiplicity 2.
Jordan block
A k × k Jordan block J(λ) corresponding to an element λ ∈ C is a matrix of the form
15
  1 0 0 0 0
 
 0  1 0 0 0
 0 0  0 0 0
 
 
 0 0 0 0  1
 
 0 0 0 0 0  
i.e. J(  ) has entries  on the diagonal, “one” on the superdiagonal, and “zero” elsewhere.
Jordan canonical form

The following remarks will be useful for a Jordan canonical form of a n × n matrix A.
1. The sum of the sizes of the blocks involving a particular eigenvalue of A equals
the algebraic multiplicity of that eigenvalue.
2. The number of blocks involving a particular eigenvalue of A equals the geometric

multiplicity of that eigenvalue.
3. The largest block involving a particular eigenvalue of A equals the multiplicity of

that eigenvalue as a root of the minimal polynomial of A.
Example.
1. Write Jordan Canonical form for the matrix given in example 5.
Soln.
The eigenvalue  = 2 has algebraic multiplicity 3 and geometric multiplicity 1.
Thus, the Jordan Canonical form of the matrix A is
 2 1 0
 
J =  0 2 1
 0 0 2
 
Note:
1. Sum of the sizes of the blocks (for  =2) = 3 = algebraic multiplicity.
2. Geometric Multiplicity of  = 2 is one = the number of blocks.
2. Write Jordan Canonical form the the matrix given in example 6.

Soln.
The eigenvalue  = 2 has algebraic multiplicity 3, and geometric multiplicity 2.
16

  2 1 0 
   
J =   0 2 0 
 0 0  2  

Note:
1. Sum of the sizes of the blocks (for  =2) = 2 + 1 = 3 = algebraic multiplicity.
2. Geometric Multiplicity of  = 2 is two = the number of blocks.
17
3. Write Jordan Canonical form the matrix given in example 7.

Soln.
The eigenvalue λ = 2 has algebraic multiplicity 3, and geometric multiplicity 3.
  2 0 0 
 
J=  0  2 0 .
 0 0  2  

Note:
1. Sum of the sizes of the blocks (for  =2) = 1 + 1 + 1 = 3 = algebraic multiplicity.
2. Geometric Multiplicity of  = 2 is three = the number of blocks.

Soln.
The eigenvalue  = -2, 2, 4 each has algebraic multiplicity 1, and geometric multiplicity
also equal to 1.
  2  0 0 
 
J=  0  2 0 .
 0 0  4  

1. Sum of the sizes of the blocks (for  = -2 ) = 1 = algebraic multiplicity.
2. Geometric Multiplicity of  = -2 is one = the number of blocks (for that λ).
3. From (1) and (2), we get similar results for the eigenvalues λ = 2 and λ = 4.
18

Soln.
The eigenvalue λ = 0, 2 each has algebraic multiplicity 2, and geometric multiplicity also
equal to 2.
  0 0 0 0 
 
 0  0 0 0 
J=  .

0 0  2  0 
 0 0 0  2 
 
Note:
1. Sum of the sizes of the blocks (for  = 0 ) = 1 + 1 = algebraic multiplicity.
2. Geometric Multiplicity of  = 0 is two = the number of blocks (for that λ).
3. From (1) and (2), we get similar result for the eigenvalues λ = 2.
Exercise.
1. Show that the following matrix has eigenvalue λ = 0 with algebraic

multiplicity 3 and geometric multiplicity 1.
 0 1 2
 
 0 0 1 .
 0 0 0
 
2. Show that the Jordan (canonical) form of the matrix given in exercise 1 is
 0 1 0
 
 0 0 1
 0 0 0
 
3. Show that the following matrix has eigenvalues λ = 2 and – 4. Show also that
the algebraic multiplicity for λ = 2 is two and geometric multiplicity is one.
Further, show that λ = – 4 has algebraic and geometric multiplicity one.
 2 1 3 
 
 4 3 3 .
 2 1 1 
 
19
Hint: Characteristic Equation is λ 3 – 0λ2 + (-12)λ - 16 = 0

  4  0 0 
 
 0  2 1  .
  
 0  0 2 
5. Find the eigenvalue, algebraic and geomentric multiplicity for the matrix
 0 1 0 0
 
 0 0 0 0
 0 0 0 0
 0 0 0 0

 
Hint:  = 0 has algebraic multiplicity 4 and geometric multiplicity 3.
6. Show that the Jordan (canonical) form for the matrix given in exercise 5 is
  0 1 0 0 
   
  0 0 0 0 
 0 0  0 0 .
 
 0 0 0  0 
 
 
Note: For exercise 6, two more Jordan forms exist. They are
 
  0 0 0 0   
    0 0 0 0 
 0  0 1 0 
 0   and  0  0 0 0 
 0 0 0   
   0 0  0 1 
 0 0 0  0 
   0  0 0 
 0   
7. Find the eigenvalue, algebraic multiplicity and geometric multiplicity for the
matrix given below:
 0 1 0 1
 
 0 0 0 0
 0 0 0 1
 
 0 0 0 0 
Hint:  = 0 has algebraic multiplicity 4 and geometric multiplicity 2.
20
8. Show that the Jordan form for the matrix given in exercise 7 is
  0 1 0 0 
   
  0 0 0 0 
 0 0  0 .
1 
 
 0 0  0 0 
   
9. Show that the following matrix has eigenvalue λ = 1 with algebraic
multiplicity 1, and geometric multiplicity 1. Also, show that the eigenvalue λ
= 2 has algebraic multiplicity 2 and geometric multiplicity 2
 1 6 3
 
 0 2 0 .
 0 0 2
 
  1 0 0 
 
 0  2 0 
 0 0  2  

21
Procedure for finding minimal polynomial for a given square matrix

Step 1. Let A be a given square matrix.
Step 2. Find the characteristic polynomial.
Step 3. Factorize it completely.
Step 4. List all the possibilities for the minimal polynomial in increasing order of
degree.
Step 5. Evaluate each possibility at A, starting at the one of lowest degree, until a
polynomial which gives zero matrix.
Example 10. Find the minimal polynomial for the matrix shown in example 4.
Soln.
 2 1 0
 
A =  0 2 1 .
 0 0 2
 
The characteristic polynomial for the matrix A is
PA() = | A - I |.
2 1 0
| A - I | = 0 2 1
0 0 2
= (2 - )3.
Now, we list all the factors in increasing order of its degree.
i.e. The three possibilities for the minimal polynomial are ( - 2), ( - 2)2, ( - 2)3.
Now, we evaluate  - 2 for  = A.
 2 1 0  1 0 0  0 1 0
     
A – 2I =  0 2 1  - 2  0 1 0 =  0 0 1  ≠ 0 (zero matrix)
 0 0 2  0 0 1  0 0 0
     
22
Now, we evaluate (λ - 2)2 for λ = A.
 0 1 0  0 1 0  0 0 1
    
(A – 2I) =  0 0 1   0 0 1  =
2
 0 0 0 ≠ 0
 0 0 0 0 0 0  0 0 0
     
Now, we evaluate ( - 2)3 for  = A.
 0 0 1  0 1 0  0 0 0
    
(A – 2I)3 = (A – 2I)2(A – 2I) =  0 0 0   0 0 1  =  0 0 0 .
 0 0 0 0 0 0  0 0 0
     
Thus, the minimal polynomial is ( - 2)3.
Note:
The largest block involving a particular eigenvalue of A equals the multiplicity of
that eigenvalue as a root of the minimal polynomial of A
Thus,
Jordan form for the example 10 (from minimal polynomial approach) is
 2 1 0
 
J =  0 2 1 .
 0 0 2
 
Soln.
 2 1 0
 
Given that A =  0 2 0 .
 0 0 2
 
PA() = | A - I |.
2 1 0
| A - I | = 0 2 0
0 0 2
= (2 - )3.
i.e. The three possibilities for the minimal polynomial are ( - 2), ( - 2)2, ( - 2)3.
23
Now, we evaluate λ - 2 for λ = A.
 2 1 0  1 0 0  0 1 0
     
A – 2I =  0 2 0  - 2  0 1 0  =  0 0 0  ≠ 0 (zero matrix)
 0 0 2  0 0 1  0 0 0
     
Now, we evaluate ( - 2)2 for  = A.
 0 1 0  0 1 0  0 0 0
    
(A – 2I)2 =  0 0 0   0 0 0  =  0 0 0 .
 0 0 0 0 0 0  0 0 0
     
Thus, the minimal polynomial is ( - 2)2.
Note:
Thus,
  2 1 0 
   
J =   0 2 0 .
 0 0  2  

Soln.
 2 0 0
 
Given that A =  0 2 0 .
 0 0 2
 
PA() = | A - I |.
2 0 0
| A - I | = 0 2 0
0 0 2
= (2 - )3.
24
i.e. The three possibilities for the minimal polynomial are (λ - 2), (λ - 2)2, (λ - 2)3.
Now, we evaluate λ - 2 for λ = A.
 2 0 0  1 0 0  0 0 0
     
A – 2I =  0 2 0  - 2  0 1 0 =  0 0 0
 0 0 2  0 0 1  0 0 0
     
Thus, the minimal polynomial is ( - 2).
Note:
Thus,
  2 0 0 
 
J=  0  2 0 .
 0 0  2  

13. Find the minimal polynomial for the matrix shown in example 9.
Soln.
 1 0 1 0
 
0 1 0 1
Given that A =  .
 1 0 1 0
 
 0 1 0 1 
The characteristic polynomial is 2 ( - 2)2 = 0. (Refer example 9).
i.e. The four possibilities for the minimal polynomial are ( - 2),  2( - 2), ( - 2)2,
and 2(-2)2.
Now, we evaluate  ( - 2) for  = A.
 1 0 1 0  1 0 1 0   0 0 0 0
    
0 1 0 1  0 1 0 1   0 0 0 0
A ( A – 2I) =  =
 1 0 1 0  1 0 1 0   0 0 0 0
     
 0 1 0 1  0 1 0 1    0 0 0 0 
Thus, the minimal polynomial is ( - 2).
25

  0 0 0 0 
 
 0  0 0 0 
J=  .

0 0  2  0 
 0 0 0  2 
 
Exercise
1. Convince yourself that the result obtained in the examples 10 – 13 coincides
with the result already obtained in examples 5 – 7; 9.
2. Let A be a 7 × 7 matrix whose characteristic polynomial is (2 - λ)4 (3-λ)3, and
whose minimal polynomial is (2-λ)2(3-λ)2.
Convince yourself that one of the Jordan form is
  2 1 0 0 0 0 0 
   
  0 2 0 0 0 0 0 
 0 0  2 0 0 0 0 
 
J=  0 0 0  2 0 0 0 
 0 0 0 

0 0  3 1 
 0 0   
0 0  0 3  0
 
 0 0 0 0 0 0  3 
3. Some of the other Jordan forms for Exercise 2 are
  2 0 0 0 0 0 0    2 0 0 0 0 0 0 
   
 0  2 1 0 0 0 0   0  2 0 0 0 0 0 
 0  
 0 2 0 0 0 0   0 0  2 1 0 0 0 
     
 0 0 0  2 0 0 0  and  0 0  0 2 0 0 0 
   
 0 0 0 0  3 1 0   0 0 0 0  3 1 0 
 0 3 0 
 0 0 0 0    0 0 0 0  0 3 
 0 
   
 0 0 0 0 0 0  3 
  0 0 0 0 0 0  3 
4. More Jordan form exists for the exercise 2. (Convince yourself!).
26
Generalized eigenvectors
A vector xm is a generalized eigenvector of type m corresponding to the matrix A and the
eigenvalue λ if (A - λI) m xm = 0 but (A - λI) m-1 xm ≠ 0.
Example 1. Show that the vector (0, 0, 1)t is a generalized vector of type 3 corresponding
to the eigenvalue λ = 2 for the following matrix :
 2 1 1 
 
A=  0 2 1 
 0 0 2
 
Soln.
We have to show that (A - 2I) 3 x3 = 0 but (A - 2I) 2 x3 ≠ 0.
 2 1 1   1 0 0  0 1 1 
     
A - 2I =  0 2 1  - 2  0 1 0  =  0 0 1 
 0 0 2   0 0 1  0 0 0 
     
 0 1  1   0 1 1   0 0 1
     
(A - 2I)2 =  0 0 1   0 0 1  =  0 0 0 
 0 0 0  0 0 0   0 0 0
     
 0 0 1   0 1 1   0 0 0
     
(A - 2I)3 = (A - I)2(A - I) =  0 0 0   0 0 1  =  0 0 0 
 0 0 0 0 0 0   0 0 0
     
 0  0 0 0  0
    0
 0  0 0 0 
    1
3 1
(A - 2I) =  0 0 0   = 0
 0 0 0 1  0  1
    0  
 0  0 0 0   0
    1  
2 1
(A - 2I) =  0 0 0   =  0 ≠ 0
Thus, the required result is proved.
27
Example 2.
Show that the vector (-1, 1, 0)t is a generalized vector of type 2 corresponding to the
eigenvalue λ = 2 for the following matrix :
 2 1 1 
 
A=  0 2 1 
 0 0 2
 
Soln.
We have to show that (A - 2I)2 x2 = 0 and (A – 2I)x2 ≠ 0.
Note that A and  values are same as that of the previous question.
 0 0 1   1   0
    
(A - 2I)2 x2 =  0 0 0   1  =  0
 0 0 0 0   0
     
Further,
 0 1  1   1   1
    
(A - 2I) x2 =  0 0 1   1  =  0 ≠ 0
 0 0 0  0   0
     
Thus, x2 is a generalized eigenvector of type 2 with eigenvalue  = 2 for the matrix A.
Example 3.
Show that (1, 0 , 0)t is a generalize eigenvector of type 1 corresponding to λ = 2 for
the matrix
 2 1 1 
 
A=  0 2 1 
 0 0 2
 
Soln.
Again the matrix A and eigenvalue  are same as that of example 1.
Thus,
 0 1 1 
 
A – 2 I =  0 0 1 .
 0 0 0
 
We have to show that (A – 2I)x1 = 0 and (A – 2I)0x1 ≠ 0.
28
 0 1 1   1   0
    
(A – 2I)x1 =  0 0 1   0  =  0 
 0 0 0  0  0
     
 1
 
(A – 2I)0 x1 = x1 =  0  ≠ 0
 0
 
Thus, x1 is a generalized eigenvector of type 1 corresponding to  = 2 for the matrix A.
Example 4.
It is known that the matrix
 5 0 2
 
A=  2 1 1 
 5 1 1
 
has a generalized eigenvector of type 2 corresponding to λ = 2. Find it.
Soln.
Let x2 = (x , y, z)t be the required generalized eigenvector.
Then x2 has to satisfy

a) (A – 2I)2x2 = 0, and
b) (A – 2I)x2 ≠ 0.
 5 0 2  1 0 0  3 0 2
     
A – 2I =  2 1 1  - 2  0 1 0  =  2 1 1 
 5 1 1   0 0 1   5 1 3 
     
 3 0 2  3 0 2
  
(A – 2I)2 =  2 1 1   2 1 1 
 5 1 3   5 1 3 
   
 9  10 2 66   1 2 0 
   
=  625 11 4  1  3  =  1 2 0 
 15  2  15 1  3 10  1  9   2 4 0 
   
29
 1 2 0   x   0
    
(A – 2I) x2 = 0 ⇒  1 2 0   y  =  0 
2
 2 4 0   z   0 
     
 x  2 y   0 
   
  x  2 y   0 
 2x  4 y   0 
   
-x+2y=0
-x+2y=0
2x–4y=0
 x = 2y.
Also, (A – 2I)x2 ≠ 0
 3 0 2   x
  
(A – 2I)x2 =  2 1 1   y 
 5 1 3   z 
   
 3x  2 z 
 
= 2 x  y  z  ------------(*)
 5 x  y  3 z 
 
Since we need x = 2y, by using this in (*) we get,
3(2 y )  2 z   6 y  2 z   2(3 y  z ) 

     
(A – 2I)x2 =  2(2 y )  y  z    3 y  z    3 y  z 
 5(2 y )  y  3 z   9 y  3z   3(3 y  z ) 
     
Note that (A – 2I)x2 ≠ 0.
Thus, 3y + z ≠ 0.
So, Any value of x, y, z satisfying the conditions x = 2y and 3y + z ≠ 0 is the required

generalized eigenvector of type 2.
One such generalized eigenvector is y = 1; x = 2; z = 0
i.e. generalized eigenvector of type 2 is (2, 1, 0)t.
30
Example 5.
It is known that the matrix
 5 1 2 4
 
0 5 2 2
A= 
 0 0 5 3
 0 0 0 4

 
has a generalized eigenvector of type 3 corresponding to λ = 5. Find it.
Soln.
Let x3 = (w, x, y, z)t be the required generalized eigenvector.
Then x3 has to satisfy (A – 5I)3x3 = 0 and (A – 5I)2 ≠ 0.
 5 1 2 4  1 0 0 0  0 1 2 4 
     
0 5 2 2  0 1 0 0  0 0 2 2
A – 5I =  -5 =
 0 0 5 3  0 0 1 0  0 0 0 3
     
 0 0 0 4   0 0 0 1   0 0 0 1  
 0 1 2 4   0 1 2 4 
  
0 0 2 2  0 0 2 2
(A – 5I) = 
2
 0 0 0 3  0 0 0 3
   
 0 0 0 1   0 0 0 1  
 0 0 2 2  6  4  0 0 2 8 
   
0 0 0 62   0 0 0 4
= =
 0 0 0 3   0 0 0 3 
   
 0 0 0 1    0 0 0 1 
 0 0 2 8   0 0 2 8   0 0 0 6  8 
    
0 0 0 4  0 0 0 4   0 0 0 4 
(A – 5I) =(A – 5I) (A – 5I ) = 
3 2
=
 0 0 0 3   0 0 0 3   0 0 0 3 
   
  
 0 0 0 1   0 0 0 1  0 0 0 1  
31
 0 0 0 14   w   0
    
0 0 0 4   x   0 
(A – 5I) x3 = 0 ⇒ 
3
=
 0 0 0 3  y   0
      0 
 0 0 0 1    z  

 14 z   0
   
 4 z   0
 =
 3z   0
 
   
 z   0
 z = 0.
 0 0 2 8   w 
  
0 0 0 4  x 
(A – 5I) x3 = 
2
 0 0 0 3   y 
   
 0 0 0 1   z

 2 y  8z 
 
 4z 
= --------------------- (*)
 3 z 
 
 z 

Using z = 0 in (*), we get
 2y
 
2  0
(A – 5I) x3 =
 0
 

 0
Note that x3 should satisfy (A – 5I)2x3 ≠ 0.
Thus,
y ≠ 0.
So, any value of w, x, y, z satisfying the conditions z = 0 and y ≠ 0 can be a required

generalized eigenvector x3.
Thus, One such generalized eigenvector is x3 = (0, 0, 1, 0).
Note: we have chosen w = 0; x = 0; y = 1; and z = 0.
32
SINGULAR VALUE DECOMPOSITION (SVD)

Result 1.
Let A be an m × n matrix with complex entries. Then all of the eigenvalues of the
n × n Hermitian matrix A t A are nonnegative.
Note: A is obtained from A by changing all its complex entries into its conjugate.
Singular Value
Let A be an m × n matrix with complex entries. The singular values of A are σ1, σ2, σ3,
…. , σn where σ1 ≥ σ2 ≥ σ3 ≥ …. ≥ σn > 0 and σ12, σ22, σ32, ……, σn2 are the
eigenvalues of A t A.
Rank of a matrix
A matrix is said to be of rank r when
a) it has at least one non-zero minor of order r, and
b) every minor of order higher than r vanishes.
Example
 1 0 0
 
Find the rank of the matrix  0 2 0  .
 0 0 3
 
Soln.
 1 0 0
 
Let A =  0 2 0 
 0 0 3
 
Given matrix is of order 3 (it is a 3 × 3 matrix).
Rank(A) ≤ Min (row, column) = Min (3, 3) = 3.
1 0 0
| A | = 0 2 0 = 1(6) = 6 ≠ 0.
0 0 3
Thus, rank (A) = 3.
33
Example
 1 0 0
 
Find the rank of the matrix  0 2 0  .
 0 0 0
 
Soln.
 1 0 0
 
Let A =  0 2 0 
 0 0 0
 
Given matrix is of order 3 (it is a 3 × 3 matrix).
Rank(A) ≤ Min (row, column) = Min (3, 3) = 3.
But, all the entries in the third row/column is zero. So, |A| = 0.
Thus, we have to consider the matrix after removing third row/column.
 rank(A) ≤ 2.
 1 0
It is not difficult to check that the minor   has a non-zero determinant value.
 0 2
Thus, rank (A) = 2.
Result 2.
Let A be an m × n matrix with complex entries.
Then rank A = rank A tA = rank A A t.
Unitary Matrix
A square matrix U is said to unitary if U tU = I.
Theorem (SVD)
Let A be an m × n matrix with complex entries. Let 1, 2, 3, …. , r be the non-zero
singular values of A. Where r = rank A, and i are written in non-decreasing order
1 ≥ 2 ≥ 3 ≥ …. ≥ n > 0. Let D be the r × r matrix with entries 1, 2, 3, …. , r on
the main diagonal and zero elsewhere. Let ∑ be the m × n matrix with D as the top left-
hand r × r block and zero elsewhere. Then there exists a unitary m × m matrix U and a
unitary matrix V such that U tAV = ∑.
34
Note: The columns of U are a full orthonormal set of eigenvectors of the m × m

matrix A A t, and the columns of V are a full orthonormal set of eigenvectors of the
n × n matrix A tA.
Example.
 2 1 
 
Find the singular value decomposition of A =  2 1 
 4 2 
 
Soln.
Since the given matrix is a real matrix (entries are real numbers), A = A.
 2 2 4 
Therefore, A t = At, and At =  
 1 1 2 
Now,
 2 1 
 2 2 4   
  2 1 
t
A A= 
 1 1 2   4 2 
 
 4  4  16 2  2  8   24 12 
= =  
 2  2  8 1  1  4   12 6 
The characteristic polynomial of At A is
 24 12   1 0
=     
 12 6   0 1
24   12
=
12 6  
= (24-)(6-) – 144
= 144 – 24 - 6 + 2 – 144
= 2 - 30
=  ( - 30).
The eigenvalues of At A are  = 0, 30.
35
Among these eigenvalues the only positive eigenvalue is 30.

 30 0
 
The singular value decomposition of A will be the matrix ∑ =  0 0 .
 0 0
 
We can determine the unitary matrices U and V such that U tAV = ∑.
Note that the matrix U is the orthonormal basis of the eigenvector of AAt.
 2 1 
   2 2 4 
AAt =  2 1   
 4 2   1 1 2 
 
 2  2  (1)  (1) 2  (2)  (1)  1 2  4  (1)  (2) 

 
=  (2)  2  1 (1) (2)  (2)  11 (2)  4  1 (2) 
 4  2  ( 2)  (1) 4  (2)  (2) 1 4  4  (2)  (2) 
 
 4  1 4  1 8  2 
 
=  4  1 4  1 8  2 
 8  2 8  2 16  4 
 
 5 5 10 
 
=  5 5 10 
 10 10 20 
 
Characteristic polynomial of AAt
 5 5 10   1 0 0
   
=  5 5 10    0 1 0 
 10 10 20   0 0 1 
   
5   5 10
= 5 5   10
10 10 20  
= (5 -  )[(5 - )(20 - ) – 100] + 5 [(-5)(20-)+100] + 10[50 – 10(5 - )]

= (5 -  )[100 - 5 - 20 +2 - 100] +5[-100+5 + 100] + 10[50 – 50 + 10]
= (5 - )[2 - 25] + 5(5) + 10(10)
= 52 - 125 - 3 + 252 + 25 + 100
36
= -λ3 + 30λ2
= λ2(30-λ)
The eigenvalues are 0, 0, 30.
For the eigenvalue λ = 0, the eigenvector is calculated as follows:
Solve for v, the equation:
(AAt - λI)v = 0.
i.e. AAt v = 0. [since λ = 0
Let v = (x, y, z)t.
Then
 5 5 10   x   0
    
AA v = 0 ⇒  5 5 10   y  =  0 
t
 10 10 20   z   0 
     
 5 x  5 y  10 z   0
   
 5 x  5 y  10 z  =  0 
 10 x  10 y  20 z   0 
   
 5(x – y + 2z) = 0
-5(x – y + 2z) = 0
10(x – y + 2z) = 0
 x = y – 2z
 x  y  2z   y   2z   1  2 
           
 y =  y =  y   0  = y  1  z  0 
 z  z   0  z   0  1 
           
For y = 1 and z = 0 we get one eigenvector as v1 = (1, 1, 0)t.
For y = 0 and z = 1 we get another eigenvector as v2 = (-2, 0, 1)t.
Thus for the eigenvalue  = 0, we have two eigenvectors v1 and v2.
Now, we find eigenvector for the eigenvalue λ = 30.
Solve for v, the equation:
(AAt - 30I)v = 0.
Let v = (x, y, z)t.
37
(AAt - 30I)v = 0
  5 5 10   1 0 0   x   0
       
⇒   5 5 10  30  0 1 0  y    0 
  10 10 20   0 0 1  z   0 
         
 25 5 10   x   0 
    
  5 25 10  y    0 
 10 10 10  z   0 
     
 25 x  5 y  10 z   0 
   
  5 x  25 y  10 z    0 
 10 x  10 y  10 z   0 
   
 -5(5x + y – 2z) = 0 ------------(i)
-5(x + 5y + 2z) = 0 -----------(ii)
10(x – y - z) = 0 -----------(iii)
Solving (i) and (iii), we get
5x + y – 2z = 0
x–y -z =0
----------------------------
6x - 3z = 0 ----------(iv)
Solving (ii) and (iii), we get
x + 5y + 2z = 0
5x - 5y - 5z = 0
______________________
6x - 3z = 0 --------- (v)
 6x = 3z
 z = 2x
(iii)  y = x – z = x – 2x = - x
Thus,
 x  x   1
     
 y =   x   x  1 
 z  2x   2 
     
For x = 1, the eigenvector is v3 = (1, -1, 2)t.
38
Now, we find orthonormal basis {w1, w2, w3} from {v1, v2, v3} using Gram-schmidt
(orthonormalization) process
||v1|| = 12  12  0  2
 1 
 1  2
1 1    1 
w1 = v1 = 1 
|| v1 || 2     2
   0 
0

 
Let y2 = v2 - <w1, v2> w1
1 1
<w1, v2> = (2)  (0)  0(1)   2
2 2
Thus,
 1 
 2   2  2   1  1 
   1 =      
y2 =  0 - (-2)   0 +  1 =  1
 1 2  1  0  1
         
 0 
 
 1 
 1  3
1 1    1 
 1  
w2 = y2 =
|| y2 || (1) 2  12  12    3
 1   
1 3
 
Let y3 = v3 - <w1, v3>w1 - <w2, v3> w2
1 1
< w1, v3> = (1)  (1)  0(2)  0
2 2
1 1 1
<w2, v3> =  (1)  ( 1)  (2)  0
3 3 3
Thus, y3 = v3.
 1 
 1  6 
1 1    
Now, w3 = y3 =
 1     1 
|| y3 || 12  (1) 2  22   6
 2   
2 6 
 
39
 1 1  1 
 6 2 3
 
The orthonormal vector U =  1 1 1 .
6 2 3 
 
 2 6 0 1 3 
 
 24 12 
Now, we find orthonormal basis for V from the eigenvectors of AtA =  .
 12 6 
We already found the eigenvalue of AtA as 30 and 0.
We find the eigenvector corresponding to λ = 0
Let v = (x , y)
We have to find v such that (AtA – 0I)v = 0.
i.e. AtAv = 0.
 24 12   x   0
i.e.    =  
 12 6   y   0
24x – 12y = 0  12(2x – y) = 0  2x = y
-12x + 6y = 0  -6(2x – y) = 0
 x  x  1
Thus,  =    x  .
 y  2x   2
Thus, the eigenvector corresponding to  = 0 is v1 = (1, 2)t [for the choice of x = 1]
We find the eigenvector corresponding to λ =30
Let v = (x , y)
We have to find v such that (AtA – 30I)v = 0.
 6 12   x   0
i.e.    =  
 12 24   y   0
 -6x – 12y = 0  -6(x + 2y) = 0  y = - x/2
-12x – 24y = 0  -12(x + 2y) = 0
 x  x   1 
=  
    x    1 
 x
 y 
 2  2
For x = 1, the eigenvector is v2 = (1, - ½)t.
40
We shall find orthonormal basis {w1, w2} from {v1, v2}

 1 
1 1  1  5
w1 = v1 =    
|| v1 || 12  22  2   2 
 5
Let y2 = v2 - <w1, v2> w1
1 2 1
<w1, v2> = (1)  ( )  0
5 5 2
Thus, y2 = v2.
 1 
y2 =  
 1 
 2
2
1  4 1
|| y2 || = 1  
5
  
2
 2  4 2
Now,
1 2  1   2 5 
w2 = y2 =    
|| y2 || 5  1 2   1 5 

Thus,
 2 5 1 5
V =  .

  1 5 2 5 
Now, it is not difficult to verify that UtAV = ∑.
Important Note: The columns of U and V are written in such a way that the basis
vector corresponding to their respective eigenvalues are in non-increasing order.
Note: In the above example, in U, the first column is for eigenvalue λ = 30 then
second the third columns for λ = 0. Similarly, in V, the first column is for the
eigenvalue λ = 30 and second column is for λ = 0.
Exercise
 2 2 
 
Find the singular value decomposition of the matrix  1 1 .
 2 2
 
41
Moore-Penrose Generalized Inverse (Pseudo Inverse)
Generalized Inverse
Let A be an m × n matrix with complex entries. The generalized inverse of A is the
unique n × m matrix B which satisfies each of the following four conditions.
i) AB = AB t
ii) BA = BA t
iii) ABA = A
iv) BAB = B.
Applications: Generalized inverses are useful in a number of different areas including
differential and difference equations, statistics, numerical analysis and electrical
networks.
Procedure for finding Generalized Inverse
Step 1. Find the Singular value decomposition of A.
i.e. Find the unitary matrices U and V such that U tAV = ∑.
Note that the m × n matrix ∑ has the r × r diagonal block D in the top left corner.
The diagonal entries of D being the non-zero singular values of A, and r being the
rank of A = rank (A A t ) = rank( A tA).
–1
Step 2. Let ∑1 be the n × m matrix with the r × r block D in the top left corner and
zero everywhere else.
Step 3. The Moore-Penrose generalized inverse, denoted by A+, is computed as follows:
A+ = V∑1 U t
Example. Find the Moore-Penrose generalized inverse (Pseudo inverse) for the matrix
 2 1 
 
A =  2 1  .
 4 2 
 
Soln.
We already discussed the singular value decomposition of the matrix A given in this
question (refer the previous example).
Thus,
42
 1 1  1 
 6 2 3
 
U =  1 1 1 .
6 2 3 
 
 2 6 0 1 3 
 
 30 0
 2 5 1 5  
V =   , and ∑=  0 0 .
 1 5 2 5 

 0 0 
 1 0 0
Thus, ∑1 =  30 .
 0 0 0
 
The required Moore-Penrose generalized inverse
A+ = V ∑1 U t = V ∑1Ut [since U is real matrix
 1 1 2 
 6 6 6
 2 51 5  1 0 0  
=   30  1 1 0 
 
 1 5 2 5   0 0 
2 2
1 3
0
 1 3 1 3 
 
1 1 2 
 2 0 0  6 6 6
 5 6  1 1 
=  0 
 1 0 0 2 2

 5 6   1 3 1 3 1 3 
 
 2 2 4 
30 30 30 
=
 1 1 2 
 30 30 30 
 1 1 2 
15 15 15 
=
 1 1 1 
 30 30 15 
43
Least Square Approximation

Defn. The least-squares straight line ( y = mx + c) is the line that minimizes the least-
square error.
Normal equations for the least-squares fit in two variables is
∑y = m ∑x + nc --------------(1)
∑xy = m ∑x2 + c∑x --------------(2)
Here n is the number of data/points.
Example
Find the least-square straight line for the following x – y data:
x 0 1 2 3 4
y 1 5 3 6 9
Soln.
We know that y = mx + c is the best least-square fit for the x – y data.
Normal equations for the least-squares fit in two variables is
∑y = m ∑x + nc --------------(1)
∑xy = m ∑x2 + c∑x --------------(2)

Here n is the number of data/points.
Here, n = 5.
We compute ∑x, ∑y, ∑xy and ∑x2 from the following table.
x y x2 xy
0 1 0 0
1 5 1 5
2 3 4 6
3 6 9 18
4 9 16 36
∑x =10 ∑y = 24 ∑x2 = 30 ∑xy = 65
44
(i) ⇒ 24 = 10m + 5c
(ii) ⇒ 65 = 30m + 10c
(ii) ⇒ 65 = 30m + 10c

(i) × 2 ⇒ 48 = 20m + 10c
_______________
17 = 10m
⇒ m = 1.7.
(i) ⇒ 24 = 10(1.7) + 5c ⇒ 24 – 17 = 5c ⇒ c = 7/5 = 1.4
Thus, the required straight line is y = 1.7x + 1.4.

Exercise
1. The monthly sales figures (in thousands of Rupees) for a newly opened ratail
store are
Month 1 2 3 4 5
Sales 9 16 14 15 21
a) Find the least-square straight line that best fits this data.
b) Use this line to predict sales revenue for month 6.
Hint. Find the straight line equation. For (b), substitute x = 6 to find y value.
Example
Find the least-squares solution to
x + 2y + z = 1
3x – y =2
2x + y – z = 2
x + 2y + 2z = 1
45
Soln.
The system written in matrix form is
 1 2 1  1
  x   
 3 1 0  y    2 
 2 1 1    2
 z   1 
 1 2 2  
   
 1 2 1  1
   x  
 3 1 0     2 .
Let A =
 2 1 1 
;X=  y ; B =  2
   z  
   
 1 2 2  1
The normal equation for this type of system is
At A X = At B
 1 3 2 1
 
At =  2 1 1 2 .
 1 0 1 2 
 
 1 2 1
 1 3 2 1  
   3 1 0 
A A =  2 1 1 2 
t
 1 0 1 2   2 1 1
   1 2 2 
 
 1 1  3  3  2  2  1 1 1 2  3  (1)  2 1  1 2 11  3  0  2  (1)  1 2 
 
=  2 1  (1)  3  1 2  2 1 2  2  (1)  (1)  11  2  2 2 1  (1)  0  1 (1)  2  2 
 11  0  3  (1)  2  2 1 1 2  0  (1)  (1) 1  2  2 11  0  0  (1)  (1)  2  2 
 
 1 9  4 1 2  3  2  2 1 0  2  2 
 
=  2  3  2  2 4 1 1 4 2  0 1  4 
 1  0  2  2 2  0 1 4 1 0 1 4 
 
 15 3 1 
 
=  3 10 5 
 1 5 6
 
46
 1
 1 3 2 1     1 1  3  2  2  2  1 1   1  6  4  1   12 
  2      
AtB =  2 1 1 2    =  2 1  (1)  2  1 2  2 1=  2  2  2  2   4 
 
 1 0 1 2   2   1 1  0  2  (1)  2  2 1   1 0  2  2   1 
   1      
 
Thus, the normal equation is
 15 3 1   x   12 
   =   -------------(*)
 3 10 5   y   4 
 1 5 6 z   1 
     
We solve the system (*) using Gaussian elimination process as follows:
The Augmented matrix

 15 3 1 12 
 
[A : B] =  3 10 5 4 
 1 5 6 1
 
 
 15 3 1 12 
 
3 3 3 3
= 3  (15) 10  (3) 5  (1) 4  (12) 
 15 15 15 15 
 1 1 1 1 
 1  (15) 5  (3) 6  (1) 1  (12)  
 15 15 15 15 
1
R2 ← R2 - 3( R1)
15
1
R3 ← R3 - R1
15
 
 3 1 12   3 1 12 
 15   15
3 1 12
=  0 10  5 4   =  0 47 24 8 
 5 5 5  5 5 5
 0   0 24 89 1  
1 1 4
 5 6 1    5 15 5
 5 15 5
47
 
 3 1 12 
 15 
= 0 47
5
24
5
8
5

 
 0 24 24  47  89 24  24  1 24  8  
         
 5 47  5  15 47  5  5 47  5  
24 5 24
R3 ← R3 -  R2 = R3 - R2
5 47 47
 15 3 1 12 
 
=  0 9.4 4.8 1.6 
 0 0 3.48 -0.62 
 
The solution is
3.48 z = - 0.62  z = - 0.178
9.4y + 4.8z = 1.6  9.4y = 1.6 – 4.8(-0.178)  y = 0.26
15x + 3y + z = 12  15x = 12 – 3(0.26) – (-0.178)  x = 0.75.
i.e. x = 0.75; y = 0.26; z = - 0. 178.
Exercise
Find the least-square solution to the following systems
1. 2x + 3y = 8
3x – y = 5
x+y =6
2. 2x + 3y – 4z = 1
x – 2y + 3z = 3
x + 4y + 2z = 6
2x + y – 3z = 1
48
QR Decomposition
Result:
Let A be an n × n matrix over a complex field. Then there exists a unitary n × n matrix Q
and an upper triangular matrix R such that A = QR.
Procedure for QR decomposition

Suppose A is non-singular (i.e. det(A) ≠ 0). Let v1, v2, …., vn be the columns of A.
1. Transform the basis {v1, v2, ….., vn} into an orthonormal basis {w1, w2, …,wn}
using Gram-Schmidt process.
2. Now Q is the unitary matrix with columns w1, w2, …,wn.
3. To find R, we write vi as follows:
vi = α 1i w1 + α 2i w2 + α 3i w3 + …… + α ni wn, for i = 1,2, …., n
The solution set (α 1i , α 2i, α 3i , …… , α ni ) for each i gives the ith column of the
Matrix R.
Example 1.
 3 2
Find a QR decomposition for the matrix A =  
 4 5
Soln.
 3  2
Let v1 =   and v2 =  .
 4  5
|| v1 || = 32  42 = 5
 3 
1 1  3
w1 = v1 =   =  5 
|| v1 || 5  4  4 
 5
Let y2 = v2 - <w1, v2> w1
<w1, v2> = (3/5)× (2) + (4/5)× (5) = 6/5 + 20/5 = 26/5.
 3   78   28 
 2 26  5   2  25   25 
y2 =   - =  - =
 5 5  4   5  104   21 
 5  25   25 
2 2
 28   21  784  441 35 7
|| y2|| =       
 25   25  25 25 5
49
1  28   4 
5 25   5
w2 = y 2 =  .
|| y2 || 7  21   3 
 25   5 
 3 4 
5 5
Thus, Q =  .
 4 3 
 5 5 
Now, we find R.
Let v1 = w1 + w2.
Then, we get
 3   4 
 3  5 +   5
 = 
 4 4   3 
 5  5 
3 4
3=  -   15 = 3 - 4 ------------(1)
5 5
4 3
4=  +   20 = 4 + 3 ------------(2)
5 5
(2) × 4  80 = 16 + 12

(1) × 3  45 = 9 - 12
----------------------------------
125 = 25   = 5
(1)  4 = 15 – 3 = 15 – 3(5) = 0   = 0.
 5
First column of R is  
 0
Let v2 = w1 + w2.
Then, we get
 3   4 
 2  5 5
 =  + 
 5 4   3 
 5  5 
50
3 4
⇒2= α - β ⇒ 10 = 3α - 4β ------------(3)
5 5
4 3
5= α + β ⇒ 25 = 4α + 3β ------------(4)
5 5
(4) × 4 ⇒ 100 = 16α + 12β
(3) × 3 ⇒ 30 = 9α - 12β
_________________
130 26
130 = 25α α=  .
25 5
 26   26  78  50 28 7
(3)  10 = 3  – 4  4 = 3  -10 =  = .
 5   5  5 5 5
 26 
5
The second column of R is  .
 7 
 5
 5 26 
5
Thus, R =  .
 0 7 
 5 
51

Matrix Theory

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Matrix Theory

Загружено:

Авторское право:

Доступные форматы

Lecture Notes in Applied Mathematics Dr. K.C.

Advanced Matrix Theory

1. Given any two elements v1 and v2 of V there exists a unique element of V,

These two operations must satisfy the following axioms:

Length or Norm of a vector

Characteristic Root (Eigenvalue)

Algebraic Multiplicity of an eigenvalue

2, …. n  R, 1v1 + 2v2 + 3v3 + ….. + nvn = 0  i = 0 for all i = 1, 2, 3, … ,

Geometric Multiplicity of an eigenvalue

Let v = (x, y, z)t.

Thus, the characteristic equation is

(i) - (iii)  3y1 = 0  y1 = 0

i.e. 2x2 + 6y2 – z2 – 4x2 = 0  - 2x2 + 6y2 – z2 = 0 -----(i)

i.e. 2x3 + 6y3 – z3 + 2x3 = 0  4x3 + 6y3 – z3 = 0 -----(i)

First, we find the characteristic equation.

(1 - ) (1 - )[(1 - )2 – 1] - 1 [(1- )2 – 1 ] = 0 [ when expanded along row 2

Now, we find the eigenvector corresponding to λ = 2.

Jordan canonical form

2. The number of blocks involving a particular eigenvalue of A equals the geometric

3. The largest block involving a particular eigenvalue of A equals the multiplicity of

2. Write Jordan Canonical form the the matrix given in example 6.

Thus, the Jordan Canonical form of the matrix A is

3. Write Jordan Canonical form the matrix given in example 7.

4. Write Jordan Canonical form the matrix given in example 8.

5. Write Jordan Canonical form the matrix given in example 9.

1. Show that the following matrix has eigenvalue λ = 0 with algebraic

Hint: Characteristic Equation is λ 3 – 0λ2 + (-12)λ - 16 = 0

Procedure for finding minimal polynomial for a given square matrix

Now, we evaluate  - 2 for  = A.

Now, we evaluate (λ - 2)2 for λ = A.

Now, we evaluate λ - 2 for λ = A.

Now, we evaluate λ - 2 for λ = A.

Jordan form for the example 13 (from minimal polynomial approach) is

Thus, x2 is a generalized eigenvector of type 2 with eigenvalue  = 2 for the matrix A.

Let x2 = (x , y, z)t be the required generalized eigenvector.

Then x2 has to satisfy

Note that (A – 2I)x2 ≠ 0.

So, Any value of x, y, z satisfying the conditions x = 2y and 3y + z ≠ 0 is the required

So, any value of w, x, y, z satisfying the conditions z = 0 and y ≠ 0 can be a required

SINGULAR VALUE DECOMPOSITION (SVD)

Thus, rank (A) = 3.

Note: The columns of U are a full orthonormal set of eigenvectors of the m × m

Among these eigenvalues the only positive eigenvalue is 30.

 2  2  (1)  (1) 2  (2)  (1)  1 2  4  (1)  (2) 

= (5 -  )[(5 - )(20 - ) – 100] + 5 [(-5)(20-)+100] + 10[50 – 10(5 - )]

We shall find orthonormal basis {w1, w2} from {v1, v2}

Moore-Penrose Generalized Inverse (Pseudo Inverse)

Least Square Approximation

∑xy = m ∑x2 + c∑x --------------(2)

(ii) ⇒ 65 = 30m + 10c

Thus, the required straight line is y = 1.7x + 1.4.

The Augmented matrix

3.48 z = - 0.62  z = - 0.178

9.4y + 4.8z = 1.6  9.4y = 1.6 – 4.8(-0.178)  y = 0.26

15x + 3y + z = 12  15x = 12 – 3(0.26) – (-0.178)  x = 0.75.

i.e. x = 0.75; y = 0.26; z = - 0. 178.

Procedure for QR decomposition

(2) × 4  80 = 16 + 12

Вам также может понравиться