0 views

Uploaded by akhil akki

- Gen inversion
- Matrices Sssss From Wiki
- Std11 Maths EM 1
- Essentials of Robust Control
- 03
- Nla Teach Dutta
- Linear Guest
- 8 Sistem Persamaan Linier SPL
- Operations on Vectors and Matrices
- Chapter_1-Slides(Sem 1 2013)
- Bca 1st Sem. Maths
- Lecture Notes Matrices
- matrix change basisbasis
- notes.pdf
- INTERNET TV
- MST-001
- Matrices and System of Equation
- mpdf (9).pdf
- _14_ELC4340_Spring13_Power_System_Matrices.doc
- HJS Smith

You are on page 1of 71

Murali K. Srinivasan

Jugal K. Verma

2

Contents

1.1 Matrix Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2 Gauss Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.3 Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.1 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.2 Linear transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.1 Length, Projection, and Angle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.2 Projections and Least Squares Approximations . . . . . . . . . . . . . . . . . . . . . 54

3.3 Determinant and Volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4.1 Algebraic and Geometric multiplicities . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.2 Spectral Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

3

4 CONTENTS

Chapter 1

Determinants

Convention 1.1.1. We shall write F to mean either the real numbers R or the complex numbers

C. Elements of F will be called scalars.

arranged in a rectangular array of m rows and n columns:

a11 a12 · · · a1n

a21 a22 · · · a2n

A = · · ··· · .

· · ··· ·

am1 am2 · · · amn

The entry in row i and column j is aij . We also write A = (aij ) to denote the entries. When all

the entries are in R we say that A is a real matrix. Similarly, we define complex matrices. For

example,

1 −1 3/2

5/2 6 11.2

is a 2 × 3 real matrix.

A 1 × n matrix [a1 a2 · · · an ] is called a row vector and a m × 1 matrix

b1

b2

·

·

bn

Matrix Addition

5

6 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

the (i, j) entries of M and N . For example,

2 1 0 1 0 3 3 1 3

+ = .

1 3 5 4 −3 1 5 0 6

Note that addition is defined only when both matrices have the same size.

Scalar multiplication

Let α ∈ F and let M be a m × n matrix. Then αM is a m × n matrix whose (i, j) entry is α ×

(i, j) entry of M . For example

0 1 0 2

2 2 3 = 4

6 .

2 1 4 2

Matrix multiplication

b1

·

First we define the product of a row vector a = [a1 . . . an ] and a column vector b =

· ,

bn

both with n components.

Define ab to be the scalar ni=1 ai bi .

P

The product of two matrices A = (aij ) and B = (bij ), denoted AB, is defined only when the

number of columns of A is equal to the number of rows of B. So let A be a m × n matrix and let

B be a n × p matrix. Let the row vectors of A be A1 , A2 , . . . , Am and let the column vectors of B

be B1 , B2 , . . . , Bp . We write

A1

A2

· , B = [B1 B2 · · · Bp ] .

A=

·

Am

by

n

X

mij = Ai Bj = aik bkj .

k=1

For example,

2 0

1 3 1 1 1 = 5 4 .

2 4 2 8 6

0 1

The usefulness and “meaning” of this definition will emerge as this course progresses. Meanwhile,

let us note other ways of thinking about matrix multiplication. First a definition. By a linear

combination of n × 1 column vectors v1 , v2 , . . . , vr we mean a column vector v of the form v =

α1 v1 + · · · + αr vr , where αi ∈ F for all i are called the coefficients. Similarly we define a linear

combination of row vectors.

Matrix times a column vector

1.1. MATRIX OPERATIONS 7

x1

·

Lemma 1.1.2. Let B = [B1 B2 · · · Bp ] be a n × p matrix with columns B1 , . . . , Bp . Let x =

·

xp

be a column vector with p components. Then

Bx = x1 B1 + x2 B2 + · · · + xp Bp .

Pp

b x

Ppj=1 1j j b1j xj

j=1 b2j xj p b2j xj p

X X

Bx = · =

· =

xj Bj ,

·

Pp ·

j=1 j=1

j=1 bnj xj bnj xj

as desired. So, Bx can be thought of as a linear combination of the columns of B, with column l

having coefficient xl . This way of thinking about Bx is very important.

Example 1.1.3. Let e1 , e2 , . . . , ep denote the standard column vectors with p components, i.e., ei

denotes the p × 1 column vector with 1 in component i and all other components 0. Then Bei = Bi ,

column i of B.

Let A = be a m × n with rows A1 , . . . , Am . Let y = [y1 · · · ym ] be a row vector with m

components. Then (why?)

yA = y1 A1 + y2 A2 + · · · + ym Am .

So, yA can be thought of as a linear combination of the rows of A, with row i having coefficient yi .

Columns and rows of product

Let A and B be as above. Then (why?)

A1 B

A2 B

AB = [AB1 AB2 · · · ABp ] =

· .

·

Am B

So, the jth column of AB is a linear combination of the columns of A, the coefficients coming from

the jth column Bj of B. For example,

2 0

1 3 1 5 4

1 1 = .

2 4 2 8 6

0 1

1 3 1 4

0 +1 +1 = .

2 4 2 6

8 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

Similarly, ith row Ai B of AB is a linear combination of the rows of B, the coefficients coming from

the ith row Ai of A.

Properties of Matrix Operations

Theorem 1.1.4. The following identities hold for matrix sum and product, whenever the sizes of

the matrices involved are compatible (for the stated operations).

(i) A(B + C) = AB + AC.

(ii) (P + Q)R = P R + QR.

(iii) A(BC) = (AB)C.

(iv) c(AB) = (cA)B = A(cB).

Proof. We prove item (iii) (leaving the others as exercises). Let A = (aij ) have p columns,

B = (bkl ) have p rows and q columns, and C = (crs ) have q rows. Then the entry in row i and

column s of A(BC) is

p

X

= a(i, m){entry in row m, column s of BC}

m=1

p q

( )

X X

= a(i, m) b(m, n)c(n, s)

m=1 n=1

q

( p )

X X

= a(i, m)b(m, n) c(n, s),

n=1 m=1

Matrix multiplication is not commutative. For example :

1 0 0 1 0 1

= ,

0 0 0 0 0 0

but

0 1 1 0 0 0

=

0 0 0 0 0 0

Definition 1.1.5. A matrix all of whose entries are zero is called the zero matrix. The entries

aii of a square matrix A = (aij ) are called the diagonal entries. If the only nonzero entries of a

square matrix A are the diagonal entries then A is called a diagonal matrix. An n × n diagonal

matrix whose diagonal entries are 1 is called the n × n identity matrix. It is denoted by In . A

square matrix A = (aij ) is called upper triangular if all the entries below the diagonal are zero,

i.e., aij = 0 for i > j. Similarly we define lower triangular matrices.

A square matrix A is called nilpotent if Ar = 0 for some r ≥ 1.

Example 1.1.6. Let A = (aij ) be an upper triangular n × n matrix with diagonal entries zero.

Then A is nilpotent. In fact An = 0.

Since column j of An is An ej , it is enough to show that An ej = 0 for j = 1, . . . , n. Denote

column j of A by Aj .

1.1. MATRIX OPERATIONS 9

Similarly

A3 e3 = A2 (Ae3 ) = AA3 = A2 (a13 e1 + a23 e2 ) = 0.

Continuing in this fashion we see that all columns of An are zero.

Inverse of a Matrix

Definition 1.1.7. Let A be an n×n matrix. If there is an n×n matrix B such that AB = In = BA

then we say A is invertible and B is the inverse of A. The inverse of A is denoted by A−1 .

Remark 1.1.8. (1) Inverse of a matrix is uniquely determined. Indeed, if B and C are inverses of

A then

B = BI = B(AC) = (BA)C = IC = C.

(2) If A and B are invertible n × n matrices, then AB is also invertible. Indeed,

(3) We will see later (in Chapter 3) that if there exists an n × n matix B for an n × n matrix A

such that AB = I or BA = I, then A is invertible. This fact fails for non-square matrices. For

example

1 1 1 2

[1 2] = [1] = I1 , but [1 2] = 6= I2 .

0 0 0 0

1 0 a b

(4) Inverse of a square matrix need not exist. For example, let A = . If is any

0 0 c d

2 × 2 matrix, then

1 0 a b a b

= 6= I2

0 0 c d 0 0

for any a, b, c, d.

Transpose of a Matrix

the matrix n × n matrix (bij ) such that bij = aji for all i, j.

Thus rows of A become columns of At and columns of A become rows of At . For example, if

2 1

2 0 1

A= then At = 0 0 .

1 0 1

1 1

(ii) For any invertible square matrix A, (A−1 )t = (At )−1 .

10 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

Proof. For any matrix C, let Cij denote its (i, j)th entry.

(i) Let A = (aij ), B = (bij ). Then, for all i, j,

X

= ajk bki

X

= (At )kj (B t )ik

X

= (B t )ik (At )kj

= (B t At )ij

(ii) Since AA−1 = I = A−1 A, we have (AA−1 )t = I = (A−1 A)t . By (i), (A−1 )t At = I =

At (A−1 )t . Thus (At )−1 = (A−1 )t .

if At = −A.

Lemma 1.1.12. (i) If A is a symmetric matrix then so is A−1 . (ii) Every square matrix A is a

sum of a symmetric and a skew symmetric matrix in a unique way.

(ii) Since

1 1

A = (A + At ) + (A − At ),

2 2

every matrix is a sum of a symmetric and a skew-symmetric matrix. To see the uniqueness, suppose

that P is a symmetric matrix and Q is a skew-symmetric matrix such that

A = P + Q.

We discuss a widely used method called the Gauss elimination method to solve a system of m linear

equations in n unknowns x1 , . . . , xn :

where the aij ’s and the bi ’s are known scalars in F. If each bi = 0 then the system above is called

a homogeneous system. Otherwise, we say it is inhomogeneous.

Set A = (aij ), b = (b1 , . . . , bm )t , and x = (x1 , . . . , xn )t . We can write the system above in the

matrix form

Ax = b.

The matrix A is called the coefficient matrix. By a solution, we mean any choice of the unknowns

x1 , . . . , xn which satisfies all the equations.

1.2. GAUSS ELIMINATION 11

F. Set U = EA and c = Eb. Then Ax = b has the same solutions as U x = c.

Ax = b.

The idea of Gauss elimination is the following:

(i) Find a suitable invertible E so that U is in “row echelon form” or “row canonical form”

(defined below).

(ii) All solutions to U x = c, when U is in row echelon form or row canonical form , can be

written down easily.

We first describe step (ii) and then step (i).

Definition 1.2.2. A m × n matrix M is said to be in row echelon form (ref ) if it satisfies the

following conditions:

(a) By a zero row of M we mean a row with all entries zero. Suppose M has k nonzero rows

and m − k zero rows. Then the last m − k rows of M are the zero rows.

(b) The first nonzero entry in a nonzero row is called a pivot. For i = 1, 2, . . . , k, suppose that

the pivot in row i occurs in column ji . Then we have j1 < j2 < · · · < jk . The columns {j1 , . . . , jk }

are called the set of pivotal columns of M . Columns {1, . . . , n} \ {j1 , . . . , jk } are the nonpivotal

or free columns.

the following conditions:

(a) By a zero row of M we mean a row with all entries zero. Suppose M has k nonzero rows

and m − k zero rows. Then the last m − k rows of M are the zero rows.

(b) The first nonzero entry in every nonzero row is 1. This entry is called a pivot. For

i = 1, 2, . . . , k, suppose that the pivot in row i occurs in column ji .

(c) We have j1 < j2 < · · · < jk . The columns {j1 , . . . , jk } are called the set of pivotal columns

of M . Columns {1, . . . , n} \ {j1 , . . . , jk } are the nonpivotal or free columns.

(d) The matrix formed by the first k rows and the k pivotal columns of M is the k × k identity

matrix, i.e., the only nonzero entry in a pivotal column is the pivot 1.

Note that a matrix in row canonical form is automatically in row echelon form. Also note that,

in both the definitions above, the number of pivots k is ≤ m, n.

0 1 a13 a14 0 a16 0 a18

0 0 0 0 1 a26 0 a28

,

0 0 0 0 0 0 1 a38

0 0 0 0 0 0 0 0

where the aij ’s are arbitrary scalars. It may be checked that U is in rcf with pivotal columns 2, 5, 7

12 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

0 a a13 a14 a15 a16 a17 a18

0 0 0 0 b a26 a27 a28

,

0 0 0 0 0 0 c a38

0 0 0 0 0 0 0 0

where a, b, c are nonzero scalars and the aij ’s are arbitrary scalars. It may be checked that R is in

ref with pivotal columns 2, 5, 7 and nonpivotal columns 1, 3, 4, 6, 8.

Example 1.2.5. Let U be the matrix from the example above. Let c = (c1 , c2 , c3 , c4 )t . We want

to write down all solutions to the system U x = c.

(i) If c4 6= 0 then clearly there is no solution.

(ii) Now assume that c4 = 0. Call the variables x2 , x5 , x7 pivotal and the variables x1 , x3 , x4 , x6 , x8

nonpivotal or free.

Give arbitrary values x1 = s, x3 = t, x4 = u, x6 = v, x8 = w to the free variables. These values

can be extended to values of the pivotal variables in one and only one way to get a solution to the

system U x = c:

x7 = c3 − a38 w

x5 = c2 − a26 v − a28 w

x2 = c1 − a13 t − a14 u − a16 v − a18 w

Thus (why?) the set of all solutions to U x = c can be written as

s

c1 − a13 t − a14 u − a16 v − a18 w

t

u

,

c2 − a26 v − a28 w

v

c3 − a38 w

w

where s, t, u, v, w are arbitrary scalars.

(iii) The column vector above can be written as

0 1 0 0 0 0

c1 0 −a13 −a14 −a16 −a18

0 0 1 0 0 0

0

+ s + t 0

0 1 0 0

+ u +v +w

c2

0

0

0

−a26

−a28

0 0 0 0 1 0

c3 0 0 0 0 −a38

0 0 0 0 0 1

Thus every solution to U x = c is of the form above, for arbitrary scalars s, t, u, v, w. Note that the

first vector in the expression above is the unique solution to U x = c that has all free variables zero

and that the other vectors (without the coefficients) are the unique solutions to U x = 0 that have

one free variable equal to 1 and the other free variables equal to zero.

1.2. GAUSS ELIMINATION 13

Example 1.2.6. Let R be the matrix from the example above. Let c = (c1 , c2 , c3 , c4 )t . We want

to write down all solutions to the system U x = c.

(i) If c4 6= 0 then clearly there is no solution.

(ii) Now assume that c4 = 0. Call the variables x2 , x5 , x7 pivotal and the variables x1 , x3 , x4 , x6 , x8

nonpivotal or free.

Give arbitrary values x1 = s, x3 = t, x4 = u, x6 = v, x8 = w to the free variables. These values

can be extended to values of the pivotal variables in one and only one way to get a solution to the

system Rx = c:

x5 = (c2 − a26 v − a27 x7 − a28 w)/b

x2 = (c1 − a13 t − a14 u − a15 x5 − a16 v − a17 x7 − a18 w)/a

The process above is called back substitution. Given arbitrary values for the free variables,

we first solve for the value of the largest pivotal variable, then using this value (and the values of

the free variables) we get the value of the second largest pivotal variable, and so on.

We extract the following Lemma from the examples above and its proof is left as an exercise.

Lemma 1.2.7. Let U be a m × n matrix in ref. Then the only solution to the homogeneous system

U x = 0 which is zero in all free variables is the zero solution.

Note that a matrix in rcf is also in ref and the lemma above also applies to such matrices.

Theorem 1.2.8. Let Ax = b, with A an m × n matrix. Let c be a solution of Ax = b and S the

set of all solutions of the associated homogeneous system Ax = 0. Then the set of all solutions to

Ax = b is

c + S = {c + v : v ∈ S}.

Conversely, let v ∈ S. Then A(c + v) = Ac + Av = b + 0 = b. Hence c + v is a solution to Ax = b.

The proof of the following important result is almost obvious and is left as an exercise.

Theorem 1.2.9. Let U be a m × n matrix in ref with k pivotal columns P = {j1 < j2 < · · · < jk }

and nonpivotal or free columns F = {1, . . . , n} \ P . Let c = (c1 , . . . , cm )t .

(i) The system U x = c has a solution iff ck+1 = · · · = cm = 0.

(ii) Assume ck+1 = · · · = cm = 0. Given arbitrary scalars yi , i ∈ F , there exist unique scalars

yi , i ∈ P such that y = (y1 , . . . , yn )t satisfies U y = c.

(iii) For i ∈ F , let si be the unique solution of U x = 0 which is zero in all free components

except component i, where it is 1. Then every solution of U x = 0 is of the form

X

ai si ,

i∈F

14 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

(iv) Let p be the unique solution of U x = c having all free variables zero. Then every solution

of U x = c is of the form X

p+ ai s i ,

i∈F

Example 1.2.10. In our previous two examples P = {2, 5, 7} and F = {1, 3, 4, 6, 8}. To make sure

the notation of the theorem is understood write down p and si , i = 1, 3, 4, 6, 8.

We now discuss the first step in Gauss elimination, namely, how to reduce a matrix to ref or

rcf. We define a set of elementary row operations to be performed on the equations of a system.

These operations transform a system of equations into another system with the same solution set.

Performing an elementary row operation on Ax = b is equivalent to replacing this system by the

system EAx = Eb, where E is an invertible elementary matrix.

Let eij denote the m × n matrix with 1 in the ith row and jth column and zero elsewhere. Any

matrix A = (aij ) of size m × n can be written as

m X

X n

A= aij eij .

i=1 j=1

For this reason eij ’s are called the matrix units. Let us see the effect of multiplying e13 with a

matrix A written in terms of row vectors :

0 0 1 ··· 0

.. .. .. — R1 — — R3 —

. . . .

..

—

R2 —

— 0 —

e13 A = ... ... ... .. — R3 — — 0 —

= .

.

.. .

.. .. .. .. — ..

— . — —

. . . .

— Rm — m×n — 0 —

0 0 0 · · · 0 m×m

— 0 —

..

— . —

— Rj —

ith row.

eij A =

— 0 —

.

— ..

—

— 0 —

— —

We now define three kinds of elementary row operations and elementary matrices. Consider the

system Ax = b, where A is m × n, b is m × 1, and x is a n × 1 unknown vector.

1.2. GAUSS ELIMINATION 15

(i) Elementary row operation of type I: For i 6= j and a scalar a, add a times equation j to

equation i in the system Ax = b.

What effect does this operation have on A and b? Consider the matrix

1 1

1 a

1

E=

1 or

.. = I + aeij , i 6= j.

.

..

. a

1 1

This matrix has 1’s on the diagonal and a scalar a as an off-diagonal entry. By the above observation

— R1 — — R1 — — 0 —

— R2 — — R2 — — Rj —

ith row

(I + aeij ) = + a

.. .. ..

. . .

— Rm — — Rm — — 0 —

— R1 —

..

.

= — Ri + aRj —

ith row

..

.

— Rm —

It is now clear that performing an elementary row operation of type I on the system Ax = b we

get the new system EAx = Eb.

Suppose we perform an elementary row operation of type I as above. Then perform the same

elementary row operation of type I but with the scalar a replaced by the scalar −a. It is clear that

we get back the original system Ax = b. It follows (why?) that E −1 = I − aeij .

(ii) Elementary row operation of type II: For i 6= j interchange equations i and j in the system

Ax = b.

What effect does this operation have on A and b?. Consider the matrix

1

1

..

.

0 1

F = = I + eij + eji − eii − ejj .

..

.

1 0

..

.

1

Premultiplication by this matrix has the effect of interchanging the ith and jth rows. Performing

this operation twice in succession gives back the original system. Thus F 2 = I.

16 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

(iii) Elementary row operation of type III: Multiply equation i in the system Ax = b by a

nonzero scalar c.

What effect does this operation have on A and b?. Consider the matrix

1

1

..

.

c

= I + (c − 1)eii , c 6= 0

G=

1

1

..

.

1

Premultiplication by G has the effect of multiplying the ith row by c. Do this operation twice

in succession, first time with the scalar c and the second time with scalar 1/c, yields the original

system back. It follows that G−1 = I + (c−1 − 1)eii .

The matrices E, F, G above are called elementary matrices of type I,II,III respectively. We

summarize the above discussion in the following result.

Theorem 1.2.11. Performing an elementary row operation (of a certain type) on the system

Ax = b is equivalent to premultiplying A and b by an elementary matrix E (of the same type),

yielding the system EAx = Eb.

Elementary matrices are invertible and the inverse of an elementary matrix is an elementary

matrix of the same type.

Since elementary matrices are invertible it follows that performing elementary row operations

does not change the solution set of the system. We now show how to reduce a matrix to row

reduced echelon form using a sequence of elementary row operations.

Theorem 1.2.12. Every matrix can be reduced to a matrix in rcf by a sequence of elementary row

operations.

Proof. We apply induction on the number of rows.If the matrix A is a row vector, the conclusion

is obvious. Now suppose that A is m × n, where m ≥ 2. If A = 0 then we are done. If A is not

the zero matrix then there is a nonzero column in A. Find the first nonzero column, say column

j1 , from the left. Interchange rows to move the first nonzero in column j1 to the top row. Now

multiply by a nonzero scalar to make this entry (in row 1 and column j1 ) 1. Now add suitable

multiples of the first row to the remaining rows so that all entries in column j1 , except the entry

in row 1, become zero. The resulting matrix looks like

0 ··· 0 1 ∗ ··· ∗

0 ··· 0 0 ∗ ··· ∗

· · · · ·

A1 =

·

· · · ·

· · · · ·

0 ··· 0 0 ∗ ··· ∗

1.2. GAUSS ELIMINATION 17

echelon form. So now the resulting matrix looks like

1 v

A2 =

D

where blank space consists of 0’s, v is a row vector with n−j1 components, and D is a (m−1)×(n−j1 )

matix in rcf. Let the pivotal columns of D be j2 < j3 < · · · < jk , where j1 < j2 . By subtracting

suitable multiples of rows 2, . . . , k of A2 from row 1 of A2 we can make the entries in columns

j2 , . . . , jk of row 1 equal to 0. The resulting matrix is in rcf.

Before giving examples of row operations we collect together some results on systems of linear

equations that follow from Gauss elimination.

Theorem 1.2.13. Let Ax = b, with A an m × n matrix.

(i) Suppose m < n. Then there is a nontrivial solution to the homogeneous system Ax = 0.

(ii) The number of solutions to Ax = b is either 0, 1, or ∞.

Proof. (i) Reduce A to rcf U by Gauss elimination. Since m < n there is atleast one free variable.

It follows that there is a nontrivial solution.

(ii) Reduce Ax = b to EAx = Eb using Gauss elimination, where U = EA is in rref. Put

c = Eb = (c1 , . . . , cm )t . Suppose U has k nonzero rows. There cases arise:

(a) atleast one of ck+1 , . . . , cm is nonzero: in this case there is no solution.

(b) ck+1 = · · · = cm = 0 and k = n: there is a unique solution (why?).

(c)ck+1 = · · · = cm = 0 and k < n: there are infinitely many solutions (why?).

No other cases are possible (why?). That completes the proof.

In the following examples an elementary row operation of type I is indicated by Ri + aRj , of

type II is indicated by Ri ↔ Rj , and of type III is indicated by aRi .

Example 1.2.14. Consider the system

2 1 1 x1 5

Ax = 4 −6 0 x2 = −2 = b.

−2 7 2 x3 9

2 1 1 5 R2 − 2R1 2 1 1 5 2 1 1 5

4 −6 0 −2 −→ 0 −8 −2 −12 R3 + R2 0 −8 −2 −12

−→

−2 7 2 9 R3 + R1 0 8 3 14 0 0 1 2

R1 − R3 2 1 0 3 2 0 0 2

R1 + (1/8)R2

−→ 0 −8 0 −8 0 −8 0 −8

−→

R2 + 2R3 0 0 1 2 0 0 1 2

(1/2)R1 1 0 0 1

−→ 0 1 0 1

(−1/8)R2 0 0 1 2

Since there are no free columns the problem has a unique solution given by x1 = x2 = 1 and x3 = 2.

18 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

x1

1 3 −2 0 2 0

x2

0

2 6 −5 −2 4 −3 x3 −1

Ax =

0

5 = b.

=

0 5 10 0 15

x4

2 6 0 8 4 18 x5 6

x6

1 3 −2 0 2 0 0 1 3 −2 0 2 0 0

2 6 −5 −2 4 −3 −1 R2 − 2R1 0 0 −1 −2 0 −3 −1

0 0

−→

5 10 0 15 5 0 0 5 10 0 15 5

R4 − 2R1

2 6 0 8 4 18 6 0 0 4 8 0 18 6

1 3 −2 0 2 0 0 1 3 −2 0 2 0 0

3R − 5R2

−R2 0 0 1 2 0 3 1 0 0 1 2 0 3 1

−→

−→ 0 0 5 10 0 15 5 0 0 0 0 0 0 0

R4 − 4R2

0 0 4 8 0 18 6 0 0 0 0 0 6 2

1 3 −2 0 2 0 0 1 3 −2 0 2 0 0

R3 ↔ R4 0 0

1 2 0 3 1 (1/6)R3 0 0

1 2 0 3 1

−→ 0 0 0 0 0 6 2 −→ 0 0 0 0 0 1 1/3

0 0 0 0 0 0 0 0 0 0 0 0 0 0

1 3 −2 0 2 0 0 1 3 0 4 2 0 0

R2 − 3R3 0 0 1 2 0 0 0 R1 + 2R2 0 0

1 2 0 0 0

−→ 0 0 0 0 0 1 1/3 −→ 0 0 0 0 0 1 1/3

0 0 0 0 0 0 0 0 0 0 0 0 0 0

−3 −4 −2

0

0 1 0 0

0 0 −2

+ t 0 ,

+ s +r

0 0 1 0

0 0 0 1

1/3 0 0 0

x1

1 3 −2 0 2 0

x2

0

2 6 −5 −2 4 −3

x3 −1

Ax =

0

=

6 = b.

0 5 10 0 15

x4

2 6 0 8 4 18 x5 6

x6

1.2. GAUSS ELIMINATION 19

1 3 −2 0 2 0 0 1 3 −2 0 2 0 0

2 6 −5 −2 4 −3 −1 2 R − 2R1

0 0 −1 −2 0 −3 −1

0 0

−→

5 10 0 15 6 0 0 5 10 0 15 6

R4 − 2R1

2 6 0 8 4 18 6 0 0 4 8 0 18 6

1 3 −2 0 2 0 0 1 3 −2 0 2 0 0

R3 − 5R2

−R2 0 0

1 2 0 3 1

0 0 1 2 0 3 1

−→

−→ 0 0 5 10 0 15 6 0 0 0 0 0 0 1

R4 − 4R2

0 0 4 8 0 18 6 0 0 0 0 0 6 2

1 3 −2 0 2 0 0 1 3 −2 0 2 0 0

R3 ↔ R4 0 0 1 2 0 3 1 (1/6)R3 0 0

1 2 0 3 1

−→ 0 0 0 0 0 6 2 −→ 0 0 0 0 0 1 1/3

0 0 0 0 0 0 1 0 0 0 0 0 0 1

1 3 −2 0 2 0 0 1 3 0 4 2 0 0

R2 − 3R3 0 0

1 2 0 0 0 R1 + 2R2 0 0

1 2 0 0 0

−→ 0 0 0 0 0 1 1/3 −→ 0 0 0 0 0 1 1/3

0 0 0 0 0 0 1 0 0 0 0 0 0 1

It follows that the system has no solution.

Lemma 1.2.17. Let A be a square matrix. Then the following are equivalent:

(b) A is a product of elementary matrices.

(c) A is invertible.

(d) The system Ax = 0 has only the trivial solution x = 0.

E1−1 . . . Ek−1 .

(d) ⇒ (a) First observe that a square matrix in rcf is either the identity matrix or its bottom row

is zero. If A can’t be reduced to I by elementary row operations then U = the rcf of A has a zero

row at the bottom. Hence U x = 0 has atmost n − 1 nontrivial equations. which have a nontrivial

solution. This contradicts (d).

This proposition provides us with an algorithm to calculate inverse of a matrix if it exists. If A

is invertible then there exist invertible matrices E1 , E2 , . . . , Ek such that Ek · · · E1 A = I. Multiply

by A−1 on both sides to get Ek · · · E1 I = A−1 .

20 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

Lemma 1.2.18. (Gauss-Jordan Algorithm) Let A be an invertible matrix. To compute A−1 , apply

elementary row operations to A to reduce it to an identity matrix. The same operations when

applied to I, produce A−1 .

Example 1.2.19. We find the inverse of the matrix

1 0 0

A = 1 1 0 .

1 1 1

1 0 0 1 0 0

[A | I] = 1 1 0 0 1 0 .

1 1 1 0 0 1

Now perform row operations to reduce the matrix A to I. In this process the identity matrix will

reduce to A−1 .

1 0 0 1 0 0 R2 − R1 1 0 0 1 0 0

[A | I] = 1 1 0 0 1 0 −→ 0 1 0 −1 1 0

1 1 1 0 0 1 R3 − R1 0 1 1 −1 0 1

1 0 0 1 0 0 1 0 0

R3 − R2 0 1 0 −1 1 0 . Hence A−1 = −1 1 0

−→

0 0 1 0 −1 1 0 −1 1

1.3 Determinants

In this section we study determinants of matrices. Recall the formula for determinants of k × k

matrices, for k = 1, 2, 3.

a b

det[a] = a, det = ad − bc

c d

a b c

and det d e f = aei − ahf − bdi + bgf + cdh − ceg.

g h i

Our approach to determinants of n × n is via their properties (rather than via an explicit formula

as above). It makes their study more elegant. Later, we will give a geometric interpretation of

determinant in terms of volume.

Let d be a function that associates a scalar d(A) ∈ F with every n × n matrix A over F. We use

the following notation. If the columns of A are A1 , A2 , . . . , An , we write d(A) = d(A1 , A2 , . . . , An ).

Definition 1.3.1. (i) d is called multilinear if for each k = 1, 2, . . . , n; scalars α, β and n × 1

column vectors A1 , . . . , Ak−1 , Ak+1 , . . . , An , B, C

α d(A1 , . . . , Ak−1 , B, Ak+1 , . . . , An ) + β d(A1 , . . . , Ak−1 , C, Ak+1 , . . . , An ).

1.3. DETERMINANTS 21

(iii) d is called normalized if d(I) = d(e1 , e2 , . . . , en ) = 1, where ei is the ith standard column

vector with 1 in the ith coordinate and 0’s elsewhere.

(iv) A normalized, alternating, and multillinear function d on n × n matrices is called a deter-

minant function of order n.

Our immediate objective is to show that there is only one determinant function of order n. This

fact is very useful in proving that certain formulas yield the determinant. We simply show that

the formula defines an alternating, multilinear and normalized function on the columns of n × n

matrices.

n × n matrices. Then

(a) If some Ak = 0 then d(A1 , A2 , . . . , An ) = 0.

(b) d(A1 , A2 , . . . , Ak , Ak+1 , . . . An ) = −d(A1 , A2 , . . . , Ak+1 , Ak , . . . , An ).

(c) d(A1 , A2 , . . . , Ai , . . . , Aj , . . . , An ) = −d(A1 , A2 , . . . , Aj , . . . , Ai , . . . , An ).

0 = d(A1 , A2 , . . . , B + C, B + C, . . . , An )

= d(A1 , A2 , . . . , B, B + C, . . . , An ) + d(A1 , A2 , . . . , C, B + C, . . . , An )

= d(A1 , A2 , . . . , B, C, . . . , An ) + d(A1 , A2 , . . . , C, B, . . . , An )

(c) Follows from (b).

Remark 1.3.3. Note that the properties (a), (b), (c) have been derived by properties of determinant

functions without having any formula at our disposal yet.

Computation of determinants

Example 1.3.4. We now derive the familiar formula for the determinant of 2×2 matrices. Suppose

d(A1 , A2 ) is an alternating multilinear normalized function on 2 × 2 matrices A = (A1 , A2 ). Then

x y

d = xu − yz.

z u

To derive this formula, write the first column as A = xe1 + ze2 and the second column as A2 =

22 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

= d(xe1 + ze2 , ye1 ) + d(xe1 + ze2 , ue2 )

= d(xe1 , ye1 ) + d(ze2 , ye1 )

+d(xe1 , ue2 ) + d(ze2 , ue2 )

= yzd(e2 , e1 ) + xud(e1 , e2 )

= (xu − yz)d(e1 , e2 )

= xu − yz.

Similarly, the formula for 3 × 3 determinants can also be derived as above. We leave this as an

exercise.

Lemma 1.3.5. Suppose f is a multilinear alternating function on n×n matrices and f (e1 , e2 , . . . , en ) =

0. Then f is identically zero.

X

f (A1 , . . . , An ) = ah(1)1 ah(2)2 · · · ah(n)n f (eh(1) , eh(2) , . . . , eh(n) ),

h

Since f is alternating we have (why?)

X

f (A1 , . . . , An ) = ah(1)1 ah(2)2 · · · ah(n)n f (eh(1) , eh(2) , . . . , eh(n) ),

h

where the sum is now over all 1 − 1 onto functions h : {1, 2, . . . , n} → {1, 2, . . . , n}.

By using part (c) of the lemma above we see that we can write

X

f (A1 , . . . , An ) = ±ah(1)1 ah(2)2 · · · ah(n)n f (e1 , e2 , . . . , en ),

h

where the sum is over all 1 − 1 onto functions h : {1, 2, . . . , n} → {1, 2, . . . , n}.

Thus f (A) = 0.

Existence and uniqueness of determinant function

tion of order n and d a determinant function of order n. Then for all n × n matrices A =

(A1 , A2 , . . . , An ),

f (A1 , A2 , . . . , An ) = d(A1 , A2 , . . . , An )f (e1 , e2 , . . . , en ).

In particular, if f is also a determinant function then f (A1 , A2 , . . . , An ) = d(A1 , A2 , . . . , An ).

1.3. DETERMINANTS 23

g(e1 , e2 , . . . , en ) = 0

We have proved uniqueness of determinant function of order n. It remains to show their exis-

tence.

Assume that we have shown existence of determinant function of order (n − 1) × (n − 1). The

determinant of an n×n matrix A can be computed in terms of certain (n−1)×(n−1) determinants.

Let Aij = the (n − 1) × (n − 1) matrix obtained from A by deleting the ith row and jth column

of A.

Suppose that the columns Aj and Aj+1 of A are equal. Then A1i have equal columns except

when i = j or i = j + 1. By induction f (A1i ) = 0 for i 6= j, j + 1. Thus

Since Aj = Aj+1 , a1j = a1j+1 and A1j = A1j+1 . Thus f (A) = 0. Therefore f (A1 , A2 , . . . , An ) is

alternating.

If A = (e1 , e2 , . . . , en ) then by induction

The formula in the lemma above above is called expansion by first row. Just like in the

lemma above we can also prove the following formula for expansion by row k. We leave its proof

as an exercise.

n

X

detA = (−1)k+j akj detAkj .

j=1

24 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

Theorem 1.3.10. (i) Let U be an upper triangular or a lower triangular matrix. Then detU =

product of diagonal entries of U .

(ii) Let E be an elementary matrix of the type I + aeij , for some i 6= j. Then detE = 1.

(iii) Let E be an elementary matrix of the type I + eij + eji − eii − ejj , for some i 6= j. Then

detE = −1.

(iv) Let E be an elementary matrix of the type I + (a − 1)eii , a 6= 0. Then detE = a.

Proof. (i) Let U = (uij ) be upper triangular. Arguing as in Lemma 3.5 we see that

X

detU = ±uh(1)1 uh(2)2 · · · uh(n)n ,

h

where the sum is over all 1 − 1 onto functions h : {1, 2, . . . , n} → {1, 2, . . . , n}. Since U is upper

triangular the only choice of h yeilding a nonzero term is the identity function (and this gives a

plus sign).

The proof for a lower triangular matrix is similar.

(ii) Follows from part (i).

(iii) E is obtained from the identity matrix by exchanging columns i and j. The result follows

since determinant is an alternating function.

(iv) Follows form part (i).

Determinant and Invertibility

Theorem 1.3.11. Let A, B be two n × n matrices. Then

det(AB) = detAdetB.

(AB)i = ABi .

f (B1 , . . . , Bk + αC, . . . , Bn ) = det(AB1 , . . . , A(Bk + αC), . . . , ABn )

= det(AB1 , . . . , ABk + αAC, . . . , ABn )

= det(AB1 , . . . , ABk , . . . , ABn )

+ det(AB1 , . . . , αAC, . . . , ABn )

= f (B1 , . . . , Bn ) + αf (B1 , . . . , C, . . . , Bn ).

1.3. DETERMINANTS 25

Therefore

= det(A1 , . . . , An )

= detA

Lemma 1.3.12. (i) If A is an invertible matrix then detA 6= 0 and

1

detA−1 = .

detA

(iii) Suppose A, B are square matrices with AB = I. Then A is invertible and B = A−1 .

(ii) Suppose A is not invertible. Then, by Chapter 2, there is a nontrivial column vector x such

that Ax = 0. So some column of A is a linear combination of other columns (i.e., excluding itself)

of A. It now follows from multilinearity and alternating properties that detA = 0.

(iii) Taking determinants we have detAdetB = 1. So detA 6= 0 and A is invertible. Now

B = (A−1 A)B = A−1 (AB) = A−1 .

Theorem 1.3.13. For any n × n matrix A,

detA = detAt .

Proof. Let B be the rcf of A. Then EA = B, where E is a product of elementary matrices. Since

inverses of elementary matrices are elementary matrices (of the same type) we can write

A = E1 · · · Ek B,

At = B t Ekt · · · E1t ,

Now the transpose of an elementary matrix is also an elementary matrix (of the same type) and

has the same determinant (by Theorem 1.3.10). Thus, by multiplicativity of determinant, we need

to show that det(B) = det(B t ).

Case (i) A is not invertible (i.e., det(A) = 0): In this case det(B) = 0 and the last row of B or the

last column of B t is 0. Thus det(B t ) = 0.

Case (ii) A is invertible: In this case B (and B t ) are both equal to the the identity matrix.

The lemma above shows that the determinant is also a normalized, alternating, and multilinear

functions of the rows of a square matrix and we have the following formula for the determinant,

called expansion by column k.

26 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

n

X

detA = (−1)k+i aik detAik .

i=1

Example 1.3.15. (Computation by Gauss–Elimination Method). This is one of the most efficient

ways to calculate determinant functions. Let A be an n × n matrix. Suppose

E = the n × n elementary matrix for the row operation Ai + cAj

F = the n × n elementary matrix for the row operation Ai ∼ Aj

G = the n × n elementary matrix for the row operation Ai ∼ cAi .

Suppose that U is the rcf of A. If c1 , c2 , . . . , cp are the multipliers used for the row operations

Ai ∼ cAi and r row exchanges have been used to get U from A then for any alternating multilinear

function d, d(A) = (−1)r c1 c2 . . . cp d(U ). To see this we simply note that

Suppose that u11 , u22 , . . . , unn are the diagonal entries of U then

Definition 1.3.16. Let A = (aij ) be an n × n matrix. The cofactor of aij , denoted by cofaij is

defined as

cofaij = (−1)i+j detAij .

cofA = (cofaij ).

(cofA)t , hence A is invertible.

detA

If i = j, it is easy to see that it is detA. When i 6= j consider the matrix B obtained by replacing ith

column of A by j th column of A. So B has a repeated column. The expansion by minors formula

for detB shows that detB = 0. The other equation A(cofA)t = (detA)I is proved similarly.

1.3. DETERMINANTS 27

a11 a12 · · · a1n x1 b1

a21 a22 · · · a2n x2 b2

=

.. .. ..

. . .

an1 an2 · · · ann xn bn

(aij ) is invertible. Let Cj be the matrix obtained from A by replacing j th column of A by b =

(b1 , b2 , . . . , bn )t . Then for j = 1, 2, . . . , n,

detCj

xj = .

detA

det(b, A2 , A3 , . . . , An ) = x1 detA (why?). So x1 = detC1 . Similarly for x2 , . . . , xn .

detA

28 CHAPTER 1. MATRICES, LINEAR EQUATIONS AND DETERMINANTS

Chapter 2

Transformations

Definition 2.1.1. A nonempty set V of objects (called elements or vectors) is called a vector

space over the scalars F if the following axioms are satisfied.

I. Closure axioms:

1. (closure under vector addition) For every pair of elements x, y ∈ V there is a unique element

x + y ∈ V called the sum of x and y.

2. (closure under scalar multiplication of vectors by elements of F) For every x ∈ V and every

scalar α ∈ F there is a unique element αx ∈ V called the product of α and x.

II. Axioms for vector addition:

3. (commutative law) x + y = y + x for all x, y ∈ V.

4. (associative law) x + (y + z) = (x + y) + z for all x, y, z ∈ V.

5. (existence of zero element) There exists a unique element 0 in V such that x + 0 = 0 + x = x

for all x ∈ V.

6. (existence of inverse or negatives) For x ∈ V there exists a unique element written as −x such

that x + (−x) = 0.

III. Axioms for scalar multiplication

7. (associativity) For all α, β ∈ F, x ∈ V,

α(βx) = (αβ)x.

8. (distributive law for addition in V ) For all x, y ∈ V and α ∈ F,

α(x + y) = αx + αy.

9. (distributive law for addition in F) For all α, β ∈ F and x ∈ V,

(α + β)x = αx + βx

29

30 CHAPTER 2. VECTOR SPACES AND LINEAR TRANSFORMATIONS

1x = x.

Remark 2.1.2. When F = R we say that V is a real vector space. If we replace real numbers in

the above definition by complex numbers then we get the definition of a complex vector space.

In the examples below we leave the verification of the vector addition and scalar multiplication

axioms as exercises.

and scalar multiplication. This gives a real vector space.

multiplication. This gives a complex vector space.

multiplication. This gives a real vector space.

addition and multiplication of a row vector by a real number as scalar multiplication. This

gives a real vector space. We can similarly define a real vector space of column vectors with

n real components. Depending on the context Rn could refer to either row vectors or column

vectors with n real components.

addition and multiplication of a row vector by a complex number as scalar multiplication.

This gives a complex vector space. We can similarly define a complex vector space of column

vectors with n complex components. Depending on the context Rn could refer to either row

vectors or column vectors with n complex components.

6. Let a < b be real numbers and set V = {f : [a, b] −→ R}, F = R. If f, g ∈ V then we set

(f + g)(x) = f (x) + g(x) for all x ∈ [a, b]. If a ∈ R and f ∈ V then (af )(x) = af (x) for all

x ∈ [a, b]. This gives a real vector space. Here V is also denoted by R[a,b] .

real vector space under usual addition of polynomials and multiplication of polynomials with

real numbers.

8. C[a, b] = {f : [a, b] −→ R|f is continuous on [a, b]} is a real vector space under addition and

scalar multiplication defined in item 6 above.

9. V = {f : [a, b] −→ R|f is differentiable at x ∈ [a, b], x fixed } is a real vector space under the

operations described in item 6 above.

00 0

10. The set of all solutions to the differential equation y + ay + by = 0 where a, b ∈ R form a

real vector space. More generally, in this example we can take a = a(x), b = b(x) suitable

functions of x.

2.1. VECTOR SPACES 31

11. Let V = Mm×n (R) denote the set of all m × n matrices with real entries. Then V is a real

vector space under usual matrix addition and multiplication of a matrix by a real number.

The above examples indicate that the notion of a vector space is quite general. A result proved

for vector spaces will simultaneously apply to all the above different examples.

Subspace of a Vector Space

Definition 2.1.4. Let V be a vector space over F. A nonempty subset W of V is called a subspace

of V if

(i) 0 ∈ W .

(ii) u, v ∈ W implies u + v ∈ W .

(iii) u ∈ W, α ∈ F implies αu ∈ W .

Linear span

LetP V be a vector space over F. Let x1 , . . . , xn be vectors in V and let c1 , . . . , cn ∈ F. The

vector ni=1 ci xi ∈ V is called a linear combination of xi ’s and ci is called the coefficient of xi

in this linear combination.

Definition 2.1.5. Let S be a subset of a vector space V over F. The linear span of S is the

subset of all vectors in V expressible as linear combinations of finite subsets of S, i.e.,

( n )

X

L(S) = ci xi |n ≥ 0, x1 , x2 , . . . , xn ∈ S and c1 , c2 , . . . , cn ∈ F

i=1

The empty sum of vectors is the zero vector. Thus L(∅) = {0}. We say that L(S) is spanned by

S.

Proof. Note that L(S) is a subspace (why?). Now, if S ⊂ W ⊂ V and W is a subspace of V then

by L(S) ⊂ W (why?). The result follows.

C1 , . . . , Cn . The row space of A, denoted R(A), is the subspace of Fn spanned by the rows of

A. The column space of A, denoted C(A), is the subspace of Fm spanned by the columns of

A. The null space of A, denoted N (A), is defined by

N (A) = {x ∈ Fn : Ax = 0}.

2. Different sets may span the same subspace. For example L({e1 , e2 }) = L({e1 , e2 , e1 + e2 }) =

R2 . The vector space Pn (R) is spanned by {1, t, t2 , . . . , tn } and also by {1, (1 + t), . . . , (1 + t)n }

(why?).

32 CHAPTER 2. VECTOR SPACES AND LINEAR TRANSFORMATIONS

We have introduced the notion of linear span of a subset S of a vector space. This raises some

natural questions:

(i) Which spaces can be spanned by finite number of elements ?

(ii) If a vector space V = L(S) for a finite subset S of V then what is the size of smallest such

S?

To answer these questions we introduce the notions of linear dependence and independence,

basis and dimension of a vector space.

Linear independence

Definition 2.1.8. Let V be a vector space. A subset S ⊂ V is called linearly dependent (L.D.)

if there exist distinct elements v1 , v2 , . . . , vn ∈ S (for some n ≥ 1) and scalars α1 , α2 , . . . , αn not

all zero such that

α1 v1 + α2 v2 + . . . + αn vn = 0

A set S is called linearly independent (L.I.) if it is not linearly dependent, i.e., for all n ≥ 1

and for all distinct v1 , v2 , . . . , vn ∈ S and scalars α1 , α2 , . . . , αn

α1 v1 + α2 v2 + . . . + αn vn = 0 implies αi = 0, for all i.

Elements of a linearly independent set are called linearly independent. Note that the empty

set is linearly independent.

Remark 2.1.9. (i) Any subset of V containing a linearly dependent set is linearly dependent.

(ii) Any subset of a linearly independent set in V is linearly independent.

Example 2.1.10. (i) If a set S contains the zero vector 0 then S is dependent since 1.0 = 0.

(ii) Consider the vector space Rn and let S = {e1 , e2 , . . . , en }. Then S is linearly independent.

Indeed, if α1 e1 + α2 e2 + . . . + αn en = 0 for some scalars α1 , α2 , . . . , αn then (α1 , α2 , . . . , αn ) = 0.

Thus each αi = 0. Hence S is linearly independent.

(iii) Let V be the vector space of all continuous functions from R to R. Let S = {1, cos2 t, sin2 t}.

Then the relation cos2 t + sin2 t − 1 = 0 shows that S is linearly dependent.

(iv) Let α1 < α2 < . . . < αn be real numbers. Let V = {f : R −→ R|f is continuous }. Consider

the set S = {eα1 x , eα2 x , . . . , eαn x }. We show that S is linearly independent by induction on n. Let

n = 1 and βeα1 x = 0. Since eα1 x 6= 0 for any x, we get β = 0. Now assume that the assertion is

true for n − 1 and

β1 eα1 x + . . . + βn eαn x = 0.

Then β1 e(α1 −αn )x + . . . + βn e(αn −αn )x = 0

Let x −→ ∞ to get βn = 0. Now apply induction hypothesis to get β1 = . . . = βn−1 = 0.

(v) Let P denote the vector space of all polynomials p(t) with real coefficients. Then the set

S = {1, t, t2 , . . .} is linearly independent. Suppose that 0 ≤ n1 < n2 < . . . < nr and

α1 tn1 + α2 tn2 + . . . + αr tnr = 0

for certain real numbers α1 , α2 , . . . , αr . Differentiate n1 times to get α1 = 0. Continuing this way

we see that all α1 , α2 , . . . , αr are zero.

2.1. VECTOR SPACES 33

Bases and dimension are two important notions in the study of vector spaces. A vector space

may be realized as linear span of several sets of different sizes. We study properties of the smallest

sets whose linear span is a given vector space.

Definition 2.1.11. A subset S of a vector space V is called a basis of V if elements of S are

independent and V = L(S). A vector space V possessing a finite basis is called finite dimensional.

Otherwise V is called infinite dimensional.

Exercise 2.1.12. Let {v1 , . . . , vn } be a basis of a finite dimensional vector space V . Show that

every v ∈ V can be uniquely expressed as v = a1 v1 + · · · + an vn , for scalars a1 , . . . , an .

We show that all bases of a finite dimensional vector space have same cardinality (i.e., they

contain the same number of elements) For this we prove the following result.

Lemma 2.1.13. Let S = {v1 , v2 , . . . , vk } be a subset of a vector space V. Then any k + 1 elements

in L(S) are linearly dependent.

(first proof ) Suppose T = {w1 , . . . , wn } are linearly independent vectors in L(S). We shall

show that n ≤ k. This will prove the result.

We shall construct a sequence of sets

S = S0 , S1 , . . . , Sn

such that

(i) each Si spans L(S), i = 0, 1, . . . , n.

(ii) |Si | = k, i = 0, 1, . . . , n.

(iii) {w1 , . . . , wi } ⊆ Si , i = 0, 1, . . . , n.

We shall produce this sequence of sets inductively, the base case i = 0 being clear. Now suppose

we have sets S0 , . . . , Sj satisfying (i), (ii), (iii) above, for some j < n.

Since Sj spans L(S) we can write

X

wj+1 = cs s,

s∈Sj

for some scalars cs , s ∈ S. Since w1 , . . . , wj+1 are linearly idependent there exists t ∈ Sj −

{w1 , . . . , wj } with ct 6= 0 (why?). It follows that

1 X

t = (wj+1 − cs s)

ct

s∈Sj −{t}

and hence the set (Sj − {t}) ∪ {wj+1 } satisfies conditions (i), (ii), and (iii) above for i = j + 1. That

completes the proof.

(second proof ) Let T = {u1 , . . . , uk+1 } ⊆ L(S). Write

k

X

ui = aij vj , i = 1, . . . , k + 1.

j=1

34 CHAPTER 2. VECTOR SPACES AND LINEAR TRANSFORMATIONS

Since A has more rows than columns there exists (why?) a nonzero row vector c = [c1 , . . . , ck+1 ]

such that cA = 0, i.e., for j = 1, . . . k

k+1

X

ci aij = 0.

i=1

We now have

k+1

X k+1

X Xk

ci ui = ci ( aij vj )

i=1 i=1 j=1

k X

X k+1

= ( ci aij )vj

j=1 i=1

= 0,

Theorem 2.1.14. Any two bases of a finite dimensional vector space have same number of ele-

ments.

Proof. Suppose S and T are bases of a finite dimensional vector space V. Suppose |S| < |T |. Since

T ⊂ L(S) = V, T is linearly dependent. This is a contradiction.

called the dimension of V. It is denoted by dim V.

(2) Let A be a n × n matrix. Then the columns of A form a basis of Fn iff A is invertible.

(why?)

(3) Pn (R) = {a0 + a1 t + . . . + an tn | a0 , a1 , . . . , an ∈ R} is spanned by S = {1, t, t2 , . . . , tn }. Since

S is independent, dim Pn (R) = n + 1.

(4) Let Mm×n (F) denote the vector space of all m × n matrices with entries in F. Let eij

denote

P theP m × n matrix with 1 in (i, j) position and 0 elsewhere. If A = (aij ) ∈ Mm×n (F) then

A= m i=1

n

j=1 aij eij . It is easy to see that the mn matrices Eij are linearly independent. Hence

Mm×n (F) is an mn−dimensional vector space.

(5) The space of solutions of the differential equation

00 0

y − 2y − 3y = 0

has dimension 2. A basis is {e−x , e3x }. Every solution is a linear combination of the solutions e−x

and e3x .

Exercise 2.1.17. What is the dimension of Mn×n (C) as a real vector space?

Lemma 2.1.18. Suppose V is a finite dimensional vector space. Let S be a linearly independent

subset of V . Then S can be enlarged to a basis of V .

2.1. VECTOR SPACES 35

Proof. Suppose that dim V = n and S has less than n elements. Let v ∈ V \ L(S). Then S ∪ {v}

is a linearly independent subset of V (why?). Continuing this way we can enlarge S to a basis of

V.

Gauss elimination, row space, and column space

Lemma 2.1.19. Let A be a m × n matrix over F and E a nonsingular m × m matrix over F. Then

(a) R(A) = R(EA). Hence dim R(A) = dim R(EA).

(b) Let 1 ≤ i1 < i2 < · · · < ik ≤ n. Columns {i1 , . . . , ik } of A are linearly independent if and

only if columns {i1 , . . . , ik } of EA are linearly independent. Hence dim C(A) = dim C(EA).

Proof. (a) R(EA) ⊆ R(A) since every row of EA is a linear combination of the rows of A.

Similarly,

R(A) = R(E −1 (EA)) ⊆ R(EA).

α1 (EA)i1 + α2 (EA)i2 + · · · + αk (EA)ik = 0

iff E(α1 Ai1 + α2 Ai2 + · · · + αk Aik ) = 0

iff −1

E (E(α1 Ai1 + α2 Ai2 + · · · + αk Aik )) = 0

iff α1 Ai1 + α2 Ai2 + · · · + αk Aik = 0

iff α1 = · · · = αk = 0.

Thus columns {i1 , . . . , ik } of EA are linearly independent. The proof of the converse is similar.

Theorem 2.1.20. Let A be a m × n matrix. Then dim R(A) = dim C(A).

(first proof ) Let r be dim C(A). Then there are m × 1 column vectors v1 , v2 , . . . , vr that form

a basis for the column space of A. Form a m × r matrix C with columns v1 , . . . , vr .

For each 1 ≤ j ≤ n, there exists (why?) a r × 1 column vector uj such that the jth column of

A is equal to Cuj .

Form a r × n matrix B with columns u1 , . . . , un . Then (why?) A = CB.

Now the row space of A is contained in the row space of B and so dim R(A) is ≤ r = dim C(A).

Applying this argument to the transpose of A shows that dim C(A) is ≤ dim R(A).

(second proof ) Apply row operations to reduce A to the rcf U . Thus A = EU , where E is a

product of nonsingular elementary matrices. Suppose the first k rows of U are nonzero.Thus U has

k pivotal columns.

Then (why?) the first k rows of U are a basis of R(A). Let j1 , . . . , jk be the pivotal columns of

U . Then (why?) columns j1 , . . . , jk of A form a basis of C(A).

Example 2.1.21. Let A be a 4 × 6 matrix whose row echelon form is

1 2 3 4 5 6

0 0 0 1 7 8

U = 0 0 0 0

0 1

0 0 0 0 0 0

36 CHAPTER 2. VECTOR SPACES AND LINEAR TRANSFORMATIONS

Columns 1,4,6 of A form a basis of C(A) and the first 3 rows of U form a basis of R(A).

Definition 2.1.22. The rank of an m × n matrix A, denoted by r(A) or rank (A) is dim R(A) =

dim C(A). The nullity of A is the dimension of the nullspace N (A) of A.

Theorem 2.1.23. Let A be an m × n matrix. Then

rank A + nullity A = n.

Proof. Let k = r(A). Reduce A to rcf (or even ref) U using elementary row operations. Then U

has k nonzero rows and k pivotal columns. We need to show that dim N (A) = dim N (U ) = n − k.

Let j1 , . . . , jk be the indices of the pivotal columns of U . Set P = {j1 , . . . , jk } and F =

{1, 2, . . . , n} \ P , so |F | = n − k. Recall from Chapter 2 the following:

(i) Given arbitrary scalars xi for i ∈ F , there are unique scalars xi for i ∈ P such that

x = (x1 , . . . , xn )t satisfying U x = 0.

(ii) Given i ∈ F , there is a unique si = (x1 , . . . , xn ) satisfying U si = 0, xi = 1, and xj = 0, for

all j ∈ F − {i}.

Then si , i ∈ F forms a basis of N (A) (why?).

Fundamental Theorem for systems of linear equations

Theorem 2.1.24. Consider the following system of m linear equations in n unknowns x1 , x2 , . . . , xn :

a11 a12 · · · a1n x1 b1

a21 a22 · · · a2n x2 b2

= .. or Ax = b.

.. ..

. . .

am1 am2 · · · amn xn bm

(3) If r(A) = r([A | b]) = r < n then Ax = b has infinitely many solutions.

a2 , . . . , xn = an . Then

b = a1 C1 + a2 C2 + · · · + an Cn .

Hence b ∈ C(A) so A and [A | b] have same column space. Thus they have equal rank. Conversely if

r(A) = r([A | b]), then b ∈ C(A). Hence b = d1 C1 + · · · + dn Cn for some scalars d1 , d2 , . . . , dn . Then

d1

d2

d1 C1 + · · · + dn Cn = A . = b.

..

dn

Hence x1 = d1 , . . . , xn = dn is a solution.

2.2. LINEAR TRANSFORMATIONS 37

(2) Let r(A) = r([A | b]) = n. Then by the rank-nullity theorem, nullity (A) = 0. Hence Ax = 0

has a unique solution, namely x1 = · · · = xn = 0. If Ax = b = Ay then A(x − y) = 0. Hence

x − y = 0. Thus x = y.

(3) Suppose r(A) = r([A | b]) = r < n. Then n − r = dim N (A) > 0. Thus Ax = 0 has infinitely

many solutions. Let c ∈ Fn and Ac = b. Then we have seen before that all the solutions of Ax = b

are in the set c + N (A) = {c + x | Ax = 0}. Hence Ax = b has infinitely many solutions.

Rank in terms of determinants

We characterize rank in terms of minors of A. Recall that a minor of order r of A is a

submatrix of A consisting of r columns and r rows of A.

Theorem 2.1.25. An m × n matrix A has rank r ≥ 1 iff detM 6= 0 for some order r minor M of

A and detN = 0 for all order r + 1 minors N of A.

Proof. Let the rank of A be r ≥ 1. Then some r columns of A are linearly independent. Let B

be the m × r matrix consisting of these r columns of A. Then rank(B) = r and thus some r rows

of B will be linearly independent. Let C be the r × r matrix consisting of these r rows of B. Then

det(C) 6= 0 (why?).

Let N be a (r + 1) × (r + 1) minor of A. Without loss of generality we may take N to consist of

the first r + 1 rows and columns of A. Suppose det(N ) 6= 0. Then the r + 1 rows of N , and hence

the first r + 1 rows of A, are linearly independent, a contradiction.

The converse is left as an exercise.

Let A be an m × n matrix with real entries. Then A “acts” on the n-dimensional space Rn by left

multiplication : If v ∈ Rn then Av ∈ Rm .

In other words, A defines a function

TA : Rn −→ Rm , TA (v) = Av.

(ii) TA (cv) = cTA (v)

where c ∈ R and v, w ∈ Rn . We say that TA respects the two operations in the vector space Rn . In

this section we study such maps between vector spaces.

function satisfying

T (v + w) = T (v) + T (w) and T (cv) = cT (v)

where v, w ∈ V and c ∈ F.

38 CHAPTER 2. VECTOR SPACES AND LINEAR TRANSFORMATIONS

Example 2.2.3.

(1) Let c ∈ R, V = W = R2 . Define T : R2 −→ R2 by

x c 0 x cx

T = = .

y 0 c y cy

T stretches each vector v in R2 to cv. Hence

T (v + w) = c(v + w) = cv + cw = T (v) + T (w)

T (dv) = c(dv) = d(cv) = dT (v).

Hence T is a linear transformation.

(2) Rotation

Fix θ and define T : R2 −→ R2 by

x cos θ − sin θ x x cos θ − y sin θ

T = = .

y sin θ cos θ y x sin θ + y cos θ

Then T (e1 ) = (cos θ, sin θ)t and T (e2 ) = (− sin θ, cos θ)t . Thus T rotates the whole space by θ.

(Draw a picture to convince yourself of this. Another way is to identify the vector (x, y)t with the

complex number z = x + iy. Then we can write T (z) = zeiθ ).

(3) Let D be the vector space of differentiable functions f : R −→ R such that f (n) exists for all n.

Define D : D −→ D by

0

D(f ) = f .

0 0

Then D(af + bg) = af + bg = aD(f ) + bD(g). Hence D is a linear transformation.

(4) Define I : D −→ D by Z x

I(f )(x) = f (t) dt

0

By properties of integration, I is a linear transformation.

00 0

y − 3y + 2y = 0.

00

Let D : D −→ D be the linear transformation defined as above. Then D ◦ D(y) = y . Let I be the

identity map I(y) = y. Then the differential equation can be written as

(D2 − 3D + 2I)(y) = 0.

It can be shown that ex and e2x are solutions of the differential equation. Let T = D2 − 3D + 2I.

Then for any α, β ∈ R

T (αex + βe2x ) = αT (ex ) + βT (e2x ) = 0

(7) Let V = Mn×n (F) be the vector space of all n × n matrices over F. Fix A ∈ V . The map

T : V → V given by T (N ) = AN is linear (why?).

2.2. LINEAR TRANSFORMATIONS 39

Let T : V → W be a linear transformation of vector spaces. There are two important subspaces

associated with T.

• Image of T = Im (T ) = {T (v) | v ∈ V }.

then T (αv + βw) = αT (v) + βT (w) = 0. Hence αv + βw ∈ N (T ). Thus N (T ) is a subsapce of V.

The dimension of N (T ) is called the nullity of T and it is denoted by nullity (T ). Suppose that

v, w ∈ V. Then

αT (v) + βT (w) = T (αv + βw).

Thus Im (T ) is a subspace of W. The dimension of Im T, denoted by rank(T ), is called the rank of

T.

Lemma 2.2.4. Let T : V → W be a linear map of vector spaces. Then T is 1-1 if and only if

N (T ) = {0}.

(only if) T (v) = 0 = T (0) implies v = 0.

Lemma 2.2.5. Let V, W be vector spaces. Assume V is finite dimensional with (v1 , . . . , vn ) as an

ordered basis. Let (w1 , . . . , wn ) be an arbitrary sequence of vectors in W . Then there is a unique

linear map T : V → W with T (vi ) = wi , for all i = 1, . . . , n.

Then T (v) = a1 T (v1 ) + · · · + an T (vn ) = a1 w1 + · · · + an wn . So T is determined by (w1 , . . . , wn ).

(existence) Define T as follows. Given v ∈ V write (uniquely) v = a1 v1 + · · · + an vn , for scalars

ai . Define T (v) = a1 w1 + · · · + an wn . Show that T is linear (exercise).

Theorem 2.2.6 (The Rank-Nullity Theorem). Let T : V → W be a linear transformation of vector

spaces where V is finite dimensional. Then

rank(T ) + nullity (T ) = dim V.

basis C = {v1 , v2 , . . . , vl , w1 , w2 , . . . , wn−l } of V. We show that

D = {T (w1 ), T (w2 ), . . . , T (wn−l )}

is a basis of Im (T ). Any v ∈ V can be expressed uniquely as

v = α1 v1 + α2 v2 + · · · + αl vl + β1 w1 + · · · + βn−l wn−l .

Hence

T (v) = α1 T (v1 ) + · · · + αl T (vl ) + β1 T (w1 ) + · · · + βn−l T (wn−l )

= β1 T (w1 ) + · · · + βn−l T (wn−l ).

40 CHAPTER 2. VECTOR SPACES AND LINEAR TRANSFORMATIONS

Then

T (β1 w1 + · · · + βn−l wn−l ) = 0.

α1 v1 + α2 v2 + · · · + αl vl = β1 w1 + β2 w2 + · · · + βn−l wn−l .

0. Hence D is a basis of Im T. Thus

In a later exercise in this section we ask you to derive the rank-nullity theorem for matrices

from the result above.

Coordinate vectors

Let V be a finite dimensional vector space (fdvs) over F. By an ordered basis of V we mean a

sequence (v1 , v2 , . . . , vn ) of distinct vectors of V such that the set {v1 , . . . , vn } is a basis. Let u ∈ V .

Write uniquely (why?)

u = a1 v1 + a2 v2 + · · · + an vn , ai ∈ F.

Define the coordinate vector of u with respect to (wrt) the ordered basis B by

a1

a2

[u]B =

. .

.

an

Suppose C = (u1 , . . . , un ) is another ordered basis of V . Given u ∈ V , what is the relation between

[u]B and [u]C .

Define MBC , the transition matrix from C to B, to be the n × n matrix whose jth column

is [uj ]B :

[u]B = M [u]C .

2.2. LINEAR TRANSFORMATIONS 41

Proof. Let

a1

a2

[u]C =

. .

.

an

Then u = a1 u1 + a2 u2 + · · · + an un and we have

[u]B = [a1 u1 + · · · + an un ]B

= a1 [u1 ]B + · · · + an [un ]B

a1

a2

= [[u1 ]B [u2 ]B · · · [un ]B ]

.

.

an

= M [u]C .

1 0 0 1 0 0

v1 = 1 , v 2 = 1 , v 3 = 0 , u 1 = 0 , u 2 = 1 , u 3 = 0 .

1 1 1 0 0 1

1 0 0

M = MBC = −1 1 0 .

0 −1 1

2 2

Let u = 3 . So (why?) [u]C = 3 .

4 4

Then

1 0 0 2 2

[u]B = −1 1 0 3 = 1 .

0 −1 1 4 1

Check that

2 1 0 0

3 = 2 1 + 1 + 0 .

4 1 1 1

Lemma 2.2.9. Let V be a fdvs and B and C be two ordered bases. Then

42 CHAPTER 2. VECTOR SPACES AND LINEAR TRANSFORMATIONS

We have, for all u ∈ V ,

[u]B = N [u]C , [u]C = M [u]B .

It follows that, for all u ∈ V ,

[u]C = M [u]B = M N [u]C

Thus (why?) M N = N M = I.

Example 2.2.10. Let M be the (n + 1) × (n + 1) matrix, with rows and columns indexed by

{0, 1, . . . , n}, and with entry in row i and column j, 0 ≤ i, j ≤ n, given by ji . We show that M is

invertible and find the inverse explicitly.

Consider the vector space Pn (R) of real polynomials of degree ≤ n. Then B = (1, x, x2 , . . . , xn )

and C = (1, x − 1, (x − 1)2 , . . . , (x − 1)n ) are both ordered bases (why?).

We claim that M = MCB . To see this note the following computaion. For 0 ≤ j ≤ n we have

xj = (1 + (x − 1))j

j

X j

= (x − 1)i

i

i=0

n

X j

= (x − 1)i ,

i

i=0

j

where in the last step we have used the fact that i = 0 for i > j.

Thus M −1 = MBC and its entries are given by the following computation. For 0 ≤ j ≤ n we

have

j

j−i j

X

j

(x − 1) = (−1) xi

i

i=0

n

j−i j

X

= (−1) xi

i

i=0

j

Thus, the entry in row i and column j of M −1 is (−1)j−i

i .

Let V and W be finite dimensional vector spaces with dim V = n and dim W = m. Suppose

E = (e1 , e2 , . . . , en ) is an ordered basis for V and F = (f1 , f2 , . . . , fm ) is an ordered basis for W .

Let T : V −→ W be a linear transformation. We define MFE (T ), the matrix of T with respect

to the ordered bases E and F , to be the m × n matrix whose jth column is [T (ej )]F :

2.2. LINEAR TRANSFORMATIONS 43

Exercise 2.2.11. Let A be a m × n matrix over F and consider the linear map TA : Fn → Fm

given by TA (v) = Av, for v ∈ Fn (we are considering column vectors here).

Consider the ordered basis E = (e1 , . . . , en ) and F = (e1 , . . . , em ) of Fn and Fm respectively.

Show that MFE (TA ) = A.

Let L(V, W ) denote the set of all linear transformations from V to W . Suppose S, T ∈ L(V, W )

and c is a scalar. Define S + T and cS as follows :

(S + T )(x) = S(x) + T (x)

(cS)(x) = cS(x)

for all x ∈ V. It is easy to show that L(V, W ) is a vector space under these operations.

Lemma 2.2.12. Fix ordered bases E and F of V and W respectively. For all S, T ∈ L(V, W ) and

scalar c we have

(ii) MFE (cS) = cMFE (S)

(iii) MFE (S) = MFE (T ) ⇔ S = T.

Proof. Exercise.

Lemma 2.2.13. Suppose V, W are vector spaces of dimensions n, m respectively. Suppose T :

V −→ W is a linear transformation. Suppose E = (e1 , . . . , en ), F = (f1 , . . . , fm ) are ordered bases

of V, W respectively. Then

[T (v)]F = MFE (T )[v]E , v ∈ V.

Proof. Let

a1

a2

[v]E =

. .

.

an

Then v = a1 e1 + a2 e2 + · · · + an en and hence T (v) = a1 T (e1 ) + a2 T (e2 ) + · · · + an T (en ).

We have

[T (v)]F = [a1 T (e1 ) + · · · + an T (en )]F

a1

a2

= [[T (e1 )]F [T (e2 )]F · · · [T (en )]F ]

.

.

an

= MFE (T )[v]E .

44 CHAPTER 2. VECTOR SPACES AND LINEAR TRANSFORMATIONS

T : U −→ V and S : V −→ W are linear transformations. Suppose E = (e1 , . . . , en ), F, G are

ordered bases of U, V, W respectively. Then

= MGF (S)[T (ej )]F

= [S(T (ej ))]G .

ordered bases of V . The square matrix MBB (T ) is said to be the matrix of T with respect to

the ordered basis B. Note that the transition matrix MBC from C to B is the matrix MBC (I) of

the identity map wrt C and B. Thus it follows that (why?) MBC (I) = MCB (I)−1 .

An easy induction gives the following generalization the lemma above. Its proof is left as an

exercise.

Vi+1 , 1 ≤ i ≤ m are linear maps. Suppose Ei is an ordered basis of Vi , for 1 ≤ i ≤ m + 1. Then

MEEm+1

1

(Tm ◦ Tm−1 ◦ · · · ◦ T2 ◦ T1 ) = MEEm+1

m

(Tm ) ◦ · · · ◦ MEE32 (T2 ) ◦ MEE21 (T1 ).

MBB (T ) = (MCB )−1 MCC (T )MCB .

E1 = E4 = B, E2 = E3 = C we get

T : R2 → R2 , T (e1 ) = e1 , T (e2 ) = e1 + e2 .

C 1 1 B 1 1 C 1/2 1/2

MC (T ) = , MC = , MB = .

0 1 1 −1 1/2 −1/2

2.2. LINEAR TRANSFORMATIONS 45

Hence

1/2 1/2 1 1 1 1

MBB =

1/2 −1/2 0 1 1 −1

1 3 −1

= .

2 1 1

3 1

T (e1 + e2 ) = 2e1 + e2 = (e1 + e2 ) + (e1 − e2 )

2 2

1 1

T (e1 − e2 ) = −e2 = − (e1 + e2 ) + (e1 − e2 ).

2 2

Exercise 2.2.19. (i) Deduce the rank-nullity theorem for matrices from the rank-nullity theorem

for linear maps.

(ii) Let T : V → W be a linear map of rank r between fdvs V and W . Show that there are

ordered bases E of V and F of W such that

I 0

MFE (T ) = ,

0 0

where I is the r × r identity matrix and 0 stands for a matrix of zeros of appropriate size.

V + W = L(V ∪ W ).

Proof. We shall give a sketch of a proof leaving the reader to fill in the details.

Consider the set V × W = {(v, w) : v ∈ V, w ∈ W }. This set is a vector space with component

wise addition and scalar multiplication. Check that the dimension of this space is dim V + dim W .

Define a linear map T : V × W → V + W by T ((v, w)) = v − w. Check that T is onto and that

the nullspace of T is {(v, v) : v ∈ V ∩ W }. The result now follows from the rank nullity theorem

for linear maps.

Example 2.2.21. Let V, W be finite dimensional vector spaces over F with dimensions n, m re-

spectively. Fix ordered bases E, F for V, W respectively.

Consider the map f : L(V, W ) → Mm×n (F) given by f (T ) = MFE (T ), for T ∈ L(V, W ). Lemma

2.2.12 shows that f is linear and 1-1 and Lemma 2.2.5 shows that f is onto. It follows (why?) that

dim L(V, W ) = mn.

Example 2.2.22. Often we see statements like “If every vector in a vector space is uniquely

determined by t parameters the dimension of V is t”. In this example we show one possible way of

making this precise.

46 CHAPTER 2. VECTOR SPACES AND LINEAR TRANSFORMATIONS

Let V be a vector space over F. A linear functional is a linear map f : V → F. We shall refer

to a linear functional as a “parameter”. Suppose we have t parameters fi : V → F, i = 1, 2, . . . , t.

Suppose every vector in V is uniquely determined by these t parameters, i.e., given arbitrary scalars

a1 , a2 , . . . , at in F, there is a unique vector v ∈ V with fi (v) = ai , i = 1, . . . , t. Then dim V = t.

We show this as follows.

For i = 1, . . . , t, let vi ∈ V be the (unique) vector with fi (vi ) = 1 and fj (vi ) = 0, for j 6= i. We

claim that v1 , . . . , vt is a basis of V .

Let v ∈ V . Put ai = fi (v), i = 1, . . . , t. Consider the vector v − (a1 v1 + · · · + at vt ). Check that

fi (v − (a1 v1 + · · · + at vt )) = 0, for i = 1, . . . , t. Since each of the fi is linear and the parameters

f1 , . . . , ft uniquely determine the vectors in V it follows that the only vector with all parameters 0

is the 0 vector. Thus v − (a1 v1 + · · · + at vt ) = 0 and v1 , . . . , vt span V .

Now suppose a1 v1 + · · · + at vt = 0. Then, for all i, fi (a1 v1 + · · · + at vt ) = ai = 0 and thus linear

independence follows.

Example 2.2.23. Given a n × n matrix, by ri we mean the sum of elements in row i. Similarly,

by cj we mean the sum of elements in column j.

A real magic square of order n is a real n × n matrix satisfying

r1 = r2 = · · · = rn = c1 = c2 = · · · = cn .

Let RM S(n) denote the set of all real magic squares of order n. It is easy to see that RM S(n) is

a subspace of Mn×n (R), the vector space of all n × n real matrices. The dimension of Mn×n (R) is

n2 . What is the dimension of RM S(n)?

We show that dim RM S(n) = (n − 1)2 + 1 using the previous example.

For 1 ≤ i, j ≤ n − 1, define a linear map fij : RM S(n) → R by

Check that the (n − 1)2 + 1 parameters f, fij satisfy the hypothesis of the previous example.

Let V be a finite dimensional vector space and let V = R ⊕ N be a direct sum decomposition.

It is easy to see that there is a unique linear map pR : V → V satisfying

Lemma 2.2.24. Let P : V → V satisfy P 2 = P . Then P is the projection onto Im(P ) along

N (P ).

Proof. Let v ∈ V . Write v = P (v) + (v − P (v)). Now P (v − P (v)) = P (v) − P 2 (v) = 0. It follows

that V = Im(P ) + N (P ).

2.2. LINEAR TRANSFORMATIONS 47

Now let P (u) ∈ Im(P ) ∩ N (P ). Then P (u) = P 2 (u) = P (P (u)) = 0. It follows that V =

Im(P ) ⊕ N (P ).

Let P (u) ∈ Im(P ). Then P (P (u)) = P 2 (u) = P (u). Clearly P (v) = 0 for v ∈ N (P ). That

completes the proof.

48 CHAPTER 2. VECTOR SPACES AND LINEAR TRANSFORMATIONS

Chapter 3

The concept of a (real) vector space abstracts the operations of adding directed line segments and

multiplying a directed line segment by a real number. In plane geometry we also speak of other

geometric concepts such as length, angle, perpendicularity, projection of a point on a line etc.

Remarkably, we need to put only a single additional algebraic structure, that of an inner product,

on a vector space in order to have these geometric concepts available in the abstract setting.

We shall use the following notation. Recall that F = R or C. Given a ∈ F, a will denote a if

F = R and is the complex conjugate of a if F = C. Given a matrix A over F we denote by A∗ the

conjugate transpose of A, i.e., if A = (aij ) then A∗ = (aji ). We call A∗ the adjoint of A.

Definition 3.1.1. Let V be a vector space V over F. An inner product on V is a rule which

to any ordered pair of elements (u, v) of V associates a scalar, denoted by hu, vi satisfying the

following axioms: for all u, v, w in V and c any scalar we have

(1) hu, vi = hv, ui (Hermitian property or conjugate symmetry),

(2) hu, v + wi = hu, vi + hu, wi (additivity),

(3) hcu, vi = chu, vi (homogeneity),

(4) hv, vi ≥ 0 with hv, vi = 0 iff v = 0 (positive definite).

Example

Pn 3.1.2. (1) Let v = (x1 , x2 , . . . , xn )t and w = (y1 , y2 , . . . , yn )t ∈ Rn . Define hv, wi =

t n

i=1 xi yi = v w. This is an inner product on the real vector space R . This is often called the

standard inner product.

(2) Let v = (x1 , x2 , . . . , xn )t and w = (y1 , y2 , . . . , yn )t ∈ Cn . Define hv, wi = ni=1 xi yi = v ∗ w.

P

This is an inner product on the complex vector space Cn . This is often called the standard inner

product.

(3) Let V = the vector space of all real valued continuous functions on the unit interval [0, 1].

For f, g ∈ V, put

49

50 CHAPTER 3. INNER PRODUCT SPACES

Z 1

hf, gi = f (t)g(t)dt.

0

Simple properties of the integral show that hf, gi is an inner product.

(4) Let B be a nonsingular n × n complex matrix. Set A = B ∗ B. Given x, y ∈ Cn define

hx, yi = x∗ Ay. Denote the standard inner product on Cn by the dot product (i.e., the inner

product of u and v is u · v). We have

hx, yi = x∗ Ay = x∗ B ∗ By = (Bx)∗ By = Bx · By

Definition 3.1.3. Given an inner product space V and an element v ∈ V we define its length or

norm by p

kvk = hv, vi.

We say v is a unit vector if kvk = 1. Elements v, w of V are said to be orthogonal or perpen-

dicular if hv, wi = 0. We write this as v ⊥ w.

p p

kcvk = hcv, cvi = cchv, vi = |c|kvk.

Proof. We have

hw, vi

pw (v) = w

hw, wi

to be the projection of v on w.

Note that the map pw : V → V given by v 7→ pw (v) is linear. This is the reason for using hw, vi

instead of hv, wi in the definition of pw (v).

The next lemma, whose geometric content is clear, explains the use of the term projection.

Lemma 3.1.7. Let v, w ∈ V with w 6= 0. Then

w (v), i.e., the projection of v on w is the same as the projection of v on the unit

(i) pw (v) = p kwk

vector in the direction of w.

(ii) pw (v) and v − pw (v) are orthogonal.

(iii) kpw (v)k ≤ kvk with equality iff {v, w} are linearly dependent.

3.1. LENGTH, PROJECTION, AND ANGLE 51

hw, vi hw, vi w w

pw (v) = w= 2 w=h , vi = p kwk

w (v),

hw, wi kwk kwk kwk

w w

where in the last step we have used the fact that h kwk , kwk i = 1.

(ii) In view of part (i) we may assume that w is a unit vector. We have

hpw (v), v − pw (v)i = hpw (v), vi − hpw (v), pw (v)i

= 0.

(iii) We have (in the third step below we have used part (ii) and Pythogoras)

kvk2 = hv, vi

= hpw (v) + v − pw (v), pw (v) + v − pw (v)i

= kpw (v)k2 + kv − pw (v)k2

≥ kpw (v)k2 .

Clearly, there is equality in the last step iff v = pw (v), i.e., iff {v, w} are dependent.

Theorem 3.1.8. Cauchy-Schwarz inequality For v, w ∈ V

|hw, vi| ≤ kvkkwk,

with equality iff {v, w} are linearly dependent.

Case (i): w is a unit vector. In this case the lhs of the C-S inequality is kpw (v)k and the result

follows from part (iii) of the previous lemma.

w

Case (ii): w is not a unit vector. Set u = kwk . We have |hw, vi| = kwk(|hu, vi|) and kvkkwk =

kwk(kvkkuk). The result follows from Case (i) above.

Theorem 3.1.9. Triangle Inequality For v, w ∈ V

kv + wk ≤ kvk + kwk.

kv + wk2 = hv + w, v + wi

= hv, vi + hv, wi + hw, vi + hw, wi

= hv, vi + hv, wi + hv, wi + hw, wi

≤ kvk2 + kwk2 + 2kvkkwk

= (kvk + kwk)2 .

52 CHAPTER 3. INNER PRODUCT SPACES

Definition 3.1.10. Let V be a real inner product space. Given v, w ∈ V with v, w 6= 0, by C-S

inequality

hv, wi

−1 ≤ ≤ 1.

kvkkwk

hv, wi

So, there is a unique 0 ≤ θ ≤ π satisfying cos(θ) = kvkkwk . This θ is the angle between v and w.

The distance between u and v in V is defined as d(u, v) = ku − vk.

Lemma 3.1.11. Let u, v, w ∈ V . Then

(i) d(u, v) ≥ 0 with equality iff u = v.

(ii) d(u, v) = d(v, u).

(iii) d(u, v) ≤ d(u, w) + d(w, u).

Proof. Exercise.

Definition 3.1.12. Let V be an n-dimensional inner product space. A basis {v1 , v2 , . . . , vn } of V

is called orthogonal if its elements are mutually perpendicular, i.e., if hvi , vj i = 0 for i 6= j. If,

in addition, kvi k = 1, for all i, we say that the basis is orthonormal.

Example 3.1.13. In Fn , with the standard inner product, the basis {e1 , . . . , en } is orthonormal.

Lemma 3.1.14. Let U = {u1 , u1 , . . . , un } be a set of nonzero vectors in an inner product space V.

If hui , uj i = 0 for i 6= j, 1 ≤ i, j ≤ n, then U is linearly independent.

c1 u1 + c2 u2 + . . . + cn un = 0.

ci hui , ui i = 0.

1 1

Example 3.1.15. (1) Consider R2

with standard inner product. Then v1 = and v2 =

1 −1

are orthogonal. Dividing v1 and v2 by their lengths we get an orthonormal basis.

(2) Let V denote the real inner product space of all continuous real functions on [0, 2π] with

inner product given by Z 2π

hf, gi = f (x)g(x)dx.

0

Z 2π √

2 2π, n = 0,

kgn (x)k = cos nx dx = √

0 π, n ≥ 1,

R 2π

and hgm , gn i = 0 cos(mx) cos(nx) dx = 0, m 6= n.

So, {g0 , . . . , gn } are orthogonal.

3.1. LENGTH, PROJECTION, AND ANGLE 53

Theorem 3.1.16. Let V be a finite dimensional inner product space. Let W ⊆ V be a subspace and

let {w1 , . . . , wm } be an orthogonal basis of W . If W 6= V , then there exist elements {wm+1 , . . . , wn }

of V such that {w1 , . . . , wn } is an orthogonal basis of V .

Taking W = {0}, the zero subspace, we see that V has an orthogonal, and hence orthonormal,

basis.

Proof. The method of proof is as important as the theorem and is called the Gram-Schmidt

orthogonalization process.

Since W 6= V , we can find a vector vm+1 such that {w1 , . . . , wm , vm+1 } is linearly independent.

The idea is to take vm+1 and subtract from it its projections along w1 , . . . , wm }. Define

hw, vi

(Recall that pw (v) = hw, wi w.)

Clearly, wm+1 6= 0 as otherwise {w1 , . . . , wm , vm+1 } would be linearly dependent. We now check

that {w1 , . . . , wm+1 } is orthogonal. For this, it is enough to check that wm+1 is orthogonal to each

of wi , 1 ≤ i ≤ m.

For i = 1, 2, . . . , m we have

m

X

hwi , wm+1 i = hwi , vm+1 − pwj (vm+1 )i

j=1

m

X

= hwi , vm+1 i − hwi , pwj (vm+1 )i

j=1

= hwi , vm+1 i − hwi , pwi (vm+1 )i, (since hwi , wj i = 0 for i 6= j)

= hwi , vm+1 − pwi (vm+1 )i

= 0, (by part (ii) of 3.1.7).

Example 3.1.17.

Find

anorthonormal

basis

for the subspace of R4 (under standard inner product)

1 1 1

1 −2 0

spanned by

0 , 0 , and −1

1 0 2

Denote these vectors by a, b, c respectively. Set

b·a

b0 = b − a

a · a

4

1 −5 .

=

3 0

1

54 CHAPTER 3. INNER PRODUCT SPACES

c·a c · b0

c0 = c − a − 0 0 b0

a·a b ·b

−4

1 −2 .

=

7 −7

6

Now a, b0 , c0 are orthogonal and generate the same subspace as a, b, c. Dividing by the lengths we

a 0 0

get the orthonormal basis kak , kbb0 k , kcc0 k .

Example 3.1.18. Let V = P3 [−1, 1] denote the real vector space of polynomials of degree atmost

3 defined on [−1, 1]. V is an inner product space under the inner product

Z 1

hf, gi = f (t)g(t)dt.

−1

To find an orthonormal basis, we begin with the basis {1, x, x2 , x3 }. Set v1 = 1. Then

1

v2 = x − hx, 1i

k1k2

1 1

Z

= x− tdt = x,

2 −1

1 x

v3 = x2 − hx2 , 1i − hx2 , xi

2 (2/3)

1 1 2

Z Z 1

3

= x2 − t dt − x t3 dt

2 −1 2 −1

1

= x2 − ,

3

1 x 1 x2 − 1

v4 = x3 − hx3 , 1i − hx3 , xi − hx3 , x2 − i p 3

2 (2/3) 3 ( 8/45)

3

= x3 − x.

5

orthonormal basis. ( r √ √ )

1 3 2 1 3 5 3 3 5 7

√ , x , (x − ) √ , (x − x) √ .

2 2 3 2 2 5 2 2

You will meet these polynomials later when you will learn about differential equations.

Let V be a finite dimensional inner product space. We have seen how to project a vector onto a

nonzero vector. We now discuss the (orthogonal) projection of a vector onto a subspace.

3.2. PROJECTIONS AND LEAST SQUARES APPROXIMATIONS 55

W ⊥ = {u ∈ V | u ⊥ w for all w ∈ W }.

v = x + y,

where x ∈ W and y ∈ W ⊥ .

we have

hy, vi i = hv − x, vi i

= hv, vi i − hx, vi i

k

X

= hv, vi i − h hvj , vivj , vi i

j=1

k

X

= hv, vi i − hvj , vihvj , vi i

j=1

= hv, vi i − hv, vi i (by orthonormality)

= 0.

(uniqueness) Let v = x + y = x0 + y 0 , where x, x0 ∈ W and y, y 0 ∈ W ⊥ . Then x − x0 = y 0 − y ∈

W ∩ W ⊥ . But W ∩ W ⊥ = {0} (why?). Hence x = x0 and y = y 0 .

Proof. Exercise.

Exercise 3.2.3. Consider Rn with standard inner product. Given a nonzero vector v ∈ Rn , by Hv

we mean the hyperplane (i.e., a subspace of dimension n − 1) orthogonal to v

Hv = {u ∈ Rn : u · v = 0}.

to −v and fixes every vector in Hv , i.e., Tv (v) = −v and Tv (u) = u, for u ∈ Hv .

Show that, for all w ∈ Rn ,

2(w · v)

Tv (w) = w − v.

v·v

56 CHAPTER 3. INNER PRODUCT SPACES

express v (uniquely) as v = x + y, where x ∈ W and y ∈ W ⊥ . Define pW (v) = x. We call pW (v)

the orthogonal projection of v onto W . Note that v − pW (v) ∈ W ⊥ . Note also that the map pW

is linear.

The diligent reader should observe that, in the language of the previous chapter, pW is the

projection onto W along W ⊥ .

Example 3.2.5. Consider V = Fn with the standard inner product. Let P be a n × n matrix over

F with associated linear map TP : Fn → Fn . Assume that P 2 = P and P = P ∗ . Then we claim

that TP = pim(TP ) = pC(P ) . To see this proceed as follows.

We have already seen that P 2 = P implies that TP is the projection onto C(P ) along N (P ). It

is enough to show that C(P ) and N (P ) are orthogonal. Let v ∈ C(P ) and u ∈ N (P ). Then

This example is our first hint of the connection between adjointness and orthogonality. We shall

come back to this theme when we discuss the spectral theorem.

Definition 3.2.6. Let W be a subspace of V and let v ∈ V . A best approximation to v by

vectors in W is a vector w in W such that

k v − w k ≤ k v − u k, for all u ∈ W.

The next result shows that orthogonal projection gives the unique best approximation.

Theorem 3.2.7. Let v ∈ V and let W be a subspace of V . Let w ∈ W . Then the following are

equivalent:

(i) w is a best approximation to v by vectors in W .

(ii) w = pW (v).

(iii) v − w ∈ W ⊥ .

Proof. We have

k v − w k2 = k v − pW (v) + pW (v) − w k2

= k v − pW (v) k2 + k pW (v) − w k2 ,

where the second equality follows from Pythogoras’ theorem on noting that pW (v) − w ∈ W and

v − pW (v) ∈ W ⊥ . It follows that (i) and (ii) are equivalent. To see the equivalence of (ii) and (iii)

write v = w + (v − w) and apply Theorem 3.2.1.

Consider Rn with the standard inner product (we think of Rn as column vectors). Let A be an

n × m (m ≤ n) matrix and let b ∈ Rn . We want to project b onto the column space of A. Here is

a method for doing this:

(i) Use Gauss elimination find a basis of C(A).

(ii) Now use Gram-Schmidt process to find an orthogonal basis B of C(A).

3.2. PROJECTIONS AND LEAST SQUARES APPROXIMATIONS 57

(iii) We have X

pC(A) (b) = pw (b).

w∈B

We now discuss another method for doing this. The projection of b onto the column space of A

will be a vector of the form p = Ax for some x ∈ Rm . From Theorem 3.2.7, p is the projection iff

b − Ax is orthogonal to every column of A. In other words, x should satisfy the equations

At (b − Ax) = 0, or

At Ax = At b.

The above equations are called normal equations in the Gauss-Markov theory in statistics. Thus,

if x is any solution of the normal equations, then Ax is the required projection of b.

Lemma 3.2.8.

rank (A) = rank (At A).

Proof. We have rank (A) ≥ rank (At A) (why?). Let At Az = 0, for z ∈ Rm . Then At w = 0,

where w = Az, i.e., w is in the column space of A and is orthogonal to every column of A.

This implies (why?) that w = Az = 0. Thus nullity (A) ≥ nullity (At A). It follows that

rank (At A) = rank (A) .

If the columns of A are linearly independent, the (unique) solution to the normal equations is

(At A)−1 At b and the projection of b onto the column space of A is A(At A)−1 At b. Note that the

normal equations always have a solution (why?), although the solution will not be unique in case

the columns of A are linearly dependent (since rank (At A) = rank (A) < m).

1 1 1

Example 3.2.9. Let A = 1 0 and b = 0 .

0 1 −5

2 1 1

Then At A = and At b = . The unique solution to the normal equations is

1 2 −4

2

2

x= and b − Ax = −2 (note that this vector is orthogonal to the columns of A).

−3

−2

−1

The projection of b onto the column space of A is p = Ax = 2 .

−3

1 1 1 2 1 3/2 1

Now let B = 1 0 1/2 . We have B t B = 1 2 3/2 and B t b = −4 .

0 1 1/2 3/2 3/2 3/2 −3/2

Note that A and B have the same column spaces (the third column of B is the average of

the first two columns). So the projection of b onto the column space of B will be the same as

before.

However

the normal

equations do not have a unique solution in this case. Check that

2 3

x = −3 , −2 are both solutions of the normal equations B t Bx = B t b.

0 −2

58 CHAPTER 3. INNER PRODUCT SPACES

Suppose we have a large number of data points (xi , yi ), i = 1, 2, . . . , n collected from some

experiment. Frequently there is reason to believe that these points should lie on a straight line. So

we want a linear function y(x) = s + tx such that y(xi ) = yi , i = 1, . . . , n. Due to uncertainity in

data and experimental error, in practice the points will deviate somewhat from a straightline and

so it is impossible to find a linear y(x) that passes through all of them. So we seek a line that fits

the data well, in the sense that the errors are made as small as possible. A natural question that

arises now is: how do we define the error?

Consider the following system of linear equations, in the variables s and t, and known coefficients

xi , yi , i = 1, . . . , n:

s + x 1 t = y1

s + x 2 t = y2

.

.

s + x n t = yn

Note that typically n would be much greater than 2. If we can find s and t to satisfy all these

equations, then we have solved our problem. However, for reasons mentioned above, this is not

always possible. For given values of s and t the error in the ith equation is |yi − s − xi t|. There

are several ways of combining the errors in the individual equations to get a measure of the total

error. The following are three examples:

v

u n n

uX X

t (yi − s − xi t)2 , |yi − s − xi t|, max 1≤i≤n |yi − s − xi t|.

i=1 i=1

Both analytically and computationally, a nice theory exists for the first of these choices and this is

what we shall study. The problem of finding s, t so as to minimize

v

u n

uX

t (yi − s − xi t)2

i=1

Let

1 x1 y1 s + tx1

1 x2 y2 s + tx2

, and x = s , so that Ax =

A= . . , b =

. . .

t

. . . .

1 xn yn s + txn

The least squares problem is finding an x such that ||b − Ax|| is minimized, i.e., find an x such

that Ax is the best approximation to b in the column space of A. This is precisely the problem of

finding x such that b − Ax is orthogonal to the column space of A.

A straight line can be considered as a polynomial of degree 1. We can also try to fit an mth

degree polynomial

y(x) = s0 + s1 x + s2 x2 + · · · + sm xm

3.3. DETERMINANT AND VOLUME 59

to the data points (xi , yi ), i = 1, . . . , n, so as to minimize the error (in the least squares sense). In

this case s0 , s1 , . . . , sm are the variables and we have

x1 x21 . . xm

1 1 y1 s0

1

x2 x22 . . xm

2

y2

s1

A= . . . . . , b = . , x = . .

. . . . . . .

1 2

xn xn . . xn m yn sm

Example 3.2.10. Find s, t such that the straight line y = s + tx best fits the following data in the

least squares sense:

y = 1 at x = −1, y = 1 at x = 1, y = 3 at x = 2.

1 1 −1

We want to project b = 1 onto the column space of A = 1 1 . Now At A =

3 1 2

3 2 5

and At b = . The normal equations are

2 6 6

3 2 s 5

= .

2 6 t 6

9

The solution is s = 9/7, t = 4/7 and the best line is y = 7 + 47 x.

The proper definition of volume needs multivariable calculus. We do not discuss this here but

instead give an elementary geometric definition of the k-volume of a parallelopiped with k sides in

Rn and show that this is given by a determinant.

Definition 3.3.1. Consider Rn with standard inner product. For 1 ≤ k ≤ n, let v1 , . . . , vk be k

vectors in in Rn . For i = 1, . . . , k define Wi to be the subspace spanned by v1 , . . . , vi .

Inductively define the volume V (v1 , . . . , vi ) of the i-parallelopiped with sides v1 , . . . , vi as follows

V (v1 ) = kv1 k, if i = 1,

V (v1 , . . . , vi ) = kvi − pWi−1 (vi )kV (v1 , . . . , vi−1 ), if 1 < i ≤ k.

While this definition is geometrically meaningful, it is not clear that the volume is independent

of the order in which the v1 , . . . , vk are listed. This will follow from the next result.

Exercise 3.3.2. Show that if v1 , . . . , vk are linearly dependent then V (v1 , . . . , vk ) = 0.

Exercise 3.3.3. Show that V (v1 , . . . , vk ) ≤ kv1 kkv2 k · · · kvk k, with equality iff the vi ’s are orthog-

onal.

For i = 1, 2, . . . , k, let Ai be the n × i matrix with v1 , . . . , vi as the columns. Note that the

entry in row r and column s of Ati Ai is vr · vs .

60 CHAPTER 3. INNER PRODUCT SPACES

V (v1 , . . . , vk )2 = det(Atk Ak ), k = 1, . . . , n.

Set uk = vk − pWk−1 (vk ) and write

vk = uk + a1 v1 + · · · + ak−1 vk−1 .

Set a = (a1 , . . . , ak−1 )t and let Bk be the n × k matrix with v1 , . . . , vk−1 , uk as the columns. Then

(why?)

I a

Ak = Bk ,

0 1

where the second matrix on the rhs is k × k with I the (k − 1) × (k − 1) identity matrix. Thus

t

I a I a

Atk Ak = Bkt Bk .

0 1 0 1

t

t Bk−1 Bk−1 0

Bk Bk = ,

0 uk · uk

t

By induction det(Bk−1 Bk−1 ) = V (v1 , . . . , vk−1 )2 and the result follows by taking determinants

in the formula above for Atk Ak .

Exercise 3.3.5. Show that V (v1 , . . . , vk ) is invariant under permutation of the vi ’s.

Exercise 3.3.6. Let A be a square n × n real matrix with columns v1 , . . . , vn . Prove Hadamard’s

inequality: det(A) ≤ kv1 kkv2 k · · · kvn k, with equality iff the vi ’s are orthogonal.

Chapter 4

Definition 4.1.1. Let V be a vector space over F and let T : V → V be a linear operator. A scalar

λ ∈ F is said to be an eigenvalue of T if there is a nonzero vector v ∈ V such that T (v) = λv.

We say that v is an eigenvector of T with eigenvalue λ.

eigenvalue and eigenvector of the linear map

TA : Fn → Fn

vector x ∈ Fn with Ax = λx.

Example 4.1.3. (i) Let V be the real vector space of all smooth real valued functions on R. Let

D : V → V be the derivative map. The function f (x) = eλx is an eigenvector with eigenvalue λ

since D(eλx ) = λeλx .

(ii) Let A be a diagonal matrix with scalars µ1 , . . . , µn on the diagonal. We write this as A =

diag(µ1 , . . . , µn ). Then Aei = µi ei and so {e1 , . . . , en } are eigenvectors of A with (corresponding)

eigenvalues µ1 , . . . , µn .

Vλ = {v ∈ V : T (v) = λv}

eigenvector with eigenvalue λ. In this case we say that Vλ is the eigenspace of the eigenvalue λ.

T and let vi ∈ Vλi , i = 1, . . . , n. Then

61

62 CHAPTER 4. EIGENVALUES AND EIGENVECTORS

v1 + v2 + · · · + vn = 0.

So

λ1 v1 + λ1 v2 + · · · + λ1 vn = 0.

Applying T to the equation v1 + v2 + · · · + vn = 0 gives

λ1 v1 + λ2 v2 + · · · + λn vn = 0.

Subtracting gives

(λ2 − λ1 )v2 + · · · + (λn − λ1 )vn = 0.

Since λ1 , λ2 , . . . , λn are distinct we get (why?), by induction, that v2 = · · · = vn = 0. And now we

get (why?)v1 = 0.

Example 4.1.5. Continuation of Example 4.1.3 (i). {eλ1 x , . . . , eλn x } (λ1 , . . . , λn distinct) are

linearly independent functions.

Definition 4.1.6. Let V be a fdvs over F and let T : V → V be a linear operator. T is said

to be diagonalizable if there exists a basis (or, equivalently, an ordered basis) of V consisting of

eigenvectors of T . If B = (v1 , . . . , vn ) is such an ordered basis with T (vi ) = λi vi , λi ∈ F then

MBB (T ) = diag(λ1 , . . . , λn ).

Definition 4.1.7. A n × n matrix A over F is said to be diagonalizable iff the linear map TA :

Fn → Fn , given by TA (x) = Ax, x ∈ Fn , is diagonalizable.

some invertible matrix P over F. In that case, the columns of P are eigenvectors of A and the ith

diagonal entry of P −1 AP is the eigenvalue associated with the ith column of P .

Proof. (only if ) Let {v1 , . . . .vn } be a basis of Fn with TA (vi ) = Avi = λi vi . Let P be the n × n

matrix with ith column vi and let D = diag(λ1 , . . . , λn ). Then check that

AP = P D.

(if ) Suppose P −1 AP = D, where D is diagonal. Then AP = P D. It now follows, as in the only if

part, that the ith column of P is an eigenvector with eigenvalue the ith diagonal entry of D.

PA of A to be

PA (t) = det(tI − A).

Note that PA (t) is a monic polynomial of degree n (i.e., the coefficient of the term tn is 1).

4.1. ALGEBRAIC AND GEOMETRIC MULTIPLICITIES 63

Proof. We have

PA (t) = det(tI − P BP −1 )

= det(P (tI − B)P −1 )

= det(P )det(tI − B)det(P −1 )

= PB (t).

Lemma 4.1.11. (1) Eigenvalues of a square matrix A are the roots of PA (t) lying in F.

(2) For a scalar λ ∈ F, Vλ = nullspace of A − λI.

Proof. (1) λ ∈ F is an eigenvalue of A iff Av = λv for some nonzero v iff (A − λI)v = 0 for some

nonzero v iff the nullity of A − λI is positive iff rank(A − λI) is less than n iff det(A − λI) = 0.

(2) Vλ = {v | Av = λv} = {v | (A − λI)v = 0} = N (A − λI).

1 2

Example 4.1.12. (1) Let A = . To find the eigenvalues of A we solve the equation

0 3

λ − 1 −2

det(λI − A) = det = (λ − 1)(λ − 3) = 0.

0 λ−3

Hence the eigenvalues of A are 1 and 3. Let us calculate the eigenspaces V1 and V3 . By definition

0 2 0 2 x 2y 0

A−I = . Suppose = = . Hence V1 = L{(1, 0)}.

0 2 0 2 y 2y 0

1−3 2 −2 2 −2 2 x 0

A − 3I = = . Suppose = .

0 3−3 0 0 0 0 y 0

−2x + 2y 0

Then = . Hence x = y. Thus V3 = L({(1, 1)}).

0 0

cos θ −sin θ

(2) Let A = , where θ 6= 0, π. Now

sin θ cos θ

t − cos θ sin θ

PA (t) = det( )

−sin θ t − cos θ

= (t − cos θ)2 + sin2 θ

= (t − eiθ )(t − e−iθ ),

√

where i = −1.

So, as a real matrix A has no eigenvalues (and thus no eigenvectors). This is clear geometrically

as A represents counter clockwise rotation by θ. But as a complex matrix A has two distinct eigen-

values eiθ and e−iθ . An eigenvector corresponding to eiθ is (1, −i)t and an eigenvector corresponding

to e−iθ is (−i, 1)t .

64 CHAPTER 4. EIGENVALUES AND EIGENVECTORS

4 −3

Example 4.1.13. Find A8 , where A = .

2 −1

3

Check that the eigenvalues of A are 2, 1 and that the corresponding eigenvectors are and

2

1

.

1

3 1 2 0 1 −1

Set P = and D = . Then P −1 = and A = P DP −1 .

2 1 0 1 −2 3

We have

A8 = (P DP −1 )8 = (P DP −1 ) · · · (P DP −1 ) = P D8 P −1

8

3 1 2 0 1 −1

=

2 1 0 18 −2 3

3 1 256 0 1 −1

=

2 1 0 1 −2 3

766 −765

= .

510 −509

polynomial PT (t) of T to be PA (t), where A = MBB (T ) wrt an ordered basis of V . By Lemma

4.1.10 it is immaterial which ordered basis B we take.

Definition 4.1.14. (i) Let f (x) be a polynomial with coefficients in F. Let µ ∈ F be a root of f (x).

Then (x − µ) divides f (x). The multiplicity of the root µ is the largest positive integer k such

that (x − µ)k divides f (x).

(ii) Let V be a fdvs over F and let T : V → V be a linear operator. Let µ be an eigenvalue of

T . The geometric multiplicity of µ is the dimension of the eigenspace Vµ and the algebraic

multiplicity of µ is the multiplicity of µ as a root of PT (t).

Theorem 4.1.15. Let V be a fdvs over F and let T : V → V be a linear operator. Then the

geometric multiplicity of an eigenvalue µ ∈ F of T is less than or equal to the algebraic multiplicity

of µ.

Proof. Suppose that the algebraic multiplicity of µ is k. Let g = geometric multiplicity of µ. Hence

Vµ has a basis of g eigenvectors v1 , v2 , . . . , vg . We can extend this basis of Vµ to an (ordered) basis

of V say B = (v1 , v2 , . . . , vg , . . . , vn ). Now

µIg D

MBB (T ) =

0 C

we see that (λ − µ)g is a factor of det(A − λI). Thus g ≤ k.

Theorem 4.1.16. Let T : V → V be a linear operator, where V is a n-dimensional vector space

over F.

4.1. ALGEBRAIC AND GEOMETRIC MULTIPLICITIES 65

(i) T is diagonalizable iff the sum of the dimensions of the eigenspaces of T equals n.

(ii) Assume F = C. Then T is diagonalizable iff the algebraic and geometric multiplicities are

equal for all eigenvalues of T .

Proof. (i)(if ) Let λ1 , . . . , λk be the distinct eigenvalues of T . Let Bi be a basis of Vλi . By Theorem

4.1.4 B1 ∪ B2 ∪ · · · ∪ Bk is a basis of V consisting of eigenvectors of T .

(only if ) Exercise.

(ii) Let λ

Q1 k, . . . , λk bemithe distinct eigenvalues of T . By the FundamentalPtheorem of Algebra

PT (t) = i=1 (t − λi ) , where mi is the algebraic multiplicity of λi . Since i mi = n the result

now follows from part (i).

3 0 0

Example 4.1.17. (1) Let A = −2 4 2 . Then det(λI − A) = (λ − 3)2 (λ − 6).

−2 1 5

Hence eigenvalues of A are 3 and 6. The eigenvalue λ = 3 has algebraic multiplicity 2. Let

us find the eigenspaces V3 and V6 .

0 0 0

λ = 3 : A − 3I = −2 1 2 . Hence rank(A − 3I) = 1.

−2 1 2

−3 0 0

λ = 6 : A − 6I = −2 −2 2 . Hence rank(A − 6I) = 2.

−2 1 −1

Thus dim V6 = 1. It can be shown that {(0, 1, 1)t } is a basis of V6 . Thus the algebraic and geometric

multiplicities of λ = 6 are one.

If we define

1 1 0

P = 0 2 1

1 0 1

then we have P −1 AP = diag(3, 3, 6).

1 1

(2) Let A = . Then det(λI − A) = (λ − 1)2 . Thus λ = 1 has algebraic multiplicity 2.

0 1

0 1

A−I = . Hence nullity (A − I) = 1 and V1 = L{e1 }. In this case the geometric

0 0

multiplicity of λ = 1 < algebraic multiplicity of λ = 2.

Exercise 4.1.18. Show that an upper triangular matrix with distinct scalars on the diagonal is

diagonalizable.

66 CHAPTER 4. EIGENVALUES AND EIGENVECTORS

Theorem 4.1.19. (Schur) Let V be a n-dimensional vector space over C and let T : V → V be a

linear operator. Then there exists an ordered basis B of V such that MBB (T ) is upper triangular.

Proof. By induction on the dimension of V the dimension 1 case being clear. Let λ be an eigenvalue

of T (it exists since we have taken the scalars to be C). Choose an eigenvector v with eigenvalue

λ. Extend v to an ordered basis (v, u2 , . . . , un ) of V and let W denote the subspace spanned by

{u2 , . . . , un }.

Define a linear map S : W → W as follows: given u ∈ W , write (uniquely) T (u) = av + w,

where a ∈ C and w ∈ W . Define S(u) = w. Check that this defines a linear map.

By induction, there is an ordered basis C = (w2 , . . . , wn ) of W such that MCC (S) is upper

triangular. Set B = (v, w2 , . . . , wn ). It is now easy to check that MBB (T ) is upper triangular.

Exercise 4.1.20. Let A be a complex n × n matrix with eigenvalues λ1 , . . . , λn (in this notation

eigenvalues may possibly be repeated. The number of times an eigenvalue is repested is equal to

its algebraic multiplicity). The trace of A, denoted tr(A), is the sum of its diagonal entries.

1. Show that det(A) = λ1 λ2 · · · λn .

2. Show that tr(A) = λ1 + λ2 + · · · + λn . (Hint: tr(P −1 AP ) = tr(A). Now use the previous

theorem.)

3. Show that the eigenvalues of Ak are λk1 , . . . , λkn .

Example 4.1.21. Let A be a complex n × n . Then A is nilpotent iff all eigenvalues of A are 0.

Choose an ordered basis B of Cn such that M = MBB (TA ) = (mij ) is upper triangular. So the

eigenvalues of A are mii , i = 1, . . . , n. If all mii are 0 then A is nilpotent (by an example form

Chapter 1). On the other hand, if some mjj 6= 0, then the jth diagonal entry of all powers of A

will be nonzero and A cannot be nilpotent.

Example 4.1.22. Let A be a complex n × n matrix. Then A is nilpotent iff tr(Ai ) = 0, for

i = 1, . . . , n.

The only if part is clear. For the if part, we prove only the n = 3 case. The reader is urged

to work out the n = 4 case (which is similar) after which the general case becomes clear. In the

process the reader will have discovered Newton’s identities (on symmetric functions).

Assume that the eigenvalues of A are µ1 , µ2 , µ3 . Since trace is the sum of eigenvalues we have

pi = µi1 + µi2 + µi3 = 0, i = 1, 2, 3.

We want to show µ1 = µ2 = µn = 0.

Set e1 = p1 , e2 = µ1 µ2 + µ1 µ3 + µ2 µ3 and e3 = µ1 µ2 µ3 .

First we show e1 = e2 = e3 = 0. We have e1 = p1 = 0. Now consider

0 = (µ1 + µ2 + µ3 )2 = p2 + 2e2 .

It follows that e2 = 0. Similarly consider

X

0 = (µ1 + µ2 + µ3 )3 = p3 + 6e3 + 3 µ2i µj

i6=j

= p3 + 6e3 + 3{p2 p1 − p3 }.

It follows that e3 = 0. Now e1 = e2 = e3 = 0 easily implies that µ1 = µ2 = µ3 = 0.

4.2. SPECTRAL THEOREM 67

In this section we prove one of the most important results in Linear Algebra, namely, the Spectral

Theorem. We shall see that the three concepts of symmetry or conjugate symmetry (of the entries

of a matrix), orthogonality (of eigenvectors), and commutativity (of operators) are linked together

in a deep way. More generally, adjointness, orthogonality, and commutativity go hand in hand.

This is the beginnings of the beautiful subject of representation theory.

Recall the definition of a diagonalizable complex matrix: An n × n complex matrix A is diago-

nalizable if there exists a basis of Cn consisting of eigenvectors of A. Now, Cn is not just a vector

space. Endowed with the dot product, it is an inner product space. The following definition is thus

natural.

Definition 4.2.1. (i) An n × n complex matrix A is orthonormally diagonalizable if there

exists an orthonormal basis of Cn consisting of eigenvectors of A. Similarly,

(ii) An n × n real matrix A is orthonormally diagonalizable if there exists an orthonormal

basis of Rn consisting of eigenvectors of A.

Theorem 4.2.2. (i) If a square real matrix A is orthonormally diagonalizable then At = A, i.e.,

A is symmetric.

(ii) If a square complex matrix A is orthonormally diagonalizable then A∗ A = AA∗ .

orthonormal basis of Rn with Avi = λi vi , λi ∈ R and let D = diag(λ1 , . . . , λn ). Let P be the n × n

matrix with ith column vi . Then AP = P D.

Since the vi are orthonormal we have (why?) P t P = I. Thus

A = P DP t and At = P Dt P t .

(ii) Let A be a complex n × n orthonormally diagonalizable matrix. Let v1 , v2 , . . . , vn be an or-

thonormal basis of Cn with Avi = λi vi , λi ∈ C and let D = diag(λ1 , . . . , λn ). Let P be the n × n

matrix with ith column vi . Then AP = P D.

Since the vi are orthonormal we have (why?) P ∗ P = I. Thus

A = P DP ∗ and A∗ = P D∗ P ∗ .

we have AA∗ = A∗ A.

A square complex matrix A is called normal if A∗ A = AA∗ . The spectral theorem asserts the

converse of the theorem above : any real symmetric matrix is orthonormally diagonalizable (over

reals) and any complex normal matrix 1 is orthonormally diagonalizable (over complexes).

A square complex matrix A is called Hermitian or self-adjoint if A = A∗ . Note that

Hermitian matrices are normal. Also note that a real matrix (treated as a complex matrix) is

Hermitian iff it is symmetric. We shall first prove the spectral theorem for Hermitian matrices and

then deduce the normal case from this.

1

unlike human beings it is possible for a matrix to be both complex and normal.

68 CHAPTER 4. EIGENVALUES AND EIGENVECTORS

(v ∗ Av)∗ = v ∗ A∗ v = v ∗ Av,

so v ∗ Av is a real number.

Now let λ be an eigenvalue of A with eigenvector v. Then

Since v ∗ Av is real and kvk is a nonzero real number it follows that λ is real.

Though a proof of the spectral theorem for self-adjoint matrices can be given working only with

matrices, a coordinate free approach is more intuitive and more memorable. Therefore, we first

develop a coordinate free version of the concept of a self adjoint matrix. The following definition

covers both the real and complex cases.

Definition 4.2.4. Let V be a finite dimensional inner product space over F. A linear operator

T : V → V is said to be Hermitian or self-adjoint if

Theorem 4.2.5. Let V be a finite dimensional inner product space over F and let T : V → V

be a linear operator. Then T is self-adjoint iff MBB (T ) is a self-adjoint matrix for every ordered

orthonormal basis of V .

Proof. Let B = (v1 , . . . , vn ) be an ordered orthonormal basis of V and let A = (aij ) = MBB (T ).

(only if ) T (vj ) = nk=1 akj vk . So

P

Xn

hT (vj ), vi i = h akj vk , vi i = aij .

k=1

Thus

n

X

aij = hT (vj ), vi i = hvj , T (vi )i = hvj , aki vk i = aji .

k=1

(if ) We are given that aij = aji . From the only if part we have

Pn Pn

Let x = j=1 aj vj and y = i=1 bi vi .

We have

X X X

hx, T (y)i = h aj vj , bi T (vi )i = aj bi hvj , T (vi )i,

j i j,i

X X X

hT (x), yi = h aj T (vj ), bi vi i = aj bi hT (vj ), vi i.

j i j,i

4.2. SPECTRAL THEOREM 69

Theorem 4.2.6. (Spectral Theorem for Self-Adjoint Operators) Let V be a finite dimen-

sional inner product space over F and let T : V → V be a self-adjoint linear operator. Then there

exists an orthonormal basis of V consisting of eigenvectors of T .

Proof. By the fundamental theorem of algebra and Theorem 4.2.3 (why?) there exists λ ∈ R and

0 6= v ∈ V with T (v) = λv. We may assume that v is a unit vector.

Put W = L({v})⊥ . We claim that

(i) w ∈ W implies T (w) ∈ W .

(ii) T : W → W is self-adjoint.

Proof of claim (i): We have

Proof of claim (ii): This is clear.

By induction on dimension, there is an orthonormal basis B of W consisting of eigenvectors of

T . Now {v} ∪ B is the required orthonormal basis of V .

Corollary 4.2.7. Let T : V → V be a self-adjoint linear operator on a finite dimensional inner

product space over F. Let µ1 , . . . , µk be the distinct (real) eigenvalues of T . Then the eigenspaces

Vµ1 , . . . , Vµk are mutually orthogonal and the sum of their dimensions is equal to the dimension of

V.

Proof. Exercise.

An n × n matrix U over C is said to be unitary if U ∗ U = U U ∗ = I, i.e., the columns (and

rows) of U are orthonormal vectors in Cn (why?).

An n × n matrix O over R is said to be orthogonal if Ot O = OOt = I, i.e., the columns (and

rows) of O are orthonormal vectors in Rn (why?).

Corollary 4.2.8. (Spectral Theorem for Real Symmetric matrices and Self-Adjoint

Matrices) (i) Let A be a n × n real symmetric matrix with (real) eigenvalues λ1 , . . . , λn . Set

D = diag(λ1 , . . . , λn ). Then there exists an n × n real orthogonal matrix O such that

Ot AO = D.

diag(λ1 , . . . , λn ). Then there exists an n × n unitary matrix U such that

U ∗ AU = D.

Proof. Exercise.

Here is a procedure for orthonormally diagonalizing a self-adjoint n × n matrix A:

(i) Find the eigenvalues of A. These should all be real. Let us call them µ1 , . . . , µk .

(ii) For each eigenvalve µi of A find a basis of Vµi = N (A − µi I) (this can be done with Gauss

elimination). Suppose this basis has di vectors. Then d1 + · · · + dk = n.

70 CHAPTER 4. EIGENVALUES AND EIGENVECTORS

(iii) Apply Gram-Schmidt process to the basis obtained in step (ii) to get an orthonormal basis

B(µi ), for all i.

(iv) Form an n × n matrix as follows: the first d1 columns are the (column) vectors in B(µ1 ) (in

any order), the next d2 columns are the vectors in B(µ2 ) (in any order), and so on. This is the

matrix U (in the complex case) or O (in the real case).

(v) The diagonal matrix D is the following: the first d1 elements on the diagonal are µ1 , the next

d2 elements are µ2 , and so on.

1 2 −2

A= 2 1 2 .

−2 2 1

The eigenvectors for λ = 3 (the null space of N (A − 3I)) are solutions of

−2 2 −2 x 0

2 −2 2 y = 0 .

−2 2 −2 z 0

V3 = {(y − z, y, z) | y, z ∈ R}

= L({u1 = (0, 1, 1)t , u2 = (−1, 1, 2)t }).

t r !t

1 1 2 1 1

v1 = 0, √ , √ and v2 = − , −√ , √ .

2 2 3 6 6

√ √ √

Similarly, check that the unit vector v3 = (1/ 3, 1/ 3, 1/ 3)t is an orthonormal basis of V−3 .

The orthogonal matrix O for diagonalization is O = [v1 , v2 , v3 ] and D = diag(3, 3, −3).

Lemma 4.2.10. A skew-self-adjoint matrix is orthonormally diagonalizable and its eigenvalues are

purely imaginary (i.e., of the form ia, for some real a).

Lemma 4.2.11. Let V be a n-dimensional complex inner product space. Let A, B be two commuting

self-adjoint operators on V , i.e., AB = BA. Then there exists an (ordered) orthonormal basis

(v1 , . . . , vn ) of V such that each vi is an eigenvector of both A and B.

V1 , . . . , Vt are mutually orthogonal and the sum of their dimensions is n.

4.2. SPECTRAL THEOREM 71

Thus, for all i, B : Vi → Vi is a self-adjoint operator. Hence each Vi has an orthonormal

basis consisting of eigenvectors of B and all of these vectors are already eigenvectors of A. That

completes the proof.

Theorem 4.2.12. (Spectral Theorem for Normal Matrices) A complex normal matrix is

orthonormally diagonalizable.

N + N∗ N − N∗

N= + .

2 2

Put A = (N + N ∗ )/2 and B = (N − N ∗ )/2. Check that A is self-adjoint, B is skew-self-adjoint,

and AB = BA.

Now C = iB is self-adjoint and AC = CA. By the previous lemma there is a common orthogonal

eigenbasis B of A and C. Since B = −iC, B is also an orthogonal eigenbasis of B and of N = A+B.

every eigenvalue λ of U satisfies |λ| = 1.

Let x, y ∈ Cn . Then

U x · U y = (U x)∗ U y = x∗ U ∗ U y = x∗ y = x · y.

Hence

kU xk2 = U x · U x = x · x = kxk2 .

Thus |λ| = 1.

- Gen inversionUploaded bySmiling Person
- Matrices Sssss From WikiUploaded bylux
- Std11 Maths EM 1Uploaded byHariprasad Manoharan
- Essentials of Robust ControlUploaded by王鸿博
- 03Uploaded byKunj Dipin Katyal
- Nla Teach DuttaUploaded bysachin2302
- Linear GuestUploaded byAbdulsalam
- 8 Sistem Persamaan Linier SPLUploaded byIgor M Farhan
- Operations on Vectors and MatricesUploaded bysamiseech
- Chapter_1-Slides(Sem 1 2013)Uploaded byVinoth Kumar
- Bca 1st Sem. MathsUploaded byNIRALI
- Lecture Notes MatricesUploaded byRR886
- matrix change basisbasisUploaded byMilli Yeshiwas
- notes.pdfUploaded byakhil akki
- INTERNET TVUploaded bysam
- MST-001Uploaded byRiya Gupta
- Matrices and System of EquationUploaded bySufi Jutawan Kosmetik
- mpdf (9).pdfUploaded byAzmat Farooq
- _14_ELC4340_Spring13_Power_System_Matrices.docUploaded byAnonymous AFFiZn
- HJS SmithUploaded byVũ Hải Lê
- GREProb.pdfUploaded bymurad
- Areas of HypersufacesjghjUploaded bymanny2011
- ExampleUploaded byMaria Ilia Yudi Rosita
- 6502handout9Uploaded byRoy Vesey
- 3. Matrix Algebra.pdfUploaded byHoney Grace Tre-inta Perez
- 5.MATRIX ALGEBRA.docUploaded byBalasingam Prahalathan
- Lab 2Uploaded byAnnas
- 4-47-nabiUploaded byEngr Rizwan Khattak
- NotUploaded byWorse To Worst Satittamajitra
- test3-98sUploaded byBeauponte Pouky Mezonlin

- Course OutlineUploaded bySuvajit Majumder
- hwk9solnUploaded byakhil akki
- 1212.6206Uploaded byakhil akki
- Are a Moment of InertiaUploaded byakhil akki
- notes.pdfUploaded byakhil akki
- notes1Uploaded byakhil akki
- CQ2 SolutionUploaded byakhil akki
- Rosat SpacecraftUploaded byakhil akki
- Impulse Turbine KrishnaUploaded byakhil akki

- Mom-curvUploaded byalexander1176
- Inverse[1]Uploaded byscribdsurya4u
- Z-matrix (mathematics).pdfUploaded bymars
- cis515-11-sl4.pdfUploaded bySufyan Sahoo
- Reflectores de HouseholderUploaded byPablo Burgos
- vectorUploaded byksuzuk
- lec7Uploaded bysuraj
- A norm inequality for pairs of commuting matrices.pdfUploaded byDanang Stwn
- Bài Tập Lớn Lê Bá Long DSTTUploaded byBuon Trong Bong Dem
- Tin-Yau Tam Multilinear Algebra [Lecture Notes]Uploaded byjhoan gaviria
- IGNOU Unit-9Uploaded bySajal Kumar
- Self Adjoint OperatorsUploaded bytheninjapirate
- EigenvectorsUploaded byDaniel Menéndez Crespo
- APMUploaded byAnindya Chatterjee
- 202_tut-9_2018Uploaded byTuanHung
- Definition of MatrixUploaded byGilbert Bersola
- Linear Algebra SummaryUploaded byManoj Ravi
- AMS 210- Applied Linear AlgebraUploaded byNick As Tello
- 07 ooo - The Square Root Function of a Matrix.pdfUploaded bycrisandy
- matrix review worksheetUploaded byapi-314332531
- Lecture Notes on Self-Adjoint OperatorsUploaded byAlex L Wang
- Convex Optimization — Boyd & VandenbergheUploaded byPremSuman8
- Matrix Appendix[Katsuhiko Ogata] System Dynamics (4th Edition)(B-ok.org)Uploaded byTracerLife
- Singh Studentproblems Chapter1Uploaded byHelbert Agluba Paat
- Engineering MathematicsUploaded bysangeethsreeni
- Matrix Estimate and the Perron-Frobenius TheoremUploaded byChenhui Hu
- Matrix factorizations and low rank approximationUploaded bytantantan
- Linear Algebra NotesUploaded byMaxwell Chomas
- Fallsem2017-18 Ece2006 Eth Tt631 Vl2017181000257 Reference Material i Dft-mkhUploaded byBharghav Roy
- Some Mathematical Preliminaries Linear Vector SpacesUploaded byjasmon