Vector spaces

2
Chapter 1

Vector Spaces

Throughout these notes F will denote a field. The reader unfamiliar with fields
should think of F as the rational numbers Q or the real numbers R or the complex
numbers C. Elements of F are called scalars.

1.1 Definition of a vector space

Definition 1.1.1. A vector space over a field F is a set V , whose elements are
called vectors, on which two operations, addition and scalar multiplication, are
defined.
Addition Given vectors x, y ∈ V we can add them and form the vector x+y ∈ V .
The addition operation is assumed to satisfy the following axioms.
1. (commutative law) x + y = y + x for all x, y ∈ V.
2. (associative law) x + (y + z) = (x + y) + z for all x, y, z ∈ V.
3. (existence of zero element) There exists an element 0 in V such that x + 0 =
0 + x = x for all x ∈ V.
4. (existence of inverse or negatives) For x ∈ V there exists an element written as
−x such that x + (−x) = 0.
Scalar multiplication Given a vector x ∈ V and a scalar α ∈ F we can multiply
x by α to form the vector αx ∈ V .
The scalar multiplication operation is assumed to satisfy the following axioms.
5. (associativity) For all α, β ∈ F, x ∈ V,
α(βx) = (αβ)x.

3
6. (distributive law for addition in V ) For all x, y ∈ V and α ∈ F,

α(x + y) = αx + αy.

7. (distributive law for addition in F) For all α, β ∈ F and x ∈ V,

(α + β)x = αx + βx

8. (existence of identity for multiplication) For all x ∈ V,

1x = x.

Remark 1.1.2. When F = R we say that V is a real vector space. If we replace

real numbers in the above definition by rational numbers or complex numbers
then we get the definition of a rational vector space or a complex vector space.
Definition 1.1.3. Let V be a vector space over F. A subset W ⊆ V is said to be a
subspace if
(i) 0 ∈ W .
(ii) u, v ∈ W implies u + v ∈ W .
(iii) u ∈ W, α ∈ F implies αu ∈ W .

Examples of Vector Spaces

In the examples below we leave the verification of the vector addition and
scalar multiplication axioms as exercises.
Example 1.1.4. 1. Set V = Fn = Set of all column vectors with n compo-
nents belonging to F. Then, under component wise addition and scalar
multiplication by elements of F, V is a vector space over F. We can simi-
larly define a vector space of row vectors. Unless stated otherwise Fn refers
to the space of column vectors.
The set W = {(x1 , . . . , xn )t ∈ Fn | x1 + · · · + xn = 0} is a subspace of V .
Note that, by restricting the scalars to the reals, we can also treat Cn as a
real vector space.

2. Let A be an m × n matrix over F. Then

{x ∈ Fn | Ax = 0}

is a subspace of Fn , called the null space of A and denoted N (A), and

{x ∈ Fm | x = Ay for some y ∈ Fn }

4
 is a subspace of Fm , called the column space of A and denoted C(A).
The row space of A, denoted R(A), is defined to be the subspace
{xt | x ∈ C(At )}
of Fn (now treated as row vectors).
3. Let V = Mm×n (F) denote the set of all m × n matrices with entries in F.
Then V is a vector space over F under usual matrix addition and multipli-
cation of a matrix by a scalar. We set Mn (F) = Mn×n (F).
Recall that, for a square matrix A, tr(A) denotes the sum of the diagonal
entries of A. The set W = {A ∈ Mn (F) | tr(A) = 0} is a subspace of
Mn (F).
4. Let S be a set. The set of all functions V = {f : S → F} is a vector space
over F under pointwise addition and scalar multiplication, i.e., for f, g ∈ V
(f + g)(s) = f (s) + g(s), s ∈ S,
(αf )(s) = αf (s), α ∈ F, s ∈ S.

5. Let a < b be real numbers and set S = [a, b]. Let V be the real vector space
of all real valued functions on S as in the example above. The set C[a, b] of
continuous real valued functions on S is a subspace of V . Another subspace
of V is the set of all functions in V differentiable at a fixed s ∈ S.
6. A function f : F → F is a polynomial with coefficients in F if there exist
scalars a0 , . . . , ak ∈ F (called the coefficients of f ) such that
f (s) = a0 + a1 s + a2 s2 + · · · + ak sk , s ∈ F.
If the function f is the zero function then all the coefficients ai are zero (see
the exercises). This result implies that the coefficients of a polynomial are
uniquely determined (why?).
If a polynomial f can be written in the form above with ak 6= 0 then we
say that f has degree k. We define the degree of the zero polynomial to be
−∞.
The set
P (F) = {a0 + a1 s + a2 s2 + · · · | a0 , a1 , a2 , . . . ∈ F}
of all polynomials is a vector space over F, under usual addition of polyno-
mials and multiplication of polynomials with scalars, and the set
Pn (F) = {a0 + a1 s + · · · + an sn | a0 , a1 , . . . , an ∈ F}
of all polynomials with degree at most n is a subspace of P (F).

5
 00 0
7. The set of all solutions to the differential equation y + ay + by = 0 where
a, b ∈ R form a real vector space. More generally, in this example we can
take a = a(x), b = b(x) to be suitable functions of x.

Two vector spaces V, W over F are said to be isomorphic if there is a bijective

map f : V → W such that f (x + y) = f (x) + f (y) and f (αx) = αf (x) for all
x, y ∈ V and α ∈ F. In other words, if we identify x ∈ V with f (x) ∈ W then
V and W are essentially the same vector spaces. For example, the vector space of
column vectors Fn and matrices Mm×n (F) is easily seen to be isomorphic to the
space of functions on {1, 2, . . . , n} and {1, 2, . . . , m}×{1, 2, . . . , n} respectively.
However, isomorphisms can sometimes be subtle and two very different looking
vector spaces may be isomorphic. Consider the real vector space V of all solutions
to the differential equation
d2
x + x = 0.
dt2
Once the initial values x(0) = a and dtd x(0) = b are known the solution is com-
pletely determined for all t (this
 requires a proof). So every element of V can be
a
described by an element of R2 . It can be checked that this mapping from V
b
to R2 is an isomorphism.

1.2 Basis and dimension

Let V be a vector
Pspace over F. Let v1 , . . . , vn be vectors in V and let c1 , . . . , cn ∈
n
F. The vector i=1 ci vi ∈ V is called a linear combination of vi ’s and ci is
called the coefficient of vi in this linear combination.
Definition 1.2.1. Let S be a subset of a vector space V over F. The linear span
of S is the subset of all vectors in V expressible as linear combinations of finite
subsets of S, i.e.,
( n )
X
L(S) = ci vi |n ≥ 0, v1 , v2 , . . . , vn ∈ S and c1 , c2 , . . . , cn ∈ F
i=1

The empty sum of vectors is the zero vector. Thus L(∅) = {0}. We say that L(S)
is spanned by S.

The linear span L(S) is actually a subspace of V . In fact, we have

Lemma 1.2.2. The smallest subspace of V containing S is L(S), i.e., L(S) is a
subspace and any subspace of V containing S contains L(S).

6
Proof. Note that L(S) is a subspace (why?). Now, if S ⊂ W ⊂ V and W is a
subspace of V then by L(S) ⊂ W (why?). The result follows. 2

Example 1.2.3. 1. Different sets may span the same subspace. For example,
let ei , i = 1, . . . , n denote the standard vectors in Fn (i.e., ei is the column
vector with 1 in the ith coordinate and 0’s elsehwere). Then (check this)

L({e1 , e2 , . . . , en }) = L({e1 , e1 + e2 , . . . , e1 + e2 + · · · + en }) = Rn .

The vector space Pn (R) is spanned by {1, t, t2 , . . . , tn } and also by {1, (1 +

t), . . . , (1 + t)n } (why?).

2. Let A be an m × n matrix over F, with rows R1 , . . . , Rm and columns

C1 , . . . , Cn . Then R(A), the row space of A, is the subspace of Fn (treated
as row vectors) spanned by the rows of A, and C(A), the column space of
A, is the subspace of Fm spanned by the columns of A.

3. Let A, B, C be matrices such that AB and CA are defined. Then (why?)

C(AB) ⊆ C(A) and R(CA) ⊆ R(A).

We have introduced the notion of linear span of a subset S of a vector space.

This raises some natural questions:
(i) Which spaces can be spanned by finite number of elements ?
(ii) If a vector space V = L(S) for a finite subset S of V then what is the size
of smallest such S?
To answer these questions we introduce the notions of linear dependence and
independence, basis and dimension of a vector space.

Definition 1.2.4. Let V be a vector space. A subset S ⊂ V is called linearly

dependent if there exist finitely many distinct elements v1 , v2 , . . . , vn ∈ S (for
some n ≥ 1) and scalars α1 , α2 . . . , αn not all zero such that

α1 v1 + α2 v2 + . . . + αn vn = 0

A set S is called linearly independent if it is not linearly dependent, i.e., for

all n ≥ 1 and for all distinct v1 , v2 , . . . , vn ∈ S and scalars α1 , α2 , . . . , αn

α1 v1 + α2 v2 + . . . + αn vn = 0 implies αi = 0 for all i.

Note that the empty set is linearly independent.

7
Remark 1.2.5. (i) Any subset of V containing a linearly dependent set is linearly
dependent.
(ii) Any subset of a linearly independent set in V is linearly independent.
Example 1.2.6. (i) If a set S contains the zero vector 0 then S is dependent since
1.0 = 0.
(ii) Consider the vector space Fn and let S = {e1 , e2 , . . . , en }. Then S is
linearly independent. Indeed, if α1 e1 + α2 e2 + . . . + αn en = 0 for some scalars
α1 , α2 , . . . , αn then (α1 , α2 , . . . , αn )t = 0. Thus each αi = 0 and S is linearly
independent.
Check that the set S = {e1 , e1 + e2 , . . . , e1 + e2 + · · · + en } is linearly inde-
pendent.
(iii) Let V be the vector space of all continuous functions from R to R. Let
S = {1, cos2 t, sin2 t}. Then the relation cos2 t + sin2 t − 1 = 0 shows that S is
linearly dependent.
(iv) Let α1 < α2 < · · · < αn be real numbers. Let V = {f : R →
R | f is continuous}. Consider the set S = {eα1 x , eα2 x , . . . , eαn x }. We show
that S is linearly independent by induction on n. Let n = 1 and βeα1 x = 0. Since
eα1 x 6= 0 for any x, we get β = 0. Now assume that the assertion is true for n − 1
and
β1 eα1 x + . . . + βn eαn x = 0.
Then β1 e(α1 −αn )x + · · · + βn e(αn −αn )x = 0
Let x → ∞ to get βn = 0. Now apply induction hypothesis to get β1 = · · · =
βn−1 = 0.
(v) In the vector space P (R) of all polynomials p(t) with real coefficients the
set S = {1, t, t2 , . . .} is linearly independent. Suppose that 0 ≤ n1 < n2 < . . . <
nr and
α1 tn1 + α2 tn2 + . . . + αr tnr = 0
for certain real numbers α1 , α2 , . . . , αr . Differentiate n1 times to get α1 = 0.
Continuing this way we see that all α1 , α2 , . . . , αr are zero.
Lemma 1.2.7. (Exchange lemma) Let S = {v1 , v2 , . . . , vk } be a subset of a
vector space V. Then any k + 1 elements in L(S) are linearly dependent.

Proof. Suppose T = {w1 , . . . , wn } are linearly independent vectors in L(S). We

shall show that n ≤ k.
We shall construct a sequence of sets
S = S0 , S1 , . . . , Sn

8
such that
(i) each Si spans L(S), i = 0, 1, . . . , n.
(ii) |Si | = k, i = 0, 1, . . . , n.
(iii) {w1 , . . . , wi } ⊆ Si , i = 0, 1, . . . , n.
We shall produce this sequence of sets inductively, the base case i = 0 being
clear. Now suppose we have sets S0 , . . . , Sj satisfying (i), (ii), (iii) above, for
some j < n.
Since Sj spans L(S) we can write
X
wj+1 = cs s,
s∈Sj

for some scalars cs , s ∈ S. Since w1 , . . . , wj+1 are linearly independent there

exists t ∈ Sj − {w1 , . . . , wj } with ct 6= 0 (why?). It follows that
 
1 X
t = wj+1 − cs s 
ct
s∈Sj −{t}

and hence the set (Sj − {t}) ∪ {wj+1 } satisfies conditions (i), (ii), and (iii) above
for i = j + 1. That completes the proof. 2
Example 1.2.8. 1. Let A be a m × n matrix with n > m. Then there exists a
nonzero column vector x such that Ax = 0. That is, homogeneous linear equa-
tions with more variables tha equations always have a nontrivial solution.
Indeed, let a1 , a2 , . . . , an ∈ Fm be the columns of A. Since Fm is spanned
by {e1 , . . . , em } and n > m we have from the Theorem above that a1 , . . . , an
are linearly dependent. So there are scalars x1 , . . . , xn , not all zero, such that
x1 a1 + x2 a2 + · · · + xn an = 0. Thus Ax = 0 where x = (x1 , . . . , xn )t .
2. Let A, B be matrices of order 9 × 7 and 4 × 3 respectively. We claim that
there is a nonzero 7 × 4 matrix X such that AXB = 0 (the zero matrix). Think
of the entries of X as variables. So there are 28 variables. Then AXB = 0 is
equivalent to a system of 27 homogeneous linear equations in these 28 variables.
Thus there is a nonzero solution.
Definition 1.2.9. A subset S of a vector space V is called a basis of V if elements
of S are independent and V = L(S). A vector space V possessing a finite basis is
called finite dimensional. Otherwise V is called infinite dimensional.
Theorem 1.2.10. Any two bases of a finite dimensional vector space have same
number of elements.

9
Proof. Suppose S and T are bases of a finite dimensional vector space V. Suppose
|S| < |T |. Since T ⊂ L(S) = V, T is linearly dependent. This is a contradiction.
2
Definition 1.2.11. The number of elements in a basis of a finite-dimensional vec-
tor space V is called the dimension of V. It is denoted by dim V.
Example 1.2.12. (1) The n coordinate vectors e1 , e2 , . . . , en in Fn form a basis of
Fn .
(2) Let A be a n × n matrix. Then the columns of A form a basis of Fn iff A
is invertible. (why?)
(3) Pn (F) = {a0 + a1 t + . . . + an tn | a0 , a1 , . . . , an ∈ F} is spanned by
S = {1, t, t2 , . . . , tn }. Since S is independent, dim Pn (R) = n + 1.
(4) Consider the vector space Mm×n (F). Let Eij denote the m × n ma-
Pm1 in
trix with Pn(i, j) position and 0 elsewhere. If A = (aij ) ∈ Mm×n (F) then
A = i=1 j=1 aij Eij . It is easy to see that the mn matrices Eij are linearly
independent. Hence Mm×n (F) is an mn−dimensional vector space.
Lemma 1.2.13. Suppose V is a finite dimensional vector space. Let S be a lin-
early independent subset of V . Then S can be enlarged to a basis of V .

Proof. Suppose that dim V = n and S has less than n elements. Let v ∈ V \L(S).
Then S ∪ {v} is a linearly independent subset of V (why?). Continuing this way
we can enlarge S to a basis of V . 2
Theorem 1.2.14. Let A be a m × n matrix. Then dim R(A) = dim C(A) and this
number is called the rank of A, denoted rank(A).

Proof. Let r be dim C(A). Then there are m × 1 column vectors v1 , v2 , . . . , vr

that form a basis for the column space of A. Form a m × r matrix C with columns
v1 , . . . , vr .
For each 1 ≤ j ≤ n, there exists (why?) a r × 1 column vector uj such that
the jth column of A is equal to Cuj .
Form a r × n matrix B with columns u1 , . . . , un . Then (why?) A = CB.
Now the row space of A is contained in the row space of B and so dim R(A)
is ≤ r = dim C(A).
Applying this argument to the transpose of A shows that dim C(A) is ≤ dim R(A).2
Let V1 , . . . , Vk be subspaces of a vector space V . Define

V1 + V2 + · · · + Vk = {v1 + v2 + · · · + vk | vi ∈ Vi for all i}.

10
It is easy to check that V1 + · · · + Vk is a subspace of V , called the sum of
V1 , V2 , . . . , Vk .
If each vector v ∈ V1 + · · · + vk can be written uniquely as v = v1 + · · · + vk ,
vi ∈ Vi for all i then we say that the sum is direct and write

V = V1 ⊕ V2 ⊕ · · · ⊕ Vk .

Example 1.2.15. (i) Let V1 be the subspace of Fn consisting of all vectors (x1 , . . . , xn )t
with x1 +· · ·+xn = 0 and let V2 be the subspace spanned by the vector (1, 1, . . . , 1)t .
Then Fn = V1 ⊕ V2 .
Clearly V1 ∩ V2 = {0} so it is enough to show that Fn = V1 + V2 . This follows
from (why?)

(x1 , . . . , xn )t = (x1 − α/n, . . . , xn − α/n)t + α/n(1, . . . , 1)t ,

where α = x1 + · · · + xn .
(ii) Let V1 be the subspace of Mn (F) consisting of all matrices of trace zero
and let V2 be the subspace spanned by the identity matrix. Then (why?)

Mn (F) = V1 ⊕ V2 .

11