Inner Product Spaces

Where normed vector spaces incorporate the concept of length into a vector
space, inner product spaces incorporate the concept of angle.
Definition 1. Let V be a vector space over K. An inner product is a function
h, i : V V K such that for all x, y, z V, K

IP1. hx, yi = hy, xi

IP2. hx, yi = hx, yi
IP3. hx + y, zi = hx, zi + hy, zi

IP4. hx, xi 0 with equality iff x = 0.

A vector space together with an inner product is called an inner product space.

Examples:
PN
V = CN , hx, yi := i=1 xi yi = y x
Rb
V = C[a, b], hx, yi := a
x(t)y(t)dt
Note that p
a valid inner product space induces a normed vector space with
norm kxk = hx, xi. (Proof relies on Cauchy-Schwartz inequality.) In RN
or CN , the standard inner product induces the `2 -norm. We summarize the
relationships between the various spaces introduced over the last few lectures
in Fig. 1.

vector metric
spaces spaces

normed vector spaces

inner product spaces

Figure 1: Venn diagram illustrating the relationship between vector and metric
spaces.

1
Properties of inner products
Inner products and their induced norms have some very useful properties:
Cauchy-Schwartz Inequality: |hx, yi| kxk kyk with equality iff C
such that y = x
2 2 2 2
Pythagorean Theorem: hx, yi = 0 kx + yk = kx yk = kxk + kyk
2 2 2 2
Parallelogram Law: kx + yk + kx yk = 2 kxk + 2 kyk
kx+yk2 kxyk2
Polarization Identity: Re[hx, yi] = 4

In R2 and R3 , we are very familiar with the geometric notion of an angle

between two vectors. For example, if x, y R2 , then from the law of cosines,
hx, yi = kxk kyk cos . This relationship depends only on norms and inner prod-
ucts, so it can easily be extended to any inner product space.

Definition 2. The angle between two vectors x, y in an inner product space

hx,yi
is defined by cos = kxkkyk
Definition 3. Vectors x, y in an inner product space are said to be orthogonal
if hx, yi = 0.

Complete Vector Spaces

Definition 4. A complete normed vector space is called a Banach space.

Examples:
C[a, b] with L norm, i.e., kf k = ess sup|f (t)| is a Banach space.
t[a,b]

Lp [a, b] = {f : kf kp < } for p [1, ] and a < b is a

Banach space.
`p (N) = {sequences x : kxkp < } for p [1, ] is a Banach space.
Any finite-dimensional normed vector space is Banach, e.g., RN or CN
with any norm.
C[a, b] with Lp norm for p < is not Banach.

2
Definition 5. A complete inner product space is called a Hilbert space.

Examples:
L2 [a, b] is a Hilbert space.
`2 (N) is a Hilbert space.

Any finite-dimensional inner product space is a Hilbert space.

Note that every Hilbert space is Banach, but the converse is not true. Hilbert
spaces will be extremely important in this course.