You are on page 1of 36

Inner Product Space (IPS)

Sub Chapter
Definitions
Orthonormal
Gramm Schmidt

Applications :
- least square method
- Control Theory.

30/11/2014 21:38

Definition
Let V be a vector space and u , v V
Then < , > is called inner product
If hold the following aksiom :
1.

u, v v, u

(Simetry)

2.

u v, w u , w v, w

(Aditivity)

3.

For some kR, k u , v u, k v k u , v

(Homogenity)
4.

u , u 0 , for every u

(Sifat Positivity)
and u , u 0 u 0
Vector space with inner product operations
Is called Inner Product Space (IPS)
30/11/2014 21:38

Let V be a IPS,
Norm (length) of a vector u :

u u ,u

12

(u u )

12

2
u1

u2 ... un 0
2

The angle of two vectors :


u, v
cos
u v
Example 1 :
Euclidean IPS ( Rn )
Let u , v Rn then u , v u1v1 u 2 v 2 ... u n v n
u

u, u

= (u12 + u22 + ..+un2)


30/11/2014 21:38

Example 2 :

Let W R3 with yhe inner product

u , v 2u1v1 u2 v2 3u3v3

where u , v W
Show that W is IPS
answer :
Let

u , v , w W

u, v 2u1v1 + u2v2 + 3u3v3


= 2 v1u1 + v2u2+ 3v3u3

v, u

30/11/2014 21:38

(simetry)

(ii ) u v , w <(u1+v1, u2+v2, u3+v3), (w1, w2, w3)>

= 2(u1+ v1)w1 + (u2+v2)w2 + 3(u3+v3)w3


= 2u1w1+2v1w1+u2w2 +v2w2+3u3w3+3v3w3
= 2u1w1+u2w2+3u3w3+2v1w1+v2w2+3v3w3
u, w v, w

(aditivity)

(iii) For some kR,


k u , v <(ku1, ku2, ku3), (v1, v2, v3)>
= 2ku1v1 + ku2v2 + 3ku3v3
= k2u1v1 + ku2v2 + k3u3v3

= k(2u1v1 + u2v2 + 3u3v3)


k u , v u, k v
(homogenity)
30/11/2014 21:38

(iv ) u , u 2u1 u2 3u3


2

It is clear that u , u 0 for every u


and u , u 0 only if u 0

Example 3:
Show that u , v u1v1 2u2v2 3u1v1
is not IPS
Answer :
We Know
2
2
2
u , u u1 2u2 3u3

When 3u32 > u12 + 2u22


Then u , u 0

30/11/2014 21:38

Not positivity

Example 4:
If u (1,0) and v (0,1)
Find u and d (u , v )
a. In R2 euclid
b. In weight euclid : u , v 3u v 2 u v
Answer :

a. u 12 02 1
d (u , v ) u v (1, 1) 12 (1) 2 2

b. u (3.1.1) (2.0.0) 3
d (u , v ) u v (1, 1) (3.1.1) (2. 1. 1) 5
30/11/2014 21:38

If u, v, and w are any three vectors in IPS, and


k scalar, then:
a)
b)
c)
d)
e)

< 0, v > = < v, 0 >


< u, v + w > = < u, v > + < u, w >
< u, kv > = k < u, v >
< u v, w > = < u, w > - < v, w >
< u, v - w > = < u, v > - < u, w >

30/11/2014 21:38

Example :
< u 2v, 3u + 4v >
= < u, 3u + 4v > - < 2v, 3u + 4v >
= < u, 3u > + < u, 4v > - < 2v, 3u > - < 2v, 4v >
= 3 < u, u > + 4 < u, v > - 6 < v, u > - 8 < v, v >
= 3 ||u||2 + 4 < u, v > - 6 < u, v > - 8 ||v||2
= 3 ||u||2 - 2 < u, v > - 8 ||v||2

30/11/2014 21:38

Exercise :
If < u,v > = 2, < v,w > = -3, < u,w > = 5
||u|| = 1, ||v|| = 2, and ||w|| = 7
Find the expressions :
a. < u+v, v+w >
b. < 2v-w, 3u+2w >
c. < u-v-2w, 4u+v >

30/11/2014 21:38

Orthonormal Set
A Set in the inner product space is called
orthogonal set
IF every couple of vectors on the set is orthogonal.
Orthonormal set orthogonal set which each vector
has norm equal to one.
Let, T c1 , c 2 ,..., c n in the IPS,
T is called orthogonal set if
ci , c j 0 for every i, j =1,2,... n and i j
T is called a ortonormal set
if ci 1 for every i

30/11/2014 21:38

Example :
1. A 1

-1

0 , 0

in the Euclidean IPS, A is not orthogonal set.


2.

1 0
B
0 , - 1

in the Euclidean IPS, B is orthonormal set.

30/11/2014 21:38

Let S v1 , v 2 ,..., v n
Is a orthonormal basis for a IPS V
If u is a vector inV,
then u u , v1 v1 u , v 2 v2 ... u , v n v n
Proof : u k1v1 k 2 v 2 ... k n v n
For some i , we have :

u , vi k1v1 k 2 v 2 ... k n v n , vi
k1 v1 , vi k2 v2 , vi ... ki vi , vi ... kn vn , vi

Because S is a orthonormal set

vi , v j 0 for every i j
30/11/2014 21:38

and vi , vi

1 for every i

Such, for every i, we have

u , vi ki

linear combination u k1v1 k 2 v 2 ... k n v n


Can be written :

u u , v1 v1 u , v 2 v2 ... u , v n v n
Example :

1
Let u in Euclidean IPS S .
2
1

2
S is spanned by v1
1

2
and v2
1

Write u as linear combination of v1 and v2


30/11/2014 21:38

Answer :
Remember ..
{u , v} is a orthonormal basis

u k1v1 k2v2
u u , v1 v1 u , v2 v2
1
1
u ,
2
2
u

3
2

v1

1
1

1
2

1
2
2
v1 ,
v2
1

2
2

(u ) s ( u , v1 , u , v2 ) ( 3

)
2

(u ) s is coordinate vector from u relative to S


30/11/2014 21:38

Exercise
If :
v1 = (0, 1, 0),
v2 = (-4/5, 0, 3/5),
v3 = (3/5, 0, 4/5)
S = {v1 , v2 , v3 } is ortonormal basis in IPS R3
a. Write u = (1, 1, 1) as a linear combination from S
b. Find (u ) s and ||u||

30/11/2014 21:38

Let S v1 , v 2 ,..., v n
Is a orthogonal basis for a IPS V
If u is a vector inV,
then
vn
vn
v1
v1
v2
v2
u u,

u,

... u ,

v1
v1
v2
v2
vn
vn
u

u , v1
v1

30/11/2014 21:38

v1

u , v2
v2

v2 ...

u , vn
vn

vn

ORTHOGONAL PROJECTION
Definition

A set of vectors in an inner product space is called an orthogonal set if all pairs of
distinct vectors in the set are orthogonal. An orthogonal in which each vectors has
norm 1 is called orthonormal set.

Orthogonal Projection
Let W be a subspace of an inner product space V, if {s1,s2,,sr } is orthonormal
basis for W and a any vectors in V, then
Remember

projb a

a,b
b

30/11/2014 21:38

projW a projs a projs a ... projs a


1

projW a a, s1 s1 a, s2 s2 ... a, sn sn

ORTHOGONAL PROJECTION
Orthogonal Projection
Let W be a subspace of an inner product space V, if {s1,s2,,sr } is orthogonal
basis for W and a any vectors in V, then

projW a projs a projs a ... projs a

Remember

projb a

a,b
b

30/11/2014 21:38

projW a

a, s1
s1

s1

a, s2
s2

s2 ...

a, sn
sn

sn

ORTHOGONAL
PROJECTION
Example
Let R2 have the Euclidean inner product and W be the subspace has
orthonormal basis:

a 0,1,0,

4 3
b ,0,
5 5
Find component of u = (1,1,1) orthogonal to each vector in the space W
Solution

1 4
5 5

3
5

projW u u , a a u , b b 1.( 0,1,0) ,0,


Component of u orthogonal to each vector in the space W is

u projW u ...

30/11/2014 21:38

1
21,0,28
25

1
4,1,3
25

Gramm-Schmidt Process

S c1 , c2 ,

cn

B w1 , w2 , ... , wn

a basis for IPS V

orthonormal basis of V

The following squence of steps as Gramm Schmidt


Process
1. w1

c1
c1

30/11/2014 21:38

2.

w2

c2

c2

q1
w2

p1 proyw c2
1

c2 , w1 w1
w1

c2 , w1 w1

q1 c2 p1

w1

w2

p1

c 2 c 2 , w1 w1
c 2 c 2 , w1 w1

30/11/2014 21:38

q1 c2 c2 , w1 w1

Unit vector in direction

q1

3. c 3

w3

wi

c3

q2

ci projW i 1ci
ci projW i 1ci

Wi-1 : subspace spanned by W1,,Wi-1

w3

p2

w1

w2

p 2 proyW c3 c3 , w1 w1 c3 , w2 w2
q2 c3 p2
q2 c3 ( c3 , w1 w1 c3 , w2 w2 )

c3 ( c3 , w1 w1 c3 , w2 w2 )
w3
c3 ( c3 , w1 w1 c3 , w2 w2 )
30/11/2014 21:38

Unit vector which


orthogonal with W

Example :
Let :

1
0
0


B u1 1, u 2 1 , u 3 0
1
1
1

be basis of Euclidean IPS in R3.


Find the Orthonormal basis
Answer :
Step 1.
u1
1, 1, 1
v1

u1
3

30/11/2014 21:38

3
1

3
1

Step 2
u2 proyv1 u2
v2
u2 proyv1 u2
u 2 proyv1 u 2 u 2 u 2 , v1 v1
2 1
1
1

0, 1, 1
,
,

3 3
3
3
2 1 1
, ,
3 3 3

We obtained

Hence,
u 2 proyv1 u 2

Such :

30/11/2014 21:38

v2

6
1

6
1

4
9

19 19

6
3

Step 3
u3 proyW u3
v3
u3 proyW u3

Then
u3 proyW u3 u3 u3 , v1 v1 u3 , v2 v2
1 1 1 1
1 2 1 1

0, 0, 1
,
,
,
,
3 3 3 3
6
6 6 6
1 1

0, ,
2 2

We get :

30/11/2014 21:38

1
v3 2
1
2

So,

v1, v2 , v3 =

1
3
1
3
1
3

2 0
6

, 16 , 12
1 1
6 2

Is a basis ortonormal for vector space R3


With Euclidean IPS

30/11/2014 21:38

Example :

1 0

Let plane shape which spanned by 0 , 1
1 1

is subspace of Euclidean IPS in R3

Find orthogonal projection of vector

1

u 1
1

To the plane shape.

30/11/2014 21:38

Answer :

Let

1
0


v1 0 ,v 2 1
1
1

be basis of Subspace of IPS

Because v1 , v 2

is linear independent
1st Step :
Basis orthonormal basis .
30/11/2014 21:38

v
w1 1
v1

1 , 0 ,1
12 02 12
1 , 0 ,1

1
1

,0 ,

2
2

So

1
1
v2 , w1 0 ,1 ,1
,0 ,

2
2
1
00
2
1

30/11/2014 21:38

Sehingga:

1 1
1
v2 , w1 w1

,0 ,

2 2
2
1
1
,0 ,
2
2

1
1
v2 v2 , w1 w1 0 ,1 ,1 , 0 ,
2
2
1
1
,1 ,
2
2

We obtained :
2

1
1
v2 v2 , w1 w1 12
2
2

30/11/2014 21:38

1
1
1
4
4

6
4

1
6
2

We get
v2 v2 , w1 w1
w2
v2 v2 , w1 w1
1
1
,1 ,
2
2

1
6
2
1
2 1
=

,
,
6
6
6

The Orthonormal Basis :

30/11/2014 21:38

2
0 ,
1
2

Orthogonal Projection of Vector


On W is :

1

u 1
1

Pr oy W u u , w1 w1 u , w2 w2

So,
1
1
u , w1 1 ,1 ,1
,0 ,

2
2
and
1
1

0
1
2 1
2
2

u , w2 1 ,1 ,1
,
,
6
6
6

1
2
1
2
2

2
30/11/2014 21:38

Pr oy W u u , w1 w1 u , w2 w2
1

= 0
1

30/11/2014 21:38

3
2
3
4

1

3
2

3
1

Exercise
1. Check that this operations is a Inner product or no
a. u,

= u12v1 + u2v22

in R2

b. u, v = u1v1 + 2u2v2 u3v3 in R3


c. u, v = u1v3 + u2v2 + u3v1 in R3
2. Find k such that vector (k, k, 1)
and vector (k, 5, 6 ) is orthogonal
in the euclidean !

30/11/2014 21:38

3. Let W be subspace of 3 with euclidean IPS


and span by
1

1
0

1

and 0
1

1

Find a orthogonal projection 1 on W
2

30/11/2014 21:38