Department of Aerospace Engineering

AE602 Mathematics for Aerospace Engineers

Assignment No. 6
6.1 Find the best least squares solution to 3 = 10, 4 = 5. What error 2 is minimized?
Check that the error vector (10 3, 5 4 ) is perpendicular to the column 3,4 .
SOLUTION:

2 = (10 3)2 + (5 4)2

2

= 6 10 3 8(5 4)

Putting the derivative equal to zero, we are minimising the error and the 'x' corresponding
to this becomes the best least squares solution .
Thus 6 10 3 + 8 5 4 = 0
=
Thus = is the best least squares solution.
Now moving to the second part of the problem, we know that the error vector is (10 3,
5 4 ) and =
Hence =

4
3

If = 0, then the error vector (10 3, 5 4 ) is perpendicular to the column 3,4

of 'A'.
= 4

3
=0
4

Since the above inner product is zero, 'e' and 'A' are perpendicular.

6.2 Suppose the values 1 = 1 and 2 = 7 at times 1 = 1 and 2 = 2 are fitted by a line =
through the origin. Solve = 1 and 2 = 7 by least squares, and sketch the best line.
SOLUTION:
Given are two line equations = 1 and 2 = 7. Writing them in matrix form
1
1
=
which is of the form Ax = b
2
7

The normal equation is given by = where is the best least squares

solution.
Thus substituting A, x and b in the above normal equation, we get
1 2

1
1
= 1 2
2
7
5 = 15

= is the best least squares solution and = is

the corresponding best fit line through the origin.
6.3 Solve = by least squares and find = if
1 0
1
= 0 1 ,
= 1 .
1 1
0
Verify that the error is perpendicular to the columns of .
SOLUTION:
1 0
1

Given = 0 1 , = 1 and let = 1

2
1 1
0
The best least squares solution is obtained from the normal equations =
1 0
1 0 1
1 0
1
0 1 =
0 1 1
0 1
1 1 2

1
1
1
1
0

2 1 1
1
=
1 2 2
1
21 + 2 = 1 ..........1
1 + 22 = 1 ..........2
Solving equations 1 and 2, we get

which is the best least squares solution.

=

To find = :

p=

To verify that the error is perpendicular to the columns of :

1
3
1
1
Error, b - p = 1
3
0
2
3

e=

1 = 2 3

2 = 2 3

1
2 3 0 = 0
1

0
2 3 1 = 0
1

Since the above two inner products equal zero, the error (b - p) is perpendicular to the
columns of 'A'.
6.4 Write out 2 = = and set to zero its derivatives with respect to and , if
1 0
1

= 0 1 , =
,
= 3 .

1 1
4
Compare the resulting equations with = , confirming that calculus as well as
geometry gives the normal equations. Find the solutions and the projection = . Why is
= ?
SOLUTION:
1
= 0
1

0
1 ,
1

=
, = 3

1 0
1
Therefore Ax = b implies 0 1
= 3

1 1
4
The equations obtained are
u=1
v=3
u+v = 4
The error is given by 2 = =

= ( 1)2 + ( 3)2 + ( + 4)2

Minimizing the error,

2

=2 1 +2 +4 =0
2 + = 5 ............1

=2 3 +2 +4 =0
+ 2 = 7

.............2

The normal equations are given by =

1 0
1 0 1
1 0
0 1 =
0 1 1
0 1
1 1

1
1
3
1
4

2 1
5
=
1 2
7
2 + = 5 ...........3
+ 2 = 7 ...........4
Comparing equations 1&2 with equations 3&4, we can confirm that calculus as well
as geometry gives the normal equations.
To find the best least squares solution :
Solving equations 3&4, we get

1
=
is the best least squares solution

p=

= =

Now solving the given system of equations without least squares,

1
0
1
1 0 + 2 1 = 3
1
1
4
=
=
+ =
Thus the given system of equations are consistent and hence 'b' lies in the column space of
'A'. Hence the projection of 'b' (which is already in the column space of 'A') onto the
column space of 'A' results in 'b' itself. Hence p=b.

6.5 The following system has no solution:

4
1 1

= 1
= 5 = .
0

1
1
9
Sketch and solve a straight line fit that leads to the minimization of the quadratic
4 2 + ( 5)2 + ( + 9)2 . What is the projection of onto the column space of ?
SOLUTION:

Given the error is 2 = 4

+ ( 5)2 + ( + 9)2

Minimizing the error,

2

= 2 4 + 2 5 + 2( + 9) = 0
=

= 2 4 + 2( + 9) = 0

=
Now = + is the equation of the best fit line .

Hence the best fit line equation is = +

Projection of 'b' onto the column space of 'A' is =

6.6 Find the projection of onto the column space of :

1
1
1
= 2 1 ,
= 2
2
4
7
Split into + , with in the column space and perpendicular to that space. Which of the
four subspaces contains ?
SOLUTION:
The projection of onto the column space of is given by =
The normal equations are given by =
1 2 2
1 1 4
We get

1
2
2

= =

1
1
1 2 2
1
=
1
2
1 1 4
2
4
7
as the best least squares solution
1
1
3
1
=
2 1
0
2
2
4
6

Error is given by e = b - p =

2
2
1

Now = + where 'p' is in the column space of 'A' .

Now consider = 2 2

Also

3
1 0 = 0. Hence 'e' is perpendicular to 'p' .
6

1 = 2 2 1

1
2 =0
2

1
2 = 2 2 1 1 = 0
4
Hence 'e' is perpendicular to the space containing 'p'.

3
2
Thus here q = e and = + = 0 + 2
6
1
with in the column space and (= ) perpendicular to that space.
The space which is perpendicular to the column space is the left Null space. Hence 'q' is in
the left Null space of 'A'.
1
1 2 2
Left Null space of 'A' is found by setting = =
2
1 1 4
3

Solving the above system of equations yields = which is the basis for the left Null

space of 'A' and it is exactly the error vector 'e'. Hence 'e' is in the left Null space of 'A'.

6.7 Find the projection matrix onto the space spanned by 1 = (1, 0, 1) and 2 = 1, 1, 1 .
SOLUTION:
Let A =

1
0
1

1
1
1

The projection matrix 'P' is given by p = Pb and hence P = ( )

=

2
0

0
3

3
( )1 = 1 6
0

P=

0
2

is the projection matrix.

6.8 Find the best straight line fit (least squares) to the measurements
= 4 at = 2, = 3 at = 1,
= 1 at
= 0, = 0 at = 2

1 2
Then find the projection of = (4, 3, 1, 0) onto the column space of A= 1 1 .
1
0
1
2

SOLUTION:
Let the equation of the best fit line be of the form = + . Then we can write
1 2
4
1 1 = 3
1
1
0
0
1
2
To find the best least squares solution
1
1 1 1 1
1
1
2
2 1 0
1

, we use the normal equations =

2
1 = 1 1 1 1
0
2 1 0 2
2

4
3
1
0

Solving the above system of equations, we get

=
=
Hence the best fit line is b =
Projection of = (4, 3, 1, 0) onto the column space of A is =
133
1 2 61
35 =
= 1 1
1
0 36
35
1
2

97
61
11

35
35
35
35

6.9 We want to fit a plane = + + to the four points

= 3 at = 1, = 1
= 6 at = 0, = 3
= 5 at = 2, = 1
= 0 at = 0, = 0
(1) Find 4 equations in 3 unknowns to pass a plane through the points (if there is such a
plane).
(2) Find 3 equations in 3 unknowns for the best least squares solution.
SOLUTION:
Equation of the plane to be fitted is = + +
=3
=5

at = 1, = 1
at = 2, = 1

= 6 at = 0, = 3
= 0 at = 0, = 0

The system can be expressed in matrix form as follows

1
1
1
1

1
0
2
0

1
3

3 = 6
1
5

0
0

The four equations in three unknowns obtained are

+ + = .....1
+ = ......2
+ + = .....3
= .....4
which implies = (from 2) and also from 1&3, = and = . respectively.
Hence the equations are not consistent and the 4 equations 3 unknowns do not
represent a plane.
To find the 3 equations and 3 unknowns, we use the least squares method.
=
1 1
1 0
1 3

1 1 1
1
1 1

1 0 3
2 0 1 2 1 = 1
1
1 0 1 0 0

1 1
0 2
3 1

3
1 6
0 5
0 0

4 + + =
+ + =
+ + =
are the 3 equations in 3 unknowns for the best least squares solution.