MAT 233: Linear Algebra

Activity on Least-Squares Solutions to Linear Systems

This activity gets us to the main goal of this chapter: Constructing a line (or curve) of best fit
through a collection of data. It will involve all the concepts we’ve seen in Chapter 6.
Consider the points (1, 1), (2, 1), and (3, 2). Plot these points on a piece of paper; do it neatly
and accurately, because you’ll be coming back to this later. Let’s suppose we wanted to find a line,
which has the equation y = ax + b where a and b are unknown constants, that goes through these
three points.

1. Why are we not going to succeed perfectly at this task?

2. Well, impossibility never stopped us before, so... suppose you had a line y = ax + b that
went through these three points. This would give us a system of three equations with two
unknowns. Write out that system below. (Don’t put this in matrix form yet. Make sure you
have the right unknowns.)

3. Now write that equation in matrix-vector form:

4. Is this system consistent? Why shouldn’t the result surprise you?

5. Let A be the coefficient matrix (3 × 2) from your system, let x be the vector containing the
unknowns (2 × 1), and let y be the “answer” vector (3 × 1). Fill in the blanks:

The system Ax = y is consistent if and only if y is in the of A.

Therefore, what can we say about y for the system you wrote above?

1
 6. What vector in the column space of A is closest to y? (The first thing you have to do to
answer this question is find an orthogonal basis for ColA.)

7. Call the vector from the previous question z. Without doing any calculations whatsoever,
why must the system Ax = z be consistent?

8. Find the solution to the system Ax = z. Call this solution x̂.

9. The entries of x̂ represent the slope and y-intercept of a line. Below, draw a neat and accurate
plot of the three points we started with and this line.

Notice: This is the line of best fit.

Following this activity:

• Least-squares solutions to linear systems

• The normal equations and least-squares solutions

• (Time permitting) Finding regression curves through data