Lectures. MTH141: Linear Algebra Mathematics Department. Ryerson University Instructor: Marcos Escobar (escobar@ryerson.

ca, Eng213) Textbook: Contemporary Linear Algebra by Howard Anton and Robert C. Busby (Wiley, 2003) Course Website: Blackboard

What is linear algebra? Like calculus, it is an area of mathematics. Algebra sets with binary operations and rules. Linear additional conditions that make the structure act like straight lines. [hence the term linear] Recall that calculus started with two main problems or ideas. 1. Tangent line to a curve 2. Area under a curve. These lead to the study of differential and integral calculus. We begin with the problem of solving linear equations.

CHAPTER 1
Vectors in the plane (R2) and three-space (R3). objects with length and direction These vectors are the same since they have the same length and direction (equivalent).

Vectors in the plane: Naming them:

(v1 ,v2 )
v = (v1,v2 )

Each vector in the plane corresponds in a one-to-one way with the points in the plane. Combining vectors: Geometrically and Algebraically Addition u = (u1 ,u2 ) u v
3

v = (v1 ,v2 )

u+v

u + v = (u1 +v1 , u 2+v2 )

Subtraction u v Scalar Multiplication u-v u - v = (u 1-v1 , u2 -v2 )

2u u

ku = k (u 1,u 2) = (ku1 ,ku 2)

-u

Vectors in 3-space. (u1,u 2,u 3) rules. z

Analogous geometric and algebraic

(u 1,u2 ,u3) = u y x

Vectors from one point to another: See what vector you get when going from the point P=(1,5,-2) to Q=(3,4,5): Answer: QP=(2,-1,7). This vector is the same as performing u-v with u the vector (3,4,5) and v the vector (1,5,-2). Same in two dimensions.
4

Norm of a Vector and Vector Arithmetic Rules that hold for addition and scalar multiplication: 1. 2. 3. 4. 5. 6. 7. 8. u+v =v+u (u+v)+w = u+(v+w) u+0=0+u=u u+(-u)=0 a(bu)=ab(u) a(u+v)=au+av (a+b)u = au+bu 1u = u

These are the same as the rules for matrices. (recall we called nx1 and 1xn matrices column and row vectors.) Length of a vector v = (v1 ,v2 ) ||v|| = (v12 + v2 2 )1/2 for vectors in the plane [ ||v|| = (v12 + v2 2 + v32 )1/2 for vectors in the 3-space ] Do problems 6 and 9 in class. 6. Find the length of (1/||v||) v [use the definition of length] Discuss the idea that normalizing v.

9. Use knowledge about triangles to show ||u+v|| ||u|| + ||v|| [look at the drawing we had of u+v earlier.
5

Dot Product (scalar product) (1.2) Geometrically u v ||u|| ||v|| cos 0 if u 0 and v 0 u = 0 or v = 0

uv=

Algebraically:

u v = u1 v1 + u2 v2

in the plane in 3-space

u v = u 1 v1 + u 2v2 + u 3 v3 Observations If u 0 and v 0 then cos = u v / ||u|| ||v||

u v if and only if u v = 0.

OTHER PROPERTIES OF THE DOT PRODUCT a) b) c) d) uv=vu u (v + w) = u v + u w k(u v) = (ku) v = u (kv) v v > 0 if u 0, and v v = 0 if v = 0.
6

Orthogonal Projection u

proja u = ua a ||a||2
( if >0 )

why? ||proja u|| = cos = ua ||u|| ||u||||a||

( solve for ||proja u|| )

What is the direction of ||proja u|| ? Conclude proja u = ua a ||a|| ||a||

EUCLIDEAN n-SPACE In previous slides we talked about vectors in 2 and 3 dimensional space. (a 1,a 2,a 3) is point or vector in 3-space. This idea generalizes to higher dimensions. This is important since most applications in areas such as physics, economics, and statistics involve problems with more than three variables. Definition: If n is an positive integer and a i 's are real numbers, then (a 1 ,a2 , . . . ,an) is an ordered n-tuple. The set of all possible n-tuples is called n-space (n dimensional space) or Rn . An n-tuple is called a point or vector in n-space. ALGEBRAIC DEFINITIONS AND RULES: Suppose u = (u1 ,u2, . . . ,u n) v = (v1 ,v2 , . . . ,vn). We define

u + v = u = (u 1+v1 , u2 +v2 , . . . , u n+vn ) ku = u = (ku 1,ku2 , . . . ,kun) 0 = (0,0, . . . ,0) It follows that -u = (-u 1,-u2, . . . ,-un ) u - v = u = (u1 -v1, u 2-v2 , . . . , u n-vn ) Theorem: 9. u+v =v+u 10.(u+v)+w = u+(v+w) 11.u+0=0+u=u 12.u+(-u)=0 13.a(bu)=ab(u) 14.a(u+v)=au+av
8

15.(a+b)u = au+bu 16.1u = u Generalized Dot Product - Euclidean Inner Product u v = u1 v1 + u2 v2 + . . . + u n vn = u i vi

i=1 n

(5,4,1,2,3) (-2,3,1,4,1) = -10 + 12 + 1 + 8 + 3 = 14 PROPERTIES OF DOT PRODUCT a) u v = v u b) (u +v) w = u w + v w c) k(u v) = (ku) v = u (kv) d) v v > 0 if u 0, and v v = 0 if v = 0. Proof of b) and d). (u+v) w = (u i +vi ) w i = (u i w i + vi w i) = u i w i + vi w i =uw + vw v v = vi vi = vi 2 > 0 v v = 0 iff vi 2 = 0 iff vi = 0 for each i. Euclidean Norm (length) Distance ||u|| = (u u)1/2 = (u 12 +u 22 +. . . + un2 )1/2

d(u,v) = ||u-v|| = ((u 1 -v1 )2 +(u 2 -v2 )2. . . + (u n -vn )2 )1/2

Cauchy Schwarz Inequality (for Rn) If u and v are in Rn then |u v| ||u|| ||v||
9

Proof: (Special Cases: 2 and 3 dimension.) |u v| = | ||u|| ||v|| cos | ||u|| ||v|| (since |cos | 1) FACTS ABOUT LENGTH AND DISTANCE Length a) ||u|| 0 b) ||u|| = 0 iff u = 0 c) ||ku|| = |k| ||u|| d) ||u +v|| ||u|| + ||v|| Proof of d. (using the Cauchy-Schwarz inequality ||u +v||2 = (u +v)(u +v) = u u + 2uv + v v = ||u ||2 + 2uv + ||v ||2 ||u ||2 + 2 ||u || || v|| + ||v ||2 = (||u|| + ||v||)2 Distance a) d(u,v) 0 b) d(u,v) = 0 iff u=v c) d(u,v) = d(v,u) d) d(u,v) d(u,w) + d(w,v) Proof of d. d(u,v) = ||u -v|| = ||u-w + w-v|| ||u -w|| + ||w -v|| = d(u,w) + d(w,v) Fact: uv = 1/4 ||u +v||2 - 1/4 ||u -v||2 Proof: (see work done earlier in class to get the first equation) ||u +v||2 = ||u ||2 + 2uv + ||v ||2 ||u - v||2 = ||u ||2 - 2uv + ||v ||2 Subtract second from first and solve for uv.
10

ORTHOGONAL (generalization of perpendicular to n-space) The vectors u and v are called orthogonal iff uv = 0. Theorem: u and v are orthogonal iff ||u +v||2 = ||u||2 + ||v||2 Proof: ||u +v||2 = (u +v)(u +v) = u u + 2uv + v v = ||u ||2 + 2uv + ||v ||2 = ||u ||2 + 0 + ||v ||2 = ||u ||2 + ||v ||2 NOTATION FOR VECTORS IN n-SPACE u1 u = u2 (column vector) or u = (u 1, u 2, . . . , u n) (row vector) . un Using column vectors then one gets the following uv = vTu (do an example)

11

Lines and Planes (1.3) Lines in 2-space. Standard Equation of a line in R2: Ax+By+C=0. Where C/B is the interception with the y-axis and A/B is the slope. Alternative expression (parametric equation): Assume a point on the line (x0 ,y0 ) and a vector v=(a,b) with the direction of the line are given then the equation of the line in vector notation is: P0 P=tv or (x,y)= (x0 ,y0 )+tv where P is any point in the line and t is a scalar called the parameter. Note the parametric equation implies (x- x0 )/a=(y- y0 )/b <=> (1/a) x + (-1/b) y + (y0 /b- x0 /a)=0 so we have a relationship between A,B,C and a,b,x0 ,y0 . The parametric approach (vector notation) is beneficial because it is intuitive and it can be extended to n-dimensions. Lines and Planes in 3-space Straight lines in 3-space Given a point in the line (x0 , y0 , z 0) and a vector v=(a,b,c) in the direction of the line then the parametric equation of the line in vector notation is: (x,y,z)=(x0 ,y0 ,z 0 )+t v.
12

x = x0 + at y = y0 + bt z = z 0 + ct

Line parallel to vector v=(a,b,c) through (x0 , y0 , z0 )

or (x - x0 )/a = (y - y0 )/b =(z- z 0)/c Planes in 3-space. Equation of a plane in vector notation: nP0 P=0 where P0 is a point in the plane. P is any other point in the plane and n is a normal vector. (a,b,c) (x-x0 ,y-y0 ,z-z 0 )=0 <=> a(x-x0 )+b(y-y0 )+c(z-z 0 )=0 <=> ax+by+cz+d=0 n P P0 . normal vector (a,b,c)=n

If d=0 then it goes through the origin (0,0,0) Recall nP0 P=0, therefore nP0 =0 if (0,0,0) in on the plane. Parametric equation of a Plane in vector notation: Assume a point P0 and two vectors v1 , v2 (both in the direction of the plane) are given, then the parametric equation is:
13

P0 P=t 1 v 1+t 2 v2 . Where P is any point in the plane and t 1 and t 2 are the parameters. Examples: 1 Equation of a Plane passing through the point P 1 =(1,2,-1) with normal n=(3,2,-5). 2 Equation of a Plane passing through the point P 1 =(1,2,-1), P 2 =(2,3,1), P 3 =(3,-1,2). 3 - Equation of a line passing through the points P 1=(2,4,-1) and P 2 =(5,0,7).

14

Chapter 2: Linear Equations (2.1) Variables Constants x1 , x2 , x3 , ... , xn a1 , a 2 , a 3 , ... , an , b a1 x1 + a2 x2 + a 3 x3 + ... + a nxn = b n a ixi = b i=1 Remark: xi has to appear linear, i.e. xi 2 or sin(xi ) are not allowed Definition: s 1 , s 2 , s 3 , ... , s n is called a solution of the linear equation if a 1 s 1 + a 2 s 2 + a 3 s 3 + ... + an s n = b Example: Solving: 1, 1 is a solution to 3x+2y = 5 (or 3x1 + 2x2 = 5) x = 5/3 - 2/3 y

Linerar equation

Parametric form of the solution (general solution) x = 5/3 - 2/3 t y=t specific solutions (t=1) (t=2) x =1 y=1 x = 1/3 y = 2

15

Example 2 x1 - 5x2 + x3 - 4x4 = 6 Solution x1 = x2 = x3 = x4 = (or x1 = 3 + 5/2 x2 - x3 /2 + 2x 4 )

3 + 5/2 s - t/2 + 2u s t u

System of Linear equations - finite set of linear equations A solution to the system must satisfy all the equations. A solution is the intersection of the solution sets of each equation. Example: 2 equations in 2 unknowns. Solution is the intersection of 2 straight lines in the plane

Consistent (one solution)

Consistent (infinite solutions)

Inconsistent (no solutions)

16

General System of Linear Equations a 11 x1 + a 12x2 + a 13 x3 + ... + a1n xn = b 1 a 21 x1 + a 22x2 + a 23 x3 + ... + a2n xn = b 2 . . a m1 x1 + am2 x2 + am3 x3 + ... + amnxn = bm a ij ith row jth column

Augmented Matrix (shorthand) a 11 a 12 a 13 ... a 21 a 22 a 23 ... a m1 a m2 a m3 ... a1n b1 a2n b2 a mn b m

What is the augmented matrix for the following? x1 - 2x2 + 3x3 = 5 2 x1 + x2 - 7x3 = 6 To solve systems of equations we manipulate these matrices.

17

Elementary Row Operations Three operations can be used to change equations (augmented matrices) and not change the solution set. 1) Multiply a equation (row) by a non-zero constant 2) Interchange 2 equations (rows). 3) Add a multiple of an equation (row) to another.

Example: (Gauss Jordan Reduction) x1 + 2x2 = 3 3 x1 - x2 = -5 1 2 3 -1 -3I+II 3 -5

1 2 3 0 -7 -14 1 2 0 1 1 0 3 2

-1/7 II

-2II + I

0 -1 1 2

Solution

x1

= -1 x2 = 2
18

Gaussian Elimination (2.2)

A matrix is in reduced row-echelon form (RREF) if it has the following properties: 1. If any row is not all zeros then the first non-zero number is a 1. 2. All rows that are entirely zero are at the bottom of the matrix. 3. The leading 1 in any row is farther to the right than the leading one in any row above it. 4. Each column with a leading 1 has zero everywhere else. 0 0 0 0 1 0 0 0 2 0 0 0 0 1 0 0 0 0 1 0 5 6 8 0

A matrix is in row-echelon form (REF) if it satisfies 1-3 above. 0 0 0 0 1 0 0 0 2 0 0 0 6 1 0 0 0 2 1 0 5 6 8 0

Remark: Gauss elimination takes a matrix to its REF while a Gauss-Jordan takes a matrix to its RREF.

19

Solving RREF matrices 1 0 0 0 2 0 0 0 0 1 0 0 0 0 1 0 3 1 2 0 0 0 0 1 4 5 6 2

x1 = 4 - 2x2 - 3x5 x3 = 5 - x5 x4 = 6 - 2x5 x6 = 2 x1 = 4 - 2s - 3t x2 = s x3 = 5 - t x4 = 6 - 2t x5 = t x6 = 2 Solving a matrix in REF. 1 0 0 2 0 0 4 1 0 5 -1 0 -3 2 1 6 5 3

x1 = 6 - 2x2 - 4x3 - 5x4 + 3x 5 x3 = 5 + x4 - 2x5 x5 = 3 x1 = 6 - 2s - 4(t-1) + 5t + 3(3) = 19 - 2s + t x2 = s x3 = 5 + t - 2(3) = t-1 x4 = t

20

x5 = 3

21

Example of Gauss-Jordan Reduction (Process that leads to a RREF) 0 3 2 3 0 2 1 0 2 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 0 6 4 6 0 4 2 0 4 2 0 0 2 0 0 2 0 0 2 0 0 2 0 0 2 0 0 -2 0 5 0 -2 5 0 -2 5 0 -2 5 0 1 5 0 1 0 0 1 0 0 1 0 0 1 0 0 9 6 9 0 6 3 0 6 3 0 0 3 0 0 3 0 0 3 0 0 3 0 0 3 0 0 7 18 -5 18 7 -5 6 7 -5 6 7 -17 6 -7/2 -17 6 -7/2 1/2 6 -7/2 1 0 -7/2 1 0 0 1 6 12 -2 switch I and II 12 6 -2 1/3 I 4 6 -2 -2I + III 4 6 -10 -1/2 II 4 -3 -10 -5II + III 4 -3 5 2 III 4 -3 10 -6 III + I -56 -3 10 7/2 III + II -56 32 10

22

x 1 = -56 - 2s - 3t x2 = s x 3 = 32 x4 = t x 5 = 10

23

Homogeneous Systems of Linear Equations a 11 x1 + a 12x2 + a 13 x3 + ... + a1n xn = 0 a 21 x1 + a 22x2 + a 23 x3 + ... + a2n xn = 0 . . a m1 x1 + am2 x2 + am3 x3 + ... + amnxn = 0 A homogeneous systems stays homogeneous after applying an elementary row operation. A homogeneous system is always consistent (it has one or more solutions). All xi = 0 works. This is called the trivial solution. Either there is only the trivial solution or there are infinitely many solutions. Is there a way to tell if there are infinitely many solutions without solving. Theorem: A homogeneous system of linear equations with more unknowns than equations has infinitely many solutions. [n>m] idea of proof: After applying Gauss-Jordan method, since there are more variables than equations, not all the variables are leading 1's so some will be free to vary (will be a parameter). 0 1 0 2 0 -2 0 0 0 1 3 0 1 0 0 0 0 0 1 0 0 x1 = s x2 = -2t + 2u x3 = -3t - u x4 = t
24

x2 = -2x4 + 2x6 x3 = -3x4 - 2x6 x5 = 0

x5 = 0 x6 = u

25