Second Order Conditions (a preview)

If FOC conditions are satised, could have, max, min, or neither. specically, for x, dx Rn , f ( x + dx) f ( x)

1 f ( x)dx + dx Hf( x)dx 2 f ( x) = 0.

if dx satises the FOC for an unconstrained maximum then

is a local max, min or neither depending on sgn(dx Hf( x x)dx) it will be a local max if dx Hf( x)dx < 0, dx,
i.e., if Hf( x) is a negative denite matrix. min if dx Hf( x)dx > 0, dx, i.e., if Hf( x) is a positive denite matrix. neither if dy, dz Rn s.t. dy Hf( x)dy > 0 > dz Hf( x)dz i.e., if Hf( x) is an indenite matrix.

what follows is graphical intuition for what deniteness means

and its relationship to the eigen-vectors and eigen-values of H .
() October 10, 2013 1 / 11

v2

1.5

v1

0.5

0.5

1.5

(a) The columns of A

x2 Ax 2 Ax 3

1
x4 x3

0.5
Ax 4

Ax 1

-1

-0.5

0.5

x1

-2

-1

-1 -0.5

-2 -1

(b) Selected elements of the unit circle

(c) The image of the circle under A

Figure 1. What the matrix A does to the unit circle

()

October 10, 2013

2 / 11

-2

-1.5

-1

-0.5

-0.5

-1
v1

-1.5

-2
v2

(a) The columns of B

x2

1
x4 x3

0.5

-1

-0.5

0.5

x1

-2

-1

B x4

-0.5

B x1

-1

-2 -1
B x3 B x2

(b) Selected elements of the unit circle

(c) The image of the circle under B

Figure 1. What the matrix B does to the unit circle

()

October 10, 2013

3 / 11

v1

1.5

v2

0.5

0.5

1.5

(a) The columns of C

x2 C x1 C x3

1
x4 x3

0.5

C x2

-1

-0.5

0.5

x1

-2

-1

Cx

-1 -0.5

-2 -1

(b) Selected elements of the unit circle

(c) The image of the circle under C

Figure 1. What the matrix C does to the unit circle

()

October 10, 2013

4 / 11

LINALGEBRA3: THU, SEP 22, 2011

What matrix A does to the unit circle

PRINTED: OCTOBER 18, 2011

(LEC# 9)

What matrix C does to the unit circle 3

Cv 2 Cw

Av

Aw 1

w 2

3 3 2 1 0 1 2 3

3 3 2 1 0 1 2 3

Figure 4. Relationship between orientation and determinant

preserve A 2x2 matrix is denite indenite iff its rotations reverse orientation
() October 10, 2013 5 / 11

 2.040 0.432 0.576 1.205 0.940 A = 0.432 0.576 0.940 1.754

()

October 10, 2013

6 / 11

 0.640 0.288 0.384 A = 0.288 2.050 1.267 0.384 1.267 1.310

()

October 10, 2013

7 / 11

 0.000 0.000 0.000 A = 0.000 1.920 1.440 0.000 1.440 1.080

()

October 10, 2013

8 / 11

()

October 10, 2013

9 / 11

()

October 10, 2013

10 / 11

Eigenvectors and values: summary

1

For any symmetric n n matrix, there exists a set of eigenvectors which each have unit length and are pairwise orthogonal, i.e., for any two elements in the set, v1 and v2 , then v1 v2 = 0. A symmetric n n matrix, has exactly 2n unit eigenvectors iff all of the corresponding eigenvalues are distinct in absolute value; if two eigenvalues have the same absolute values, then there are an innite number of unit eigenvectors For any symmetric n n matrix, the matrix is positive (negative) denite if and only if all of its eigenvalues are positive (negative). For any symmetric n n matrix, the rank of the matrix is equal to the number of nonzero eigenvalues. More precisely, let A be a symmetric n n matrix, let V = v 1 , ..., v n be a set of pair-wise orthogonal eigenvectors for A and let = {1 , ..., n } be a set of corresponding eigenvalues, i.e., for each i , Av i = i v i . Then the rank of A is equal to the number of nonzero elements of .
() October 10, 2013 11 / 11