Homework # 4

1. (a) Consider a function f (x) with f (x) : R → R. Let r(x)

be the second order Taylor series of f (x) about the base
point z ∈ R. (2nd order means up to the term including
the second derivative of f (x).) Show that r(z) = f (z),
r0 (z) = f 0 (z) and r00 (z) = f 00 (z). Explain why we can
think of r(x) as the tangent parabola to f (x) at z.
(b) Now consider a function f (x) with f (x) : Rn → R. (Notice,
now the domain is n-dimensional.) Let r(x) be the second
order Taylor series of f (x) about the base point z ∈ Rn .
Show that r(z) = f (z), ∇r(z) = ∇f (z) and Hr(z) =
Hf (z), and that from these equalities you can conclude
that the two functions share the same partial derivatives
and second order partial derivatives at z. Explain why we
can think of r(x) as the n-d tangent paraboloid to f (x) at
z.
(c) Continuing from (b), derive the Newton’s method iteration
for optimizing f (x) based on construction of replacement
functions.
2. Consider the function
20 3
f (x) = x4 − x − 24x2 + 19x − 10 (1)
3
(a) Graph this function and visually estimate all local maxes
and mins.
(b) Apply Newton’s method as an optimization to f (x) in or-
der to find all local maxs and mins. You will have to use
different starting values to find the different max/mins.
Explain why different starting values cause the iteration
to converge to different stationary (i.e. critical) points. Do
you have control of whether a max or min is found? Why
or why not?
3. Consider the banana function of the previous hw: f (x, y) =
100(y−x2 )2 +(1−x)2 . Pretend that we can’t solve ∇f (x, y) = 0
and instead attempt to optimize using Newton’s method start-
ing at the point (4, 4). Compare the number of iterations

1
 needed to optimize using Newton’s method (should be just a
few iterations) to steepest descent.
4. Recall the o-ring dataset from the previous hw and the associ-
ated log likelihood function:
N
X
log L(α) = ((1 − yi )(−α0 − α1 xi ) − log(1 + exp(−α0 − α1 xi )))
i=1
(2)
Use Newton’s method to maximize the log likelihood. Do this
first using the starting point (0, 0) and then using the starting
point (1, 1). Using (0, 0) you should converge in just a few
iterations, compare to the number of iterations needed using
steepest ascent. Using (1, 1) Newton’s method will fail, explain
why.