AMATH 731: Applied Functional Analysis Fall 2018

From Multivariable Calculus to Gâteaux Derivatives to Fréchet

Derivatives
Some preliminary remarks with regard to Section 3.10, “Fréchet Derivative,” in
the AMATH 731 Course Notes

Recall that a vector-valued function F : Rn → Rm may be expressed as follows,

F(x1 , · · · , xn ) = (F1 (x1 , · · · , xn ), · · · , Fm (x1 , · · · , xn )) (1)

or compactly as
F(x) = (F1 (x), · · · , Fm (x)) , (2)
where F1 · · · , Fm are the component functions of F. Each function Fi is a real-valued function of
x1 , · · · , xn , i.e.,
Fi : Rn → R , 1 ≤ i ≤ m . (3)
Now define the following n × m matrix of partial derivatives of F evaluated at a point x,
 
∂F1 ∂F1 ∂F1
∂x1 (x) ∂x2 (x) ··· ∂xm (x)
 ∂F2 ∂F2 ∂F2 
∂x1 (x) ∂x2 (x) ··· ∂xm (x)
 
DF(x) =  . (4)
 
.. .. ..

 . . . 

∂Fn ∂Fn ∂Fn
∂x1 (x) ∂x2 (x) · · · ∂xm (x)

Here we are assuming that all partial derivatives exist at x ∈ Rn .

Note that DF(x) may be viewed as a linear operator from Rn to Rm . It is the formal derivative
(operator) of F → Rn → Rm at x ∈ Rn .

Definition: A vector-valued function F → Rn → Rm is said to be differentiable at a point a ∈ Rn

if

1. all partial derivatives of the components F1 , · · · , Fn of F exist at a – in other words, all elements
of DF(a) exist – and

2. the derivative matrix DF(a) of F at a satisfies

kF(x) − F(a) − DF(a)(x − a)k

lim = 0, (5)
x→a kx − ak

where k · k in the numerator denotes the (Euclidean) length/norm in Rm and k · k in the

denominator denotes the (Euclidean) length/norm in Rn .

The derivative operator DF(a) is the Fréchet derivative of F at a. It defines the following linear
approximation of F at a: For x near a,

F(x) ≃ La (x) = F(a) + DF(a)(x − a) . (6)

Theorem: If F is Fréchet differentiable at a ∈ Rn , then F is continuous at a.

1
Proof: (similar to proof in AMATH 731 notes) From Eq. (5), it follows that

kF(x) − F(a) − DF(a)(x − a)k = o(kx − ak) . (7)

This implies that

lim kF(x) − F(a) − DF(a)(x − a)k = 0 , (8)
x→a
which, in turn, implies that

lim [F(x) − F(a) − DF(a)(x − a)] = lim [F(x) − F(a)] = 0 . (9)

x→a x→a

Therefore,
lim F(x) = F(a) , (10)
x→a
and the proof is complete.

Corollary: If F is not continuous at a, then F is not Fréchet differentiable at a.

Fréchet differentiability of the function F(x) is a rather strict requirement on the component
functions Fi (x) and their partial derivatives. A less restrictive requirement is that of the directional
derivative of F:

Definition: Let F → Rn → Rm as defined above. Also let u = (u1 , · · · , un ) be a unit vector in Rn ,

i.e., kuk = 1. The directional derivative of F at the point a ∈ Rn in the direction u is defined as
follows,
F(a + tu) − F(a)
Du F(a) = lim , (11)
t→0 t
provided that the limit exists.

Note that the directional derivative Du F(a) gives the rate of change of F at point a when we restrict
our motion away from a in the direction of the unit vector u. This rate of change is an m-vector. The
directional derivative Du F(a) is also known as the Gâteaux derivative of F at a in the direction u.
If the Gâteaux derivative of F at a exists for all unit directions u ∈ Rn , then F is said to be Gâteaux
differentiable at a.

Here we simply state that Gâteaux differentiability is a weaker requirement on F and its partial
derivatives – it does not necessarily imply Fréchet differentiability. Gâteaux differentiability implies
that a rate of change at the point of concern can be defined at all directions. The rate of change in
a given direction u is defined by the limit in Eq. (11). On the other hand, Fréchet differentiability
implies that the limit in Eq. (5) exists. Since this limit involves the norm kx − ak, the convergence to
the limit is uniform in all directions. This is a stronger requirement on F. As such, Fréchet differen-
tiability implies Gâteaux differentiability. To see this, let us assume that DF(a) exists and show how
the directional derivative may be expressed in terms of DF(a).

The ith component of the vector-valued RHS of (11) is

Fi (a + tu) − Fi (a) Fi (a1 + tu1 , · · · , an + tun ) − Fi (a1 , · · · , an )
lim = lim
t→0 t t→0 t
n
X ∂Fi
= (a) uk . (12)
j=1
∂xj

2
But the RHS is simply the inner product of the ith row of the matrix DF(a) with the (unit) vector
u ∈ Rn . As such, the directional derivative Du F(a) is given by

Du F(a) = DF(a) · u . (13)

(Note that this is an element in Rm .) In the special case m = 1, i.e., f = F = F1 is real-valued,

~ (a), the gradient vector of f and we have the well-known result,
Df (a) = ∇f
~ · u.
Du f (a) = ∇f (14)

The following example will illustrate the major ideas discussed above.

Example: Consider the function f : R2 → R defined as follows,

(
0 , (x, y) = (0, 0) ,
f (x, y) = x4 y (15)
x6 +y 3
, (x, y) 6= (0, 0) .

First of all, we show that f (x, y) is not continuous at (0, 0). If it were continuous at (0, 0), then for
any ǫ > 0, there exists a δ > 0 such that
 x4 y 
 
 x6 + y 3  < ǫ
  for all x2 + y 2 < δ . (16)

Let (x, y) = (r cos θ, r sin θ) for r > 0 and 0 ≤ θ < π so that

r 5 cos4 θ sin θ r 2 cos4 θ sin θ
f (x, y) = g(r, θ) = = . (17)
r 6 cos6 θ + r 3 sin3 θ r 3 cos6 + sin3 θ
If we fix θ and let r → 0+ , i.e., approaching (0, 0) along the straight line with slope tan θ,

lim g(r, θ) = 0 . (18)

r→0+

It might appear that g, hence f , is continuous at (0, 0). Once again, keeping θ fixed, we look for the
maximum value of
r 2 cos4 θ sin θ
hθ (r) = 3 . (19)
r cos6 + sin3 θ
1 sin θ
After a little work, we find that hθ (r) achieves a maximum value of at r̄(θ) = . Note that
2 cos2 θ
+ +
as θ → 0 , r̄(θ) → 0 . This implies that any open ball of radius δ > 0 centered at (0, 0) contains at
least one point (in fact, an infinite number of points) at which hθ (r), hence f (x, y) achieves the value
1
of . Therefore, f (x, y) is not continuous at (0, 0).
2

We now show that f (x, y) is Gâteaux differentiable at (0, 0). For a fixed θ ∈ [0, 2π), let u =
(cos θ, sin θ). From Eq. (11), directional derivative of f at (0, 0) in the direction u is given by
f (0 + t cos θ, 0 + t sin θ) − 0
Du f (0, 0) = lim
t→0 t !
1 t cos4 sin θ
5
 
= lim
t→0 t t6 cos6 θ + t3 sin3 θ
t cos4 sin θ
= lim
t→0 t3 cos6 θ + sin3 θ
= 0. (20)

3
Since the above result holds for all θ ∈ [0, 2π), it follows that f (x, y) is Gâteaux differentiable at (0, 0).

Note that Eq. (20) implies the existence of the partial derivatives of f with respect to x and y at
(0, 0):
∂f ∂f
(0, 0) = 0 (θ = 0) , (0, 0) = 0 (θ = π/2) . (21)
∂x ∂y
One might be tempted to conclude that f (x, y) admits a linear approximation of the following form,

∂f ∂f
f (x, y) ∼ f (0, 0) + (0, 0) x + (0, 0) y
∂x ∂y
= 0, (22)

for (x, y) near (0, 0). But this approximation is not valid since neither f (x, y) nor its partial derivatives
are continuous at (0, 0). For example, we saw that any ball of radius ǫ > 0 centered at (0, 0) contains
an infinite number of points at which f (x, y) assumes the value 1/2. This clearly contradicts the linear
approximation conjectured above.

Finally, we compute the partial derivatives of F with respect to x and y,

∂f 4x3 y x4 y
= − (6x5 )
∂x x6 + y 3 (x6 + y 3 )2
∂f x4 1
= 6 3
− 6 (3y 2 ) .
∂y x +y (x + y 3 )2
(23)

These formulas do not give us any information about the partial derivatives of f at (0, 0), i.e., whether
or not they exist. For this information, we must examine the directional derivatives at (0, 0) using Eq.
(20) as we did above.

The ideas of Fréchet and Gâteaux differentiability discussed above may be extended in a rather
straightforward manner to mappings F : X → Y where X and Y are normed linear spaces (typically
Banach spaces). Fréchet derivatives of such mappings are the subject of Section 3.10 in the AMATH
731 Course Notes.