Fréchet & Gâteaux Derivatives1 and the Chain Rule

by
Francis J. Narcowich
January, 2021

There are two types of derivatives that get used in connection with non-
linear functions (and functionals), the Gâteaux (weak) derivative and the
Fréchet (strong) derivative.
Let V and W be Banach spaces, Ω an open set in V , and F a function
that maps Ω into W . Fix y in Ω.

Definition 0.1 (Gâteaux Derivative). If there is a bounded linear map2

∆y F : V → W given by3

F (y + εv) − F (y)
∆y F (v) := lim , for all v ∈ V, (0.1)
ε→0 ε
then we say that F is Gâteaux differentiable at y and that ∆y F is the Gáteaux
derivative of F at y.

It is easy to show that, if the derivative exists, it is unique.

As a simple example of the Gâueax derivative, let V = W = R2 and
define F by  
f1 (x, y)
F (x, y) := , (0.2)
f2 (x, y)
where f1 and f2 are differentiable functions in a neighborhood of y = (x0 , y0 ).
Let v = (u, v). Using the chain rule, we see that
! ! ! 
∂f1 ∂x ∂f1 ∂y ∂f1 ∂f1 ∂f1 ∂f1
∂x ∂ε
+ ∂y ∂ε ∂x
u + ∂y
v ∂x ∂y u
∆y F (v) := ∂f 2 ∂x ∂f2 ∂y = ∂f ∂f2 = ∂f ∂f2
∂x ∂ε
+ ∂y ∂ε ε=0
2
∂x
u + ∂y
v ∂x
2
∂y
v

The matrix on the far right is called the Jacobian matrix, and it’s usually de-
noted by J or Jy . The calculation above works for any system of equations of
the form F (x1 , . . . , xn ) = [f1 (x1 , . . . , xn ) · · · fm (x1 , . . . , xn )]T . The Jacobian
1
The main reference for these notes is Jordan Bell, Fr’echet and Gâteaux Derivatives,
https://individual.utoronto.ca/jordanbell/.
2
This is Gâteaux’s definition. Other authors use a directional derivative and even allow
nonlinear derivatives.
3
Note that ε is not required to be positive.

1
 ∂fi
matrix for the larger system is m × n and has entries Jij = ∂x j
. The same
calculation as above works for the more general case. Thus, ∆y F = Jy .
Another example, which differs from the one above, is the following: Let
H : R2 → R be defined by
( 3
|x| y
x4 +y 2
(x, y) 6= (0, 0)
H(x, y) = (0.3)
0 x = 0, y = 0.

H(x, y)) is continuous at (0, 0). At (x, y) 6= (0, 0),

|x|x2 |y| 1 x4 + y 2 1
|H(x, y| = ≤ |x| = |x|,
x4 + y 2 2 x4 + y 2 2
Hence, as (x, y) → (0, 0), H(x, y) → 0, so H is continuous at (0, 0).
Let’s compute the Gâteaux derivative. Let v 6= (0, 0) and y = (0, 0),
then the difference quotient is
H(ε(u, v)) − H(0, 0) |ε|3 ε|u|3 v |ε||u|3 v
= = .
ε ε(ε4 u4 + ε2 v 2 ) ε2 u 4 + v 2
Letting ε → 0 results in the Gâteaux derivative being ∆(0,0) H(v) = 0. It is
important to note that ∆(0,0) H is linear in v. If it were not, the Gâteaux
would not exist. For example, the function
( 2
x y
x2 +y 2
xy > 0
K(x, y) :=
0 xy ≤ 0

satisfies
K(ε(u, v)) − G(0, 0)
= K(u, v),
ε
which is obviously nonlinear in v. Consequently it’s Gâteaux doesn’t exist.
One last remark. The existence of the Gâteaux derivative doesn’t neces-
sarily imply continuity of a function. For instance, let
( 4
x y
x6 +y 3
(x, y) 6= (0, 0)
M (x, y) :=
0 (x, y) = (0, 0).

A computation nearly identical to the one above shows that ∆(0,0) M (v) = 0.
However, M (x, x2 ) = 12 6= 0. Thus, M is discontinuous at (0, 0), even though
the Gâteaux derivative exists.

2
 The Fréchet derivative is defined in a way that is somewhat different than
the Gâteaux derivative. Let V , W , Ω and F be as defined earlier. Again, fix
y ∈ Ω.
Definition 0.2 (Fréchet Derivative). If there is a bounded linear map ∆y F :
V → W that satisfies
kF (y + v) − F (y) − ∆y F (v)kW
lim = 0, (0.4)
kvk→0 kvkV
then we say that F is Fréchet differentiable at y and we call ∆y F the the
Fréchet derivative of F at y.
Like the Gâteaux, one can show that the Fréchet derivative is unique,
if it exists. However, unlike the definition of the Gâteaux derivative, where
the Gâteaux derivative is given directly via the formula in (0.1), the Fréchet
derivative is defined indirectly, without a formula, as a linear operator satisfy-
ing (0.4). This would seem to make it more difficult to compute the Fréchet
derivative. However, if the Fréchet derivative does exist, it can always be
computed via the Gâteaux derivative:
Proposition 0.3. If the Fréchet derivative exists, then the Gâteaux deriva-
tive also exists, and the two are equal.
Proof. Suppose that the Fréchet derivative exists at y. Fix v and note that
the linearity of ∆y F implies that ∆y F (εv) = ε∆y F (v), so that

F (y + εv) − F (y) F (y + εv) − F (y) − ∆y F (εv)

− ∆y F (v) = kvk .
ε kεvk

By (0.4) and the fact that any approach to zero for kvk is allowed in the
Fréchet case, the limit on the right is zero. Hence, the limit on the left is
also zero. Thus, the Gâteaux derivative exists and is equal to the Fréchet
derivative, whenever the latter exists.
Consider a function F (x1 , . . . , xn ) = [f1 (x1 , . . . , xn ) · · · fm (x1 , . . . , xn )]T .
When is F guaranteed to be Fréchet diffentiable? We leave it as an exercise
to show that F is Fréchet differentiable at a given point if and only if all of
∂f
the first partials ∂kj are continuous there.
The Gâteaux derivative may exist, but the Fréchet derivative may not.
Earlier, we showed that the Gâteaux derivative for the function defined in

3
(0.3) was ∆(0,0) H(v) = 0. If the Fréchet derivative exists, it is equal to the
Gâteaux derivative. Thus, by (0.4), we have that
kH(u, v) − H(0, 0) − ∆(0,0) H(v)k |u|3 v
lim = lim 1 = 0.
kvk→0 kvk kvk→0 (u2 + v 2 ) 2 (u4 + v 2 )

Choose v = u2 . Then, as u → 0 we have

|u|3 v |u|5 1
1 = 1 = 1 −→ 1/2,
(u2 + v 2 ) (u4 + v 2 )
2 (1 + u2 ) |u|(2u4 )
2 2(1 + u2 ) 2
which is a contradiction. Consequently, the Fréchet derivative doesn’t exist.
There are other important properties of these derivatives, such as the sum
rule, product rule and the chain rule. To state and prove these, we note that
the Fréchet derivative may also be defined in terms of “little oh” notation:
The Fréchet derivative exists at y if and only if F (y + v) = F (y) + ∆y F (v) +
o(kvk) for all v. The same is true for the Gáteaux derivative; it will exist at
y if and only if for every fixed v we have F (y +εv) = F (y)+ε∆y F (v)+o(ε).
Our aim is to prove the chain rule.
Theorem 0.4 (Chain Rule). Suppose that U , V and W are normed spaces,
e is an open subset of V , and that F : U → V ,
that Ω is an open subset of U , Ω
with F (Ω) ⊆ Ω. In addition, assume that F is Fréchet differentiable at x ∈ Ω
e
and the F (x) = y ∈ Ω.
e If G : Ω e → W is Fréchet differentiable at y. Then

∆x (G ◦ F ) = ∆y G ◦ ∆x F,
which is the chain rule, holds.
Proof. Let v be in Ω. Since F is Fréchet differentiable at x, F (x + v) =
F (x) +∆x F (v) + o(kvk). In addition,
| {z }
y

G ◦ F (x + v) = G(y + ∆x F (v) + o(kvk))

= G(y) + ∆y G(∆x F (v) + o(kvk))
= G(y) + ∆y G ◦ ∆x F (v) + ∆y G(o(kvk))
Because ∆y G is a bounded linear operator, k∆y G(o(kvk))k ≤ k∆y Gk ko(kvk)k =
o(kvk). From this, we have that
G ◦ F (x + v) = G(y) + ∆y G ◦ ∆x F (v) + o(kvk),
so ∆x (G ◦ F )(v) = ∆x G ◦ ∆x F (v). Thus, ∆x (G ◦ F ) = ∆y G ◦ ∆x F