APPENDIX

DIFFERENTIATION WITH
RESPECT TO A VECTOR

The first derivative of a scalar-valued function f (x) with respect to a vector

x = [x1 x2 ]T is called the gradient of f (x) and defined as
f (x) =




d
f/x1
f (x) =
f/x2
dx

(C.1)

Based on this definition, we can write the following equation.


T

T
y
x y=
y x=
(x1 y1 + x2 y2 ) = 1 = y
y2
x
x
x


2
T
x
x x=
(x1 + x22 ) = 2 1 = 2x
x2
x
x

(C.2)
(C.3)

Also with an M N matrix A, we have

T
x Ay =
x
T
y Ax =
x
where
xT Ay =

T T
y A x = Ay
x
T T
x A y = AT y
x

N
M 


amn xm yn

(C.4a)
(C.4b)

(C.5)

m=1 n=1

Applied Numerical Methods Using MATLAB, by Yang, Cao, Chung, and Morris
Copyright 2005 John Wiley & Sons, Inc., ISBN 0-471-69833-4

471

472

DIFFERENTIATION WITH RESPECT TO A VECTOR

Especially for a square, symmetric matrix A with M = N , we have

if A is symmetric
T
x Ax = (A + AT )x 2Ax
x

(C.6)

The second derivative of a scalar function f (x) with respect to a vector x =

[x1 x2 ]T is called the Hessian of f (x) and is defined as
H (x) = 2 f (x) =

2
d2
f/x12
f
(x)
=
2
2
f/x2 x1
dx

2 f/x1 x2
2 f/x22


(C.7)

Based on this definition, we can write the following equation:

if A is symmetric
d2 T
x Ax = A + AT 2A
2
dx

(C.8)

On the other hand, the first derivative of a vector-valued function f(x) with
respect to a vector x = [x1 x2 ]T is called the Jacobian of f (x) and is defined as



d
f1 /x1 f1 /x2
J (x) =
(C.9)
f(x) =
f2 /x1 f2 /x2
dx