PHY400W:

Computational physics
Dr. Roger Fearick
Room 506, R.W.James Bldg
http://www.phy.uct.ac.za/courses/phy400w/
Books
de Vries First course in computational physics. Wiley 1994.
Garcia Numerical methods for physics. Prentice Hall 1994.
Giordano Computational physics. Prentice Hall 1997.
Gould and Tobochnik Introduction to computer simulation
methods. Addison-Wesley 1996.
Press et al. Numerical recipes. Cambridge UP.
Part I
Introduction
Computers in physics
Presentation.
Writing papers, email, etc.
Data acquisition
Data analysis
Numerical analysis
Simulation
Physics?
Traditional:
Experimental physics
Theoretical physics.
Now we must add:
Computational physics
Computational physics?
Analysing a general potential system with two degrees of
freedom is beyond the capabilities of modern science
V.I. Arnold
Mathematical methods of classical mechanics
The need for computational physics?
Arnold quotes the set of solvable problems in classical mechanics:
one-dimensional problems, motion of a point in a central
eld, Eulerian and Lagrangian motions of a rigid body,
the problem of two xed centers, and motion along
geodesics on the ellipsoid.
We can add the the Toda lattice this was only found to be
solvable (integrable) after numerical solutions had hinted at it.
The H enon-Heiles potential
Simple model of stellar motion in galaxy
H =
u
2
+ v
2
2m
+ m
2
x
2
+ y
2
2
+ (x
2
y
y
3
3
)
Solution shows deterministic chaos
h1 h2 h3 h4
What is computational physics?
Obtaining solutions to known equations which cannot be
solved analytically.
Exploring model systems for which no solution is known.
In general, using the computer to generate data, by means of
the above, which can be used to confront theory.
Programming
I dont know what the language of the year 2000 will look
like, but I know it will be called Fortran.
C.A.R. Hoare
(1982)
Basics of programming
How do we transform a physics problem into something that can
be solved on a computer?
Analyse the problem to nd a suitable numerical method of
solution.
Derive an algorithm a recipe to generate the result.
Write a computer program to embody the algorithm.
Get the program debugged and running.
Solve the problem!
Basics of programming
A simple differential equation is the equation that gives rise to the
exponential decay law,
dy(t )
dt
= y(t )
where is a decay constant.
Basics of programming
Numerical analysis: approximation.
Change in a time interval is proportional to y:
y(t + t ) y(t ) = y(t ) t (1)
or
y(t + t ) = y(t ) y(t ) t (2)
Basics of programming
Recipe to solve exponential decay law:
Dene the initial condition at t = 0
Repeat the following until some nal value for t is reached:
Use equation (2) to determine the solution at t + t
Basics of programming
# initial conditions
y=100.0
lamb=1.0
deltaT=0.02
# use equation (2) for 100 steps
for i in range(3*lamb/deltaT):
print i*deltaT,y
y=y-lamb*y*deltaT
Exponential decay
Output of program
0
10
20
30
40
50
60
70
80
90
100
0 0.5 1 1.5 2 2.5 3
y
(
t
)
Time / s
decay.data
100*exp(-x)
Numbers
The purpose of computing is insight, not numbers.
R.W. Hamming
Numerical methods for scientists and engineers
Data elements
The basic data elements are of course numbers.
In physics, we deal with various kinds of numbers: integers, real
numbers and complex numbers.
Computers, and computer languages, attempt to supply some
realisations of these numbers for us to use: there are major
differences from the idealised mathematical quantities.
Data elements: numbers
Integers are available in some limited range, with a maximum
value of 2
31
1 for 32-bit integers.
Floating point numbers, the real numbers available on a
computer, have little resemblance to the real numbers of
mathematics.
The range is limited, they are not complete and continuous, and
the spacing between available numbers varied from number to
number.
Complex numbers are found in Fortran, but not by default in most
other languages.
Data elements: characters
Languages usually have some representation for characters and
character strings.
Usually there are facilities for embedding numbers, etc., in strings:
this is known as formatting.
Statements
Usually, one can assign names to numbers and treat them much
like arithmetic variables.
e.g.
v=12.0
distance=v*3.0+5.0
In many languages, the types (integer, real, . . . ) of these variables
must be declared before use.
Control structures
There are a number of control structures that are useful:
Sequence Usually statements are executed in sequence. Also
known as a block.
Repetition These structures are generically known as loops.
Decision Logical choice is implemented by if/then/else
structures. Multiple choice is implemented as a case
statement.
Higher-level structures
Programs are usually assembled out of higher level structures
called procedures.
In object oriented languages, these may be combined with data to
form classes.
Combinations of classes may form modules in some languages.
These are often in separate les.
Languages technology
Compiled
e.g. Fortan, C, C++
Interpreted
e.g. Java, Python, Matlab,. . .
Languages classication
Procedural
e.g. Fortran, C, Algol,. . .
Object oriented
e.g. C++, Java, C#, Smalltalk, Python,. . .
Functional, declarative,. . .
e.g. Lisp, Haskell, Prolog,. . .
Languages numerics
Arrays
Fortran, C, C++, Python
Array arithmetic
Fortran, Python, (C++)
Complex numbers
Fortran, Python, C++
Specied oating point precision
Fortran
Array sections, generics, operator overloading, . . .
Numbers in more detail
The purpose of computing is insight, not numbers.
R.W. Hamming
The purpose of computing is not yet in sight.
Anon
Nevertheless. . .
The computer number system
There are two types of number represented on the computer:
integers and
oating point numbers, the counterparts of real numbers.
The representations depend on the hardware.
Integers
Integers are usually stored in four bytes, each byte consisting of
eight bits.
31 bits are used to represent an integer, and one bit for the sign.
Thus integers range over 2
31
to 2
31
1
(-2147483648 to 2147483647).
Floating point numbers
Real numbers are represented as oating point numbers.
These are written in the form
sMB
ee
o
(s is a sign bit, M is a mantissa,
B is the base of the number system (usually 2),
e is an exponent and e
o
is an offset.
Floating point numbers
If we take a mantissa of 3 decimal digits and a signed exponent of
2 decimal digits as an example, a oating point number might be
0.123 10
01
Floating point numbers
On a PC oating point number will be stored in 32 or 64 bits.
The number of bits used for portions of the number are:
Precision sign mantissa exponent
single 1 23 8
double 1 52 11
extended 1 64 15
The leading bit in the mantissa is always set.
Floating point numbers
On a PC the range of represented numbers is given approximately
in the table:
Precision minimum maximum sig. decimals
single 1.2E-38 3.4E+38 6-9
double 2.2E-308 1.8E+308 15-17
extended 3.4E-4932 1.2E+4932 18-21
Note that the minimum value is normalised.
Floating point exceptions
Errors in representation cause (or raise) exceptions.
Underow to zero.
Overow to innity. (INF)
Inexact operation.
Zero division (INF,NaN)
Invalid operation.
Loss of signicance
Consider base 10 with four digits in the mantissa
i.e. take B=10 and, say, e
0
= 50.
The number a = 10.3213 can be represented as
+ 5 2 1 0 3 2
Note that there has already been some truncation of the number to
t into the oating point scheme.
How the number is rounded is determined by the hardware.
Loss of signicance
The subtraction a = 10.3213 10.3122
+ 5 2 1 0 3 2 - + 5 2 1 0 3 1
results in the unnormalised number
+ 5 2 0 0 0 1
The normalised result is then:
+ 4 9 1 0 0 0
Signicant digits have been lost from the right hand side.
Floating point precision
An important quantity in numerical programming is the machine
epsilon.
This is the smallest number that, added to 1.0, gives a result
different to 1.0.
Example (On Intel hardware. . . )
Precision epsilon
single 1.19E-7
double 2.22E-16
Part 2: Ordinary differential equations
Classical mechanics
The equations of classical mechanics can be written as differential
equations. We look at how to solve these.
Differential equations
A wide range of differential equations can be written in the form
dy(t )
dt
= f (y, t )
These equations rarely have analytical solutions, and generally
one must apply numerical methods in order to solve them.
Differential equations (2)
Differential equations involving higher order derivatives can often
be written as a set of simultaneous rst order equations
dy(t )
dt
= f(y, t )
where y and f are now vectors.
Classical mechanics
The equations of classical mechanics can typically be written in
this form:
dx
dt
= v
dv
dt
=
1
m
= a
These could be combined by identifying
y = (x, v)
and
f = (v,
1
m
F) = (v, a)
Classical mechanics (2)
In Hamiltonian form:
dq
i
dt
=
H
p
i
dp
i
dt
=
H
q
i
or
dq
dt
=
p
H
dp
dt
=
q
H
Solution of equations of motion
In order to solve the equations of motion, we need
1
Initial conditions.
2
A numerical method that permits an approximate solution
using only arithmetic.
Numerical methods
An approximation to the derivative is suggested by the denition
from basic calculus:
dx(t )
dt
= lim
t 0
x(t + t ) x(t )
t

x(t + t ) x(t )
t
(for t suciently small)
How good is this as an approximation?
Use the Taylor expansion to derive it:
x(t + t ) x(t ) = t x

(t ) +
(t )
2
2
x

(t ) +
so that
x(t + t ) x(t )
t
=
dx(t )
dt
+
t
2
x

(t ) +
The leading error term vanishes as t , i.e. we say that the
approximation is of order t , i.e. O(t ).
Discretising the derivatives
The differential equations for the motion (in 1-d) can now be written
x(t + t ) x(t )
t
= v(t ) +
v(t + t ) v(t )
t
= a(t ) +
(3)
with an approximate solution (the Euler method)
x(t + t ) = x(t ) + v(t )t
v(t + t ) = v(t ) + a(t )t
(4)
Given an initial x and v, and knowing a, these can be solved
recursively.
The not-so-simple pendulum
A real pendulum with nite amplitude is represented as the
mathematical pendulum
d
2

dt
2
+
2
sin = 0
Pendulum solution
The solution is only found in terms of elliptic integrals, with the
period T being
T = 4
_
L
g
K(sin

m
2
)
where
K(x) =
_
/2
0
dz
_
1 x
2
sin
2
z
is the complete elliptic integral of the rst kind.
The angular position is
= 2 arcsin (k sn(t + , k))
where k = sin(
m
/2) and sn(u, k) is a Jacobian elliptic function.
The not-so-simple pendulum
T = 4
_
L
g
K(sin

m
2
)
2
2.5
3
3.5
4
4.5
K
0 0.2 0.4 0.6 0.8 1 1.2 1.4
x
Solving the differential equation
Write the equation as a system of two coupled rst order
equations:
d
dt
=
2
sin (5)
d
dt
= (6)
where is the angular velocity and is the angular position.
These equations can now be integrated using the Euler method.
The explicit Euler solution
Suppose we know the position and velocity at a time t = it . Then
the above pair of coupled equations can be rewritten as

i+1
=
i
t
2
sin
i
(7)

i+1
=
i
+ t
i
(8)
A simple pseudocode algorithm is:
initialise variables
for i varying over the range of timesteps:
velocity[i+1]=velocity[i]-dt*Omega**2*sin(position[i])
position[i+1]=position[i]+dt*velocity[i]
from Numeric import *
g=9.81 # m/s**2
l=1.00 # length in m
w0sq = g/l
dt=0.1 # time step
vn=0.0 # initial speed zero at extreme posn.
xn=30.0*pi/180.0 # initial angular position 30 degrees
Nsteps = 100 # no. of steps to take
for i in range(Nsteps+1):
print i, xn, vn
vnp1 = vn - w0sq*sin(xn)*dt
xnp1 = xn + vn*dt # Euler method
xn = xnp1
vn = vnp1
program pendulum
implicit none
real(kind=8), parameter :: pi = 3.1415926535897932384_8
real(kind=8), parameter :: g=9.81_8 ! m/s**2
real(kind=8), parameter :: l=1.00_8 ! length in m
real(kind=8) :: xn, xnp1, vn, vnp1, dt, w0sq
integer :: i
integer, parameter :: Nsteps = 100
w0sq = g/l
dt=0.1_8 ! time step
vn=0.0_8 ! initial speed zero at extreme posn.
xn=30.0_8*pi/180.0_8 ! initial angular position 30 degrees
do i=0,Nsteps
write(*,fmt=(i4,2x,2(f10.4))) i, xn, vn
vnp1 = vn - w0sq*sin(xn)*dt
xnp1 = xn + vn*dt ! Euler
xn = xnp1
vn = vnp1
end do
stop
end program pendulum
The explicit Euler solution
Result
-10
-8
-6
-4
-2
0
2
4
0 20 40 60 80 100 120 140 160 180 200
A
n
g
u
l
a
r

p
o
s
i
t
i
o
n
Step
Euler t=0.05
A better solution
The computed solution clearly does not conserve the energy
E =
1
2
mL
2

2
+ mgL(1 cos ).
A slight modication, however, gives a system which does
conserve energy (at least, at a period).

i+1
=
i
t
2
sin
i
(9)

i+1
=
i
+ t
i+1
(10)
This Euler-Cromer method (also Euler-Ruth) is a rst order
area-preserving mapping. Such mappings exploit the symplectic
structure of Hamiltons equations they preserve the canonical
structure of these equations.
The Euler-Cromer solution result
-1
-0.5
0
0.5
1
0 20 40 60 80 100 120 140 160 180 200
A
n
g
u
l
a
r

p
o
s
i
t
i
o
n
Step
Euler-Cromer Dt=0.05
Phase space diagram
Euler-Cromer
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
-0.6 -0.4 -0.2 0 0.2 0.4 0.6
v
e
l
o
c
i
t
y
position
dt=0.01
dt=0.1
Integrating Newton II
Equations of motion:
dv
dt
= F/m
dx
dt
= v
Conservative: Symplectic methods (e.g. Euler-Cromer) better
Dissipative: Simple Euler may be better.
Higher order methods better still
Simple higher-order methods
(Velocity) Verlet:
y
n+1
= y
n
+ v
n
t +
1
2
a
n
(t )
2
v
n+1
= v
n
+
1
2
(a
n+1
+ a
n
) t
Second order
Simple symplectic method.
Exists in several forms.
Simple higher-order methods
Euler-Richardson: (second order)
a
n
= F(x
n
, v
n
, t
n
)/m
v
m
= v
n
+ a
n
t /2
y
m
= y
n
+ v
n
t /2
a
m
= F(x
m
, v
m
, t
n
+ t /2)/m
v
n+1
= v
n
+ a
m
t
y
n+1
= y
n
+ v
m
t
Useful for velocity-dependent forces (dissipative problems).
Can be made adaptive.
Loop idioms
Loops can be written in various ways.
Plain loop
vn=v0
xn=x0
do i = 1,N
t=i*dt
vnp1=vn+accel(xn,i*dt)*dt
xnp1=xn+vn*dt
vn=vnp1
xn=xnp1
end do
Arrays
Ordered sequence of numbers of same type
0.23 0.55 0.32 1.45 0.56 3.0 2.1
Access individual elements by indexing
e.g x(3)
Index starts from 1 in Fortran
Dene an array using, e.g
real(kind=8),dimension(4) :: x
or
real(kind=8),dimension(4) :: x=(/1.2,1.3,3.4,5.6/)
Can treat as vectors, e.g x=y+z
Loops with arrays
v
i+1
= v
i
+ a(x
i
, t ) t
x
i+1
= x
i
+ v
i
t
v(1)=v0
x(1)=x0
do i = 1,N
t=i*dt
v(i+1)=v(i)+accel(x(i),t)*dt
x(i+1)=x(i)+v(i)*dt
end do
Vector solver 1
Use arrays as position and velocity vectors
real(kind=8),dimension(3) :: x,v
...
v=(/vx,vy,vz/)
x=(/x0,y0,z0/)
do i =1,N
t=i*dt
v=v+accel(x,v,t)
x=x+v*dt
end do
Vector solver 2
Combine position and velocity into one vector
function rhs( x,t )
real(8),dimension(2),intent(in) :: x
real(8),dimension(2) :: rhs
real(8) :: t
rhs(1)=x(2)
rhs(2)=accel(x(1),t)
return rhs
end function rhs
x(1)=x0
x(2)=v0
do i =1,N
t=i*dt
x=x+rhs(x,t)*dt
end do
Symplectic methods
For simplicity, take a 1-d problem:
dq
i
dt
=
H
p
i
dp
i
dt
=
H
q
i
Write a vector z = (p, q) with
z
= (

q
,

q
)
The equations become
dz
dt
= J
z
H(z), where J =
_
0 1
1 0
_
Symplectic methods (2)
This equation has the formal solution
z(t ) = exp (tJ
z
H) (z
0
)
for initial conditions z
0
.
The exponential term is an operator dened by its power series.
Write H = T + U and truncate BCH expansion ater rst terms.
z(t ) = exp (tJ
z
T) exp (tJ
z
U) (z
0
)
Expand exponentials to rst order:
z(t ) (1 + tJ
z
T) (1 + tJ
z
U) (z
0
)
Symplectic methods (3)
E.g. consider harmonic oscillator, H = (1/2)(p
2
+ x
2
)
z(t ) =
_
1 tp()
0 1
_ _
1 0
tx() 1
_ _
x
0
p
0
_
=
_
1 tp()
0 1
_ _
x
0
p
0
x
0
t
_
=
_
1 tp()
0 1
_ _
x
0
p
1
_
=
_
x
0
+ tp
1
p
0
x
0
t
_
Instability as step size increases
The explicit solution can be unstable for large step sizes.
E.g. simple rst order
y(t + t ) = y(t ) t y(t )
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
0 0.1 0.2 0.3 0.4 0.5 0.6
y
t
dt=0.01
dt=0.05
dt=0.15
exponential
The implicit solution
The equation can also be written, with about the same accuracy, as
y(t + t ) = y(t ) t y(t + t )
The solution is now implicitly in terms of the new value.
The solution in this particular case is easy:
y(t + t ) =
y(t )
1 + t
Greater stability
This implicit solution is stable for large step sizes.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6
y
t
dt=0.01
dt=0.05
dt=0.15
exponential
How small can t be?
The errors in the derivative are approximately
e = E
_
y(t + t ) y(t )
t
_
=
y
t

m
+
t y

2
the relative error is then
e
y
=

m
t
+
t
2
y

y
We then nd the optimum value of t to be
t

m
How small can t be?
For single precision oating point numbers on the PC

m
= 1.1920929 10
7
.
Thus for values of t 0.0005 (and = 1.0) we are limited by
truncation error.
If we make t smaller than 0.0005 we run into numerical problems.
Better approximations
These are of O((t )
2
). First derivative:
dy(t )
dt

y(t + t ) y(t t )
2t
Second derivative:
d
2
y(t )
dt
2

y(t + t ) 2y(t ) + y(t t )
(t )
2
Algorithm
Starting from (2) we can write down a simple algorithm to generate
a solution at a number of points (in other words, we generate a
table of values).
Dene the initial condition at t = 0
Repeat the following until some nal value for t is reached:
Use equation (2) to determine the solution at t + t
# solution of differential equation dy/dt=-lambda y
# using Euler method
# rwf, July 2000
#(NOTE: lambda is a reserved word in Python!)
from Numeric import *
y = 1.0 # initial y
y0 = 1.0 # initial y0
dt = 0.1 # time step
lamb = 1.0 # decay constant
Nmax=101 # no. of steps
y=y0
for i in range(Nmax):
print i*dt, y
y = (1.0 - dt*lamb)*y
program decay
!
! solution of differential equation dy/dt=-lambda y
! using Euler method
! rwf, July 1997
!
implicit none
real :: y, y0 = 1.0 ! y and initial y0
real :: dt = 0.1 ! time step
real :: lambda = 1.0 ! decay constant
integer :: i, N=101 ! loop index and no. of steps
y = y0
do i = 1,N
print *, (i-1)*dt, y
y = (1.0 - dt*lambda)*y
end do
stop
end program decay
#include <iostream>
//
// solution of differential equation dy/dt=-lambda y
// using Euler method
// rwf, April 2001
//
int main() {
double y, y0 = 1.0 ; // y and initial y0
double dt = 0.1 ; // time step
double lambda = 1.0 ; // decay constant
int N=101 ; // loop index and no. of steps
y = y0 ;
for(int i=0; i<N; i++){
cout << i*dt << << y <<\n ;
y = (1.0 - dt*lambda)*y ;
}
}
Pathology
Various problems arise in numerical calculations.
The solution of the differential equation is subject to truncation
errors: the approximation of the problem gives rise to these.
Some of the exercises introduce another problem: that of
stability. The solution can oscillate around or diverge from the
true solution, even though the truncation error might be small.
This arises from exponentially growing solutions of the
difference equations that we use.
The oating point number system used in the computer is only
an approximation to the reals. Real numbers have to be
rounded to the nearest representable oating point number
giving rise to rounding errors. Also, the range of numbers is
restricted.
The Fast Fourier Transform
One of the most important algorithms.
Straightforward discrete Fourier transform: O(N
2
)
FFT: O(N log N)
Fourier series
A periodic function f (t ) = f (t + T) can be expanded in the Fourier
series:
f (t ) =

n=
c
n
e
int
where = 2/T.
c
n
=
1
T
_
T
0
f (t ) e
int
dt
Fourier series synthesis
-1.5
-1
-0.5
0
0.5
1
1.5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Fourier transform
A function f (t ) can be expressed as a Fourier transform:
f (t ) =
_

F() e
it
d
where
F() =
1
2
_

f (t ) e
it
d t
Transforming between the time and frequency domains.
Discrete Fourier transform
Time or frequency represented by a nite set of N points on a grid.
Grid spacing t = T/N, t
n
= nt . Let = 2/T. and
n
= n
F(
n
) =
1
N
N1

m=0
f (t
m
) e
i
n
t
n
=
N1

m=0
f (mt ) e
i2nm/N
Discrete Fourier transform (2)
The inverse transform is then
f (t
m
) =
N1

n=0
F(
n
) e
i2mn/N
Note
Time interval of data is T
Step size in frequency d = 2/(Ndt )
Frequencies from (N/2)d to (N/2)d
Discrete Fourier transform (3)
Use periodic boundary conditions.
Aliasing occurs for frequencies higher than the Nyquist
frequency
ny
= N/2 = (2/t )/2
Transform vector is complex.
The fast Fourier transform
The fast Fourier transform (FFT) is used to compute the DFT.
In Python we can use:
from Numeric import *
from FFT import * # In windows : from Fft
...
F=fft(f) # f is a Numeric array
g=inverse_fft(F)
Note that F and g will be complex.
FFT f (t ) = 2t
-1
-0.5
0
0.5
1
0 20 40 60 80 100 120 140 160 180
-100
0
100
200
300
400
500
600
0 5 10 15 20 25 30
real
imaginary
FFT f (t ) = 2t
-1
-0.5
0
0.5
1
0 20 40 60 80 100 120 140 160 180
-600
-400
-200
0
200
400
600
0 5 10 15 20 25 30
real
imaginary
Fourier series synthesis
-1.5
-1
-0.5
0
0.5
1
1.5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
FFT analysis
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 2 4 6 8 10 12
Power spectrum
The power spectrum gives essentially the intensity of the
time-domain data as a function of frequency.
It is dened by
P() = F()
2
and may be computed by
transform=fft(data) # transform is complex
powerspectrum=abs(transform*conjugate(transform))/len(data)**2
FFT power spectrum
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
0 20 40 60 80 100 120 140 160 180
0
0.05
0.1
0.15
0.2
0.25
0 5 10 15 20 25 30
Power spectrum
Convolution
The convolution f g of f (t ) and g(t ) is dened by
f g =
_

f () g(t )d
By the convolution theorem,
/[f g] = /[f ]/[g]
This provides a fast convolution algorithm via the FFT.
Convolution by FFT (1)
0
1
2
3
4
5
6
0 200 400 600 800 1000
Spectrum
Gaussian*100, s.d 10
0
0.05
0.1
0.15
0.2
0.25
0 200 400 600 800 1000
Convolution
Convolution by FFT (2)
0
0.5
1
1.5
2
2.5
3
3.5
4
0 200 400 600 800 1000
Spectrum
Gaussian*100, s.d 10
0
0.5
1
1.5
2
2.5
0 200 400 600 800 1000
Convolution
Correlation
The correlation corr(f , g) of f (t ) and g(t ) is dened by
corr(f , g) =
1

2
_

() g(t + )d
From the Fourier transforms F() = /[f (t )] and G() = /[g(t )],
/[corr(f , g)] = F

()G()
Autocorrelation
The Wiener-Khinchine theorem relates the autocorrelation of a
function to the power spectrum:
/[corr(f , f )] = F()
2
Part 4: Partial differential equations
Quantum mechanics
The time dependent Schr odinger Equation (TDSE) is a partial
differential equation related to the diffusion equation.
The Time Dependent Schr odinger equation
The Time Dependent Schr odinger equation is
i

t
(x, t ) =

2
2m

2
x
2
(x, t ) + V(x)(x, t )
= (7 + +)(x, t )
= 1 (x, t )
This describes the evolution (in time) of the wavefunction.
Potential barrier
As a problem involving the TDSE, consider the case of a Gaussian
wavepacket colliding with a barrier.
The barrier is described by the potential; we can take this to be
V(x) =
_

_
V
0
for x
L
x x
R
0 otherwise
cn
fft
fftp
Gaussian wavepacket
(x, 0) =
1
(2
2
)
1/4
e
(xx
0
)
2
/4
2
e
ik
0
x
The average momentum k
0
and the initial width are initial data
for the solution.
Recall the kinetic energy is
2
k
2
0
/2m.
Classically, the wavepacket will be reected if the kinetic energy is
less than the maximum barrier energy.
In QM, it will tunnel through the barrier.
Formal solution
The TDSE is:
i

t
(x, t ) = 1 (x, t )
The formal solution of the TDSE is given by
(x, t ) = e

1t
(x, 0)
with time-reversed solution
(x, 0) = e
+
i

1t
(x, t )
But 1 is an operator: what are the consequences?
Exponential of operator
The exponential of the operator 1 is dened by its series
expansion
e
1
= 1 + 1 +
1
2!
11 +
1
3!
111 + . . .
But here there is a problem:
the operators 7 and + do not commute
e

(7++)t
e

7t
e

+t
BCH theorem
One can use the Baker-Campbell-Hausdorff theorem in order to
generate approximations.
This states that
e
7
= e
J
e
(
iff
7 = J + ( +
1
2
[J, (] + . . .
Approximations
Using the rst two terms in the BCH expansion:
e
1
e

7t
e

+t
By including the rst commutator in the BCH expansion,
e
1
e

i
2
+t
e

7t
e

i
2
+t
to O((t )
2
).
Note that the order here is important!.
Propagating the wave function
We have an approximate form of the TDSE
i

t
(x, t ) = 1 (x, t ) e

i
2
+t
e

7t
e

i
2
+t
We can thus propagate the wave function over a small time interval
t
(x, t + t ) = e

1t
(x, t )
= e

i
2
+t
e

7t
e

i
2
+t
(x, t )
= e

i
2
+t
e

7t
(x, t )
(now what?)
Potential energy operator
Let us consider the problem of stepping a wave function (x, t )
through an interval t .
The term
(x, t ) = e

i
2
+t
(x, t )
is easily evaluated straightforward multiplication.
Kinetic energy operator
However, the kinetic energy operator is a (second) derivative:
the exponential of this is required:
e

7t
(x, t ) = exp
_

i
2m
t

2
x
2
_
_
(x, t )
_
This is more easily evaluated by exploiting the Fourier transform
(k, t ) =
1

2
_

(x, t )e
ikx
dk
Fourier transform of derivative
A fundamental property of the Fourier transform is
/
_
d
n
f (x)
dx
n
_
= i
n
k
n

f (k)
i.e. The Fourier transform of a derivative is multiplication.
To evaluate kinetic energy operator: FT, multiply, inverse FT.
Kinetic energy operator
e

7t
(x, t ) = /
1
_
/
_
e

7t
(x, t )
__
= /
1
_
/
__
1 +
i

2
2m
t

2
x
2
+ . . .
_
(x, t )
__
= /
1
__
1 +
i

2
k
2
2m
t + . . .
_

(k, t )
_
= /
1
_
e

7(k)t
/
_
(x, t )
_
_
where 7(k) =
2
k
2
/2m and
(x, t ) = e

i
2
+t
(x, t )
Solution
Thus
(x, t + t ) e

i
2
+t
/
1
_
e

7(k)t
/
_
e

i
2
+t
(x, t )
__
which may be easily programmed:
! psi, expV ,expT, temp are arrays of size nfft
! work is a workspace array
temp = expV*psi ! at t
call cfftf( nfft, temp, work )
temp=expT*temp
call cfftb( nfft, temp, work )
psi=expV*temp ! now at t+dt
Time step
Suppose we solve a 1-d problem on a domain of size L, with
x = L/N.
Then the step in k is k = 2/L = 2/Nx.
Note: must have negative k in upper half of array
expT(N+1-i)=expT(i) for i=1..N/2.
Use periodic boundary conditions (natural for FFT):
psi(N)=psi(1).
tdsea
tdseb
Part 5: Stochastic methods
Using random numbers
Simulate physical situation using (pseudo)random numbers.
Average to obtain results.
Also: stochastic methods from
oo: to aim at a mark; to guess.
Also: Monte Carlo methods.
Example: Radioactive decay
Example: simulate radioactive decay by choosing decaying atom
at random at every time step.
Start with N atoms.
For T time steps:
For N atoms:
decay probability is lambda
generate random x in (0,1)
if x <= lambda: decrease N by 1
decay
Example: Random walk
Take steps of xed size in random directions. e.g. diffusion.
Consider a 3-d situation.
Start at the origin.
For N steps:
Compute a random x, y and z displacement
Calculate the next position
Repeat M times and compute a histogram of the endpoints.
From this, compute whatever moments of the
distribution are required.
walk
Example: Phase transitions
Consider ferromagnetism.
Iron is magnetised alignment of atomic magnetic moments.
This alignment disappears above a certain temperature (Curie
temperature, T
C
)
(second order) phase transition.
Spontaneous reappearance of alignment as temperature drops
below Curie temperature.
Second order phase transitions are very interesting universal
features.
Example: Phase transitions
Magnetisation: m(T) (T
C
T)

Specic heat: C(T) T T

C

Similar critical exponents for magnetic susceptibility and

correlation length.
Example: Ising model
Very simple model of ferromagnetism:
Consider spins on a lattice.
Spins can only take two values (up or down).
Simple spin dependent interaction between nearest
neighbours.
System is in thermal equilibrium (canonical ensemble).
How can we compute ensemble averages, e.g. magnetisation at
temperature T?
Generating a random number
Generating a random number is not easy.
What is random? WE generally use random enough numbers.
e.g. the linear congruential generator.
Generates a sequence of integers N
1
, N
2
, . . . between 0 and m
with
N
i+1
= (aN
i
+ c) mod m
where a, c and m are constants. N
0
is the seed.
e.g. a = 16807, c = 0, m = 2
31
1.
(infamous: a = 65539, c = 0, m = 2
31
)
A random number in (0, 1) can be obtained by dividing by m.
Non-uniform random numbers
It is often required to have random numbers from some other
distribution than uniform (on (0, 1)).
These can often be generated with the aid of uniform random
numbers.
We consider two methods: transformation and rejection.
Transformations
The density of standard uniform random numbers is
p(x) =
_

_
1 if x (0, 1)
0 otherwise
Suppose we have a function y = f (x). We can then take a random
deviate x and generate a value y. Then the probability distribution
of y is given by p(y)dy or
p(y) =

dx
dy

p(x)
Transformations
Since p(x) = 1 we can nd the transformation by solving
dx
dy
= p(y)
i.e.
x(y) = P(y) =
_
y

p(y

) dy

In order to generate y from x we have to invert this function,

y(x) = P
1
(x)
and y has the required distribution.
Ex: Uniform distribution
Uniform distribution for x (a, b) rather than (0, 1), i.e.
p(y) =
_

_
1
ba
if y (a, b)
0 otherwise
Let us however apply the transformation method. Then
x(y) = P(y) =
_
y
a
1
b a
dy

=
y a
b a
and so y = a + (b a)x
where x is drawn from a uniform distribution.
Ex: Uniform distribution (a)
0
5
10
15
20
0 20 40 60 80 100
h1000.dat
Ex: Uniform distribution (b)
0
50
100
150
200
0 20 40 60 80 100
h10000.dat
Ex: Uniform distribution (c)
0
500
1000
1500
2000
0 20 40 60 80 100
h100000.dat
Ex: Exponential distribution
p(y) =
_

_
1

e
y/
for y 0
0 for y < 0
Typical of Poisson processes.
e.g. path lengths between collisions in a gas, or waiting times in
radioactive decay.
Ex: Exponential distribution
Then
x = P(y) =
_
y
0
1

e
y

/
dy

= 1 e
y/
and
y = ln(1 x)
where x is drawn from a uniform distribution.
Ex: Normal distribution
A common technique to generate normally distributed random
numbers is the Box-Muller transform.
A generalisation of the transformation technique to two dimensions
can be applied to yield two normal deviates from two uniform
deviates.
y
1
=
_
2 ln x
1
cos 2x
2
y
2
=
_
2 ln x
1
sin 2x
2
are independent Gaussian random variables with zero mean and
unit variance.
Ex: Normal distribution (a)
0
20
40
60
80
100
120
140
160
0 1 2 3 4 5 6 7 8 9
g200.dat
f(x)
Ex: Normal distribution (b)
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
g1000.dat
f(x)
Ex: Normal distribution (c)
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 10 20 30 40 50 60 70 80 90 100
g100000.dat
f(x)
Rejection method
Suppose that the required density is y = p(x) where x (a, b) and
y (0, p
max
).
Then
1
We choose two numbers (x, y) from a uniform random
distribution in the above ranges.
2
If y p(x) then x is accepted as the random number, else the
value is rejected and we start again at step 1.
Phase transitions and the Ising model
Local interactions give rise to long-range behaviour.
Changes of state.
First and second order transitions.
The Ising model
Model ferromagnetism by simple (very!) model.
Atom i interacts with neighbour j in external magnetic eld B:
E = J(i, j)s
i
s
j
gB s
i
where is Bohr magneton and g the gyromagnetic ratio.
The Ising model simplies this to
E = J s
i
s
j
Bs
i
with s
i
= 1.
The Ising model (2)
Model is implemented on lattice (we will consider 2-d):
The Ising model (3)
We can study the spin congurations as a function of temperature.
First look at system in absence of external magnetic eld: do we
get spontaneous ferromagnetism as temperature is lowered?
Congurations
Consider a particular conguration of spins.
The energy is
E

= J

(i,j)
s
i
s
j
The magnetisation is
M

i
s
i
Thermodynamics
To study the temperature dependence we can connect the system
to a heat bath at temperature T.
The probability of nding a particular conguration is determined
by a Boltzmann factor exp (E

/kT) and is
P

=
1
Z(T)
e
E

/kT
where Z is the partition function
Z(T) =

e
E

/kT
Ensemble averages
Thermodynamically interesting quantities are obtained by weighted
averages over ensembles of all possible congurations
The mean (internal) energy and the mean square energies are
E =
1
Z

e
E

/kT
and E
2
=
1
Z

E
2

e
E

/kT
The mean magnetisation of the system is then
M =

1
Z(T)
e
E

/kT
=

i
s
i
()
More thermodynamics
Quantities of interest in the Ising model are the specic heat
capacity
C =
1
N
s
E
T
=
1
N
s
1
kT
2
E

where = 1/kT, and the magnetic susceptibility

=
M
H
More thermodynamics (2)
Now,
E =
1
Z

e
E

=
1
Z
Z

ln Z
and
E
2
=
1
Z

2
Z

2
Hence the variance is
(E)
2
= (E E)
2
= E
2
E
2
=

2
ln Z
More thermodynamics (3)
It is difcult and time consuming to calculate C (or ) from
derivatives, hence the following is used
C =
1
N
s
1
kT
2
E

=
1
N
s
1
kT
2

2
ln Z
=
1
N
s
(E)
2
kT
2
These quantities can be calculated from the uctuations.
Solving the Ising model computationally
Simulate the system by generating a sequence of N
c
independent
congurations drawn from the Boltzmann distribution. The partition
function does not have to be calculated.
The magnetisation is then
M
1
N
c
N
c

i
s
i
()
The problem is to generate this sequence.
Metropolis algorithm
Generate sequence (Markov chain) of equilibrium congurations
by specifying transition probability.
Probability of nding the system in conguration at time t is P

(t ).
In equilibrium, dP

(t )/dt = 0, and the equilibrium probability P

is
given by the Boltzmann distribution.
P

=
1
Z
e
E

/kT
Metropolis algorithm (2)
The time evolution of a Markov chain is described by a master
equation
dP

(t )
dt
=

_
W( )P

(t ) W( )P

(t )
_
W( ) is the probability of a transition from state to .
For a discrete time system this becomes
P

(t + 1) = P

(t ) +

_
W( )P

(t ) W( )P

(t )
_
Metropolis algorithm (3)
At equilibrium, the loss and gain terms must be equal (detailed
balance).
W( )P

(t ) = W( )P

(t )
and thus
W( )
W( )
=
P

= e
E/kT
where E = E

.
Metropolis algorithm (4)
Metropolis et al. chose a simple transition probability which gives
this ratio:
W( ) =
_

_
e
E/kT
E > 0
1 E 0
Transition from to always occurs if the new conguration has a
lower energy, E

< E

.
If the new conguration has a higher energy, the transition can
occur with a Boltzmann probability.
Ising model: algorithm
An algorithm to generate congurations is then
Set up a lattice with random spins; it will have certain energy.
Choose a spin i at random and invert it; calculate the energy
change
E = 2Js
i

j
s
j
If E 0 accept the new conguration with the inverted spin.
If E > 0 accept it with probability p = exp(E/kT):
Generate a uniform random number x (0, 1). If x < p accept
the new spin, else leave it unchanged.
Ising model: practice
No need to ip spin; just calculate energy and ip if Metropolis
step will succeed.
One time step of a simulation corresponds to inverting and
testing every spin in the lattice.
A number of steps are required to thermalize the
congurations before samples are drawn for computing
averages.
Periodic boundary conditions
Precompute Boltzmann factors
Ising model: Metropolis step
E
i
= Js
i
_
neighbours
s
j
E=0 E=4 E=2 E=2 E=4
If ith spin is ipped:
E
i
= 2Js
i

neighbours
s
j
Magnetisation, 100x100 Ising model
0
2000
4000
6000
8000
10000
1 1.5 2 2.5 3 3.5 4 4.5 5
M
a
g
n
e
t
i
s
a
t
i
o
n
Temperature
100x100 Ising model
Magnetisation
Internal energy, 100x100 Ising model
-20000
-15000
-10000
-5000
0
1 1.5 2 2.5 3 3.5 4 4.5 5
I
n
t
e
r
n
a
l

e
n
e
r
g
y
Temperature
100x100 Ising model
Internal energy
Specic heat, 100x100 Ising model
0
0.5
1
1.5
2
2.5
3
1 1.5 2 2.5 3 3.5 4 4.5 5
S
p
e
c
i
f
i
c

h
e
a
t
Temperature
100x100 Ising model
Specific heat
Susceptibility, 100x100 Ising model
0
100
200
300
400
500
600
700
800
1 1.5 2 2.5 3 3.5 4 4.5 5
S
u
s
c
e
p
t
i
b
i
l
i
t
y
Temperature
100x100 Ising model
Susceptibility
Fluctuations in 25x25 Ising model
-600
-400
-200
0
200
400
600
800
0 100 200 300 400 500 600 700 800 900 1000
M
a
g
n
e
t
i
s
a
t
i
o
n
time
25x25 Ising model
T=2.0
T=2.3
T=3.5
Ising model: 100x100, T=1.5
Ising model: 100x100, T=2.2
Ising model: 100x100, T=2.3
Ising model: 100x100, T=3.0