Вы находитесь на странице: 1из 8

LECTURE 1: INTRODUCTION

1. Introduction

NONLINEAR DYNAMICS AND CHAOS

2. Maps
3. Flows
4. Fractals and Attractors

Patrick E McSharry
Systems Analysis, Modelling & Prediction Group

5. Bifurcations
6. Quantifying Chaos

www.eng.ox.ac.uk/samp

7. Nonlinear Time Series Analysis

patrick@mcsharry.net

8. Nonlinear Modelling and Forecasting

Tel: +44 20 8123 1574

9. Real-World Applications

Trinity Term 2007, Weeks 3 and 4


Mondays, Wednesdays & Fridays 09:00 - 11:00

10. Weather Forecasting


11. Biomedical Models
12. Time Series Analysis Workshop

Seminar Room 2
Mathematical Institute
University of Oxford

c 2007 Patrick McSharry p.1


Nonlinear dynamics and chaos

Suggested Reading

c 2007 Patrick McSharry p.2


Nonlinear dynamics and chaos

Relevant journal articles

Strogatz, S. H., Nonlinear Dynamics and Chaos: With Applications to Physics, Biology,
Chemistry, and Engineering, Reading, MA: Addison-Wesley (1994)

Yorke, J. and Li, T. Y., Period Three Implies Chaos, American Mathematical Monthly
82:985-992 (1975)

Eubank, S., and D. Farmer, An introduction to chaos and randomness. In Jen, E. (Ed.), 1989
Lectures in Complex Systems. Santa Fe Institute Studies in the Sciences of Complexity,
Lecture Vol. II, pp. 75-190. Reading, MA: Addison-Wesley, (1990)

May, R., Simple mathematical models with very complicated dynamics. Nature 261:
459-467 (1976)

Ott, E. and Sauer, T. and Yorke, J. Coping with Chaos, J. A. John Wiley & Sons, New York
(1984)
Ott, E., Chaos in Dynamical Systems, Cambridge: Cambridge University Press (1993)

Packard, N. and Crutchfield, J. and Farmer, J. D. and Shaw, R., Geometry from a time
series, Phys. Rev. Lett. 45: 712-716 (1980)
Crutchfield, J. P, N. H. Packard, J. D. Farmer, and R. S. Shaw. (1986) Chaos, Scientific
American 255:46-57 (1996)

Schuster, H.G., Deterministic Chaos: An Introduction, VCH, (1995).


Wiggins, S., Introduction to Applied Nonlinear Dynamical Systems and Chaos, Springer
(1990)
Kantz, H. and Schreiber, T., Nonlinear Time Series Analysis, Cambridge Univ. Press (1997)
Abarbanel, H.D.I, Analysis of Observed Chaotic Data, Springer, (1996)

c 2007 Patrick McSharry p.3


Nonlinear dynamics and chaos

c 2007 Patrick McSharry p.4


Nonlinear dynamics and chaos

Sources of information

Suggested special topics

Journals:
Physica D
Physical Review Letters
Physical Review E
Physics Letters A
International Journal of Bifurcation and Chaos

Look for evidence of low-dimensionality in real time series


Investigate the predictability of a real time series using nonlinear methods

Online discussion groups:


sci.nonlinear
www.jiscmail.ac.uk/lists/allstat.html
www.jiscmail.ac.uk/lists/timeseries.html
Websites:
www.societyforchaostheory.org (Society for chaos theory)
www.physionet.org (MIT-Harvard biomedical database and tools)
www.comdig.org (Complexity Digest)

Available time series:


electronic circuits
lasers
sunspot record
electricity data (demand, price, grid frequency)
weather data (temperature, precipitation, wind speed)
economic data (GDP, inflation, interest rates, unemployment)
financial data (stockmarket tick data, S&P500, Dow Jones, FTSE100)
biomedical signals (ECG, EEG, blood pressure, respiration, blood gases)
Construct and investigate a nonlinear model of a particular system
Compare nonlinear methods for forecasting, (e.g. RBFs vs. local linear)
Develop new methods for classification of health/disease using biomedical signals
TISEAN package
Matlab software

c 2007 Patrick McSharry p.5


Nonlinear dynamics and chaos

Special topic instructions

c 2007 Patrick McSharry p.6


Nonlinear dynamics and chaos

Linear analysis
Definition of linearity:
L(ax) = aL(x)

Should be one week of work

L(x + y) = L(x) + L(y)

Marking scheme:content 20 pts


presentation 5 pts

Principle of superposition: If x and y are solutions, then z = ax + by is also a solution


Advantages of linear models:
Often have analytical solutions
A large body of historical knowledge for helping with model specification and estimation
Less parameters - smaller chance of overfitting
Can employ Fourier spectral analysis and associated techniques

Two copies of final report required

Disadvantages of linear models:


Real-world systems are usually nonlinear
Linearity is a first order approximation and neglects higher orders
Stanislaw Ulam: nonlinear science is like non-elephant zoology
In practice, while underlying dynamics may be nonlinear, observed data may only provide
sufficient resolution for linear models
Need relevant null hypothesis tests for nonlinearity (surrogate data)

c 2007 Patrick McSharry p.7


Nonlinear dynamics and chaos

c 2007 Patrick McSharry p.8


Nonlinear dynamics and chaos

Normal distributions

Dynamical systems I
Variables

Linear

n=1

Growth, decay, equilibrium

Mean x0 and variance 2 :


(x x0 )2
exp
p(x) =
2 2
2 2
1

Easily manipulated: if x N (0, x2 ) and y N (0, y2 ), then x + y N (0, x2 + y2 )


Central limit theorem: sum of a large number of IID random variables (with finite mean and
variance) is normally distributed
For linear models:
Normal distributions are preserved by principle of superposition
Normally distributed forecast errors: Maximum likelihood gives least squares
Useful for calculating prediction intervals
Problems for nonlinear systems:
Use of normal distributions neglects possibility of asymmetric distributions
Fat tailed distributions imply larger probability of worse case scenarios (risk
management)

Exponential growth

Fixed points

RC circuit

Bifurcations

Radioactive decay

Overdamped systems, relaxational dynamics


Logistic equation for single species

n=2

All higher order moments are given in terms of x0 and

Nonlinear

Oscillations
Linear oscillator

Pendulum

Mass and spring

Anharmonic oscillator

RLC circuit

Limit cycles

2-body problem

Biological oscillators
Predator-prey cycles
Nonlinear electronics (van Der Pol)

n3

Chaos
Civil engineering

Strange attractors (Lorenz)

Electrical engineering

3-body problem (Poincare)


Chemical kinetics
Iterated maps (Feigenbaum)
Fractals (Mandelbrot)
Forced nonlinear oscillators (Levison, Smale)

c 2007 Patrick McSharry p.9


Nonlinear dynamics and chaos

Dynamical systems II
Variables

Linear

n1

Collective phenomena

c 2007 Patrick McSharry p.10


Nonlinear dynamics and chaos

Napoleans Armys Russian Campaign


Nonlinear

Coupled harmonic oscillators

Coupled nonlinear oscillators

Solid-state physics

Lasers, nonlinear optics

Molecular dynamics

Nonequilibrium statistical mechanics

Equilibrium statistical mechanics

Nonlinear solid-state physics


Heart cell synchronisation
Neural networks
Economics

Continuum

Waves and patterns

Spatio-temporal complexity

Elasticity

Nonlinear waves (shocks, solitons)

Wave equations

Plasmas

Electromagnetism (Maxwell)

Earthquakes

Quantum mechanics (Schrodinger,


Heisenberg)

General relativity (Einstein)

Heat and diffusion

Quantum field theory

Acoustics

Reaction-diffusion, biological waves

Viscous fluids

Fibrillation
Epilepsy
Turbulent fluids

Map drawn by the French engineer Charles Joseph Minard in 1861 to show the tremendous
losses of Napoleans army during his Russian Campaign of 1812

Life
c 2007 Patrick McSharry p.11
Nonlinear dynamics and chaos

c 2007 Patrick McSharry p.12


Nonlinear dynamics and chaos

Happiness time evolution

Happiness versus GDP

From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002)
From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002)
c 2007 Patrick McSharry p.13
Nonlinear dynamics and chaos

Time series from nonlinear systems

c 2007 Patrick McSharry p.14


Nonlinear dynamics and chaos

Mathematical characterisation

Time series: variables are recorded as a function of time


A

Steady state: a constant solution of a mathematical equation


e.g. Homeostasis: relative constancy of the internal environment with respect to variables
such as blood sugar, blood gases, blood pressure and pH. Control mechanisms constrain
variables to narrow limits. e.g. Following a hemorrhage, reflex mechanisms quickly restore
blood pressure to equilibrium values.

Oscillations: periodic solutions of mathematical equations


e.g. Heartbeat, respiration, sleep-wake cycles and reproduction
Irregular activity: intrinsic fluctuations, can even be present when external parameters are
relatively constant. Two distinct mathematical descriptions: noise and chaos

Noise:
Variability cannot be linked with any underlying stationary or periodic process
e.g. Fluctuating environment: eating, exercise, rest and posture affects heart rate, blood
pressure, blood-sugar levels and insulin levels
e.g. Respiratory Sinus Arrhythmia (RSA): heart rate increases during inspiration

Chaos:
Irregularity that arises in a deterministic system
Chaos can exist without influence of external noise

c 2007 Patrick McSharry p.15


Nonlinear dynamics and chaos

c 2007 Patrick McSharry p.16


Nonlinear dynamics and chaos

Noise versus Chaos

An exponential distribution, but not a Poisson process


2

Inter-event time sequences (e.g. heartbeat, neurons firing)

1.5

Believed to be a random process

ti+1

Simplest model for such a random process is a Poisson process:


Probability of event to occur in a very short time increment dt is Rdt
The probability R is independent of the previous history
Probability of two or more events occurring during dt is negligible
(Rt)k Rt
e
k!
st
1) following

(Poisson distribution)

Probability that interval between event and (k +

event is

R(Rt)k Rt
e
k!

0.5

0.1

0.2

0.3

0.4

0.5
ti

0.6

0.7

0.8

0.9

90

100

0.9

ln2/3
1.4
1.2

Probability of k events in time interval t is Pk (t) =


pk (t) =

1
ti

0.8
0.6

(PDF of Poisson process)

0.4
0.2

Average time between events is 1/R and variance is 1/R2

Nonlinear map, ti+1 = (1/R) ln |1 2 exp(Rti )| also gives p(t) = ReRt

10

20

30

40

50
i

60

70

80

0.1

0.2

0.3

0.4

0.5
t

0.6

0.7

0.8

Observation of an exponential probability density is not sufficient to identify a Poisson


process!

3.5
3
2.5

Use recurrence plots (e.g. ti+1 versus ti ) to identify structural equations

p(t)

2
1.5
1
0.5
0

c 2007 Patrick McSharry p.17


Nonlinear dynamics and chaos

Determinism and Predictability

c 2007 Patrick McSharry p.18


Nonlinear dynamics and chaos

Poincar

After Newton people believed in a deterministic, and hence, predictable Universe


Given for one instant an intelligence which could comprehend all the forces by which nature
is animated and the respective situation of the beings who compose itan intelligence and
sufficiently vast to submit these data to analysisit would embrace in the same formula the
movements of the greatest bodies of the universe and those of the lightest atom; for it,
nothing would be uncertain and the future, as the past, would be present before its eyes.
[P.S. Laplace, 1814]
Laplacian dream excludes stochastic laws of physics
Laplace acknowledged that we would never achieve the intelligence requireda tacit
appreciation that deterministic systems might not, in practice, be predictable
deterministic 6= predictable

After tackling the 3-body problem Poincar identified the phenomenon of sensitive
dependence on initial conditions (SDIC), this provided a definition of chaos
If we knew exactly the laws of nature and the situation of the universe at the initial moment,
we could predict exactly the situation of that same universe at a succeeding moment. But
even if it were the case that the natural laws had no longer any secret for us, we could still
only know the initial situation approximately. If that enabled us to predict the succeeding
situation with the same approximation, that is all we require, and we should say that the
phenomenon had been predicted, that is is governed by laws. But it is not always so; it may
happen that small differences in the initial conditions produce very great ones in the final
phenomena. A small error in the former will produce an enormous error in the latter.
Prediction becomes impossible, and we have the fortuitous phenomenon.[H. Poincar,
1903]

Laplace saw probabilities as a way to describe our ignorance of a deterministic system


Analytic expediency means most mathematics revolves around linear systems

c 2007 Patrick McSharry p.19


Nonlinear dynamics and chaos

c 2007 Patrick McSharry p.20


Nonlinear dynamics and chaos

Lorenzs butterfly effect

Sensitive dependence on initial condition

50

20

45

15
40
35

10

30
z

25
20

x(t) 0

15

10
5

10
0
20

15

15

10

0
x

10

15

20

Perfect model and perfect knowledge of observational uncertainty


Predictability varies with position

20

4
time [secs]

c 2007 Patrick McSharry p.21


Nonlinear dynamics and chaos

Chaos

c 2007 Patrick McSharry p.22


Nonlinear dynamics and chaos

Two faces of Chaos

Advent of digital computer allowed numerical investigation of nonlinear equations

The word chaos refers to disorder and extreme confusion

Lorenz found SDIC in a numerical model of the atmosphere and constructed the Lorenz
system to illustrate the effect in a simple system [1963]

To a scientist, it implies deterministic disorder

Yorke and Li coined the word chaos in 1975

On the contrary, a chaotic deterministic system is, in principle, perfectly predictable

May demonstrates chaos in the one-dimensional Logistic map in 1976

The sensitive dependence of the system dynamics to the initial conditions (SDIC) implies
that, in reality, any error in specifying the initial condition will lead to an erroneous prediction

Chaos becomes trendy, Chaos is published by James Gleick, 1987


Claims of chaos in the brain, heart, economy, stockmarket, ...
Investigations of nonlinear dynamical systems, claims of chaos are played down!

This might suggest that a chaotic system should be unpredictable

Laplace suggests using probabilistic predictions to overcome the problems, of diverging


trajectories, posed by chaotic systems
Chaos is sometimes used as a scapegoat: meteorologists blame chaos for inaccurate
predictions when it is often model inadequacy that is at fault
Important research: separating model inadequacy (structural and parametrical errors) from
effects of observational uncertainty

c 2007 Patrick McSharry p.23


Nonlinear dynamics and chaos

c 2007 Patrick McSharry p.24


Nonlinear dynamics and chaos

Deterministic versus Stochastic

Deterministic versus Stochastic

stochastic system

Deterministic:

xt+1 = axt + t

t : N (0, 1)

deterministic system

Everything is described by a single point in state space. This description


completely determines the future.

Stochastic:

Knowledge of present states does not determine the evolution of future states.

Can you tell if a system is stochastic or deterministic?

yt+1 = ayt + zt

Should a system be modelled as stochastic or deterministic?

zt+1 = 4zt (1 zt )

High dimensional deterministic chaotic system might be modelled as a stochastic system


State space trajectory of an autonomous, deterministic system never crosses itself
Sources of forecast uncertainty:
Uncertainty in the initial condition
Model inadequacy (parametrical and structural)

c 2007 Patrick McSharry p.25


Nonlinear dynamics and chaos

Introduction to Dynamical Systems

c 2007 Patrick McSharry p.26


Nonlinear dynamics and chaos

Properties of dynamical systems

A state is an array of numbers that provides sufficient information to describe the future
evolution of the system.
If m numbers are required, then these form an m-dimensional state vector x.
The collection of these state vectors defines an m-dimensional state space.
The rule for evolving from one state to another may be expressed as a discrete map or a
continuous flow:
map xt+1 = F (xt )

flow x(t)
= f (x(t))
Fixed point of a map: x0 = F (x0 )
Fixed point of a flow: x 0 = f (x0 ) = 0

Determinism: trajectories should not diverge when going forward in time


Invertibility: A dynamical system is invertible if each state x(t) has a unique predecessor
x(t 1). This implies that trajectories should never merge.
Thus continuous deterministic flows are always invertible!
Maps derived from flows (Poincar maps) are also invertible
Reversibility: if the dynamical system obtained by the transformation t t is equivalent to
the original one
Invariance under coordinate transforms: an invariant of a dynamical system represents a
fundamental property of that dynamical system, e.g. dimensions and Lyapunov exponents
System invariant offer a means of summarising the behaviour of a particular system: (e.g.
health and disease)

Non-autonomous system: x = f (x, t)


Autonomous system: x = f (x)
If the non-autonomous nature is due to periodic terms it can be made autonomous
Dissipative flow: f < 0 implies contracting state space volume

Hamiltonian systems are non-dissipative or conservative, preserving state space volume


(Liouville theorem)
c 2007 Patrick McSharry p.27
Nonlinear dynamics and chaos

c 2007 Patrick McSharry p.28


Nonlinear dynamics and chaos

Simple Pendulum

Fixed Points

An example of a simple two-dimensional dynamical system

Consider the fixed point of a flow f (x0 ) = 0

From Newtonss second law, knowledge of the forces, position and velocity are sufficient to
determine future motion

Let x(t) = x0 + (t)


= f (x0 + )

Pendulum (constrained to move in the plane)

= f (x0 ) + Dx f (x0 ) + O(||||2 )

Dynamics fully specified by the displacement angle (t) and the angular velocity (t)

Dx f (x0 ) = J

State vector given by x(t) = [(t), (t)]


Let m be the mass of the pendulum

J is Jacobian matrix of partial derivatives

g is the acceleration due to gravity

The solution is
(t) = eJt 0

l is the length of the pendulum


Let i be (distinct) eigenvalues of J

Tangential restoring force due to gravity: mg sin


Tangential force due to angular acceleration: ml

P 1 JP =

In the absence of friction, dynamics are governed by


d

dt
d

dt

where ii = i and ij = 0 if i 6= j
Let = P y so
y = et y0

g
sin
l
c 2007 Patrick McSharry p.29
Nonlinear dynamics and chaos

Classification of fixed points

<(i ) 6= 0 for all i: hyperbolic fixed point


Otherwise non-hyperbolic fixed point
Hyperbolic fixed points:
sinks: all <(i ) < 0

sources: all <(i ) > 0


saddle point: some positive and some negative

If =(i ) = 0: node

If =(i ) 6= 0: focus
Non-hyperbolic fixed points:
if <(i ) > 0 for some i: unstable

some <(i ) < 0 some = 0: neutrally stable


all <(i ) = 0: centre

c 2007 Patrick McSharry p.31


Nonlinear dynamics and chaos

c 2007 Patrick McSharry p.30


Nonlinear dynamics and chaos