Introduction to Time Series

and ARIMA Modeling

05/11/2011 1
 Types of Data Series
• Cross Sectional Data
• Time Series Data
• Panel Data

05/11/2011 2
 Time Series Data

• A time series is defined as a set of quantitative

observations arranged in chorological order.

• Statistically every observation is a random

realization of a stochastic process.

05/11/2011 K N Badhani/KU/Jaipuria 3
 0 1 2 3 4

05/11/2011 K N Badhani/KU/Jaipuria 4
 4

3
2
1
0

05/11/2011 K N Badhani/KU/Jaipuria 5
 Features of Time Series Data
• Time Series data may be related to their own past
(such as autoregressive process, moving average
process)
• They may be subject to deterministic trends,
seasonality, cycles etc.
• In multi-variate context, they may have cross-
correlations. (For example; advertisement expenses for the month
of January will affect the sale not only in the month of January but may
affects the sale of February and coming months.)

05/11/2011 K N Badhani/KU/Jaipuria 6
Tools to Identify a underlying behavior
of Time Series
• Time Series Plot
• Correlogram

05/11/2011 K N Badhani/KU/Jaipuria 7
 Stationarity

05/11/2011 K N Badhani/KU/Jaipuria 8
 Stationary Process
• A time series process is called strictly
stationary if all of its parameters are time
invariant.

05/11/2011 K N Badhani/KU/Jaipuria 9
 Weakly Stationary Process
• A process is called weakly stationary (or
covariance stationary) if following three
parameters are time invariant
– Mean
– Variance
– Auto covariance

05/11/2011 K N Badhani/KU/Jaipuria 10
 Characteristics of a Stationary Process
• It is mean reverting.
• It has a finite memory.
• The impact of a shock dies out with time.
• It has a defined variance.

05/11/2011 K N Badhani/KU/Jaipuria 11
 Characteristics of a Non-Stationary
Process
• It is not mean-reverting.
• It has an infinite memory.
• The impact of a shock does not die out with
time.
• It does not have a defined variance. Rather
the variance may increase with time.

05/11/2011 K N Badhani/KU/Jaipuria 12
 Stationary of a AR Process
• An AR(1) process is stationary if its AR-
coefficient is less than 1.
• In the followinng process

y t  y t 1   t
When  is 1 it becomes a random walk

05/11/2011 K N Badhani/KU/Jaipuria 13
 Stationary of a AR Process
• The variance of a AR(1) process is

2
variance of ( yt ) 
1  2

• If AR coefficient is 1 this will become infinite.

05/11/2011 K N Badhani/KU/Jaipuria 14
 Why do we need a Stationary Process
• We can not predict a non-stationary process.
• It produce spurious regression

05/11/2011 K N Badhani/KU/Jaipuria 15
 Spurious Regression
• Regression between two non-stationary series
may be spurious.
• In case of spurious regression t-statistic is not
valid.
• Symptoms:
– High R2
– Low Darwin-Watson
– R2 > d

05/11/2011 K N Badhani/KU/Jaipuria 16
05/11/2011 K N Badhani/KU/Jaipuria 17
 Types of Non-Stationarity

Trend Difference
Stationary Stationary
Process Process

05/11/2011 K N Badhani/KU/Jaipuria 18
 Trend Stationary Process

The process is basically non-

stationary as its mean is not constant

However, it is stationary around a

deterministic trend. It has a constant
variance around that trend

It can be made ‘stationary’

after de-trending.

05/11/2011 K N Badhani/KU/Jaipuria 19
 Difference Stationary Process
This process is following a Stochastic Trend,. Its
variance around the trend is not constant.

This process can be made by differencing

it.
y(t) - y(t-1).
The process is also called Integrated
05/11/2011
Process
K N Badhani/KU/Jaipuria 20
 A Difference Stationary Process with Constant Mean, but
non-stationary Variance

05/11/2011 K N Badhani/KU/Jaipuria 21
 Dickey-Fuller Unit Root Test
y t  y t 1   t
 y t  y t 1  y t 1  y t 1   t
 yt  (1   ) yt 1   t
 yt  yt 1   t
05/11/2011 K N Badhani/KU/Jaipuria 22
 Augmented Dickey-Fuller Unit Root
Test

p
yt  c  T  yt 1    j yt  j  t
j 1

05/11/2011 K N Badhani/KU/Jaipuria 23
 The Null Hypothesis
• The process has a Unit Root

05/11/2011 K N Badhani/KU/Jaipuria 24
 Alternative Tests
Depending on which terms we include in model
specification, the following hypotheses are
tested:
– Unit root vs. stationary process with zero mean
– Unit root with drift vs. stationary process with
constant mean
– Unit Root with drift and trend vs. stationary
process with trend.

05/11/2011 K N Badhani/KU/Jaipuria 25
 What to Do when Process is not
Stationary:
• For Multivariate Analysis: Cointegration Test.
• For Univariate Analysis:
– Detrend it
– Make it Stationary by differencing

05/11/2011 K N Badhani/KU/Jaipuria 26