Chapter 5: Box-Jenkins Methodology

Chapter 5:
5.1

Box-Jenkins Methodology

Introduction
ARIMA modeling was first introduced by George E. P. Box and Gwilym
M. Jenkins in 1976.
ARIMA is in short stands for Autoregressive Integrated Moving Average
models.
ARIMA models
Statistically sophisticated methods of extrapolating time series.
Commonly applied to time series analysis, forecasting and control.
Requires large run of time series data and technical expertise on the
part of the forecaster.

5.2

Stages in ARIMA Model Development

The basic of the Box-Jenkins modeling approach consists of three main
stages.
Stage 1: Model identification
Stage 2: Model estimation and validation
Stage 3: Model application

5.2.1 Stage 1: Model Identification

Identify the class of model most suitable to be applied to the given data set.
Common statistics used to identify the model type is the autocorrelation
function (ACF) and the partial autocorrelation function (PACF) of a
stationary data.
Although the class of model suited to a particular data series can be
determined by simply plotting the ACF and PACF, however this may not
necessarily be the best ultimate procedure.
This is because human eyes and minds may in many cases, can be
deceiving and inefficient.
In order to select the best fitted model, one needs to run several models.
The best fitted model (among the suitable models) can be determined
by applying certain statistical test procedures.

5.2.2 Stage 2: Model Estimation and Validation

Estimating the selected model, performing the necessary diagnostic testing
procedures and selecting the best model for forecasting purposes.
There are two important objectives that need to be achieved when fitting
the model.
The fitted values should be as close as possible to the actual values
Search for the estimated parameter values that minimize the error.
This can be done in an iterative manner, at each step in the process,
a new and improved model is obtained and these iterations are
repeated until a model with the minimum errors is obtained.

SQQS 3033 Business Forecasting

Chapter 5: Box-Jenkins Methodology

The models should require the least possible parameters consistent

with a good model fit
It is desirable to keep the model simple by excluding all those
irrelevant variables from the model

5.2.3 Stage 3: Model Application

If all test criteria are met and that the models fitness has been confirmed,
it is then ready to be used to generate the forecasts values.
Develop a system to monitor the forecast values produced.
When new data is available, the model needs to be revised and updated.
New and latest data are collected and incorporated into the existing
series.
New model is formulated and re-estimated.

5.3

The Assumption of Box-Jenkins Methodology

The application of Box-Jenkins methodology lies on the assumption that
concerns the characteristic of the initial data series.
It is assumed that the data series is stationary.
If the data is not stationary, then necessary procedures, such as using
suitable transformation, are performed in order to achieve stationary
condition in the series.

5.3.1 Stationary and Non-stationary Time Series

A series is said to be stationary if it fluctuates randomly around some fixed
values, generally either around the mean value of the series or it could be
some other constant values or even zero value.
A series is stationary if it does not show growth or decline over time
In other words, the data series does not indicate presence of trend
component.
A stationary series has
Constant mean
Constant variance
Constant autocorrelation structure
Example of stationary series
200
150
100
50
0
-50
-100
-150
-200

SQQS 3033 Business Forecasting

Chapter 5: Box-Jenkins Methodology

On the other hand, the time series that are not stationary is called nonstationary time series.
Series that is not constant around the mean or level (due to trend or
seasonal pattern) and hence can be expressed as deterministic function
of t.
Example: A non-stationary series in mean

1400
1200
1000
800
600
400
200
0

A series which in addition to be time dependent (with trend) also

shows an increase in the variability of the observations through time
Example: A non-stationary series in variance

14
12
10
8
6
4
2
0

Transformation: non-stationary to stationary

i) The non-stationary series can be made stationary (by removing the
trend/seasonal) by taking successive differences of the data
ii) Log transformation is commonly used to stabilize the variance

SQQS 3033 Business Forecasting

Chapter 5: Box-Jenkins Methodology

5.3.2 Autocorrelation Function (ACF) and Partial Autocorrelation Function

(PACF)
The non-stationarity of a series can be determined by
i) Through simple observation of plotted data
Calculate and plotting autocorrelation and partial autocorrelation
function.
ii) Using statistical test procedures (more accurate)
Unit root test
Augmented Dickey-Fuller (ADF) procedure is currently the most
commonly used.
For the time series, y1 , y 2 , , y n , the autocorrelation at lag k is defined as
nk

rk

(y
t 1

y )( y t k y )

(y
t 1

y) 2

where
y

1 n
yt
n t 1

The autocorrelation function (ACF) is a listing, or graph, of the

autocorrelations at lags k = 1, 2,
Example of behavior for ACF

SQQS 3033 Business Forecasting

Chapter 5: Box-Jenkins Methodology

In general, it can be shown that for non-seasonal data,

If the ACF of the time series values y1 , y 2 , , y n either cuts off fairly
quickly or die down fairly quickly, then the time series values should
be considered stationary.

If the ACF of the time series values y1 , y 2 , , y n dies down

extremely slowly, then the time series values should be considered
non-stationary.

Some example of time series and its ACF

SQQS 3033 Business Forecasting

Chapter 5: Box-Jenkins Methodology

Example 1
Table 1 shows the weekly sales of Ultra Shine Toothpaste.

yt
235
239
244
252
264
277

1
2
3
4
5
6

t
7
8
9
10
11
12

yt
286
295
310
325
336
344

Table 1
t
13
14
15
16
17
18

yt
355
368
384
398
413
423

19
20
21
22
23
24

yt
435
446
456
466
478
491

a) Using the first 10 observations, calculate r2 and r4 .

b) The autocorrelation function for the data is given in Figure 1 and the
autocorrelation function for data after first differencing is given in Figure 2.
Comment about the stationarity of the data.
SALES

SALES

Coefficient

1.0

Coefficient

1.0

Upper
Confidence
Limit

Upper
Confidence
Limit
Lower
Confidence
Limit

0.0

Lower
Confidence
Limit

0.5

ACF

ACF

0.5

0.0

-0.5

-0.5

-1.0

-1.0
1

9 10 11 12 13 14 15 16

Lag Number

Figure 1

9 10 11 12 13 14 15 16

Lag Number

Figure 2

Solution:

SQQS 3033 Business Forecasting

Chapter 5: Box-Jenkins Methodology

For the time series, y1 , y 2 , , y n , the partial autocorrelation at lag k is

defined as
r1
if k 1

k 1

rk rk 1, j rk j
rkk
j 1
if k 2, 3,
k 1

1 rk 1, j r j
j 1

where
rk , j rk 1, j rkk rk 1,k j for j 1, 2, , k 1
The partial autocorrelation function (PACF) is a listing, or graph, of the
partial autocorrelations at lags k = 1, 2,
The PACF is being used to determine the Box-Jenkins model.
Example 2
Table 2 shows the autocorrelation function for data in Example 1. Calculate
the partial autocorrelation at lag 1, 2 and 3.

k
1
2
3
4

rk
0.887
0.769
0.694
0.527

k
5
6
7
8

Table 2
k
rk
0.406
9
0.290
10
0.176
11
0.064
12

rk
-0.035
-0.123
-0.199
-0.268

k
13
14
15
16

rk
-0.326
-0.375
-0.410
-0.426

Solution:

SQQS 3033 Business Forecasting

Chapter 5: Box-Jenkins Methodology

5.4

Differencing
A non-stationary series can easily be made stationary by the process of
differencing.
This is analogous to the process of removing the trend pattern from the
actual data.
This is usually referred to as detrending process, and the resulting time
series is called detrended series which is now stationary.
Let the time series with linear trend, yt t
Then the first order differencing of y t ,
yt yt yt 1
( t ) [ (t 1)]
( t ) ( t )
t t

which is a constant.

Occasionally, taking the first difference may not be sufficient to induce

stationarity in the series. This calls for the second degree differencing.
The second order differencing of y t ,
2 yt (yt )
yt yt 1

( yt yt 1 ) ( yt 1 yt 2 )
yt yt 1 yt 1 yt 2
yt 2 yt 1 yt 2
A series that requires first difference to be stationary is said to be
integrated of order one or first order integrated series.

SQQS 3033 Business Forecasting

Chapter 5: Box-Jenkins Methodology

Table 3: Example differencing non-stationary time series to stationary

Non-stationary (with trend)
After first differencing

5.4.1 Backward Shift Operator

A useful notational device used when differencing is performed is called
the backward shift operator, denoted by B.
The operator B is used to indicate the number of backward steps a time
series value may take.
That is, B yt yt 1
B 2 y t B( B y t ) y t 2

two-period shift backward

B yt yt 12
twelve-period shift backward
Therefore, the first order differencing can be written as
yt yt yt 1
yt B yt
12

(1 B) y t
NOTE:

The d th order differencing is differencing d times is not the same

as d th difference, which is essentially yt yt d .

SQQS 3033 Business Forecasting

Chapter 5: Box-Jenkins Methodology

Example 3
Write the following using backward shift operator
a) 3 yt
b) yt 1 yt 1 2 yt 2 t 1 t 1 2 t 2
Solution:

5.5

The Basic Model of the Box-Jenkins Methodology

There are three basic models for stationary data
Autoregressive model, AR(p)
Moving average, MA(q)
Mixed autoregressive and moving average, ARMA(p, q)
Model for data with trend
Mixed autoregressive integrated moving average model,
ARIMA(p, d, q)
Model for data with seasonal component
ARIMA with seasonal component, ARIMA(p, d, q)(P, D, Q)s

5.5.1 Autoregressive Model, AR(p)

In the AR model, the current value of the series is defined as a function of
its previous values plus an error term.
Mathematically, it is written as
yt 1 yt 1 2 yt 2 p yt p t
where and j , j 1, 2, , p are constant terms or parameters to
be estimated
y t is the dependent or current value
y t p is the p th order of lagged dependent or current value

t is the error term which is normally and independent

distributed with mean zero and variance, 2
Using the backward shift operator, AR(p) model can be written as
yt 1 yt 1 2 yt 2 p yt p t

yt 1 B yt 2 B 2 yt p B p yt t
yt 1 B yt 2 B 2 yt p B p yt t

1 B B
1

SQQS 3033 Business Forecasting

p B p yt t

10

Chapter 5: Box-Jenkins Methodology

Example 4
Write the equation for the following AR model.
a) AR(1)
b) AR(2)
c) AR(4)
Solution:

5.5.2 Moving Average Model, MA(q)

Note that in Chapter 2 and 3, the moving average is the average of the
actual observations, y t whereas in this chapter, it is a function of the error
terms.
MA model links the current values of the time series to the random errors
that have occurred in the previous periods.
Mathematically, it is written as
yt 1 t 1 2 t 2 q t q t
where is the mean about which the series fluctuates
j , j 1, 2, , q are constant terms or parameters to be
estimated
t q are the error terms assumed to be independently
distributed over time
Using the backward shift operator, MA(q) model can be written as
yt t 1 t 1 2 t 2 q t q

yt t 1 B t 2 B 2 t q B q t

y t 1 1 B 2 B 2 q B q t
Example 5
Write the equation for the following MA model.
a) MA(1)
b) MA(2)
c) MA(5)
Solution:

SQQS 3033 Business Forecasting

11

Chapter 5: Box-Jenkins Methodology

5.5.3 Mixed Autoregressive and Moving Average Model, ARMA(p, q)

Mathematical model of ARMA is given as
yt 1 yt 1 2 yt 2 p yt p 1 t 1 2 t 2 q t q t
Using backward shift operator,
1 1 B 2 B 2 p B p yt 1 1 B 2 B 2 q B q t

Example 6
Write the equation for the following ARMA model.
a) ARMA(1, 1)
b) ARMA(2, 1)
c) ARMA(2, 2)
d) ARMA(1, 3)
Solution:

SQQS 3033 Business Forecasting

12

Chapter 5: Box-Jenkins Methodology

5.5.4

Mixed Autoregressive Integrated and Moving Average Model,

ARIMA(p, d, q)
When the stationarity assumption of the variable is not met, the data series
needs, firstly, to be differenced in order to achieve stationarity.
d denotes the number of time the variable y t needs to be differenced in
order to achieve stationarity.
Example 7
Write the equation for the following ARIMA model.
a) ARIMA(1, 1, 1)
b) ARIMA(2, 1, 1)
c) ARIMA(1, 1, 2)
d) ARIMA(2, 1, 2)
Solution:

SQQS 3033 Business Forecasting

13

Chapter 5: Box-Jenkins Methodology

5.5.5 ARIMA with Seasonal Component, ARIMA(p, d, q)(P, D, Q)s

For series with seasonal component presence, then the additional
differencing is necessary to be performed in order to eliminate the
seasonality effect.
This is called seasonal differencing and is performed as followed
For a monthly data, z t yt yt 12
For a quarterly data, z t yt yt 4
If even after performing the seasonal difference, the resulting z t remained
non-stationary, then further differencing is performed, wt z t z t 1 .
Example 8
Write the equation for the following model.
a) ARIMA(1, 1, 1)(1, 1, 1)4
b) ARIMA(1, 1, 2)(1, 1, 1)4
c) ARIMA(2, 1, 1)(1, 1, 1)4
d) ARIMA(1, 1, 1)(1, 1, 0)12
Solution:

SQQS 3033 Business Forecasting

14

Chapter 5: Box-Jenkins Methodology

5.6

Steps in the Identification of the Box-Jenkins Model

Step 1
Construct a time plot for the original data series
Try to identify any unusual observation
If exist, decide whether a transformation is necessary
If necessary, transform to achieve stationary in variance, for
example do log transformations.
Plot also the ACF and PACF to confirm the stationarity condition of
the series.
Step 2
If the series appears non-stationary, perform the first difference.
For non-seasonal data series the first difference is sufficient.
If seasonality exists, perform the seasonal difference.
If non-stationarity condition persists after the seasonal difference,
perform the non-seasonal difference.
Plot also the ACF and PACF to confirm the stationarity condition of
the series
Step 3
When stationarity condition has been achieved, examined the ACF and
PACF to see whether any discernible pattern of the data series exists.
The ACF and PACF may reveal the type of model appropriate
Model with seasonal component is suggested by larger autocorrelation
/ partial autocorrelation at seasonal lags.
For example: at lags 12, 24, for monthly data and lags 4, 8,
for quarterly data.
Step 4
In the model identification stage, it may be slightly easier to select pure
AR or pure MA models.
But when selecting a mixed ARIMA model, the decision to decide on
the values of p and q is much more difficult.
More so for model with seasonal component.
Hence, it is worth considering several possible models.
To finally determine the best fitted model, one needs to use several
statistical measures such as MSE, AIC/BIC or Box-Pierce (Ljung-Box)
statistic.
Practical tips
In many practical applications, it is very difficult to tell whether data is
from an AR(p) or MA(q)
Choose best-fitting model
DO NOT build models with
Large numbers of MA terms
Large numbers of AR and MA terms together

SQQS 3033 Business Forecasting

15

Chapter 5: Box-Jenkins Methodology

5.6.1 General Guidelines for Model Identification

If ACF decays exponentially and the PACF has spikes, the process can
best be captured by an AR model.
In this case, the order of the AR model (the value of p) equals to the
number of significant spikes in the PACF.
If PACF decays and the ACF has spikes, the process can best be captured
by MA model.
In this case, the order of the MA model (the value of q) equals to the
number of significant spikes in the ACF.
If both the ACF and PACF are characterized by irregular patterns, the
process can best be captured by an ARMA model.
The order of the ARMA model (the value of p and q) equals to the
number of significant spikes in the PACF and ACF respectively.
Table 4: General Nonseasonal Models
ACF
PACF
Model Type
Exponential decay (die down) Cut off after lag p
AR(p) where p is
and/or damped sinusoid
the number of
spikes in the PACF

Cut off after lag q

Exponential decay (die down)

and/or damped sinusoid

MA(q) where q is
the number of
spikes in the ACF

Exponential decay (die down) Exponential decay (die down) ARMA(p, q) where
and/or damped sinusoid
and/or damped sinusoid
p is the number of
spikes in the PACF
and q is the number
of spikes in the
ACF

SQQS 3033 Business Forecasting

16

Chapter 5: Box-Jenkins Methodology

Table 5: Some examples of ACF and PACF for different models

Model
ACF / PACF
AR(1)

AR(2)

MA(1)

SQQS 3033 Business Forecasting

17

Chapter 5: Box-Jenkins Methodology

Model
MA(2)

ACF / PACF

ARMA(1,1)

Example 9
Based on the given ACF and PACF of the original data (given in the Excel
file), decide the suitable Box-Jenkins model for this time series. Write down
the equation of the model.
45.00
40.00
35.00
30.00
25.00
20.00
1

16 31 46 61 76 91 106 121 136

Time series plot for original data

SQQS 3033 Business Forecasting

18

Chapter 5: Box-Jenkins Methodology

ACF and PACF of the original data

The ACF cuts off after lag 2 (there are 3 spikes) and the PACF die down.
The suitable models are MA(2) or MA(3).
Equation of MA(2)
yt 1 t 1 2 t 2 t

y t 1 1 B 2 B 2 t

Equation for MA(3)

yt 1 t 1 2 t 2 3 t 3 t

y t 1 1 B 2 B 2 3 B 3 t

Example 10
Based on the given ACF and PACF of the original data (given in the Excel
file), decide the suitable Box-Jenkins model for this time series. Write down
the equation of the model.
1200
1000
800
600
400
200
0
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89

Time series plot for original data

SQQS 3033 Business Forecasting

19

Chapter 5: Box-Jenkins Methodology

ACF and PACF of the original data

From the time series plot and the ACF (die down extremely slow) of the
original data, it shows that the series is not stationary (with trend), thus need to
use first order differencing to transform the data to obtain stationary condition.

ACF and PACF of the first order differencing for the original data

From the ACF of the first order differencing for the original data, it die down
quickly, thus the data is stationary. The PACF of the first differencing for the
data cut off after lag 1.
Thus the suitable model is ARIMA(1, 1, 0).
The equation of the model is
wt 1 wt 1 t
where
wt yt yt 1
wt 1 yt 1 yt 2

SQQS 3033 Business Forecasting

20

Chapter 5: Box-Jenkins Methodology

Example 11
Based on the given ACF and PACF of the original data (given in the Excel
file), decide the suitable Box-Jenkins model for this time series. Write down
the equation of the model.
450.00
400.00
350.00
300.00
250.00
200.00
1

12 23 34 45 56 67 78 89 100 111 122 133 144 155 166 177

Time series plot for original data

ACF and PACF of the original data

From the time series plot, the ACF (die down extremely slow) and PACF
(with spikes at lag 1 and 13) of the original data, it shows that the series is not
stationary (with trend and seasonal component), thus need to use seasonal
differencing and ordinal differencing to transform the data to obtain stationary
condition.

SQQS 3033 Business Forecasting

21

Chapter 5: Box-Jenkins Methodology

ACF and PACF of the first seasonal differencing for the original data

The data is still not yet stationary after the first seasonal differencing, so need
to apply ordinal differencing to the data.

ACF and PACF of original data after the first seasonal differencing and first
order ordinal differencing

From the ACF and PACF of the data after the first seasonal differencing and
the first order ordinal differencing, the data achieved the stationary condition.
From the ACF, it suggests q may take the values 1 or 2 and Q = 1 (2 spikes at
lag 1 and 4, 1 spike at lag 12), while from the PACF, it suggests p may take
the value of 1 or 2 and P = 1 (2 spikes at lag 1 and 4, 1 spike at lag 12).
Thus the possible model may be
ARIMA(1, 1, 1)(1, 1, 1)12
ARIMA(2, 1, 1)(1, 1, 1)12
ARIMA(1, 1, 2)(1, 1, 1)12
ARIMA(2, 1, 2)(1, 1, 1)12

SQQS 3033 Business Forecasting

22

Chapter 5: Box-Jenkins Methodology

Equation for ARIMA(1, 1, 1)(1, 1, 1)12

Let zt yt yt 12 (1 B12 ) yt and wt zt zt 1 (1 B) zt
The non-seasonal model ARIMA(1, 1, 1)
wt 1 wt 1 1 t 1 t
(1 1 B) wt (1 1 B) t
(1 1 B)(1 B) z t (1 1 B) t
(1 1 B)(1 B)(1 B12 ) yt (1 1 B) t ---(1)

The seasonal model ARIMA(1, 1, 1)12

z t 1 z t 12 1 t 12 t

(1 1 B12 ) z t (1 1 B12 ) t
(1 1 B12 )(1 B12 ) yt (1 1 B12 ) t

---(2)

Combine the common terms in (1) and (2):

(1 1 B)(1 B)(1 1 B12 )(1 B12 ) yt (1 1 B)(1 1 B12 ) t

SQQS 3033 Business Forecasting

23

Chapter 5: Box-Jenkins Methodology

5.7

Estimation and Validation of the Box-Jenkins Model

The Box-Jenkins models are usually estimated based on sample statistic
that must be tested to ensure their validity as estimates of the true
population parameter values.
There are three components of the validation process
Statistical validation or residual diagnostics
Parameter validation
Model validation
Three common statistical measure used when validating the ARIMA
models are
Akaikes Information Criteria (AIC)
Bayesian Information Criterion (BIC)
The Box-Pierce Q statistic

5.7.1 The Akaikes Information Criteria (AIC)

A common measure of the fitness of an ARIMA model
Measures the penalty on the likelihood for each additional term included in
the model.
If the extra term does not improve the likelihood more than the penalty
amount, it is not worth adding into the model
A model is considered as having a better fit if the value of its AIC is the
smallest.
The value of AIC standing alone is meaningless
The AIC of the competing models need to be pitted against one another
so that one best model (with the lowest value of AIC) can be picked
among the competing models
5.7.2 Bayesian Information Criterion (BIC)
Developed by Schwarz (1978) for the purpose of selecting models that
achieve the most accurate out-of-sample forecasts by balancing between
the models complexity and goodness-of-fit.
The best model is the one that has the lowest BIC value.
It needs to be noted that the BIC is usually used as model selective
criterion when no firm theoretical or empirical reasons are available to
decide one model over the other.
In SPSS output, BIC is represented by SBC.
5.7.3 The Box-Pierce Q Statistic
The Box-Jenkins framework assumes that the residuals are not correlated
with each other
It is assumed that there is no systematic pattern in the residuals
The residuals are independent of one another
When there is systematic pattern in the behaviour of the residuals, then the
model is said to be mis-specified.
A procedure to check for mis-specification is to check for the presence of
correlation among the residuals
Carried out by calculating the chi-squared value of the error terms

SQQS 3033 Business Forecasting

24

Chapter 5: Box-Jenkins Methodology

Hypotheses
H 0 : The residuals are white noise (the residuals are random and
independent of each another)
H1 : The residuals are not white noise (model is mis-specified or
inadequate)
Test statistic
Box-Pierce Q statistic
h

Q N ' rk2
k 1

where

N ' N d
N is the number of observations in the time series
h is the maximum lags being tested
p is the number of AR terms
q is the number of MA terms
rk is the autocorrelation of the residual terms
d is the degrees of the differencing applied to the original data
Ljung-Box statistic
h
r2
Q* N ' ( N '2) k
k 1 N ' k
Decision
Critical value: h2 p q ,

Reject H 0 if the test statistic is larger than the critical value.

If H 0 is rejected, the best course of action is to try other model variations.
NOTE: Some of the statistic software does not show the value of BoxPierce Q and Ljung-Box statistics

SQQS 3033 Business Forecasting

25

Chapter 5: Box-Jenkins Methodology

Example 12
Using the data for Example 9, the model fitted using SPSS is given as follow.
Write the equation of the fitted models and decide which model is better.

SPSS Output for ARIMA(0, 0, 2)

SPSS Output for ARIMA(0, 0, 3)

From the SPSS output, the estimated model for MA(2) is given by
y t 35.1825 0.5135 t 1 0.6415 t 2
AIC = 693.3416
SBC = 702.3735
From the SPSS output, the estimated model for MA(3) is given by
y t 35.1848 0.4354 t 1 0.6027 t 2 0.1375 t 3
AIC = 692.9668
SBC = 705.0093

SQQS 3033 Business Forecasting

26