System Identification

Control Engineering B.Sc., 3rd year

Technical University of Cluj-Napoca
Romania

Lecturer: Lucian Busoniu

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Part IV
Frequency analysis. Correlation analysis

Frequency analysis

Correlation analysis

Table of contents

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Correlation analysis: Matlab example

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Classification

Recall Types of models from Part I:

1
2
3

Mental or verbal models

Graphs and tables (nonparametric)
Mathematical models, with two subtypes:
First-principles, analytical models
Models from system identification

Frequency analysis produces a model in the form of the frequency

response the Bode diagram.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Response to sinusoidal input

Apply unit sine of frequency to system with transfer function H:

u(t) = sin(t)
In steady-state, the output is a sine of the same frequency, possibly
with different amplitude and phase:
yss (t) = k() sin[t + ()]
Gain: k() = |H(j)|, where j is the imaginary unit, so H(j)
is a complex number.
Phase: () = arg(H(j)).

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Frequency analysis
By walking through a grid of frequencies , and measuring the gains
k () and phase shifts (), a Bode diagram for the system is
experimentally created:

a nonparametric model, which can be used to analyze the system

further or to perform control design.

Frequency analysis

Correlation analysis

Table of contents

Frequency analysis

Correlation analysis
Analytical development
A practical algorithm. FIR model
Accuracy guarantee

Correlation analysis: Matlab example

Correlation analysis: Matlab example

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Context

Transient analysis of step and impulse responses gives a rough,

heuristic model of the system.
The upcoming methods provide:
Fully implementable algorithms, which derive a solution
programatically.
Solution accuracy guarantees (under appropriate conditions).

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Classification

Recall Types of models from Part I:

1
2
3

Mental or verbal models

Graphs and tables (nonparametric)
Mathematical models, with two subtypes:
First-principles, analytical models
Models from system identification

Correlation analysis is still a nonparametric method; it produces an

impulse response model.

Frequency analysis

Correlation analysis

Recall: discrete-time model

Correlation analysis: Matlab example

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Discrete-time impulse response

Discrete-time, unit impulse signal:

(
1 k =0
uI (k ) =
0 k >0
Discrete-time impulse response:
yI (k ) = h(k),

k 0

h(k ), k 0 is also called the weighting function of the system.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Convolution
The (disturbance-free) response to an arbitrary signal u(k ) is the
convolution of the input and the impulse response:
y (k ) =

h(j)u(k j)

j=0

j (k ) equal to u(j) at k = j, and 0

Intuition: Consider a signal u
elsewhere; just a shifted and scaled unit impulse:
j (k ) = u(j)uI (k j)
u
j (k) is a shifted and scaled impulse response:
So, the response to u
yj (k ) = u(j)h(k j)
j , and due to linearity:
Now, u(k ) is the superposition of all signals u
y (k ) =

k
X
j=0

yj (k ) =

k
X
j=0

u(j)h(k j) =

k
X
j=0

h(j)u(k j) =

h(j)u(k j)

j=0

where zero initial conditions were assumed, i.e. u(j) = 0j < 0.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Impulse-response model

y (k ) =

h(j)u(k j) + v (k)

j=0

Includes, in addition to the ideal model, a disturbance term v (k).

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Assumptions

Assumptions
1
2

The input u(k) is a stationary stochastic process.

The input u(k) and the disturbance v (k ) are independent.

Recall:
Independence of random variables.
Stationary stochastic process: constant mean at every time step,
covariance only depends on difference between time steps and
not on absolute time.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Covariance functions

The covariance functions are defined as follows:

ryu ( ) = E {y(k + )u(k)}
ru ( ) = E {u(k + )u(k)}
Note: These quantities are true covariances only when the input and
output are zero-mean.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Relationship of covariances and impulse response

If there were no disturbance, then:

ryu ( ) = E {y(k + )u(k)}

X
=E
h(j)u(k + j) u(k)

j=0

X
j=0

h(j)E {u(k + j)u(k )} =

h(j)ru ( j)

j=0

The errors coming from the disturbance are dealt with later, using
linear regression.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Impulse response identification

Writing the covariance relationship for all :
ryu (0) =

ryu (1) =

X
j=0

h(j)ru (j)
h(j)ru (1 j)

j=0

...
we obtain (in principle) an infinite system of linear equations:
Coefficients ru ( ), ryu ( ).
Unknowns h(0), h(1), . . . , solution of the system.
Next, a practical algorithm working with finite data is given.

Frequency analysis

Correlation analysis

Table of contents

Frequency analysis

Correlation analysis
Analytical development
A practical algorithm. FIR model
Accuracy guarantee

Correlation analysis: Matlab example

Correlation analysis: Matlab example

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Covariances from data

Consider we are given signals u(k ), y (k ) with k = 1, . . . , N.
Denote by {a(k)}k=b,c,... the signal a(k ) at time steps b, c, . . . .
We have:

ru ( ) = E {u(k + )u(k)}
mean{u(k + )u(k )}k=1,...,N
=: b
ru ( ),
0

and b
ru ( ) = b
ru ( ), due to u being a stationary process.
ryu ( ) = E {y(k + )u(k)}
mean{y (k + )u(k )}k=1,...,N
=: b
ryu ( ),
0
Note: to allow negative in b
ryu , the range of k would need to be
1 min{, 0}, . . . , N max{, 0}.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Finite impulse response model

Impose the condition h(k ) = 0 for k M. We obtain the finite impulse

response (FIR) model:
y (k ) =

M1
X

h(j)u(k j) + v (k)

j=0

The covariance relationship is similarly truncated:

ryu ( ) =

M1
X

h(j)ru ( j)

j=0

Note: M must be taken so that MTs  dominant time constants!

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Linear system
Using b
ryu , b
ru estimated from data, write the truncated equations for
= 0, . . . , T 1:
b
ryu (0) =

M1
X

h(j)b
ru (j)

j=0

b
ryu (1) =

M1
X

h(j)b
ru (1 j)

j=0

...
b
ryu (T 1) =

M1
X

h(j)b
ru (T 1 j)

j=0

This is a linear system of T equations in M unknowns

h(0), . . . , h(M 1).

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Linear system (continued)

In matrix form:

b
b
b
ryu (0)
ru (0)
ru (1)
...
ryu (1) b
b
r
(1)
r
(0)
.
..
u

=
..
..

.
.
b
ru (T 1) b
ru (T 2) . . .
ryu (T 1)

b
ru (M 1)
h(0)

b
ru (M 2)
h(1)

..

.
b
ru (T M)
h(M 1)

(keep in mind that b

ru ( ) = b
ru ( )).
Taking T = M gives (if input is informative enough) an exact solution.
Due to disturbances and errors, it is a good idea to take T > M.
Then we can apply the machinery of linear regression (see Part 3) to
solve this problem.

Frequency analysis

Correlation analysis

Table of contents

Frequency analysis

Correlation analysis
Analytical development
A practical algorithm. FIR model
Accuracy guarantee

Correlation analysis: Matlab example

Correlation analysis: Matlab example

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Special case: White noise input

Assumptions
3

The input u(k) is zero-mean white noise.

Then ru ( ) =P
0 whenever 6= 0 (since white noise is uncorrelated),

and ryu ( ) = j=0 h(j)ru ( j) simplifies to:

ryu ( ) = h( )ru (0)
leading to the easy algorithm:
h( ) =

b
ryu ( )
b
ru (0)

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Simplified guarantee

Theorem
In the white-noise case, as the number of data points N grows to
b ) converge to the true values h( ).
infinity, the estimates h(
Remark: This type of property, where the true solution is obtained in
the limit of infinite data, is called consistency.

Frequency analysis

Correlation analysis

Table of contents

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Correlation analysis: Matlab example

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Experimental data
Consider we are given the following, separate, identification and
validation data sets.
plot(id); and plot(val);

Note the identification input is white noise, while the validation input is
not. There are 2000 samples in the identification data.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Applying correlation analysis

fir = cra(id, M, 0); or fir = cra(id, M, 0, plot);
Arguments:
1
2
3

Identification data.
FIR length M, here it is set to 40.
Third argument 0 means no input whitening is performed.

Dealing with non-ideal inputs:

If input is not zero-mean, pass the data through detrend to
remove the means.
If input is not white noise, the third argument should be left to
default (by not specifying it or setting it to an empty matrix).

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Applying correlation analysis (continued)

With the plot switch the covariance functions are shown, as well
as the FIR parameters with a 99% confidence interval.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Results on the identification data

yhat = conv(fir, id.u); yhat = yhat(1:length(id.u);
To simulate the FIR model, a convolution between the FIR
parameters and the input is performed. The simulated output is
longer than needed so we cut it off at the right length.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Validation of the FIR model

yhat = conv(fir, val.u); yhat = yhat(1:length(val.u);

Results OK, not great.

Frequency analysis

Correlation analysis

Correlation analysis: Matlab example

Alternative: impulse function

impulse(id, sd, 5);
This function produces a more complicated model than the one
studied in the lectures. Called as shown, it plots the impulse
response with the 5-sigma confidence level.

Negative-time coefficients correspond to possible feedback in the

data (input depends on output); here the data is open-loop.
Statistically significant coefficients appear to exist after the default
time interval. We therefore manually set a time interval [0, 4].

Frequency analysis

Correlation analysis

impulse function (continued)

impulse(id, sd, 5, [0 4]);

To obtain the actual model:

mod = impulse(id, [0 4]);

Correlation analysis: Matlab example

Frequency analysis

Correlation analysis

Validation of impulse model

compare(mod, val);

The fit is very good.

Correlation analysis: Matlab example