SYSTEMS Identification

1
SYSTEMSIdentification

• Ali Karimpour
• Assistant Professor
• Ferdowsi University of Mashhad

Reference System Identification Theory For The
User Lennart Ljung(1999)
2
Lecture 4
Models of Linear Systems

• Topics to be covered include
• Frequency Response Function (FRF).
• Data in Time and Frequency Domain.
• Linear Model Structures (Without Noise).
• Linear Model Structures (With Noise).
• Fitting Parameterized Linear Models to Data.
• Time Domain Data
• Frequency Domain Data
• The Asymptotic Properties of the Estimate.
• Model Sets, Model Structures and Identifiability.

3
Identification of Linear Systems

• Topics to be covered include
• Frequency Response Function (FRF).
• Data in Time and Frequency Domain.
• Linear Model Structures (Without Noise).
• Linear Model Structures (With Noise).
• Fitting Parameterized Linear Models to Data.
• Time Domain Data
• Frequency Domain Data
• The Asymptotic Properties of the Estimate.
• Model Sets, Model Structures and Identifiability.

4
Frequency Response Function (FRF)

• A linear system is characterized by its transfer
  function G(s)

This leads to FRF G(j?)
This could be a way of determining G. All
frequency at the same time
5
Identification of Linear Systems

• Topics to be covered include
• Frequency Response Function (FRF).
• Data in Time and Frequency Domain.
• Linear Model Structures (Without Noise).
• Linear Model Structures (With Noise).
• Fitting Parameterized Linear Models to Data.
• Time Domain Data
• Frequency Domain Data
• The Asymptotic Properties of the Estimate.
• Model Sets, Model Structures and Identifiability.

6
Data in Time and Frequency Domain

• Spectral Analysis.

7
Data in Time and Frequency Domain

• Spectral Analysis.

8
Data in Time and Frequency Domain

• Spectral Analysis.

Exercise 1 Derive the figure by an experimental
data.
9
Identification of Linear Systems

• Topics to be covered include
• Frequency Response Function (FRF).
• Data in Time and Frequency Domain.
• Linear Model Structures (Without Noise).
• Linear Model Structures (With Noise).
• Fitting Parameterized Linear Models to Data.
• Time Domain Data
• Frequency Domain Data
• The Asymptotic Properties of the Estimate.
• Model Sets, Model Structures and Identifiability.

10
Linear Model Structures (Without Noise)
A complete model is given by
A particular model thus corresponds to
specification of the function G.
We try to parameterize coefficients so
Where ? is a vector in Rd space.
We thus have
11
Linear Model Structures (Without Noise)
Finite Impulse Response Model
If we suppose that the relation between input and
input can be written as
With
So
FIR model
12
Linear Model Structures (Without Noise)
Output error model structure
If we suppose that the relation between input and
undisturbed output w can be written as
With
So
OE model
13
Linear Model Structures (Without Noise)
For most physical systems it is easier to
construct models with physical insight in
continuous time
State Space model
We can derive the transfer operator from u to y
Another important model that is used in process
system is
Static gain, time constant, delay
Process model
14
Linear Model Structures (With Noise)
For most physical systems it is easier to
construct models with physical insight in
continuous time
? is a vector of parameters that typically
correspond to unknown values of physical
coefficients, material constants, and the like.
Let ?(t) be the measurements that would be
obtained with ideal, noise free sensors
We can derive the transfer operator from u to ?
14
15
Linear Model Structures (With Noise)
Sampling the transfer function
Let
Then x(kTt) is
So x(kTT) is
We can derive the transfer operator from u to ?
15
16
Identification of Linear Systems

• Topics to be covered include
• Frequency Response Function (FRF).
• Data in Time and Frequency Domain.
• Linear Model Structures (Without Noise).
• Linear Model Structures (With Noise).
• Fitting Parameterized Linear Models to Data.
• Time Domain Data
• Frequency Domain Data
• The Asymptotic Properties of the Estimate.
• Model Sets, Model Structures and Identifiability.
• .

17
Linear Model Structures (With Noise)
A complete model is given by
Same as
A particular model thus corresponds to
specification of the function G, H.
We try to parameterize coefficients so
Where ? is a vector in Rd space.
We thus have
18
Linear Model Structures (With Noise)
Equation error model structure
Adjustable parameters in this case are
Define
ARX model
So we have
where
19
Linear Model Structures (With Noise)
ARMAX model structure
with
So we have
now
where
Let
20
Linear Model Structures (With Noise)
Other equation error type model structures
ARARX model
With
We could use an ARMA description for error
ARARMAX model
21
Linear Model Structures (With Noise)
Output error model structure
If we suppose that the relation between input and
undisturbed output w can be written as
Then
With
So
OE model
22
Linear Model Structures (With Noise)
Box-Jenkins model structure
A natural development of the output error model
is to further model the properties of the output
error. Let output error with ARMA model then
BJ model
This is Box and Jenkins model (1970)
23
Linear Model Structures (With Noise)
A general family of model structure
The structure we have discussed in this section
may give rise to 32 different model sets,
depending on which of the five polynomials A, B,
C, D, F are used.
For convenience, we shall therefore use a
generalized model structure
General model structure
24
Linear Model Structures (With Noise)
Sometimes the dynamics from u to y contains a
delay of nk samples, so
So
But for simplicity
24
25
Linear Model Structures (With Noise)
The structure we have discussed in this section
may give rise to 32 different model sets,
depending on which of the five polynomials A, B,
C, D, F are used.
General model structure
B
FIR (finite impulse response)
AB
ARX
ABC
ARMAX
AC
ARMA
ABD
ARARX
ABCD
ARARMAX
BF
OE (output error)
BFCD
BJ (Box-Jenkins)
26
State Space models
Noise Representation and the time-invariant
Kalman filter
A straightforward but entirly valid approach
would be
with e(t) being white noise with variance ?.
Note The ?-parameter in H(q, ?) could be partly
in common with those in G(q, ?) or be extra.
w(t) and v(t) are assumed to be sequences of
independent random variables with zero mean and
26
27
State Space models
Noise Representation and the time-invariant
Kalman filter
w(t) and v(t) may often be signals whose
physical origins are known.
The load variation Tl(t) was a process noise.
The inaccuracy in the potentiometer angular
sensor is the measurement noise.
In such cases it may of course not always be
realistic to assume that the signals are white
noises.
27
28
State Space models
Exercise 2 (4G.2) Colored measurement noise
28
29
State Space models
29
30
State Space models
The conditional expectation of x(t) is
The predictor filter can thus be written as
30
31
State Space models
Innovation representation
InnovationAmounts of y(t) that cannot be
predicted from past data
Let it e(t)
The innovation form of state space description
31
32
State Space models
Innovation representation
The innovation form of state space description
Let suppose
Directly Parameterized Innovations form
Which one involve with lower parameters?
Both according to situation.
32
33
State Space models
Innovation representation
It is ARMAX model
33
34
State Space models
Example 4.2 Companion form parameterization
34
35
Identification of Linear Systems

• Topics to be covered include
• Frequency Response Function (FRF).
• Data in Time and Frequency Domain.
• Linear Model Structures (Without Noise).
• Linear Model Structures (With Noise).
• Fitting Parameterized Linear Models to Data.
• Time Domain Data
• Frequency Domain Data
• The Asymptotic Properties of the Estimate.
• Model Sets, Model Structures and Identifiability.

36
Fitting Models to Data
Curve fitting
Time Domain
Frequency Domain
36
37
Fitting Models to Data
Pre filtering A Further Degree of Freedom
The fit is focused to the frequency ranges where
L is large.
38
Linear Regression
Model structures such as
That are linear in ? are known in statistics as
linear regressions.
Now define
39
First order difference equation
Consider the simple model
Then
Now we have
40
First order difference equation
Consider the simple model
41
Linear Regression for ARX model
Equation error model structure
So we have
ARX model
Now if we introduce
Linear regression
41
42
Linear Regression for ARX model
Linear regression in statistic
43
Pseudo Linear Regression for ARMAX model
ARMAX model structure
Now
Let
Now if we introduce
Pseudo linear regressions
43
44
Pseudo Linear Regression for ARMAX model
Let
w(t) is never observed instead it is constructed
from u
So
45
Pseudo Linear Regression for general model
structure
A pseudolinear form for general model structure
Predictor error is
46
Pseudo Linear Regression for general model
structure
So we have
47
Pseudo Linear Regression for general model
structure
48
Identification of Linear Systems

• Topics to be covered include
• Frequency Response Function (FRF).
• Data in Time and Frequency Domain.
• Linear Model Structures (Without Noise).
• Linear Model Structures (With Noise).
• Fitting Parameterized Linear Models to Data.
• Time Domain Data
• Frequency Domain Data
• The Asymptotic Properties of the Estimate.
• Model Sets, Model Structures and Identifiability.

48
49
The Asymptotic Properties of the Estimate

• Assume that data have been generated by

• Properties of the estimate ?N

• A parameter free assessment of model quality
  for linear systems and models

• How to affect the model quality?

49
50
Identification of Linear Systems

• Topics to be covered include
• Frequency Response Function (FRF).
• Data in Time and Frequency Domain.
• Linear Model Structures (Without Noise).
• Linear Model Structures (With Noise).
• Fitting Parameterized Linear Models to Data.
• Time Domain Data
• Frequency Domain Data
• The Asymptotic Properties of the Estimate.
• Model Sets, Model Structures and Identifiability.

50
51
General Notation
Some notation
It is convenient to introduce some more compact
notation
One step ahead predictor is
52
General Notation
Definition 4.1. A predictor model of a linear,
time-invariant system is a stable filter W(q).
Definition 4.2. A complete probabilistic model of
a linear, time-invariant system is a pair
(W(q),fe(x)) of a predictor model W(q) and the
PDF fe(x) of the associated errors.
Clearly, we can also have models where the PDFs
are only partially specified (e.g., by the
variance of e)
We shall say that two models W1(q) and W2(q) are
equal if
53
General Notation
Example Unstable system.
If A(q) has zeros outside the unit disc, then the
map from u ? y is unstable.
But
Is always stable.
54
Model Sets
Definition A model set is a collection of models
Examples
55
Parameterization of Model Sets
Let a model be index by a parameter ?, W(q, ?)
We require W(q, ?) to be differentiable with
respect to ?
So
56
Model Structure
Definition A model structure m is a
differentiable mapping from a connected subset Dm
of Rd to a model set m , such that the
gradients of the predictor functions are stable.
57
Parameterization of Model Sets
Example An ARX model.
58
Model Structure
Definition A model structure m is a
differentiable mapping from a connected subset Dm
of Rd to a model set m , such that the
gradients of the predictor functions are stable.
Differentiability of T ( G and H )
Differentiability of W
59
Model Structure
Definition A model structure m is a
differentiable mapping from a connected subset Dm
of Rd to a model set m , such that the
gradients of the predictor functions are stable.
Lemma The predictor
For ? confined to Dm? F(q)C(q) has no zeros
outside the unit disc is a model structure.
Proof We need only verify that the gradients of
60
Model Structure
Definition A model structure m is a
differentiable mapping from a connected subset Dm
of Rd to a model set m , such that the
gradients of the predictor functions are stable.
Lemma For the Kalman filter predictor
Assume that the entries of the matrices A(?),
B(?), C(?) and K(?) are differentiable with Dm?
all eigenvalues of A(?)-K(?)C(?) are inside
the unit circle
Then the parameterization is a model structure.
Exercise 7 Proof the Lemma. (4D.1)
61
Independent Parameterization
Definition A model structure m is said to have
an independently parameterized transfer function
and noise model if
We can define a model set as the range of a model
structure
We can define union of different model structures
62
Identifiability
Identifiability properties
The problem is whether the identification
procedure will yield a unique value of the
parameter ?, and/or whether the resulting model
is equal to the true system.
Definition 4.6. A model structure M is globally
identifiable at ? if
63
Identifiability
Definition A model structure M is globally
identifiable at ? if
This definition is quite demanding. A weaker and
more realistic property is
For corresponding local property, the most
natural definition of local identifiability of M
at ? would be to require that there exist an e
such that
64
Identifiability
Use of the Identifiability concept
The identifiability concept concerns the unique
representation of a given system description in a
model structure. Let such a description as
Let M be a model structure based on
one-step-ahead predictors for
Then define the set DT(S,M) as those ?-values in
DM for which SM (?)
Very important property
65
Identifiability
SISO Transfer function models
Consider the model structure
Idenifiability of ARX Model?
Idenifiability of OE Model?
Idenifiability of other Models?
66
Identifiability
ARX Model
So ARX Model is strictly identifiable
67
Identifiability
OE Model
So OE Model is not generally strictly identifiable
68
Identifiability
Theorem Consider the general model structure
Exercise 8 (4E.6)