GUARANTED SET COMPUTATION Ellipsoidal approach

1
GUARANTED SET COMPUTATION Ellipsoidal approach

• S. Lesecq
• GIPSA-Lab, Département Automatique
• INPG-UJF-CNRS
• Suzanne.lesecq_at_gipsa-lab.inpg.fr

2
Outline

• Motivation
• Notations and definitions
• Outer bounding ellipsoid
• Algorithm(s)
• Parameterised family
• Simple example
• Factorisation

• Convergence conditions
• Common difficulties
• Choice of the bound
• Industrial context input?
• Extra
• Inner bounding
• Uncertainty in the model
• References

3
Motivation - 1

• Discrete-time dynamic system
• Observation
• yk measured output vector
• uk known input vector
• unknown state, process perturbation,
  measurement noise
• Identification (output linear in the parameters)

4
Motivation - 2

• Consider a SISO model
• dk regressor, ? parameter vector
• Error bound ek

5
Motivation - 3

• Possible simpler sets
• Polytopes
• Parallelotopes
• Orthotopes
• Ellipsoids
• Outer bounding ellipsoid
• Mininum size
• Inner bounding ellipsoid
• Maximum size

?2
? Criterion to be optimised
?1
6
Motivation - 4

• Usual criteria
• Determinant criterion (actually log(det))
• Trace criterion

7
Notations definitions - 1

• Norm Euclidian
• Unit ball centred on the origin
• Bounded ellipsoid E(c,P) with non empty interior
• c centre
• P PT gt 0 shape and orientation

8
Notations definitions - 2

• Particular cases
• Strip unbounded ellipsoid
• Empty interior ellipsoid

Centre c not unique, just satisfy y Cc
?2
?1
9
Notations definitions - 3

• Sum of K ellipsoids (prediction)
• Intersection of K ellipsoids (correction)

10
Outline

• Motivation
• Notations and definitions
• Outer bounding ellipsoid
• Algorithm(s)
• Parameterised family
• Simple example
• Factorisation
• Examples Identification

• Convergence conditions
• Common difficulties
• Choice of the bound
• Industrial context input?
• Extra
• Inner bounding
• Uncertainty in the model
• References

11
Outer bounding ellipsoid - 1

• Recursive algorithm (Fogel and Huang, 1982)
• not normalised form (intersection)

P PT gt 0
12
Outer bounding ellipsoid - 2

• OBE algorithm

Choice of ?k, ?k ?
13
Outer bounding ellipsoid - 3

• Basic OBE algorithm not optimal (Belforte and
  Bona, 1985 and 1990)

• Void intersection
• ? must be detected
• ? add intersection test (e.g. Pronzato and
  Walter, 1994)

• Other classical situation

14
Outer bounding ellipsoid - 4

• Usual criteria
• In the litterature, two sets of programs
• (Favier and Arruda, 1996 Tran Dinh, 2005)
• Set 1 minimize the geometrical size of Ek
• Set 2 insure the convergence of ? 2

15
Outer bounding ellipsoid - 5

• Set 1 (size)

16
Outer bounding ellipsoid - 6

• Set 2

17
Outer bounding 7 (parameterised family)

• Normalised problem (Durieu, et al., 2001)
• MIMO models
• State estimation (identification) ? Sum,
  Intersection
• Analytical results K ellipsoids
• Optimisation problem addressed in details

Empty interior
Unbounded (strip)
18
Outer bounding 8 (parameterised family)

• Sum of K ellipsoids (Durieu, et al., 2001)
• Problem find
• Theorem 4.1
• The centre of the optimal ellipsoid E for both
  problems is given by

Empty interior possible
19
Outer bounding 9 (parameterised family)

• Parameterized family ? optimisation can be done
• Theorem 4.2
• c independent of ?
• Solve for ? the problem
• ? generally, suboptimal solution of

20
Outer bounding 10 (parameterised family)

• Necessary condition (Lemma 4.1) for E? to be
  optimal solution of
• Theorem 4.4 Trace criterion ? explicit solution
• Theorem 4.5 recursive ? nonrecursive
  approximating ellipsoid
• Determinant criterion ? no explicit solution

21
Outer bounding 11 (parameterised family)

• Intersection of K ellipsoids (Durieu, et al.,
  2001)
• Problem find

22
Outer bounding 12 (parameterised family)

• Theorem 5.1
• Proposition 5.1

23
Outer bounding ellipsoid - 13

• Ellipsoid with parallel cuts algorithm (Goldfarb
  and Todd, 1982)
• Sequential algorithm
• Equivalent to (modified) OBE (Pronzato and
  Walter, 1994)
• Recursively optimal
• ? i.e. minimal volume ellipsoid containing
  E(ck-1,Pk-1) ? Bk

x2
Hk
Ek
Ek-1
x1
24
Outer bounding ellipsoid - (example)

• Simulated data
• yk measured output vector
• unknown measurement noise
• (uniform distribution)
• SNR ? 50 dB
• Identification

25
Outer bounding ellipsoid - (example)

• Results
• xtheoretic 0.95, 0.05
• c0 0, ?02P 106I

Algo Param min  max c ? ?V
FH a 0.9496 0.9509 0.9502 6.51 10-4 1.85 10-14
FH b 0.0497  0.0502 0.0499 2.76 10-4 1.85 10-14
DH a 0.9094 0.9914 0.9504 4.10 10-2 6.14 10-10
DH b 0.0380 0.0618 0.0499 1.19 10-2 6.14 10-10
TAN a 0.0509 1.3912 0.9504 4.41 10-1 1.04 10-3
TAN B -0.0499 0.1498 0.0499 9.99 10-2 1.04 10-3
LO A -60.8 62.7 0.9495 61.79 1.12 106
LO B -39.9 40. 1 0.0510 40.04 1.12 106
Set 1 determinant
Set 2
26
Outer bounding ellipsoid - (example)

• Evolution of Det (?k2P)
• LO Lozano-Leal and Ortega, 1987
• TAN Tan, et al., 1997
• DH Dasgupta and Huang, 1987
• FH Fogel and Huang, 1982

• Evolution of ?k2

27
Outer bounding ellipsoid - (example)

• Parameter a evolution

Figure 2.4  Evolution des paramètres estimés.
28
Outer bounding ellipsoid - (example)

• Parameter b evolution

29
Outer bounding ellipsoid summary

• Two sets of algorithms (Favier and Arruda, 1996
  Tran Dinh, 2005)
• Minimise the geometrical size of E
• Minimise ?2
• Different formulations of the algorithm
• Equivalence provable for some of them (OBE-EPC)
  (Pronzato and Walter, 1994, Tran Dinh, 2005)
• (Durieu, et al., 1996 and 2001)
• Parameterised family
• K ellipsoids
• Convexity of criteria

30
Outline

• Motivation
• Notations and definitions
• Outer bounding ellipsoid
• Algorithm(s)
• Parameterised family
• Simple example
• Factorisation
• Examples Identification

• Convergence conditions
• Common difficulties
• Choice of the bound
• Industrial context input?
• Extra
• Inner bounding
• Uncertainty in the model
• References

31
Factorisation - 1

• Prerequisite
• Orthogonal matrix HT H-1
• Numerically highly suitable
• Orthogonal factorisation
• Factorisation of a product of matrices

32
Factorisation - 2

• Factorisation of a sum of matrices
• Least Square problem solution
• Recursive Least Square problem also factorised

33
Factorisation - 3

• Parameterised family (Durieu, et al., 1996)
• hypothesis (not restrictive) y? R
• Sum of 2 symmetric matrices ? factorise!

Intersection

• Goldfarb and Todd, 1982
• LDL factorisation
• Cholesky suggested

34
Factorisation - 4

• Reformulation ? optimisation problem (Lesecq and
  Barraud, 2002)
• Let then

35
Factorisation - 5

• Factorised algorithm (Lesecq and Barraud, 2002)
• Theoretical property ? ? 0, 1 simpler
  demonstration

36
Factorisation - 6

• Directly with P (general formulation) (Tran Dinh,
  2005)

ck, Mk dependent
37
Factorisation - summary

• Absolutely necessary to ensure numerical
  stability
• Academic example (Lesecq and Barraud, 2002)
• n 8, M hilb(8)hij 1/(ij-1), c ones
  (8,1), y 1
• temp invhilb(9), d temp(18,9)
• ? 0.001 ? ?not factorised - 47.6 and
  ?factirised 1.7 10-2
• Practical problem (identification)
• Theoretical properties easier demonstration
• Parameterised family (Durieu, et al., 1996) ? P
  and M algorithms (Lesecq and Barraud, 2002)
• General formulation ? P and M algorithms (Tran
  Dinh, 2005)

38
Outline

• Motivation
• Notations and definitions
• Outer bounding ellipsoid
• Algorithm(s)
• Parameterised family
• Simple example
• Factorisation
• Examples Identification

• Convergence conditions
• Common difficulties
• Choice of the bound
• Industrial context input?
• Extra
• Inner bounding
• Uncertainty in the model
• References

39
Examples - 1

• Industrial Data
• 1st example industrial
• Looks like 1st order, 2 parameters
• aim model identification ? diagnosis
• 2nd example LIRMM robot
• 14 parameters
• 14 000 regressors!
• aim model identification, large problem

40
Examples - 2

• 1st example recorded on a process (valve)

output
input
41
Examples - 3

• 1st example
• Data re-used several times
• Determinant criterion
• ? 0.002
• Measurement
• and regressor
• known

Determinant criterion 1st circulation of data
42
Examples - 4
Ellipsoid updating

• 1st example parameters

Circulations param. min  max centre ? determinant R()
1 a 0.7967 1.0205 0.9086 0.1119 4.85 10-7 13.7
1 b -0.0133 0.0637 0.0252 0.0385 4.85 10-7 13.7
10 a 0.8479 0.9462 0.8971 0.0491 1.08 10-7 2.4
10 b 0.0145 0.0441 0.0293 0.0148 1.08 10-7 2.4
43
Examples - 5

• 1st example

det(10)
det(1)
Gain properly identified
Trace(10)
Trace(1)
44
Examples - 7

• 2nd example LIRMM parallel robot

N 3500
45
Examples 8

• 2nd example Recorded data (for instance)

Sequential algorithm
46
Examples - 9
criterion
No empty intersection
? 6 Nm

• 2nd example parameters (60 circulations)

Parameter min  max c ? ?() a priori
Imot1 -0.0251  0.1158 0.0454 0.0704 155.2 0.012
Imot2 0.0418  0.0580 0.0499 0.0081 16.28 0.012
Imot3 -0.0504 0.0998 0.0247 0.0752 304.1 0.012
Imot4 -0.0469 0.1035 0.0283 0.0753 266. 2 0.012
Mnac 0.1395 0.2893 0.2144 0.0749 34.92 1.0
Inac -0.03613 0.03524 -0.00045 0.0357 799.2 0.0008
F?1 -2.9720 3.7908 0.40939 3.38 825.9 X
F?2 -0.2689 1.2673 0.4992 0.768 153.9 X
F?3 -2.8062 2.9045 0.0491 2.86 581.0 X
F?4 -2.6674 2.7023 0.0174 2.68 1539.1 X
Fs1 -8.0750 9.8545 0.8898 8.96 1007.5 X
Fs2 -1.2478 1.4495 0.1008 1.35 1337.5 X
Fs3 -16.212 16.868 0.3282 16.5 5039.5 X
Fs4 -14.437 15.390 0.47671 14.9 3128.4 X
47
Factorisation interest
Parallel robot, det, family of ellipsoids
No factorisation
Thanks to N. Ramdani Work done by N. Ramdani P.
Poignet LIRMM
Factorisation
48
Examples - 10

• 2nd example Model reconsidered
• ? split in several models

? 6 Nm
1st model
Parameter min  max c ? ?() a priori
Imot1 0.0063 0.0805 0.0434 0.037 85.4 0.012
Mnac -0.1299 0.6809 0.2755 0.405 147. 2 1.0
Inac -0.1994 0.2864 0.0435 0.243 558.4 0.0008
F?1 -1.6799 2.3432 0.3316 2.01 606.6 X
Fs1 -2.9494 5.1822 1.1164 4.07 364. 2 X
49
Examples - 11
Center, obtained with ? 6 Nm and 60 circulations

• 2nd example Adaptation of the bound

Far from hypothesis!

• Analysis of data
• Reject outliers
• heavy tail

50
Examples - 12
60 circulations

• 2nd example Adaptation of the bound ? heavy
  tail

Param. ? 6 Nm ? 6 Nm ? 6 Nm heavy heavy heavy
Param. centre min  max ? centre min  max ?
Imot1 4.54 10-2 -0.025  0.116 7.04 10-2 1.66 10-2 0.014  0.019 2.40 10-3
Imot2 4.99 10-2 0.042 0.058 8.12 10-3 1.64 10-2 0.0117 0.021 4.62 10-3
Imot3 2.49 10-2 -0.050 0.099 7.52 10-2 1.75 10-2 0.011 0.024 6.11 10-3
Imot4 2.84 10-2 -0.047 0.104 7.53 10-2 2.32 10-2 0.019 0.027 3.52 10-3
Mnac 2.14 10-1 0.139 0.289 7.49 10-2 9.87 10-1 0.923 1.051 6.42 10-2
Inac -6.48 10-4 -0.036 0.035 3.57 10-2 2.94 10-3 0.002  0.004 1.19 10-3
F?1 4.15 10-1 -2.972 3.791 3.38 2.13 10-1 0.145  0.281 6.78 10-2
F?2 4.99 10-1 -0.269 1.267 7.68 10-1 1.23 10-1 0.067  0.179 5.64 10-2
F?3 5.37 10-2 -2.806 2.905 2.86 1.29 10-1 0.059  0.199 7.04 10-2
F?4 1.61 10-2 -2.667 2.702 2.68 1.15 10-1 0.011  0.219 1.04 10-1
Fs1 8.81 10-1 -8.075 9.855 8.96 1.22 1.059  1.374 1.58 10-1
Fs2 1.01 10-1 -1.248 1.449 1.35 1.02 0.807  1.239 2.16 10-1
Fs3 2.38 10-1 -16.21 16.87 1.65 101 7.68 10-1 0.378  1.159 3.90 10-1
Fs4 4.82 10-1 -14.44 15.39 1.49 101 1.03 0.840  1.219 1.89 10-1
det 3.1312 10-9 3.1312 10-9 3.1312 10-9 1.5719 10-45 1.5719 10-45 1.5719 10-45
51
Examples - 13

• 2nd example reorganise data

Angle between 2 regressors
52
Examples - 14
criterion
?(Para.) 1st circulation 1st circulation 60th circulation 60th circulation
?(Para.) Initial Reorganised Initial Reorganised
?(Imot1) 1.6423 1.5004 10-1 7.0448 10-2 6.9419 10-3
?(Imot2) 5.3308 10-1 1.5788 10-1 8.1202 10-3 7.9782 10-3
?(Imot3) 2.1952 10-1 9.4689 10-2 7.5151 10-2 7.4153 10-2
?(Imot4) 9.6392 10-2 8.9899 10-2 7.5253 10-2 7.5824 10-2
?(Mnac) 3.5286 3.1971 7.4868 10-2 7.3429 10-2
?(Inac) 4.0685 10-2 5.7488 10-2 3.5688 10-2 3.6735 10-2
?(F?1) 7.2910 101 5.5816 3.3814 3.3284
?(F?2) 2.7041 101 5.3418 7.6816 10-1 7.5179 10-1
?(F?3) 7.1152 3.2917 2.8553 2.8145
?(F?4) 2.6920 2.9740 2.6849 2.6892
?(Fs1) 1.9207 102 2.0027 101 8.9647 8.8031
?(Fs2) 8.3393 101 1.8595 101 1.3487 1.3250
?(Fs3) 2.6252 101 1.7720 101 1.6540 101 1.6520 101
?(Fs4) 1.1166 101 1.5965 101 1.4913e 101 1.4925 101
det 7.0667 1014 6.2676 103 3.1312 10-9 2.5372 10-9
53
Criterion
T17
det
Initial data
Outliers removed
T23
Nb of Circulations
54
Summary

• Ellipsoidal approach
• Large size problem
• (14 parameters,
• 14 000 regressors)
• Data analysis
• Several strips together
• (optimisation)

• Algorithm
• Factorisation
• Uncertain model
• Industrial/real data
• Assumptions? Convergence condition?
• Known inputs?
• Choice of the bound?
• Data analysis
• Choice of the input?
• Excitation
• Analysis of results
• Physical meaning

Bound?
55
References

• Bai E. W., Tempo R., Cho H. (1995), Membership
  Set Estimators Size, Optimal Inputs, Complexity
  and Relations with Least Squares, IEEE
  Transactions on Circuits and Systems I
  Fundamental Theory and Application, Vol. 42(5),
  pp. 266-277.
• Bai E. W., Huang Y. F. (1999), Convergent of
  optimal sequential outer bounding sets in bounded
  error parameter estimation, Mathematics and
  Computers in Simulation, Vol. 49, pp. 307-317.
• Belforte G. Bona B. (1985), An improved parameter
  identification algorithm for signals with
  unknown-but- bounded error, 7th IFAC/IFORS Symp.
  on identification and system parameter
  estimation, pp. 1507-1512.
• Belforte G. Bona B. (1990), Parameter estimation
  algorithms for a set-membership description of
  uncertainty, Automatica, Vol. 26, pp. 887-898.
• Durieu C., Polyak B., Walter E. (1996), Trace
  versus determinant in ellipsoidal outer bounding,
  with application to state estimation, 13th IFAC
  World Congress, San Francisco, USA, Vol. I, pp.
  9-12.
• Durieu C., Walter E., Polyak B. (2001),
  Multi-input multi output ellipsoidal state
  bounding, Journal of Optim. Theory and
  Application, Vol. 111(2), pp. 273-303.
• Fogel E., Huang Y. (1982), On the Value of
  Information in System Identification Bounded
  noise case, Automatica, Vol. 18, pp. 229-238.
• Nayeri M., Liu M. S, Deller J. R (1993), An
  interpretable and converging set-membership
  algorithm, Proceeding IEEE of International
  Conference on Acoustics, Speech and Signal
  Processing, Vol IV, Minneapolis, New York, pp.
  472-475.
• Pronzato L., Walter E. (1994), Minimal volume
  ellipsoids, International Journal of Adaptive
  Control and Signal Processing, Vol. 8, , pp.
  15-30.
• Schweppe C. F. (1973), Uncertain dynamic systems,
  Englewood Cliffs, NJ, Prentice-Hall.

56
References

• Dasgupta S., Huang Y. F. (1987), Asymptotically
  Convergent Modified Recursive Least-square with
  Data-dependent Updating and Forgetting Factor for
  Systems with Bounded Noise, IEEE Trans. on
  Information Theory, Vol. IT 33(3), pp. 383-393.
• Lozano-Leal R., Ortega R. (1987), Reformulation
  of the Parameter Identification Problem for
  Systems with Bounded Disturbances, Automatica,
  Vol. 23(2), pp. 247-251.
• Favier G., Arruda L.V.R. (1996), Review and
  Comparison of Ellipsoidal Algorithms, In Bounding
  approaches to System Identification, edited by M.
  Milanese et al., Plenium Press, New York, pp.
  43-68.
• Tran Dinh Q.K. (2005), Contribution à
  lestimation ensembliste ellipsoidale, PhD
  thesis, INPG, Grenoble, France (in French).
• Goldfarb D., Told M.J. (1982), Modification and
  implementation of the ellipsoid algorithm for
  linear programming, Mathematical Programming,
  Vol. 23, pp. 1-19.
• Lesecq S., Barraud A. (2002), Une approche
  factorisée plus simple et numérique-ment stable
  pour l'estimation ensembliste, JESA, Vol. 36(4),
  pp. 505-518.
• Veres S.M., Norton J.P. (1991), Structure
  selection for bounded-parameter models
  consistency conditions and selection criterion,
  IEEE Transactions on Automatic Control, Vol.
  36(4), pp. 474-481.
• Sedda E. (1998), Estimation en ligne de létat et
  des paramètres dune machine asynchrone par
  filtrage à erreur borne et par filtrage de
  Kalman, PhD thesis, ENS Cachan (in French).

57
References

• Lin T.M., Nayeri M., Deller J.R. (1998), A
  consistently convergent OBE algorithm with
  automatic estimation of error bounds,
  International Journal of Adaptive Control and
  Signal Processing, Vol. 12, pp. 305-324.
• Polyak B., Nazin S., Durieu C., Walter E. (2004)
  Ellipsoidal parameter or state estimation under
  model uncertainty, Automatica, Vol. 40, pp.
  1171-1179.
• Clement T., Gentil S. (1990), Recursive
  membership set estimation for output error
  models, Mathematics and Computers in Simulation,
  Vol. 32, pp. 505-513.
• Ferreres G., MSaad M. (1997), Estimation of
  output error models in the presence of unknown
  but bounded disturbances, International Journal
  of Adaptive Control and Signal Processing, Vol.
  11, pp. 115-140.