APPLICATION OF ORDER STATISTICS TO TERMINATION OF STOCHASTIC ALGORITHMS

1
APPLICATION OF ORDER STATISTICS TO TERMINATION OF
STOCHASTIC ALGORITHMS

• Vaida Bartkute,
• Leonidas Sakalauskas

2
Outline

• Introduction
• Application of order statistics to optimality
  testing and termination of the algorithm
• Stochastic Approximation algorithms
• Simulated Annealing algorithm
• Experimental results
• Conclusions.

3
Outline

• Introduction
• Application of order statistics to optimality
  testing and termination of the algorithm
• Stochastic Approximation algorithms
• Simulated Annealing algorithm
• Experimental results
• Conclusions.

4
Introduction

• Termination of the algorithm is a topical
  problem in stochastic and heuristic
  optimization.
• We consider the application of order
  statistics to establish the optimality in Markov
  type optimization algorithms.
• We build a method for the estimation of minima
  confidence intervals using order statistics,
  which is implemented for optimality testing and
  termination.

5
Statement of the problem

• The optimization problem is (minimization) as
  follows
• where is a bounded from below
  locally Lipshitz
• function. Denote the generalized gradient of this
  function by
• Let be the sequence constructed by
• stochastic search algorithm, where ?tf(xt), t
  0, 1, . .

6
The Markovian algorithms for optimization

• The Markovian algorithm of random searching
  represents a Markov chain in which the
  distribution of probabilities of a point xt1
  depends on a location of the previous point xt
  and value of function ?tf(xt) in it, that
• Examples Stochastic Approximation
• Simulated Annealing
• Random Search (Rastrigin method) and
  etc.

7
Order statistics and target values for optimality
testing and termination

• Beginning of the problem
• Mockus (1968)
• Theoretical background
• Zilinskas, Zhigljavsky (1991)
• Application to maximum location
• Chen (1996)
• Time-to-target-solution value
• Aiex, Resende, Ribeiro, (2002),
• Pardalos (2005).

8
Method for optimality testing by order statistics

• We build a method for estimation of minimum M
  of
• the objective function using values of the
  function provided in
• optimization
• Let only k1 order statistics from the sample H
  to be chosen
• ,
• where .

9
Let apply linear estimators for estimation of
the minimum where . We examine a
simple set (Hall (1982))
Let apply linear estimators for estimation of
the minimum where . We examine a
simple set (Hall (1982))

• Let apply linear estimators for estimation of
  the minimum
• where .
• We examine a simple set (Hall (1982))

10
The one side confidence interval for the minimum
value of the objective function is

• where , where ? is a confidence level.

- the parameter of extreme values
distribution n dimension ?- the parameter of
homogeneity of the function f(x) (Zilinskas
Zhigliavsky (1991)).
11
Stochastic Approximation

• The smoothing is the standard way for the
  nondifferentiable
• optimization. We consider a function smoothed by
  Lipshitz
• perturbation operator
• where is the value of the
  perturbation parameter,
• is a random vector distributed with density p(.).
  
• If density p(.) is locally Lipshitz then
  functions smoothed by this operator are
  twice continuously differentiable (Rubinstein
  Shapiro (1993), Bartkute Sakalauskas (2004)).

12
where stochastic gradient,
is a scalar multiplier.
This scheme is the same for different
Stochastic Approximation algorithms whose
distinguish only by approach to stochastic
gradient estimation. The minimizing sequence
converges a.s. to solution of the optimization
problem under conditions typical for SA
algorithms (Ermolyev (1976), Mikhalevitch et at
(1987), Spall (1992), Bartkute Sakalauskas
(2004)).
13
- smoothing parameter.
14
Rate of Convergence

• Let consider that the function f(x) has a
  sharp
• minimum in the point , in which the
  algorithm converges
• when
• Then
• where Agt0, Hgt0, Kgt0 are certain constants,
  is minimum point of the smoothed function
  (Sakalauskas, Bartkute (2007)).

15
Experimental results

• Unimodal testing functions
• (SPSAL, SPSAU, FDSA)

• Generated funkcions with sharp minimum-
• CB3-
• Rozen Suzuki-

Multiextremal testing functions (Simulated
Annealing (SA))

• Branin-
• Beale-
• Rastrigin-

16
The samples of T500 test functions were
generated, when and minimized by SPSA with
Lipshitz perturbation.

• The coefficients of the optimizing sequence were
  chosen according to convergence conditions
  (Bartkute Sakalauskas (2006))

17
Testing hypothesis about Pareto distribution

• If order statistics follows from Weibull
  distribution, then
• distributed with respect to Pareto distribution
  (ilinskas, Zhigljavsky (1991))
• Thus, statistical hypothesis tested

.
H0
18
Testing hypothesis about Pareto distribution

• The hypothesis tested by criteria ?2 (
  ) for various stochastic algorithms (critical
  value 0,46)

19
One side confidence interval
,
?0.95
20
Confidence bounds of the minimum
21
Confidence bounds of the hitting probability
22
Termination criterion of the algorithms

• To stop the algorithm when minima confidence
  interval
• becomes less admissible value ?

23
Number of iterations after the termination of
the algorithm
24
Simulated Annealing Algorithm

• I. Choose temperature updating function
  neighborhood size
• function solution generation density
  function
• and initial solution x0 (Yang
  (2000)).

II. Construct the optimizing sequence
25
Experimental results

• Let consider results of optimality testing with
  Beale testing function
• F(x,y) (1.5-xxy)2 (2.25-xxy2)2
  (2.625-xxy3)2,
• where search domain -4.5 x,y 4.5.
• It is known that this function has few local
  minima and
• global minimum is 0 at the point (3 0.5).

26
Confidence bounds of the minimum
27
Confidence bounds of the hitting probability
28
Number of iterations after the termination of
the algorithm
29
Conclusions

• Linear estimator for minimum has been proposed
  using theory of order statistics, which was
  studied by experimental way
• Developed procedures are simple and depend only
  on the parameter of extreme values distribution
  ?
• Parameter ? is easily estimated using a
  homogeneity of the objective function or by
  statistical way
• Theoretical considerations and computer examples
  have shown that we can estimate the confidence
  interval of a function extremum with an
  admissible accuracy, when the number of
  iterations increased
• Termination rule using the minimum confidence
  interval was proposed and implemented to
  Stochastic Approximation and Simulated Annealing.