# CHAPTER 17 OPTIMAL DESIGN FOR EXPERIMENTAL INPUTS - PowerPoint PPT Presentation

PPT – CHAPTER 17 OPTIMAL DESIGN FOR EXPERIMENTAL INPUTS PowerPoint presentation | free to download - id: 129b03-YTY1N

The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
Title:

## CHAPTER 17 OPTIMAL DESIGN FOR EXPERIMENTAL INPUTS

Description:

### Building approximation to existing large-scale simulation via 'metamodel' ... Metamodels are 'curve fits' that approximate simulation input/output ... – PowerPoint PPT presentation

Number of Views:218
Avg rating:3.0/5.0
Slides: 20
Provided by: Jhu96
Category:
Tags:
Transcript and Presenter's Notes

Title: CHAPTER 17 OPTIMAL DESIGN FOR EXPERIMENTAL INPUTS

1
CHAPTER 17 OPTIMAL DESIGN FOR EXPERIMENTAL
INPUTS
Slides for Introduction to Stochastic Search and
Optimization (ISSO) by J. C. Spall
• Organization of chapter in ISSO
• Background
• Motivation
• Finite sample and asymptotic (continuous) designs
• Precision matrix and D-optimality
• Linear models
• Connections to D-optimality
• Key equivalence theorem
• Response surface methods
• Nonlinear models

2
• Building approximation to existing large-scale
simulation via metamodel
• Building simulation model itself
• Metamodels are curve fits that approximate
simulation input/output
• Usual form is low-order polynomial in the inputs
linear in parameters ?
• Linear design theory useful
• Building simulation model
• Typically need nonlinear design theory
• Some terminology distinctions
• Factors (statistics term) ? Inputs (modeling
and simulation terms)
• Levels ? Values
• Treatments ? Runs

3
• Simulation experiments may be considered special
case of general experiments
• Some unique benefits occur due to simulation
structure
• Can control factors not generally controllable
(e.g., arrival rates into network)
• Direct repeatability due to deterministic nature
of random number generators
• Variance reduction (CRNs, etc.) may be helpful
• Not necessary to randomize runs to avoid
systematic variation due to inherent conditions
• E.g., randomization in run order and input levels
in biological experiment to reduce effects of
change in ambient humidity in laboratory
• In simulation, systematic effects can be
eliminated since analyst controls nature

4
Design of Computer Experiments in Statistics
• There exists significant activity among
statisticians for experimental design based on
computer experiments
• T. J. Santner et al. (2003), The Design and
Analysis of Computer Experiments, Springer-Verlag
• J. Sacks et al (1989), Design and Analysis of
Computer Experiments (with discussion),
Statistical Science, 409435
• Etc.
• Above statistical work differs from experimental
design with Monte Carlo simulations
• Above work assumes deterministic function
evaluations via computer (e.g., solution to
complicated ODE)
• One implication of deterministic function
evaluations no need to replicate experiments for
given set of inputs
• Contrasts with Monte Carlo, where replication
provides variance reduction

5
General Optimal Design Formulation (Simulation or
Non-Simulation)
• Assume model
• z h(?,?x) v ,
• where x is an input we are trying to pick
optimally
• Experimental design ? consists of N specific
input values x ?i and proportions (weights) to
these input values wi
• Finite-sample design allocates n ? N available
measurements exactly asymptotic (continuous)
design allocates based on n ? ?

6
D-Optimal Criterion
• Picking optimal design ? requires criterion for
optimization
• Most popular criterion is D-optimal measure
• Let M(?,??) denote the precision matrix for an
estimate of ? based on a design ?
• M(?,??) is inverse of covariance matrix for
estimate
• and/or
• M(?,??) is Fisher information matrix for estimate
• D-optimal solution is

7
Equivalence Theorem
• Consider linear model
• Prediction based on parameter estimate and
future measurement vector hT is
• Kiefer-Wolfowitz equivalence theorem states
• D-optimal solution for determining ? to be used
in forming is the same ? that minimizes the
maximum variance of predictor
• Useful in practical determination of optimal ?

8
Variance Function as it Depends on Input Optimal
Asymptotic Design for Example 17.6 in ISSO
9
Orthogonal Designs
• With linear models, usually more than one
solution is D-optimal
• Orthogonality is means of reducing number of
solutions
• Orthogonality also introduces desirable secondary
properties
• Separates effects of input factors (avoids
aliasing)
• Makes estimates for elements of ? uncorrelated
• Orthogonal designs are not generally D-optimal
D-optimal designs are not generally
orthogonal
• However, some designs are both
• Classical factorial (cubic) designs are
orthogonal (and often D-optimal)

10
Example Orthogonal Designs, r 2 Factors
11
Example Orthogonal Designs, r 3 Factors
xk3
12
Response Surface Methodology (RSM)
• Suppose want to determine inputs x that minimize
the mean response z of some process (E(z))
• There are also other (nonoptimization) uses for
RSM
• RSM can be used to build local models with the
aim of finding the optimal x
• Based on building a sequence of local models as
one moves through factor (x) space
• Each response surface is typically a simple
regression polynomial
• Experimental design can be used to determine
input values for building response surfaces

13
Steps of RSM for Optimizing x
• Step 0 (Initialization) Initial guess at optimal
value of x.
• Step 1 (Collect data) Collect responses z from
several x values in neighborhood of current
estimate of best x value (can use experimental
design).
• Step 2 (Fit model) From the x, z pairs in step 1,
fit regression model in region around current
best estimate of optimal x.
• Step 3 (Identify steepest descent path) Based on
response surface in step 2, estimate path of
steepest descent in factor space.
• Step 4 (Follow steepest descent path) Perform
series of experiments at x values along path of
steepest descent until no additional improvement
in z response is obtained. This x value
represents new estimate of best vector of factor
levels.
• Step 5 (Stop or return) Go to step 1 and repeat
process until final best factor level is
obtained.

14
Conceptual Illustration of RSM for Two Variables
in x Shows More Refined Experimental Design Near
Solution
Adapted from Montgomery (2001), Design and
Analysis of Experiments, Fig. 11-3
15
Nonlinear Design
• Assume model
• z h(?,?x) v ,
• where ? enters nonlinearly
• D-optimality remains dominant measure
• Maximization of determinant of Fisher information
matrix (from Chapter 13 of ISSO Fn(?, x) is
Fisher information matrix based on n data points)
• Fundamental distinction from linear case is that
D-optimal criterion depends on ?
• Choosing x to best estimate ?, yet need to know
? to determine x

16
Strategies for Coping with Dependence on ?
• Assume nominal value of ? and develop an optimal
design based on this fixed value
• Sequential design strategy based on an iterated
design and model fitting process.
• Bayesian strategy where a prior distribution is
assigned to ?, reflecting uncertainty in the
knowledge of the true value of ?.

17
Sequential Approach for Parameter Estimation and
Optimal Design
•   Step 0 (Initialization) Make initial guess at
?, Allocate n0 measurements to initial
design. Set k 0 and n 0.
• Step 1 (D-optimal maximization) Given Xn , choose
the nk inputs in X to maximize
• Step 2 (Update ? estimate) Collect nk
measurements based on inputs from step 1. Use
measurements to update from to
• Step 3 (Stop or return) Stop if the value of ? in
with the new k set to the former k 1 and the
new n set to the former n nk (updated Xn now
includes inputs from step 1).

18
• Note two optimization problems being solved one
for ?, one for ?
• Determine next nk input values (step 1)
conditioned on current value of ?
• Each step analogous to nonlinear design with
fixed (nominal) value of ?
• Full sequential mode (nk 1) updates ? based
on each new input?ouput pair (xk , zk)
• Can use stochastic approximation to update ?
• where

19
Bayesian Design Strategy
• Assume prior distribution (density) for ?, p(?),
reflecting uncertainty in the knowledge of the
true value of ?.
• There exist multiple versions of D-optimal
criterion
• One possible D-optimal criterion
• Above criterion related to Shannon information
• While log transform makes no difference with
fixed ?, it does affect integral-based solution.
• To simplify integral, may be useful to choose
discrete prior p(?)