CHAPTER 17 OPTIMAL DESIGN FOR EXPERIMENTAL INPUTS - PowerPoint PPT Presentation

Loading...

PPT – CHAPTER 17 OPTIMAL DESIGN FOR EXPERIMENTAL INPUTS PowerPoint presentation | free to download - id: 129b03-YTY1N



Loading


The Adobe Flash plugin is needed to view this content

Get the plugin now

View by Category
About This Presentation
Title:

CHAPTER 17 OPTIMAL DESIGN FOR EXPERIMENTAL INPUTS

Description:

Building approximation to existing large-scale simulation via 'metamodel' ... Metamodels are 'curve fits' that approximate simulation input/output ... – PowerPoint PPT presentation

Number of Views:218
Avg rating:3.0/5.0
Slides: 20
Provided by: Jhu96
Learn more at: http://www.jhuapl.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: CHAPTER 17 OPTIMAL DESIGN FOR EXPERIMENTAL INPUTS


1
CHAPTER 17 OPTIMAL DESIGN FOR EXPERIMENTAL
INPUTS
Slides for Introduction to Stochastic Search and
Optimization (ISSO) by J. C. Spall
  • Organization of chapter in ISSO
  • Background
  • Motivation
  • Finite sample and asymptotic (continuous) designs
  • Precision matrix and D-optimality
  • Linear models
  • Connections to D-optimality
  • Key equivalence theorem
  • Response surface methods
  • Nonlinear models

2
Optimal Design in Simulation
  • Two roles for experimental design in simulation
  • Building approximation to existing large-scale
    simulation via metamodel
  • Building simulation model itself
  • Metamodels are curve fits that approximate
    simulation input/output
  • Usual form is low-order polynomial in the inputs
    linear in parameters ?
  • Linear design theory useful
  • Building simulation model
  • Typically need nonlinear design theory
  • Some terminology distinctions
  • Factors (statistics term) ? Inputs (modeling
    and simulation terms)
  • Levels ? Values
  • Treatments ? Runs

3
Unique Advantages of Design in Simulation
  • Simulation experiments may be considered special
    case of general experiments
  • Some unique benefits occur due to simulation
    structure
  • Can control factors not generally controllable
    (e.g., arrival rates into network)
  • Direct repeatability due to deterministic nature
    of random number generators
  • Variance reduction (CRNs, etc.) may be helpful
  • Not necessary to randomize runs to avoid
    systematic variation due to inherent conditions
  • E.g., randomization in run order and input levels
    in biological experiment to reduce effects of
    change in ambient humidity in laboratory
  • In simulation, systematic effects can be
    eliminated since analyst controls nature

4
Design of Computer Experiments in Statistics
  • There exists significant activity among
    statisticians for experimental design based on
    computer experiments
  • T. J. Santner et al. (2003), The Design and
    Analysis of Computer Experiments, Springer-Verlag
  • J. Sacks et al (1989), Design and Analysis of
    Computer Experiments (with discussion),
    Statistical Science, 409435
  • Etc.
  • Above statistical work differs from experimental
    design with Monte Carlo simulations
  • Above work assumes deterministic function
    evaluations via computer (e.g., solution to
    complicated ODE)
  • One implication of deterministic function
    evaluations no need to replicate experiments for
    given set of inputs
  • Contrasts with Monte Carlo, where replication
    provides variance reduction

5
General Optimal Design Formulation (Simulation or
Non-Simulation)
  • Assume model
  • z h(?,?x) v ,
  • where x is an input we are trying to pick
    optimally
  • Experimental design ? consists of N specific
    input values x ?i and proportions (weights) to
    these input values wi
  • Finite-sample design allocates n ? N available
    measurements exactly asymptotic (continuous)
    design allocates based on n ? ?

6
D-Optimal Criterion
  • Picking optimal design ? requires criterion for
    optimization
  • Most popular criterion is D-optimal measure
  • Let M(?,??) denote the precision matrix for an
    estimate of ? based on a design ?
  • M(?,??) is inverse of covariance matrix for
    estimate
  • and/or
  • M(?,??) is Fisher information matrix for estimate
  • D-optimal solution is

7
Equivalence Theorem
  • Consider linear model
  • Prediction based on parameter estimate and
    future measurement vector hT is
  • Kiefer-Wolfowitz equivalence theorem states
  • D-optimal solution for determining ? to be used
    in forming is the same ? that minimizes the
    maximum variance of predictor
  • Useful in practical determination of optimal ?

8
Variance Function as it Depends on Input Optimal
Asymptotic Design for Example 17.6 in ISSO
9
Orthogonal Designs
  • With linear models, usually more than one
    solution is D-optimal
  • Orthogonality is means of reducing number of
    solutions
  • Orthogonality also introduces desirable secondary
    properties
  • Separates effects of input factors (avoids
    aliasing)
  • Makes estimates for elements of ? uncorrelated
  • Orthogonal designs are not generally D-optimal
    D-optimal designs are not generally
    orthogonal
  • However, some designs are both
  • Classical factorial (cubic) designs are
    orthogonal (and often D-optimal)

10
Example Orthogonal Designs, r 2 Factors
11
Example Orthogonal Designs, r 3 Factors
xk3
12
Response Surface Methodology (RSM)
  • Suppose want to determine inputs x that minimize
    the mean response z of some process (E(z))
  • There are also other (nonoptimization) uses for
    RSM
  • RSM can be used to build local models with the
    aim of finding the optimal x
  • Based on building a sequence of local models as
    one moves through factor (x) space
  • Each response surface is typically a simple
    regression polynomial
  • Experimental design can be used to determine
    input values for building response surfaces

13
Steps of RSM for Optimizing x
  • Step 0 (Initialization) Initial guess at optimal
    value of x.
  • Step 1 (Collect data) Collect responses z from
    several x values in neighborhood of current
    estimate of best x value (can use experimental
    design).
  • Step 2 (Fit model) From the x, z pairs in step 1,
    fit regression model in region around current
    best estimate of optimal x.
  • Step 3 (Identify steepest descent path) Based on
    response surface in step 2, estimate path of
    steepest descent in factor space.
  • Step 4 (Follow steepest descent path) Perform
    series of experiments at x values along path of
    steepest descent until no additional improvement
    in z response is obtained. This x value
    represents new estimate of best vector of factor
    levels.
  • Step 5 (Stop or return) Go to step 1 and repeat
    process until final best factor level is
    obtained.

14
Conceptual Illustration of RSM for Two Variables
in x Shows More Refined Experimental Design Near
Solution
Adapted from Montgomery (2001), Design and
Analysis of Experiments, Fig. 11-3
15
Nonlinear Design
  • Assume model
  • z h(?,?x) v ,
  • where ? enters nonlinearly
  • D-optimality remains dominant measure
  • Maximization of determinant of Fisher information
    matrix (from Chapter 13 of ISSO Fn(?, x) is
    Fisher information matrix based on n data points)
  • Fundamental distinction from linear case is that
    D-optimal criterion depends on ?
  • Leads to conundrum
  • Choosing x to best estimate ?, yet need to know
    ? to determine x

16
Strategies for Coping with Dependence on ?
  • Assume nominal value of ? and develop an optimal
    design based on this fixed value
  • Sequential design strategy based on an iterated
    design and model fitting process.
  • Bayesian strategy where a prior distribution is
    assigned to ?, reflecting uncertainty in the
    knowledge of the true value of ?.

17
Sequential Approach for Parameter Estimation and
Optimal Design
  •   Step 0 (Initialization) Make initial guess at
    ?, Allocate n0 measurements to initial
    design. Set k 0 and n 0.
  • Step 1 (D-optimal maximization) Given Xn , choose
    the nk inputs in X to maximize
  • Step 2 (Update ? estimate) Collect nk
    measurements based on inputs from step 1. Use
    measurements to update from to
  • Step 3 (Stop or return) Stop if the value of ? in
    step 2 is satisfactory. Else return to step 1
    with the new k set to the former k 1 and the
    new n set to the former n nk (updated Xn now
    includes inputs from step 1).

18
Comments on Sequential Design
  • Note two optimization problems being solved one
    for ?, one for ?
  • Determine next nk input values (step 1)
    conditioned on current value of ?
  • Each step analogous to nonlinear design with
    fixed (nominal) value of ?
  • Full sequential mode (nk 1) updates ? based
    on each new input?ouput pair (xk , zk)
  • Can use stochastic approximation to update ?
  • where

19
Bayesian Design Strategy
  • Assume prior distribution (density) for ?, p(?),
    reflecting uncertainty in the knowledge of the
    true value of ?.
  • There exist multiple versions of D-optimal
    criterion
  • One possible D-optimal criterion
  • Above criterion related to Shannon information
  • While log transform makes no difference with
    fixed ?, it does affect integral-based solution.
  • To simplify integral, may be useful to choose
    discrete prior p(?)
About PowerShow.com