Value of Information for Complex Economic Models - PowerPoint PPT Presentation

1 / 29
About This Presentation
Title:

Value of Information for Complex Economic Models

Description:

Title: PowerPoint Presentation Author: Jeremy Last modified by: Jeremy Created Date: 10/12/2002 12:13:35 PM Document presentation format: On-screen Show – PowerPoint PPT presentation

Number of Views:47
Avg rating:3.0/5.0
Slides: 30
Provided by: Jeremy202
Category:

less

Transcript and Presenter's Notes

Title: Value of Information for Complex Economic Models


1
Value of Information for Complex Economic Models
  • Jeremy Oakley
  • Department of Probability and Statistics,
  • University of Sheffield.
  • Paper available from
  • www.sheffield.ac.uk/chebs/papers.html

2
Outline
  1. Motivation
  2. Expected value of perfect information (EVPI)
  3. Emulators and Gaussian processes
  4. Illustration GERD model

3
1) Introduction
  • An economic model is to be used to predict the
    cost-effectiveness of a particular treatment(s).
  • The economic model will require the specification
    of various input parameters. Values of some or
    all of these are uncertain.
  • This implies the output of the model, the
    cost-effectiveness of the treatment is also
    uncertain.

4
Introduction
  • We wish to identify which input parameters are
    the most influential in driving this output
    uncertainty.
  • Should we learn more about these parameters
    before making a decision?

5
Introduction
  • A measure of importance for an input variable
    have been proposed, based on the expected value
    of perfect information (EVPI) (Felli and Hazen,
    1998, Claxton 1999).
  • Computing the values of these measures is
    conventionally done using Monte Carlo techniques.
    These invariably require a very large numbers of
    runs of the economic model.

6
Introduction
  • For computationally expensive models, this can be
    completely impractical.
  • We present an efficient alternative to Monte
    Carlo, in terms of the number of model runs
    required.

7
2) EVPI
  • We work with net benefit the monetary value or
    utility of a treatment is
  • K x efficacy cost
  • with K the monetary value of a unit increase
    in efficacy.
  • The net benefit of any treatment option will be a
    function of the parameters in the economic model.

8
EVPI
  • Denote the net benefit of treatment option t
    given model parameters X to be
  • NB (t , X )
  • Given X, the economic model returns NB
    (t , X ) for each t .
  • The true values of the model parameters X are
    uncertain.

9
EVPI
  • The baseline decision is to choose t with the
    largest expected net benefit
  • NB maxt EX NB (t , X )
  • The decision maker will have utility NB if they
    choose the best treatment now with no additional
    information.

10
EVPI
  • Now suppose the decision-maker chooses to learn
    the value of all the uncertain input variables X
    before choosing a treatment.
  • They would then choose the treatment with the
    highest net benefit conditional on X, i.e., they
    would consider
  • maxt NB (t , X )

11
EVPI
  • Before actually observing X, they will expect to
    achieve a net benefit of
  • EX maxt NB (t , X )
  • The expected value of this course of action is
    the expected gain in net benefit over the
    baseline decision
  • EX maxt NB (t , X ) NB.
  • This is the (global) EVPI.

12
Partial EVPI
  • Now suppose the decision-maker chooses to learn
    the value of a single uncertain input variable Y
    , an element of X before making a decision.
  • They would then choose the treatment with the
    highest net benefit conditional on Y , i.e., they
    would consider
  • maxt EX Y NB (t , X )

13
Partial EVPI
  • The expected value of learning Y before Y is
    actually observed is then
  • EY maxt EX Y NB (t , X ) NB
  • This is the partial expected value of perfect
    information (partial EVPI) for Y .
  • The partial EVPI is zero if the decision-maker
    would choose the same treatment for any
    (plausible) value of Y .

14
Computing partial EVPIs
  • We need to evaluate
  • EY maxt EX Y NB (t , X )
  • for each element Y in X.
  • The outer expectation EY is a one-dimensional
    integral, and can be evaluated using numerical
    integration.
  • The term maxt EX Y is the maximum of (several)
    higher-dimensional integrals. This requires a
    large Monte Carlo sample to be evaluated.

15
Patient Simulation Models
  • Computing partial EVPIs for computationally cheap
    models, while not trivial, is relatively
    straightforward.
  • However, for one class of models, patient
    simulation models, a sensitivity analysis using
    Monte Carlo methods will be out of reach for the
    model user.

16
Patient Simulation Models
  • An example is given in Kanis et al (2002) for
    modelling osteoporosis
  • For an osteoporosis patient, a bone fracture
    significantly increases the risk of a subsequent
    fracture.
  • Residential status of a patient needs to be
    tracked, in order that costs are not
    double-counted.

17
Patient Simulation Models
  • Progress is to be modelled over a 10 year period.
    Including the approptiate features in the model
    necessitates a patient simulation approach.
  • The net benefit for a given set of input
    parameters is obtained by sampling events for a
    large number of patients.
  • The model takes over an hour for a single run at
    one set of input parameters.

18
Patient Simulation Models
  • For a model with 20 uncertain input variables,
    computing the partial EVPI reliably using Monte
    Carlo for each input variable would require a
    possible minimum of 500,000 model runs.
  • At one hour for each run, this would take 57
    years!
  • Something more efficient is needed

19
3) Emulators
  • For each treatment option t, and given values for
    the input parameters X x, the economic model
    returns NB (t , x )
  • We think of the model as a collection of
    functions
  • NB (t , x ) ft (x)
  • Partial EVPIs can be computed more efficiently by
    exploiting the smoothness of each ft (x)

20
Emulators
  • We can compute partial EVPIs more efficiently
    through the use of an emulator.
  • An emulator is a statistical model of the
    original economic model which can then be used as
    a fast approximation to the model itself.
  • An approach used by Sacks et al (1989) for
    dealing with computationally expensive computer
    models.

21
Gaussian processes
  • Any regression technique can be used. We employ a
    nonparametric regression technique based on
    Gaussian processes (OHagan, 1978).
  • The gaussian process model for the function
    ft (x) is non-parametric the only assumption
    made about ft (x) is that it is a continuous
    function.

22
Gaussian processes
  • In the Gaussian process model, ft (x) is thought
    of as an unknown function, and uncertainty about
    ft (x) is described by a normal distribution.
  • Correlation between ft (x1) and ft (x2) is
    modelled parametrically as a function of
    x1-x2

23
Gaussian processes
  • The partial EVPI for input variable Y is given by
  • EY maxt EX Y NB (t , X ) NB
  • We need to evaluate EX Y NB (t , X ) for each
    t at various values of Y.
  • Denote G (X Y) to be the distribution of X given
    Y. Then
  • EX Y NB (t , X ) ? ft (x) dG (x y )

24
Gaussian processes
  • We can use Bayesian quadrature (OHagan, 1993) to
    rapidly speed up the computation
  • Under the Gaussian process model for ft (x),
  • ? ft (x) dG (x y )
  • has a normal distribution, and can be
    evaluated (almost) instantaneously.
  • This reduces the number of model runs required
    from 100,000s to 100s.

25
4) Example GERD model
  • The GERD model, presented in OBrien et al (1999)
    predicts the cost-effectiveness of a range of
    treatment strategies for gastroesophageal reflux
    disease.
  • Various uncertain inputs in the model related to
    treatment efficacies, resource uses by patients.
  • Model outputs mean number of weeks free of GERD
    symptoms, and mean cost of treatment for a
    particular strategy.

26
Example GERD model
  • We consider a choice between three treatment
    strategies
  • Acute treatment with proton pump inhibitors
    (PPIs) for 8 weeks, then continuous maintenance
    treatment with PPIs at the same dose.
  • Acute treatment with PPIs for 8 weeks, then
    continuous maintenance treatment with hydrogen
    receptor antagonists (H2RAs).
  • Acute treatment with PPIs for 8 weeks, then
    continuous maintenance treatment with PPIs at the
    a lower dose.

27
Example GERD model
  • There are 23 uncertain input variables.
  • Distributions for uncertain inputs detailed in
    Briggs et al (2002).
  • We estimate the partial EVPI for each input
    variable, based on 600 runs of the GERD model.
  • We assume a value of 250 for each week free of
    GERD symptoms. (It is straightforward to repeat
    our analysis for alternative values).

28
Example GERD model
29
Conclusions.
  • The use of the Gaussian process emulator allows
    partial EVPIs to be computed considerably more
    efficiently.
  • Sensitivity analysis feasible for computationally
    expensive models.
  • Can also be extended to value of sample
    information calculations.
Write a Comment
User Comments (0)
About PowerShow.com