Optimal revision of uncertain estimates in project portfolio selection

1
Optimal revision of uncertain estimates in
project portfolio selection

• Eeva Vilkkumaa, Juuso Liesiö, Ahti Salo
• Department of Mathematics and Systems Analysis,
• Aalto University School of Science and Technology

2
Contents

• Project portfolio selection
• Optimizers curse
• Revised estimates
• Discussion

3
Project portfolio selection

• Select a subset of projects within a budget,
  e.g., k out of n projects with the aim of
  maximizing the sum of the projects values µi,
  i1,...,n
• The values µi are generally unknown, whereby
  decisions about which projects to select are made
  based on estimates Vi about µi.

Estimates
Portfolio selection
Values
t
4
Optimizers curse in portfolio selection

• Assume that the estimates are unbiased
• Portfolio maximization selects, on average,
  overestimated projects ? the value of the
  portfolio is less than expected based on the
  estimation information (optimizers curse cf.
  Smith and Winkler, 2006)
• where is the index set of the selected
  projects.

5
Optimizers curse in portfolio selection

• Choosing 10 projects out of 100
• Values i.i.d with
• Unbiased estimates
• The larger the estimation error variance, the
  harder it is to identify the best projects, and
  the larger the difference between the estimated
  and realized portfolio value

µi N(0,12)
Portfolio value
Vi µi ei, ei N(0,s2)
Standard deviation of estimation error
6
Optimal revision of the estimates

• Estimates do not account for the uncertainties
• Use Bayesian revised estimates instead as a
  basis for project selection
• For instance, with µi N(mi,si2), Vi
  N(µi,ti2)
• The estimate V and the prior information m are
  weighted according to their uncertainty.

where
7
Optimal revision of the estimates

• With revised estimates the optimizers curse is
  eliminated, that is
• where is the index set of the projects
  selected using revised estimates
• Previous example
• Choosing 10 projects out of 100
• True values i.i.d. with
• Unbiased estimates

Portfolio value
µi N(0,12)
Standard deviation of estimation error
Vi µi ei, ei N(0,s2)
8
Revised estimates and portfolio composition

• In the previous example, the projects values
  were identically distributed, and the estimation
  errors had equal variances
• Then, prioritization among the projects remains
  unchanged when the estimates are revised, because
• In general, using revised estimates may result in
  a different project prioritization than estimates

9
Revised estimates and portfolio composition
Same error variances
Different error variances

• Choosing 3 projects out of 8
• True values i.i.d. With µi N(0,12)
• On the left, estimates with equal error variance
  for all projects
• On the right, four projects (dashed) more
  difficult to estimate

Project value
Project value
Vi µi ei, ei N(0,0.52)
Vi µi ei, ei N(0,12)
Estimate
Revised estimate
Estimate
Revised estimate
10
Revised estimates and portfolio composition
Same error variances
Different error variances

• On the left, equal error variances ? estimates
  are shifted towards the common prior mean (zero)
  in the same proportion
• On the right the revised estimates of the
  dashed projects are more drawn towards zero,
  because the estimation information is less
  reliable
• Selection of 3 projects leads to different
  portfolios depending on whether the estimates are
  revised or not

Project value
Project value
Estimate
Revised estimate
Estimate
Revised estimate
11
Revised estimates and portfolio value

• The use of revised estimates yields at least as
  high overall portfolio value as the use of
  initial estimates, i.e.
• Example
• Selection of 10 out of 100 projects with values
  µi N(3,12)
• Population contains two types of projects
• Revised estimates yield higher portfolio value
  for any non-trivial division between projects
  with small and large estimation error variances

Optimal
Portfolio value
Revised estimates
Estimates
1) ei N(0,0.12) - small error variance 2)
ei N(0,12) - large error variance
Share of projects with large error variance
12
Revised estimates and correct choices

• The share of correctly selected projects
  increases with revised estimates in the normally
  distributed case, i.e.,
• where K is the index set of the projects in the
  optimal portfolio
• In the previous example, the difference between
  the two portfolios is statistically significant
  (a0.05), when the share of projects with large
  error variance is between 25-55

Share of correct choices
Share of projects with large error variance
13
Discussion

• Selection based on project prioritization
  resulting from estimates
• The value of the portfolio will, on average, be
  lower than expected
• If there are differences in the projects
  estimation error variances, too many projects
  with large error variance will be selected
• Suggestions for improving the selection process
• Accounting for the uncertainties by using revised
  estimates
• Sorting the projects in terms of estimation error
  variances by, e.g., budget division