From theory to empirical evidence and back: integrating datadriven research into your portfolio

1
From theory to empirical evidence and back
integrating data-driven research into your
portfolio

• Serguei Netessine

2
Signs of distress

• Enrollments in OM courses are shrinking (OR/MS
  Today).
• There is an on-going initiative to market the
  profession (INFORMS).
• The gap between salaries of professors in OM and
  Finance/Accounting is widening.
• Academics in other areas (Finance, Physics,
  Medicine) have created big ideas that impacted
  the world while OM only followed practice.
• OM/OR traditionally focused on low-level,
  tactical problems that are hard to sell to top
  management.
• We had a good run with Supply Chain Management
  but the interest in fading away.
• What is the next big thing?
• Not a different area but a different methodology!

3
Some observations

• Mathematical ideas originate in empiricsbut
  then the subject begins to live a peculiar life
  of its own and after much abstract
  inbreeding, is in danger of degeneration
  whenever this stage is reached, the only remedy
  seems to me to be the reinjection of more or
  less directly empirical ideas
• John von Neumann
• Economics, Finance, Accounting and Marketing all
  started up as theoretical disciplines, then
  enjoyed an explosion of empirical studies and
  came to a balance between empirics and modeling.
• How much of our (modeling) research can we
  utilize in teaching?
• How much of our (modeling) research interests top
  managers?
• Empirical research can make us more relevant as
  researchers, teachers, consultants as well as
  stimulate better models!

4
Some myths about empirical research

• It is hard to get a job with empirical work.
• Counterexamples
• Kamalini Ramdas (Darden)
• Taylor Randall (Chicago offer)
• Vishal Gaur (NYU)
• Vinayak Deshpande (Purdue)
• Frances Frei (Rochester, Harvard)
• Sharon Novak (Kellogg)
• Nicole De Horatius (Chicago)
• Christian Terwiesch (Wharton)
• Anita Tucker (Wharton)
• It is hard to get tenure with empirical research.
• Counterexamples
• Raman, Ramdas, Terwiesch, (Deshpande, Gaur)

5
More myths about empirical research

• OM data is hard to get.
• Have you tried?
• 136 universities subscribe to WRDS (Wharton
  Research Data Services) which contains COMPUSTAT,
  CRISP and other information.
• Government agencies provide free data (e.g.,
  CENSUS, BEA). DOT provides detailed info on the
  aviation industry that economists used for years
  yet there are no empirical papers on revenue
  management!!!
• Consulting companies have tons of data ready to
  be shared for free because they dont know how to
  analyze it.
• Even more sources are available at a nominal fee
  that most academic advisors can cover.
• Many industry journals publish industry-specific
  data which can be accessed at a price of a
  regular subscription (e.g., Internet Retailer,
  Automotive News).

6
More myths about empirical research

• Empirical papers are hard to publish in top
  journals.
• Just one persons counterexample
• Terwiesch, Christian, Justin Z. Ren, Teck H. Ho,
  Cohen, Morris, An Empirical Analysis of Forecast
  Sharing in the Semiconductor Equipment Industry,
  forthcoming in Management Science.
• Terwiesch, Christian, Il-Horn Hann, Sergei Savin,
  Online Haggling in a Name-Your-Own-Price
  Channel Theory and Application, forthcoming in
  Management Science. 
• Cohen, Morris, Teck H. Ho, Justin Z. Ren,
  Christian Terwiesch, Measuring Imputed Costs in
  the Semiconductor Equipment Supply Chain,
  Management Science, Vol. 49, No. 12, 2003, (pp.
  1653-1670).  
• Hann, Il-Horn, Christian Terwiesch, Measuring
  the Frictional Costs of Online Transactions The
  Case of a Name-Your-Own-Price Channel,
  Management Science, Vol. 49, No. 11, 2003, (pp.
  1563-1579). 
• Terwiesch, Christian, Arnoud De Meyer, Christoph
  H. Loch, Exchanging Preliminary Information in
  Concurrent Engineering Alternative Coordination
  Strategies, Organization Science, Vol. 13,
  Number 4, 2002, (pp. 402-419) a 2-page summary
  of the initial working paper appeared in Harvard
  Business Review.
• Terwiesch, Christian, Christoph H. Loch,
  Measuring the Effectiveness of Overlapping
  Development Activities, Management Science, Vol.
  45, Number 4, 1999 (pp. 455-465).
• See also Marshall Fisher, Morris Cohen, Karl
  Ulrich, Taylor Randall, Kamalini Ramdas

7
Some observations/advices

• It is hard to get job/tenure with empirical
  research at some schools.
• Write both modeling and empirical papers (a
  de-facto standard at Wharton).
• The specific data you might want is often hard to
  get so
• You have to be opportunistic.
• You have to be inventive with proxies.
• You should leverage one data collection effort in
  several papers.
• Certain types of empirical research take long
  time to finish, require extensive resources,
  experience, and are harder to publish in top
  journals (Case studies, Surveys, Field studies)
• It is prudent to begin by focusing on econometric
  work and use secondary data sources.
• Keep the modeling community in mind and address
  its needs.
• Work at a high level (whole economy/industry).
• Jump on the bandwagon now before all interesting
  research is done!

8
Story 1 Internet retailing or how I got into
empirical research

• Supply Chain Structures on the Internet and the
  role of marketing-operations interaction. With
  Nils Rudi. 2004, in "Handbook of Quantitative
  Supply Chain Analysis Modeling in the E-Business
  Era", D. Simchi-Levi, S. D. Wu and M. Shen, Eds,
  Kluwer. (job talk)
• Supply Chain choice on the Internet. With Nils
  Rudi. Rev. Aug 2005. (extension of the job talk).
• An empirical examination of the decision to
  invest in fulfillment capabilities a study of
  Internet retailers. With Taylor Randall, Nils
  Rudi. Rev. August 2005.
• Should you take the virtual fulfillment path?
  With Taylor Randall, Nils Rudi. 2002, Supply
  Chain Management Review, November-December,
  54-58.

9
Analytical model and empirical predictions
Drop-shipping
Traditional
c
c
Q
A hybrid
?
??
q
q
q
q
r
r
Single-period newsvendor model. Exogenous prices.
10
Factors Influencing Inventory Choice
Own
Drop-Ship
Firm Age Firm Size Ratio of retailers to
wholesalers Product Variety Demand
Uncertainty Product Weight/Size Obsolescence
Risk Gross Margin
Older Large Low Low Low Low Low High
Younger Small High High High Higher Higher
Low
11
Sample Description

• Small survey of 64 publicly held e-tailers
• 56 responses, 54 usable responses (84.4)
• Between 60 and 70 of US-wide e-tailing
  revenue.
• Financial data from COMPUSTAT data base
• Example Companies
• Amazon.com Pets.com
• BarnesNoble.com Egghead.com
• CDNow.com Delias.com
• Fogdog.com Autobytel.com
• Webvan.com Buy.com
• 36 companies choose to hold inventory (67)
• 11 bankrupt companies (20)

12
Inventory Model Logistic Regression Results
Variable Expected Sign Coefficient Es
timate Constant ? 17.47 Age
?1 0.05 Firm Sales
?2 -0.02 Retailers/wholesalers
- ?3 -0.91 Product Variety
- ?4 -0.01 Demand Uncertainty
- ?5 -62.51 Size and Weight
- ?6 -1.38 Obsolescence Risk
- ?7 -0.11 Industry
Margins ?8 14.97
13
Bankruptcy Model Cox proportional hazards model
MISFIT is a measure of deviation from the
theoretical model.
Variable Coefficient All Firms
Surviving Firms MISFIT2 ?1
2.50 2.83 Entry order ?2
0.05 0.07 Market Share
?3 -23.12 -18.28 Altmans Z Score ?4
0.07 0.12
Companies that do not adhere to the theoretical
model go bankrupt more often. So we know we have
a good model!
14
Summary

• An example of combining data from public sources
  with a survey.
• Insights from simple inventory models apply on
  the Internet as well.
• Matching supply with demand affects financial
  performance (bankruptcy).
• Research results
• theoretically obtained criteria for Supply
  Chain choice,
• confirmed hypothesis empirically,
• linked Supply Chain choice and bankruptcy.
• Details http//www.netessine.com.
• Most of the data is publicly available
  (COMPUSTAT, 10K filings, Internet news sites). A
  minimal survey was conducted over the phone look
  for Media Contacts.
• A flood of emails from e-commerce executives,
  newswires, speeches, advising, consulting,
  litigation witnessing etc.
• These slides are a part of OPIM 632 Supply Chain
  Management class at Wharton.

15
Closing the loop feeding empirical results into
models

• Most original Internet retailers are start-ups
  with a high risk of bankruptcy which drives their
  investment decisions.
• Is it appropriate to describe their behavior
  using expected profit maximization criterion or
  even the risk-averse utility function?
• How should decisions of startups differ from
  decisions of established firms that launched
  Internet retailing? (Read The Innovators
  Dilemma by Clayton Christensen).
• Who should do better in a new market a startup
  or an established firm?
• The competitive start-up capacity, pricing, and
  uncertain demand. With G. Cachon and R. Swinney.
  Rev. Sep 2005.
• Empirical study inventory management by
  start-ups

16
Story 2 Investment into manufacturing
flexibility

• Strategic technology choice and capacity
  investment under demand uncertainty. With Manu
  Goyal. Rev. Sep 2005.
• Deployment of manufacturing flexibility an
  empirical analysis of the North American
  automotive industry. With M. Goyal and T.
  Randall. Rev. Sep 2005.
• Capacity investment and the interplay between
  volume-flexibility and product-flexibility. With
  M. Goyal. Rev. May 2005.

PhD dissertation
17
The Model
18
Empirical predictions
Product-flexible plant
Nonflexible plant
Demand Uncertainty Demand Correlation Number of
flexible competitors Mean Demand Demand
Difference Product Substitutability
Uncertainty Product Substitutability Demand
Difference
High Low High Low Low High Low
Low High Low High High Low High
19
Data

• Harbour Associates consulting company that
  publishes annually The Harbour Report which
  rates productivity of North American automotive
  plants.
• Detailed plant-level data for 100 automotive
  plants in the USA.
• Made a trip to Michigan to meet Ron Harbour.
• Paid a nominal cost for reports.
• Used slave labor to code the data.
• WardsAuto.com (demand/supply data).
• AutomotiveNews.com.
• A visit to GMs Wilmington Delaware plant.
• 20 interviews of automotive industry experts.
• Aside check International Motor Vehicle Program
  at MIT http//imvp.mit.edu/ for data, research
  (mostly survey-based) and funding.

20
Logistic regression results (N367 plant/years).
Dependent variable 1 if more than one platform
per assembly line, 0 otherwise.
21
Observations and the feedback loop

• An example of leveraging data collected by a
  consulting company.
• Automotive manufacturers deploy flexibility to
  respond to competition, demand uncertainty and
  demand correlation.
• First empirical evidence to analyze the link
  between product flexibility and correlation,
  product flexibility and competition.
• More flexible plants are lower utilized which is
  contrary to the idea behind manufacturing
  flexibility (see Jordan and Graves 1995). Why?
• A hypothesis product flexibility is intertwined
  with volume flexibility. Product-flexible plants
  can profitably operate under lower utilization.
• There are no extant models that analyze more than
  one flexibility type at a time.
• Capacity investment and the interplay between
  volume-flexibility and product-flexibility. With
  M. Goyal. Rev. May 2005.
• More information http//www.netessine.com

22
Story 3 Testing predictions from the classical
inventory models

• Numerous classical inventory models (newsvendor,
  periodic review, multi-echelon, EOQ etc.)
• What can be learned from classical inventory
  models cross-industry empirical examination.
  With S. Roumiantsev. Rev. April 2005.
• Some new operational drivers of financial
  performance. With S. Roumiantsev. In final
  stages.
• Find out predictive power of classical inventory
  models.
• Gain credibility with top management.
• Find out if managing inventories right
  contributes to financial performance.
• Define directions for modifications of these
  models.

23
What can top managers learn from inventory models?

• High-level managers are typically concerned with
  the aggregate inventory behavior at the firm
  level.
• Inventory models do not account for competition,
  business cycles, industry trends, companies
  financial distress, seasonality, etc.
• Can high-level managers and industry analysts
  benefit from understanding classical inventory
  models? this is an empirical question!
• Hypotheses
• Relative inventory level is negatively associated
  with company size.
• Inventory level is positively associated with
  demand uncertainty.
• Inventory level is positively associated with
  lead times.
• Inventory level is positively associated with
  product margins.
• Inventory level is negatively associated with
  inventory holding costs.

24
Data

• Quarterly data from COMPUSTAT containing 44 time
  points between 1992 and 2002 for every company in
  our sample.
• A sample of 722 public companies including 233
  SP500 companies with 8 segments represented oil
  and gas, wholesale, retail, consumer electronics,
  food, chemicals, hardware, and machinery.
• Non-widely diversified companies accounting for
  30 of US inventory.
• Average company holds 396M in inventory, 2108M
  in annual COGS.

25
Results inventory models are helpful at any
level!
Economies of scale
Cost of underage
Lead times
Demand uncertainty
Cost of overage
Explanatory power
Note denote statistical significance at the
1 level.
26
Insights and further research

• An example of leveraging publicly available data
  sources.
• Teaching simple inventory models to MBAs is
  justified insights from these models can help
  evaluate companies aggregate inventories.
• Next step do companies that behave according to
  predictions from classical inventory models
  achieve better financial performance?
• Some new operational drivers of financial
  performance. With S. Roumiantsev. In final
  stages.
• The ability to adjust inventories quickly to
  changes in the environmental variables increases
  ROA.
• Also read David Berman case, HBS, Ananth Raman
  and Vishal Gaur.
• The feedback loop
• we need to spend more time analyzing
  non-stationary models,
• we need to endogenize the speed of adjusting
  inventory levels,
• more research is needed to link finance and
  inventories this is where all the money is.

27
Story 4 More on linking finance and operations.
The price of a supply chain glitch

• Kevin Hendricks and Vinod R. Singhal Supply
  Chain Disruptions and Shareholder Value
• 800 announcements of supply chain disruptions
  (production or shipment delays) from Wall Street
  Journal and Dow Jones News
• Sun Microsystems delays shipments of workstations
  and servers, Dow Jones News Service, December,
  14, 2000.
• Sony Sees Shortage of Playstation 2s for Holiday
  Season, The Wall Street Journal, September 28,
  2000.
• Boeing pushing for record production, finds parts
  shortages, delivery delays, Wall Street Journal,
  June 26, 1997.
• Hershey will miss earnings estimate by as much as
  10 because of problems in delivering order, Wall
  Street Journal, September 14, 1999.
• Compare performance of disruption experiencing
  firms with portfolios of similar type of firms
  (Size, Book to Market value, Prior performance).
  Data from COMPUSTAT.

28
Average stock returns over different intervals
29
Comparison with stock market reaction to other
corporate events (on announcement)
Marketing events Change in firm
name 0.7 Brand leveraging
0.3 Celebrity endorsement 0.2 New
product introduction 0.7 Affirmative
action awards 1.6
Operational events Increase in capital
expenditure 1.0 Increase in RD
expenditure 1.4 Effective TQM
implementation 0.7 Internal corporate
restructuring 1.0 Decrease in capital
expenditure -1.8 Plant closing
-0.7 Delay introduction new products
-5.3 Supply Chain Glitch -7.2
Financial events Stock splits
3.3 Open market share repurchase 3.5 Proxy
contest 4.2 Increasing
financial leverage 7.6 Decreasing financial
leverage -5.4 Seasoned equity offerings
-3.0
Information technology events IT
Investments 1.0 IT
problems -1.7
30
Story 5 Is there a bullwhip effect?

• Dozens (if not hundreds) of papers study the
  bullwhip effect and propose solutions to
  eliminate it. The Beer Game. Barilla Case.
• An often forgotten effect countering the bullwhip
  effect is production smoothing which has been
  studied by economists.
• Which one dominates an empirical question!
• Cachon, Randall and Schmidt. In search of the
  bullwhip effect.
• Data Sources
• Census Department, Bureau of Economic Analysis.
• Data
• U.S., 1992-2004, monthly.
• Amplification ratio VarProduction /
  VarDemand (1 for Bullwhip, smoothing).

31
General merchandise stores margin and price
adjusted
32
Telecom margin and price adjusted plus logged
and first differenced
33
Amplification ratios across industries
seasonally unadjusted

• Almost all retail industries and the majority of
  manufacturing industries smooth production.
• The bullwhip effect is observed among most
  wholesalers (need better models?)
• There is little evidence that demand variability
  increases as one moves up the supply chain from
  retailer to manufacturer.

34
Integration with teaching OPIM632 SCM
35
Empirical Research in MSOM and Management Science
36
Conclusion

• If we want to be relevant as a discipline, we
  must strike a balance between the empirical data
  and models. This is the way to differentiate
  ourselves from OR, IE and MS.
• Empirical research appeals to much wide
  audiences, leads to more consulting, better
  teaching and media exposure.
• Every other area of business is doing it so there
  is little hope that we can get away with just
  models.
• Now there is a critical mass of people on various
  editorial boards that can judge empirical
  research.
• Take econometrics courses while you still can.
• Read Freakonomics by Levitt and some other guy,
  The machine that changed the world by Womack et
  al., a book by Clark and Fujimoto.
• Read trade publications, look for unique data
  sources.
• Talk to your advisor about working with data.
• Attend empirical sessions at INFORMS (I am
  co-chairing two with Gerard Cachon, Vinayak
  Deshpande chairs another one) to at least see
  what is happening.