Methods of researching things that haven

1
Methods of researching things that havent
happened especially good things and bad things

• Michael Wood
• Portsmouth Business School
• michael.wood_at_port.ac.uk
• http//userweb.port.ac.uk/woodm/NotHappened.ppt
• Slightly revised after comments from PBS Research
  Conference
• 8 June 2010

2
Abstract

• My aim in this talk is twofold. First, I will
  argue that most management research gives too
  much status to facts and observations, focusing
  on what has happened in the past on transcripts
  of people saying what theyve done or what they
  think, or on subtle statistical distinctions
  between one aspect of past history and another.
  Such data may be important, but the danger is it
  encourages us to ignore the potentially much more
  interesting task of exploring possibilities
  what could be or what should be and predicting,
  or shaping, or making recommendations about, the
  future. Second, I will tentatively suggest
  some approaches for doing such research these
  include methods involving experimentation,
  mathematics, modeling and simulation, works of
  fiction (including thought experiments), analysis
  of common sense data and hypothetical surveys.
  (The presentation will be on the web at
  http//userweb.port.ac.uk/woodm/NotHappened.ppt
  .)

3
Where am I coming from?

• Vague feeling that I really dont care about
  endless empirical data, but Im more interested
  in new, exciting possibilities.
• What might be instead of what is or was.
• (Perhaps I just dont like the real world?)
• Im often asked how big should a sample be ...
  Expected answer is 30 or 50, so its rather fun
  to say 0!
• I suspect everything I want to say is entirely
  obvious, except possibly to those who know a
  little about research methods.
• Not about OR and economics, which already have a
  tradition of being flexible with the facts
• Not final word in the spirit of anarchistic
  epistemology (Feyerabend, 1993 and Law, 2004)

4
I will

• Discuss what I mean and why its important
• Suggest some approaches for researching important
  things that havent happened
• Consider one or two case studies, with a class
  exercise

5
What hasnt happened?

• In one sense everything in the future hasnt
  happened yet researching this is very important
  ... and obviously difficult!
• But if we can assume future is similar to past,
  then we can study past happenings to try to learn
  about the future. But ...
• May be difficult to find past examples
• Talebs black swans and the fate of the turkey
  (Taleb, 2008)
• If future happenings are not likely to be similar
  to past happenings then we have a problem!

6
Things that havent happened that are worth
studying include

• Things that may happen in the future that arent
  obviously similar to things that have happened in
  the past (and about which we can find get data)
• Especially
• Good things, that we want to recommend or help to
  happen
• Bad things, that we would like to avoid (e.g.
  typical risk analysis)
• (Also counterfactuals to understand the past ?)

7
Typical MBA project research into how to improve
X

• 1 Investigate the current problems with X.
• 2 Investigate how other organisations deal with X
• 3 Taking account of the findings from (1) and
  (2), and the literature and any ideas within the
  organisation, and any other sources of
  inspiration, produce some recommendations for
  improving X
• 4 Test the proposals to see how they work

8
The research bit of this project is normally seen
as 1 and 2 only

• The rhetoric of research methods (books, courses,
  etc) focuses on Steps 1 and 2 because these
  concern facts or observations about what has
  happened in the past methods typically viewed
  as qualitative or quantitative
• Management tries to be scientific which
  supposedly means that we must focus on the
  facts about what has happened in the past. Hence
  the obsession with data, questionnaires
  (dismissed as autoerotic fantasies by Salancik,
  1979), interview transcripts, etc.

9
But

• This ignores Steps 3 and 4 in which we want to
  formulate some recommendations for the future and
  test them
• This is surely the main problem with management
  research how to investigate things which
  havent yet happened answer in a minute, but
  first

10
Taken-for-granted assumptions of research
methods

• Empiricism everything must be based on facts
  ie on surveys, interviews, observations
• Methodism approved methods only must be used
• These assumptions seem to outlaw research on
  things that havent happened like Steps 3 and 4

11
Facts and empiricism in physics

• Physics often seen as the exemplar for science,
  and Emc2 is probably the most famous result in
  physics. However
• unlike results in management, this is assumed to
  hold everywhere and for all time
• it summarises an infinite collection of possible
  events, most which havent happened. E.g. my
  conversion to energy (about 20,000 million
  million KwH)
• it was derived from thought experiments and only
  indirectly from observations, and certainly not
  on the basis of a statistical survey of matter in
  the universe.

12
Methods of researching things that havent
happened in management

• Not easy for obvious reasons
• May be design or invention, rather than discovery
  (e.g. Ackoffs idealized design)
• What follows is just some ideas. Not
  comprehensive or fully worked out

13
Methods might be based on ...

• Experimentation ranging from informal to
  randomized controlled trials (see Ayres (2007)
  for examples)
• Mathematics (which is a way of exploring
  possibilities without trying them eg stock
  control formulae)
• the subject in which we never know what we are
  talking about, nor whether what we are saying is
  true (Russell, 1901)
• Modelling and simulation (another way of
  exploring possibilities without trying them)
  e.g. Simulating reactions to disasters, or sales
  situations
• Scenario planning work out details of
  possibilities
• Works of fiction (including thought experiments,
  hypothetical examples, fables, utopias,
  dystopias)
• Analysis of common sense data (if simple called
  philosophy, if more complicated called maths)
• Surveys asking what would you do if ...?

14
Any others methods of researching things that
havent happened ?
15
Case study 1 Alternative vote (AV) vs first past
the post (FPP)

• This case is fictional Im making it up rather
  than looking at a real project, but does this
  matter?

16
Method of researching what has happened

• Interview key stakeholders to get their opinions
  about FPP and AV
• Survey voters views about FPP and AV
• Look at other countries
• Typical management research stops here!
• But if people havent experienced AV will they
  have a useful opinion? This does not really give
  adequate information about the new system ...

17
Methods of researching what hasnt happened

• Mathematical analysis of properties of different
  systems (Method Type 2 eg the GibbardSatterthwai
  te theorem)
• Simulate different voting patterns and compare AV
  with FPP (Method Type 3)
• Ask a sample of people how they would have voted
  under AV (get them to vote) and compare results
  with actual results (Method Type 7)

18
Case study 2 Problems and opportunities of
blackberries for WLB

• Actual MBA project

19
Methods of researching what has happened

• Interviews with 12 blackberry users
• But not much information on possible serious
  problems, or useful innovations. Small snapshot
  of typical present uses.
• Perhaps research should aim to go past this ...
• What would you do?

20
Methods of researching what hasnt happened

• Experimentation ask people to not use a
  blackberry, use it in a particular way etc (Type
  1)
• Fiction imagine extreme scenarios involving
  blackberries and WLB (Type 5)
• Ask people what they would do if ... to explore
  things that havent happened to them (Type 7)
• Mathematical modelling of WLB and blackberries
  ...? (Type 2)
• These allow research to explore future
  possibilities

21
Very tentative math approach to WLB and
Blackberry use

• For each email away from work devise a way of
  measuring (relative to work)
• Disruption caused
• Time off work gained
• Response time improvement
• Look at patterns and thus model uses which
  havent happened

22
In conclusion

• The most difficult and important thing for
  management research is investigating things that
  havent happened
• Obsessively cataloguing views of what has
  happened can only help to a limited extent
• So need to be imaginative
• Any further thoughts tomichael.wood_at_port.ac.uk
  please.

23
References

• Ayres, I. (2007). Super crunchers how anything
  can be predicted. London John Murray.
• Feyerabend, P.K. (1993) Against method an
  outline of an anarchistic theory of knowledge
  (3rd edition) London Verso.
• Law, J. (2004) After method mess in social
  science research. Abingdon and New York
  Routledge.
• Salancik, G. R. (1979) Field stimulations for
  organizational behavior research. Administrative
  Science Quarterly, 24, 638-649.
• Russell, B. (1901) 'Recent Work on the Principles
  of Mathematics', International Monthly, 4, 84.
• Taleb, N. N. (2008). The black swan the impact
  of the highly improbable. London Penguin.