Research Methods

1
Research Methods

• Michael Wood
• michael.wood_at_port.ac.uk
• http//userweb.port.ac.uk/woodm/rm/rm.ppt
• This file contains draft slides which will be
  updated.
• 30 November 2009

2
Reading

• There are many books available e.g.
• Saunders et al (2007)
• Robson (2002)
• Easterby-Smith et al (2002)
• And many others browse in the library
• These books vary a lot some are better on the
  practical aspects, others on the theoretical
  aspects. Sometimes you will get different advice
  from different sources, so you need to consider
  the rationale behind the advice. Robson is good
  on most aspects, although Saunders et al is
  probably more student-friendly

3
Contents

• Overview of academic business research
• What must be in a project plan and a project?
• Formulating research aims
• The design of research projects
• Evaluating research
• Statistical analysis for research
• Qualitative data analysis
• Analysing data and presenting results
• Philosophy of research
• Questionnaire design
• Interview design and qualitative research
• Reminders about the project
• Interviews and qualitative research more detail
• More on literature reviews

4
Overview of academic business research

• Reading browse through a book on research
  methods e.g. Saunders et al (2007), Robson
  (2002)
• These slides intended as a brief summary of the
  important points
• Reread them when you are starting your project

5
Advice on research methods

• Common sense dont forget this!
• Articles and books reporting similar research
  should be discussed in the project
• Books on research methods in general
• Focus on chapters relevant to your project.

6
Purpose and characteristics of academic research

• Purpose
• Discover truth about something and/or
• Find a good way of doing something
• Must be
• Systematic and as thorough and trustworthy as
  possible
• Clearly written and with sufficient detail for
  readers to check credibility
• Ethical

7
Types of research include

• Large scale surveys (of people, organisations,
  events, etc) analysed statistically
• Small scale surveys with emphasis on
  qualitative detail
• Case studies (to see how something works in
  detail)
• Experiments (change something to see what
  happens)
• Models can be set up, tested and used for
• Participant observation (observe as participant)
• Action research (combine research and action)
• Evaluation
• and may other possibilities be imaginative!
• Many projects combine several of these

8
Sources of data many possibilities

• Interviews
• Including focus groups, Delphi technique (Robson,
  200257), various approaches to eliciting
  comments (e.g. photo elicitation Sam Warren)
• Questionnaires, including via email (be careful
  )
• Documents (minutes of meetings, company reports,
  etc)
• The web
• Databases within organisation, of share prices,
  etc
• Observations of various kinds
• Etc . Be imaginative!
• Sources of literature is a different issue
  (Judiths session is very important for this)

9
Experiments (randomised controlled trials)

• Put people (or whatever you investigating) in
  randomly assigned groups, give the groups
  different treatments, and compare groups to see
  what differences emerge.
• Used for testing drugs, diets (http//tinyurl.com/
  yp2t2o, http//tinyurl.com/489hns), educational
  methods, different designs for websites, social
  policies, etc. Lots of examples in Ayres (2007).
• Advantages of experiments over non-interventionist
  research
• Disentangle cause and effect. Can control
  variables you havent even thought of. If done
  well evidence can be very convincing.
• Can investigate new things
• Ayres, Ian. (2007). Super Crunchers how
  anything can be predicted. London John Murray.

10
But

• Experiments are often impractical or unethical
• Difficulties include
• Hawthorne effect
• Failure to assign groups at random (this matters
  a lot because )
• So use less rigorous quasi-experiments instead
  (Grant Wall, 2008) e.g. in action research
  you may do a before and after comparison. This is
  a sort of crude experiment but it is not as
  convincing as a proper RCT.
• Grant, A. M. Wall, T. D. (2008). The
  Neglected Science and Art of Quasi-Experimentation
  Why-to, When-to, and How-to Advice for
  Organizational Researchers. Organizational
  Research Methods (published online, July 18,
  2008).

11
Finding a suitable topic

• Interest
• Career
• Feasibility
• Usefulness

12
How to do research

• Read about topic
• Draft aims of research. Clear, simple, focused.
• Draft literature review.
• Draft research plan check it is really likely
  to meet your research aims. Check again.
• Do research/analysis
• Draft research/analysis and recommendations/conclu
  sions
• Check it fits together and revise all sections
• If it doesnt fit together revise aims and

13
Practical issues

• Timing
• Plan this remembering that your supervisor may
  suggest extensive changes.
• Gantt chart may help.
• Ethics (remember the form!)
• Access to information.
• Take care this is often difficult!

14
What must be in a project and a project plan?

• Reading
• Project guidelines
• Proposal guidelines
• Saunders et al (2007), or another similar book

15
What must be in a project?

• Abstract (short summary of project including
  conclusions)
• Background and aims (what youre trying to find
  out and why its important)
• Literature review (of relevant previous research
  which you will build on or extend)
• Research methods plan and justification (what
  you did to meet the aims, and why it was a
  sensible approach)
• Analysis (in detail, to convince sceptical
  readers and impress examiners important tables,
  diagrams etc must be in the text, only details in
  appendix)
• Results, conclusions, recommendations,
  limitations, further research
• References (list works cited in text in
  alphabetical order)
• Appendices Ethics form, extra details for the
  reader
• Flexible designs can be more flexible but
  everything must be there!

16
Features of a good project

• Obviously important and interesting
• Difficult to disagree with because
• Arguments and analysis detailed, clear and
  obviously valid
• Possible objections considered and if possible
  answered
• Fits together
• Aims met by methods (check this in your project
  plan)
• Conclusions follow from analysis

17
References and citations

• You must give references to publications which
  you draw on or quote
• Exact (word for word) quotes must be in and
  the reference must be given
• Maximum about one paragraph
• Use one of the standard referencing systems
  preferably the Harvard (see university website)
• Copying word for word without and reference
  is treated as cheating and you will fail!

18
Harvard referencing system

• Very important to use this (or another
  established system)
• Seems easy to me, but causes a lot of difficulty
• Check library website (search for Harvard) and/or
  copy an academic article or book.
• All references in text like Smith (2001)
• Then alphabetical list of references at the end.
  Should include everything referred to, and
  nothing else.

19
What must be in your project plan (proposal)?

• See assignment description
• You may be able to put parts of it in your
  project!
• You should describe and justify your research
  methods in as much detail as possible

20
Writing style (1)

• Keep it simple.
• Short sentences
• Clear, short paragraphs
• Clear subheadings
• Read it through to make sure you can follow it.
  Swap with a friend and check each others

21
Writing style (2)

• I think the EMH was true in this situation
• In my opinion the EMH was true
• In the authors opinion the EMH was true
• 4 The evidence suggests that the EMH was true
• 5 This shows that the EMH was true
• Use 4 or 5.
• Avoid 1, 2 or 3 because it gives the impression
  that its just your opinion and that other, even
  wiser, people may see it differently.

22
Writing style (3)

• I work for and the problems are / I
  interviewed three managers.
• The author works for and the problems are /
  The author interviewed three managers.
• Then problems of this organization are / Three
  managers were interviewed.
• Opinions vary here. I (MW) prefer (1). Others
  prefer (2) or (3).
• Check with your supervisor.

23
Formulating research aims

• Reading most research methods books, e.g.
  Saunders et al, 2007

24
Research aims or questions

• Usually start from vague idea
• Then formulate a clear aim, or list of aims, that
  your research will achieve. Think of these as
  hoped-for outcomes.
• Alternativelyformulate a clear question or list
  of questions.
• This process may require some creative thinking
• Techniques like brainstorming and mind maps may
  be useful

25
Aims, objectives, questions

• You can formulate your research aims as aims (or
  objectives if you prefer that word) or questions.
  
• These are different ways of saying the same
  thing. Doesnt matter which you use, but dont
  confuse things by having aims and questions
• May be helpful to have a list or hierarchy of
  aims, but keep it simple

26
Hypotheses

• Hypotheses are statements whose truth you want to
  test, or predicted answers to research
  questions (Robson, 2002)
• Occasionally appropriate as a top level research
  aim
• e.g. to test the hypothesis that Working at home
  improves quality of life
• Usually best to avoid hypotheses when formulating
  main research aims because questions or aims tend
  to be more flexible
• e.g. How does working at home affect quality of
  life?
• Null hypotheses have a (controversial) role in
  some statistical analysis ( as you will see),
  but they are not relevant to formulating your
  overall research aims

27
Research aims or questions

• Research aims or questions should
• Be clearly and simply expressed
• Fit together (so that you have a coherent
  project)
• Clarify the intended outcome and scope of the
  research
• Your research aims or questions should also
• Be relevant to your degree
• Be achievable
• Present a reasonable level of challenge

28
Research aims or questions

• Must be research aims, not business or personal
  aims.
• However, business or personal aims may be part of
  the background motivating your research aims, and
  research aims would normally include the aim of
  making recommendations to people or
  organisations.
• Should generally have a limited scope or focus.
• The danger with general aims is that they lead to
  superficial research.
• May relate to theoretical issues. You may be
  aiming to test, modify or create a theory

29
Theory

• Theory includes models, explanatory frameworks,
  generalisations, recommendations
• Examples .
• Your research should link with any relevant
  theory. It may
• Use a theory
• Demonstrate that a theory is useful
• Test a theory
• Modify a theory or create a new theory

30
Also ask yourself

• Is the research worth doing?
• Are there any ethical or political problems?
• Is it possible? Have you got access to the
  necessary data?

31
Is it really going to be useful?

• What use do you want the results to be? This may
  be a practical use to find out how to make more
  money, or to make life easier or a contribution
  to theory, but it should be something that is
  really worth achieving. Must pass the so what?
  test.
• May help to clarify your aims if you imagine
  youve done the research and write down what you
  think your conclusions and recommendations might
  be.
• Then work backwards from what you want to achieve
  to the best methods to achieve it.

32
Example of research aims

• The aims of this research are to
• Describe the decision making strategies of small
  investors
• Determine the effectiveness of these strategies
• Any comments? Does this seem reasonable for a
  Masters project?

33
Another example of research aims

• The aims of this research project are to
• Evaluate Method X for planning mountaineering
  expeditions
• If necessary propose and justify Amended Method X
  for planning mountaineering expeditions

34
Another example of research aims

• What are the important quality problems in
  Company X?
• How serious are these problems?
• What is the best strategy for reducing these
  problems?
• Any comments? Does this seem reasonable for a
  Masters project? Does it matter that they are
  expressed as questions?

35
Three more examples of research aims

• The aim of this research is to investigate the
  role of the internet in banking.
• This research project aims to explain activity
  based costing.
• The aim of this project is to
• Test the efficient market hypothesis for the
  Athens stock exchange, and
• Determine how global warming will influence share
  prices.
• Any comments? These are not reasonable for an
  Masters projects! Why not?

36
Possible research topics

• Research in a specific organisation
• Best if they are likely to implement any
  recommendations
• Take care you have adequate access to data
• Easier if you have a recognised / paid job there
  and / or know key players well.
• Research based on publicly available data
• Eg share prices, the www, published statistics
• Research based on surveys of the public
• These are just some possibilities. There are more
  

37
Design of research projects

• Design means deciding on the methods and
  approaches which will best achieve your aims
• Needs thinking out carefully starting from your
  aims
• Check the proposed design will achieve all your
  aims
• The design may require the use of a theoretical
  framework which should be explained and its use
  justified
• May incorporate several approaches (e.g. earlier
  slide)
• Some advocate flexible designs (E.g. Robson,
  2002)
• E.g. Poppy Jamans summary. Any comments?
• E.g. check aims and designs of these projects.

38
Designing research is not easy!

• Think about how you can best achieve your aims
• Consider all possible types of research
• Be imaginative
• Think about it again
• and again
• Check youve found the best way you can for
  meeting all your aims

39
Group exercise

• Design a research plan for one of the projects
  below, and do a pilot study for part of it. (You
  may find you need to make the aims / questions
  more precise.)
• Michaels project. The provisional aims are
• To evaluate the suitability of the PBS website
  for prospective PhD students
• To suggest improvements to the website from this
  perspective
• Alisons project on the impact of a Blackberry on
  family/work-life balance. What are the problems
  and opportunities, and what would you recommend?
• or

40
Email project

• How much time do people spend on emails, is it
  time well spent, and if not how can things be
  improved?
• Provisional method Survey to find how much time
  is spent on emails, and respondents opinions on
  whether this is time well spent, and on
  recommendations (is this a satisfactory method?)
• And / or other possibilities ?

41
A general design for a typical Masters degree
project

• If the aim is to find a good strategy to
  "improve" X in org Y, then a possible design may
  be
• Survey/case studies of Org Y to investigate
  problems and opportunities
• Survey/case studies to see how other
  organisations do X and which approaches work well
• Based on (1), (2), the literature, and perhaps
  creative inspiration, consultations within the
  organisation, simulation or modelling, devise a
  strategy likely to improve X
• Try/test/pilot/monitor the proposed strategy,
  probably in a limited domain

42
Take care with opinion surveys

• Suppose your research is about risk management
  and its effectiveness. You decide to investigate
  by means of a questionnaire and come up with
• 70 of people in the organisation think our risk
  management is unsatisfactory
• 60 think Method X is the best way of improving
  it
• You then present this as the rationale behind
  your recommendations to improve risk management.
• But how do they know?
• Surely the researcher should find out by rigorous
  and sensible methods, rather than asking people
  who may neither know nor care?

43
Exercise

• There are many problems with interviews and
  questionnaires. Your respondents may
• Not know the answers
• Not understand the questions
• Be too lazy to think about the issues
• Want to deceive you
• Try to design the methods for a research project
  without using interviews or questionnaires. (This
  is not usually a good idea but it should help you
  to consider alternatives.)

44
Then

• Having designed your research get someone to act
  as a devils advocate and tell you
• Whats wrong with it why it may fail to deliver
  what you are aiming for
• What may go wrong
• Would they trust the answer?

45
Evaluating research

• Relevant to
• Planning your own research. Use the following
  slides to
• Check your proposal
• Check your final project
• Critically reviewing published research
• These slides are intended as a checklist for your
  research and others

46
Good research should be

• As User-friendly as possible
• Simple as possible given the message?
• As Uncritisable (trustworthy) as possible
• Trustworthiness or credibility is particularly
  important. Can you trust the conclusions? Do you
  believe them? Are there any flaws? Essential to
  give readers enough detail to check.
• As Useful or interesting as possible
• Clear implications for future? New results?

47
In groups

• Choose one of the articles you have been given
• Assess its
• User-friendliness
• Trustworthiness (pay particular attention to
  this)
• Usefulness
• Brief feedback session, then we will compare your
  critiques with my slides

48
Trustworthiness of research main things to check

• C
• R
• I
• T
• I
• C
• Each letter represents an issue you should
  consider

49
Jargon

• Most of these checks are covered by technical
  jargon, concepts and techniques e.g. lots of
  types of validity (internal, external, construct,
  face ), lots of types of reliability, ideas
  about test and scale construction (see Robson,
  2002), etc
• Read up only those areas which you think are
  relevant. I have largely avoided jargon here.
• Always check sampling always necessary to
  consider whether your sample is likely to be
  representative of your area of interest.

50
Deciding what is Cause and what is effect

• Important to try to work out what causes what,
  and how strongly and under what circumstances, so
  that you know what you should change to achieve a
  particular effect.
• Take care may be more complicated than it
  appears (ISO 9000 and profitability drinking and
  thinking, storks and babies).
• Variable you havent thought of may be the
  important cause!
• Experiments (randomised controlled trials) for
  definitive answers, but may be difficult, so
• Quasi-experiments (e.g. a before/after comparison
  of a trial of a new innovation) insead, but
• May be lots of causes. Be suspicious of simple
  explanations (see Taleb, 2008).

51
Deciding what is Cause and what is effect more
examples

• A survey of organizations showed that those that
  used the balanced scorecard were more profitable
  than those that didnt.
• Does this show that the balanced scorecard makes
  firms more profitable?
• A survey showed that the average job satisfaction
  score for a department rose substantially and
  significantly between 2006 and 2008. In 2007
  everyone was sent on a weeks computer course in
  the Seychelles.
• Would you recommend a computer course for other
  departments?
• Does high staff turnover cause poor performance
  or vice versa? (Glebbeek and Bax, 2004). Does
  extraversion help people get promoted, or vice
  versa (Moutafi et al, 2007). Does it matter?
• What caused the fall of the Berlin Wall?

52
To ensure results Representative check Sampling

• Decide what youre interest in often called the
  population or target population.
• Usually this is too big to look at everything so
  take a sample. Normally we want the sample to be
  representative of the population or wider
  contextso you must check if this is likely.
• Need to consider how the sample is selected and
  its size. Badly chosen samples can be biased and
  give very misleading results.
• E.g. TV audience research, word length, NRE,
  non-response bias in surveys, survivor bias in
  stock price samples

53
How to sample

• Clarify target population (the whole group of
  interest)
• May be a population of people, organisations or
  
• Decide sampling approach. There are many methods
  of taking a sample from your target population,
  including
• Random
• Stratified
• Purposive
• Convenience (or opportunity, haphazard,
  accidental)
• Cluster, snowball, quota, etc (see a book)
• Decide size of sample need to balance cost with
  information obtained. If you analysis is
  statistical, statistical theory can help

54
Random sampling

• Make a numbered list of the target population (a
  sampling frame)
• Use random numbers to choose sample
• Each member of population has the same chance of
  being selected (and its independent of any
  biases)
• Each member of sample selected independently
• In practice, likely that some members of the
  sample cant be found or wont help, so the
  sample may be biased. Difficult to deal with this
  possibilities
• The principle is to ignore all variables and
  choose at random. This allows for all noise
  variables.

55
Which sampling method?

• Usually random samples are best for large
  samples, and purposive samples for small samples
  analysed qualitatively.
• Done properly, with a large enough sample, random
  or stratified samples (probability samples)
  should be reasonably representative of the
  population. Cant assume this about purposive or
  convenience samples (non-probability samples)
  because these are selected by factors that are
  likely to bias the result in one direction or
  another.

56
Sampling in practice

• Many samples are biased and so will not give a
  good idea of the population regardless of
  sample size.
• E.g. NRE, non-response bias in surveys, survivor
  bias in
• Ideal for large samples is random sampling, but
  this is often difficult to do properly.
• E.g. Iraq war death rate (see http//www.iraqbodyc
  ount.org/ for another approach), TV audience
  research.
• Be suspicious of statistical results from
  purposive or convenience samples
• Need to be especially careful with small,
  purposive samples for detailed analysis
  consider the purpose and choose accordingly

57
Three surveys to check accuracy of NRE phone
service which is right?

• A Consumers Association survey used a sample of
  60 calls, mainly about fares. The worst mistake
  was when one caller asking for the cheapest fare
  from London to Manchester was told 162 instead
  of the cheaper 52 fare which was available via
  Sheffield and Chesterfield. The percentage
  correct was
• 32
• A reporter rang four times and each time asked
  for the cheapest route from London to Manchester.
  The proportion of the four answers which were
  correct was
• 25
• An NRE sponsored survey found that the answers
  were
• 97 correct
• (Source Breakfast programme, BBC1 TV, April 30
  2002.)

58
More sampling problems

• An MBA student sends out 100 questionnaires to
  100 organisations asking if they would be
  interested in a particular service. Twenty are
  returned, and of these 6 indicated they may be
  interested in the service
• There are 650 firms in the relevant industry
  sector. How big is the market for the service?
  Are you sure?
• Suppose you wanted to find out how common it is
  for women aged 30-40 to enjoy running.
• How would you choose a sample to ask?
• Other examples and exercises attached

59
Measurements (Indicators)

• If you want to find out whether customer
  satisfaction, or quality or profits have improved
  you must have a sensible way of measuring them.
• Moreno-Luzon (1993) used managers perceived
  achievement of objectives as a measure. Can you
  see any problems with this?
• How would you measure quality of service in a
  casino?
• How would you check if your proposed measure is
  valid / reliable / right / accurate?

60
Things to remember with measurements (1)

• Conventional to distinguish between validity (are
  you measuring the right thing?) and reliability
  (consistency)
• If possible use an existing measurement system
  (with acknowledgement / permission). This has two
  advantages there may be evidence validating it,
  and you can compare your results with previous
  results.
• Remember that some informants may be biased, or
  too lazy to give good answers, or just ignorant.
• If possible use triangulation (check with
  information from different sources)
• Ask yourself whether your proposed method of
  measurement really measures the right thing

61
Things to remember with measurements (2)

• Be especially careful with measures of value.
  This may have several dimensions (Keeney, 1992).
  E.g. the success of a firm might depend on
  profitability, worker satisfaction, contribution
  to the community
• If you are measuring the success of a change,
  remember there may be several different criteria.
  E.g.
• May be useful to use the average (mean) response
  to a series of questions. Use your common sense
  to see if this is reasonable, or if they should
  be kept separate. (See literature on Tests and
  scales e.g. Robson, 2002 292-308).
• Keeney, R. L. (1992). Value-focused thinking a
  path to creative decisionmaking. Cambridge,
  Massachusetts Harvard University Press.

62
Reliability of measurements

• Same answer at different times?
• If anything depends on subjective judgments,
  check agreement between different judges
• Eg marking projects
• If youre asking a number of questions to get at
  the same information, check the relationship
  between answers to these questions with two
  questions use a correlation coefficient, with
  more than two use Cronbachs Alpha (if you are
  keen on stats!) see http//www.statsoft.com/text
  book/stathome.htm

63
Exercise how would you measure

• ??

64
Theoretical assumptions

• If the research uses a theory, is the theory
  right for the purpose? And is it a valid
  theory? (Some theories, of course, are stupid or
  wrong!) You need a critical evaluation in your
  literature review.
• A questionnaire or interview plan may be based on
  assumptions about what is relevant. Are these
  assumptions OK?

65
Is the research sufficiently Imaginative?

• Imagination helpful in
• Thinking of hypotheses to explain things
• Thinking of new methods for researching
• Thinking of new ways of doing things
• Many recommendations for boosting imagination and
  thinking creatively e.g.
• Brainstorming
• Doing something else and coming back to the task
• etc

66
Making sure that you are not being misled by
Chance

• Could your results just be due to chance?
• Have you taken account of sampling error? (If you
  repeated your research with another sample are
  you sure the answer would be the same?)
• Is the sample large enough?
• Null hypothesis tests or confidence intervals can
  be used to answer these questions.
• Are the measurements reliable?

67
The first CRITIC

• Cause and effect assumptions OK?
• Representative sample?
• Indicators (measurements) OK?
• Theoretical assumptions OK?
• Imaginative enough?
• Chance ruled out as explanation?
• (Checks needed are mostly common sense except
  for Chance.)

68
The second CRITIC

• C Claim?
• R Role of the claimant?
• I Information backing the claim?
• T Test?
• I Independent testing?
• C Cause proposed?
• Teaching skepticism via the CRITIC acronym and
  the skeptical inquirer
• Skeptical Inquirer,  Sept-Oct, 2002  by Wayne R.
  Bartz

69
Two extra checks

• Use of a devils advocate or critical friend.
  Remember the problem of confirmation bias you
  are likely to be more enthusiastic about evidence
  that confirms your pet ideas than about evidence
  that undermines it! Get someone to try and be
  critical and find difficulties with your research
  then fix or (if unfixable) discuss the
  problems.
• Triangulation compare results from different
  sources. Applies to data, methods, observers,
  theories (Robson, 2002 174).

70
Anything else?

• Is this list complete?
• Does it address all the flaws you noticed in the
  paper you looked at?
• What would you add or change?

71
Checklist the 3 Us, the CRITIC and Extra checks

• User-friendly?
• UnCRITICisable (trustworthy)?
• CRITIC
• Useful?
• Extra checks
• Triangulation
• Devils advocate (critical friend)

72
Another measurement problem

• Andy had answers from lots of questions on a SD,
  D, N, A, SA scale
• He wanted a measure to tell him which questions
  produced responses which gave a a clear overall
  view (COV) from his respondents
• His defined his measurement as
• COV abs(SDDASA) N
• (where SD is the number of SD responses, etc, abs
  absolute value)
• He then highlighted questions for which COV gt 0
• Do you think this is a sensible measurement?

73
Critique of an article

• Do you accept what the article says, or are there
  flaws in the research?
• Think about the article! Use your common sense.
• Check the CRITIC.
• Is it worth publishing? Could you do better?
• Read round the subject e.g. other research on
  the same theme.
• Would the research benefit from some qualitative
  work, p values or confidence intervals, case
  studies, different perspectives, experiments

74
Statistical data analysis

• Go to http//userweb.port.ac.uk/woodm/stats/StatN
  otes0.ppt

75
Qualitative data analysis

• Aim is detail and depth of understanding
• Demonstrate and understand possibilities, but not
  how frequently they occur
• Use direct quotes () as evidence and to reduce
  danger of imposing your perspective
• Sometimes may be helpful to
• Summarise in a table or similar
• Use coding scheme to analyse statistically (but
  be careful if the sample is very small!)
• Further possibilities in Saunders et al, Robson,
  www.qual.auckland.ac.nz/, Thorpe and Holt (2008)

76
Analysing data and presenting results

• Questionnaires and interview plans, and possibly
  some data, in appendix
• Graphs and tables and important quotes from
  interviewees etc in the main text
• Focus on your research aims, not the questions in
  your questionnaire
• Readers want an analysis which shows how your
  aims are met. They dont want to know the answers
  to all the questions in your questionnaire!
• Use appropriate summaries e.g. tables of
  averages, or of main points from interviews

77
Literature review

• See Saunders et al (2003) Chapter 3
• Focus on relevant books, articles and theories
• Brief on general points
• More detailed on research of particular relevance
  to your project you will need to search for
  articles using the library databases
• Critical
• Should lead into your method and analysis
• Must be properly referenced!

78
Philosophy of research

• In the textbooks you will find discussions of
  positivism, social constructivism, phenomenology,
  etc, etc.
• In my view, Robson (2002) is the best research
  methods text for philosophical concepts.
• Almost all concepts and distinctions here open to
  serious criticism see Robson (2002). Most
  management research articles dont mention
  philosophy.
• I wouldnt suggest focusing on these ideas unless
  you are interested in which case be critical of
  what you read!
• If you do want to go into philosophy, use a book
  like the Penguin Dictionary of Philosophy
  (Mautner, 2005) or Thorpe and Holt (2008) to
  check what the words mean.
• Also note that there are arguments against being
  prescriptive about research methods and
  philosophy in books with titles like
• After method mess in social science research
  (Law, 2004), and
• Against method outline of an anarchistic theory
  of knowledge (Feyerabend, 1993)

79
Further reading and references

• http//userweb.port.ac.uk/woodm/qualquant.pdf

80
Some ideas which are worth mulling over

• Detailed study of a small sample vs less detailed
  study of a large sample
• Induction vs the Hypothetico-deductive method
  (Popper) vs Following a framework / paradigm /
  theory vs Deduction
• Subjective vs Objective Facts vs Values

81
Some misguided platitudes

• The following are often assumed (I think
  wrongly)
• There are two distinct kinds of research
• Quantitative (positivisthypothetico-deductive),
  and
• Qualitative (phenomenologicalinductive).
• Instead
• Positivist research (only) starts from
  hypotheses.
• Instead ...
• Academics tend to disagree about many of these
  issues. If you do decide to go into them, please
  think hard, and dont accept everything you read
  in the textbooks uncritically!

82
Qualitative vs quantitative

• Quantitative usually means statistical often
  with largish samples
• Qualitative means focusing on qualities usually
  with smallish samples studied in depth
• Disadvantage with statistical approaches is that
  the data on each case is often very superficial
• Disadvantage with qualitative approaches is that
  case(s) studied may be untypical and cant be
  used for statistical generalisation
• Often best to use both approaches. This is known
  as mixed methods search for this keyword in
  library.
• This distinction often confused with other
  distinctions

83
Regrettable tendency to reduce things to a simple
dichotomy

• If youre a soft and cuddly person
• Soft and cuddly (e.g. interpretivist,
  qualitative, inductivist ) is good
• Hard and spiky (e.g. positivist, quantitative,
  deductivist, ) is bad
• But if you are a hard person you will probably
  reverse good and bad above. There are really many
  different dichotomies. Reducing them all to one
  is neither right nor useful.

84
And

• To hard and spiky people, soft and cuddly
  research is lacking in rigour
• To soft and cuddly people, hard and spiky
  research is naïve and lacking in richness

85
Induction vs hypothetico-deductive method

• Generalise from the data without preconceptions
  (induction)
• Grounded theory. Rigour is in process used to
  generate theory from data
• Versus
• Use data to test hypotheses or theories
  (hypothetico-deductive method)
• Karl Popper. Rigor is in the testing.
• Theory building vs theory testing is much the
  same distinction (see Saunders et al, 2007, pp
  117-119)
• However, I dont think these are the only choices
  

86
Other useful approaches besides induction and
hypothetico-deduction

• Use a framework or theory or paradigm (Kuhn,
  1970) to define concepts, questions, and
  measurements, but without trying to test the
  theory
• Arguably what most scientists do most of the time
  (c.f. Kuhn, 1970). Rigour is in ensuring the
  theory is a good one, and in using it properly.
• Deduction from data, theories and framework. E.g.
  the differences between two quality standards can
  be deduced.
• Rigour is in checking the deduction and the info
  you start with
• Differs from the hypothetico deductive method in
  that the result is the deduction itself, not a
  confirmation, rejection or revision of a
  hypothesis or theory
• Note that this contradicts the assumption in
  Saunders et al (2007 117-119) that there are
  just two approaches deductive and
  inductive. I think they mean
  hypothetico-deductive, and they omit the two
  very important possibilities above.

87
An example

• How would these four approaches work with a
  project of interest to you

88
Karl Poppers ideas (1)

• Science works by putting forward bold theories
  (or hypotheses) and then testing them as
  thoroughly as possible
• Provisionally accept theories that have withstood
  this testing process
• Theories must be sufficiently precise to be
  falsifiable otherwise not proper science (eg
  Freuds theories are too vague)

89
Karl Poppers ideas (2) - eg

• Einsteins theory of general relativity predicts
  that light from a distant star will be bent by a
  small amount by passing close to the sun.
  Newtons theory predicts the light will not be
  bent.
• Only possible to check during a total eclipse of
  the sun. In an eclipse in 1918 light was bent as
  Einsteins theory predicted
• Newtons theory is falsified Einsteins lives on
  and seemed much more credible.

90
Karl Poppers ideas (3)

• Theories can come from anywhere guesswork,
  intuition, other theories, etc
• The process of criticising theories and trying to
  show they are wrong is vital for science
• The method applies to both natural and social
  sciences
• How would you apply Poppers ideas to a
  management research project? in practice, has
  elements in common with a critical attitude

91
Critical attitude

• Try to anticipate and discuss criticisms
• Get a friend to act as a devils advocate
• Your work should be so convincing that it cant
  be disputed!
• Think of any criticisms you have of articles you
  have read. Make sure the same faults dont apply
  to your work.
• Word critical sometimes used in a slightly
  different, more specific, sense.

92
Questionnaire design summary

• Read a (chapter of a) book on questionnaires
• Develop a pilot. Remember questionnaires are far
  more difficult to design than they appear! Check
  with your pilot respondents
• Is it clear?
• Is it interesting / appealing / user-friendly /
  not too long? Would you answer it?
• Does it provide (only) the information you want?
• Are you still sure a questionnaire is a good idea?

93
Questionnaire design (1)

• Write down what you want to find out
• Closed questions
• Tick boxes
• Rating (Likert) scales
• Open questions
• Pros and cons of each
• Check your questions will enable you to find out
  what you need to for your research

94
Questionnaire design (2)

• Covering letter
• Pilot it
• 3-4 nice friendly people to tell you whats wrong
  with it
• Pilot the analysis too
• Consider sample to send it to
• Anonymity / confidentiality
• How to send it / get it back (email?)
• What to do about non-response?

95
Questionnaire design (3)

• Far too many questionnaires about - many of them
  very silly. What is the response rate likely to
  be? Would you fill it in?
• Are you sure a questionnaire is necessary???
• Many companies have a policy of not responding to
  questionnaires
• Are there any alternatives?
• Check with your supervisor before sending it out

96
Take care with opinion surveys

• You can ask someone
• What she did last week
• What she does in general terms
• Her opinion of what she does
• What she thinks other people do
• Her opinion of what she thinks other people do
• How she thinks things can be improved
• What she thinks about particular suggestions
  about how things can be improved
• What she likes / wants / values
• Etc
• Think about what type of question you are using
  and whether it is really useful

97
Interview design in brief (1)

• Read a (chapter of a) book on interviews
• Follows, or precedes questionnaire, or stands
  alone
• Be clear what you want to find out
• Consider telephone interviews
• Small sample. Dont do too many interviews.
• Plan your questions. Should be open-ended and
  flexible, and aim for a detailed understanding
• Probes and prompts

98
Interview design (2)

• Ask for permission to tape record
• Transcribe interesting bits to get quotes for
  your project
• Get interviewee relaxed. Anonymity /
  confidentiality (take care here!)
• Check youve covered everything
• Send interviewee transcript afterwards?
• Some transcripts or parts of transcripts in
  appendix?

99
Reminders about the project

• Research aims should be simple, explicit,
  focussed, motivated, useful
• Literature review should focus on relevance to
  your project
• References should be complete and in order
• Methods should be the best which are feasible.
• Analysis chapter should show how hard and
  skilfully youve worked, and why readers should
  believe you. You need to convince a sceptical
  reader who may want to know details of how your
  data was obtained e.g. source of samples,
  location of interviews (pub or office?), etc, etc
  and analyzed.
• Conclusions and recommendations should summarise
  what you have found, and clearly meet the
  research aims. Also discuss limitations.
• Changing your mind is to be expected if
  necessary rewrite aims after doing the research!

100
Reminders (2)

• Docs/links at http//userweb.port.ac.uk/woodm/pro
  jects
• Keep to the 15,000 word limit. You can get a good
  mark with 13,000 words but not with 17,000 words.
• Remember the ethics form no form, no pass!
• Be particularly careful about NHS ethics
  clearance
• Make use of your supervisor (see Project
  Guidelines)
• Plan the timescale (Gantt chart) allow time for
  delays
• Allow time at the end for your supervisor to read
  it for you to make any necessary amendments
• If its good, consider publishing a summary in a
  journal. Ask your supervisor.

101
When starting your project you should

• Have a clear aim, and a rough idea of your
  methods and the relevant literature, and a few
  ideas about problems
• Make an appointment with your supervisor and
  discuss what you will do and the timescale. Take
  your proposal and comments
• Remember your supervisor may have a holiday
  planned agree when you will meet / email. Usual
  to send drafts of chapters when completed
• Remember the deadline and plan back from this.
  Send your supervisor a draft of the project at
  least a month before the deadline
• Project guidelines at http//userweb.port.ac.uk/w
  oodm/rm
• Practical guidelines on statistical analysis at
  http//userweb.port.ac.uk/woodm/stats/statnotes0.
  pdf
• Any questions to michael.wood_at_port.ac.uk

102
Interviews and qualitative research more detail

• I am grateful to Alan Rutter for the next few
  slides, some of which I have edited slightly

103
Primary data collection interviewing

• Useful for accessing peoples perceptions,
  meanings, definitions of situations, eliciting
  their constructions of reality, etc.
• Alternative types
• structured
• semi-structured
• in-depth
• Ethical considerations

104
Forms of qualitative interviews
F F
f
Qualitative interviews
One to one
One to many
Focus group interviews
Face to face interviews
Telephone interviews
After Saunders, et al, 2000
105
Interview respondents

• Who will be interviewed and why?
• How many will be interviewed and how many times?
• When and for how long will each person be
  interviewed?
• Where will each person be interviewed?
• How will access to the interview situation be
  organised?

106
Sampling for small sample qualitative research

• Usually best to use theoretical (purposive)
  sampling - the selection of individuals who you
  think will best contribute to the development of
  a theory
• Results apply to immediate situations
• May be tentatively generalised, but the small
  sample means

107
Difficulties with interviews

• Mistrust by respondents
• e.g. researcher is a management spy
• Loyalty to organisation/colleagues
• Adherence to stereotypical views rather than
  their own inner feelings and knowledge
• Complete indifference
• An opportunity for respondent to sell their
  ideas

108
Managing the interview

• Preparation for the interview
• the interview schedule
• Beginning the interview - establishing rapport
• Communication and listening skills
• Asking questions
• sequence and types of questions
• Closing the interview

109
Verifying interview data

• Body language
• Material evidence
• e.g. company/factory tour
• Writing notes
• as soon as possible after interview
• Use informant verification and secondary sources

110
Remember

• Need to demonstrate rigour
• Good research acknowledges bias and the need to
  expose it.
• Devils advocates are useful for revealing bias
  and other problems, but are seldom used.
• Is all research is biased?

111
More on Literature reviews

• I am grateful to Andreas Hoecht for the next 16
  slides
• Dont forget the literature should be clearly
  focused on your research aims, and it should be
  critical in the sense that you should point out
  strengths and weaknesses where appropriate

112
Research methods writing a literature review
(Andreas Hoecht)

• 1.Finding material
• 2. Mapping relevant literatures
• 3. Evaluating literature
• 4.Some practical hints

113
Writing a literature reviewFinding material

• There is no prescribed number of sources you
  should use, it depends on the topic
• Be wary if you feel that you are drowning in
  material you found for your topic, it probably
  means you have not narrowed it down enough
• Be wary if you find no sources or very little
  sources. You normally need some academic sources
  to be able to write a meaningful literature review

114
What secondary sources should you use?

• Books
• Use textbooks only to get an overview over a
  topic
• Academic monographs (edited books with chapters
  by different authors) can be very useful. They
  often explore a topic from different angles or
  cover different aspects of a topic
• Dont use airport bookstall books as serious
  sources

115
Secondary sources

• Journals
• Peer-reviewed academic journal articles should
  normally be the backbone of your literature
  review
• They provide up-to date discussions of topics and
  are usually more narrowly focused than textbooks
• Trade journals (non peer-reviewed) can provide
  good introductions to topics and overviews of
  developments but carry considerably less academic
  weight than academic journals.

116
(Secondary) sources

• Sometimes you may be able to find article titles
  like A review of the literature in academic
  journals. They can save you lots of work
• Internet
• Make sure you are able to distinguish between
  credible sources and Joe Blocks unsubstantiated
  views
• Reputed organisations websites can be good
  sources of information (but may have a
  bias/self-interest). (gov. Agencies, internat.
  Organisations)

117
(Secondary) sources

• Dissertations and PhDs
• Checking dissertations stocked in the library may
  help you to get a feel for what is expected in a
  dissertation as well as provide information on a
  topic
• Government reports/EU reports/other
  organisations reports can be very useful (but
  are sometimes biased).

118
Searching for literature

• The key is the use of electronic databases
• Some databases are full text (you can download
  articles directly), others are bibliographic
  databases (you need to check with library or use
  inter-library loan requests)
• Business Sources Premier/Emerald Full
  Text/Econlit/Science Direct are all recommended
• Be patient and creative in the use of keywords

119
Searching for literature

• CD-Rom newspaper databases (FT, Economist) can be
  useful tools
• Financial Data and Marketing databases mainly
  provide primary data

120
Mapping out relevant literatures

• Dont put everything you find or everything you
  read in your literature review
• Time spent on familiarising yourself with and
  assessing literature for relevance is never
  wasted
• Only after you have gained a good overview over
  the literature will you be able to decide on your
  particular angle and your research questions

121
Mapping out relevant literature

• Your database search should tell you how much and
  what type of literature is available
• For some well-researched topic you will be able
  to concentrate on the literature directly dealing
  with your specific topic
• For other research ideas, you may need to think
  about related areas or similar experiences in
  other industries or possible insights from other
  subject disciplines for enlightenment

122
Mapping out relevant literature

• An simple example If you are interested in TQM
  and small firms you may wish to
• Look at the TQM literature in general for the
  pros and cons, constraints and motives
• Identify success and failure factors from the TQM
  literature
• Check the small business literature for general
  business conditions and constraints
• Check the small business literature to find out
  if these success factors apply there

123
Mapping out relevant literature

• You can draw this as a conceptual map of
  overlapping circles or as a flow diagram if this
  suits your learning style
• Brainstorming and drawing conceptual maps is best
  done after you have gained a feel for the
  literature from your literature search

124
Evaluating literature

• This becomes easier with experience
• When reading literature
• identify the key arguments that are made
• The reason(s) for the conclusions drawn
• They should be either derived from logical
  deduction (a conclusion following necessarily
  from premises) and /or based on empirical
  evidence

125
Evaluating literature

• Check the logic of the arguments made
• Does this necessarily follow?
• Check the supporting evidence
• Is this data relevant? Is it meaningful and
  accurate? Could it be interpreted in another way?
  Which data would I need to challenge this?
• Check for flaws tautologies, simplistic
  analogies, redefinition of terms, moral
  judgements (ought to)

126
Some practical hints

• Make sure you refer to key texts that are
  frequently cited in the literature
• Find out whether there are different schools or
  camps in the literature and cover their
  positions.
• Use your research questions to structure your
  literature review
• Check the validity (logic, empirical evidence) of
  arguments made
• Make clear on what basis you decide to side with
  a camp or author or why you remain unconvinced
  or oppose a judgement

127
Some practical hints

• Dont overstate your case and be realistic about
  what you can conclude
• Be particularly fair to views and arguments you
  dont agree with (avoid to be seen as biased)
• Dont be shy to critique established trade
  names(academic gurus)
• Write your literature review for non-specialists
  and avoid jargon
• Write it well structured and easy to read