Methodology Matters: Doing Research in The Behavioral and Social Sciences

1
Methodology MattersDoing Researchin The
Behavioraland Social Sciences

• Student Way Chang Cai
• Scott Lippert

2
Introduction

• Doing Research simply means the systematic use of
  some set of theoretical and empirical tools to
  try to increasing our understanding of some set
  of phenomena or events.

3
Some Basic Features of The Research Process

• Some content that is of interest
• Some ideas that give meaning to that content
• Some techniques or procedures by means of which
  those ideas and contents can be studied.

4
Some Basic Features of The Research Process

• Substantive domain, which we draw contents that
  seem worthy of our study and attention.
• Understand both actor and context here refer
  to human system
• Conceptual domain, which we draw ideas that seem
  likely to give meaning to our results.
• The idea give the meaning that we study
• Methodological domain, which we draw techniques
  that seem useful in conducting that research.
• Methods are the tools the instruments,
  techniques and procedures by which a science
  gathers and analyzes information

5
Methods are the tools

• The instruments, techniques and procedures.
• Ex. If you want to cut a apple, a hammer dont
  help much. For that you need a saw.
• All research method help you gain knowledge.
• All method used to gather and analyze evidence

6
However

• All methods have inherent flaws, through each has
  certain potential advantages. You cant avoid
  these flaws, but you can bring more than one
  method to bear on each of aspect of a problem.

7
Research Strategies Choosing a Setting for a
Study

• Research evidence in social and behavioral
  science always involves somebody doing something
  in some situation, which are who, what and where.
  

8

• Three criteria
• Generalizability of evidence over the population
  of Actors.
• Precision of measurement of the behaviors that
  are being studied.
• Realism of situation or context within which the
  evidence is gathered, in relation to the contexts
  to which you want evidence to apply.
• You may always want to maximize these criteria,
  but increasing one of these three features
  reduces one or both of the other two.

9
Quadrant III Respondent Strategies
Obtrusive
Unobtrusive
Abstract
Sample Survey
Judgment Study
Laboratory Experiment
Formal Theory
Quadrant II Experimental Strategies
Quadrant IV Theoretical Strategies
Experimental Simulation
Computer Simulation
Field Study
Field Experiment
Maxima for each criterion A Generalizability B
Precision C Realism
Concrete
Quadrant I Field Strategies
10
Quadrant I

• Field Study
• The researcher sets out to make direct
  observations of natural, ongoing systems, while
  intruding on and disturbing those systems as
  little as possible. (case studies)
• Field experiment
• Researcher gives up some of the unobtrusiveness
  of the plain field study, in interest of gaining
  more precision in the information resulting from
  the study.

11
Quadrant II

• Laboratory experiment
• Researcher is able to study the behaviors of
  interest with considerable precision, and to do
  so under conditions where many extraneous factors
  have been eliminated or brought under
  experimental control.
• Experimental simulation
• The researcher attempts to achieve much of the
  precision and control of the laboratory
  experiment but to gain some of the realism (
  apparent realism ) of field studies.

12
Quadrant III

• Sample survey
• The investigator tries to obtain evidence that
  will permit him or her to estimate the
  distribution of some variables, and/or some
  relationships among them, within a specified
  population.
• Judgment Study
• Researcher concentrates on obtaining information
  about the properties of a certain set of stimulus
  materials, usually arranged so that they
  systematically reflect the properties of some
  broad stimulus domain.

13
Quadrant IV

• Formal theory
• The researcher focuses on formulating general
  relations among a number of variables of
  interest.
• Computer Simula
• Complete and closed system that models the
  operation of the concrete system without any
  behavior by any system participants.

14
Strategic Issues
15
Baserates

• Provide a basis for comparison
• Eg 3 out of 10 smokers develop lung cancer
• Is this high, low or normal?
• A baserate will give this information

16
Correlation

• Does X covary with Y?
• E.g. Does happiness vary with age?
• Covariance can have high positive or negative
  value
• Covariance can be linear or nonlinear
• Nonlinear is often overlooked
• Covariance shows that X and Y have a
  relationship, but not necessarily a causal one

17
The Difference Question

• Does the presence of X alter the state or
  magnitude of Y?
• If so, to what degree?

18
Randomization in Experiments

• Attempt to minimize the effect of extraneous
  factors
• Randomization doesnt guarantee equal
  distribution of extraneous factors
• Does make highly unequal distribution unlikely

19
Sampling, Allocation

• A large sample minimizes chance results
• Probability of chance causing results can be
  estimated
• Accuracy limited by knowledge of extraneous
  factors
• Even if results are likely not caused by chance,
  results dont prove X caused Y
• Could be that ignored variable caused Y
• This can be prevented with careful design

20
Validity of Findings

• Internal Validity
• Degree the results allow causal relations to be
  determined
• Construct Validity
• How well defined is the theory behind the study
• External Validity
• Concerned with the limits under which the results
  are valid

21
Classes of Measures and Manipulation
22
Potential Classes of Measures

• Self report
• Questionnaire, diary
• Observations
• Visible or Invisible
• Refers to whether subject knows he is being
  observed
• Archival records
• Records usually made for non-research purposes
• Trace measures

23
Self Reports

• Strengths
• Inexpensive, Easy to make, low rate of
  information discarded
• Weaknesses
• Reactivity
• This means that people respond as they think they
  should, rather than how they normally would

24
Observations

• Strengths
• Large amounts of information created
• Weaknesses
• Large amounts of information created, reactivity,
  high cost, ethics issues

25
Trace Measures

• Strengths
• Unobtrusive, non-reactive
• Weaknesses
• Information lacks context, information is not
  specific enough

26
Archival

• Strengths
• Sometimes only option, inexpensive
• Weaknesses
• Reactive, loose link between the information
  desired and that archived

27
Techniques for Manipulating Variables

• Selection
• Direct intervention
• Induction

28
Selection

• Group based on one criteria
• E.g. Age, sex
• Lose random distribution of the variable you are
  manipulating

29
Direct Intervention

• Set up the desired situation
• E.g. creating juries of 12, 6
• Inexpensive, gives reliable information
• Can still distribute randomly
• Only works for tangible, superficial variables
• Some reactivity exists

30
Induction

• Different types
• Misleading instructions
• Lie to subjects about what is being measured
• False feedback
• Give feedback based on the needs of the
  experiments, not the results obtained by subject
• Experimenters pretend to be subjects
• Carry out pre-determined activities to measure
  results