Experimental Research

1
Experimental Research

• Jeremy Kees, Ph.D.

2
Evidence that supports a causal inference

• Concomitant variation--evidence of the extent to
  which X and Y occur together or vary together in
  the way predicted by the hypothesis
• Time order of occurrence of variables--evidence
  that shows X occurs before Y
• Elimination of other possible causal
  factorsevidence that allows the elimination of
  factors other than X as the cause of Y
• X -- the presumed cause
• Y -- the presumed effect

3
Types of Experiments
Laboratory Experiment
Research investigation in which investigator
creates a situation with exact conditions, so as
to control some, and manipulate other, variables
Experiment
Scientific investigation in which an
investigator manipulates and controls one or more
independent variables and observes the
dependent variable for variation concomitant to
the manipulation of the independent variables
Field Experiment
Research study in a realistic situation in which
one or more independent variables are
manipulated by the experimenter under as
carefully controlled conditions as the
situation will permit
3
4
Definitions and Concepts

• Independent variables (IV) are variables or
  alternatives that are manipulated and whose
  effects are measured and compared, e.g., price
  levels.
• Test units are individuals, organizations, or
  other entities whose response to the independent
  variables or treatments is being examined, e.g.,
  consumers or stores.
• Dependent variables (DV) are the variables which
  measure the effect of the independent variables
  on the test units, e.g., sales, profits, and
  market shares.
• Extraneous variables are all variables other than
  the independent variables that affect the
  response of the test units, e.g., store size,
  store location, and competitive effort.
• Covariates

5
Validity

• Internal validity refers to whether the
  manipulation of the independent variables or
  treatments actually caused the observed effects
  on the dependent variables. Control of
  extraneous variables is a necessary condition for
  establishing internal validity.
• External validity refers to whether the
  cause-and-effect relationships found in the
  experiment can be generalized. To what
  populations, settings, times, independent
  variables and dependent variables can the results
  be projected?

6
Causal Research (Experimental Design)

• Internal Validity

7
Causal Research (Experimental Design)

• External Validity

8
Threats to Validity

• History--Specific events external to an
  experiment, but occurring at the same time, which
  may affect the criterion or response variable
• Maturation--Processes operating within the test
  units in an experiment as a function of the
  passage of time per se
• Testing--Contaminating effect in an experiment
  due to the fact that the process of
  experimentation itself affected the observed
  response

9
Threats to Validity

• Instrument Variation--Any and all changes in the
  measuring device used in an experiment that might
  account for differences in two or more
  measurements
• Statistical Regression--Tendency of extreme cases
  of a phenomenon to move toward a more central
  position during the course of an experiment
• Selection Bias--Contaminating influence in an
  experiment occurring when there is no way of
  certifying that groups of test units were
  equivalent at some prior time
• Experimental Mortality--Experimental condition in
  which test units are lost during the course of an
  experiment

10
Controlling Extraneous Variables

• Randomization refers to the random assignment of
  treatment conditions to experimental groups by
  using random numbers. This is the key to internal
  validity (extraneous variables are equal across
  groups due to random assignment).
• Assumed to produce balancing across groups --
• Comparable groups due to randomness of
  assignment average participant is the same
  across groups for non-manipulated variables
  (e.g., Distribution of extraneous variance and
  variables are constant across groups)
• if 65 female in one group, about same in others

11
Controlling Extraneous Variables (less effective
ways)

• Matching involves comparing test units on a set
  of key background variables before assigning them
  to the treatment conditions.
• Statistical control involves measuring the
  extraneous variables and adjusting for their
  effects through statistical analysis.
• Design control involves the use of experiments
  designed to control specific extraneous
  variables.

12
Characteristics of Good Experiments

• Random assignment
• Comparison group/control group
• As a source of comparison
• As a control for rival hypotheses
• Generalizability/external validity
• Random selection

13
Limitations of Experimentation

• Experiments can be time consuming, particularly
  if the researcher is interested in measuring the
  long-term effects.
• Experiments are often expensive. The requirements
  of experimental group, control group, and
  multiple measurements significantly add to the
  cost of research.
• Experiments can be difficult to administer. It
  may be impossible to control for the effects of
  the extraneous variables, particularly in a field
  environment.
• Competitors may deliberately contaminate the
  results of a field experiment.

14
Causal Research

• Who Can Resist an Oreo? Choice Behavior and
  Gender Differences when Body Image Anxiety is
  made Salient, Presented at the Marketing and
  Public Policy Conference (2005).
• Lit Review
• The majority of women and men are unhappy with
  their appearance (Warner 2002)
• Media images may result in body dissatisfaction
  (Shaw 1995), decreased perception of personal
  attractiveness (Odgen and Mundray 1996) and body
  image anxiety (Richins 1991)
• Media images are a primary factor that leads to
  body image anxiety (Richins 1991, 1995)
• Social Comparison (Festinger 1954)
• Differences in self-monitoring determine to what
  extent people use internal versus external info
  to guide decisions and behaviors (Snyder 1980)

15
Causal Research

• Who Can Resist an Oreo? Choice Behavior and
  Gender Differences when Body Image Anxiety is
  made Salient, Presented at the Marketing and
  Public Policy Conference (2005).
• Research Questions
• Can exposure to models in advertisements affect
  body esteem and choice behavior in addition to
  body image anxiety?
• Do individual differences in self-monitoring a)
  impact BIA, body esteem, and behavior and b)
  moderate the impact of model exposure?
• Are there differences in how women versus men are
  impacted by model ideals in ads?
• Hypos
• H1 Exposure to a model in an advertisement will
  result in (a) higher levels of body image anxiety
  and (b) lower levels of body esteem.
• H3 Consumers with higher tendencies to
  self-monitor will have stronger reactions to the
  exposure to a model in an advertisement than
  those consumers who do not self-monitor.

16
Causal Research

• Who Can Resist an Oreo? Choice Behavior and
  Gender Differences when Body Image Anxiety is
  made Salient, Presented at the Marketing and
  Public Policy Conference (2005).
• Research Design
• 3 (model exposure) X 2 (self-monitoring) for both
  males and females
• N 240 Undergraduates
• Dependent Variables

Females Body Image Anxiety Physical Condition (7
items, a .96) Body Weight (9 items, a
.98) Sexual Attractiveness (2 items, a
.72) Cookie Choice
Males Body Image Anxiety Physical Condition (10
items, a .85) Upper Body Strength (6 items, a
.81) Physical Attractiveness (3 items, a
.73) Cookie Choice
17
Causal Research

• Who Can Resist an Oreo? Choice Behavior and
  Gender Differences when Body Image Anxiety is
  made Salient, Presented at the Marketing and
  Public Policy Conference (2005).
• Results

18
Causal Research

• Who Can Resist an Oreo? Choice Behavior and
  Gender Differences when Body Image Anxiety is
  made Salient, Presented at the Marketing and
  Public Policy Conference (2005).
• Results

19
Causal Research

• Who Can Resist an Oreo? Choice Behavior and
  Gender Differences when Body Image Anxiety is
  made Salient, Presented at the Marketing and
  Public Policy Conference (2005).
• Results

20
To conclude

• Experiments are the only way to show causation
• But often take a back seat to descriptive studies
  due to time, cost, and control issues
• Exploratory and descriptive studies are useful,
  but be careful not to infer too much
• Correlation is not causation
• Again, let your research questions dictate your
  design!