KNR 497

1
Experimental Design

• Ch. 8 Lets not kid ourselves, this is going to
  hurt

2
Experimental Design

• How on Earth can you ensure that 2 groups of
  different people are equal (in all respects, not
  just on the measure of choice) at the beginning
  of an experiment?
• You cant
• But you can make it more probable (and to
  experimenters, good enough)

Remember though, even if you achieve this, groups
can still grow different after they have been
formed
3
Experimental Design

• Searching for group equivalence
• What we do
• Random assignment
• Does it work?
• Maybe!
• Sample size, power c.

4
Experimental Design

• If random assignment is the solution, and
  increased internal validity is the benefit, is
  there a cost?
• Undoubtedly
• Sample size big enough?
• Control of social threats, mortality
• Its unreal, so improved internal validity comes
  at the cost of external validity

5
Experimental Design

• 2-group experimental designs

Two-group, post-test only randomized experimental
design
6
Experimental Design

• More on probabilistic equivalence
• Random assignment will distribute folk to groups
  such that their scores on any measure will be
  distributed randomly (duh)this means they will
  probably be different, but that it is
  statistically improbable that this will be a
  significant difference

7
Experimental Design

• More on probabilistic equivalence

8
Experimental Design
Random selection ? Random assignment
External validity control
Internal validity control
9
Experimental Design

• Classifying experimental designs
• Signal enhancing vs. noise reducing
• The signal vs. noise idea

Strong treatment enhances signal
Good measurement reduces noise
10
Experimental Design

• Classifying experimental designs
• Signal enhancing vs. noise reducing
• Designs differ in their strengths
• Factorial designs focus on isolating aspects or
  combinations of treatments that seem to affect
  the measurement most (signal enhancer)
• Covariance/blocking designs focus on lessening
  the effects of known sources of noise (noise
  reducers)

11
Experimental Design

• Factorial designs
• Imagine an educational program
• You are interested in (IVs)
• Time of instruction (1 hour vs. 4 hr)
• Setting (in-class or pulled out of class)
• You measure via study scores (DV)

Note we are now dealing with 2 independent
variables for the first time
12
Experimental Design Factorial
13
Experimental Design Factorial
14
Experimental Design Factorial
15
Experimental Design Factorial
16
Experimental Design Factorial
17
Experimental Design Factorial
18
Experimental Design Factorial
19
Experimental Design Factorial

• A silly example - The marshmallow peeps study
• Factor 1 Alcohol (presence/absence)
• Factor 2 Smoking (yes/no)

20
Experimental Design Factorial

• Does alcohol have an effect?
• Imbibed liberally

• Moderate headache
• Nausea
• No permanent damage

21
Experimental Design Factorial

• It can give up any time it wants tono effect

22
Experimental Design Factorial

• So, alcohol nicotine are benign?
• Wait..what if you combined them?
• Sum of the parts?
• More than the sum of the parts?

23
Experimental Design Factorial

• Is there an interaction?
• Combine the elements

• Faint flameblackening

• smell of caramel

• Metamorphosis

• ball of charred goo

• less sweet
• crunchier
• gross

24
Experimental Design Factorial

• Variations i. 2 x 3

25
Experimental Design Factorial

• Variations i. 2 x 3

26
Experimental Design Factorial

• Variations ii. 2 x 2 x 3 (3 factor)

27
Experimental Design Factorial

• Variations iii. 2 x 3 control

28
Experimental Design Blocking

• Reducing noise Randomized block designs
• Key point unexplained variation in a sample
  reduces power
• The solution is to reduce the variation within
  the sample by splitting the sample up
• You split across some factor that you know causes
  the sample to differ with respect to the measure
  of interest (making multiple blocks)
• You do not include this as a factor in the
  experiment, because it is not of interest
• Each block will have less variability on the
  measure, and therefore more power

29
Experimental Design Blocking

• Reducing noise Randomized block designs

Here is the design notation for what was
described on the last slide
30
Experimental Design Blocking

• Reducing noise Randomized block designs

s show scores for all treatment group members
(average of all gives treatment group score
average on x-axis is for pretest, and on y-axis
is for posttest
os show scores for all control group members
(average of all o gives control group score
average on x-axis is for pretest, and on y-axis
is for posttest
31
Experimental Design Blocking

• Reducing noise Randomized block designs

Note that, regardless of the block, the spread of
scores on the post-test is less within the block
than across the entire measure
32
Experimental Design Covariates

• Reducing noise Covariance designs
• Design can vary, but basic is this
• Lingo controlling for, removing the effect
  of
• Both terms imply use of covariates

33
Experimental Design Covariates

• Reducing noise Covariance designs

34
Experimental Design Hybrids

• Solomon 4 group

To examine control testing effects in pre-post
arrangements
35
Experimental Design Hybrids

• Switched replication design

To examine control social interaction threats
36
Experimental Design Hybrids

• Reducing social interaction threats
• (other than via switched replication)
• Blind double blind set ups
• Placebos
• Isolation of groups