Loading...

PPT – Group comparisons: Intro to factorial design PowerPoint presentation | free to view - id: 2a60a-MTBkM

The Adobe Flash plugin is needed to view this content

Group comparisonsIntro to factorial design

- 3143/Josh Rodefer, Ph.D.

Previously (?)

- Experimental design
- Design Between vs Within
- Compare 2 groups (t test different types)
- Paired (within repeated measures)
- Unpaired (between)
- Control confounds
- Extraneous variables (random vs systematic)
- How to control or minimize?

Basic issues in experimental

- What do you need for an experiment?
- An independent variable (what the researcher

varies minimally 2 groups) - Equal assignment to groups (lots of ways to do

this) - Controlling extraneous variables (alternative

explanations confounds)

Hypothesis testing 2 samples

- Want to test relationship between the means
- Use a t-test
- Used when you have 2 levels of 1 IV
- Dependent samples (1 sample, tested twice)
- Independent samples (2 different samples)
- aka Within- (dependent) and Between (independent)

designs

Hypothesis testing 2 samples

- Dependent samples (Within)
- Eg, before and after treatment (or pre/post)
- 20 people tested on attitude towards recycling

after watching nature video - IVviewing nature video
- Level 1 test score pre-video
- Level 2 test score post-video
- DV score on test
- 2 samples, same people in each sample

Hypothesis testing 2 samples

- Independent samples (between)
- 2 scores are taken, but from different groups 2

independent samples (random) - Eg, study effects of age on swearing
- 20 ten yr olds and 20 fifteen yr olds, record

swearing in 2 hr period - IVage
- Level 1 10 yrs old
- Level 2 15 yrs old

More than 2 groups ANOVA

- Analysis of Variance
- Tests the differences between treatment groups

(conditions) to see if they are significant - Look at variance in DV, partition it into 2

components - variance due to IV (good)
- variance due to error (bad extraneous variables)
- Asks if the ratio (IV variance/error variance)

between the two types of variance is greater than

would be expected due to chance (or equal or

about 1.00) - F test examines this ratio

ANOVA when to use?

- One IV at least 3 conditions
- Between subjects One-way ANOVA
- Eg, effect of psychiatric diagnosis (depression,

panic, no disorder) on memory - Within subjects Repeated measures ANOVA
- Eg, experiment where subjects experience all

conditions example Stroop effect, colors/words - Two (or more) IVs
- Factorial ANOVA
- Effect of psychiatric diagnosis on Stroop effect

multiple IVs

First One way ANOVA

- One IV 3 (or more conditions)
- Single test
- Tests Null that all group means are equal
- Alternative all group means are not equal
- Null must be non-directional (vs t test

directional) - Between subjects ANOVA
- Analogous to independent samples t-test
- (stats trivia In fact.if you have 2 independent

groups, then Ft2)

One way ANOVA

- Why not just perform 3 t-tests?
- Psychiatric diagnosis memory example
- Depression vs panic disorder
- Depression vs no disorder
- Panic disorder vs no disorder
- Same thing, right?

One way ANOVA

- Multiple tests result in an inflated alpha
- Alpha for each test 0.05 (risk for Type I)
- If you do 3 tests, then each is at 0.05 and they

are additive - Thus, doing 3 t-test increases the

experiment-wise Type I error rate from 0.05 to

0.15 (not acceptable) - Must use ANOVA to test all combinations

simultaneously, keeping alpha at 0.05

One way ANOVA

- So, we can calculate the total variance and

determine how much is due to IV and how much is

error - F ratio variance due to IV /error variance
- Variance due to IV is called MSB (Mean square

between groups) - Variance due to extraneous variables or error is

called MSW (Mean square within groups) - F test formula F MSB/MSW

ANOVA

- Most data sets have both error variance and

variance due to the IV (or, both between and

within group variability) - We want to know if the between group variance is

due to a true effect of the IV (is this effect

real?) - FMSb/MSw
- If F about 1, then no effect of IV
- If F 1 then may have an effect of IV (need to

look up value on F table, depends on critical F

and df)

Factorial ANOVA 2 way designs

- Have 2 (or more) IVs in the same experiment eg,

test effects of gender and age on humor - IV gender (M/F)
- IV age (10, 20, or 30 yrs old)
- DV freq of laughing during comedy show
- Why not conduct 2 separate experiments?
- Gender on humor age on humor

Factorial ANOVA 2 way designs

- Why do you want to look at 2 IVs?
- More efficient study factor A then study factor

Boryou could study them together - More interesting can see how things relate

together - More representative of the real world

Factorial designs

- Have 2 (or more) IVs
- Each IV is called a factor
- Each factor has at least 2 levels/conditions
- The design must be complete each level of one IV

must occur with each level of the other IV(s) - Described as 2x2 or 3x5 or2x3x4
- Terminology Factors/Main effects levels cells
- Can be experiment or quasi-experiment (whats

quasi?)

Factorial designs

- Example examine effect of sleep and caffeine on

reaction time study - IV-1 Sleep 2 hrs vs 8 hrs
- IV-2 Caffeine zero (water) vs 3 Cokes
- DV computer game reaction time
- Both IVs are between subjects variables

Factorial design

- Randomly select subject for each condition equal

cell sizes - 2 IVs test for 3 effects
- 2 Main effects (effects of sleep and caffeine)
- 1 interaction (interaction of sleep caffeine)
- Thus, 3 null hypotheses, 3 alternative

hypotheses, and 3 F tests

Factorial designs

- Main effect
- The effect of one factor (IV) collapsed across

levels of the other factor (IV) (ie, you ignore

the other IV) - Test one main effect for each factor (IV)
- Need to analyze statistics to check for

significance

Factorial design Main effect graphic

8 hrs

2 hrs

0 Cokes

3 Cokes

Did different amounts of sleep have an effect?

Mean of 2 hrs

Mean of 8 hrs

Factorial design Main effect graphic

8 hrs

2 hrs

0 Cokes

Mean of 0 Cokes

3 Cokes

Mean of 3 Cokes

Did different quantities of Coca-cola have an

effect?

Post hoc tests

- Post hoc means after the fact
- Tests that are performed after finding a

significant effect (sig F value) - ANOVAs tell you if there is a difference, but

they dont tell you where it is - Post hoc tests answer that question

Post hoc tests

- Would you need a follow up post hoc test after a

significant t-test? - Would you need to follow up a significant F-test

where there are 3 levels of the IV with a post

hoc test (say, for the sleep experiment you used

2, 4 or 8 hrs of sleep as three different levels

of the IV).

Post hoc tests

- So, need to use post hoc tests when
- You have 3 or more means
- You have a significant difference with a

general/omnibus test (F test) - Post hoc tests (pairwise comparisons) examine all

possible combinations of means

Post hoc tests

- Many types Tukeys HSD (honestly significant

difference), Bonferroni, Scheffes,

Kruskal-Wallis, Dunnetts - Each tests pairwise hypotheses
- Each calculates a particular statistic that

resembles a t-test (but it isnt that simple)

Post hoc tests

- Consider the 3 level IV, where you find a

significant F value - Next need to assess all possible pairwise

comparison - Mean 1 /diff mean 2
- Mean 1 /diff mean 3
- Mean 2 /diff mean 3
- Compute pairwise comparison for each

testascertain where the difference(s) are

statistically significant

Next time

- More factorials interactions

Previously -- 2nd Lecture

- Experimental design (Between vs Within)
- Compare 2 groups (paired vs unpaired)
- Control confounds (random vs systematic)
- ANOVAs (3 or more groups)
- Do overall F tests
- Pairwise compartions (post hoc tests)
- (like t tests done after significant F tells

you where the difference is that is, which two

means are significantly different from each other)

ANOVA when to use?

- One IV at least 3 conditions
- Between subjects One-way ANOVA
- Eg, effect of psychiatric diagnosis (depression,

panic, no disorder) on memory - Within subjects Repeated measures ANOVA
- Eg, experiment where subjects experience all

conditions example Stroop effect, colors/words - Two (or more) IVs
- Factorial ANOVA
- Effect of psychiatric diagnosis on Stroop effect

multiple IVs

Factorial designs

- Have 2 (or more) IVs
- Each IV is called a factor
- Each factor has at least 2 levels/conditions
- The design must be complete (sometimes called

completely crossed) each level of one IV must

occur with each level of the other IV(s) - Described as 2x2 or 3x5 or2x3x4
- Can be experiment or quasi-experiment

Factorial designs

- Example examine effect of sleep and caffeine on

reaction time study - IV-1 Sleep 2 hrs vs 8 hrs
- IV-2 Caffeine zero (water) vs 3 Cokes
- DV computer game number correct
- Both IVs are between subjects variables

Factorial design

- Randomly select subject for each condition equal

cell sizes - 2 IVs test for 3 effects
- 2 Main effects (effects of sleep and caffeine)
- 1 interaction (interaction of sleep caffeine)
- Thus, 3 null hypotheses, 3 alternative

hypotheses, and 3 F tests

Factorial designs

- Main effect
- The effect of one factor (IV) collapsed across

levels of the other factor (IV) (ie, you ignore

the other IV) - Test one main effect for each factor (IV)

Factorial designs

- Interactions are a way to qualify findings
- Revisit coke and sleep study

8 hrs

2 hrs

0 Cokes

3 Cokes

Factorial Design

- Calculate cell means and plot

2 hrs

8 hrs

0 Cokes

3 Cokes

Factorial Design

- Calculate cell means and plot

2 hrs

8 hrs

0 Cokes

3 Cokes

What does the parallel shift suggest to you?

Factorial Design

- Calculate cell means and plot

2 hrs

8 hrs

0 Cokes

3 Cokes

Factorial Design

- Calculate cell means and plot

2 hrs

8 hrs

0 Cokes

3 Cokes

Spreading interaction (aka ordinal) usually at

least one main effect

Factorial Design

- Calculate cell means and plot

2 hrs

8 hrs

0 Cokes

3 Cokes

But what about if we plotted this differently?

Factorial Design

- Calculate cell means and plot

2 hrs

8 hrs

0 Cokes

3 Cokes

Factorial Design

- Calculate cell means and plot

2 hrs

8 hrs

0 Cokes

3 Cokes

Cross-over interaction (aka disordinal) opposite

effects of IV1 at different levels of IV2

usually no main effects, but still can have

interaction (!)

Interactions

- So look at main effects
- Look for interactions
- Graphing things helps
- parallel lines no interactions
- crossing or close to crossing lines interaction
- But ultimately you need stats and p values to

make judgement about significance

Factorials

- So with 2 Independent variables you can have

multiple types of designs - Both IV are within
- Both IV are between
- One IV is within and one IV is between
- Also can have 1 true IV, and 1 quasi-IV

(sometimes called mixed or experi-corr) - Lets you consider unmanipulated factors like

subject variables (sex, age, diagnoses, etc.) - The more correlational something is, the more

external validity (and less internal validity)

you have

Within designs

- Simple 2 group comparison or ANOVA, doesnt

matter. - Limitations?
- How to fix?

Counter balancing schemes

- Reverse order of group presentation (AB vs BA)

but remember ANOVA usually have more conditions - Gets complicated quickly.
- Consider a within subjects design IV that has 4

levels, how many different orders of conditions?

Counter balancing schemes

- Reverse order of group presentation (AB vs BA)

but remember ANOVA usually have more conditions - Gets complicated quickly.
- Consider a within subjects design IV that has 4

levels, how many different orders of conditions? - ABCD

Counter balancing schemes

- Reverse order of group presentation (AB vs BA)

but remember ANOVA usually have more conditions - Gets complicated quickly.
- Consider a within subjects design IV that has 4

levels, how many different orders of conditions? - ABCD, ABDC,

Counter balancing schemes

- Reverse order of group presentation (AB vs BA)

but remember ANOVA usually have more conditions - Gets complicated quickly.
- Consider a within subjects design IV that has 4

levels, how many different orders of conditions? - ABCD, ABDC, ACBD,

Counter balancing schemes

- Reverse order of group presentation (AB vs BA)

but remember ANOVA usually have more conditions - Gets complicated quickly.
- Consider a within subjects design IV that has 4

levels, how many different orders of conditions? - ABCD, ABDC, ACBD, ACDB, ADBC, ADCB
- BCDA, BCAD, BDAC, BDCA, BACD, BADC
- CDAB, CDBA, CABD, CADB, CBAD, CBDA
- DABC, DACB, DBAC, DBCA, DCAB, DCBA

Counterbalancing

- Too confusingmore simple way is a Latin Square

design - Number conditions number of orders
- Each condition appears at each position only once

C

D

A

B

1st condition

2nd condition

3rd condition

4th condition

Fancy control groups

- Might be possible that the IV isnt all that is

involved in a particular effect - Other factors (timing, duration, pairing with

other events) may be critical - How to control? A Yoked control group
- The yoked control subject is paired directly with

an experimental subject (matching of experiences

in a way)

Summary

- How many IVs?
- 1 IV, 2 group/levels t-test
- Paired (subjects serve in both conditions) vs

Unpaired (different subjects in both conditions) - 1 IV, 2 groups/levels one-way ANOVA
- 2 IV two-way ANOVA (factorial design)
- Do subjects serve in only one condition?
- Yes between-subjects design (aka independent)
- No within-subjects design (aka repeated

measures, aor dependent) - Yes and No mixed between/within combo design
- Main effects (did the IV have an effect?)

interactions (was the effect dependent upon

different levels of the other IV?)