Experimental designs and Analysis of Variance Factorial ANOVA

1
Experimental designsandAnalysis of
VarianceFactorial ANOVA
2
Overview

• Factorial ANOVA
• Random factors and generalization
• Nested designs

3
Types of experimental designs

• Randomized-blocks design
• Second factor defines blocks error control
• Within-subject design
• Each case is measured once at each level
• Differences due to treatments are tested within
  subjects
• Individual differences between subjects are
  removed, thereby reducing the error variance
• ? Blocking on subjects
• Special case of blocking design only 1
  observation in each cell

4
Example Stevens

• The effect of four drugs on reaction time
• Written as a block-design block on subjects to
  remove within-subjects variability from error
  variance

5
Example Stevens
Treated as one-way randomized-group design
Blocking on subjects two-way randomized-group
design
Blocking variable random factor?
6
Random factors

• Repeated measures
• Blocking on subjects
• Remove variance within subjects (blocks) from
  error variance to increase precision
  within-subjects design
• Use subjects as a factor
• Subjects are random sample from a population of
  subjects not a fixed level of some treatment
• ? Random factor
• The treatment levels (subjects) are a random
  sample from a larger population of treatment
  levels (subjects)
• Generalization

7
Generalization

• Make statements that apply to a larger collection
  of individuals and instances than it is possible
  to explicitly study
• Statistical generalization sampling form a
  larger population to make inferences about the
  population
• E.g. F test in ANOVA
• Extrastatistical generalization generalization
  to larger population than sample was drawn from
• E.g. treat sample of students as representative
  for all people

8
Example Keppel Wickens

• Three-factor design
• How does the organization of a story affects how
  people remember it?
• Factor A organization (yes/no)
• Factor B version of story (three stories)
• Factor C lab sections of class (five sections)
• Generalization to
• population of subjects (students)
• population of stories, not just the three
  picked
• population of sections, not just the five
  picked
• irrelevant factor? (blocking)

9
Example Keppel Wickens

• Study sample of subjects
• Generalize to larger population use F test
  (statistical generalization) and extrastatistical
  generalization
• Extend these ideas to factors stories and
  sections
• Analysis (fixed factors)
• generalization only
• extrastatistical
• F test only statistical
• generalization to larger
• population of subjects

10
Fixed and random factors

• Fixed factor
• factor whose levels have been chosen rationally
• The levels of are the only ones of possible
  interest
• Factor A (organization of story) is the point
  of study
• Random factor
• factor whose levels are chosen unsystematically
  by random sampling from a larger population
• Allows statistical generalization
• Subject factor individual levels (subjects) no
  particular meaning, only representatives of
  larger population

Example
Example
11
Fixed factors

• Fixed effects The levels of all factors included
  are the only ones of possible interest
• Goal Estimate treatment (group) means and
  differences between them
• Data Analysis ANOVA
• Model (one-way) with
• Test
• H0 µ1 µ2 µa vs. Ha not all µs equal
• H0 a1 a2 0 vs. Ha not all as equal
  to 0

12
Experimental error

• Differences between means due to
• Treatment effect
• Chance (motivation, attention, measurement, etc)
• Estimate extent to which differences are due to
  experimental error ? evaluate hypothesis of equal
  group means
• Variability within treatment groups of subjects
  provides estimate
• If null hypothesis is true and subjects are
  randomly assigned to treatments, than variability
  between treatment groups also provides estimate
• If null hypothesis is false there is a treatment
  effect (systematic differences), than variability
  between treatment groups reflects treatment
  effects and experimental error

13
Logic of hypothesis testing

• Experimental error reflected in
• differences (variability) among subjects given
  same treatment
• differences (variability) among groups of
  subjects given different treatment
• Inspect the ratio of variabilities
• If H0 is true
• If H0 is false

14
Fixed factors

• Assumptions
• ais are fixed and
• errors eij are normally and independently
  distributed, with zero mean and SD s (also
  denoted se)
• Treatment effect
• F test

15
Random factors

• Random effects The levels of the factors
  included are a random sample from a population of
  levels
• Goal Estimate the variation among the treatment
  (group) means
• New source of random variability must be
  accommodated
• F test

Error term for treatment effect includes
variability of every random effect that
influences the treatment mean square
16
Sources of variance

• Identifying sources of variability (Table 11.2)
• 1. List each factor as source
• 2. Examine each combination of factors
  completely crossed ? include interaction as
  source
• 3. When effect is repeated, with different
  instances, at every level of another factor ?
  include factor as source after slash (/)
• 1. Factor A, factor B, and subjects S
• 2. A and B completely crossed A, B, A?B, and S
• 3. S is repeated in every level of A and B A, B,
  A?B, and S/AB

Example
17
Assigning error terms

• Assigning error terms (Table 24.3)
• 1. List sources of variability
• 2. List influences affecting each source source
  itself, crossing with random factors, any source
  in which random effects are nested within
  original source
• 3. Denominator F source that includes all
  influences except effect that is to be tested
• (two factors B random)
• 0. Factors A fixed
• B and S/AB random
• 1. Sources of variability A, B, A?B, and S/AB

Example
18
Assigning error terms

• Assigning error terms (Table 24.3)
• 1. List sources of variability
• 2. List influences affecting each source source
  itself, crossing with random factors, any source
  in which random effects are nested within
  original source
• 3. Denominator F source that includes all
  influences except effect that is to be tested
• (two factors B random)
• 2. Influence on A A, A?B, and S/AB
• Influence on B B and S/AB
• Influence on A?B A?B and S/AB
• Influence on S/AB S/AB

Example
19
Assigning error terms

• Assigning error terms (Table 24.3)
• 1. List sources of variance
• 2. List influences affecting source (source
  itself, crossing with random factors, any source
  in which random effects are nested within
  original source
• 3. Denominator F source that includes all
  influences except effect to be tested itself
• (two factors B random)
• Influence on A A (effect) and A?B, S/AB
  (error)
• 3. Thus error for A includes A?B and S/AB
• MS of A?B is influenced by A?B and S/AB
• Error MSA?B

Example
20
Random factors

• Random effects the levels of the factors
  included are a random sample from a population of
  levels
• Goal estimate the variation among the treatment
  (group) means
• Example dose of drugs, subjects, etc
• Model

21
Random effects models

• Assumptions
• ais are normally and independently distributed
  with zero mean and SD sa
• errors eij are normally and independently
  distributed, with zero mean and SD se
• ai and eij are independent
• Two sources of variation
• 1. sa variation due to treatment effects
  (explained)
• 2. se variation due to error/noise/etc.
  (unexplained)

22
Random effects models

• Calculations for the ANOVA are the same under
  both models, however inferences differ
• Fixed effects model tests
• H0 µ1 µ2 µa vs. Ha not all µs
  equal
• Random effects model tests
• H0 sa2 0 vs. Ha sa2 gt 0
• i.e., the hypothesis that there is no variation
  among the treatment (group) means against the
  alternative that the means vary

23
ANOVA models
Fixed
Random
24
Example Keppel Wickens
25
Example Keppel Wickens
Influence on A A, A?B, A?C, A?B?C, and S/ABC
Error for A includes A?B, A?C, A?B?C, and S/ABC
26
Random effects

• Random factors change the error term in the F
  test
• Results in larger error terms than without the
  random factor
• Results in fewer dfs for error term
• Pay a price for statistical generalization
• reduced power
• Powerful tests need random factors with as many
  levels as possible
• Powerful tests require adequate samples of both
  subjects and levels of the random factor

27
Nested designs

• How are the levels of multiple factors combined?
• Crossing all levels of one variable completely
  cross (occur in combination with) all levels of
  another variable
• ? Factorial designs
• Nesting different levels of a factor occur at
  each combination of other factors
• Example of an nested design A is nested within
  levels of B

28
Nested designs

• These designs are also called hierarchical
  designs
• Factor A is nested with levels of factor B
• This means that
• For each level of B there are several
  (different) levels of A (outer or nesting factor)
• For each level of A there is only one level of B
  (inner or nested factor)

Example
29
Nested designs

• Notation use slash /
• Crossed design subjects nested in cells S/AB
• Nested design inner factor nested in outer
  factor B/A
• Nested design subjects nested in cells S/B/A
• Nested factor is typically a random factor
• Used to increase generality
• Not chosen systematically, and not of primary
  interest
• Analysis of design (ANOVA)
• Determine sources of variability
• Find the Sums of Squares, dfs, and Mean Squares
• Determine proper error term for the F test

30
Nested designs

• Not all effects can be tested
• Not every Interaction can be determined
• Main effect of nested factor cannot be assessed
• Simple main analysis
• Test main effects B within A
• Main effect of non-nested factor can be tested
  specification of right error variance
• Nested designs are designs with missing cells