Multiple comparisons for one-way ANOVA (Chapter 15.7)

1
Lecture 13

• Multiple comparisons for one-way ANOVA (Chapter
  15.7)
• Analysis of Variance Experimental Designs
  (Chapter 15.3)

2
15.7 Multiple Comparisons

• When the null hypothesis is rejected, it may be
  desirable to find which mean(s) is (are)
  different, and how they rank.
• Three statistical inference procedures, geared at
  doing this, are presented
• Fishers least significant difference (LSD)
  method
• Bonferroni adjustment to Fishers LSD
• Tukeys multiple comparison method

3
Example 15.1

• Sample means
  
• 
  
  
• 
  
• Does the quality strategy have a higher mean
  sales than the other two strategies?
• Do the quality and price strategies have a
  higher mean than the convenience strategy?
• Does the price strategy have a smaller mean
  sales than quality but a higher mean than
  convenience?
• Pairwise comparison Are two population means
  different?
• 
  

4
Fisher Least Significant Different (LSD) Method

• This method builds on the equal variances t-test
  of the difference between two means.
• The test statistic is improved by using MSE
  rather than sp2.
• We conclude that mi and mj differ (at a
  significance level if gt LSD, where

5
Multiple Comparisons Problem

• A hypothetical study of the effect of birth
  control pills is done.
• Two groups of women (one taking birth controls,
  the other not) are followed and 20 variables are
  recorded for each subject such as blood pressure,
  psychological and medical problems.
• After the study, two-sample t-tests are performed
  for each variable and it is found that one null
  hypothesis is rejected. Women taking birth pills
  have higher incidences of depression at the 5
  significance level (the p-value equals .02).
• Does this provide strong evidence that women
  taking birth control pills are more likely to be
  depressed?

6
Experimentwise Type I error rate (aE) versus
Comparisonwise Type I error rate

• The comparisonwise Type I error rate is the
  probability of committing a Type I error for one
  pairwise comparison.
• The experimentwise Type I error rate ( ) is
  the probability of committing at least one Type I
  error when C tests are done and all null
  hypotheses are true.
• For a one-way ANOVA, there are k(k-1)/2 pairwise
  comparisons (knumber of populations)
• If the comparisons are not planned in advance and
  chosen after looking at the data, the
  experimentwise Type I error rate is the more
  appropriate one to look at.

7
Experimentwise Error Rate

• The expected number of Type I errors if C tests
  are done at significance level each is
• If C independent tests are done,
• aE 1-(1 a)C
• The Bonferroni adjustment determines the required
  Type I error probability per test (a) , to secure
  a pre-determined overall aE.

8
Bonferroni Adjustment

• Suppose we carry out C tests at significance
  level
• If the null hypothesis for each test is true, the
  probability that we will falsely reject at least
  one hypothesis is at most
• Thus, if we carry out C tests at significance
  level , the experimentwise Type I error
  rate is at most

9
Bonferroni Adjustment for ANOVA

• The procedure
• Compute the number of pairwise comparisons
  (C)all Ck(k-1)/2, where k is the number of
  populations.
• Set a aE/C, where aE is the true probability of
  making at least one Type I error (called
  experimentwise Type I error).
• We conclude that mi and mj differ at a/C
  significance level (experimentwise error rate at
  most ) if

10
Fisher and Bonferroni Methods

• Example 15.1 - continued
• Rank the effectiveness of the marketing
  strategies(based on mean weekly sales).
• Use the Fishers method, and the Bonferroni
  adjustment method
• Solution (the Fishers method)
• The sample mean sales were 577.55, 653.0, 608.65.
• Then,

11
Fisher and Bonferroni Methods

• Solution (the Bonferroni adjustment)
• We calculate Ck(k-1)/2 to be 3(2)/2 3.
• We set a .05/3 .0167, thus t.0167/2, 60-3
  2.467 (Excel).

Again, the significant difference is between m1
and m2.
12
Tukey Multiple Comparisons

• The test procedure
• Assumes equal number of obs. per populations.
• Find a critical number w as follows

k the number of populations n degrees of
freedom n - k ng number of observations per
population a significance level qa(k,n) a
critical value obtained from the studentized
range table (app. B17/18)
13
Tukey Multiple Comparisons

• Select a pair of means. Calculate the difference
  between the larger and the smaller mean.

• If there is
  sufficient evidence to conclude that mmax gt mmin
  .

• Repeat this procedure for each pair of samples.
  Rank the means if possible.

14
Tukey Multiple Comparisons

• Example 15.1 - continued We had three populations
  (three marketing strategies).K 3,
• Sample sizes were equal. n1 n2 n3 20,n
  n-k 60-3 57,MSE 8894.

Take q.05(3,60) from the table 3.40.
Population Sales - City 1 Sales - City 2 Sales -
City 3
Mean 577.55 653 698.65
City 1 vs. City 2 653 - 577.55 75.45 City 1
vs. City 3 608.65 - 577.55 31.1 City 2 vs.
City 3 653 - 608.65 44.35
15
15.3 Analysis of Variance Experimental Designs

• Several elements may distinguish between one
  experimental design and another
• The number of factors (1-way, 2-way, 3-way,
  ANOVA).
• The number of factor levels.
• Independent samples vs. randomized blocks
• Fixed vs. random effects
• These concepts will be explained in this lecture.

16
Number of factors, levels

• Example 15.1, modified
• Methods of marketing price, convenience,
  quality gt first factor with 3 levels
• Medium advertise on TV vs. in newspapers
  gt second factor with 2 levels
• This is a factorial experiment with two crossed
  factors if all 6 possibilities are sampled or
  experimented with.
• It will be analyzed with a 2-way ANOVA. (The
  book got this term wrong.)

17
One - way ANOVA Single factor
Two - way ANOVA Two factors
Response
Response
Treatment 3 (level 1)
Treatment 2 (level 2)
Treatment 1 (level 3)
Level 3
Level2
Factor A
Level 1
Level 1
Level2
Factor B
18
Randomized blocks

• This is something between 1-way and 2-way ANOVA
  a generalization of matched pairs when there are
  more than 2 levels.
• Groups of matched observations are collected in
  blocks, in order to remove the effects of
  unwanted variability. gt We improve the chances
  of detecting the variability of interest.
• Blocks are like a second factor gt 2-way
  ANOVA is used for analysis
• Ideally, assignment to levels within blocks is
  randomized, to permit causal inference.

19
Randomized blocks (cont.)

• Example expand 13.03
• Starting salaries of marketing and finance MBAs
  add accounting MBAs to the investigation.
• If 3 independent samples of each specialty are
  collected (samples possibly of different sizes),
  we have a 1-way ANOVA situation with 3 levels.
• If GPA brackets are formed, and if one samples
  3 MBAs per bracket, one from each
  specialty, then one has a blocked design.
  (Note the 3 samples will be of equal size due to
  blocking.)
• Randomization is not possible here one cant
  assign each student to a specialty, and one
  doesnt know the GPA beforehand for matching.
  gt No causal
  inference.

20
Models of fixed and random effects

• Fixed effects
• If all possible levels of a factor are included
  in our analysis or the levels are chosen in a
  nonrandom way, we have a fixed effect ANOVA.
• The conclusion of a fixed effect ANOVA applies
  only to the levels studied.
• Random effects
• If the levels included in our analysis represent
  a random sample of all the possible levels, we
  have a random-effect ANOVA.
• The conclusion of the random-effect ANOVA applies
  to all the levels (not only those studied).

21
Models of fixed and random effects (cont.)

• Fixed and random effects - examples
• Fixed effects - The advertisement Example
  (15.1) All the levels of the marketing
  strategies considered were included. Inferences
  dont apply to other possible strategies such as
  emphasizing nutritional value.
• Random effects - To determine if there is a
  difference in the production rate of 50 machines
  in a large factory, four machines are randomly
  selected and the number of units each produces
  per day for 10 days is recorded.

22
15.4 Randomized Blocks Analysis of Variance

• The purpose of designing a randomized block
  experiment is to reduce the within-treatments
  variation, thus increasing the relative amount of
  between treatment variation.
• This helps in detecting differences between the
  treatment means more easily.

23
Examples of Randomized Block Designs
Factor Response Units Block
Varieties of Corn Yield Plots of Land Adjoining plots
Blood pressure Drugs Hypertension Patient Same age, sex, overall condition
Management style Worker productivity Amount produced by worker Shifts
24
Randomized Blocks
Block all the observations with some commonality
across treatments
Treatment 4
Treatment 3
Treatment 2
Treatment 1
Block 1
Block3
Block2
25
Randomized Blocks
Block all the observations with some commonality
across treatments
26
Partitioning the total variability

• The sum of square total is partitioned into three
  sources of variation
• Treatments
• Blocks
• Within samples (Error)

Recall. For the independent
samples design we have SS(Total) SST SSE
SS(Total) SST SSB SSE
27
Sums of Squares Decomposition

• observation in ith block, jth treatment
• mean of ith block
• mean of jth treatment

28
Calculating the sums of squares

• Formulas for the calculation of the sums of
  squares

29
Calculating the sums of squares

• Formulas for the calculation of the sums of
  squares

30
Mean Squares

• To perform hypothesis tests for treatments and
  blocks we need
• Mean square for treatments
• Mean square for blocks
• Mean square for error

31
Test statistics for the randomized block design
ANOVA
df-T k-1 df-B b-1
df-E n-k-b1
32
The F test rejection regions

• Testing the mean responses for treatments
• F gt Fa,k-1,n-k-b1
• Testing the mean response for blocks
• Fgt Fa,b-1,n-k-b1

33
Randomized Blocks ANOVA - Example

• Example 15.2
• Are there differences in the effectiveness of
  cholesterol reduction drugs?
• To answer this question the following experiment
  was organized
• 25 groups of men with high cholesterol were
  matched by age and weight. Each group consisted
  of 4 men.
• Each person in a group received a different drug.
• The cholesterol level reduction in two months was
  recorded.
• Can we infer from the data in Xm15-02 that there
  are differences in mean cholesterol reduction
  among the four drugs?

34
Randomized Blocks ANOVA - Example

• Solution
• Each drug can be considered a treatment.
• Each 4 records (per group) can be blocked,
  because they are matched by age and weight.
• This procedure eliminates the variability in
  cholesterol reduction related to different
  combinations of age and weight.
• This helps detect differences in the mean
  cholesterol reduction attributed to the different
  drugs.

35
Randomized Blocks ANOVA - Example
Blocks
Treatments
b-1
MST / MSE
MSB / MSE
K-1