Stat 112: Lecture 22 Notes

1
Stat 112 Lecture 22 Notes

• Chapter 9.1 One-way Analysis of Variance.
• Chapter 9.3 Two-way Analysis of Variance
• Homework 6 is due on Friday.

2
Errors in Hypothesis Testing
State of World State of World
Null Hypothesis True Alternative Hypothesis True
Decision Based on Data Accept Null Hypothesis Correct Decision Type II error
Decision Based on Data Reject Null Hypothesis Type I errror Correct Decision
When we do one hypothesis test and reject null
hypothesis if p-value lt0.05, then the probability
of making a Type I error when the null hypothesis
is true is 0.05. We protect against falsely
rejecting a null hypothesis by making probability
of Type I error small.
3
Multiple Comparisons Problem

• Compound uncertainty When doing more than one
  test, there is an increased chance of a Type I
  error
• If we do multiple hypothesis tests and use the
  rule of rejecting the null hypothesis in each
  test if the p-value is lt0.05, then if all the
  null hypotheses are true, the probability of
  falsely rejecting at least one null hypothesis is
  gt0.05.

4
Individual vs. Familywise Error Rate

• When several tests are considered simultaneously,
  they constitute a family of tests.
• Individual Type I error rate Probability for a
  single test that the null hypothesis will be
  rejected assuming that the null hypothesis is
  true.
• Familywise Type I error rate Probability for a
  family of test that at least one null hypothesis
  will be rejected assuming that all of the null
  hypotheses are true.
• When we consider a family of tests, we want to
  make the familywise error rate small, say 0.05,
  to protect against falsely rejecting a null
  hypothesis.

5
Bonferroni Method

• General method for doing multiple comparisons for
  any family of k tests.
• Denote familywise type I error rate we want by
  p, say p0.05.
• Compute p-values for each individual test --
• Reject null hypothesis for ith test if
• Guarantees that familywise type I error rate is
  at most p.
• Why Bonferroni works If we do k tests and all
  null hypotheses are true , then using Bonferroni
  with p0.05, we have probability 0.05/k to make
  a Type I error for each test and expect to make
  k(0.05/k)0.05 errors in total.

6
Bonferroni on Milgrams Data
Output obtained from Fit Y by X, Compare Means,
Each Pair Students t
7
Bonferroni on Milgrams Data Continued
8
Tukeys HSD

• Tukeys HSD is a method that is specifically
  designed to control the familywise type I error
  rate (at 0.05) for analysis of variance when we
  are interested in comparing all pairs of groups.
• JMP Instructions After Fit Y by X, click the red
  triangle next to the X variable and click LSMeans
  Tukey HSD.

9
Tukeys HSD for Milgrams Data
10
Assumptions in one-way ANOVA

• Assumptions needed for validity of one-way
  analysis of variance p-values and CIs
• Linearity automatically satisfied.
• Constant variance Spread within each group is
  the same.
• Normality Distribution within each group is
  normally distributed.
• Independence Sample consists of independent
  observations.

11
Rule of thumb for checking constant variance

• Constant variance Look at standard deviation of
  different groups by using Fit Y by X and clicking
  Means and Std Dev.
• Rule of Thumb Check whether (highest group
  standard deviation/lowest group standard
  deviation) is greater than 2. If greater than 2,
  then constant variance is not reasonable and
  transformation should be considered.. If less
  than 2, then constant variance is reasonable.
• (Highest group standard deviation/lowest group
  standard deviation) (131.874/63.640)2.07.
  Thus, constant variance is not reasonable for
  Milgrams data.

12
Transformations to correct for nonconstant
variance

• If standard deviation is highest for high groups
  with high means, try transforming Y to log Y or
  . If standard deviation is highest for groups
  with low means, try transforming Y to Y2.
• SD is particularly low for group with highest
  mean. Try transforming to Y2. To make the
  transformation, right click in new column, click
  New Column and then right click again in the
  created column and click Formula and enter the
  appropriate formula for the transformation.

13
Transformation of Milgrams data to Squared
Voltage Level

• Check of constant variance for transformed data
  (Highest group standard deviation/lowest group
  standard deviation) 1.63. Constant variance
  assumption is reasonable for voltage squared.
• Analysis of variance tests are approximately
  valid for voltage squared data reanalyzed data
  using voltage squared.

14
Analysis using Voltage Squared
Strong evidence that the group mean voltage
squared levels are not all the same.
Strong evidence that remote has higher mean
voltage squared level than proximity and
touch-proximity and that voice-feedback has
higher mean voltage squared level than
touch-proximity, taking into account the multiple
comparisons.
15
Rule of Thumb for Checking Normality in ANOVA

• The normality assumption for ANOVA is that the
  distribution in each group is normal. Can be
  checked by looking at the boxplot, histogram and
  normal quantile plot for each group.
• If there are more than 30 observations in each
  group, then the normality assumption is not
  important ANOVA p-values and CIs will still be
  approximately valid even for nonnormal data if
  there are more than 30 observations in each
  group.
• If there are less than 30 observations per group,
  then we can check normality by clicking Analyze,
  Distribution and then putting the Y variable in
  the Y, Columns box and the categorical variable
  denoting the group in the By box. We can then
  create normal quantile plots for each group and
  check that for each group, the points in the
  normal quantile plot are in the confidence bands.
  If there is nonnormality, we can try to use a
  transformation such as log Y and see if the
  transformed data is approximately normally
  distributed in each group.

16
One way Analysis of Variance Steps in Analysis

1. Check assumptions (constant variance, normality,
   independence). If constant variance is violated,
   try transformations.
2. Use the effect test (commonly called the F-test)
   to test whether all group means are the same.
3. If it is found that at least two group means
   differ from the effect test, use Tukeys HSD
   procedure to investigate which groups are
   different, taking into account the fact multiple
   comparisons are being done.

17
Analysis of Variance Terminology

• The criterion (criteria) by which we classify the
  groups in analysis of variance is called a
  factor. In one-way analysis of variance, we have
  one factor.
• The possible values of the factor are levels.
• Milgrams study Factor is experimental condition
  with levels remote, voice-feedback, proximity and
  touch-proximity.
• Two-way analysis of variance Groups are
  classified by two factors.

18
Two-way Analysis of Variance Examples

• Milgrams study In thinking about the Obedience
  to Authority study, many people have thought that
  women would react differently than men. Two-way
  analysis of variance setup in which the two
  factors are experimental condition (levels
  remote, voice-feedback, proximity,
  touch-proximity) and sex (levels male, female).
• Package Design Experiment Several new types of
  cereal packages were designed. Two colors and
  two styles of lettering were considering. Each
  combination of lettering/color was used to
  produce a package, and each of these combinations
  was test marketed in 12 comparable stores and
  sales in the stores were recorded.. Two-way
  analysis of variance in which two factors are
  color (levels red, green) and lettering (levels
  block, script).
• Goal of two-way analysis of variance Find out
  how the mean response in a group depends on the
  levels of both factors and find the best
  combination.

19
Two-way Analysis of Variance

• The mean of the group with the ith level of
  factor 1 and the jth level of factor 2 is denoted
  , e.g., in package-design experiment, the
  four group means are
• As with one-way analysis of variance, two-way
  analysis of variance can be seen as a a special
  case of multiple regression. For two-way
  analysis of variance, we have two categorical
  explanatory variables for the two factors and
  also include an interaction between the factors.
  

20
Estimated Mean for Red Block group
144.929.83-11.174.5 148.08 Estimated Mean for
Red Script group 144.929.8311.17-4.5 161.42
21
The LS Means Plots show how the means of
the groups vary as the levels of the factors
vary. For the top plot for color, green refers to
the mean of the two green groups (green block and
green script) and red refers to the mean of the
two red groups (red block and red script).
Similarly for the second plot for TypeStyle,
block refers to the mean of the two block groups
(red block and green block). The third plot for
TypeStyleColor shows the mean of all four groups.
22
Two-way ANOVA in JMP

• Use Analyze, Fit Model with a categorical
  variable for the first factor, a categorical
  variable for the second factor and an interaction
  variable that crosses the first factor and the
  second factor.
• The LS Means Plots are produced by going to the
  output in JMP for each variable that is to the
  right of the main output, clicking the red
  triangle next to each variable (for package
  design, the vairables are Color, TypeStyle,
  TypestyleColor) and clicking LS Means Plot.

23
Interaction in Two-Way ANOVA

• Interaction between two factors The impact of
  one factor on the response depends on the level
  of the other factor.
• For package design experiment, there would be an
  interaction between color and typestyle if the
  impact of color on sales depended on the level of
  typestyle.
• Formally, there is an interaction if
• LS Means Plot suggests there is not much
  interaction. Impact of changing color from red
  to green on mean sales is about the same when the
  typestyle is block as when the typestyle is
  script.

24
Effect Test for Interaction

• A formal test of the null hypothesis that there
  is no interaction,
• for all levels i,j,i,j of factors 1 and 2,
  versus the alternative hypothesis that there is
  an interaction is given by the Effect Test for
  the interaction variable (here TypestyleColor).
• p-value for Effect Test 0.4191. No evidence of
  an interaction.

25
Implications of No Interaction

• When there is no interaction, the two factors can
  be looked in isolation, one at a time.
• When there is no interaction, best group is
  determined by finding best level of factor 1 and
  best level of factor 2 separately.
• For package design experiment, suppose there are
  two separate groups one with an expertise in
  lettering and the other with expertise in
  coloring. If there is no interaction, groups can
  work independently to decide best letter and
  color. If there is an interaction, groups need
  to get together to decide on best combination of
  letter and color.

26
Model when There is No Interaction

• When there is no evidence of an interaction, we
  can drop the interaction term from the model for
  parsimony and more accurate estimates

Mean for red block group 144.929.83-11.17143.5
8 Mean for red script group 144.929.8311.1716
5.92
27
Tests for Main Effects When There is No
Interaction

• Effect test for color Tests null hypothesis that
  group mean does not depend on color versus
  alternative that group mean is different for at
  least two levels of color. p-value 0.0804,
  moderate but not strong evidence that group mean
  depends on color.
• Effect test for TypeStyle Tests null hypothesis
  that group mean does not depend on TypeStyle
  versus alternative that group mean is different
  for at least two levels of TypeStyle. p-value
  0.0481, evidence that group mean depends on
  TypeStyle.
• These are called tests for main effects. These
  tests only make sense when there is no
  interaction.

28
Example with an Interaction

• Should the clerical employees of a large
  insurance company be switched to a four-day week,
  allowed to use flextime schedules or kept to the
  usual 9-to-5 workday?
• The data set flextime.JMP contains percentage
  efficiency gains over a four week trial period
  for employees grouped by two factors Department
  (Claims, Data Processing, Investment) and
  Condition (Flextime, Four-day week, Regular
  Hours).

29
Which schedule is best appears to differ by
department. Four day is best for investment
employees, but worst for data processing
employees.
30
Which Combinations Works Best?

• For which pairs of groups is there strong
  evidence that the groups have different means
  is there strong evidence that one combination
  works best?
• We combine the two factors into one factor
  (Combination) and use Tukeys HSD, to compare
  groups pairwise, adjusting for multiple
  comparisons.

31
(No Transcript)
32
Checking Assumptions

• As with one-way ANOVA, two-way ANOVA is a special
  case of multiple regression and relies on the
  assumptions
• Linearity Automatically satisfied
• Constant variance Spread within groups is the
  same for all groups.
• Normality Distribution within each group is
  normal.
• To check assumptions, combine two factors into
  one factor (Combination) and check assumptions as
  in one-way ANOVA.

33
Checking Assumptions

• Check for constant variance (Largest standard
  deviation of group/Smallest standard deviation of
  group) (44.85/33.51) lt2. Constant variance OK.
• Check for normality Look at normal quantile
  plots for each combination (not shown). For all
  normal quantile plots, the points fall within the
  95 confidence bands. Normality assumption OK.

34
Two way Analysis of Variance Steps in Analysis

1. Check assumptions (constant variance, normality,
   independence). If constant variance is violated,
   try transformations.
2. Use the effect test (commonly called the F-test)
   to test whether there is an interaction.
3. If there is no interaction, use the main effect
   tests to whether each factor has an effect.
   Compare individual levels of a factor by using
   t-tests with Bonferroni correction for the number
   of comparisons being made.
4. If there is an interaction, use the interaction
   plot to visualize the interaction. Create
   combination of the factors and use Tukeys HSD
   procedure to investigate which groups are
   different, taking into account the fact multiple
   comparisons are being done.