Multiple%20Comparison%20Procedures

1
Multiple Comparison Procedures

• Comfort Ratings of 13 Fabric Types
• A.V. Cardello, C. Winterhalter, and H.G. Schultz
  (2003). "Predicting the Handle and Comfort of
  Military Clothing Fabrics from Sensory and
  Instrumental Data Development and Application of
  New Psychophysical Methods," Textile Research
  Journal, Vol. 73, pp. 221-237.

2
Treatments
Means and standard deviations of 45 comfort
ratings for 13 military fabrics. Fabric types
10R - 50/50 Nylon/combed cotton, ripstop poplin
weave 11A - 50/50 Nylon/Polyester, oxford weave
(Australian) 12T - 50/50 Nylon/cotton, twill
weave 13P - 92/5/3 Nomex/Kevlar/P140, plain
weave 14N - 100 Cotton (former flame retardant
treated) 15B - 77/23 Cotton sheath/synthetic
core, twill (UK) 16C - 100 combed cotton,
ripstop poplin (former hot weather BDU) 17C -
65/35 Wool/Polyester, plain weave
(Canada-unlaundered) 18L - 65/35 Wool/Polyester,
plain weave (Canada-laundered) 19N - 92/5/3
Nomex/Kevlar/P140, oxford weave 20J - Carded
cotton sheath/nylon core, plain weave (Canada)
124 - 100 Pima cotton ripstop poplin
(experimental) 176 - 50/50 Nylon carded cotton
ripstop poplin weave
3
Multiple Comparisons

• Individual Combined Null Hypotheses (H0 ? H01
  H0k)
• Comparisonwise Error Rate ? Pr(Reject H0iH0i
  True)
• Experimentwise Error Rate ? Pr(Reject any H0iAll
  H0i True)
• False Discovery Rate ? E( False Rejects/Total
  Rejections)
• Strong Familywise Error Rate ? Pr(Any False
  Discoveries)
• Simultaneous Confidence Intervals ? Pr(All
  Correct)1-e
• Multiple Comparison Procedures control Type 1
  Error Rate other than per comparison

4
Data and Analysis of Variance
5
Bonferroni Based Methods

• Construct P-values for all k test statistics
• Order P-values from smallest p(1) p(k)
• Bonferroni Reject H0(i) if p(i) e/k
• Holm (Controls Strong FWER) Reject H0(i) if p(j)
  e/(k-j1) ? j I
• False Discovery Rate Reject H0(i) if p(j) je/k
  for some j i
• (Assumes independent tests, not the case for
  this example)
• Example Comparing all k13(12)/278 pairs of
  fabrics

6
Fabric Example j1,,26
7
Fabric Example j27,,52
8
Fabric Example j53,,78
9
Scheffes Method for All Contrasts

• Can be used for any number of contrasts, even
  those suggested by data. Conservative (Wide CIs,
  Low Power)

10
Example Scheffes Method All Pairwise
Tests/CIs
11
Tukeys Method for All Pairwise Comparisons

• Makes use of the Studentized Range Distribution
• Pr(max(Y1,,Yn)-min(Y1,,Yn))/S q(a,n,n) a
• Y1,,Yn ? S n ? degrees of freedom for S

12
Tukeys Method for All Pairwise Comparisons
13
Bonferronis Method for All Pairwise Comparisons

• Adjusts type I error rate for each test to e/(
  of tests)
• Increases Confidence levels of CIs to (1-(e/(
  of CIs)))

14
Bonferronis Method for All Pairwise Comparisons
15
SNK Method for All Pairwise Comparisons

• Controls False Discovery Rate at e
• Uses Different Critical Values for different
  ranges of means

16
SNK Method for All Pairwise Comparisons
17
Fishers Protected LSD for All Pairwise
Comparisons

• Controls Experimentwise Error Rate at e
• Only Conducted if F-test is significant (P-value
  e)

18
Fishers Protected LSD for All Pairwise
Comparisons
19
Multiple Comparisons with Best Treatment/Control

• Prsubset of treatments contains the best 1- e

20
Multiple Comparisons with Best Treatment/Control
Treatments 13P, 15B, and 11A all lie within 16.22
of the highest mean