Stat 100

1
Stat 100

• Chapter 23, Try prob. 5-6, 9 12
• Read Chapter 24

2
Example

• N20 teens with high blood pressure take calcium
  supplements to reduce b.p.
• After two months, mean decrease for these 20
  teens was 6 points

3
Hypotheses for Significance Test

• null mean b.p. does not change with calcium
  supplements
• alternative mean b.p. decreases with calcium
  supplements
• These statements are for larger population
  represented by the sample of 20 teens

4
How the decision is made

• Determine the probability that observed decrease
  would be as large as 6 if calcium really has no
  effect
• This is called the p-value
• Well skip the details of finding this
• Rule is reject the null if the p-value is less
  than .05 (5).

5
P-value and Conclusion

• Suppose p-value in our example is found to be .03
  (3)
• This is below 5 guideline for significance.
• We can reject the null hypothesis
• Conclude that calcium causes drop in b.p.

6
Example

• Suppose you are on a panel considering the case
  of a student accused of cheating in a class.
• If convicted the student will fail the class.
• What do you think would be appropriate null and
  alternative hypotheses in this case?

7
Hypotheses

• Null Student did not cheat
• Alternative Student did cheat

8
Possible errors

• What are the two decision errors that might
  happen over many cases like this?
• Might convict somebody who did not cheat
• Might fail to convict somebody who did cheat

9
Which error would be worse?

• Probably convicting an innocent person.
• So rules of evidence might protect against this.
• Downside would be we might often let off guilty
  people (because its hard to convict)

10
Two Possible errors in significance testing

• Type 1 Error picking alternative when null is
  really true
• Type 2 Error picking null when alternative is
  really true

11
Example

• Researchers compare effectiveness of placebo and
  Zoloft for treating depression
• What are the null and alternative hypotheses?
• Null no difference in effectiveness
• Alternative Zoloft is more effective

12
Possible errors

• Type 1 picking the alternative when the null
  is true
• saying Zoloft is more effective when its not
• Type 2 picking the null when the alternative is
  true
• saying theres no difference when there really
  Zoloft is better

13
Most Common Cause of Type 2 Error

• Small sample size
• Small study may not be definitive about
  significance so theres a risk of not be able to
  reject the null
• Similar to not having enough evidence to convict
  somebody whos really guilty

14
The Effect of Sample Size

• The larger the study, the smaller the risk of
  Type 2 error
• Put another way - The larger the study, the
  better the chance of finding a true difference.

15
Power

• The term power defines the chance of not making
  a type 2 error
• That is, power chance of finding a true
  difference
• As sample size is increased, power is increased

16
The Problem with A Huge Sample

• A small, unimportant difference may be called
  statistically significant
• Headline Spring Birth Provides Height
  Advantage
• Data N400,000 Austrian military recruits
• Observed diff in heights Spring versus rest1/4
  of inch.
• The sample was so big, it was possible to say
  there was a difference

17
Meta-Analysis Approach

• Combine results of different researchers' studies
  of the same problem
• Example - recent news item about zinc's affect on
  cold symptoms
• combined five different studies of zinc's affect
  on cold duration
• Concluded taking zinc may have some benefit

18
Why meta-analysis ?

• Basically increases sample size and generates
  more power
• Good to combine all known information
• Smaller studies might all be inconclusive, but
  the combined effect could be significant

19
Difficulties

• Studies may not be comparable
• different populations
• different methods
• Nonsignificant studies may not have been
  published (called the file drawer problem)
• Biased analyst may give more weight to studies
  proving his or her point