Hypothesis testing

1
Hypothesis testing

• Week 10 Lecture 2

2
Functions of inferential statistics

• To estimate a population parameter from a random
  sample
• If you draw two random samples of the same size
  from a population, it is very likely that the two
  means you get will be different
• Standard error of the mean is the standard
  deviation of all sample means
• To test hypotheses
• data are from sample
• The probability that our result is due to chance
  alone
• If you get data from whole population, no
  hypothesis testing is required.

3
Two types of hypotheses

• Research hypothesis (alternative hypothesis)
• A prediction of the relation between variables
• Null hypothesis (H0)
• There will be no relation between the variables
• Any relation observed are due to chance alone
• Examples
• The higher the annual income, the greater the
  Internet usage
• There is no relation between annual income and
  Internet usage
• It is the null hypothesis that is tested
• We look at the probability that our result is due
  to chance alone.

4
Result of hypothesis testing

• Reject Null hypothesis
• Indirectly accept the research hypothesis
• Research hypothesis is supported
• Fail to reject null hypothesis
• Research hypothesis is not supported
• How do we determine whether or not to reject the
  null hypothesis
• Compute some statistics reflecting the difference
• Find out the probability that any difference is
  due to chance

5
Probability

• The likelihood of something happening
• Denote by p
• Numerical value ranging from 1 to 0
• Probability of discrete variables
• Probability of continuous variables

6
Statistical significance

• How far must an outcome be away from the
  expected?
• At what level of probability we believe a result
  is more likely due to a real difference (caused
  by experiment) than to chance
• Level of significance (significance level)
• a 0.05, 0.01, 0.1, 0.001

7
Making errors in hypothesis testing

• Type I error
• Reject the null hypothesis when it is true
• Type II error
• Fail to reject the null hypothesis when it is
  false
• Significance level -- ? (0.05)
• The probability of rejecting null hypothesis when
  it is true
• Power 1-? (0.80)
• The ability to reject a false null hypothesis

Actual situation null hypothesis is Actual situation null hypothesis is
Conclusion True False
Fail to reject H0 Correct decision Type II error
Probability 1- ? Probability ?
Reject H0 Type I error Correct decision
Probability ? Probability 1-?
8
One-tailed and two tailed hypotheses

• One-tailed hypothesis
• The direction of the relation is predicted in the
  alternative hypothesis
• Example
• People with high education are more interested in
  politics than people with low education
• H1 m1 gt m2
• Two-tailed hypothesis
• No prediction about the direction of the
  relationship is made
• Example
• There is a difference in interest of politics
  between people with high education and people
  with low education
• H1 m1 ? m2

9
Drawing conclusion from hypothesis test

• p-value
• The p-value indicates the probability that one
  would obtained a test statistic which is more
  extreme than the observed one when the null
  hypothesis is true.
• The possibility that any observed difference is
  due to chance.
• if p-value lt a, reject null hypothesis

10
One-tailed test

• Upper tail test
• Critical region (reject region) locates on the
  upper tail
• The area indicates the maximum probability that
  you can reject a null hypothesis (a)
• The corresponding value of its boundary is the
  critical value

11
Two tailed test

• Two critical regions locate on two tails

12
One and two-tailed tests

• Easier to reject a null hypothesis in a
  one-tailed than in a two-tailed test if the test
  statistic falls in the expected direction
• One tailed-test can not handle the situation when
  the test statistic falls in the wrong tail

13
p-value in one and two-tailed test

• p-value from a two-tailed test is double the
  value from a one-tailed test
• p P (z gt zobserved?H0 is true ) (upper-tailed
  test)
• p 2P (z gt zobserved ?H0 is rue) (two-tailed
  test)
• Some test returns either one or two-tailed test
  results, some returns both
• Find the p-value suitable for your hypothesis

14
Simplest hypothesis testing statistic

• t-test
• Assesses whether the means of two groups are
  statistically different from each other
• Sample size is small
• Dependent variable is interval or ratio scale
• Independent variable has two levels
• Approximately normal distribution of the measure
  in the two groups is assumed

15
t-score

• t-score
• difference between means/standard error of the
  difference
• Give both one- and two-tailed p-value

16
t-test case

• Scenario
• You want to measure the acquisition of
  mathematical skills by distance learning and
  traditional classroom learning. The study
  involves the comparison of 20 students, ten
  taught in classroom and ten taught by distance
  learning program. The final test scores were
  collected as dependent variable.
• Write down your null hypothesis
• Write a two-tailed hypothesis and a one-tailed
  hypothesis