Hypothesis Testing

1
Hypothesis Testing
2
Hypothesis (1)

• A hypothesis is a claim or assertion involving
  characteristics of the context in which you are
  working.
• Typically these are statements relating
  measurable, hopefully quantitative,
  characteristics.

3
Hypotheses (2)

• An hypothesis can be thought of as a prediction.
  It describes what you expect from a study.
• Not all studies have hypotheses. Exploratory
  studies may not have explicit hypotheses since
  the purpose of the activity is to collect data
  regarding a situation.

4
Hypotheses and Measures (1)

• Hypotheses should be stated in quantifiable
  terms.

Using method X produces better quality software
than method Y.
Code produced using method X has a lower number
of defects per KLOC than that produced by method
Y.
5
Hypotheses and Measures (2)

• The relationship between measures and the factors
  they represent should be documented.

6
Hypotheses, Again

• In most studies you will see two hypotheses.
  These two hypotheses describe the potential
  results you expect in the study and should be
  mutually exclusive.
• Alternative
• The hypothesis that you look to support is the
  alternative hypothesis.
• Null
• The hypothesis that describes the remaining
  possible outcomes the null hypothesis

7
Two Hypotheses

• Alternative
• The hypothesis that you support is the
  alternative hypothesis.
• Null
• The hypothesis that describes the remaining
  possible outcomes the null hypothesis

8
Is this important?

• The hypotheses for a study are
• HO As a result of the XYZ company code
  inspection training program, there will either be
  no significant difference in number of defects
  found during inspections or there will be a
  significant decrease.
• HA As a result of the XYZ company code
  inspection training program, there will be a
  significant increase in number of defects found
  during inspections.

9
Hypotheses
The null hypothesis, denoted H0, is the claim
that is initially assumed to be true. The
alternative hypothesis, denoted by Ha, is the
assertion that is contrary to H0. Possible
conclusions from hypothesis-testing analysis are
reject H0 or fail to reject H0.
10
A Test of Hypotheses
A test of hypotheses is a method for using sample
data to decide whether the null hypothesis should
be rejected.
11
Test Procedure
A test procedure is specified by

• A test statistic, a function of the sample data
  on which the decision is to be based.
• A rejection region, the set of all test statistic
  values for which H0 will be rejected (null
  hypothesis rejected iff the test statistic value
  falls in this region.)

12
Errors in Hypothesis Testing
A type I error consists of rejecting the null
hypothesis H0 when it was true. ? describes the
probability of a type I error. A type II error
involves not rejecting H0 when H0 is false. ?
describes the probability of a type II error.
13
Example

• A certain type of automobile is known to
  sustain no visible damage 25 of the time in
  10-mph crash tests. A modified bumper design has
  been proposed in an effort to increase this
  percentage. We will test the new bumper by
  conducting 20 independent crashes. When would we
  decide that the new bumper is effective?

14
Solution (1)
15
Solution (2)
This says we stand a 10 chance of rejecting H0
when it is true using that rejection region.
16
Rejection Region
Suppose an experiment and a sample size are
fixed, and a test statistic is chosen. The
decreasing the size of the rejection region, to
obtain a smaller value of , results in a
larger value of for any particular parameter
value consistent with Ha.
17
Significance Level
Specify the largest value of that can be
tolerated and find a rejection region having that
value of . This makes as small as
possible subject to the bound on . The
resulting value of is referred to as the
significance level.
18
Level Test
A test corresponding to the significance level is
called a level test. A test with
significance level is one for which the type
I error probability is controlled at the
specified level.