Session 2b

1
Session 2b
2
Overview

• Hypothesis Testing
• Review of the Basics
• Single Parameter
• Differences Between Two Parameters
• Independent Samples
• Matched Pairs
• Goodness of Fit
• Simulation Methods
• Using Historical Data in Simulations
• Parametric Approach
• Resampling Approach

3
Basic Hypothesis Testing Method

• Formulate Two Hypotheses
• Select a Test Statistic
• Derive a Decision Rule
• Calculate the Value of the Test Statistic Invoke
  the Decision Rule in light of the Test Statistic

4
(No Transcript)
5
The p-value of a test is the probability of
observing a sample at least as unlikely as
ours. In other words, it is the minimum level
of significance that would allow us to reject
H0. Small p-value unlikely H0
6
Example Buying a Laundromat
A potential entrepreneur is considering the
purchase of a coin-operated laundry. The present
owner claims that over the past 5 years the
average daily revenue has been 675. The buyer
would like to find out if the true average daily
revenue is different from 675. A sample of 30
selected days reveals a daily average revenue of
625 with a standard deviation of 75.
7
(No Transcript)
8
Test Statistic Decision Rule, based on alpha
of 1 Reject H0 if the test statistic is greater
than 2.575 or less than -2.575.
9
(No Transcript)
10
We reject H0. There is sufficiently strong
evidence against H0 to reject it at the 0.01
level. We conclude that the true mean is
different from 675.
11
Example Reliability Analysis
12
The brand manager wants to begin advertising for
this product, and would like to claim a mean time
between failures (MTBF) of 45 hours. The product
is only in the prototype phase, so the design
engineer uses Crystal Ball simulation to estimate
the products reliability characteristics.
Extracted data
13
H0 here is that the product lasts 45 hours (on
the average). There is sufficiently strong
evidence against H0 to reject it at any
reasonable significance level. We conclude that
the true MTBF is less than 45 hours.
14
Example Effects of Sales Campaigns
In order to measure the effect of a storewide
sales campaign on nonsale items, the research
director of a national supermarket chain took a
random sample of 13 pairs of stores that were
matched according to average weekly sales volume.
One store of each pair (the experimental group)
was exposed to the sales campaign, and the other
member of the pair (the control group) was not.
15
The following data indicate the results over a
weekly period
16
Is the campaign effective? Basically this is
asking Is there a difference between the average
sales from these two populations (with and
without the campaign)?
17
Two Methods

• Independent Samples
• General Method
• Matched Pairs
• Useful Only in Specific Circumstances
• More Powerful Statistically
• Requires Logical One-to-One Correspondence
  between Pairs

18
Independent Samples Method
19
We do not reject the null hypothesis. The
campaign made no significant difference in sales.
20
(No Transcript)
21
Matched-Pairs Method
22
This time we do reject the null hypothesis, and
conclude that the campaign actually did have a
significant positive effect on sales.
23
(No Transcript)
24
TSB Problem Revisited
25
Independent Samples
26
(No Transcript)
27
(No Transcript)
28
Matched Pairs
29
(No Transcript)
30
(No Transcript)
31
Goodness-of-Fit Tests

• Determine whether a set of sample data have been
  drawn from a hypothetical population
• Same four basic steps as other hypothesis tests
  we have learned
• An important tool for simulation modeling used
  in defining random variable inputs

32
Example Warren Sapp
Financial analyst Warren Sapp wants to run a
simulation model that includes the assumption
that the daily volume of a specific type of
futures contract traded at U.S. commodities
exchanges (represented by the random variable X)
is normally distributed with a mean of 152
million contracts and a standard deviation of 32
million contracts. (This assumption is based on
the conclusion of a study conducted in 1998.)
Warren wants to determine whether this assumption
is still valid.
33
He studies the trading volume of these contracts
for 50 days, and observes the following results
(in millions of contracts traded)
34
(No Transcript)
35
Here is a histogram showing the theoretical
distribution of 50 observations drawn from a
normal distribution with µ 152 and s 32,
together with a histogram of Warren Sapps sample
data
36
The Chi-Square Statistic
37
Essentially, this statistic allows us to compare
the distribution of a sample with some expected
distribution, in standardized terms. It is a
measure of how much a sample differs from some
proposed distribution. A large value of
chi-square suggests that the two distributions
are not very similar a small value suggests that
they fit each other quite well.
38
Like Students t, the distribution of chi-square
depends on degrees of freedom. In the case of
chi-square, the number of degrees of freedom is
equal to the number of classes (a.k.a. bins
into which the data have been grouped) minus one,
minus the number of estimated parameters.
39
(No Transcript)
40
(No Transcript)
41
Note It is necessary to have a sufficiently
large sample so that each class has an expected
frequency of at least 5. We need to make sure
that the expected frequency in each bin is at
least 5, so we collapse some of the bins, as
shown here.
42
The number of degrees of freedom is equal to the
number of bins minus one, minus the number of
estimated parameters. We have not estimated any
parameters, so we have d.f. 4 1 0 3. The
critical chi-square value can be found either by
using a chi-square table or by using the Excel
function CHIINV(alpha, d.f.) CHIINV(0.05, 3)
7.815 We will reject the null hypothesis if
the test statistic is greater than 7.815.
43
Our test statistic is not greater than the
critical value we cannot reject the null
hypothesis at the 0.05 level of significance.
It would appear that Warren is justified in
using the normal distribution with µ 152 and s
32 to model futures contract trading volume in
his simulation.
44
The p-value of this test has the same
interpretation as in any other hypothesis test,
namely that it is the smallest level of alpha at
which H0 could be rejected. In this case, we
calculate the p-value using the Excel function
CHIDIST(test stat, d.f.) CHIDIST(7.439,3)
0.0591
45
Example Catalog Company
If we want to simulate the queueing system at
this company, what distributions should we use
for the arrival and service processes?
46
Arrivals
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
Services
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
(No Transcript)
58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
(No Transcript)
62
(No Transcript)
63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
The Parametric Approach
Fit the data to some theoretical distribution
(such as normal or exponential) and estimate the
parameters appropriate to the distribution (such
as mean and standard deviation for a normal
distribution, or lambda for an exponential
distribution). Advantage Simplicity (a random
variable can be described with a few parameters
instead of all the data). Disadvantage Need
assurance that the theoretical distribution we
choose is in fact a good fit to the data. This
gives rise to a special kind of hypothesis test,
called a goodness-of-fit test.
70
The Parametric Approach

• find which theoretical distribution best fits
  each variable,
• estimate the proper parameters for each, and
• specify a correlation coefficient for the
  relationship between the two variables.

71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
(No Transcript)
78
Using the same approach, we determine that the
best fit to our Y data is a Logistic distribution
with Mean 75.0 and Scale parameter 10.56.
79
(No Transcript)
80
The last step is to define the correlation
between these two variables. You can specify the
correlation between any pair of variables by
selecting either assumption cell, clicking the
Define Assumption button and then clicking on
Correlate.
81
(No Transcript)
82
The Resampling Approach
In this approach, we make no assumptions about
any theoretical distributions that may or may not
actually fit our data we use the data themselves
as the basis for our simulation. Advantages
Avoids the problem of Type II errors in the
Chi-square test. Also spares us from dealing
explicitly with correlation. Disadvantage our
model may have to include a large set of data (as
opposed to the few parameters we used in the
parametric approach).
83
Back to our example. Start the model with a
spreadsheet similar to the parametric one. Note
that we have added a column of integers in column
A.
84
Use the VLOOKUP function in E9 and F9 to look
up the X, Y pair corresponding to the integer in
E6.
85
(No Transcript)
86
(No Transcript)
87
Click OK and the cell turns green. Now the
model is ready to sample X, Y pairs randomly from
the data set.
88
Using Crystal Ball copy and paste, we can set up
a series of X, Y pairs, as shown here
89
Summary

• Hypothesis Testing
• Review of the Basics
• Single Parameter
• Differences Between Two Parameters
• Independent Samples
• Matched Pairs
• Goodness of Fit
• Simulation Methods
• Using Historical Data in Simulations
• Parametric Approach
• Resampling Approach