Chapter 4 Simple Linear Regression

1
Chapter 4Simple Linear Regression
2
Agenda

• Simple linear regression
• Properties of the least square estimators and
  estimation of variance
• Hypothesis tests in simple linear regression
• Confidence intervals on the slope and intercept

3
Simple Linear Regression

• The case of simple linear regression considers a
  single regressor or predictor x and a dependent
  or response variable Y.
• The expected value of Y at each level of x is a
  random variable
• (1)

• We assume that each observation, Y, can be
  described by the model
• (2)

4
Simple Linear Regression

• Suppose that we have n pairs of observations
  (x1, y1), (x2, y2), , (xn, yn).

Deviations of the data from the estimated
regression model.
5
Simple Linear Regression

• The method of least squares is used to estimate
  the parameters, ?0 and ?1 by minimizing the sum
  of the squares of the vertical deviations in the
  following figure.

Deviations of the data from the estimated
regression model.
6
Simple Linear Regression

• Using Equation (2), the n observations in the
  sample can be expressed as

• The sum of the squares of the deviations of the
  observations from the true regression line is

7
Simple Linear Regression
8
Simple Linear Regression
9
Simple Linear Regression
Definition
10
Simple Linear Regression
11
Simple Linear Regression
Notation
12
Example 1
13
Example 1
14
Example 1
Scatter plot of oxygen purity y versus
hydrocarbon level x and regression model y
74.20 14.97x.
15
Minitab Practice for Example 1

• Menu ? Stat ? regression ?regression
• Response y
• Predictors x
• ?Options Prediction intervals for new obs. 1
• select all options
• ? Results select regression equation

16
Table 11-2 Software Output
17
Simple Linear Regression Estimating ?2
The error sum of squares is
It can be shown that the expected value of the
error sum of squares is E(SSE) (n 2)?2.
18
Simple Linear Regression Estimating ?2
An unbiased estimator of ?2 is
where SSE can be easily computed using
19
Properties of the Least Squares Estimators

• Slope Properties

• Intercept Properties

20
Hypothesis Tests in Simple Linear Regression
1. Use of t-Tests
Suppose we wish to test
(3)
An appropriate test statistic would be
21
Hypothesis Tests in Simple Linear Regression
1. Use of t-Tests
The test statistic could also be written as
We would reject the null hypothesis if
22
Hypothesis Tests in Simple Linear Regression
1. Use of t-Tests
Suppose we wish to test
An appropriate test statistic would be
23
Hypothesis Tests in Simple Linear Regression
1. Use of t-Tests
We would reject the null hypothesis if
24
Hypothesis Tests in Simple Linear Regression
1. Use of t-Tests
An important special case of the hypotheses of
Equation (3) is
These hypotheses relate to the significance of
regression. Failure to reject H0 is equivalent to
concluding that there is no linear relationship
between x and Y.
25
Hypothesis Tests in Simple Linear Regression
The hypothesis H0 ?1 0 is not rejected.

26
Hypothesis Tests in Simple Linear Regression
The hypothesis H0 ?1 0 is rejected.
27
Example 2
28
Hypothesis Tests in Simple Linear Regression
2. Analysis of Variance Approach to Test
Significance of Regression
The analysis of variance identity is
Symbolically,
29
Hypothesis Tests in Simple Linear Regression
2. Analysis of Variance Approach to Test
Significance of Regression
If the null hypothesis, H0 ?1 0 is true, the
statistic
follows the F1,n-2 distribution and we would
reject if f0 gt f?,1,n-2.
30
Hypothesis Tests in Simple Linear Regression
2. Analysis of Variance Approach to Test
Significance of Regression
The quantities, MSR and MSE are called mean
squares. Analysis of variance table
31
Example 3
32
Example 3
33
Confidence Intervals
1. Confidence Intervals on the Slope and Intercept
Definition
34
Example 4
35
Confidence Intervals
2. Confidence Interval on the Mean Response
Definition
36
Example 5
37
Example 5
38
Example 5
Scatter diagram of oxygen purity data from
Example 1 with fitted regression line and 95
percent confidence limits on ?Yx0.
39
Correlation
40
Correlation
We may also write
41
Correlation
It is often useful to test the hypotheses
The appropriate test statistic for these
hypotheses is
Reject H0 if t0 gt t?/2,n-2.
42
Correlation
The test procedure for the hypothesis
where ?0 ? 0 is somewhat more complicated. In
this case, the appropriate test statistic is
Reject H0 if z0 gt z?/2.
43
Correlation
The approximate 100(1- ?) confidence interval is
44
Example 6
45
Scatter plot of wire bond strength versus wire
length, Example 6.
46
Minitab Practice for Example 6

• Data file Example4_6.xls
• Menu ? Stat ? Regression ? Regression
• Response y
• Predictor x1
• ? Options select PRESS and predicted R-square
• ? Results select Regression equation, table of
  coefficients .

47
Minitab Output for Example 6
48
Example 6
49
Example 6
50
Example 6