Residual%20Analysis

1
Residual Analysis
2
Residual Analysis

• Purposes
• Examine Functional Form (Linear vs. Non-Linear
  Model)
• Evaluate Violations of Assumptions
• Graphical Analysis of Residuals

3
Y
(X1, mY1)
X1
For one value X1, a population contains may Y
values. Their mean is mY1.
4
(No Transcript)
5
A Sample Regression Line
Y
The sample line approximates the population
regression line.
x
6
Histogram of Y Values at X X1
mY1 a BX1
mY a BX
7
Normal Distribution of Y Values when X X1
The standard deviation of the normal
distribution is the standard error of estimate.
mY1 a BX1
mY a BX
8
Normality Constant Variance Assumptions
f(e)
Y
X
1
X
2
X
9
A Normal Regression Surface
Every cross-sectional slice of the surface is a
normal curve.
f(e)
Y
X
1
X
2
X
10
Analysis of Residuals

• A residual is the difference between the actual
  value of Y and the predicted value .

11
Linear Regression and Correlation Assumptions

• The independent variables and the dependent
  variable have a linear relationship.
• The dependent variable must be continuous and at
  least interval-scale.

12
Linear Regression Assumptions

• Normality
• Y Values Are Normally Distributed with a mean of
  Zero For Each X.
• The residuals ( e ) are normally distributed with
  a mean of Zero.
• Homoscedasticity (Constant Variance)
• The variation in the residuals must be the same
  for all values of Y.
• The standard deviation of the residuals is the
  same regardless of the given value of X.
• Independence of Errors
• The residuals are independent for each value of X
• The residuals ( e ) are independent of each other
  
• The size of the error for a particular value of x
  is not related to the size of the error for any
  other value of x

13
Evaluating the Aptness of the Fitted Regression
Model

• Does the model appear linear?

14
Residual Plot for Linearity(Functional
Form)Aptness of the Fitted Model
15
Residual Plots for Linearityof the Fitted Model

• Scatter Plot of Y vs. X value
• Scatter Plot of residuals vs. X value

16
Using SPSS to Test for Linearity of the
Regression Model

• Analyze/Regression/Linear
• Dependent - Sales
• Independent - Customers
• Save
• Predicted Value (Unstandardized or Standardized)
• Residual (Unstandardizedor Standardized)
• Graphs/Scatter/Simple
• Y-Axis residual res_1 or zre_1
• X-Axis Customer (independent variable)

17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
ELECTRONIC FIRMS
21
TheLinear Regression Assumptions

• 1. Normality of residuals (Errors)
• 2. Homoscedasticity (Constant Variance)
• 3. Independence of Residuals (Errors)

Need to verify using residual analysis.
22
Residual Plots for Normality

• Construct histogram of residuals
• Stem-and-leaf plot
• Box-and-whisker plot
• Normal probability plot
• Scatter Plot residuals vs. X values
• Simple regression
• Scatter Plot residuals vs. Y
• Multiple regression

23
Residual Plot 1 for Normality

• Construct histogram of residuals
• Nearly symmetric
• Centered near or at zero
• Shape is approximately normal

24
Using SPSS to Test for NormalityHistogram of
Residuals

• Analyze/Regression/Linear
• Dependent - Sales
• Independent - Customers
• Plot/Standardized Residual Plot Histogram
• Save
• Predicted Value (Unstandardized or Standardized)
• Residual (Unstandardizedor Standardized)
• Graphs/Histogram
• Variable - residual (Unstandardized or
  Standardedized)

25
Histogram of Residuals of Sales and Customer
Problemfrom regression output
26
Histogram of Residuals of Sales and Customer
Problemfrom graph output
27
Residual Plot 2 for Normality

• Plot residuals vs. X values
• Points should be distributed about the horizontal
  line at 0
• Otherwise, normality is violated

0
28
Using SPSS to Test for NormalityScatter Plot

• Simple Regression
• Graph/Scatter/Simple
• Y-Axis residual res_1 or zre_1
• X-Axis Customers independent variable
• Multiple Regression
• Graph/Scatter/Simple
• Y-Axis residual res_1 or zre_1
• X-Axis predicted Y values

29
(No Transcript)
30
The Electronic Firms
An accounting standards board investigating
the treatment of research and development
expenses by the nations major electronic firms
was interested in the relationship between a
firms research and development expenditures and
its earnings.
Earnings 6.840 10.671(rdexpend)
31
List of Data, Predicted Values and Residuals
Data Predicted Residual
Standardized Standardized
Value Predicted
Value Residual
ELECTRONIC FIRMS
32
ELECTRONIC FIRMS
33
ELECTRONIC FIRMS
34
Residual Plot for HomoscedasticityConstant
Variance
Correct Specification
0
35
Using SPSS to Test for Homoscedasticity of
Residuals

• Simple Regression
• Graphs/Scatter/Simple
• Y-Axis residual res_1 or zre_1
• X-Axis rdexpend independent variable
• Multiple Regression
• Graphs/Scatter/Simple
• Y-Axis residual res_1 or zre_1
• X-Axis predicted Y values

36
Test for Homoscedasticity
DUNTONS WORLD OF SOUND
37
Test for Homoscedasticity
ELECTRONIC FIRMS
38
(No Transcript)
39
Residual Plot for Independence
Plots Reflect Sequence Data Were Collected.
40
Two Types of Autocorrelation

• Positive Autocorrelation successive terms in
  time series are directly related
• Negative Autocorrelation successive terms are
  inversely related

41
Positive autocorrelation Residuals tend to be
followed by residuals with the same sign
42
Negative Autocorrelation Residuals tend to
change signs from one period to the next
43
Problems with autocorrelated time-series data

• sy.x and sb are biased downwards
• Invalid probability statements about regression
  equation and slopes
• F and t tests wont be valid
• May imply that cycles exist
• May induce a falsely high or low agreement
  between 2 variables

44
Using SPSS to Test for Independence of Errors

• Graphs/Sequence
• Variables residual (res_1)
• Durbin-Watson Statistic

45
DUNTONS WORLD OF SOUND
46
ELECTRONIC FIRMS
47
Customers
Sales(000)
794 9 799 8 837 7 855 9 845 10 844 10 863 11 875 1
1 880 12 905 13 886 12 843 10 904 12 950 12 841 10
Customers and sales for period of 15 consecutive
weeks.
48
(No Transcript)
49
Durbin-Watson Procedure

• Used to Detect Autocorrelation
• Residuals in One Time Period Are Related to
  Residuals in Another Period
• Violation of Independence Assumption
• Durbin-Watson Test Statistic

)
50
H0 No positive autocorrelation exists
(residuals are random) H1 Positive
autocorrelation exists Accept Ho if dgt du Reject
Ho if d lt dL Inconclusive if dL lt d lt du d
51
Testing for Positive Autocorrelation
There is positive autocorrelation
There is no evidence of autocorrelation
The test is inconclusive
4
2
0
dL
du
52
Rule of Thumb

• Positive autocorrelation - D will approach 0
• No autocorrelation - D will be close to 2
• Negative autocorrelation - D is greater than 2
  and may approach a maximum of 4

53
Using SPSS with Autocorrelation

• Analyze/Regression/Linear
• Dependent Independent
• Statistics/Durbin-Watson (use only time series
  data)

54
Customers
Sales(000)
794 9 799 8 837 7 855 9 845 10 844 10 863 11 875 1
1 880 12 905 13 886 12 843 10 904 12 950 12 841 10
Customers and sales for period of 15 consecutive
weeks.
55
(No Transcript)
56
Durbin-Watson
.883
57
Using SPSS with Autocorrelation

• Analyze/Regression/Linear
• Dependent Independent
• Statistics/ Durbin-Watson (use only time series
  data)
• If DW indicates autocorrelation, then
• Analyze/Time Series/Autoregression
• Cochrane-Orcutt
• OK

58
Solutions for autocorrelation

• Use Final Parameters under Cochrane-Orcutt
• Changes in the dependent and independent
  variables - first differences
• Transform the variables
• Include an independent variable that measures the
  time of the observation
• Use lagged variables (once lagged value of
  dependent variable is introduced as independent
  variable, Durbon-Watson test is not valid