Multiple regression

1
Multiple regression

• K300 Class 29
• April 28, 2009

2
Overview

• SPSS analysissimple regression
• Multiple regression
• SPSS analysismultiple regression

3
SPSS analysis simple (2-variable) regression

• Analysis of rent versus family income
• Interactive scatterplot
• Graphs, Chart Builder, Scatter/Dot
• Add regression line in Chart Editor
• Regression analysis
• Analyze, Regression, Linear

4
Scatterplot with regression line
5
Regression output goodness of fit
Correlation
Coefficient of Determination
6
Regression output significance of model
Significance Test of null hypothesis ? 0 or
Population R2 0
7
Regression output regression coefficients
b - slope
a y intercept
8
Multiple regression

• Use of two or more independent (x) variables to
  predict value of dependent (y) variable
• For case of two independent variables, fitting
  least-squares plane to data points in
  3-dimensional space

9
Multiple regression equations

• Two independent variables
• k independent variables

10
Multiple regression estimating regression
coefficients

• Still use least-squares procedures, minimizing
  sum of squared errors
• Find regression coefficients so that errors in
  predictiondifferences between observed and
  predicted valuessquared and summed are the least

11
Multiple regression coefficient of determination

• Coefficient of determination (R-square) is still
  measure of goodness of fit of the model,
  proportion of variation in dependent variable
  accounted for by all of the independent variables
• Measure of strength of relationship between
  dependent variable and the set of independent
  variables

12
Multiple regression significance of model

• ANOVA and F test are still used to test for
  whether there is a significant relationship,
  whether the model as a whole is significant,
  whether the coefficient of determination
  (R-square) is significantly different from zero
• Null hypothesis population coefficient of
  determination (R-square) 0 no relationship

13
Multiple regression regression coefficients

• Regression coefficients now include y intercept
  or constant and regression coefficients for each
  of the dependent variables, e.g.,

14
Regression coefficients SPSS output
Regression coefficients
15
Interpretation of multiple regression coefficients

• Same as for simple regression
• Amount of change in dependent (y) variable that
  would result for a unit change in the independent
  (x) variable (holding the other independent
  variables constant)
• In example, 1 increase in median family income
  would result in 0.011 increase in rent
• Increase of 1 in percent population change would
  result in 0.921 increase in rent

16
Multiple regression tests for significance of
regression coefficients

• For multiple regression, now we have additional,
  separate tests for significance of the regression
  coefficients
• Test for whether independent variable is a
  significant predictor of the dependent variable
  in the context of the multiple regression

17
Multiple regression tests for significance of
regression coefficients

• t test for significance of regression coefficient
• Null hypothesis, bi 0
• Alternative hypothesis, bi ltgt 0

18
Significance of regression coefficients SPSS
output
Levels of significance for hypothesis tests for
regression coefficients
19
Prediction using multiple regression

• Equation estimated using multiple regression can
  be used for prediction in the same way as simple
  regression
• Predict rent for metropolitan area with median
  family income of 25,000 and percent population
  change of 30

20
Prediction using multiple regression
21
SPSS multiple regression example

• Same dataset with data for 102 largest
  metropolitan areas in 1980
• Simple regression used median family income as
  independent variable to predict rent, dependent
  variable
• Now we add percent change in population in
  preceding decade as second independent variable

22
Matrix scatterplot
23
3-d scatterplot with plane
24
Running the multiple regression
25
Regression output coefficient of determination
Coefficient of determination R-square Measure of
goodness of fit of the model Proportion of the
variation in the dependent variable accounted for
by the model
26
Regression output significance of model
Significance Test of null hypothesis of no
relationship, of population R-square 0 If
significance is less than alpha (e.g., 0.05)
reject null hypothesis, conclude significant
relationship
27
Regression output regression coefficients
Regression coefficients Give regression
equation Used for prediction Measure of change in
dependent variable associated with unit change in
independent variable (not constant)
28
Regression output significance of regression
coefficients
Significance levels for hypothesis tests of
significance of regression coefficients (null
hypothesis that regression coefficient equals
zero Reject null hypothesis (coefficient
significantly different from zero) of
significance less than alpha, e.g., 0.05