Variable Screening Methods

1
Variable Screening Methods

• Chapter 6

2
Why use a Variable Screening Method?

• In this chapter, we consider two systematic
  methods designed to reduce a large list of
  potential predictors to a more manageable one.
  These techniques, known as variable screening
  procedures, objectively determine which
  independent variables in the list are the most
  important predictors of y and which are the least
  important predictors.

3
Stepwise Regression

• One of the most widely used variable screening
  methods is known as stepwise regression. To run
  a stepwise regression, the user first identifies
  the dependent variable (response) y, and the set
  of potentially important independent variables,
  x1, x2, , xk, where k is generally large.
  Note This set of variables could include both
  first-order and higher-order terms as well as
  interactions. The data are entered into the
  computer software, and the stepwise procedure
  begins.

4
All-Possible Regression Selection Procedure

• R2 Criterion
• Adjusted R2 or MSE Criterion
• Cp Criterion
• Press Criterion

5
Caveats

• Both stepwise regression and the
  all-possible-regressions selection procedure are
  useful variable screening methods. Many
  regression analysts, however, tend to apply these
  procedures as model building methods. Why? The
  stepwise (or best subset) model will often have a
  high value of R2 and all the ß coefficients in
  the model will be significantly different from 0
  with small p-values. And, with very little work
  (other than collecting the data and entering it
  into the computer), you can obtain the model
  using a statistical software package.
  Consequently, it is extremely tempting to use the
  stepwise model as the final model for predicting
  and making inferences about the dependent
  variable, y.