Loading...

PPT – Ch14: Linear Least Squares PowerPoint presentation | free to download - id: 7911c0-NGYwN

The Adobe Flash plugin is needed to view this content

Ch14 Linear Least Squares

- 14.1 INTRO
- Fitting a pth-order polynomial will require

finding (p1) coefficients from the data. Thus,

a straight line (p1) is obtained thru its slope

and intercept. - LS (Least Squares) method finds parameters by

minimizing the sum of the squared deviations of

the fitted values from the actual observations.

Predicting y (responsedependent) from x

(predictorindependent)

- Formula

14.2 Simple Linear Regression(linear in the

parameters)

- Regression is NOT fitting line but E(YXx)
- 14.2.1 Properties of the estimated slope

Intercept

Variance-Covariance of the betas

- Under the assumptions of Theorem A

Inferences about the betas

- In the previous result,

14.2.2 Assessing the Fit

- Recall, that the residuals are the differences

between the observed and the fitted values - Residuals are to be plotted versus the x-values.
- Ideal plot should look like a horizontal blur

that is to say that one can reasonably model it

as linear. - Caution the errors have zero mean and are said

to be homoscedastic constant variance

independently of the predicator x. That is to

say

Steps in Linear Regression

- Fit the Regression Model (Mathematics)
- Pick a method Least Squares or else
- Plot the data Y versus g(x)
- Compute regression estimates residuals
- Check for linearity outliers (plot residuals)
- More diagnostics (beyond the scoop of this class)
- Statistical Inference (Statistics)
- Check for error assumptions
- Check for normality (if not transform data)
- If nonlinear form, then (beyond the scoop of this

class) - Least Squares Java applet
- http//www.math.tamu.edu/FiniteMath/Classes/LeastS

quares/LeastSquares.html

14.2.3 Correlation Regression

- A close relation exists between Correlation

Analysis Fitting straight lines by the Least

Squares method.

14.3 Matrix approach to Linear Least Squares

- Weve already fitted straight lines (p1).
- What if p gt 1 ? ? Investigate some Linear

Algebra tools

Formulation of the Least Squares problem

14.4 Statistical Properties of Least Squares

Estimates

- 14.4.1 Vector-valued Random Variables

Cross-covariance matrix

14.4.2 Mean and Covariance of Least Squares

Estimates

14.4.3 Estimation of the common variance for the

random errors

- In order to make inference about , one must

get an estimate of the parameter (if

unknown).

14.4.4 Residuals Standardized Residuals

14.4.5 Inference about

- Recall Section 14.4 for the statistical

properties of the Least Squares Estimates with

some additional assumptions about the errors

being

14.5 Multiple Linear Regression

- This section will generalize Section 14.2 (Simple

Linear Regression) by doing the Multiple Linear

Regression thru an example of polynomial

regression.