Matrix Representation of Univariate Procedures

1
Matrix Representation of Univariate Procedures

• The General Linear Model
• Associated Matrix Operations
• Non-Traditional Regression

2
Two-way ANOVA

• Two treatment factors, with g and b levels
• There are g levels of factor 1
• b levels of factor 2
•  gb combinations of levels
• N independent observations

3
Univariate Analysis of VarianceTwo-way Fixed
Effects Model with Interaction

The ANOVA model (Linear Model) can be written as
? is the grand mean ? is the fixed effect for
factor 1 ? is fixed effect of factor 2 ? is the
interaction
4
The Expected Response
are independent
5
In other words
6
Hypotheses tested by ANOVA

• 1) Does the effect of one factor on the response
  variable(s) depend on level of the other factor? 
• H0 There is no interaction between Factor 1 and
  Factor 2
• 2) Do the levels of Factor 1 differ in the
  effects on the response variable(s)
•  H0 There is no main effect of Factor 1 on the
  response
• 3) Do the levels of Factor 2 differ in their
  effects on the response variable(s)

7
ANOVA Table Variance Decomposition
8
ANOVA in Matrix Notation

• Regardless of the complexity of the ANOVA model,
  we can express it in matrix notation
• X is a matrix of 0s and 1s that follows the
  experimental plan and its linear model

y X? ?
9
The General Linear Model
10
Least Squares Estimates of b
11
Elaboration of Matrix Elements
The transpose of the parameter vector is
12
Design Matrix
Each column of the design matrix corresponds with
the appropriate element of the parameter vector.
13
Assumptions of ANOVA

• Normal distribution
• Independence of residuals
• Homoscedasticity of Variances
• Variances are Equal

14
Regression Analysis

• Most widely applied technique for assessing
  relationships among variables
• Used to investigate relationship between a
  response (dependent) variable and one or more
  predictor (independent) variables.
• Regression analysis is concerned with estimating
  and predicting the population mean value of the
  response variable Y on the basis of known (fixed)
  values of one or more predictor (or explanatory)
  variable(s)

15
The Population-based Regression Model
?0, ?1 are unknown, but fixed parameters ?0, -
intercept ?1 slope
16
Full Model
?i is referred to as an Error, Residual, or
Disturbance term.
17
Properties of Population Model

• Postulates the condition means are linear
  functions of the Xi.
• The ?s are known as regression coefficients.
• The intercept gives E(YX0)
• The slope describes the change in Y for a fixed
  unit change in X

18
Advantages and Reality

• The advantage of using population based model is
  that we can derive a regression model in a simple
  way
• Population regression model is unrealistic
  because we rarely have complete data on all
  individuals, families, etc. that define a
  population
• Population Regression model easily modified for
  sample-based estimates
• We call it linear regression because it is linear
  in the parameters

19
Disturbances

• Note that there is dispersion about the
  conditional mean of Y. As Xi increases, Yi does
  not necessarily increase.
•  Thus, there can be instances when Yi gt Yj, but
  Xi lt Xj
•  The difference Yi E(YXi) is an unobservable
  random variable that can take on positive or
  negative values
•  These random variables are called disturbances.

20
Why is it that Yi ? E(YXi)?

• We neglected to include variables that affect Y
  in the regression model.
• Randomness in the response (dependent) variable
• The es represent estimates of the true
  disturbances.

21
Assumptions of Regression Analysis

• We will cover the assumptions in greater detail
  later.
•  Briefly
• Need to assume Ys are normally distributed
• Xs are fixed,
• Disturbances (ei) are normal, independent random
  variables.

22
Sample-based Regression Model
or
23
How to estimate b0 and b1.

• Use Ordinary Least Squares approach.
• i.e., minimize error sum of squares.

minimize
24
ANOVA Table for Regression
25
Matrix Notation for Linear Regression
We can estimate the regression parameters using
the simple expression
26
New Matrix Concept
Matrix Inverse
27
Matrix Inverse Matrix Division

• Matrix Division requires computation of Matrix
  Inverse
• Inverse in turn requires computation of the
  Determinant of a Matrix
• Matrix Inverse defined for Square Matrix
• Read Legendre Legendre (pp 68 80)
• Review on Thursday

28
Statistical Software

• JMP
• SAS
• Proc LOGISTIC
• S-Plus