Title: Exploratory Factor Analysis II Sept. 22, 2004
1Exploratory Factor Analysis IISept. 22, 2004
- Introduction to Matrix Algebra
- Matrix Definitions and Notation
- Matrix Operations
- Addition, Subtraction, Multiplication, Inversion
- Fundamental Statistics in Matrix Form
- Mean, Variance, Covariance, Covariance Matrices
- Factor Analysis Expressed in Matrix Form
- But, retaining some of the mystery
- Determining the Number of Factors
2Matrix Definitions and Notations
- Matrix
- Rectangular array of numbers or functions,
contained within brackets - Entire matrix is represented by capital letter
- Individual elements by lower case
a11 1
3Matrix Definitions and Notations
- Order of the Matrix
- Size of the matrix
- Represented by subscripts
- First value number of rows
- Second value number of columns
- Frequently used subscripts
- n, v, f
- Vertical matrices
- more rows than columns
- e.g., a data matrix Xnv
- Square matrices
- equal number of rows and columns
- e.g., a correlation matrix Rvv
- Vector
- matrix with only one row or column
- e.g., scores for individual X14 1 6 5 3
- Scalar
- only one row and column
- i.e., a single value (e.g., n 135, v 20, f
4)
Column Vector
4Matrix Definitions and Notations
- Symmetric Matrices
- Square matrix with elements above diagonal equal
to those below - e.g., correlation matrix Rvv, such that aij aji
- Diagonal matrix
- all off-diagonal elements are zeros
- at least one element in the diagonal is nonzero
- special case of the symmetric matrix
- Identity matrix
- all elements in the diagonal matrix are 1
- Matrix Transposition
- interchanging rows and columns
5Matrix Definitions and Notations
- Equal Matrices
- Matrices must have the same order (size)
- Each element must be equal across both matrices
6Matrix Operations
- Addition and Subtraction
- Matrices must have the same order (size)
7Matrix Operations
- Multiplication
- Number of columns in first matrix (A) must equal
number of rows in second matrix (B) - Matrix multiplication proceeds by computing the
sum of the cross-products of the elements in the
first row of A with the elements in the first
column of B - Process then repeated for first row of A and
second column of B - Resulting matrix (C) will have same number of
rows as A and same number of columns as B
c11 a11b11a12b21 1(1)4(2)9 c12
a11b12a12b22 1(2)4(1)6 c13
a11b13a12b23 1(3)4(2)11 c21
a21b11a21b21 2(1)5(2)12 . . c33
a31b13a32b23 3(3)6(2)21
8Matrix Operations
- Inversion (Matrix Algebra Division)
- In regular algebra
- a number divided by itself 1
- In matrix algebra
- the inverse of a matrix (A) is the matrix (A-1)
such that AA-1 I - Only square matrices have inverses
- How is the inverse of a matrix calculated?
- very laboriously
- so, lets just rely on the computer
9Matrix Operations
- Whats the utility of matrix inversion?
- In regular algebra
- lets say abc
- if a and c are known, b can be solved for by
multiplying each side by 1/a (b c/a) - In matrix algebra
- lets say AjjBjk Cjk
- then, Bjk Ajj-1Cjk
10Fundamental Stats in Matrix Form
- Mean
- Variance (when raw scores are deviations from the
mean)
11Fundamental Stats in Matrix Form
- Covariance (with raw scores as mean deviations)
- Correlation (with raw scores as mean deviations)
12Fundamental Stats in Matrix Form
- Correlation (with raw scores as mean deviations)
13Fundamental Stats in Matrix Form
14Fundamental Stats in Matrix Form
- Correlation Matrix
- where S-1vv is a diagonal matrix of the standard
deviations of the variables. - If the variables are in standard score form,
then - Exploratory FA traditionally operates in standard
score form
15Factor Analysis Expressed in Matrix Form
- Component Model
- Znv is the standard score data matrix
- Fnf is the standardized factor score matrix
- Pfv is the factor by variable weight matrix
(transposed factor pattern)
recall that
substituting top equation, we (eventually) get
where Rff is the correlation between the factors
(omitted for orthogonal models)
16Factor Analysis Expressed in Matrix Form
- Component Model
- These equations have an infinite number of
solutions when both F and P are being solved for
simultaneously - Because only the variable scores are known, they
are used as a starting point and the factor
scores are defined in terms of the variables - This means the factors are defined as a linear
function of the variables - So, this is the ballgame
- A linear weighting of the variables (factor A)
is computed that minimizes the size of Rvv.A when
multiplied through the equation above - Then the process is repeated on the residual
matrix
17Common Factor Model
Factors loadings for I, II, III determined by
computer calculation based on least squares
criterion
impR1 PIPI
resR1 Rr impR1 .16 - .029 .131 .20 - .133
.067 .51 - .386 .124
18Number of Factors
- Kaiser-Guttman Rule
- Number of eigenvalues greater than one
- eigenvalues basically represent the proportion of
RVV accounted for by each factor (vector) - Note that the sum of the eigenvalues will be
equal to v - Thus, even with (nearly) complete independence,
some will be greater than one - Usually, pretty messy
19Number of Factors
- Scree Test
- Plot of the eigenvalues
- Cut-off at the point where the curve changes from
rapid to flat - Cattell (1966) Walking up the scree (trivial
factors) until you reach the base of the mountain
(real factors) - Very subjective
20Number of Factors
Extraversion / Neuroticism
BIDR
21Number of Factors
- Confirmatory Maximum Likelihood
- yields a chi-square goodness of fit index
- Models based on different numbers of extracted
factors can be contrasted using a chi-square
difference test - with large samples and numbers of variables, this
criterion often yields too many factors - Not always clear which comparison to use
22Number of Factors
- Recommendation?
- Examine all three
- If they converge, great
- If not, go with convergence of Kaiser-Guttman
(factor must be above 1) and Scree (factor must
be on the upslope) - Most importantly, replicate!
23Project 1
- Whats your plan?
- Should we meet in my lab next week?
24Next Week
- Factor Rotation
- Defining alternatives during extraction
- principal components
- unweighted least squares
- generalized least squares
- maximum likelihood
- principal axis
- image factoring
- alpha factoring
- Applications 1 and 2
- Russells critique