Factor Analysis and Inference for Structured Covariance Matrices

1
Factor Analysis and Inference for Structured
Covariance Matrices

• Shyh-Kang Jeng
• Department of Electrical Engineering/
• Graduate Institute of Communication/
• Graduate Institute of Networking and Multimedia

2
History

• Early 20th-century attempt to define and measure
  intelligence
• Developed primarily by scientists interested in
  psychometrics
• Advent of computers generated a renewed interest
• Each application must be examined on its own
  merits

3
Essence of Factor Analysis

• Describe the covariance among many variables in
  terms of a few underlying, but unobservable,
  random factors.
• A group of variables highly correlated among
  themselves, but having relatively small
  correlations with variables in different groups
  represent a single underlying factor

4
Example 9.8Examination Scores
5
Orthogonal Factor Model
6
Orthogonal Factor Model
7
Orthogonal Factor Model
8
Orthogonal Factor Model
9
Orthogonal Factor Model
10
Example 9.1 Verification
11
Example 9.2 No Solution
12
Ambiguities of L When mgt1
13
Principal Component Solution
14
Principal Component Solution
15
Residual Matrix
16
Determination of Number of Common Factors
17
Example 9.3Consumer Preference Data
18
Example 9.3Determination of m
19
Example 9.3Principal Component Solution
20
Example 9.3Factorization
21
Example 9.4Stock Price Data

• Weekly rates of return for five stocks
• X1 Allied Chemical
• X2 du Pont
• X3 Union Carbide
• X4 Exxon
• X5 Texaco

22
Example 9.4Stock Price Data
23
Example 9.4Principal Component Solution
24
Example 9.4Residual Matrix for m2
25
Maximum Likelihood Method
26
Result 9.1
27
Factorization of R
28
Example 9.5 Factorization ofStock Price Data
29
Example 9.5ML Residual Matrix
30
Example 9.6Olympic Decathlon Data
31
Example 9.6Factorization
32
Example 9.6PC Residual Matrix
33
Example 9.6ML Residual Matrix
34
A Large Sample Test for Number of Common Factors
35
A Large Sample Test for Number of Common Factors
36
Example 9.7Stock Price Model Testing
37
Example 9.8Examination Scores
38
Example 9.8Maximum Likelihood Solution
39
Example 9.8Factor Rotation
40
Example 9.8Rotated Factor Loading
41
Varimax Criterion
42
Example 9.9 Consumer-Preference Factor Analysis
43
Example 9.9Factor Rotation
44
Example 9.10 Stock Price Factor Analysis
45
Example 9.11Olympic Decathlon Factor Analysis
46
Example 9.11Rotated ML Loadings
47
Factor Scores
48
Weighted Least Squares Method
49
Factor Scores of Principal Component Method
50
Orthogonal Factor Model
51
Regression Model
52
Factor Scores by Regression
53
Example 9.12Stock Price Data
54
Example 9.12Factor Scores by Regression
55
Example 9.13 Simple Summary Scores for Stock
Price Data
56
A Strategy for Factor Analysis

• 1. Perform a principal component factor analysis
• Look for suspicious observations by plotting the
  factor scores
• Try a varimax rotation
• 2. Perform a maximum likelihood factor analysis,
  including a varimax rotation

57
A Strategy for Factor Analysis

• 3. Compare the solutions obtained from the two
  factor analyses
• Do the loadings group in the same manner?
• Plot factor scores obtained for PC against scores
  from ML analysis
• 4. Repeat the first 3 steps for other numbers of
  common factors
• 5. For large data sets, split them in half and
  perform factor analysis on each part. Compare the
  two results with each other and with that from
  the complete data set

58
Example 9.14Chicken-Bone Data
59
Example 9.14Principal Component Factor Analysis
Results
60
Example 9.14 Maximum Likelihood Factor Analysis
Results
61
Example 9.14Residual Matrix for ML Estimates
62
Example 9.14Factor Scores for Factors 1 2
63
Example 9.14Pairs of Factor Scores Factor 1
64
Example 9.14Pairs of Factor Scores Factor 2
65
Example 9.14Pairs of Factor Scores Factor 3
66
Example 9.14Divided Data Set
67
Example 9.14 PC Factor Analysis for Divided Data
Set
68
WOW Criterion

• In practice the vast majority of attempted factor
  analyses do not yield clear-cut results
• If, while scrutinizing the factor analysis, the
  investigator can shout Wow, I understand these
  factors, the application is deemed successful

69
Structural Equation Models

• Sets of linear equations to specify phenomena in
  terms of their presumed cause-and-effect
  variables
• In its most general form, the models allow for
  variables that can not be measured directly
• Particularly helpful in the social and behavioral
  science

70
LISREL (Linear Structural Relationships) Model
71
Example

• h performance of the firm
• x managerial talent
• Y1 profit
• Y2 common stock price
• X1 years of chief executive experience
• X2 memberships on board of directors

72
Linear System in Control Theory
73
Kinds of Variables

• Exogenous variables not influenced by other
  variables in the system
• Endogenous variables affected by other variables
• Residual associated with each of the dependent
  variables

74
Construction of a Path Diagram

• Straight arrow
• to each dependent (endogenous) variable from each
  of its source
• Straight arrow
• also to each dependent variables from its
  residual
• Curved, double-headed arrow
• between each pair of independent (exogenous)
  variables thought to have nonzero correlation

75
Example 9.15Path Diagram
76
Example 9.15Structural Equation
77
Covariance Structure
78
Estimation
79
Example 9.16Artificial Data
80
Example 9.16Artificial Data
81
Example 9.16Artificial Data
82
Assessing the Fit of the Model

• The number of observations p for Y and q for X
  must be larger than the total number of unknown
  parameters t lt (pq)(pq1)/2
• Parameter estimates should have appropriate signs
  and magnitudes
• Entries in the residual matrix S S should be
  uniformly small

83
Model-Fitting Strategy

• Generate parameter estimates using several
  criteria and compare the estimates
• Are signs and magnitudes consistent?
• Are all variance estimates positive?
• Are the residual matrices similar?
• Do the analysis with both S and R
• Split large data sets in half and perform the
  analysis on each half