LATENT VARIABLES AND CORRELATION

1
LATENT VARIABLESANDCORRELATION
2
Data and Information
Obs. no. x1 x2 1 1 -1 2 1/2 -1/2
3 -1/2 1/2 4 -1 1
1. Is there any common information in x1,
x2? 2. If yes, can we express such common
information quantitatively?
3
Physical Sciences
x1 - x2 x2 - x1
4
Empirical Sciences
x1
5
Statistical formulation
Cor (x1,x2)
6
Correlation is the statistician's solution to
expressing the strength in correspondence between
variables.This reduces the graph into a
number!And statisticians like numbers.
7
Latent variable (LV)

• x1 - x2

8
Obs no. x1 x2 x1 - x2 1 1 -1 2
2 1/2 -1/2 1 3 -1/2
1/2 -1 4 -1 1
-2
9
x1-x2(LV)
?1
cor (x1, x1-x2) 1
?2
x1
?3
The linear combination x1-x2 contains the same
information as x1and x2 separately
? 4
x1-x2(LV)
?1
?2
x2
?3
cor (x2, x1-x2) -1
? 4
10
Correlation is symmetrical in the involved
variables
Direction independent
11
Latent variables are also symmetrical in the
variables
In addition, they can handle an unlimited number
of variables simultaneously!
12
x1 f (LV) f (x1 - x2) x2 g (LV) g (x1 -
x2)
13
Information and variation
PC1 p1e1 p2e2 Principal Component (PC)
14
Latent variable (LV)
LV w1e1 w2e2.. wMeM
Shows common (shared) variance (information!) in
a suite of variables !
15
Key steps in the development of latent variable
analysis (LVA)
i) Correlation (Galton) ii) Factors, expressions
of partial correlations (Spearman, Pearson,
Thurstone) Psychometrics
16
Francis Galton, Proc. Royal Society of London, XL
XLV. C.Spearman, General Intelligence,
objectively determined and measured, American
Journal of Psychology, 15 (1904)
201-293. L.L.Thurstone, Multiple factor analysis,
Psycological Review, 38 (1931) 406-427.
17
the length of the arm is said to be correlated
with that of the leg, because a person with a
long arm has usually a long leg and
conversely. Sir Francis Galton, Proc. Royal
Society of London, XL XLV. (1886)
18
The hidden underlying causes of
associations C. Spearman, The proof and
measurement of association between two things,
American Journal of Psycology, 15 (1904) 72-101.
19
Spearman (1904) General factor
(intelligence) Common source of
variation hidden underlying cause of the
variations. LATENT INFORMATION!
20
Partial correlation between measured variables
and latent variables (factors)
21
we can arrive at estimating the correspondence
of whatever may be common to the first pair of
faculties with whatever may be common to the
second pair. By combining such correlations of
higher order, it is feasible to execute any
required amount of elimination and selection, so
that eventually a dissociation and exactness may
be introduced into psychology such as can only be
compared with quantitative chemical
analysis Spearman (1904), American J.
Psychology, p. 259
22
Some time ago a psychologist told me that
factors where meaningless abstractions and that
people who found them where foolish to do so. The
chief point of this article is to show that
factors are meaningful abstractions, but that
people are foolish if they dont find
them. K.J.Holzinger, Why do people factor?,
Psychometrika, 7 (1942) 147-156.
23
Let us not forget a simple principle that every
scientist takes for granted, namely, that he
would rather measure something significant
without any sampling distribution than to measure
something trivial or irrelevant because its
sampling distribution is known. L.L.Thurstone,
Multiple-factor Analysis, The University of
Chicago Press, Chicago and London, 1947