Principal Component Analysis

1
Principal Component Analysis

• IML 2004-5

2
Outline

• Max the variance of the output coordinates
• Optimal reconstruction
• Generating data
• Limitations of PCA

3
Eigenfaces
Variance in Face Pictures

• Figure/ground
• Orientation
• Lighting
• Hairline

4
Eigenfaces
100 images30x30 pixels
5
Maximizing Output Variance
The first eigenvector (highest eigenvalue)
characterizes the maximal variance in the image
figure - background
6
Maximizing Output Variance
The second eigenvector characterizes right
orientation
7
Maximizing Output Variance
8
Maximizing Output Variance
9
Optimal Reconstruction
q1
q2
q4
q8
Original Image
q16
q32
q64
q100
10
If ngtgtm
e.g. n80x80 pixels gtgt m100 images Problem
finding the eigenvectors of a 6400x6400 matrix
O(64003) Solution extract the eigenvectors Q of
ATA
11
If ngtm
q1
q2
q4
q8
Original Image
q16
q32
q64
q100
12
Generating Data
13
(No Transcript)
14
Generating Data
15
Kernel PCA
16
Limits of PCA
Should the goal be finding independent rather
than pair-wise uncorrelated dimensions

• Independent Component Analysis (ICA)

PCA
ICA
17
Limits of PCA
Are the maximal variance dimensions the relevant
dimensions for preservation?

• Relevant Component Analysis (RCA)
• Fisher Discriminant analysis (FDA)