View by Category

Loading...

PPT – Eigen Decomposition and Singular Value Decomposition PowerPoint presentation | free to view

The Adobe Flash plugin is needed to view this content

About This Presentation

Write a Comment

User Comments (0)

Transcript and Presenter's Notes

Eigen Decomposition and Singular Value

Decomposition

- Mani Thomas
- CISC 489/689

Introduction

- Eigenvalue decomposition
- Spectral decomposition theorem
- Physical interpretation of eigenvalue/eigenvectors

- Singular Value Decomposition
- Importance of SVD
- Matrix inversion
- Solution to linear system of equations
- Solution to a homogeneous system of equations
- SVD application

What are eigenvalues?

- Given a matrix, A, x is the eigenvector and ? is

the corresponding eigenvalue if Ax ?x - A must be square the determinant of A - ? I must

be equal to zero - Ax - ?x 0 ! x(A - ?I) 0
- Trivial solution is if x 0
- The non trivial solution occurs when det(A - ?I)

0 - Are eigenvectors are unique?
- If x is an eigenvector, then ?x is also an

eigenvector and ?? is an eigenvalue - A(?x) ?(Ax) ?(?x) ?(?x)

Calculating the Eigenvectors/values

- Expand the det(A - ?I) 0 for a 2 2 matrix
- For a 2 2 matrix, this is a simple quadratic

equation with two solutions (maybe complex) - This characteristic equation can be used to

solve for x

Eigenvalue example

- Consider,
- The corresponding eigenvectors can be computed as
- For ? 0, one possible solution is x (2, -1)
- For ? 5, one possible solution is x (1, 2)

For more information Demos in Linear algebra by

G. Strang, http//web.mit.edu/18.06/www/

Physical interpretation

- Consider a correlation matrix, A
- Error ellipse with the major axis as the larger

eigenvalue and the minor axis as the smaller

eigenvalue

Physical interpretation

- Orthogonal directions of greatest variance in

data - Projections along PC1 (Principal Component)

discriminate the data most along any one axis

Physical interpretation

- First principal component is the direction of

greatest variability (covariance) in the data - Second is the next orthogonal (uncorrelated)

direction of greatest variability - So first remove all the variability along the

first component, and then find the next direction

of greatest variability - And so on
- Thus each eigenvectors provides the directions of

data variances in decreasing order of eigenvalues

For more information See Gram-Schmidt

Orthogonalization in G. Strangs lectures

Spectral Decomposition theorem

- If A is a symmetric and positive definite k k

matrix (xTAx gt 0) with ?i (?i gt 0) and ei, i 1

? k being the k eigenvector and eigenvalue pairs,

then - This is also called the eigen decomposition

theorem - Any symmetric matrix can be reconstructed using

its eigenvalues and eigenvectors - Any similarity to what has been discussed before?

Example for spectral decomposition

- Let A be a symmetric, positive definite matrix
- The eigenvectors for the corresponding

eigenvalues are - Consequently,

Singular Value Decomposition

- If A is a rectangular m k matrix of real

numbers, then there exists an m m orthogonal

matrix U and a k k orthogonal matrix V such

that - ? is an m k matrix where the (i, j)th entry ?i

0, i 1 ? min(m, k) and the other entries are

zero - The positive constants ?i are the singular values

of A - If A has rank r, then there exists r positive

constants ?1, ?2,??r, r orthogonal m 1 unit

vectors u1,u2,?,ur and r orthogonal k 1 unit

vectors v1,v2,?,vr such that - Similar to the spectral decomposition theorem

Singular Value Decomposition (contd.)

- If A is a symmetric and positive definite then
- SVD Eigen decomposition
- EIG(?i) SVD(?i2)
- Here AAT has an eigenvalue-eigenvector pair

(?i2,ui) - Alternatively, the vi are the eigenvectors of ATA

with the same non zero eigenvalue ?i2

Example for SVD

- Let A be a symmetric, positive definite matrix
- U can be computed as
- V can be computed as

Example for SVD

- Taking ?2112 and ?2210, the singular value

decomposition of A is - Thus the U, V and ? are computed by performing

eigen decomposition of AAT and ATA - Any matrix has a singular value decomposition but

only symmetric, positive definite matrices have

an eigen decomposition

Applications of SVD in Linear Algebra

- Inverse of a n n square matrix, A
- If A is non-singular, then A-1 (U?VT)-1

V?-1UT where - ?-1diag(1/?1, 1/?1,?, 1/?n)
- If A is singular, then A-1 (U?VT)-1¼ V?0-1UT

where - ?0-1diag(1/?1, 1/?2,?, 1/?i,0,0,?,0)
- Least squares solutions of a mn system
- Axb (A is mn, mn) (ATA)xATb ) x(ATA)-1

ATbAb - If ATA is singular, xAb¼ (V?0-1UT)b where ?0-1

diag(1/?1, 1/?2,?, 1/?i,0,0,?,0) - Condition of a matrix
- Condition number measures the degree of

singularity of A - Larger the value of ?1/?n, closer A is to being

singular

http//www.cse.unr.edu/bebis/MathMethods/SVD/lect

ure.pdf

Applications of SVD in Linear Algebra

- Homogeneous equations, Ax 0
- Minimum-norm solution is x0 (trivial solution)
- Impose a constraint,
- Constrained optimization problem
- Special Case
- If rank(A)n-1 (m n-1, ?n0) then x? vn (? is

a constant) - Genera Case
- If rank(A)n-k (m n-k, ?n-k1? ?n0) then

x?1vn-k1??kvn with ?21??2n1

- Has appeared before
- Homogeneous solution of a linear system of

equations - Computation of Homogrpahy using DLT
- Estimation of Fundamental matrix

For proof Johnson and Wichern, Applied

Multivariate Statistical Analysis, pg 79

What is the use of SVD?

- SVD can be used to compute optimal low-rank

approximations of arbitrary matrices. - Face recognition
- Represent the face images as eigenfaces and

compute distance between the query face image in

the principal component space - Data mining
- Latent Semantic Indexing for document extraction
- Image compression
- Karhunen Loeve (KL) transform performs the best

image compression - In MPEG, Discrete Cosine Transform (DCT) has the

closest approximation to the KL transform in PSNR

Image Compression using SVD

- The image is stored as a 256264 matrix M with

entries between 0 and 1 - The matrix M has rank 256
- Select r 256 as an approximation to the

original M - As r in increased from 1 all the way to 256 the

reconstruction of M would improve i.e.

approximation error would reduce - Advantage
- To send the matrix M, need to send 256264

67584 numbers - To send an r 36 approximation of M, need to

send 36 36256 36264 18756 numbers - 36 singular values
- 36 left vectors, each having 256 entries
- 36 right vectors, each having 264 entries

Courtesy http//www.uwlax.edu/faculty/will/svd/co

mpression/index.html

About PowerShow.com

You can use PowerShow.com to find and download example online PowerPoint ppt presentations on just about any topic you can imagine so you can learn how to improve your own slides and presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

Recommended

«

/ »

Page of

«

/ »

Promoted Presentations

Related Presentations

Page of

Page of

CrystalGraphics Sales Tel: (800) 394-0700 x 1 or Send an email

Home About Us Terms and Conditions Privacy Policy Contact Us Send Us Feedback

Copyright 2014 CrystalGraphics, Inc. — All rights Reserved. PowerShow.com is a trademark of CrystalGraphics, Inc.

Copyright 2014 CrystalGraphics, Inc. — All rights Reserved. PowerShow.com is a trademark of CrystalGraphics, Inc.

The PowerPoint PPT presentation: "Eigen Decomposition and Singular Value Decomposition" is the property of its rightful owner.

Do you have PowerPoint slides to share? If so, share your PPT presentation slides online with PowerShow.com. It's FREE!