Unsupervised feature learning for audio classification using convolutional deep belief networks - PowerPoint PPT Presentation


Title: Unsupervised feature learning for audio classification using convolutional deep belief networks


1
Unsupervised feature learning for audio
classification using convolutional deep belief
networks
  • Honglak Lee, Yan Largman, Peter Pham and Andrew
    Y. Ng

Presented by Bo Chen, 5.7,2010
2
Outline
  • 1. Whats Deep Learning?
  • 2. Why use Deep Learning?
  • 3. Foundations of Deep Learning
  • 4. Convolutional Deep Belief Networks
  • 5. Results

3
Deep Architecture
  • Deep architectures compositions of many layers
    of adaptive non-linear components.
  • Difficulty parameter searching (local minima)
  • Deep belief nets probabilistic generative models
    that are composed of multiple layers of
    stochastic, latent variables. (Hinton et al.,
    2006)

Deep Learning Wiki
4
Why Use Deep Learning
  • Insufficient depth can hurt
  • Usually our experiences tell us that one-layer
    machine only gives us a set of general dictionary
    elements, unless a huge number of dictionary
    elements.
  • The brain has a deep architecture
  • Cognitive processes seem deep
  • Learn a feature hierarchies or the complicated
    functions that can represent high-level
    abstractions
  • For example,
  • Pixels?Edglets?Motifs?Parts?Objects?Sc
    enes

Some from Yoshua Bengios course notes and Yann
Lecun, et.al.,2010
5
One-layer dictionary
30 16x16 dictionary elements and reconstructed
images
250 16x16 dictionary elements and reconstructed
images
6
Restricted Boltzmann Machine
Binary-valued
Energy function
Real-valued
Contrastive divergence is used to solve the
problem. (Hinton et al., 2006)
Figure from R Salakhutdinov et. al. 
7
Deep Architectures
RBM in the different layers can be independently
trained.
8
Convolutional Network Architecture
Intuitively, in each layer the weight matrix will
catch the most consistent structures through
all of the images.
Figure from Yann LeCun et. al, 1998
9
3-dimensional Dictionary elements in the second
layer
D the first-layer dictionary element E the
second-layer dictionary element S the
convolution of the image and the first-layer
elements.
The dictionary element in the second layer is a
3-dimensional matrix.
10
Convolutional RBM with Probabilistic Max-Pooling
Layer
Max-pooling Layer
11
Convolutional Deep Belief Networks
the weight matrix Connecting pooling unit Pk to
detection unit Hl.
12
Results on Natural Images
13
Results Caltech101 Images
View by Category
About This Presentation
Title:

Unsupervised feature learning for audio classification using convolutional deep belief networks

Description:

Unsupervised feature learning for audio classification using convolutional deep belief networks Honglak Lee, Yan Largman, Peter Pham and Andrew Y. Ng – PowerPoint PPT presentation

Number of Views:8
Avg rating:3.0/5.0
Slides: 14
Provided by: BoC55
Learn more at: http://www.ece.duke.edu
Category:

less

Write a Comment
User Comments (0)
Transcript and Presenter's Notes

Title: Unsupervised feature learning for audio classification using convolutional deep belief networks


1
Unsupervised feature learning for audio
classification using convolutional deep belief
networks
  • Honglak Lee, Yan Largman, Peter Pham and Andrew
    Y. Ng

Presented by Bo Chen, 5.7,2010
2
Outline
  • 1. Whats Deep Learning?
  • 2. Why use Deep Learning?
  • 3. Foundations of Deep Learning
  • 4. Convolutional Deep Belief Networks
  • 5. Results

3
Deep Architecture
  • Deep architectures compositions of many layers
    of adaptive non-linear components.
  • Difficulty parameter searching (local minima)
  • Deep belief nets probabilistic generative models
    that are composed of multiple layers of
    stochastic, latent variables. (Hinton et al.,
    2006)

Deep Learning Wiki
4
Why Use Deep Learning
  • Insufficient depth can hurt
  • Usually our experiences tell us that one-layer
    machine only gives us a set of general dictionary
    elements, unless a huge number of dictionary
    elements.
  • The brain has a deep architecture
  • Cognitive processes seem deep
  • Learn a feature hierarchies or the complicated
    functions that can represent high-level
    abstractions
  • For example,
  • Pixels?Edglets?Motifs?Parts?Objects?Sc
    enes

Some from Yoshua Bengios course notes and Yann
Lecun, et.al.,2010
5
One-layer dictionary
30 16x16 dictionary elements and reconstructed
images
250 16x16 dictionary elements and reconstructed
images
6
Restricted Boltzmann Machine
Binary-valued
Energy function
Real-valued
Contrastive divergence is used to solve the
problem. (Hinton et al., 2006)
Figure from R Salakhutdinov et. al. 
7
Deep Architectures
RBM in the different layers can be independently
trained.
8
Convolutional Network Architecture
Intuitively, in each layer the weight matrix will
catch the most consistent structures through
all of the images.
Figure from Yann LeCun et. al, 1998
9
3-dimensional Dictionary elements in the second
layer
D the first-layer dictionary element E the
second-layer dictionary element S the
convolution of the image and the first-layer
elements.
The dictionary element in the second layer is a
3-dimensional matrix.
10
Convolutional RBM with Probabilistic Max-Pooling
Layer
Max-pooling Layer
11
Convolutional Deep Belief Networks
the weight matrix Connecting pooling unit Pk to
detection unit Hl.
12
Results on Natural Images
13
Results Caltech101 Images
About PowerShow.com