Artificial Neural Networks - PowerPoint PPT Presentation

1 / 27
About This Presentation
Title:

Artificial Neural Networks

Description:

EECP0720 Expert Systems Artificial Neural Networks Artificial Neural Networks Sanun Srisuk 42973003 sanun_at_mut.ac.th Introduction Artificial neural networks (ANNs ... – PowerPoint PPT presentation

Number of Views:356
Avg rating:3.0/5.0
Slides: 28
Provided by: sris2
Category:

less

Transcript and Presenter's Notes

Title: Artificial Neural Networks


1
Artificial Neural Networks
EECP0720 Expert Systems Artificial Neural
Networks
  • Sanun Srisuk 42973003
  • sanun_at_mut.ac.th

2
Introduction
EECP0720 Expert Systems Artificial Neural
Networks
  • Artificial neural networks (ANNs) provide a
    general, practical method for learning
    real-valued, discrete-valued, and vector-valued
    functions from examples. Algorithms such as
    BACKPROPAGATION use gradient descent to tune
    network parameters to best fit a training set of
    input-output pairs. ANN learning is robust to
    errors in the training data and has been
    successfully applied to problems such as face
    recognition/detection, speech recognition, and
    learning robot control strategies.

3
Autonomous Vehicle Steering
EECP0720 Expert Systems Artificial Neural
Networks
4
Characteristics of ANNs
EECP0720 Expert Systems Artificial Neural
Networks
  • Instances are represented by many attribute-value
    pairs.
  • The target function output may be
    discrete-valued, real-valued, or a vector of
    several real- or discrete-valued attributes.
  • The training examples may contain errors.
  • Long training times are acceptable.
  • Fast evaluation of the learned target function
    may be required.
  • The ability of humans to understand the learned
    target function is not important.

5
Perceptrons
EECP0720 Expert Systems Artificial Neural
Networks
  • One type of ANN system is based on a unit called
    a perceptron.
  • The perceptron function can sometimes be written
    as
  • The space H of candidate hypotheses considered in
    perceptron learning is the set of all possible
    real-valued weight vectors.

6
Representational Power of Perceptrons
EECP0720 Expert Systems Artificial Neural
Networks
7
Decision surface
EECP0720 Expert Systems Artificial Neural
Networks
linear decision surface
nonlinear decision surface
Programming Example of Decision Surface
8
The Perceptron Training Rule
EECP0720 Expert Systems Artificial Neural
Networks
  • One way to learn an acceptable weight vector is
    to begin with random weights, then iteratively
    apply the perceptron to each training example,
    modifying the perceptron weights whenever it
    misclassifies an example. This process is
    repeated, iterating through the training examples
    as many times as needed until the perceptron
    classifies all training examples correctly.
    Weights are modified at each step according to
    the perceptron training rule, which revises the
    weight associated with input according to
    the rule

9
Gradient Descent and Delta Rule
EECP0720 Expert Systems Artificial Neural
Networks
  • The delta training rule is best understood by
    considering the task of training an unthresholded
    perceptron that is, a linear unit for which the
    output o is given by
  • In order to derive a weight learning rule for
    linear units, let us begin by specifying a
    measure for the training error of a hypothesis
    (weight vector), relative to the training
    examples.

10
Visualizing the Hypothesis Space
EECP0720 Expert Systems Artificial Neural
Networks
initial weight vector by random
minimum error
11
Derivation of the Gradient Descent Rule
EECP0720 Expert Systems Artificial Neural
Networks
  • The vector derivative is called the gradient of E
    with respect to , written
  • The gradient specifies the direction that
    produces the steepest increase in E. The negative
    of this vector therefore gives the direction of
    steepest decrease. The training rule for gradient
    descent is

12
Derivation of the Gradient Descent Rule (cont.)
EECP0720 Expert Systems Artificial Neural
Networks
  • The negative sign is presented because we want to
    move the weight vector in the direction that
    decreases E. This training rule can also written
    in its component form
  • which makes it clear that steepest descent is
    achieved by altering each component of in
    proportion to .

13
Derivation of the Gradient Descent Rule (cont.)
EECP0720 Expert Systems Artificial Neural
Networks
  • The vector of derivatives that form the
    gradient can be obtained by differentiating E

The weight update rule for standard gradient
descent can be summarized as
14
Stochastic Approximation to Gradient Descent
EECP0720 Expert Systems Artificial Neural
Networks
15
Summary of Perceptron
EECP0720 Expert Systems Artificial Neural
Networks
  • Perceptron training rule guaranteed to succeed if
  • training examples are linearly separable
  • sufficiently small learning rate
  • Linear unit training rule uses gradient descent
  • guaranteed to converge to hypothesis with minimum
    squared error
  • given sufficiently small learning rate
  • even when training data contains noise

16
BACKPROPAGATION Algorithm
EECP0720 Expert Systems Artificial Neural
Networks
17
Error Function
EECP0720 Expert Systems Artificial Neural
Networks
  • The Backpropagation algorithm learns the weights
    for a multilayer network, given a network with a
    fixed set of units and interconnections. It
    employs gradient descent to attempt to minimize
    the squared error between the network output
    values and the target values for those outputs.
    We begin by redefining E to sum the errors over
    all of the network output units
  • where outputs is the set of output units in the
    network, and tkd and okd are the target and
    output values associated with the kth output unit
    and training example d.

18
Architecture of Backpropagation
EECP0720 Expert Systems Artificial Neural
Networks
19
Backpropagation Learning Algorithm
EECP0720 Expert Systems Artificial Neural
Networks
20
Backpropagation Learning Algorithm (cont.)
EECP0720 Expert Systems Artificial Neural
Networks
21
Backpropagation Learning Algorithm (cont.)
EECP0720 Expert Systems Artificial Neural
Networks
22
Backpropagation Learning Algorithm (cont.)
EECP0720 Expert Systems Artificial Neural
Networks
23
Backpropagation Learning Algorithm (cont.)
EECP0720 Expert Systems Artificial Neural
Networks
24
Face Detection using Neural Networks
EECP0720 Expert Systems Artificial Neural
Networks
Training Process
Face Database
Output1, for face database
Non-Face Database
Neural Network
Face or
Non-Face?
Output0, for non-face database
Testing Process
25
End of Presentation
EECP0720 Expert Systems Artificial Neural
Networks
26
Derivation of Backpropagation
EECP0720 Expert Systems Artificial Neural
Networks
27
Derivation of Backpropagation (cont.)
EECP0720 Expert Systems Artificial Neural
Networks
Write a Comment
User Comments (0)
About PowerShow.com