View by Category

Loading...

PPT – FeedForward Neural Networks PowerPoint presentation | free to view - id: 1bf8bb-ZDE4N

The Adobe Flash plugin is needed to view this content

About This Presentation

Write a Comment

User Comments (0)

Transcript and Presenter's Notes

Feed-Forward Neural Networks

- ??? ???

Content

- Introduction
- Single-Layer Perceptron Networks
- Learning Rules for Single-Layer Perceptron

Networks - Perceptron Learning Rule
- Adline Leaning Rule

Feed-Forward Neural Networks

- Introduction

Historical Background

- 1943 McCulloch and Pitts proposed the first

computational models of neuron. - 1949 Hebb proposed the first learning rule.
- 1958 Rosenblatts work in perceptrons.
- 1969 Minsky and Paperts exposed limitation of

the theory. - 1970s Decade of dormancy for neural networks.
- 1980-90s Neural network return (self-organization,

back-propagation algorithms, etc)

Nervous Systems

- Human brain contains 1011 neurons.
- Each neuron is connected 104 others.
- Some scientists compared the brain with a

complex, nonlinear, parallel computer. - The largest modern neural networks achieve the

complexity comparable to a nervous system of a

fly.

Neurons

- The main purpose of neurons is to receive,

analyze and transmit further the information in a

form of signals (electric pulses). - When a neuron sends the information we say that a

neuron fires.

Neurons

Acting through specialized projections known as

dendrites and axons, neurons carry information

throughout the neural network.

This animation demonstrates the firing of a

synapse between the pre-synaptic terminal of one

neuron to the soma (cell body) of another neuron.

A Model of Artificial Neuron

A Model of Artificial Neuron

Feed-Forward Neural Networks

- Graph representation
- nodes neurons
- arrows signal flow directions
- A neural network that does not contain cycles

(feedback loops) is called a feedforward network

(or perceptron).

Layered Structure

Hidden Layer(s)

Knowledge and Memory

- The output behavior of a network is determined by

the weights. - Weights ? the memory of an NN.
- Knowledge ? distributed across the network.
- Large number of nodes
- increases the storage capacity
- ensures that the knowledge is robust
- fault tolerance.
- Store new information by changing weights.

Pattern Classification

output pattern y

- Function x ? y
- The NNs output is used to distinguish between

and recognize different input patterns. - Different output patterns correspond to

particular classes of input patterns. - Networks with hidden layers can be used for

solving more complex problems then just a linear

pattern classification.

input pattern x

Training

Training Set

. . .

. . .

Goal

. . .

. . .

Generalization

- By properly training a neural network may produce

reasonable answers for input patterns not seen

during training (generalization). - Generalization is particularly useful for the

analysis of a noisy data (e.g. timeseries).

Generalization

- By properly training a neural network may produce

reasonable answers for input patterns not seen

during training (generalization). - Generalization is particularly useful for the

analysis of a noisy data (e.g. timeseries).

Applications

- Pattern classification
- Object recognition
- Function approximation
- Data compression
- Time series analysis and forecast
- . . .

Feed-Forward Neural Networks

- Single-Layer Perceptron Networks

The Single-Layered Perceptron

The Single-Layered Perceptron

Training a Single-Layered Perceptron

Training Set

Goal

Learning Rules

- Linear Threshold Units (LTUs) Perceptron

Learning Rule - Linearly Graded Units (LGUs) Widrow-Hoff

learning Rule

Training Set

Goal

Feed-Forward Neural Networks

- Learning Rules for
- Single-Layered Perceptron Networks
- Perceptron Learning Rule
- Adline Leaning Rule

Perceptron

Linear Threshold Unit

sgn

Perceptron

Goal

Linear Threshold Unit

sgn

Example

Goal

Class 1

g(x) ?2x1 2x220

Class 2

Augmented input vector

Goal

Class 1 (1)

Class 2 (?1)

Augmented input vector

Goal

Augmented input vector

Goal

A plane passes through the origin in the

augmented input space.

Linearly Separable vs. Linearly Non-Separable

AND

OR

XOR

Linearly Separable

Linearly Separable

Linearly Non-Separable

Goal

- Given training sets T1?C1 and T2 ? C2 with

elements in form of x(x1, x2 , ... , xm-1 , xm)

T , where x1, x2 , ... , xm-1 ?R and xm ?1. - Assume T1 and T2 are linearly separable.
- Find w(w1, w2 , ... , wm) T such that

Goal

wTx 0 is a hyperplain passes through the origin

of augmented input space.

- Given training sets T1?C1 and T2 ? C2 with

elements in form of x(x1, x2 , ... , xm-1 , xm)

T , where x1, x2 , ... , xm-1 ?R and xm ?1. - Assume T1 and T2 are linearly separable.
- Find w(w1, w2 , ... , wm) T such that

Observation

Which ws correctly classify x?

What trick can be used?

Observation

Is this w ok?

w1x1 w2x2 0

Observation

w1x1 w2x2 0

Is this w ok?

Observation

w1x1 w2x2 0

Is this w ok?

How to adjust w?

?w ?

Observation

Is this w ok?

How to adjust w?

?w ??x

reasonable?

gt0

lt0

Observation

Is this w ok?

reasonable?

How to adjust w?

?w ?x

gt0

lt0

Observation

Is this w ok?

?

?w ?

?x

??x

or

Perceptron Learning Rule

Upon misclassification on

Define error

Perceptron Learning Rule

Define error

Perceptron Learning Rule

Summary ? Perceptron Learning Rule

Based on the general weight learning rule.

correct

incorrect

Summary ? Perceptron Learning Rule

Converge?

Perceptron Convergence Theorem

- Exercise Reference some papers or textbooks to

prove the theorem.

If the given training set is linearly separable,

the learning process will converge in a finite

number of steps.

The Learning Scenario

Linearly Separable.

The Learning Scenario

The Learning Scenario

The Learning Scenario

The Learning Scenario

The Learning Scenario

w4 w3

w3

The Learning Scenario

w

The Learning Scenario

The demonstration is in augmented space.

w

Conceptually, in augmented space, we adjust the

weight vector to fit the data.

Weight Space

A weight in the shaded area will give correct

classification for the positive example.

w

Weight Space

A weight in the shaded area will give correct

classification for the positive example.

?w ?x

w

Weight Space

A weight not in the shaded area will give correct

classification for the negative example.

w

Weight Space

A weight not in the shaded area will give correct

classification for the negative example.

w

?w ??x

The Learning Scenario in Weight Space

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w2

w1

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w2

w1

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w2

w2

w1

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w2

w2

w3

w1

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w2

w4

w2

w3

w1

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w2

w4

w2

w3

w5

w1

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w2

w4

w2

w3

w5

w1

w6

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w2

w7

w4

w2

w3

w5

w1

w6

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w2

w8

w7

w4

w2

w3

w5

w1

w6

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w9

w2

w8

w7

w4

w2

w3

w5

w1

w6

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w9

w10

w2

w8

w7

w4

w2

w3

w5

w1

w6

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w9

w10

w2

w11

w8

w7

w4

w2

w3

w5

w1

w6

w1

w0

The Learning Scenario in Weight Space

To correctly classify the training set, the

weight must move into the shaded area.

w2

w11

w1

w0

Conceptually, in weight space, we move the weight

into the feasible region.

Feed-Forward Neural Networks

- Learning Rules for
- Single-Layered Perceptron Networks
- Perceptron Learning Rule
- Adline Leaning Rule

Adline (Adaptive Linear Element)

Widrow 1962

Adline (Adaptive Linear Element)

In what condition, the goal is reachable?

Goal

Widrow 1962

LMS (Least Mean Square)

Minimize the cost function (error function)

Gradient Decent Algorithm

Our goal is to go downhill.

Contour Map

?w

(w1, w2)

Gradient Decent Algorithm

Our goal is to go downhill.

How to find the steepest decent direction?

Contour Map

?w

(w1, w2)

Gradient Operator

Let f(w) f (w1, w2,, wm) be a function over Rm.

Define

Gradient Operator

df positive

df zero

df negative

Go uphill

Plain

Go downhill

The Steepest Decent Direction

To minimize f , we choose ?w ?? ? f

df positive

df zero

df negative

Go uphill

Plain

Go downhill

LMS (Least Mean Square)

Minimize the cost function (error function)

Adline Learning Rule

Minimize the cost function (error function)

Learning Modes

- Batch Learning Mode
- Incremental Learning Mode

Comparisons

Habbian Assumption

Gradient Decent

Fundamental

Converge Asymptotically

Convergence

In finite steps

Linearly Separable

Linear Independence

Constraint

About PowerShow.com

PowerShow.com is a leading presentation/slideshow sharing website. Whether your application is business, how-to, education, medicine, school, church, sales, marketing, online training or just for fun, PowerShow.com is a great resource. And, best of all, most of its cool features are free and easy to use.

You can use PowerShow.com to find and download example online PowerPoint ppt presentations on just about any topic you can imagine so you can learn how to improve your own slides and presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

You can use PowerShow.com to find and download example online PowerPoint ppt presentations on just about any topic you can imagine so you can learn how to improve your own slides and presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

presentations for free. Or use it to find and download high-quality how-to PowerPoint ppt presentations with illustrated or animated slides that will teach you how to do something new, also for free. Or use it to upload your own PowerPoint slides so you can share them with your teachers, class, students, bosses, employees, customers, potential investors or the world. Or use it to create really cool photo slideshows - with 2D and 3D transitions, animation, and your choice of music - that you can share with your Facebook friends or Google+ circles. That's all free as well!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

For a small fee you can get the industry's best online privacy or publicly promote your presentations and slide shows with top rankings. But aside from that it's free. We'll even convert your presentations and slide shows into the universal Flash format with all their original multimedia glory, including animation, 2D and 3D transition effects, embedded music or other audio, or even video embedded in slides. All for free. Most of the presentations and slideshows on PowerShow.com are free to view, many are even free to download. (You can choose whether to allow people to download your original PowerPoint presentations and photo slideshows for a fee or free or not at all.) Check out PowerShow.com today - for FREE. There is truly something for everyone!

Recommended

«

/ »

Page of

«

/ »

Promoted Presentations

Related Presentations

Page of

Home About Us Terms and Conditions Privacy Policy Contact Us Send Us Feedback

Copyright 2017 CrystalGraphics, Inc. — All rights Reserved. PowerShow.com is a trademark of CrystalGraphics, Inc.

Copyright 2017 CrystalGraphics, Inc. — All rights Reserved. PowerShow.com is a trademark of CrystalGraphics, Inc.

The PowerPoint PPT presentation: "FeedForward Neural Networks" is the property of its rightful owner.

Do you have PowerPoint slides to share? If so, share your PPT presentation slides online with PowerShow.com. It's FREE!

Committed to assisting Ttu University and other schools with their online training by sharing educational presentations for free