Neural Networks

John Riebe and Adam Profitt

What is a neuron?

PR Elements of the input vector W Weights ? Summer

b Bias n Sum of all P elements and

b ƒ Translation Function a Output

Weights Weights are scalars that multiply each

input element Summer The summer sums the input

elements, PR, together with the bias Bias A

bias is a number that is added to the total from

the summer Translation Function A translation

function is one of many specific functions used

in neural networking.

Layers of the Neural Network

- There are only three different types of layers in

a network - The Input Layer
- Moves the input vectors into each neuron of the

first hidden layer - The Hidden Layers
- Performs the bulk of the computations in most

networks - Hidden layers are not always required
- The Output Layer
- Each neuron in the output layer outputs its own

result

Translation Functions

Types of Neural Networks

- Perceptrons
- Used to classify data.
- Applies the hard-limit transfer function.
- Usually does not have any hidden layers.

- Linear Filters
- Used to solve linearly separable problems.
- Applies the linear transfer function.

- Backpropagation
- Generally has only one hidden layer.
- Can solve any reasonable problem.
- Hidden layers use sigmoid translations, outputs

use the linear transfer function

Training Neurons

Training a network sets the biases and weights in

each neuron

- To train a network you need
- A network
- An input
- A target vector

- There are many different types of
- training algorithms. To name a few
- Levenberg-Marquardt
- BFGS quasi-Newton
- Bayesian regularization
- One step secant
- Random order incremental

- Training algorithms
- Gives a network an input
- Receives the output
- Calculates error between output and target
- Adjusts weights and biases
- Goes back to step 1

Each time the algorithm goes through the steps is

called an epoch. Most networks go through many

epochs.

MatlabApplication

The newff Function

- Create a feed-forward network
- Syntax
- net newff
- net newff(PR,S1 S2...Si,TF1 TF2...TFi)
- Description
- net newff creates a new network with a dialog

box. - newff(PR,S1 S2...Si,TF1 TF2...TFi) takes,
- PR - R x 2 matrix of min and max values for R

input elements. - Si - Size of ith layer, for Nl layers.
- TFi - Transfer function of ith layer, default

'tansig'.

The train Function

Trains a neural network Syntax net

train(net,P,T) Description train trains a

network. train(net,P,T) takes, net - Neural

network object. P - Network inputs. T - Network

targets, default zeros.

The sim Function

The sim function simulates a neural network. This

function feeds the network the input, P, and

displays the results.

Syntaxsim(net,P) Descriptionsim simulates

neural networks. sim(net,P) takes, net - Network.

P - Network inputs.

Transfer Functions Revisited

- Transfer functions
- Hard-Limit
- a hardlim(n)

Outputs either a 1 or a 0

- Linear
- a purelin(n)

Outputs the scaled and summed input

- Log-Sigmoid
- a logsig(n)

Squeezes the input to between 0 and 1

- Tan-Sigmoid
- a tansig(n)

Squeezes the input to between -1 and 1

The Baum-Haussler Rule

The Baum-Haussler Rule is one of the most useful

rules for neural networks.

Nhidden (Ntrain Etolerance) / (Npts

Noutputs)

This rule helps you determine the maximum number

of neurons you will need for your network to

function properly.

This is NOT a law it will not work in all

situations. Sometimes you just have to use

another method.

Bibliography

Demuth, Howard and Mark Beale. Neural Network

Toolbox Users Guide. 1992-2003 URL

http//www.mathworks.com/access/helpdesk/help/tool

box/nnet/nnet.shtml