Lecture 3' Mathematical Models of Neurocomputing

1
Lecture 3. Mathematical Models of
Neurocomputing
Saint-Petersburg State Polytechnic
University Distributed Intelligent Systems
Dept ARTIFICIAL NEURAL NETWORKS

• Prof. Dr. Viacheslav P. Shkodyrev
• EMail shkodyrev_at_imop.spbstu.ru

Neurocomputer may be described as a
mathematical model which has been created in the
image and likeness of human brain. Mathematicians
suggest dramatically new high-performance models
of massive parallelism in data processing and
data storage, ways of analysis and calculus
results interpretation, formal models creation
and knowledge data, artificial intelligent
systems.
2
Really surprising features we get when
compose single neurons into a neural network.
In spite of apparent simplicity of each
individual neuron their interconnected
assembling or composition (to network) leads to
principally new computational models.

• Objective of Lecture 3
• The main objective of this lecture is to
  discuss the background of artificial neural net
  theory - basic models of ANN and their
  mathematical descriptions. You will learn
• Structure and mathematical description of
  Neural Networks
• One layer linear and non-linear networks
• Multilayer networks
• Topology of Neural Networks
• Learning paradigm of Neural Networks

3
(No Transcript)
4
Scalar product of vectors
Neuron activation
5
(No Transcript)
6
To introduce dependence on time, we define the
activation implicity by its rate of change
Linear dynamic model
Non-Linear dynamic model
where
where u internal potential
7
Mathematical description
Topology of Neural Network
Input pattern
or, in matrix form
- Outputs vector
where
8
Topology of Neural Network
The processing ability of networks is stored in
the inter-unit connections strengths, or weights,
are turned by a process of adaptation to, or
learning from a set of training patterns.
9
Topology of Multilayer Neural Networks (detail
model)
One of the simplest but a large class of NN
is so-named feed-forward net which can be
presented as a structure of neurons arranged in
layer. Layer is a simplest structure where all
neurons are connected to all neurons in the
adjacent (???????, ????????, ???????????) layers
through uni-directional links.
Typically, we have layer 0 (sometimes called the
input layer) which do not perform computation,
but only feed input signals to the neurons of
layer 1 called the first hidden layer. The last
layer (layer 3) is the output layer where the
response of the network comes from.
10
Topology of Multilayer Neural Networks (General
model)
Mathematical description
where
3-D Synaptic weights matrix
11
In Feed forward NN artificial neurons are
arranged in a feed worward manner which are
usually in the form of layers. Feed-forward NN
can be presented as a structure of neurons
arranged in layer. In this scheme each neuron may
receive an input from the external environment
and /or from other neurons, but no feedback is
formed.
Feed forward (Multilayer) Topology of NN
For feedback NN the dynamics are no longer
trivial since they consist of processing units
with dynamic building blocks (e.g. integrators or
unit delays) and they operate in feedback mode.
The dynamic properties of such networks are
described by a system of nonlinear ordinary
differential or difference equations.
Cellular NN, similar to cellular automata,
consist of regularly spaced special ANN called
cells which communicate directly with other
neurons only in their nearest neighbourhood.
Adjacent cells can interact with each other by
means of mutual lateral interconnections.
12
Fukushima Model
TheMcCulloch Model

• Two types of inputs
• Excitatory inputs
• Inhibitory inputs

The Hopfield Model
13
The Modified Nonlinear Model
where
The Grossberg Model
14

• Haykin S. Neural Networks A Comprehensive
  Foundation. Mac-Millan. N.York.1994.
• Laurence Fausett Fundamentals of Neural
  Networks Architecture, Algorithms, and
  Applications . Prentice Hall, 1994.
• Cichocki A. Unbehauen R. Neural Networks for
  Optimization and Signal Processing, Wiley, 1993.