Lecture 2' Paradigms of Neurocomputing

1
Lecture 2. Paradigms of Neurocomputing
Saint-Petersburg State Polytechnic
University Distributed Intelligent Systems
Dept. ARTIFICIAL NEURAL NETWORKS

• Prof. Dr. Viacheslav P. Shkodyrev
• EMail shkodyrev_at_imop.spbstu.ru

Neurocomputer Brianlike computer.
Whether it is possible to simulate work of a
human brain? Why is it so universal? In what way
does the human brain processes the information?
Cooperated with biologists, mathematicians offer
a theory of Brain Computer.
2
Brain Computer and traditional the Von Neuman
machine performs quite different paradigms and
have different properties

• Objective of Lecture 2
• The main objective of this lecture is to
  discuss the basic paradigms of Neurocomputing and
  Artificial Neural Networks via Brain computer
  theory. We will discuss
• Why and where we need new paradigms of
  neurocomputations
• What are Neurocomputer and Artificial Neural
  Networks
• Which new principles of data processing does
  Neurocomputer realize
• What is biological prototype of Neurocomputer

3

• Ill-Formalizable Tasks
• Sound and Pattern recognition
• Decision making
• Knowledge discovery
• Context-Dependent Analysis

What is difference between human brain and
traditional computer via specific approaches to
solution of ill-formalizing tasks?
Symbol manipulation

• Dove flies
• Lion goes
• Tortoise scrawls
• Donkey sits
• Shark swims

4
The main difference between human and traditional
computer comes from
Researches are analyzing such model as visual
image recognition, speech and sound recognition,
smell recognition
Our hearing, vision, taste and tactile
inputs provide us with a wealth of images
information about world. How we interpreter these
patterns?
5
How our brain manipulates with patterns ?
A process of pattern recognition and pattern
manipulation is based on
6
Artificial Neural Network Mathematical
Paradigms of Brain-Like Computer
The new paradigm of computing mathematics
consists of the combination of such artificial
neurons into some artificial neuron net.
Neurons and Neural Net
Brain-like computer is a mathematical model of
humane-brain principles of computations. This
computer consists of those elements which can
be called the biological neuron prototypes, which
are interconnected by direct links called
connections and which cooperate to perform
parallel distributed processing (PDP) in order to
solve a desired computational task.
7
Connectionizm NN is a highly interconnected
structure in such a way that the state of one
neuron affects the potential of the large number
of another neurons to which it is connected
accordiny to weights of connections
Not Programming but Training NN is trained rather
than programmed to perform the given task since
it is difficult to separate the hardware and
software in the structure. We program not
solution of tasks but ability of learning to
solve the tasks
Distributed Memory NN presents an distributed
memory so that changing-adaptation of synapse can
take place everywhere in the structure of the
network.
8
Learning and Adaptation NN are capable to adapt
themselves (the synapses connections between
units) to special environmental conditions by
changing their structure or strengths
connections.
Non-Linear Functionality Every new states of a
neuron is a nonlinear function of the input
pattern created by the firing nonlinear activity
of the other neurons.
Robustness of Assosiativity NN states are
characterized by high robustness or insensitivity
to noisy and fuzzy of input data owing to use of
a highly redundance distributed structure
9
(No Transcript)
10
Biological Neuron - The simple arithmetic
computing element
11

• Soma or body cell - is a large, round central
  body in which almost all the logical functions of
  the neuron are realized.
• The axon (output), is a nerve fibre attached to
  the soma which can serve as a final output
  channel of the neuron. An axon is usually highly
  branched.
• The dendrites (inputs)- represent a highly
  branching tree of fibres. These long irregularly
  shaped nerve fibres (processes) are attached to
  the soma.
• Synapses are specialized contacts on a neuron
  which are the termination points for the axons
  from other neurons.

12
Where w1, w2 , , wn sinaptic weights, 0
threshold level.
13
Electronic circuit model of Neuron
A simulating physical model of the above formal
neuron can be built using traditional electronic
circuit components - a voltage amplifier. It
simulates the cell body (soma), the wires
replace the input structure (dendrites) and
output structure (axon) and the variable
resistors model the synaptic weights (synapses).
where ij is the current flowing through the
resistor Rj (from neuron I to nuron j).
14
ANN is an assembly of a simple processing
elements neurons which are interconnected by
direct links and which cooperate in order to
solve a desired computational task.
15
Every neuron of ANN can be interpreted as an
elementary processing unit with multiple inputs
and non-linear transformation of bioelectrical
signal excitations .
Non-linear
16
What is mathematical description of these
paradigms ?
?
17

• Haykin S. Neural Networks A Comprehensive
  Foundation. Mac-Millan. N.York.1994.
• Laurence Fausett Fundamentals of Neural
  Networks Architecture, Algorithms, and
  Applications . Prentice Hall, 1994.
• Cichocki A. Unbehauen R. Neural Networks for
  Optimization and Signal Processing, Wiley, 1993.