Artificial Neural Networks ECE.09.454/ECE.09.560 Fall 2006

1
Artificial Neural Networks ECE.09.454/ECE.09.560
Fall 2006
Lecture 2September 25, 2006

• Shreekanth Mandayam
• ECE Department
• Rowan University
• http//engineering.rowan.edu/shreek/fall06/ann/

2
Plan

• Recall Neural Network Paradigm
• Recall Perceptron Model
• Learning Processes
• Rules
• Paradigms
• Tasks
• Perceptron Training Algorithm
• Widrow-Hoff Rule (LMS Algorithm)
• Lab Project 1

3
Recall Neural Network Paradigm
Stage 1 Network Training
Artificial Neural Network
Present Examples
knowledge
Stage 2 Network Testing
Artificial Neural Network
New Data
4
Recall ANN Model
x Input Vector
y Output Vector
Artificial Neural Network
f Complex Nonlinear Function
knowledge
f(x) y
5
Recall The Perceptron Model
6
Learning
Mathematical Model of the Learning Process
Intitialize Iteration (0)
ANN
w0
x
y(0)
w
x
y
Iteration (1)
w1
x
y(1)
desired o/p
Iteration (n)
wn
x
y(n) d
7
Learning Rules

• Error Correction Learning
• Delta Rule or Widrow-Hoff Rule
• Memory Based Learning
• Nearest Neighbor Rule
• Hebbian Learning
• Competitive Learning
• Boltzman Learning

8
Error-Correction Learning
wk1(n)
Desired Output, dk (n)
Activation/ squashing function
x1 (n)
Bias, bk
wk2(n)
x2

Output, yk (n)
S
j(.)
S
Inputs
Synaptic weights
-
Induced field, vk(n)
wkm(n)
Error Signal ek (n)
xm
9
Learning Paradigms
Unsupervised
Supervised
10
Learning Paradigms
Unsupervised
Supervised
11
Learning Tasks

• Pattern Association
• Pattern Recognition

• Function Approximation
• Filtering

Classification
12
Perceptron Training Widrow-Hoff Rule (LMS
Algorithm)
w(0) 0
n 0
y(n) sgn wT(n) x(n)
w(n1) w(n) hd(n) y(n)x(n)
n n1
Matlab Demo
13
Lab Project 1

• http//engineering.rowan.edu/shreek/fall06/ann/la
  b1.html
• UCI Machine Learning Repository
• http//www.ics.uci.edu/mlearn/MLRepository.html
• Face Recognition Generate images

14
Summary