Neural Networks Seminar: Introduction and a look at Assignment 1

1
Neural Networks SeminarIntroduction and a look
at Assignment 1

• Saqib Khan
• Bioinformatics and Machine Learning Group

2
Ice Breaker

• About Myself
• About You
• Experience with Neural Networks
• Programming in MATLAB
• Seminars
• Thursdays 1100 to 1300 (MSc)
• Fridays 0900 to 1100 (3rd Years)
• Time to do the assignments applying what you
  learn during the lectures
• A computer lab from next week onwards

3
Assignment 1

• Available on the course webpage
• Exploring the Single Layer Neural Network (SLNN)
  and the Gradient Descent Learning Algorithm
• Programming Project and a Report
• Programme in MATLAB
• Use Vector Algebra rather than programming loops
• Use plots and displays to illustrate your results
• Do not use NN tool box
• No word limit for the report
• Explain the procedure
• Try and explore with learning rates, Epochs, data
  etc as required and use that in the write up
• You are encouraged to
• read from the text books and show what you have
  learnt in your report
• consult course handouts

4
Task A Classification

• (1) Four input patterns, four outputs either 1 or
  -1
• Input to be discriminated into two classes, each
  class having two patterns
• Select target values accordingly
• Take one set of target values and train your
  network and explain in the write up.
• Use error functions and weight update and output
  equations discussed during lectures and given in
  handouts.
• Start with weights set to 0 0 0.
• This includes the bias. Dont forget the bias,
  fixed at 1.
• Also decide what you want to do with the output.
• Keep track of the number of iterations (Epochs)
• Display the learning curve and the classification
  boundary (see handouts)
• Use all other combinations to see which ones can
  be trained

5
Task A Classification..contd

• (2) Instead of given training data we now need to
  generate it from Gaussian Distributions
• Assign target value 1 to half the training data
  and -1 to the other half
• Train network as before and explain procedure in
  the report
• Instead of constraining outputs at 1 and -1 try
  using sigmoidal activation function
• Finally change gaussian data so that it is not
  linearly separable and try training again.
  Explain observations

6
Task B Regression

• (1) Single Input
• Generate linear data and pseudo-random target
  values
• Choose a learning rate ltlt 1
• Start with weights (dont forget the bias) set to
  zero as before and train your network using
  equations from lecture handouts
• Chose an appropriate number of Epochs
• Show error curve and the output of the trained
  network
• Comment on the trained weights, how do they
  relate to the output?
• Add noise to the data and train again
• Make observations
• Modify the underlying function and train again
• Again make observations

7
Task B Regression..Contd

• (2) Two Inputs
• As before generate training data for two inputs
• Generate pseudo-random target values using the
  given equation
• Repeat as task one
• Observe and Comment