Evolutionary connectionism: Amalgamating neural and evolutionary techniques

1
Evolutionary connectionism Amalgamating neural
and evolutionary techniques

• Angelos Molfetas
• Centre for Advanced Systems Engineering
• School of Computing and Information Technology
• University of Western Sydney

2
Initial pleasantries

• Details
• P.h.D research (1st year)
• Topic Evolutionary connectionism
• Research in the field
• Direction of my research
• Plan of action
• Basic concepts
• What constitutes as research in this field?
• Process of research
• Possible research topics
• Summary

3
Basic concepts

• Artificial Neural Networks (ANNs)
• Simplified imitation of biological neural
  networks.
• Synonyms Neural Networks, connectionist
  networks.
• Components Neurons, connection weights,
  activation functions, learning rules, etc.
• Types of learning Supervised training,
  unsupervised training and reinforcement learning.
• Used for classification, pattern recognition,
  filtering and prediction.

4
Basic concepts
5
Basic concepts

• Genetic Algoritms (GAs)
• Using evolutionary principles to find solutions
  (survival of the fittest)
• Key points
• Population of solutions
• Solution characteristics encoded in genes
• GA Phases Evaluation, selection, crossover and
  mutation

6
Combining neural and evolutionary computing

• Encode ANN characteristics into gene structures.
  GA finds optimal ANN characteristics.
• Motivation
• ANNs and GAs seem to work well together
• Imitate natures way
• Make ANNs simple to use
• Let the computer do the hard work.

7
What constitutes as research in this field?

• Introduce new algorithms or methods
• New architectures
• Genetic encoding schemes (mapping neural network
  characteristics to gene structures)
• Learning mechanisms
• Comparisons
• Comparisons within topic
• Comparisons with other techniques.

8
What constitutes as research in this field?

• Applying evolved ANNs to solve problems
• For e.g. Speech recognition
• Application of techniques to investigate
  phenomena

9
Research topics

• GA based connection weight training
• Pendharkar and Rodger (1999)
• Comparisons to other learning techniques.
• Performance of different GA crossover operations.
  
• GA encoding schemes for ANNs
• Dasgupta and McGregor (1992)
• Encoding scheme sGA
• Types of characteristics encoded (weights and
  structure)
• Chalmers (1990)
• Encoding learning processes.

10
Research topics

• Application of evolved ANNs to problems
• Watts and Kasabov (1998)
• Phoneme recognition (Speech recognition)
• Harvey (1995)
• Robotics
• Game theory.
• Chellapilla and Fogel (1999), Chellapilla and
  Fogel (2001).
• IPD
• TIC-TAC_TOE
• CHECKERS

11
Research topics

• Investigating Phenomena
• Jones and Konstam (1999)
• Baldwin effect in evolutionary theory
• Kun, Mak and Siu (2000)
• Lamarckian Evolution in recurrent neural networks

12
Process of Research

• Experimental methodologies dominant research
  method.
• Comparisons
• Proof of concept
• Mathematical manipulation
• Draws from multiple disciplines
• Often problem driven

13
Direction of research

• Possible Research topics
• Evolutionary ANNs to extend Ambalavanan and
  Carlos (1999) work.
• Application of evolutionary ANNs to financial
  problems.
• Extend GA encoding schemes to incorporate more
  characteristics.
• Focus thesis on the techniques rather than their
  application to a given problem domain

14
Quick Summary
15
Final note

• Leave you with one final thought
• Next wave of innovation Artificial Intelligence
  and robotics

16
References

• Chellapilla, K., Fogel, D. (1999). Evolution,
  Neural Networks, Games, and Intelligence.
  Proceedings of the IEEE.
• Chellapilla, K., Fogel, D. (2001). Evolving an
  expert checkers playing program without using
  human expertise. IEEE transactions on
  evolutionary computation, 5(4), 422-428
• Chalmers, D. (1990). The Evolution of Learning An
  experiment in Genetic Connectionism. Proceedings
  of the 1990 Connectionist Models Summer School
  (pp. 81-90).
• Dasgupta, D., McGregor, D. (1992). Designing
  application-specific neural networks using the
  Structured Genetic Algorithm. COGANN-92
  (International workshop on combinations of
  Genetic Algorithms and Neural Networks)

17
References

• Harvey, Inman (1995). The artificial evolution of
  adaptive behaviour. Unpublished thesis,
  University of Sussex, Brighton
• Jones, M., Konstam, A. (1999). The use of
  genetic algorithms and neural networks to
  investigate the Baldwin effect. Proceedings of
  the 1999 ACM symposium on applied computing, pp.
  275-279
• Kasabov, N., Watts, M. (1997). Genetic
  algorithms for structural optimisation, dynamic
  adaptation and automated design design of fuzzy
  neural networks. Internation conference on Neural
  Networks, 4, 2546-2549.
• Ku, K., Mak, M., Siu, W. (2000). A study of the
  lamarckian evolution of recurrent neural
  networks. IEEE transactions on evolutionary
  computation, 4 (1), 31 - 42

18
References

• Namasivayam, A., Carlo, W. (1999). Comparison
  of the prediction of extremely low birth weight
  neonatal mortality by regression analysis and by
  neural networks. Society of Pediatric Research
  meeting
• Pendharkar, P., Rodger, J. (1999). An empirical
  study of non-binary genetic algorithm-based
  neural approaches for classification. Proceedings
  of the 20th international conference on
  Information Systems.
• Watts, M., Kasabov, N. (1998). Genetic
  Algorithms for the design of fuzzy Neural
  Networks. Proceedings of ICONIP' 98 - The fifth
  international conference on Neural Information
  Processing.