Innovative Computer Architectures and Concepts

1
Innovative Computer Architectures and Concepts
2
Optical Neural Networks
Speech Marco Buberl
Advisor Nicoleta Pricopi
3
Overview

• 1. Introduction to Neural Networks
• 2. Nonlinear Optics
• 3. Optical Neural Networks

4
1. Introduction to Neural Networks

• 1.1. The Human Brain Model
• 1.2. The Structure of Neural Networks
• 1.3. Learning Concepts

5
1.1. The Human Brain Model
6
1.1. The Human Brain Model

• A neuron and its connections

7
1.1. The Human Brain Model

• A complex 3-dimensional network of approx.
  10,000,000,000 neurons
• Each neuron has a cell body (soma)
• Output connection axon
• Input connection dendrite
• synapse connects axon with dendrite

8
1.1. The Human Brain Model

• The synapse

9
1.1. The Human Brain Model

• Visual input

10
1.1. The Human Brain Model

• Information processing

11
1.1. The Human Brain Model

• Result

12
1.1. The Human Brain Model

• Reaction

13
1.1. The Human Brain Model

• Happy end !

14
1.2. The Structure of Neural Networks
15
1.2. The Structure of Neural Networks

• A mathematically modelled neuron

16
1.2. The Structure of Neural Networks

• Neuron modelled as a network node
• Each neuron has a threshold value ?
• input values xi
• output value y
• input edges have individual weights wi
• weights are adjusted via learning process

17
1.2. The Structure of Neural Networks

• weighted input sum is processed by threshold
  function f with parameter B
• e.g. the Fermi-function or tangens hyperbolicus

18
1.2. The Structure of Neural Networks

• An example 3-layer network

hidden layer
output layer
input layer
19
1.3. Learning Concepts
20
1.3. Learning Concepts

• Training the network
• Applying of training input patterns
• Comparing result with desired output
• Learning by adjusting weights
• Minimising the error

21
1.3. Learning Concepts

• Backpropagation algorithm
• 1. Initialise weights to small random values
• 2. Present the input values and the desired
  output values
• 3. Calculate the network output
• 4. Adapt weights by using a recursive algorithm
  starting at the output layer(for detailed
  formulas see Denz (1998), page 26)
• 5. Repeat from step 2

22
1.3. Learning Concepts

• The Energy Landscape

23
2. Nonlinear Optics

• 2.1. Advantages of Optical Systems
• 2.2. Linear vs. Nonlinear Optics
• 2.3. Photorefractive Optics

24
2.1. Advantages of Optical Systems
25
2.1. Advantages of Optical Systems

• Characteristics of Electrical Neural Networks (1)
• Integrated Circuit Realisations (1)
• analogue devices
• variable resistors, field emitting transistors
  and capacities for weight structure and data
  storage
• huge amount of space necessary
• digital devices
• register storage
• adding and multiplying units necessary (space)

26
2.1. Advantages of Optical Systems

• Characteristics of Electrical Neural Networks (2)
• Integrated Circuit Realisations (2)
• fixed weights
• simulation to get weight values before creating
  the network
• no adaptability
• disadvantages for all three methods
• interference between connections
• isolation space needed

27
2.1. Advantages of Optical Systems

• Characteristics of All-Optical Neural Networks
• data processing at the speed of light
• no interference between light waves
• parallel interconnections are possible
• e.g. with lenses
• optical switches can go faster than electrical

28
2.2. Linear vs. Nonlinear Optics
29
2.2. Linear vs. Nonlinear Optics

• Characteristics of Linear Optics (1)
• effects that are familiar to the visual sense
• diffraction, refraction, interference,
  scattering, absorption
• normal intensities of illumination
• refraction index n
• absorption coefficient ?

30
2.2. Linear vs. Nonlinear Optics

• Characteristics of Linear Optics (2)
• Principle of superposition
• two light waves dont interact with each other
• Conservation of frequency
• light wave keeps its frequency when interacting
  with matter
• n and ? are independent of intensity

31
2.2. Linear vs. Nonlinear Optics
32
2.2. Linear vs. Nonlinear Optics

• Characteristics of Nonlinear optics (1)
• high intensity of illumination
• n and ? depend on intensity
• interaction between waves
• wave can change medium

33
2.2. Linear vs. Nonlinear Optics

• Characteristics of Nonlinear optics (2)
• Possibility to construct optical devices
• signal processing
• optical displays
• telecommunication
• information processing
• neural networks

34
2.2. Linear vs. Nonlinear Optics

• Characteristics of Nonlinear optics (3)
• Light wave can be
• manipulated
• stored
• processed

35
2.3. Photorefractive Optics
36
2.3. Photorefractive Optics

• Photorefractive Materials
• various crystals mainly used in dynamic and
  stationary three-dimensional holography
• most important materials
• Lithiumniobate (LiNbO3)
• Lithiumtantalate (LiTaO3)
• Bariumtitanate (BaTiO3)

37
2.3. Photorefractive Optics

• Photorefractive Effects (1)
• Parametric Amplification
• signal and pump waves meet in a photorefractive
  material
• interference grating with high and low energy
  zones
• impurities (e.g. iron) move in the conduction
  band
• charge carrier density is changed
• also change of refraction indices
• signal data can be stored

38
2.3. Photorefractive Optics

• Photorefractive Effects (2)
• Phase Conjugation
• two pump waves moving towards each other and meet
  the signal wave
• special designed phase conjugate mirror
• sending back light exactly to where it came from
• removal of unwanted phase changes by matter
  interaction on the way back

39
2.3. Photorefractive Optics
40
3. Optical Neural Networks

• 3.1. Interconnection and Storage
• 3.2. The Optical Neuron
• 3.3. A Realisation of an Optical Neural Network

41
3.1. Interconnection and Storage
42
3.1. Interconnection and Storage

• Holographic Interconnection and Storage methods
• Analogy between real and optical neuron
• soma (cell body) ? source/detector-device
• synapse ? hologram (weight simulated by
  modulation depth of the holographic grating)
• axon ? output light beam
• dendrite ? input light beam

43
3.1. Interconnection and Storage
44
3.1. Interconnection and Storage

• Storage by gratings in volume holographic
  materials
• Two types of waves are necessary
• signal waves
• reference waves

45
3.1. Interconnection and Storage

• Gratings are superimposed by several multiplexing
  methods
• angular multiplexing
• the angle of the reference beam is altered
• phase-coded multiplexing
• several reference waves that are individually
  phase-encoded

46
3.1. Interconnection and Storage

• wavelength multiplexing
• frequency of laser source is changed
• spatioangular multiplexing
• angular change combined with slight distance
  between recordings

47
3.2. The Optical Neuron
48
3.2. The Optical Neuron

• The optical neuron must fulfil several tasks
• adding the weighted beams
• applying the threshold-function
• refreshing the stored data

49
3.2. The Optical Neuron

• a good solution for the threshold device is the
  use of nonlinear Fabry-Perot etalons
• running in two modes
• normal mode
• probe mode
• through that backpropagation learning is possible

50
3.2. The Optical Neuron

• Adding and subtracting is done by phase-coded
  multiplexing
• Special phase-code patterns are used
• Suitable for high-speed data processing

51
3.2. The Optical Neuron

• Recording of new data partially erases previous
  stored information
• Therefore a refreshing of the stored data
  necessary
• Coherent refreshment by using phase conjugate
  mirrors

52
3.2. The Optical Neuron
Coherent refreshment of stored information
53
3.3. A Realisation of an Optical Neural Network
54
3.3. A Realisation of an Optical Neural Network

• A combination of the discussed devices
• System should be All-Optical for high-speed
  performance
• Backpropagation algorithm should be possible
• Access of both directions possible

55
3.3. A Realisation of an Optical Neural Network
56
3.3. A Realisation of an Optical Neural Network

• Nonlinear Fabry-Perot etalons representing array
  of optical neurons
• volume holograms for weighted interconnection
• phase conjugate mirrors for weight refreshing

57
3.3. A Realisation of an Optical Neural Network

• high-speed data input by laser diodes
• also an array of optical fibers possible
• error-detection circuit can be realised
  electrooptical or all-optical

58
Summary

• The optical neural network is a brilliant
  invention derived from medical science, network
  theory and nonlinear optics.

59
The End

• No rivalry between human and machines, but an
  attractive and powerful tool to solve complex
  engineering problems.