Associative memory using coupled non-linear oscillators - PowerPoint PPT Presentation

About This Presentation
Title:

Associative memory using coupled non-linear oscillators

Description:

Associative memory using coupled non-linear oscillators Semester project Final Presentation Vlad TRIFA – PowerPoint PPT presentation

Number of Views:81
Avg rating:3.0/5.0
Slides: 26
Provided by: Vlad220
Category:

less

Transcript and Presenter's Notes

Title: Associative memory using coupled non-linear oscillators


1
Associative memory using coupled non-linear
oscillators
  • Semester project
  • Final Presentation

Vlad TRIFA
2
Project summary
  • Litterature review
  • Implementation of an associative memory using
    coupled oscillators and analysis of
    performance/drawbacks.
  • Mixture with the BIRG model
  • Generalization to complex signals
  • Better control on the capacity
  • Final discussion about relevant issues concerning
    the performances of both models
  • Conclusion

3
Associative Memory
  • Animal and human memory works by association.
  • Able to retrieve a stored pattern upon
    presentation of a partial and noisy
    representation of an input signal.
  • Many models developed since early 80s
  • Concepts taken from statisic mechanics and
    hebbian learning rule turned neural networks into
    dynamic systems.
  • Useful into understanding dynamics of networks
    (emergence)
  • but
  • Lack of biologically plausible mechanisms
    (coupling, binary,)
  • Low capacity and performance (Global coupling N2
    parameters)

4
Oscillators
  • Oscillating systems are very common in nature
    and possess very intersting properties.
  • Synchronization
  • Energy efficient mechanism for temporal
    correlation
  • Many brain processes rely on interaction of
    oscillators
  • CPG
  • Olfactory and visual cortex
  • Temporal correlation hypothesis and binding
    problem
  • Information can be stored as phase relationships
    patterns, where coupled oscillators converge.

5
Analyzed model
  • Can be found in Borisyuk, 2001.
  • Oscillators described by phase, amplitude, and
    frequency.

6
The model
7
(No Transcript)
8
Dynamics
9
Model performance
  • The capacity of this model is not easy to derive,
    due to the random phase shifts, and to the
    dynamics of the nonlinear term
  • We do not know what percentage of overlapping is
    possible, as memorized patterns can be
    overwritten.
  • Implies that error increases as memory is filled.
  • Robustness due to distributed memorization.
  • But, loss of groups influence strongly the
    retrieval error.

10
Discussion
  • The model is interesting as it is based on
    oscillating systems, thus can be easily
    implementable on many oscillating systems (PLL,
    etc).
  • A very nice methodology that is embedded in the
    system is proposed in order to decide where to
    store an input signal is proposed.
  • Random phase shifts ensure some robustness to the
    system, but too big influence on the performance.

11
Drawbacks
  • The all-to-all coupling into groups is not
    efficient computationally and it uses too many
    oscillators.
  • Due to the explicit input signal embedded in the
    equations, we can only learn sine functions.
  • The input dimension is annoying. Complexity is
    increased with no performance increase.
  • The time is reset after each stimulus. We need to
    present the input in-phase with the oscillators.
    We cannot learn sequences.

12
Improvements
  • We want to be able to learn complex signals.
  • Starting from the model in Righetti et al,
    2005, we want to extend the model to form a
    network.

13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
Discussion
  • Simpler model, more computationally efficient.
  • We gained a much better control on the amount of
    oscillators to dedicate for a frequency
    component.
  • We are able to memorize complex signals in a
    robust and fault tolerant manner, under some
    constraints.
  • but
  • Unfortunately, the capacity depends on the
    complexity of the signals to store.
  • We lost the selection of the storage sites based
    on phase relationships we had with the previous
    model.

23
Future work
  • We need to find a mechanism (embedded in the
    dynamics) that can select where each component
    should be stored depending on the signal.
  • It would be very intersting to create links
    between different clusters activated by the same
    signal, similar to associative connections
    forming according to the correlation of neural
    activity between assemblies, enhancing robustness
    if attenuated components.
  • Reduce parameters, so we need only to select the
    amount of oscillators allocated per component.

24
Conclusion
  • This work should be considered as an attempt to
    provide insights on how it is possible to store
    information encoded as a complex signals in a
    reliable manner, simply by using oscillating
    systems with local interactions.
  • Our approach is interesting as it uses some
    concepts that are common in biological neuronal
    networks such as
  • Oscillating components with local interactions
  • No global external process to supervise the
    learning procedure

25
Thank you!
  • References
  • Borisyuk, 2001
  • Righetti et al., 2005
  • Singer, 1995
Write a Comment
User Comments (0)
About PowerShow.com