Parallel neural Circuit SIMulator (PCSIM) - Tutorial

1
Parallel neural Circuit SIMulator (PCSIM) -
Tutorial

• Dejan Pecevski

Institute for Theoretical Computer Science Graz
University of Technology
FIAS Theoretical Neuroscience Summer School,
Frankfurt, August, 2008
2
Outline of the Tutorial

• General Info What is PCSIM?
• Features What is PCSIM useful for?
• Network elements neurons and synapses
• Scalability Using clusters for parallel
  simulation.
• Python Interface
• PCSIM basic operations
• Creating and connecting neurons
• Simulating
• Recording signals
• Populations and Projections
• Summary

3
Who am I?

• Ph.D. Student at theInstitute for Theoretical
  Computer Science, Graz University of Technology
• One of the developers of PCSIM.
• Research Interests
• Plasticity and Learning mechanisms in neural
  circuits
• Spiking Neural Networks
• Simulation technologies/algorithms for biological
  neural networks

4
PCSIM in a nutshell

• Simulator for parallel simulation of spiking and
  analog neural networks with point neuron models
• Implemented in C, with Python and Java
  interfaces
• High-level definition of neural networks
• Library of built-in neuron and synapse models
• Part of FACETS standardization efforts the pyNN
  interface http//www.neuralensemble.org
• Open-source, free, released under the GPL
  license, web page at http//www.igi.tugraz.at/pcs
  im

5
Developers

• Thomas Natschläger Ph.D.Software Competence
  Center Hagenberg Hagenberg, Austria
• Dejan Pecevski DIInstitute for theoretical
  computer scienceGraz University of Technology
  Graz, Austria

Other Contributors Klaus Schuch DI (IGI),
Christian Ernstbrunner DI (SCCH),
Walter Hargassner DI (SCCH)
6
Feature Overview

• Object Oriented Design
• The user interface/view is object oriented
• General Communication System
• Analog and spiking messages
• Network elements with multiple input and output
  ports
• Supported in distributed/multi-threaded mode
• Flexible construction of more complex network
  architectures
• Populations of simulated objects
• Projections for connectivity specification

7
Feature Overview

• Parallel simulation
• Mixed multi-thread and distributed
• Easy to use, transparent to the user
• Easily extensible
• A compilation tool for creating PCSIM extension
  packages
• Usable as backend component in C, Python, Java,
  
• Easier and faster setup of simulations in Python
  and Java
• Runs on Linux or other Unix-like platforms
• can be compiled under Windows (still not tested).

8
PCSIM high-level structure
9
Type of models PCSIM is suitable for

• Large recurrent networks with simple point
  neurons
• Distributed and multi-threaded capabilities
• Efficient simulation engine in C
• More than 105 spiking neurons and 108 synapses
• Hybrid models
• Abstract modules (filters etc.) combined together
  with circuits of analog neurons and spiking
  neurons
• Extensible with new custom elements
• Closed loop/feedback models

10
Type of models PCSIM is suitable for

• Structured models
• Networks composed of smaller subnetworks
  (Populations)
• Probabilistic connectivity patterns based on
  neurons attributes
• Diversity of neurons and synapses
• Different neuron types in the neuron populations
• Specifying random distribution for parameter
  values
• New neuron and synapse types can be implemented

11
Examples of PCSIM usage

• Spiking neural network model of the V1 area in
  the visual cortex of mammals (Schuch Rasch)
• Testing the computational performance of laminar
  cortical models based on experimental data.
  (Schuch Haeusler)
• Experiments to examine the learning capabilities
  of reward-modulated spike-timing-dependent
  plasticity (Legenstein, Pecevski, Maass)

12
A simple model represented in PCSIM
Model equations

• The equations can be decoupled into separate
  simulated elements.
• Presynaptic neurons send spikes to the synapses.
• Synapses inject currents (or modify conductances)
  in the postsynaptic neuron.

13
Network element basic building block

• Nodes of the network (dynamical systems)
• Advanced/integrated each time step of the
  simulation
• Represented by class SimObject
• Neurons, synapses, recorders are derived from
  this class
• Multiple input and output ports, either analog or
  spiking.
• Connections are formed from output to input ports
  of the same type.
• Every element in PCSIM

14
Neurons and Synapses as Network Elements

• Synapses as network elements are attached to the
  postsynaptic neuron
• Synapses have one input port, no output ports
• Neurons have only one output, no inputs
• The spiking and analog connections can connect
  neurons on different nodes

15
Built-in Neuron Models

• Spiking Models
• LifNeuron, CbLifNeuron
• Leaky integrate-and-fire
• HHNeuron
• Hodgkin-Huxley Neuron
• IzhiNeuron (Izhikevich, 2004)
• aeIFNeuron
• Adaptive Exponential IF
• (Brette and Gerstner, 2005)
• LinearPoissonNeuronLeaky integrate with Poisson
  output

• Input Neurons
• PoissonInputNeuron
• SpikingInputNeuron
• Analog Neurons
• LinearAnalogNeuron

16
Built-in Synapse Models

• Type
• current based
• conductance based
• PSR Kernel
• exponential, alpha, double-exponential, Square
• Short-term Plasticity
• (Markram et al. 1998)
• Spike-timing-dependent Plasticity
• (Froemke and Dan, 2002)
• (Gütig et al, 2003)
• each pair and nearest neighbour

• Other Mechanisms
• NMDA(Gabbiani et al. 1994)
• GabaB (Mainen et al. 1994)
• Reward-modulated STDP (Izhikevich, 2007)
• Homeostatic plasticity(Buonomano et al. 2005)
• Ornstein-Uhlenbeck noise synapse (Destexhe et
  al. 2001)
• Analog Synapse

17
Simulation Strategy

• Clock-driven simulation with a fixed time step.
• Hybrid Integration Strategy
• Neurons are integrated every time step.
• Synapses are event driven, i.e. they are
  integrated only after receiving a spike.
• Numeric Integration Algorithms
• Exponential Euler Method
• ODE solvers from GSL

18
Parallel Simulation

• Each MPI process advances subset of neurons
• MPI as underlying communication layer
• spiking and analog messages
• Each process can have multiple-threads

• Transparent to the user
• Default round-robin distribution of the neurons
  over nodes
• Custom distribution strategies
• Easy retrieval of recordings (as in single-thread
  case)

19
Parallel Simulation

• Why?
• Speed-up simulation of an existing model by using
  more processors
• Enables simulation of larger models no memory
  limit problem
• Scale up the number of neurons while keeping the
  simulation time the same.
• Comparison between single-threaded and
  distributed simulation
• lt 104 neurons and 107 synapses on a single
  machine
• gt 105 neurons and 108 synapses on clusters
• gt 106 neurons and 1010 synapses on
  supercomputers?

20
Scalability of PCSIM

• Hardware 3.4 Ghz, 512 kb cache, 4GB RAM
• Model
• 50000 LifNeurons, 4.0 Hz average firing rate
• 50?106 synapses, 1.5 ms transmission delay

21
Scalability of PCSIM

• Hardware 3.4 Ghz, 512 kb cache, 4GB RAM
• Cuba Model
• 5000 LifNeurons per node, 4.0 Hz average firing
  rate
• 103 synapses per neuron, 1.0 ms transmission
  delay, broad distribution of time synaptic
  constants

22
Extending PCSIM

• What are extensions?
• User implemented classes that are plugged into
  the PCSIM OO framework
• usualy coded in C (can be coded also in Python)
  
• Compilation of additional separate Python
  extension packages, without the need to recompile
  the main PCSIM code.
• The extension classes are automatically wrapped
  in Python
• Usual extensions for PCSIM
• new neuron/synapse types
• other user defined network elements
• custom rules for specific network construction

23
Python Interface

• What is Python? (taken from www.python.org)
• Python is a dynamic object-oriented programming
  language
• offers strong support for integration with other
  languages and tools.
• comes with extensive standard libraries
  (batteries included).
• can be learned in a few days.
• Many Python programmers report substantial
  productivity gains and feel the language
  encourages the development of higher quality,
  more maintainable code.
• many scientific packages scipy, numpy,
  matplotlib, ipython, Rpy etc.

24
Python Interface

• PCSIM can be imported as a package within Python
• The model is represented as a network object
• Individual neurons and synapses are accessible as
  python objects.
• Population and Projection objects for high-level
  construction of networks composed of populations
• Together with the other Python packages for
  scientific computing provides a complete working
  environment for neural simulations.

from pypcsim import
net SingleThreadNetwork()
25
Creating and connecting neurons

• Creates one LIF neuron with default parameter
  values
• nrn_model LifNeuron()
• nrn_id net.create( nrn_model )
• Creates array of 10 LIFneurons
• nrn_array net.create( LifNeuron(), 10)
• Connects the first and the second neuron in the
  array
• syn_id net.connect( nrn_array0, nrn_array1,
  StaticSpikingSynapse())

26
Specifying neuron/synapse parameters

• nrn net.create(LifNeuron( Cm2e-10,
• Rm1e8,
• Vthresh-50e-3,
  Vresting-60e-3,
• Vreset-60e-3,
  Trefract5e-3, Vinit-60e-3 ))
• syn net.connect( pre_nrn, post_nrn,
• StaticSpikingSynapse( W1e-10,
  
• tau
  5e-3,
• 
  delay1e-3 ))

27
Recording

• record the output spikes of a neuron
• rec_spk net.record( nrn, SpikeTimeRecorder() )
• record the membrane potential
• rec_vm net.record( nrn, Vm, AnalogRecorder()
  )
• record the weight of a synapse
• rec_syn net.record( syn, W, AnalogRecorder()
  )
• record any field in an element
• rec net.record( myElement,
• anyFieldName, AnalogRecorder())

28
Simulate the model

• simulate the network for 1 second
• net.simulate(1.0)
• or advance it for 200 time steps
• net.advance(200)

29
Accessing individual neurons/synapses

• creates return an ID of the neuron
• nrn_id net.create( LifNeuron() )
• get the actual neuron object from the network
• nrn_object net.object(nrn_id)
• Manipulate the neuron object
• nrn_object.Vthresh -59e-3
• getting the recorded values from a recorder
• values list(net.object(rec_id).getRecordedValues
  ())

30
Synapse classes nomenclature

• StaticCurrExpSynapse (StaticSpikingSynapse)
• DynamicCondDblExpSynapse
• StaticStdpCondAlphaSynapse
• DynamicCurrAlphaSynapse
• DynamicNMDAAlphaSynapse

Current/conductance based?
Shape of PSR kernel
Short-term plasticity
31
Hello, world!

• from pypcsim import
• net SingleThreadNetwork()
• pre_nrn net.create( LifNeuron(Iinject 5e-8) )
• post_nrn net.create( LifNeuron(Iinject 5e-8)
  )
• net.connect( pre_nrn, post_nrn,
  StaticSpikingSynapse())
• rec net.record( post_nrn, SpikeTimeRecorder() )
• net.simulate(0.2)
• print spike times, list(net.object(rec).getReco
  rdedValues())

32
Specifying inputs

• Create input neuron which spikes at specific
  times
• net.create(SpikingInputNeuron( 0.1, 0.2, 0.4))
• Create input neuron with Poisson process spike
  times
• net.create( PoissonInputNeuron(rate 10,
  duration 10) )
• Create analog input neuron with an
• output analog signal specified as array
• net.create( AnalogInputNeuron( arange(0,10,1e-4)
  1 ) )

33
Creating and connecting many neurons

• Population of 1000 neurons randomly
  interconnected with 0.1 probability
• Population of recorders one for each neuron in
  the population

• create a population of 1000 LIF neurons
• popul SimObjectPopulation(net, LifNeuron(),
  1000)
• create the projection
• proj ConnectionsProjection( popul, popul,
• StaticSpikingSynapse(), RandomConnections
  ( 0.1 ) )
• create a recorder population
• rec_popul popul.record( SpikeTimeRecorder() )

34
Random distributions for parameter values

• Factory an object that generates PCSIM elements
• Neuron and synapse objects (used as models for
  creation) are simple clone factories
• SimObjectVariationFactory factory that attaches
  random distributions to parameter names

nrn_factory SimObjectVariationFactory(
LifNeuron() ) nrn_factory.set( Vinit, Normal
Distribution( -55e-3, 0.1 ) ) popul
SimObjectPopulation(net, nrn_factory, 1000)
35
Spatial heterogenous populations

• Each neuron has coordinates in 3D space
• The population is composed of different families
  of neurons (that can be of different type)

• exc_nrn LifNeuron( Cm 2e-10,
• Inoise 1e-10)
• inh_nrn LifNeuron( Cm 3e-10 )
• popul SpatialFamilyPopulation(net,
• exc_nrn, inh_nrn,
• RatioBasedFamilies((4,1) ),
• CuboidIntegerGrid3D(20,10,10))
• exc_popul, inh_popul popul.splitFamilies()

36
Distance dependent random connections
Creates random connections with
probability where D is the euclidean
distance between the two neurons

• proj ConnectionsProjection( popul, popul,
  
• StaticSpikingSynapse(),
• EuclideanDistanceRandomConnections( C, lambda )
  )

37
Saving recorded data

• from pypcsimplus import
• Create the recording class
• r Recordings()
• Add the recorder populations as attributes to r
• r.spikes popul1.record( SpikeTimeRecorder() )
• r.vm_traces popul2.record( Vm,
  AnalogRecorder() )
• Some additional variables to be saved
• r.v 5
• Save the Recordings class to a HDF5 file
• r.saveInOneH5File(results.h5)

38
Running a parallel simulation

• Change the network class from SingleThreadNetwork
  to either
• MultiThreadNetwork( nThreads 4 )
• DistributedSingleThreadNetwork
• DistributedMultiThreadNetwork
• then run the script with mpirun

mpirun n 10 python my_experiment.py
39
Summary

• PCSIM is a tool for simulation structured random
  neural networks composed of spiking and analog
  point neuron models
• Has various built-in neuron and synapse models
• Supports parallel simulation distributed and
  multi-threaded
• Flexible high-level construction of networks
  based on probabilistic rules
• The Object-oriented framework is designed to
  support easy extensions
• Can be used as a package within the Python
  interpreting programming language, together with
  other useful scientific packages

40
PCSIM Resources

• Web page
• http//www.lsm.tugraz.at/pcsim
• On Sourceforge http//www.sourceforge.net/project
  s/pcsim
• Mailing listhttp//sourceforge.net/mailarchive/fo
  rum.php?forum_namepcsim-users
• User manualhttp//www.lsm.tugraz.at/pcsim/userman
  ual/html/index.html
• C class referencehttp//www.lsm.tugraz.at/pcsim
  /cppclassreference/html/hierarchy.html

41

• Thank you for your attention!