{"title": "Event-Driven Simulation of Networks of Spiking Neurons", "book": "Advances in Neural Information Processing Systems", "page_first": 927, "page_last": 934, "abstract": null, "full_text": "Event-Driven Simulation of Networks of \n\nSpiking l'Ieurons \n\nLloyd Watts \nSynaptics Inc. \n\n2698  Orchard  Parkway \n\nSan Jose CA  95134 \n\n11oydGsynaptics.com \n\nAbstract \n\nA fast event-driven software simulator has been  developed for sim(cid:173)\nulating large networks of spiking neurons and synapses.  The prim(cid:173)\nitive  network  elements  are  designed  to exhibit  biologically  realis(cid:173)\ntic  behaviors,  such  as  spiking,  refractoriness,  adaptation,  axonal \ndelays,  summation of post-synaptic  current  pulses,  and tonic  cur(cid:173)\nrent  inputs.  The efficient  event-driven  representation  allows  large \nnetworks  to  be  simulated in a  fraction  of the  time  that  would  be \nrequired  for  a  full  compartmental-model simulation.  Correspond(cid:173)\ning analog CMOS  VLSI  circuit  primitives have  been  designed  and \ncharacterized,  so  that  large-scale  circuits  may  be simulated  prior \nto fabrication. \n\n1 \n\nIntroduction \n\nArtificial  neural  networks  typically  use  an  abstraction  of real  neuron  behaviour, \nin  which  the  continuously  varying  mean firing  rate  of the  neuron  is  presumed  to \ncarry  the information about the neuron's time-varying state of excitation  [1].  This \nuseful  simplification allows the  neuron's  state to  be  represented  as  a  time-varying \ncontinuous-amplitude quantity.  However,  spike timing is known to be important in \nmany biological systems.  For  example,  in  nearly  all  vertebrate  auditory  systems, \nspiral ganglion cells from the cochlea are known  to phase lock  to pure-tone stimuli \nfor  all but the  highest  perceptible frequencies  [2].  The barn owl  uses  axonal delays \nto compute azimuthal spatial localization [3].  Studies in the cat [4]  have shown that \n\n927 \n\n\f928 \n\nWatts \n\nrelative timing of spikes is  preserved  even  at  the highest  cortical levels.  Studies in \nthe  visual  system of the  blowfly  [5]  have shown  that  the information contained in \njust three spikes is enough for  the fly  to make a decision to turn, if the visual input \nIS  sparse. \nThus,  it  is  apparent  that  biological  neural  systems  exploit  the spiking  and  time(cid:173)\ndependent  behavior of the neurons and synapses to perform system-level computa(cid:173)\ntion.  To  investigate  this  type  of computation,  we  need  a  simulator that  includes \ndetailed  neural  behavior,  yet  uses  a  signal representation  efficient  enough  to  allow \nsimulation of large networks in a  reasonable  time. \n\n2  Spike:  Event-Driven Simulation \n\nSpike is  a  fast  event-driven simulator optimized for simulating networks of spiking \nneurons and synapses.  The key  simplifying assumption in Spike is that all currents \ninjected into a  cell  are composed of piecewise-constant  pulses  (i.e.,  boxcar pulses), \nand  therefore  all  integrated  membrane voltage trajectories  are  piecewise  linear in \ntime.  This very simple representation is capable of surprisingly complex and realistic \nbehaviors,  and  is  well  suited  for  investigating system-level  questions  that  rely  on \ndetailed spiking behavior. \n\nThe simulator operates by maintaining a queue of scheduled events.  The occurrence \nof one event  (i.e.,  a neuron spike)  usually causes later events to be scheduled in the \nqueue  (Le., end of refractory  period, end of post-synaptic current  pulse).  The total \ncurrent injected into a  cell is integrated into the future to predict the time of firing, \nat which  time a  neuron spike event  is scheduled.  If any  of the current  components \nbeing injected into the cell subsequently change, the spike event is rescheduled.  The \nsimulator runs  until  the  queue  is empty or until the desired  run-time has  elapsed. \nA similar event-driven  neural simulator was developed  by  Pratt [6]. \nThe simulator output may be plotted by a number of commercially available plotting \nprograms, including Gnuplot, Mathematica, Xvgr,  and Cview. \n\n3  N euraLOG:  Neural Schematic Capture \n\nNeuraLOG is a schematic entry tool, which allows the convenient entry of \"neural\" \ncircuit diagrams, consisting of neurons, synapses,  test inputs,  and custom symbols. \nNeuraLOG is a  customization of the program AnaLOG, by John Lazzaro and Dave \nGillespie. \n\nThe parameters of the  neural  elements  are  entered  directly  on  the  schematic  dia(cid:173)\ngram; these parameters include the neuron refractory period, duration and intensity \nof the post-synaptic current pulse following an action potential, saturation value of \nsummating post-synaptic  currents,  tonic  input  currents,  axonal  delays,  etc.  Cus(cid:173)\ntom symbols  can  be  defined,  so  that arbitrarily  complex  hierarchical designs  may \nbe made.  It is common to create  a  complex  \"neuron\"  containing many neuron and \nsynapse  primitive elements.  Spiking inputs may be supplied as external stimuli for \nthe  circuit in a  number of different  formats,  including single spikes,  periodic  spike \ntrains,  periodic bursts,  poisson  random spike trains,  and gaussian-jittered periodic \n\n\fEvent-Driven Simulation of Networks of Spiking Neurons \n\n929 \n\nspike trains.  Textual input to Spike is also supported,  to allow simulation of circuit \ntopologies that would  be too time-consuming to enter graphically. \n\n4  A  Simple Example \n\nA simple example of a  neural circuit  is shown  in Figure  1.  This circuit  consists  of \ntwo  neurons  (the large disks),  several synapses  (the large triangles),  and two tonic \ninputs  (the  small  arrows).  The  text  strings  associated  with  each  symbol  define \nthat  symbol's  parameters:  neuron  parameters  are  identifier  labels  (Le.,  ni)  and \nrefractory period in milliseconds (ms); synapse parameters are the value of the post(cid:173)\nsynaptic current in nA, and the duration of the current pulse in ms, and an optional \nsaturation parameter, which indicates how  many post-synaptic current  pulses  may \nbe superposed  before saturation; the  tonic input parameter is  the injected  current \nin nA.  Filled symbols (tonic inputs and synapses)  indicate inhibitory behavior. \n\n-.8108> \n\n-815> \n\n6.4> \n\n.132.5> \n\n-.001 > \n\nFigure  1:  Graphical input representation  of a simple neural circuit,  as  en(cid:173)\ntered  in NeuraLOG. \n\nThe simulated behavior of the circuit is shown in Figure 2.  The neuron ni exhibits \nan adapting bursting behavior,  as seen  in the top  trace of the plot. \nThe excitatory  tonic  current  input  to neuron  ni causes  ni to fire  repeatedly.  The \nweakly excitatory synapse from ni to neuron n2 causes n2 to fire after many spikes \nfrom n1.  The synaptic current in the synapse from ni  to n2 is  plotted in the  trace \nlabeled  snin2.  The strongly  inhibitory synapse  from  n2  to  ni  causes  ni  to  stop \nfiring  after n2  fires  a  spike.  The synaptic current  in  the synapse  from  n2  to  ni is \nplotted in  the trace labeled  sn2ni.  The  combination of the excitatory  tonic input \nto ni and the inhibitory feedback  from n2 to ni causes  the bursting behavior. \nThe adapting behavior is caused  by  the self-inhibitory accumulating feedback  from \nneuron  ni  to  itself,  via  the  summating inhibitory  synapse  in  the  top  left  of the \ndiagram.  Each  spike  on  ni  causes  a  slightly increased  inhibitory  current  into ni, \nwhich  gradually  slows  the  rate  of firing  with  each  successive  pulse.  The synaptic \ncurrent  in  this inhibitory synapse is plotted in the trace  snini; it is similar to  the \n\n\f930 \n\nWatts \n\nn1  _mJJJD\"'----___ JillJJU\"'----___ OOJJU~ \nJ~ \n\nJ \n\nn2 \n\nJ \n\nsn1n1 \n\nsn2n1 \n\no \n\n10 \n\n20 \n\n30 \n\n40 \n\n50 \n\n60 \n\n70 \n\nTime (ms) \n\nFigure  2:  Simulation results  for  the  circuit  of Figure  1,  showing  adapting \nbursting behavior. \n\ncurrent  that would be generated  by  a  calcium-dependent potassium channel. \nThis simple example  demonstrates  that  the  summating synapse  primitive can  be \nused  to  model a  behavior  that is  not strictly synaptic in origin;  it can  be  thought \nof as  a  general  time-dependent  state  variable.  This  example  also  illustrates  the \nprinciple that proper network topology (summating synapse in a  negative feedback \nloop)  can lead to realistic system-level behavior (gradual adaptation), even  though \nthe basic  circuit elements may be rather primitive (boxcar current  pulses). \n\n5  Applications of the Simulation Tools \n\nNeuraLOG  and  Spike  have  been  used  by  the  author  to  model spiking  associative \nmemories, adaptive structures that learn to predict a time delay, and chaotic spiking \ncircuits.  Researchers  at Caltech  [7,  8]  and the Salk Institute have used  the tools in \ntheir studies  of locust  central  pattern generators  (CPGs)  and  cortical  oscillations. \nThe cortical oscillation circuits contain a few  hundred neurons and a few  thousand \nsynapses.  A  CPG  circuit,  developed  by  Sylvie  Ryckebusch,  is  shown  in  Figure  3; \nthe corresponding simulation output is  shown in  Figure 4. \nNeuraLOG  and Spike  are  distributed  at  no  charge  under  the  GNU  licence.  They \nare  currently supported on HP and Sun workstations.  The tools are supplied  with \na  user's manual and  working tutorial examples. \n\n\fEvent-Driven Simulation of Networks of Spiking Neurons \n\n931 \n\nin)>--~. \n\n~)>--~. \n\nchI)>---..... \n\npill \n\nIwI)>---\n\ni11 \n\ni11) \n\nlevi) \n\nItIVr) \n\n~ \n<l \n~ \n\nlevi) \n\n~  pill) \n\nI_r) \n\n~ \n<l \n<l \n\npill) \n\ndll \n\nI-I) \n\nItIVr) \n\n~in \n\nin) \n\nItIVr) \n\ndar) \n\npirr \n\nItIVr) \n\ni1r \n\ni1r) \n\nItIVr) \n\nlevi) \n\nItIVr ) \n\nI_r \n\npirr) \n\nlevi) \n\n~ \n<l \n<l \n\npirr) \n\ndar \n\nItIVr ) \n\nIwI) \n\nFigure 3:  Sylvie Ryckebusch's  locust  CPG circuit.  For clarity, the synapse \nparameters have been omitted. \n\n\f932 \n\nWatts \n\nIn  I \n\ndfl \n\ndsl \n\ni11 \n\nlevi \n\npirl \n\ndfr \n\ndsr \n\n11r \n\nlevr \n\npirr \n\nj \nt \nj \n\nj \n-L \nj \n20 \n\n0 \n\nj \n\nJ  J  J  J \nt \nJ \n\n-I  1 \nj \n.J \n40 \n\nj \n-I  L  -L \nj \nj \n80 \n60 \n\nj \n\nJ \nt \nJ \n\n\" \n-I \n\u2022 \n100 \n\nFigure 4:  Simulation results for  Sylvie Ryckebusch's locust  CPG circuit. \n\nTime (ms) \n\n6  The Link to Analog VLSI \n\nAnalog VLSI  circuit  primitives that can be  modelled by Spike  have been  designed \nand tested.  The circuits are shown in Figure 5,  and have been described  previously \n[9,  10].  These  circuits  have  been  used  by  workers  at  Caltech  to  implement VLSI \nmodels of central pattern generators.  The software simulation tools allow simulation \nof complex neural circuits  prior to fabrication,  to improve the likelihood of success \non first  silicon,  and  to allow optimization of shared parameters (bias wires). \n\n7  Conclusion \n\nNeuraLOG  and Spike  fill  a  need  for  a  fast  neural  simulator that  can  model large \nnetworks of biologically realistic spiking neurons.  The simple computational prim(cid:173)\nitives  within  Spike  can  be  used  to  create  complex  and  realistic  neural  behaviors \nin  arbitrarily  complex hierarchical  designs.  The  tools  are  publicly  available at  no \ncharge.  NeuraLOG  and  Spike  have  been  used  by  a  number  of research  labs  for \ndetailed modeling of biological systems. \n\nAcknowledgements \n\nNeuraLOG is a customization of the program AnaLOG, which was written by John \nLazzaro  and  David  Gillespie.  Lloyd  Watts  gratefully  acknowledges  helpful  dis(cid:173)\ncussions  with  Carver  Mead,  Sylvie  Ryckebusch,  Misha  Mahowald,  John  Lazzaro, \n\n\fEvent-Driven Simulation of Networks of Spiking Neurons \n\n933 \n\n.......................\u2022... ....... ..\u2022\u2022\u2022....... ~ ....................\u2022.....\u2022.............. ......\u2022...................... \n\nI emI IKJ \n\n. \n\nI \n\n~ ............ ~y.f!~.P.s.~ .............. I~~.i~ .. L ............................... ~~':Ir~!\" ............................ . \n\nFigure 5:  CMOS Analog VLSI circuit primitives.  The neuron circuit models \na  voltage-gated sodium channel  and  a  delayed  rectifier  potassium  channel \nto produce a spiking mechanism.  The tonic circuit allows constant currents \nto  be  injected  onto  the  membrane  capacitance em.  The  synapse  circuit \ncreates  a boxcar current  pulse in response  to a spike input. \n\nDavid  Gillespie,  Mike  Vanier,  Brad  Minch,  Rahul  Sarpeshkar,  Kwabena  Boahen, \nJohn Platt, and Steve Nowlan.  Thanks to Sylvie Ryckebusch for  permission to use \nher  CPG circuit  example. \n\nReferences \n\n[1]  J. Hertz,  A.  Krogh and R. Palmer,  Introduction  to  the  Theory  of Neural Com(cid:173)\n\nputation,  Addison-Wesley,  1991. \n\n[2]  N. Y-S.  Kiang, T. Watanabe, E.  C.  Thomas, L.  F. Clark,  \"Discharge Patterns \nof Single  Fibers  in  the  Cat's Auditory  Nerve\",  MIT  Res.  Monograph  No.  35, \n(MIT,  Cambridge, MA). \n\n[3]  M.  Konishi,  T.T.  Takahashi,  H.  Wagner,  W.E.  Sullivan,  C.E.  Carr,  \"Neuro(cid:173)\n\nphysiological  and  Anatomical Substrates  of Sound  Localization  in  the  Owl\", \nIn  Auditory Function,  G.M.  Edelman, W.E.  Gall, and W.M. Cowan, eds.,  pp. \n721-745, Wiley,  New  York. \n\n[4]  D.  P.  Phillips and  S.  E.  Hall,  \"Response  Timing Constraints on  the  Cortical \nRepresentation of Sound Time Structure\" , Journal of the Acoustical Society of \nAmerica, 88 (3),  pp.  1403-1411, 1990. \n\n[5]  R.R.  de  Ruyter  van  Steveninck  and  W.  Bialek,  \"Real-time Performance  of a \nmovement-sensitive  neuron  in  the  blowfly  visual  system:  Coding  and  infor-\n\n\f934 \n\nWatts \n\nmation transfer  in short  spike sequences\",  Proceedings  of the  Royal  Society  of \nLondon,  Series  B,  234, 379-414. \n\n[6]  G.  A.  Pratt,  Pulse  Computation,  Ph.D.  Thesis,  Massachusetts  Institute  of \n\nTechnology,  1989. \n\n[7]  M.  Wehr,  S.  Ryckebusch and G.  Laurent, Western Nerve Net Conference, Seat(cid:173)\n\ntle,  Washington,  1993. \n\n[8]  S.  Ryckebusch,  M.  Wehr,  and  G.  Laurent,  \"Distinct rhythmic locomotor pat(cid:173)\n\nterns can be generated by a simple adaptive neural circuit:  biology, simulation, \nand VLSI implementation\", in review,  Journal of Computational Neuroscience. \n[9]  R.  Sarpeshkar,  L.  Watts,  C.A.  Mead,  \"Refractory  Neuron  Circuits\",  Inter(cid:173)\n\nnal Memorandum, Physics of Computation Laboratory, California Institute of \nTechnology,  1992. \n\n[10]  L.  Watts,  \"Designing Networks of Spiking Silicon Neurons and Synapses\", Pro(cid:173)\n\nceedings of Computation and Neural Systems Meeting CNS*92, San Francisco, \nCA,1992. \n\n\fPART VIII \n\nVISUAL PROCESSING \n\n\f\f", "award": [], "sourceid": 834, "authors": [{"given_name": "Lloyd", "family_name": "Watts", "institution": null}]}