{"title": "A Spike Based Learning Neuron in Analog VLSI", "book": "Advances in Neural Information Processing Systems", "page_first": 692, "page_last": 698, "abstract": null, "full_text": "A  spike based learning neuron in analog \n\nVLSI \n\nPhilipp Hiifliger \n\nMisha Mahowald \n\nInstitute of Neuroinformatics \n\nInstitute of Neuroinformatics \n\nETHZjUNIZ \n\nGloriastrasse 32 \nCH-8006 Zurich \n\nSwitzerland \n\nETHZjUNIZ \n\nGloriastrasse 32 \nCH-8006  Zurich \n\nSwitzerland \n\ne-mail:  haftiger@neuroinf.ethz.ch \n\ne-mail:  misha@neuroinf.ethz.ch \n\ntel:  ++41 1 257  26  84 \n\ntel:  ++41  1 257  26  84 \n\nLloyd  Watts \nArithmos,  Inc. \n\n2730  San Tomas Expressway,  Suite 210 \n\nSanta Clara,  CA  95051-0952 \n\nUSA \n\ne-mail:  lloyd@arithmos.com \n\ntel:  408  982  4490, x219 \n\nAbstract \n\nMany  popular  learning  rules  are  formulated  in  terms  of continu(cid:173)\nous,  analog inputs  and outputs.  Biological  systems,  however,  use \naction  potentials,  which  are  digital-amplitude  events  that  encode \nanalog  information  in  the  inter-event  interval.  Action-potential \nrepresentations are now  being used  to advantage in neuromorphic \nVLSI  systems as  well.  We  report on a  simple learning rule,  based \non  the  Riccati  equation  described  by  Kohonen  [1],  modified  for \naction-potential  neuronal  outputs.  We  demonstrate  this  learning \nrule in an analog VLSI chip that uses volatile capacitive storage for \nsynaptic weights.  We  show that our time-dependent learning rule \nis  sufficient  to achieve  approximate weight  normalization and  can \ndetect temporal correlations in spike trains. \n\n\fA Spike Based Learning Neuron in Analog VLSI \n\n693 \n\n1 \n\nINTRODUCTION \n\nIt is an ongoing debate how information in the nervous system is encoded and carried \nbetween neurons.  In many subsystems of the brain it is  now believed that it is  done \nby  the exact  timing of spikes.  Furthermore spike  signals  on  VLSI  chips  allow  the \nuse of address-event busses  to solve the problem of the large connectivity in  neural \nnetworks [3, 4].  For these reasons our artificial neuron and others [2]  use spike signals \nto communicate.  Additionally  the weight  updates  at the synapses  are  determined \nby  the  relative  timing  of presynaptic  and  postsynaptic spikes,  a  mechanism  that \nhas recently been discovered to operate in cortical synapses  [5,  7,  6]. \n\nWeight normalization is  a useful property of learning rules.  In order to perform the \nnormalization,  some information about  the whole  weight  vector  must  be available \nat  every synapse.  We  use  the neuron's  output  spikes  (The  neuron's output is  the \nproduct of the weight  and the input vector), which retrogradely propagate through \nthe dendrites  to the  synapses  (as  has  been  observed  in  biological  neurons  [5]).  In \nour model  approximate normalization is  an implicit  property of the learning rule. \n\n2  THE LEARNING RULE \n\npresynaptic spikes \n\n7 \n\n.~ ____ L-~ ____ ~ ______ ~_ \n\ncorrelation signat \n\n3  ynaptic weight \n\npostsynaptic spikes \n\nFigure  1:  A  snapshot  of  the  simulation  variables  involved  at  one  synapse.  With \nr  =  0.838 \n\nThe core  of the learning  rule  is  a  local  'correlation  signal'  c  at every  synapse.  It \nrecords the 'history' of presynaptic spikes.  It is  incremented by 1 with every presy(cid:173)\nnaptic spike  and decays in  time with time constant  r: \n\nc(tm,O)  =  0 \n\nc(tm,n)  =  e-\n\nttn,n- t 171.,n-l \n\nT \n\nc(tm,n-d + 1 \n\nn>O \n\n(1) \n\n(n > 0)  is the time of the \ntm,o  is  the time of the m'th postsynaptic spike and tm,n \nn'th presynaptic spike after the m'th postsynaptic spike.  The weight changes when \nthe cell  fires  an action potential: \n\n\f694 \n\nP.  Hiijliger, M.  Mahowald and L  Watts \n\nw(tm,o)  =  W(tm-l,O) + ae-\n\ntTn  o-tTTl-l  \u2022 \n\n. \n\nT \n\n\u2022  c(tm-1,s)  - ,8w(tm- 1,o) \n\ns  =  max{v : tm-l ,v :s;  tm,o} \n\n(2) \n\nwhere  w  is  the  weight  at  this  synapse.  tm-l,s  means  the  last  event  (presynaptic \nor postsynaptic spike)  before the m'th postsynaptic spike.  a  and ,8  are parameters \ninfluencing learning speed and weight  vector normalization  (see  (5)). \n\nOur learning  rule  is  designed  to  react  to  temporal  correlations  between  spikes  in \nthe input  signals.  However,  to show the normalizing of the weights  we  analyze its \nbehavior by making some simplifying assumptions on the input and output signals; \ne.g.  the  intervals  of the  presynaptic  and  the  postsynaptic  spike  train  are  Poisson \ndistributed  and  there  is  no  correlation  between  single  spikes.  Therefore  we  can \nrepresent the signals by their instantaneous average frequencies  0  and i.  Now  the \nsimplified  learning rule can be written  as: \n\n:t w =  al(O)f - ,8wO \n\n(3) \n\n(4) \n\nl(O)  represents  the  average  percentage to  which  the  correlation  signal  is  reduced \nbetween  weight  updates  (output  spikes).  So  when  the neuron's average firing  rate \nfulfills  0  \u00bb  ~, one can  approximate l(O)  ~ 1.  (3)  is  thus  reduced  to  the  Riccati \nequation described by Kohonen [1].  This rule would not be Hebbian, but normalizes \nthe weight  vector (see  (5)).  Note that if the correlation signal does not decay,  then \nour  rule  matches  exactly  the  Riccati  equation.  We  will  further  refer  to  it  as  the \nModified  Riccati  Rule  (MRR).  Whereas  if  0  \u00ab  ~ then  l(O)  ~ Or,  which  is  a \nHebbian learning rule also described in  [1]. \nIf we assume that the spiking mechanism preserves 0  = wT f and insert it in  (3),  it \nfollows  for  the equilibrium state: \n\nIIwll  =  Jl(O)~ \n\n(5) \n\nSince l(O)  < 1 the weight vector will never be longer than Ii' This property also \n\nholds  when  the  simplifying  assumptions  are  removed.  The  vector  will  always  be \nsmaller,  as it is with no decay of the correlation signals, since the decay only affects \nthe incrementing part of the rule. \n\nMatters  get  much  more  complicated  with  the  removal  of  the  assumption  of  the \npre- and  postsynaptic  trains  being  independently  Poisson  distributed.  With  an \nintegrate-and-fire neuron for  instance,  or if there exist  correlations between spikes \nof the input trains, it is  no longer possible to express what happens in terms of rate \n\n\fA Spike Based Learning Neuron in Analog VLSI \n\n695 \n\ncoding only  (with  f  and  0).  (3)  is  still  valid  as  an  approximation  but  temporal \nrelationships between pre- and postsynaptic spikes become important.  Presynaptic \nspikes immediately followed by an action potential will have the strongest increasing \neffect  on the synapse's weight. \n\n3 \n\nIMPLEMENTATION IN  ANALOG VLSI \n\nWe  have  implemented  a  learning  rule  in  a  neuron  circuit  fabricated  in  a  2.0f..Lm \nCMOS  process.  This  neuron  is  a  preliminary  design  that  conforms  only  approxi(cid:173)\nmately to the MRR.  The neuron uses  an integrate-and-fire mechanism to generate \naction potentials  (Figure 2). \n\nFigure  2:  Integrate-and-fire neuron.  The  sarna  capacitor holds the  somatic mem(cid:173)\nbrane  voltage.  This voltage  is  compared  to  a  threshold  thresh with  a  differential \npair.  When it crosses this threshold it gets pulled up through the mirrored current \nfrom  the  differential  pair.  This  same  current  gets  also  mirrored  to the  right  and \nstarts to pull up a second leaky capacitor (setback)  through a small W / L transistor, \nso  this  voltage  rises  slowly.  This  capacitor  voltage finally  opens  a  transistor that \npulls  sarna  back to ground where it restarts integrating the incoming current.  The \nparameters tonic+  and  tonic- are  used  to  add  or subtract  a  constant  current  to \nthe soma capacitor.  tre! allows the spike-width to be changed. \n\nNot  shown,  but  also  part  of the  neuron,  are  two  non-learning  synapses:  one  ex(cid:173)\ncitatory  and  one  inhibitory.  Each  of  three  learning  synapses  contains  a  storage \ncapacitor for  the synaptic weight and for  the correlation signal (Figure 3). \n\nThe correlation  signal  c is  simplified  to  a  binary  variable  in  this  implementation. \nWhen an input spike occurs, the correlation signal is set to 1.  It is set to 0 whenever \nthe neuron produces an output-spike or after a fixed  time-period  (T in  (7))  if there \nis  no  other input spike: \n\nc(tm,o)  =  0 \nc(tm,n)  =  1 \n\nn  > 0 \n\n, \n\ntm,n::;  tm+1,O \n\n(6) \n\nThis  approximation  unfortunately  tends  to  eliminate  differences  between  highly \nactive inputs and weaker inputs.  Nevertheless the weight changes with every output \nspike: \n\n\f696 \n\nP.  Hiijiiger, M.  Mahowald and L.  Watts \n\nFigure 3:  The  CMOS  learning-synapse incorporates the learning mechanism.  The \nweight capacitor holds  the weight,  the  carr  capacitor stores the correlation signal \nrepresentation.  The  magnitude  of the  weight  increment  and  decrement  are  com(cid:173)\nputed  by  a  differential  pair  (upper  left  w50).  These  currents  are  mirrored  to  the \nsynaptic weight and gated by  digital switches encoding the state of the correlation \nsignal  and  of  the  somatic  action  potential.  The  correlation  signal  reset  is  medi(cid:173)\nated  by  a  leakage  transistor,  decayin,  which  has  a  tonic  value,  but  is  increased \ndramatically when  the output neuron fires. \n\nif C{tm-l s)  =  1 and tm 0  -\n, \notherwise \n\n\" \n\ntm-l s < T \n\n(7) \n\nw  is  the weight  on one synapse, c is  the correlation signal of that synapse, and a  is \na parameter that controls how fast the weight changes.  (See in the previous section \nfor  a  description of tm,n.)  The weight,  W50,  is  the equilibrium value of the synaptic \nweight  when  the  occurrence  of an  input  spike  is  fifty  percent  correlated  with  the \noccurrence of an output spike.  This implementation differs from the Riccati rule in \nthat either the weight increment or the weight decrement, but not both, are executed \nupon  each output  spike.  Also,  the  weight  increment  is  a  function  of the  synaptic \nweight.  The  circuit  was  implemented  this  way  to try  and  achieve  an  equilibrium \nvalue for the synaptic weight equal to the fraction of the time that the input neuron \nfired  relative  to the times  the output neuron fired.  This is  the  correct equilibrium \nvalue for  the synaptic weight in the Riccati rule.  The evolution of a synaptic weight \nis  depicted in Figure 4. \n\nThe synaptic  weight  vector  normalization  in this  implementation  is  accurate only \nwhen  the assumptions  of the  design  are met.  These assumptions  are that there is \none  or fewer  input  spikes  per  synapse  for  every  output  spike.  This  assumption  is \neasier to meet when there are many synapses formed with the neuron, so that spikes \nfrom  multiple  inputs  combine  to  drive  the  cell  to  threshold.  Since  we  have  only \nthree  synapses,  this  approximation  is  usually  violated.  Nevertheless,  the  weights \ncompete with one  another and therefore the length of the weight  vector is  limited. \nCompetition  between synaptic weights  occurs  because if one  weight  is  stronger,  it \ncauses the output neuron to spike and this suppresses the other input that has not \n\n\fA Spike Based Learning Neuron in Analog VLSI \n\n697 \n\nfired.  Future revision of the chip will  conform  more closely to the MRR. \n\n20 \n\n18 \n\n8.S \n\n9 \n\ntlma \n\n9.S \n\n10 \n\nX 10-3 \n\nFigure  4:  A  snapshot of the learning behavior of a  single  VLSI  synapse:  The top \ntrace  is  the  neuron  output  (IV/division),  the  upper  middle  trace  is  the  synap(cid:173)\ntic  weight  (lower  voltage means  a  stronger synaptic weight)  (25mV /division),  the \nlower middle trace is a representation of the correlation signal (1  V /division)(it has \ninverted sense  too)  and the bottom trace is  the presynaptic activity  (1 V / division) . \nThe weight changes only when an output spike occurs.  The timeout of the correla(cid:173)\ntion signal is realized with a decay and a threshold.  If the correlation signal is above \nthreshold, the weight  is  strengthened.  If the signal has decayed below threshold at \nthe time of an output spike, the weight is  weakened.  The magnitude of the change \nof the  weight  is  a  function  of the  absolute  magnitude  of the  weight.  This  weight \nwas weaker than W50,  so the increments are bigger than the decrements. \n\n4  TEMPORAL CORRELATION IN INPUT SPIKE \n\nTRAINS \n\nFigure 5 illustrates the ability of our learning rule to detect temporal correlations in \nspike trains.  A  simulated neuron strengthens those two synapses that receive  40% \ncoincident spikes,  although all four synapses get the same average spike frequencies. \n\n5  DISCUSSION \n\nLearning rules that make use of temporal correlations in their spike inputs/outputs \nprovide  biologically relevant  mechanisms of synapse modification  [5,  7,  6].  Analog \nVLSI implementations allow such models to operate in real time.  We plan to develop \nsuch  analog VLSI  neurons  using floating  gates for  weight  storage and  an address(cid:173)\nevent  bus  for  interneuronal  connections.  These  could  then  be  used  in  realtime \napplications in  adaptive 'neuromorphic' systems. \n\nAcknowledgments \n\nWe  thank the following  organizations for  their support:  SPP Neuroinformatik des \nSchweizerischen  Nationalfonds,  Centre  Swiss  d'Electronique et de  Microtechnique, \nU.S.  Office  of Naval  Research and the Gatsby Charitable Foundation. \n\n\f698 \n\nP.  Hiijiiger, M.  Mahowald and L.  Watts \n\nsynapse 2 \n\n0.4 \n\n0.36 \n\n0 .3 \n\n015 \n\n0.1 \n\nO\u00b7060l..----:-O:,00--='200\":----,300~~400.,----='\"=\"--\":-----=-'700 \n\ntimers) \n\nFigure  5:  In this  simulation we  use  a  neuron  with  four  synapses.  All  of them  get \ninput trains of the same average frequency  (20Hz).  Two of those  input trains  are \nthe result of independent Poisson processes (synapses 3 and 4), the other two are the \ncombination oftwo Poisson processes (synapses 1 and 2):  One that is independent of \nany other (12Hz) and one that is shared by the two with slightly different time delays \n(8Hz):  Synapse  1 gets those  coincident spikes 0.01  seconds earlier than synapse 2. \nSynapse 2 gets stronger because when it together with synapse 1 triggered an action \npotential,  it was  the last  synapse  being  active  before the postsynaptic spike.  The \nparameters were:  Q  =  0.004, {3  =  0.02, T  =  11ms \n\nReferences \n\n[1]  Thevo  Kohonen.  Self-Organization  and  Associative  Memory.  Springer,  Berlin, \n\n1984. \n\n[2]  D.K.  Ferry L.A. Akers and R.O. Grondin. Synthetic neural systems in the 1990s. \nAn introduction  to  neural  and  electronic  networks,  Academic Press  (Zernetzer, \nDavis,  Lau,  McKenna),  pages 359-387, 1995. \n\n[3]  J.  Lazzaro, J.  Wawrzynek,  M.  Mahowald,  M.  Sivilotti, and D.  Gillespie.  Silicon \nauditory  processors  as  computer  peripherals.  IEEE  Trans.  Neural  Networks, \n4:523-528,  1993. \n\n[4]  A.  Mortara and E.  A.  Vittoz.  A communication architecture tailored for  analog \nVLSI  artificial  neural  networks:  intrinsic  performance  and  limitations.  IEEE \nTranslation  on Neural  Networks,  5:459-466,  1994. \n\n[5]  G.  J.  Stuart and B.  Sakmann.  Active  propagation of somatic action  potentials \n\ninto neocortical pyramidal cell  dendrites.  Nature,  367:600,  1994. \n\n[6]  M.  V.  Tsodyks  and  H.  Markram.  Redistribution  of synaptic  efficacy  between \n\nneocortical pyramidal neurons.  Nature,  382:807-810, 1996. \n\n[7]  R.  Yuste  and W.  Denk.  Dendritic spines  as  basic functional  units  of neuronal \n\nintegration.  Nature,  375:682-684, 1995. \n\n\f", "award": [], "sourceid": 1322, "authors": [{"given_name": "Philipp", "family_name": "H\u00e4fliger", "institution": null}, {"given_name": "Misha", "family_name": "Mahowald", "institution": null}, {"given_name": "Lloyd", "family_name": "Watts", "institution": null}]}