{"title": "Synchronization, oscillations, and 1/f noise in networks of spiking neurons", "book": "Advances in Neural Information Processing Systems", "page_first": 629, "page_last": 636, "abstract": "", "full_text": "Synchronization, oscillations, and 1/ f \nnoise in  networks of spiking neurons \n\nMartin Stemmler, Marius  Usher, and Christof Koch \n\nPasadena, CA  91125 \n\nComputation and  Neural Systems,  139-74 \n\nCalifornia Institute of Technology \n\nZeev  Olami \n\nDept.  of Chemical  Physics \n\nWeizmann Institute of Science \n\nRehovot 76100,  Israel \n\nAbstract \n\nWe investigate a model for neural activity that generates long range \ntemporal correlations, 1/ f  noise, and oscillations in global activity. \nThe model  consists  of a  two-dimensional sheet  of leaky  integrate(cid:173)\nand-fire  neurons  with feedback  connectivity consisting of local ex(cid:173)\ncitation  and  surround  inhibition.  Each  neuron  is  independently \ndriven  by  homogeneous  external  noise.  Spontaneous  symmetry \nbreaking occurs,  resulting in  the formation of \"hotspots\"  of activ(cid:173)\nity in  the  network.  These  localized  patterns  of excitation  appear \nas  clusters  that coalesce,  disintegrate, or fluctuate  in size  while si(cid:173)\nmultaneously moving in a random walk constrained by the interac(cid:173)\ntion  with other  clusters.  The emergent  cross-correlation functions \nhave  a  dual  structure,  with  a  sharp  peak  around  zero  on  top  of \na  much broader  hill .  The  power  spectrum  associated  with  single \nunits  shows a  1/ f  decay for  small frequencies  and is flat  at higher \nfrequencies,  while the power spectrum of the spiking activity aver(cid:173)\naged over many cells-equivalent to the local field  potential-shows \nno  1/ f  decay  but a prominent peak  around 40  Hz. \n\n629 \n\n\f630 \n\nStemmler, Usher, Koch, and Olami \n\n1  The model \n\nThe  model  consists  of a  100-by-l00  lattice  of integrate-and-fire  units  with  cyclic \nlattice  boundary  conditions.  Each  unit  represents  the  nerve  cell  membrane  as  a \nsimple  RC  circuit  (r  = 20  msec)  with  the  addition  of  a  reset  mechanism;  the \nrefractory  period TreJ  is  equal to one iteration step  (1  msec). \n\nUnits are connected to each other within the layer by local excitatory and inhibitory \nconnections  in  a  center-surround pattern.  Each  unit  is  excitatorily  connected  to \nN  = 50  units  chosen  from a  Gaussian probability distribution of u  = 2.5  (in  terms \nof the  lattice  constant),  centered  at  the  unit's  position  N  inhibitory  connections \nper  unit  are  chosen from  a  uniform probability distribution on  a  ring eight to  nine \nlattice constants away. \n\nOnce  a  unit  reaches  the  threshold  voltage,  it emits  a  pulse  that  is  transmitted  in \none iteration (1  msec)  to connected  neighboring units,  and the potential is  reset  by \nsubtracting the threshold from  resting  potential. \n\n\\Ii(t + 1) = (exp( -l/r)\\Ii(t) + h (t)) O[vth  - V(t)]. \n\n(1) \nIi  is  the input  current,  which is  the sum of lateral currents from  presynaptic  units \nand  external  current.  The  lateral  current  leads  to  an  increase  (decrease)  in  the \nmembrane  potential  of excitatory  (inhibitorily  )  connected  cells.  The  weight  of \nthe  excitation  and  inhibition,  in  units  of voltage  threshold,  is  ~ and  J3 ~.  The \nvalues  a  =  1.275  and  J3  =  0.67  were  used  for  simulations.  The  external  input  is \nmodeled  independently  for  each  cell  as  a  Poisson  process  of excitatory  pulses  of \nmagnitude 1/ N, arriving at a  mean rate \"ext.  Such a simple cellular model mimics \nreasonably  well  the  discharge  patterns  of cortical  neurons  [Bernander et al.,  1994, \nSoftky and  Koch,  1993]. \n\n2  Dynamics and  Pattern Formation \n\nIn  the  mean-field  approximation,  the  firing  rate  of an  integrate-and-fire  unit  is  a \nfunction  of the input current  [Amit  and Tsodyks,  1991]  given  by \n\nf(I) = (TreJ  - r In[l - 1/(1 r)])-l, \n\n(2) \n\n(3) \n\nwhere  Tref  is  the refractory  period  and  r  the  membrane time constant. \n\nIn  this  approximation, the dynamics associated  with eq.  1 simplify to \n\n~i =  -Ii + L j Wijf(Ij) + It xt , \n\nwhere  Wij  represents  the connection  strength matrix from unit j  to unit i. \nHomogeneous firing  activity throughout  the network will result  as  long as  the con(cid:173)\nnectivity pattern satisfies W(k)-l < 0 for all k,  where W(k) is the Fourier transform \nof Wij .  As  one increases  the total strength  of lateral connectivity,  clusters  of high \nfiring activity develop.  These clusters form a hexagonal grid across  the network; for \neven  stronger lateral currents,  the clusters  merge to form stripes. \n\nThe transition from a homogeneous state to hexagonal clusters to stripes is generic \nto  many  nonequilibrium  systems  in  fluid  mechanics,  nonlinear  optics,  reaction(cid:173)\ndiffusion  systems,  and  biology.  (The  classic  theory  for  fluid  mechanics  was  first \n\n\fSynchronization, Oscillations, and  IlfNoise in Networks of Spiking Neurons \n\n631 \n\ndeveloped  by  [Newell and Whitehead,  1969],  see  [Cross  and  Hohenberg,  1993]  for \nan extensive review.  Cowan (1982) was the first  to suggest  applying the techniques \nof fluid  mechanics to  neural  systems.) \n\nThe richly varied dynamics of the model, however,  can not be captured by a  mean(cid:173)\nfield  description.  Clusters  in  the  quasi-hexagonal  state  coalesce,  disintegrate,  or \nfluctuate  in size  while simultaneously moving in a  random walk  constrained by the \ninteraction with other clusters. \n\nR~ndom Walk  of  Clusters \n\n16 \n\n14 \nE  12 \n... \n\" \" \nt'.: B  8 \n; \n\n10 \n\n6 \n\no~~--~~--~~--~~--~~ \n\n14 \n\n16 \n\n18 \n\no \n\n2 \n\n6 \n12 \nx  (latt~ce  un~t~) \n\n10 \n\n8 \n\nFigure  1:  On  the  left,  the  summed firing  activity for  the  network over  50  msec  of \nsimulation is shown.  Lighter shades denote higher firing rates (maximum firing rate \n120  Hz).  Note  the  nearly  hexagonal  pattern  of clusters  or  \"hotspots\"  of activity. \nOn the right, we  illustrate the motion of a typical cluster.  Each vertex in the graph \nrepresents a tracked cluster's position averaged over 50 msec.  Repulsive interactions \nwith surrounding clusters generally constrain the motion to remain within a certain \nradius.  This  vibratory  motion  of a  cluster  is  occasionally  punctuated  by longer(cid:173)\nrange  diffusion. \n\nStatistical fluctuations,  diffusion  and synchronization of clusters,  and  noise  in  the \nexternal  input  driving  the  system lead  to  1/ I-noise  dynamics,  long-range  correla(cid:173)\ntions,  and  oscillations  in  the  local  field  potential.  These  issues  shall  be  explored \nnext. \n\n3  1/ f  Noise \n\nThe  power spectra  of spike  trains from  individual  units  are  similar  to  those  pub(cid:173)\nlished  in  the  literature  for  nonbursting  cells  in  area  MT  in  the  behaving  mon(cid:173)\nkey  [Bair et  al.,  1994].  Power  spectra  were  generally  flat  for  all  frequencies  above \n100  Hz.  The effective  refractory  period  present  in  an  integrate-and-fire  model  in(cid:173)\ntroduces  a  dip  at  low  frequencies  (also  seen  in  real  data).  Most  noteworthy is  the \nl/lo.s  component  in  the  power spectrum  at  low  frequencies.  Notice  that in  order \nto see  such  a  decay for  very  low frequencies  in  the spectrum,  single  units  must  be \nrecorded for  on the order of 10-100 sec,  longer than the recording time for  a  typical \ntrial in neurophysiology. \n\nWe  traced  a  cluster  of neuronal  activity  as  it  diffused  through  the  system,  and \n\n\f632 \n\nStemmler. Usher. Koch. and Olami \n\n3r-----~----~------~----_r----~ \n\nSpike  Tra~n Power  Spectrum \n\nlSI  distribution \n\n2.5 \n\n2 \n\n1.5 \n\n1 \n\n0.5 \n\n0.7 \n0.5 \n\n... \n\n0.3 \n0.2 \n0.15 \n\n0.1 \n\n20 \n\n40 \n\nHz \n\n60 \n\n80 \n\n100 \n\n30. \n\n50. \n\n70. \n\n100. \n\n150.  200. \n\nmsec \n\nFigure 2:  Typical power spectrum  and lSI distribution of single  units over 400  sec \nof simulation.  At  low  frequencies,  the  power  spectrum  behaves  as  f- O.S\u00b1O.017  up \nto  a  cut-off frequency  of ~ 8  Hz.  The lSI  distribution on  the  right  is  shown  on  a \nlog-log scale.  The lSI histogram decays  as  a  power law  pet) ex  t-1.70\u00b1O.02  between \n25  and 300  msec.  In  contrast,  a  system with randomized network connections  will \nhave a  Poisson-distributed lSI histogram which decays exponentially. \n\nmeasured  the lSI distribution at a  fixed  point relative to the  cluster  center.  In the \ncluster frame of reference,  activity should remain fairly  constant, so  we expect  and \ndo find an interspike interval (lSI) distribution with a single characteristic relaxation \ntime: \n\nPr(t) =  A(r)exp(-tA(r)) , \n\nwhere  the firing  rate  A(r)  is  now only a function  of the distance r  in cluster  coordi(cid:173)\nnates.  Thus Pr(t)  is  always  Poisson for  fixed  r. \nIf a  cluster  diffuses  slowly  compared  to  the  mean  interspike  interval,  a  unit  at  a \nfixed  position samples  many lSI distributions of varying A(r)  as  the cluster  moves. \nThe lSI  distribution in the fixed  frame reference  is  thus \n\npet) = j A(r)2 exp( -t A(r\u00bb)dr. \n\n(4) \n\nDepending  on  the functional  form  of A(r),  pet)  (the  lSI  distribution for  a  unit  at \na  fixed position)  will decay  as  a  power law,  and  not as an exponential.  Empirically, \nthe  distribution of firing rates  as  a function of r  can  be  approximated (roughly)  by \na  Gaussian.  A  Gaussian  A(r)  in eq.  4  leads  to  pet)  f'oi  t- 2  for  t  at  long  times.  In \nturn,  a  power-law (fractal)  pet) generates  1/ f  noise  (see  Table 1). \n\n4  Long-Range  Cross-Correlations \n\nExcitatory cross-correlation functions for  units separated by small distances consist \nof a  sharp  peak  at  zero  mean time  delay followed  by  a  slower  decay  characterized \nby a  power law with exponent -0.21 until the function reaches an asymptotic level. \nNelson  et  al.  (1992)  found  this type of cross-correlation  between neurons-a  \"castle \non a hill\" -to be the most common form of correlation in cat visual cortex.  Inhibitory \n\n\fSynchronization, Oscillations, and  lifNoise in Networks of Spiking Neurons \n\n633 \n\ncross-correlations  show  a  slight  dip  that  is  much  less  pronounced  than  the  sharp \nexcitatory peak  at short time-scales. \n\nCross-Correlation  at  d \n\n1 \n\n1000  -\n\n750  -\n\n500  -\n\n250 \n\n1000  -\n\n750  -\n\n500 \n\n250 \n\n-300 \n\n-200 \n\n-100 \n\no \n\nmsec \n\n100 \n\n200 \n\n300 \n\nCross-Correlation  at  d \n\n9 \n\n-300 \n\n-200 \n\n-100 \n\no \n\nmsec \n\n100 \n\n200 \n\n300 \n\nFigure  3:  Cross-correlation  functions  between  cells  separated  by  d  units  of the \nlattice.  Given the  center-surround geometry of connections,  the  upper  curve  corre(cid:173)\nsponds  to  mutually excitatory  coupling and  the lower  to  mutually inhibitory  cou(cid:173)\npling.  Correlations  decay  as  l/t O.21 ,  consistent  with  a  power  spectrum  of single \nspike trains that behaves as  1/ fo .8. \n\nSince  correlations  decay  slowly  in  time  due  to  the  small  exponent  of the  power, \nlong temporal fluctuations in the firing  rate result, as the 1/ f-type power spectra of \nsingle spike  trains demonstrate.  These fluctuations  in turn lead to  high variability \nin the  number of events over a fixed  time period. \n\nIn  fact,  the  decay  in  the  auto-correlation  and  power  spectrum,  as  well  as  the  rise \nin  the  variability  in  the  number  of events,  can  be  related  back  to  the  slow  de(cid:173)\nIf the  lSI  distribution  decays \ncay  in  the  interspike  interval  (lSI)  distribution. \nas  a  power  law  pet)  ,....,  t- II ,  then  the  point  process  giving  rise  to  it  is  fractal \nwith  a  dimension  D  =  v-I [Mandelbrot,  1983].  Assuming  that  the  simula(cid:173)\ntion  model  can  be  described  as  a  fully  ergodic  renewal  process,  all  these  quanti(cid:173)\nties  will  scale  together  [Cox and  Lewis,  1966,  Teich,  1989,  Lowen and Teich,  1993, \nUsher  et  al.,  1994]: \n\n\f634 \n\nStemmler, Usher, Koch, and Olami \n\nTable 1:  Scaling Relations and  Empirical Results \n\nVar(N) \n\nAuto-correlation  Power Spectrum \n\nlSI  Distribution \n\nVar(N)  \"-J  Nil \n\nA(t)  \"-J  t ll - 2 \n\nS(I)  \"-J  /-11+1 \n\npet) \"\"'  t- II \n\nVar(N) '\" N1.54 \n\nA(t)  \"-J  t- 0 .21 \n\nS(I) \"\"'  /-0.81 \n\npet)  \"-J  c1. 7O \n\nThese relations will be only approximate if the process is  nonrenewal or nonergodic, \nor if power-laws hold over a  limited range.  The process  in the model is  clearly non(cid:173)\nrenewal, since  the presence  of a  cluster  makes consecutive short interspike intervals \nfor  units within that cluster more likely than in a renewal process.  Hence,  we expect \nsome  (slight)  deviations from the scaling relations outlined above. \n\n5  Cluster Oscillations and  the Local  Field Potential \n\nThe interplay between the  recurrent  excitation  that leads to nucleation of clusters \nand the  \"firewall\" of inhibition that restrains activity causes clusters of high activity \nto  oscillate  in  size.  Fig 4  is  the  power spectrum of ensemble  activity over  the size \nof a  typical cluster. \n\nPower  Spectrum  of  Cluster  ActlVlty  withln  radlus  d=9 \n\n25 \n\n20 \n\n15 \n\n10 \n\n10-4 \n(lJ \n:J: \n0 \nP... \n\n5 \n\n0 \n\n0 \n\n20 \n\n40 \n\n60 \nHz \n\n80 \n\n100 \n\nFigure 4:  Power spectrum of the  summed  spiking activity over a  circular  area the \nsize  of a  single  cluster  (with  a  radius  of 9 lattice constants)  recorded  from  a  fixed \npoint on  the  lattice for  400  seconds.  Note  the  prominent  peak  centered  at 43  Hz \nand the loss  of the  1// component seen  in  the single  unit power spectra (Fig.  2). \n\nThese oscillations  can  be  understood  by examining the  cross-correlations  between \ncells.  By the Wiener-Khinchin theorem, the power spectrum of cluster activity is the \nFourier transform of the  signal's auto-correlation.  Since  the  cluster  activity is  the \nsum of all single-unit spiking activity within a cluster of N  cells, the autocorrelation \nof the cluster spiking activity will be the sum of N  auto-correlations functions of the \n\n\fSynchronization, Oscillations, and  lifNoise in Networks of Spiking Neurons \n\n635 \n\nindividual cells  and  N  x  (N - 1)  cross-correlation  functions  among individual cells \nwithin the  cluster.  The ensemble activity is thus dominated by cross-correlations. \n\nIn general, the excitatory  \"castles\"  are sharp relative to the broad dip in the cross(cid:173)\ncorrelation  due  to  inhibition  (see  Fig.  3). \nIn  Fourier  space,  these  relationships \nare  reversed:  broader Fourier transforms of excitatory cross-correlations  are paired \nwith narrower Fourier transforms  of inhibitory cross-correlations.  Superposition  of \nsuch  transforms  leads  to a  peak  in the  30-70  Hz  range and  cancellation of the  1/ f \ncomponent which was present the  single unit  power spectrum. \n\nInterestingly, the power spectra of spike trains of individual cells within the network \n(Fig.  2)  show  no  evidence  of a  peak  in  this  frequency  band.  Diffusion  of clusters \ndisrupts  any phase  relationship between single  unit firing  and ensemble activity. \n\nThe ensemble activity corresponds  to the local field  potential in neurophysiological \nrecordings.  While  oscillations  between  30  and  90  Hz  have often  been  seen  in  the \nlocal field  potential (or  sometimes even  in the  EEG)  measured  in cortical  areas  in \nthe  anesthetized  or  awake cat  and  monkey, these  oscillations are frequently  not  or \nonly weakly visible in multi- or single-unit data (e.g., [Eeckman and Freeman,  1990, \nKreiter  and Singer,  1992,  Gray et  al.,  1990,  Eckhorn et al.,  1993]).  We here offer  a \ngeneral  explanation for  this phenomenon. \n\nAcknowledgments:  We  are  indebted  to  William Softky,  Wyeth  Bair,  Terry  Se(cid:173)\njnowski,  Michael  Cross,  John  Hopfield,  and  Ernst  Niebur,  for  insightful  discus(cid:173)\nsions.  Our  research  was  supported  by  a  Myron  A.  Bantrell  Research  Fellowship, \nthe Howard Hughes  Medical Institute,  the National Science  Foundation, the Office \nof Naval Research  and the Air  Force Office  of Scientific Research. \n\nReferences \n\n[Amit and Tsodyks,  1991]  Amit,  D.  J.  and  Tsodyks,  M.  V.  (1991).  Quantitative \nstudy of attractor neural network retrieving at low rates: 1. substrate spikes, rates \nand  neuronal gain.  Network  Com.,  2(3):259-273. \n\n[Bair et  al.,  1994]  Bair, W., Koch, C., Newsome, W., and Britten, K.  (1994).  Power \nspectrum analysis of MT neurons in the behaving monkey.  J.  Neurosci., in press. \n[Bernander  et  al.,  1994]  Bernander, 0., Koch, C.,  and Usher,  M.  (1994).  The effect \nof synchronized  inputs at the single  neuron level.  Neural  Computation,  in  press. \n[Cowan,  1982]  Cowan, J. D.  (1982).  Spontaneous symmetry breaking in large scale \n\nnervous activity.  Int.  J.  Quantum  Chemistry,  22:1059-1082. \n\n[Cox and  Lewis,  1966]  Cox, D. and Lewis,  P. A. W. (1966).  The  Statistical Analysis \n\nof Series  of Events.  Chapman and  Hall,  London. \n\n[Cross  and  Hohenberg,  1993]  Cross,  M.  C.  and  Hohenberg,  P.  C.  (1993).  Pattern \n\nformation outside of equilibrium.  Rev.  Mod.  Phys.,  65(3):851-1112. \n\n[Eckhorn et  al.,  1993]  Eckhorn, R., Frien, A.,  Bauer, R., Woelbern, T., and Harald, \nK. (1993).  High frequency (60-90 hz) oscillations in primary visual cortex of awake \nmonkey.  Neuroreport, 4:243-246. \n\n\f636 \n\nStemmler, Usher, Koch, and Olami \n\n[Eeckman  and  Freeman,  1990]  Eeckman, F . and  Freeman, W.  (1990).  Correlations \nbetween unit firing  and EEG in the rat olfactory system.  Brain Res., 528(2):238-\n244. \n\n[Grayet al.,  1990]  Gray,  C.  M.,  Engel,  A.  K.,  Konig,  P.,  and  Singer,  W.  (1990) . \nStimulus  dependent  neuronal  oscillations  in  cat  visual  cortex:  receptive  field \nproperties  and feature  dependence.  Europ.  J.  Neurosci., 2:607-619. \n\n[Kreiter  and Singer,  1992]  Kreiter,  A.  K.  and  Singer,  W.  (1992).  Oscillatory neu(cid:173)\n\nronal  responses  in  the  visual  cortex  of the  awake  macaque  monkey.  Europ.  J. \nNeurosci.,  4:369-375. \n\n[Lowen and Teich, 1993]  Lowen,  S.  B.  and  Teich,  M.  C.  (1993).  Fractal  renewal \n\nprocesses  generate  Iff noise.  Phys.  Rev.  E,  47(2):992-1001. \n\n[Mandelbrot,  1983]  Mandelbrot,  B.  B.  (1983).  The  fractal geometry  of nature.  W. \n\nH.  Freeman,  New  York. \n\n[Nelson  et al.,  1992]  Nelson,  J.  I.,  Salin,  P.  A.,  Munk,  M.  H.-J.,  Arzi,  M.,  and \nBullier, J. (1992).  Spatial and temporal coherence in cortico-cortical connections: \nA  cross-correlation  study  in  areas  17  and  18  in  the  cat.  Visual  Neuroscience, \n9:21-38. \n\n[Newell and Whitehead,  1969]  Newell,  A.  C.  and  Whitehead,  J.  A.  (1969).  Finite \n\nbandwidth, finite  amplitude convection.  J.  Fluid Mech ., 38:279-303. \n\n[Softky and  Koch,  1993]  Softky, W.  R.  and  Koch, C.  (1993).  The highly  irregular \nfiring of cortical cells is inconsistent with temporal integration of random EPSPs. \nJ.  Neurosci.,  13(1):334-350. \n\n[Teich,  1989]  Teich,  M.  C.  (1989).  Fractal  character  of the  auditory  neural  spike \n\ntrain.  IEEE  Trans.  Biomed.  Eng.,  36(1):150-160. \n\n[Usher  et al.,  1994]  Usher,  M.,  Stemmler, M., Koch, C., and Olami, Z.  (1994).  Net(cid:173)\n\nwork  amplification of local fluctuations  causes  high spike rate variability, fractal \nfiring  patterns,  and  oscillatory  local  field  potentials.  Neural  Computation,  in \npress. \n\n\fPART V \n\nCONTROL, \n\nNAVIGATION, AND \n\nPLANNING \n\n\f\f", "award": [], "sourceid": 736, "authors": [{"given_name": "Martin", "family_name": "Stemmler", "institution": null}, {"given_name": "Marius", "family_name": "Usher", "institution": null}, {"given_name": "Christof", "family_name": "Koch", "institution": null}, {"given_name": "Zeev", "family_name": "Olami", "institution": null}]}