{"title": "Correlation Functions in a Large Stochastic Neural Network", "book": "Advances in Neural Information Processing Systems", "page_first": 471, "page_last": 476, "abstract": "", "full_text": "Correlation Functions in a Large \n\nStochastic Neural Network \n\nIris Ginzburg \n\nSchool of Physics and Astronomy \n\nRaymond and Beverly Sackler Faculty of Exact Sciences \n\nTel-Aviv University \nTel-Aviv 69978, Israel \n\nRacah Institute of Physics and Center for Neural Computation \n\nHaim Sompolinsky \n\nHebrew University \n\nJerusalem 91904, Israel \n\nAbstract \n\nMost theoretical investigations of large recurrent networks focus on \nthe properties of the macroscopic order parameters such as popu(cid:173)\nlation averaged activities or average overlaps with memories. How(cid:173)\never, the statistics of the fluctuations in the local activities may \nbe an important testing ground for comparison between models \nand observed cortical dynamics. We evaluated the neuronal cor(cid:173)\nrelation functions in a stochastic network comprising of excitatory \nand inhibitory populations. We show that when the network is in \na stationary state, the cross-correlations are relatively weak, i.e., \ntheir amplitude relative to that of the auto-correlations are of or(cid:173)\nder of 1/ N, N being the size of the interacting population. This \nholds except in the neighborhoods of bifurcations to nonstationary \nstates. As a bifurcation point is approached the amplitude of the \ncross-correlations grows and becomes of order 1 and the decay time(cid:173)\nconstant diverges. This behavior is analogous to the phenomenon \nof critical slowing down in systems at thermal equilibrium near a \ncritical point. Near a Hopf bifurcation the cross-correlations ex(cid:173)\nhibit damped oscillations. \n\n471 \n\n\f472 \n\nGinzburg and Sompolinsky \n\n1 \n\nINTRODUCTION \n\nIn recent years there has been a growing interest in the study of cross-correlations \nbetween the activities of pairs of neurons in the cortex. In many cases the cross(cid:173)\ncorrelations between the activities of cortical neurons are approximately symmetric \nabout zero time delay. These have been taken as an indication of the presence of \n\"functional connectivity\" between the correlated neurons (Fetz, Toyama and Smith \n1991, Abeles 1991). However, a quantitative comparison between the observed \ncross-correlations and those expected to exist between neurons that are part of a \nlarge assembly of interacting population has been lacking. \n\nMost of the theoretical studies of recurrent neural network models consider only time \naveraged firing rates, which are usually given as solutions of mean-field equations. \nThey do not account for the fluctuations about these averages, the study of which \nrequires going beyond the mean-field approximations. In this work we perform a \ntheoretical study of the fluctuations in the neuronal activities and their correlations, \nin a large stochastic network of excitatory and inhibitory neurons. Depending on the \nmodel parameters, this system can exhibit coherent undamped oscillations. Here we \nfocus on parameter regimes where the system is in a statistically stationary state, \nwhich is more appropriate for modeling non oscillatory neuronal activity in cortex. \nOur results for the magnitudes and the time-dependence of the correlation functions \ncan provide a basis for comparison with physiological data on neuronal correlation \nfunctions. \n\n2 THE NEURAL NETWORK MODEL \n\nWe study the correlations in the activities of neurons in a fully connected recurrent \nnetwork consisting of excitatory and inhibitory populations. The excitatory con(cid:173)\nnections between all pairs of excitatory neurons are assumed to be equal to J / N \nwhere N denotes the number of excitatory neurons in the network. The excitatory \nconnections from each of the excitatory neurons to each of the inhibitory neurons \nare J' / N. The inhibitory coupling of each of the inhibitory neurons onto each of \nthe excitatory neurons is K / M where M denotes the number of inhibitory neurons. \nFinally, the inhibitory connections between pairs of inhibitory neurons are ](' / M. \nThe values of these parameters are in units of the amplitude of the local noise (see \nbelow). Each neuron has two possible states, denoted by Si = \u00b11 and Ui = \u00b11 \nfor the i-th excitatory and inhibitory neurons, respectively. The value -1 denotes \na quiet state. The value + 1 denotes an active state that corresponds to a state \nwith high firing rate. The neurons are assumed to be exposed to local noise result(cid:173)\ning in stochastic dynamics of their states. This dynamics is specified by transition \nprobabilities between the -1 and + 1 states that are sigmoidal functions of their \nlocal fields. The local fields of the i-th excitatory neuron, Ei and the i-th inhibitory \nneuron, Ii, at time t, are \n\nEi(t) = J s(t) - K u(t) -\n\n() \n\nJ's(t) - K' u(t) -\n\n() \n\n(1) \n\n(2) \n\n\fCorrelation Functions in a Large Stochastic Neural Network \n\n473 \n\nwhere () represents the local threshold and sand 0' are the population-averaged \nactivities s(t) = l/N\"'\u00a3j Sj(t), and O'(t) = l/M\"'\u00a3j O'j(t) of the excitatory and \ninhibitory neurons, respectively. \n\n3 AVERAGE FIRING RATES \n\nThe macroscopic state of the network is characterized by the dynamics of s(t) \nand O'(t). To leading order in l/N and l/M, they obey the following well known \nequations \n\nds \n\nTO dt = -s + tanh(Js - J{O' - 0) \n\nTO- = -0' + tanh J s - K 0' - 0 \n) \n\n(I \n\n-,I \n\ndO' \ndt \n\n(3) \n\n(4) \n\nwhere TO is the microscopic time constant of the system. Equations of this form for \nthe two population dynamics have been studied extensively by Wilson and Cowan \n(Wilson and Cowan 1972) and others (Schuster and Wagner 1990, Grannan, Kle(cid:173)\ninfeld and Sompolinsky 1992) \n\nDepending on the various parameters the stable solutions of these equations are \neither fixed-points or limit cycles. The fixed-point solutions represent a stationary \nstate of the network in which the popUlation-averaged activities are almost constant \nin time. The limit-cycle solutions represent nonstationary states in which there \nis a coherent oscillatory activity. Obviously in the latter case there are strong \noscillatory correlations among the neurons. Here we focus on the fixed-point case. \nIt is described by the following equations \n\nSo = tanh ( J So - K 0'0 - 0) \n\n(5) \n\n(6) \nwhere So and 0'0 are the fixed-point values of sand O'. Our aim is to estimate the \nmagnitude of the correlations between the temporal fluctuations in the activities of \nneurons in this statistically stationary state. \n\n0'0 = tanh (J' So - K'O'o - 0) \n\n4 CORRELATION FUNCTIONS \n\nThere are two types of auto-correlation functions, for the two different populations. \nFor the excitatory neurons we define the auto-correlations as: \n\n(7) \nwhere 6si (t) = Sj(t)-so and < ... >t means average over time t. A similar definition \nholds for the auto-correlations of the inhibitory neurons. In our network there are \nthree different cross-correlations: excitatory-excitatory, inhibitory- inhibitory, and \ninhibitory-excitatory. The excitatory-excitatory correlations are \n\nCij(T) = {8si(t)8sj(t + T)}t \n\n(8) \n\nSimilar definitions hold for the other functions. \n\n\f474 \n\nGinzburg and Sompolinsky \n\nWe have evaluated these correlation functions by solving the equations for the cor(cid:173)\nrelations of 6Si(t) in the limit of large Nand M. We find the following forms for \nthe correlations: \n\nGii(T) ~ (1- s~)exp(-A1T) + N La,exp(-AI T) \n\n1 3 \n\n'=1 \n\n1 3 \n\nGij(T)~ NLb,exP(-A,T) . \n\n1=1 \n\n(9) \n\n(10) \n\nThe coefficients a, and b, are in general of order 1. The three A, represent three \ninverse time-constants in our system, where Re(AI) ~ Re(A2) ~ Re(A3)' The first \ninverse time constant equals simply to Al = liTo, and corresponds to a purely \nlocal mode of fluctuations. The values of A2 and A3 depend on the parameters of \nthe system. They represent two collective modes of fluctuations that are coherent \nacross the populations. An important outcome of our analysis is that A2 and A3 \nare exactly the eigenvalues of the stability matrix obtained by linearizing Eqs. (3) \nand (4) about the fixed-point Eqs. (5) and (6) \n\n. \n\nThe above equations imply two differences between the auto-correlations and the \ncross-correlations. First, Gii are of order 1 whereas in general Gij is of 0(1/ N). \nSecondly, the time-dependence of Gii is dominated by the local, fast time constant \nTO, whereas Gij may be dominated by the slower, collective time-constants. \nThe conclusion that the cross-correlations are small relative to the auto-correlations \nmight break down if the coefficients b, take anomalously large values. To check these \npossibility we have studied in detail the behavior of the correlations near bifurcation \npoints, at which the fixed point solutions become unstable. For concreteness we will \ndiscuss here the case of Hopfbifurcations. (Similar results hold for other bifurcations \nas well). Near a Hopf bifurcation A2 and A3 can be written as A\u00b1 ~ \u20ac \u00b1 iw, \nwhere \u20ac > 0 and vanishes at the bifurcation point. In this parameter regime, the \namplitudes b1 \u00ab b2, b3 and b2 ~ b3 ~ ~. Similar results hold for a2 and a3. Thus, \nnear the bifurcation, we have \n\nGii ( T) ~ (1 - s~) exp( -T /ro)cos(wr) \n\nGij(r) ~ N\u20ac exp(-\u20acr)cos(wr) \n\nB \n\n. \n\n(11) \n\n(12) \n\nNote that near a bifurcation point \u20ac \nis linear in the difference between any of the \nparameters and their value at the bifurcation. The above expressions hold for \n1 but large compared to l/N.When \u20ac ~ liN the cross-correlation becomes of \n\u20ac\u00ab \norder 1, and remains so throughout the bifurcation. \nFigures 1 and 2 summarize the results of Eqs. (9) \nnear the Hopf \nbifurcation point at J,J',K,K',O = 225,65, 161,422,2.4. The population sizes \nare N = 10000, M = 1000. We have chosen a parameter range so that the fixed \npoint values of So and lTo will represent a state with low firing rate resembling \nthe spontaneous activity levels in the cortex. For the above parameters the rates \nrelative to the saturation rates are 0.01 and 0.03 for the excitatory and inhibitory \npopulations respectively. \n\nand (10) \n\n\fCorrelation Functions in a Large Stochastic Neural Network \n\n475 \n\n0.45 \n\n04 \n\n035 \n\n0.3 \n\n025 \n\n02 \n\n015 \n\n01 \n\n005 \n\nO~~==C==C==~~--~~--~~ \n180 \n225 \n\n205 \n\n190 \n\n215 \n\n200 \n\n220 \n\n210 \n\n185 \n\n195 \n\nJ \n\nFIG URE 1. The equal-time cross-correlations between a pair of excitatory neu(cid:173)\nrons, and the real part of its inverse time-constant,f, vs. the excitatory coupling \nparameter J. \n\nThe values of Cij (0) and of the real-part of the inverse-time constants of Cij are \nplotted (Fig. 1) as a function of the parameter J holding the rest of the parameters \nfixed at their values at the bifurcation point. Thus in this case f a(225 - J). The \nFigure shows the growth of Cij and the vanishing of the inverse time constant as \nthe bifurcation point is approached. \n0.15 ..-----r--..-----,---.,....---,---r----,----,--.,.----, \n\n0.1 \n\n0.05 \n\no \n\n-0.05 \n\n-0 .1 \n\n-0 .15 L-_....l.-_--L_---l. __ .L..-_-'--_-'-_~_-.-I __ ~_-' \n50 \n\n35 \n\n30 \n\n20 \n\n10 \n\n40 \n\n45 \n\n25 \n\n15 \n\n5 \n\no \n\nThe time-dependence of the cross-correlations near the bifurcation (J = 215) is \nshown in Fig. 2. Time is plotted in units of TO. The pronounced damped oscillations \nare, according to our theory, characteristic of the behavior of the correlations near \nbut below a Hopf bifurcation. \n\ndelay \n\n\f476 \n\nGinzburg and Sompolinsky \n\n5 CONCLUSION \n\nMost theoretical investigations of large recurrent networks focus on the properties of \nthe macroscopic order parameters such as population averaged activity or average \noverlap with memories. However, the statistics of the fluctuations in the activities \nmay be an important testing ground for comparison between models and observed \ncortical dynamics. We have studied the properties of the correlation functions in a \nstochastic network comprising of excitatory and inhibitory populations. We have \nshown that the cross-correlations are relatively weak in stationary states, except in \nthe neighborhoods of bifurcations to nonstationary states. The growth of the am(cid:173)\nplitude of these correlations is coupled to a growth in the correlation time-constant. \nThis divergence of the correlation time is analogous to the phenomenon of critical \nslowing down in systems at thermal equilibrium near a critical point. Our analysis \ncan be extended to stochastic networks consisting of a small number of interacting \nhomogeneous populations. \n\nDetailed comparison between the model's results and experimental values of auto(cid:173)\nand cross- correlograms of extracellularly measured spike trains in the neocortex \nhave been carried out (Abeles, Ginzburg and Sompolinsky). The tentative con(cid:173)\nclusion of this study is that the magnitude of the observed correlations and their \ntime-dependence are inconsistent with the expected ones for a system in a sta(cid:173)\ntionary state. They therefore indicate that cortical neuronal assemblies are in a \nnonstationary (but aperiodic) dynamic state. \n\nAcknowledgements: We thank M. Abeles for most helpful discussions. This work \nis partially supported by the USA-Israel Binational Science Foundation. \n\nREFERENCES \n\nAbeles M., 1991. Corticonics: Neural Circuits of the Cerebral Cortex. Cambridge \nUniversity Press. \nAbeles M., Ginzburg I. & Sompolinsky H. Neuronal Cross-Correlations and Orga(cid:173)\nnized Dynamics in the Neocortex. to appear \nFetz E., Toyama K. & Smith W., 1991. Synaptic Interactions Between Cortical \nNeurons. Cerebral Cortex, edited by A. Peters & G. Jones Plenum Press,NY. Vol \n9. 1-43. \nGrannan E., Kleinfeld D. & Sompolinsky H., 1992. Stimulus Dependent Synchro(cid:173)\nnization of Neuronal Assemblies. Neural Computation 4,550-559. \nSchuster H. G. & Wagner P., 1990. BioI. Cybern. 64, 77. \nWilson H. R. & Cowan J. D., 1972. Excitatory and Inhibitory Interactions m \nLocalized Populations of Model Neurons. Biophy. J. 12, 1-23. \n\n\f", "award": [], "sourceid": 854, "authors": [{"given_name": "Iris", "family_name": "Ginzburg", "institution": null}, {"given_name": "Haim", "family_name": "Sompolinsky", "institution": null}]}