{"title": "Non-Boltzmann Dynamics in Networks of Spiking Neurons", "book": "Advances in Neural Information Processing Systems", "page_first": 109, "page_last": 116, "abstract": null, "full_text": "Non-Boltzmann Dynamics in Networks of Spiking Neurons \n\n109 \n\nNon-Boltzmann Dynamics in Networks of \n\nSpiking Neurons \n\nMichael C. Crair and William Bialek \n\nDepartment of Physics, and \n\nDepartment of Molecular and Cell Biology \n\nUniversity of California at Berkeley \n\nBerkeley, CA 94720 \n\nABSTRACT \n\nWe study networks of spiking neurons in which spikes are fired as \na Poisson process. The state of a cell is determined by the instan(cid:173)\ntaneous firing rate, and in the limit of high firing rates our model \nreduces to that studied by Hopfield. We find that the inclusion \nof spiking results in several new features, such as a noise-induced \nasymmetry between \"on\" and \"off\" states of the cells and probabil(cid:173)\nity currents which destroy the usual description of network dynam(cid:173)\nics in terms of energy surfaces. Taking account of spikes also al(cid:173)\nlows us to calibrate network parameters such as \"synaptic weights\" \nagainst experiments on real synapses. Realistic forms of the post \nsynaptic response alters the network dynamics, which suggests a \nnovel dynamical learning mechanism. \n\n1 \n\nINTRODUCTION \n\nIn 1943 McCulloch and Pitts introduced the concept of two-state (binary) neurons \nas elementary building blocks for neural computation. They showed that essentially \nany finite calculation can be done using these simple devices. Two-state neurons are \nof questionable biological relevance, yet much of the subsequent work on modeling of \nneural networks has been based on McCulloch-Pitts type neurons because the two(cid:173)\nstate simplification makes analytic theories more tractable. Hopfield (1982, 1984) \n\n\f110 \n\nCrair and Bialek \n\nshowed that an asynchronous model of symmetrically connected two-state neurons \nwas equivalent to Monte-Carlo dynamics on an 'energy' surface at zero temperature. \nThe idea that the computational abilities of a neural network can be understood \nfrom the structure of an effective energy surface has been the central theme in much \nrecent work. \nIn an effort to understand the effects of noise, Amit, Gutfreund and Sompolinsky \n(Amit et aI., 1985a; 1985b) assumed that Hopfield's 'energy' could be elevated to \nan energy in the statistical mechanics sense, and solved the Hopfield model at finite \ntemperature. The problem is that the noise introduced in equilibrium statistical \nmechanics is of a very special form, and it is not clear that the stochastic properties \nof real neurons are captured by postulating a Boltzmann distribution on the energy \nsurface. \nHere we try to do a slightly more realistic calculation, describing interactions among \nneurons through action potentials which are fired according to probabilistic rules. \nWe view such calculations as intermediate between the purely phenomenological \ntreatment of neural noise by Amit et aI. and a fully microscopic description of \nneural dynamics in terms of ion channels and their associated noise. We find that \neven our limited attempt at biological realism results in some interesting deviations \nfrom previous ideas on network dynamics. \n\n2 THE MODEL \nWe consider a model where neurons have a continuous firing rate, but the generation \nof action potentials is a Poisson process. This mean~ that the \"state\" of each cell i \nis described by the instantaneous rate Ti(t), and the probability that this cell will \nfire in a time interval [t, t + dt] is given by Ti(t)dt. Evidence for the near-Poisson \ncharacter of neuronal firing can be found in the mammalian auditory nerve (Siebert, \n1965; 1968), and retinal ganglion cells (Teich et al., 1978, Teich and Saleh, 1981). \nTo stay as close as possible to existing models, we assume that the rate T( t) of a \nneuron is a sigmoid function, g(x) = 1/(1 +e- Z ), of the total input x to the neuron. \nThe input is assumed to be a weighted sum of the spikes received from all other \nneurons, so that \n\nr,(t) = rmY [~~ J,;!(t - til - e,] . \n\n(1) \n\nJii is the matrix of connection strengths between neurons, Tm is the maximum \nspike rate of the neuron, and 0i is the neuronal threshold. J(t) is a time weighting \nfunction, corresponding schematically to the time course of post-synaptic currents \ninjected by a pre-synaptic spike; a good first order approximation for this function \nis J(t) -- e- t / r , but we also consider functions with more than one time constant. \n(Aidley, 1980, Fetz and Gustafsson, 1983). \n\nWe can think of the spike train from the itA neuron, Ep .5(t - tn, as an approx(cid:173)\n\nimation to the true firing rate Ti(t); of course this approximation improves as the \n\n\fNon-Boltzmann Dynamics in Networks of Spiking Neurons \n\n111 \n\nspikes come closer together at high firing rates. If we write \n\nL <5(t - tn = ri(t) + 7]i(t) \n\nIJ \n\n(2) \n\nwe have defined the noise TJi in the spike train. The equations of motion for the \nrates then become \n\n(3) \n\nwhere Ni(t) = L:j Jij7]j(t) and f 0 rj(t) is the convolution of f(t) with the spike \nrate rj(t). The statistics of the fluctuations in the spike rate 7]j(t) are (7]j(t\u00bb = \n0, \n\n(7]i(t)7]j(t'\u00bb = <5ij(t - t')rj(t). \n\n3 DYNAMICS \nto obtain a first order equation for the normalized spike rate Yi(t) = ri{t)/rm. \nIf the post-synaptic response f(t) is exactly exponential, we can invert Eq. (3) \n\nMore precise descriptions of the post-synaptic response will yield higher order time \nderivatives with coefficients that depend on the relative time constants in f(t). vVe \nwill comment later on the relevance of these higher order terms, but consider first \nthe lowest order description. By inverting Eq. (3) we obtain a stochastic differential \nequation analogous to the Langevin equation describing Brownian motion: \n\ndg-1(Yd __ dE N.() \ndYi + \u2022 t , \n\ndt \n\n-\n\nwhere the deterministic forces are given by \n\n(4a) \n\n(4b) \n\nNote that Eq. (4) is nearly equivalent to the \"charging equation\" Hopfield (1984) \nassumed in his discussion of continuous neurons, except we have explicitly included \nthe noise from the spikes. This system is precisely equivalent to the Hopfield two(cid:173)\nstate model in the limit of large spike rate (rm T =:} 00, Jii = constant), and no \nIn a thermodynamic system near equilibrium, the noise \"force\" Ni (t) is \nnoise. \nrelated to the friction coefficient via the fluctuation dissipation theorem. In this \nsystem however, there is no analogous relationship. \n\nA standard transformation, analogous to deriving Einstein's diffusion equation from \nthe Langevin equation (Stratonovich, 1963, 1967), yields a probabilistic description \nfor the evolution of the neural system, a form of Fokker-Planck equation for the time \nevolution of P( {y;}), the probability that the network is in a state described by the \nnormalized rates {y;}; we write the Fokker-Planck equation below for a simple case. \n\n\f112 \n\nCrair and Bialek \n\nA useful interpretation to consider is that the system, starting in a non-equilibrium \nstate, diffuses or evolves in phase space, to a final stationary state. \nWe can make our description of the post-synaptic response f(t) more accurate \nby including two (or more) exponential time constants, corresponding roughly to \nthe rise and fall time of the post synaptic potential. This inclusion necessitates \nthe addition of a second order term in the Langevin equation (Eq. 4). This is \nanalogous to including an inertial term in a diffusive description, so that the system \nis no longer purely dissipative. This additional complication has some interesting \nconsequences. Adjusting the relative length of the rise to fall time of the post \nsynaptic potential effects the rate of relaxation to local equilibrium of the system. \nIn order to perform most efficaciously as an associative memory, a neural system \nwill \"choose\" critical damping time constants, so that relaxation is fastest. Thus, \nby adjusting the time course of the post synaptic potential, the system can \"learn\" \nof a local stationary state, without adjusting the synaptic strengths. This novel \nlearning mechanism could be a form of fine tuning of already established memories, \nor could be a unique form of dynamical short-term memory. \n\n4 QUALITATIVE RESULTS \nIn order to understand the dynamics of our Fokker-Planck equation, we begin by \nconsidering the case of two neurons interacting with each other. There are two lim(cid:173)\niting behaviors. If the neurons are weakly coupled (J < Je , Je = 4/rm T), then the \nonly stable state of the system is with both neurons firing at a mean firing rate, ! rm. \nIf the neurons are strongly (and positively) coupled (J > Je ), then isolated basins \nof attraction, or stationary states are formed, one stationary state corr..:sponding \nto both neurons being active, the other state has both neurons relatively (but not \nabsolutely) quiescent. In the strong coupling limit, one can reduce the problem \nto motion along the a collective coordinate connecting the two stable states. The \nresulting one dimensional Fokker-Planck equation is \n\nat P(y, t) = ay U'(y)P(y, t) + ay T(y)P(y, t) \na \n\na [ \n\na \n\n1 \n\n, \n\nwhere U(y) is an effective potential energy, \n\nU Y = y(l - y) \n'( ) \n\n[9- 1(y) \n\nT \n\n1 \n2 \n\n- -rmJ y - -) + -J rmy(3 - 5y)], \n\n( \n\n1 \n2 \n\n1 2 \n4 \n\n(5) \n\n(6) \n\nand T(y) is a spatially varying effective temperature, T(y) = ~J2rmy3(1 _ y)2. \nOne can solve to find the size of the stable regions, and the stationary probability \ndistribution, \n\n[(I U'(y) 1 \n\u2022 \nP (y) - T(y) exp - J, T(y) dy \n\nB \n\n. \n\n-\n\n(7) \n\nWe have done numerical simulations which confirm the qualitative predictions of the \none dimensional Fokker-Planck equation. This analysis shows that the non-uniform \n\n\fNon-Boltzmann Dynamics in Networks of Spiking Neurons \n\n113 \n\nand asymmetric temperature distribution alters the relative stability of the stable \nstates, in the favor of the 'off' state. This effect does have some biological pertinence, \nas it is well known that on average neurons are more likely to be quiescent then \nactive. In our model the asymmetry is a direct consequence of the Poisson nature \nof the neuronal firing. \n\nProbability Current \n\n\u2022 \n\n... -\n\n'\" -\nI -r \n\ni \n\no \n\nII \n\n2 \n\n\u2022 \n\nI \n\n14 \n\n\u2022 \u2022 \n\n! \n\u2022 \n\nrX ... \n\n\u2022 \n\n10 \n\n\u2022 \n\n12 \n\nFigure 1: Probability current in the stationary state for two neurons that are \nstrongly interacting. Computed as a ratio of the number of excess excursions in \none dire<:tion to the total number of excursions, in percent. In thermodynamic \nequillibrium, detailed balance would force the current to be zero. Shown as a \nfunction of the number of spikes in an e-folding time of the post-synaptic response. \n\nThere are further surprises to be found in the simple two neuron model. Since the \ninteraction between the neurons is not time reversal invariant, detailed balance is \nnot maintained in the system. Thus, even the stationary probability distribution \nhas non-zero probability current, so that the system tends to cycle probabilistically \nthrough state space. The presence of the current further alters the relative proba(cid:173)\nbility of the two stable states, as confirmed by numerical simulations, and renders \nthe application of equilibrium statistical mechanics inappropriate. \n\nSimulations also confirm (Fig. 1) that the probability current falls off with increas(cid:173)\ning maximum spike rate (rmT), because the effective noise is suppressed when the \nspike rate is high. However, at biologically reasonable spike rates (rm - 150s- 1), \nthe probability current is significant. These currents destroy any sense of a global \n\n\f114 \n\nCrair and Bialek \n\nenergy function or thermodynamic temperature. \nOne advantage of treating spikes explicitly is that we can relate the abstract synaptic \nstrength J to observable parameters. In Fig. 2 we compare J with the experimen(cid:173)\ntally accessible spike number to spike number transfer across the synapse, for a two \nneuron system. Note that critical coupling (see above) corresponds to a rather large \nvalue of,...- 4/5 th of a spike emitted per spike received. \n\nSpikes Generated per Spike Input \n\n. -----------------------------\n\n.' \n\no \n\n. 0 \n\n, \n\n~ ~ \ni I. \n\ne \ne \n\n0.0 \n\nD.5 \n\n1.0 \n\n1.5 \n\n2.0 \n\n2.5 \n\nFigure 2: Single neuron spike response to the receipt of a spike from a coupled \nneuron. Since response is probabilistic, fractional spikes are relevant. Computed as \na function of J /Jcritical, where Jcritical is the minimum synaptic strength necessary \nfor isolated basins of attraction. \n\nMany of the simple ideas we have introduced for the two neuron system carryover \nto the multi-neuron case. If the matrix of connection strengths obeys the \"Hebb\" \nrule (often used to model associative memory), \n\n(8) \n\nthen a stability analysis yields the same critical value for the connection strength J \n(note that we have scaled by N, and the sum on 11 runs from 1 to p, the number of \nmemories to be stored). Calculation of the spike-out/spike-in ratio for the multi(cid:173)\nneuron system at critical coupling shows that it scales like (a/N)t, where p = aN. \n\n\fNon-Boltzmann Dynamics in Networks of Spiking Neurons \n\n115 \n\nSince most neural systems naturally have a small spike-out/spike-in ratio, this (to(cid:173)\ngether with Fig. 2) suggests that small networks will have to be strongly driven in \norder to achieve isolated basins of attraction for \"memories;\" this is in agreement \nwith the one available experiment (Kleinfeld et aI., 1990). In contrast, large net(cid:173)\nworks achieve criticality with more natural spike to spike ratios. For instance, if a \nnetwork of 104 - 105 connected neurons is to have multiple stable \"memory\" states \nas in the original Hopfield model, we predict that a neuron needs to receive 100-\n500 contiguous action potentials to stimulate the emission of its own spike. This \nprediction agrees with experiments done on the hippocampus (McNaughton et al., \n1981), where about 400 convergent inputs are needed to discharge a granule cell. \n\n5 CONCLUSIONS \nTo conclude, we will just summarize our major points: \n\n\u2022 Spike noise generated by the Poisson firing of neurons breaks the symmetry \n\nbetween on/off states, in favor of the \"off\" state. \n\n\u2022 State dependent spike noise also destroys any sense of a global energy func(cid:173)\n\ntion, let alone a thermodynamic 'temperature'. This makes us suspicious of \nattempts to apply standard techniques of statistical mechanics. \n\n\u2022 By explicitly modeling the interaction of neurons via spikes, we have direct \n\naccess to experiments which can guide, and be guided by our theory. Specif(cid:173)\nically, our theory predicts that for a given connection strength between neu(cid:173)\nrons, larger net Norks of neurons will function as memories at naturally small \nspike-input to spike-output ratios. \n\n\u2022 More realistic forms of post synaptic response to the receipt of action poten(cid:173)\ntials alters the network dynamics. By adjusting the relative rise and fall time \nof the post-synaptic potential, the network speeds the relaxation ,to the local \nstable state. This implies that more efficacious memories, or \"learning\", can \nresult without altering the strength of the synaptic weights. \n\nFinally, we comment on the dynamics of networks in the N -+ 00 limit. \\Ve might \nimagine that some of the complexities we find in the two-neuron case would go away, \nin particular the probability currents. We have been able to prove that this does not \nhappen in any rigorous sense for realistic forms of spike noise, although in practice \nthe currents may become small. The function of the network as a memory (for \nexample) would then depend on a clean separation of time scales between relaxation \ninto a single basin of attraction and noise-driven transitions to neighboring basins. \nArranging for this separation of time scales requires some constraints on synaptic \nconnectivity and firing rates which might be testable in experiments on real circuits. \n\n\f116 \n\nCrair and Bialek \n\nReferences \n\nD. J. Aidley (1980), Physiology of Excitable Cells, 2nd Edition, Cambridge Univer(cid:173)\nsity Press, Cambridge. \nD. J. Amit, H. Gutfreund and H. Sompolinsky (1985a), Phys. Rev. A, 2, 1007-1018. \nD. J. Amit, H. Gutfreund and H. Sompolinsky (1985b), Phys. Rev. Lett., 55, \n1530-1533. \nE. E. Fetz and B. Gustafsson (1983), J. Physiol., 341, 387. \nJ. J. Hopfield (1982), Proc. Nat. Acad. Sci. USA, 79,2554-2558. \n\nJ. J. Hopfield (1984), Proc. Nat. Acad. Sci. USA, 81,3088-3092. \n\nD. Kleinfeld, F. Raccuia-Behling, and H. J. Chiel (1990), Biophysical Journal, in \npress. \n\nW. S. McCulloch and W. Pitts (1943), Bull. of Math. Biophys., 5, 115-133. \n\nB. L. McNaughton, C. A. Barnes and P. Anderson (1981), J. Neurophysiol. 46, \n952-966. \n\nW. M. Siebert (1965), Kybernetik, 2, 206. \n\nW. M. Siebert (1968) in Recognizing Patterns, p104, P.A. Kohlers and ~L Eden, \nEds., MIT Press, Cambridge. \nR. 1. Stratonovich (1963,1967), Topics in the Theory oj Random Noise, Vol. I and \nII, Gordon & Breach, New York. \n\nM. C. Teich, L. Martin and B.1. Cantor (1978), J. Opt. Soc. Am., 68, 386. \nM. C. Teich and B.E.A. Saleh (1981), J. Opt. Soc. Am.,71, 771. \n\n\f", "award": [], "sourceid": 276, "authors": [{"given_name": "Michael", "family_name": "Crair", "institution": null}, {"given_name": "William", "family_name": "Bialek", "institution": null}]}