{"title": "Spike-Based Compared to Rate-Based Hebbian Learning", "book": "Advances in Neural Information Processing Systems", "page_first": 125, "page_last": 131, "abstract": null, "full_text": "Spike-Based Compared to Rate-Based \n\nHebbian Learning \n\nRichard Kempter* \n\nInstitut fur Theoretische Physik \nTechnische Universitat Munchen \n\nD-85747 Garching, Germany \n\nWulfram Gerstner \n\nSwiss Federal Institute of Technology \n\nCenter of Neuromimetic Systems, EPFL-DI \n\nCH-1015 Lausanne, Switzerland \n\nJ. Leo van Hemmen \n\nInstitut fur Theoretische Physik \nTechnische Universitat Munchen \n\nD-85747 Garching, Germany \n\nAbstract \n\nA correlation-based learning rule at the spike level is formulated, \nmathematically analyzed, and compared to learning in a firing-rate \ndescription. A differential equation for the learning dynamics is \nderived under the assumption that the time scales of learning and \nspiking can be separated. For a linear Poissonian neuron model \nwhich receives time-dependent stochastic input we show that spike \ncorrelations on a millisecond time scale play indeed a role. Corre(cid:173)\nlations between input and output spikes tend to stabilize structure \nformation, provided that the form of the learning window is in \naccordance with Hebb's principle. Conditions for an intrinsic nor(cid:173)\nmalization of the average synaptic weight are discussed. \n\n1 \n\nIntroduction \n\nMost learning rules are formulated in terms of mean firing rates, viz., a continuous \nvariable reflecting the mean activity of a neuron. For example, a 'Hebbian' (Hebb \n1949) learning rule which is driven by the correlations between presynaptic and \npostsynaptic rates may be used to generate neuronal receptive fields (e.g., Linsker \n1986, MacKay and Miller 1990, Wimbauer et al. 1997) with properties similar to \nthose of real neurons. A rate-based description, however, neglects effects which are \ndue to the pulse structure of neuronal signals. During recent years experimental and \n\n* email: kempter@physik.tu-muenchen.de (corresponding author) \n\n\f126 \n\nR. Kempter. W Gerstner and J L. van Hemmen \n\ntheoretical evidence has accumulated which suggests that temporal coincidences \nbetween spikes on a millisecond or even sub-millisecond scale play an important \nrole in neuronal information processing (e.g., Bialek et al. 1991, Carr 1993, Abeles \n1994, Gerstner et al. 1996). Moreover, changes of synaptic efficacy depend on \nthe precise timing of postsynaptic action potentials and presynaptic input spikes \n(Markram et al. 1997, Zhang et al. 1998). A synaptic weight is found to increase, if \npresynaptic firing precedes a postsynaptic spike and decreased otherwise. In contrast \nto the standard rate models of Hebbian learning, the spike-based learning rule \ndiscussed in this paper takes these effects into account. For mathematical details \nand numerical simulations the reader is referred to Kempter et al. (1999) . \n\n2 Derivation of the Learning Equation \n\n2.1 Specification of the Hebb Rule \nWe consider a neuron that receives input from N \u00bb 1 synapses with efficacies Ji , \n1 :::; i :::; N. We assume that changes of Ji are induced by pre- and postsynaptic \nspikes. The learning rule consists of three parts. (i) Let tf be the time of the m th \ninput spike arriving at synapse i. The arrival of the spike induces the weight Ji to \nchange by an amount win which can be positive or negative. (ii) Let tn be the nth \noutput spike of the neuron under consideration. This event triggers the change of all \nN efficacies by an amount wout which can also be positive or negative. (iii) Finally, \ntime differences between input spikes influence the change of the efficacies. Given \na time difference s = tf - t n between input and output spikes, Ji is changed by an \namount W(s) where the learning window W is a real valued function (Fig. 1). The \nlearning window can be motivated by local chemical processes at the level of the \nsynapse (Gerstner et al. 1998, Senn et al. 1999). Here we simply assume that such \na learning window exist and take some (arbitrary) functional dependence W(s) . \n\nFigure 1: An example of a learning win(cid:173)\ndow W as a function of the delay s = \ntf - tn between a postsynaptic firing time \ntn and presynaptic spike arrival tf at \nsynapse i. Note that for s < 0 the presy(cid:173)\nnaptic spike precedes postsynaptic firing. \n\nStarting at time t with an efficacy Ji(t), the total change 6.Ji(t) = Ji(t + T) - Ji(t) \nin a time interval T is calculated by summing the contributions of all input and \noutput spikes in the time interval [t, t + 7]. Describing the input spike train at \nsynapse i by a series of 8 functions, s:n(t) = Lm 8(t - tf), and, similarly, output \nspikes by sout(t) = Ln 8(t - tn), we can formulate the rules (i)--:(iii): \n\nb.J,(t) = ! dt' Wi\" S;\"(t') + wont 8\"nt(t') + ! dt\" W(t\" - t') S;\"(t\") 8\"nt(t') \n\nt+T \n\nt+T \n\n[ \n\n] \n\n(1) \n\n2.2 Separation of Time Scales \n\nThe total change 6..Ji (t) is subject to noise due to stochastic spike arrival and, \npossibly, stochastic generation of output spikes. We therefore study the expected \ndevelopment of the weights Ji , denoted by angular brackets. We make the substi(cid:173)\ntution s = til -\n\nt' on the right-hand side of (1), divide both sides by T, and take \n\n\fSpike-Based Compared to Rate-Based Hebbian Learning \n\nthe expectation value: \n\n(tlJt\u00b7)(t) \n\nT \n\n_1 I t+Tdt' \nT t \n1 It+T \n+-\nT \n\nt \n\ndt' \n\nt-t' \n\n[win (s!n)(t') + W out (sout) (t')] \nIt+T-t' \n\nds W(s) (s!n(t' + s) sout(t')) \n\n127 \n\n(2) \n\nWe may interpret (s~n)(t) for 1 ::; i ::; Nand (sout)(t) as instantaneous firing \nrates. I They may vary on very short time scales - shorter, e.g., than average \ninterspike intervals. Such a model is consistent with the idea of temporal coding, \nsince it does not rely on temporally averaged mean firing rates. \nWe note, however, that due to the integral over time on the right-hand side of (2) \ntemporal averaging is indeed important. If T is much larger than typical interspike \nintervals, we may define mean firing rates v!n(t) = (s~n)(t) and vout(t) = (sout)(t) \nwhere we have used the notation f(t) = T- l Itt+T dt' f(t'). The mean firing rates \nmust be distinguished from the previously defined instantaneous rates (s~n) and \n(sout) which are defined as an expectation value and have a high temporal resolu(cid:173)\ntion. In contrast, the mean firing rates vin and vout vary slowly (time scale of the \norder of T) as a function of time. \nIf the learning time T is much larger than the width of the learning window, the \nintegration over s in (2) can be extended to run from -00 to 00 without introducing \na noticeable error. With the definition of a temporally averaged correlation, \n\n(3) \nthe last term on the right of (2) reduces to I~oo ds W(s) Ci(s; t). Thus, correlations \nbetween pre- and postsynaptic spikes enter spike-based Hebbian learning through \nCi convolved with the learning window W. We remark that the correlation Ci(s; t) \nmay change as a function of s on a fast time scale. Note that, by definition, s < 0 \nimplies that a presynaptic spike precedes the output spike - and this is when we \nexpect (for excitatory synapses) a positive correlation between input and output. \n\nAs usual in the theory of Hebbian learning, we require learning to be a slow process. \nThe correlation Ci can then be evaluated for a constant Ji and the left-hand side \nof (2) can be rewritten as a differential on the slow time scale of learning \n\n:t Ji(t) == ji = win v!n(t) + Wout vout(t) + i: ds W(S) Ci(S; t) \n\n(4) \n\n2.3 Relation to Rate-Based Hebbian Learning \n\nIn neural network theory, the hypothesis of Hebb (Hebb 1949) is usually formulated \nas a learning rule where the change of a synaptic efficacy Ji depends on the corre(cid:173)\nlation between the mean firing rate vln of the i th presynaptic and the mean firing \nrate vout of a postsynaptic neuron, viz. , \n\nji = ao + al v!n + a2 vout + a3 v!n vout + a4 (v~n)2 + a5 (vout )2 \n\n(5) \nwhere ao, aI, a2, a3 , a4, and a5 are proportionality constants. Apart from the decay \nterm ao and the 'Hebbian' term vin vout proportional to the product of input and \n\n, \n\n1 An example of rapidly changing instantaneous rates can be found in the auditory \nsystem . The auditory nerve carries noisy spike trains with a stochastic intensity modulated \nat the frequency of the applied acoustic tone. In the barn owl, a significant modulation of \nthe rates is seen up to a frequency of 8 kHz (e.g., Carr 1993). \n\n\f128 \n\nR. Kempler, W Gerstner and J L. van Hemmen \n\noutput rates, there are also synaptic changes which are driven separately by the pre(cid:173)\nand postsynaptic rates. The parameters ao, ... , as may depend on Ji . Equation (5) \nis a general formulation up to second order in the rates; see, e.g., (Linsker 1986). \n\nTo get (5) from (4) two approximations are necessary. First, if there are no correla(cid:173)\ntions between input and output spikes apart from the correlations contained in the \nrates, we can approximate (SJn(t + s) sout(t)) ~ (s~n)(t + s) (SOUtHt). Second, if \nthese rates change slowly as compared to T, then we have Ci(s; t) ~ v;n(t+s) vout(t). \nSince we have assumed that the learning time T is long compared to the width of \nthe learning window, we may simplify further and set vJn(t + s) ~ v!n(t), hence \nJ~oo ds W(s) Ci(s; t) ~ W(O) vjn(t) vout(t), where W(O) = J~oo ds W(s). We may \nnow identify W(O) with a3. By further comparison of (5) with (4) we identify \nwin with al and wout with a2, and we are able to reduce (4) to (5) by setting \nao = a4 = as = O. \nThe above set of of assumption which is necessary to derive (5) from (4) does, \nhowever, not hold in general. According to the results of Markram et aI. (1997) the \nwidth of the learning window in cortical pyramidal cells is in the range of ~ 100 ms. \nA mean rate formulation thus requires that all changes of the activity are slow on \na time scale of lOOms. This is not necessarily the case. The existence of oscillatory \nactivity in the cortex in the range of 50 Hz implies activity changes every 20 ms. \nMuch faster activity changes on a time scale of 1 ms and below are found in the \nauditory system (e.g., Carr 1993). Furthermore, beyond the correlations between \nmean activities additional correlations between spikes may exist; see below. Because \nof all these reasons, the learning rule (5) in the simple rate formulation is insufficient . \nIn the following we will study the full spike-based learning equation (4). \n\n3 Stochastically Spiking Neurons \n\n3.1 Poisson Input and Stochastic Neuron Model \n\nTo proceed with the analysis of (4) we need to determine the correlations Ci between \ninput spikes at synapse i and output spikes. The correlations depend strongly on \nthe neuron model under consideration. To highlight the main points of learning \nwe study a linear inhomogeneous Poisson neuron as a toy model. \nInput spike \ntrains arriving at the N synapses are statistically independent and generated by an \ninhomogeneous Poisson process with time-dependent intensities (Sin) (t) = ,\\~n (t), \nwith 1 ~ i ~ N. A spike arriving at tf at synapse i , evokes a postsynaptic potential \n(PSP) with time course E(t - tf) which we assume to be excitatory (EPSP). The \namplitude is given by the synaptic efficacy Ji(t) > O. The membrane potential u of \nthe neuron is the linear superposition of all contributions \n\nu(t) = Uo + L L Ji(t) E(t - t~) \n\nN \n\ni=l m \n\n(6) \n\nwhere Uo is the resting potential. Output spikes are assumed to be generated \nstochastically with a time dependent rate ,\\out(t) which depends linearly upon the \nmembrane potential \n\n,\\out(t) = f3 [u(t)l+ = Vo + L L Ji(t) E(t - tf)\u00b7 \n\nN \n\n(7) \n\ni=l m \n\nwith a linear function f3[ul+ = f30 + f31 u for u > 0 and zero otherwise. After the \nsecond equality sign, we have formally set Vo = Uo + f30 and f31 = 1. vo > can \n\n\fSpike-Based Compared to Rate-Based Hebbian Learning \n\n129 \n\nbe interpreted as the spontaneous firing rate. For excitatory synapses a negative \nu is impossible and that's what we have used after the second equality sign. The \nsums run over all spike arrival times at all synapses. Note that the spike generation \nprocess is independent of previous output spikes. In particular, the Poisson model \ndoes not include refractoriness. \n\nIn the context of (4), we are interested in the expectation values for input and \noutput. The expected input is (s~n)(t) = A~n(t). The expected output is \n\n(sout)(t) = va + L Ji(t) 10 00 d8\u20ac(s) A~n(t - 8) , \n\n(8) \n\nt \n\nThe expected output rate in (8) depends on the convolution of \u20ac with the input rates. \nIn the following we will denote the convolved rates by A~n(t) = 1000 d8 \u20ac(8)A~n(t - 8). \nNext we consider the expected correlations between input and output, (s!n(t + \n8) sout(t)), which we need in (3): \n\n(s~n (t + 8) sout(t)) = A~n (t + 8) [Va + Ji (t) \u20ac( -8) + L Jj (t) A~n(t)] \n\n(9) \n\nj \n\nThe first term inside the square brackets is the spontaneous output rate. The second \nterm is the specific contribution of an input spike at time t + 8 to the output rate \nat t. It vanishes for 8 > 0 (Fig. 2). The sum in (9) contains the mean contributions \nof all synapses to an output spike at time t. Inserting (9) in (3) and assuming the \nweights Jj to be constant in the time interval [t, t + T] we obtain \n\nCi(8; t) = L Jj(t) A~n(t + 8) A~n(t) + A~n(t + 8) [Va + Ji(t) \u20ac( -8)]. \n\n(10) \n\nj \n\nFor excitatory synapses, the second term gives for 8 < 0 a positive contribution \n(Recall that 8 < 0 means that a \nto the correlation function - as it should be. \npresynaptic spike precedes postsynaptic firing.) \n\n[ ... ](t') \n\n---\u00b7r ---------\n\no -'-----r----,,---'---------___. t' \n\nt+s \n\nt \n\nFigure 2: Interpretation of the term in square \nbrackets in (9). The dotted line is the contri(cid:173)\nbution of an input spike at time t + 8 to the \noutput rate as a function of t', viz., Ji (t) \u20ac (t' -\nt - 8). Adding this to the mean rate contribu(cid:173)\ntion, Va + Lj Jj(t') A~n(t') (dashed line), we \nobtain the rate inside the square brackets of \n(9) (full line). At time t' = t the contribution \nof an input spike at time t + 8 is Ji(t) \u20ac( -s). \n\n3.2 Learning Equation \n\nThe assumption of identical and constant mean input rates, A~n(t) = v:n(t) = vin \nfor all i, reduces the number of free parameters in (4) and eliminates all effects of \nrate coding. We introduce r~n(t) := [W(O)]-l J.~oo d8 W(8)A~n(t + 8) and define \n\n(11) \nUsing (8), (10), (11) in (4) we find for the evolution on the slow time scale oflearning \n\nji(t) = kl + L Jj(t) [Qij(t) + k2 + k3 bij] , where \n\n(12) \n\nj \n\n\f130 \n\nR. Kempter. W Gerstner and 1. L. van Hemmen \n\n[w out + W(O) vin] Vo + win v in \n[w out + W(O) vin] v in \n\nv in / ds\u20ac(-s) W(s) . \n\n(13) \n(14) \n\n(15) \n\n4 Discussion \n\nEquation (12), which is the central result of our analysis, describes the expected \ndynamics of synaptic weights for a spike-based Hebbian learning rule (1) under the \nassumption of a linear inhomogeneous Poisson neuron. Linsker (1986) has derived a \nmathematically equivalent equation starting from (5) and a linear graded response \nneuron, a rate-based model. An equation of this type has been analyzed by MacKay \nand Miller (1990). The difference between Linsker's equation and (12) is, apart from \na slightly different notation, the term k3 6ij and the interpretation of Qij. \n\n4.1 \n\nInterpretation of Qij \n\nIn (12) correlations between spikes on time scales down to milliseconds or below \ncan enter the driving term Qij for structure formation; cf. (11). In contrast to that, \nLinsker 's ansatz is based on a firing rate description, where the term Qij contains \ncorrelations between mean firing rates only. In his Qij term, mean firing rates take \nthe place of r~n and A~n. If we use a standard interpretation of rate coding, a mean \nfiring rate corresponds to a temporally averaged quantity with an averaging window \nor a hundred milliseconds or more. \nFormally, we could define mean rates by temporal averaging with either \u20ac( s) or \nW(s) as the averaging window. In this sense, Linsker's 'rates' have been made \nmore precise by (11). Note, however, that (11) is asymmetric: one of the rates \nshould be convolved with \u20ac\n\n, the other one with W. \n\n4.2 Relevance of the k3 term \n\nThe most important difference between Linsker's rate-based learning rule and our \nEq. (12) is the existence of a term k3 I: O. We now argue that for a causal chain of \nevents k3 ex: I dx \u20ac(x) W( -x) must be positive. [We have set x = -s in (15).] First, \nwithout loss of generality, the integral can be restricted to x > 0 since \u20ac(x) \nis a \nresponse kernel and vanishes for x < O. For excitatory synapses, \u20ac(x) \nis positive for \nx > O. Second, experiments on excitatory synapses show that W(s) is positive for \ns < 0 (Markram et al. 1997, Zhang et al. 1998). Thus the integral I dx \u20ac(x) W( -x) \nis positive - and so is k 3 . \nThere is also a more general argument for k3 > 0 based on a literal interpretation of \nHebb's statement (Hebb 1949). Let us recall that s < 0 in (15) means that a presy(cid:173)\nnaptic spike precedes postsynaptic spiking. For excitatory synapses, a presynaptic \nspike which precedes postsynaptic firing may be the cause of the postsynaptic ac(cid:173)\ntivity. [As Hebb puts it, it has 'contributed in firing the postsynaptic cell'.] Thus, \nthe Hebb rul~ 'predicts' that for excitatory synapses W(s) is positive for s < O. \nHence, k3 = vln I ds \u20ac( - s) W (s) > 0 as claimed above. \nA positive k3 term in (12) gives rise to an exponential growth of weights. Thus any \nexisting structure in the distribution of weights is enhanced. This contributes to the \nstability of weight distributions, especially when there are few and strong synapses \n(Gerstner et al. 1996). \n\n\fSpike-Based Compared to Rate-Based Hebbian Learning \n\n131 \n\n4.3 \n\nIntrinsic Normalization \n\nLet us suppose that no input synapse is special and impose the (weak) condition \nthat N - 1 Li Qij = Qo > 0 independent of the synapse index j . We find then from \n(12) that the average weight Jo := N-l Li Ji has a fixed point Jo = -kd[Qo + \nk2 + N- 1 k 3 ]. The fixed point is stable if Qo + k2 + N- 1 k3 < O. We have shown \nabove that k3 > O. Furthermore, Qo > 0 according to our assumption. The only \nway to enforce stability is therefore a term k2 which is sufficiently negative. Let \nus now turn to the definition of k2 in (14). To achieve k2 < 0, either W(O) (the \nintegral over W) must be sufficiently negative; this corresponds to a learning rule \nwhich is, on the average, anti-Hebbian. Or, for W(O) > 0, the linear term wout in \n(1) must be sufficiently negative. In addition, for excitatory synapses a reasonable \nfixed point Jo has to be positive. For a stable fixed point this is only possible for \nkl > 0, which, in turn, implies win to be sufficiently positive; cf. (13). \nIntrinsic normalization of synaptic weights is an interesting property, since it allows \nneurons to stay at an optimal operating point even while synapses are changing. \nAuditory neurons may use such a mechanism to stay during learning in the regime \nwhere coincidence detection is possible (Gerstner et al. 1996, Kempter et al. 1998). \nCortical neurons might use the same principles to operate in the regime of high \nvariability (Abbott, invited NIPS talk, this volume). \n\n4.4 Conclusions \n\nSpike-based learning is different from simple rate-based learning rules. A spike(cid:173)\nbased learning rule can pick up correlations in the input on a millisecond time \nscale. Mathematically, the main difference to rate-based Hebbian learning is the \nexistence of a k3 term which accounts for the causal relation between input and \noutput spikes. Correlations between input and output spikes on a millisecond time \nscale playa role and tend to stabilize existing strong synapses. \n\nReferences \nAbeles M. , 1994, In Domany E. et al., editors, Models of Neural Networks II, pp. \n\n121- 140, New York. Springer. \n\nBialek W . et al. , 1991, Science, 252:1855- 1857. \nCarr C. E., 1993, Annu. Rev. Neurosci. , 16:223-243. \nGerstner W. et al., 1996, Nature, 383:76-78. \nGerstner W. et al., 1998, In W. Maass and C. M. Bishop., editors, Pulsed Neural \n\nNetworks, pp. 353-377, Cambridge. MIT-Press. \n\nHebb D.O. , 1949, The Organization of Behavior. Wiley, New York. \nKempter R. et al., 1998, Neural Comput. , 10:1987- 2017. \nKempter R. et al., 1999, Phys. Rev. E, In Press. \nLinsker R., 1986, Proc. Natl. Acad. Sci. USA, 83:7508- 7512. \nMacKay D. J. C., Miller K. D. , 1990, Network, 1:257- 297. \nMarkram H. et al., 1997, Science, 275:213-215. \nSenn W. et al., 1999, preprint, Univ. Bern. \nWimbauer S. et al., 1997, BioI. Cybern., 77:453-461. \nZhang L.I. et al. , 1998, Nature, 395 :37- 44 \n\n\f", "award": [], "sourceid": 1635, "authors": [{"given_name": "Richard", "family_name": "Kempter", "institution": null}, {"given_name": "Wulfram", "family_name": "Gerstner", "institution": null}, {"given_name": "J.", "family_name": "van Hemmen", "institution": null}]}