{"title": "Spike-timing Dependent Plasticity and Mutual Information Maximization for a Spiking Neuron Model", "book": "Advances in Neural Information Processing Systems", "page_first": 1409, "page_last": 1416, "abstract": null, "full_text": "Spike-Timing Dependent Plasticity and Mutual\nInformation Maximization for a Spiking Neuron\n\nModel\n\nTaro Toyoizumiyz,\n\nJean-Pascal P\ufb01sterz\n\nKazuyuki Aiharax \u2044,\n\nWulfram Gerstnerz\n\ny Department of Complexity Science and Engineering,\n\nThe University of Tokyo, 153-8505 Tokyo, Japan\n\nz Ecole Polytechnique F\u00b4ed\u00b4erale de Lausanne (EPFL),\nSchool of Computer and Communication Sciences and\n\nBrain-Mind Institute, 1015 Lausanne, Switzerland\n\nx Graduate School of Information Science and Technology,\n\nThe University of Tokyo, 153-8505 Tokyo, Japan\n\ntaro@sat.t.u-tokyo.ac.jp,\naihara@sat.t.u-tokyo.ac.jp,\n\njean-pascal.pfister@epfl.ch\nwulfram.gerstner@epfl.ch\n\nAbstract\n\nWe derive an optimal learning rule in the sense of mutual information\nmaximization for a spiking neuron model. Under the assumption of\nsmall \ufb02uctuations of the input, we \ufb01nd a spike-timing dependent plas-\nticity (STDP) function which depends on the time course of excitatory\npostsynaptic potentials (EPSPs) and the autocorrelation function of the\npostsynaptic neuron. We show that the STDP function has both positive\nand negative phases. The positive phase is related to the shape of the\nEPSP while the negative phase is controlled by neuronal refractoriness.\n\n1\n\nIntroduction\n\nSpike-timing dependent plasticity (STDP) has been intensively studied during the last\ndecade both experimentally and theoretically (for reviews see [1, 2]). STDP is a variant\nof Hebbian learning that is sensitive not only to the spatial but also to the temporal corre-\nlations between pre- and postsynaptic neurons. While the exact time course of the STDP\nfunction varies between different types of neurons, the functional consequences of these\ndifferences are largely unknown. One line of modeling research takes a given STDP rule\nand analyzes the evolution of synaptic ef\ufb01cacies [3\u20135]. In this article, we take a different\n\n\u2044Alternative address: ERATO Aihara Complexity Modeling Project, JST, 45-18 Oyama, Shibuya-\n\nku, 151-0065 Tokyo , Japan\n\n\fapproach and start from \ufb01rst principles. More precisely, we ask what is the optimal synap-\ntic update rule so as to maximize the mutual information between pre- and postsynaptic\nneurons.\n\nPreviously information theoretical approaches to neural coding have been used to quantify\nthe amount of information that a neuron or a neural network is able to encode or trans-\nmit [6\u20138]. In particular, algorithms based on the maximization of the mutual information\nbetween the output and the input of a network, also called infomax principle [9], have been\nused to detect the principal (or independent) components of the input signal, or to reduce\nthe redundancy [10\u201312]. Although it is a matter of discussion whether neurons simply\n\u2019transmit\u2019 information as opposed to classi\ufb01cation or task-speci\ufb01c processing [13], strate-\ngies based on information maximization provide a reasonable starting point to construct\nneuronal networks in an unsupervised, but principled manner.\n\nRecently, using a rate neuron, Chechik applied information maximization to detect static\ninput patterns from the output signal, and derived the optimal temporal learning window;\nthe learning window has a positive part due to the effect of the postsynaptic potential and\nhas \ufb02at negative parts with a length determined by the memory span [14].\n\nIn this paper, however, we employ a stochastic spiking neuron model to study not only\nthe effect of postsynaptic potentials generated by synaptic input but also the effect of the\nrefractory period of the postsynaptic neuron on the shape of the optimal learning window.\nWe discuss the relation of mutual information and Fisher information for small input vari-\nance in Sec. 2. Optimization of the Fisher information by gradient ascent yields an optimal\nlearning rule as shown in Sec. 3\n\n2 Model assumptions\n\n2.1 Neuron model\n\nThe model we are considering is a stochastic neuron with refractoriness. The instantaneous\n\ufb01ring rate \u2030 at time t depends on the membrane potential u(t) and refractoriness R(t):\n\n\u2030(t) = g(\ufb02u(t))R(t);\n\n(1)\n\nwhere g(\ufb02u) = g0 log2[1+e\ufb02u] is a smoothed piecewise linear function with a scaling vari-\nable \ufb02 and a constant g0 = 85Hz. The refractory variable is R(t) = (t\u00a1^t\u00a1\u00bfabs)2\nrefr+(t\u00a1^t\u00a1\u00bfabs)2 \u00a3(t \u00a1\n\u00bf 2\n^t \u00a1 \u00bfabs) and depends on the time elapsed since the last \ufb01ring time ^t, the absolute refrac-\ntory period \u00bfabs = 3 ms, and the time constant of relative refractoriness \u00bfrefr = 10 ms. The\nHeaviside step function \u00a3 takes a value of 1 for positive arguments and zero otherwise.\nThe postsynaptic potential depends on the input spike trains of N presynaptic neurons. A\npresynaptic spike of neuron i 2 f1; 2; : : : ; N g emitted at time tf\ni evokes a postsynaptic\npotential with time course \u2020(t \u00a1 tf\n\ni ). The total membrane potential is\n\nN\n\n\u2020(t \u00a1 tf\n\ni ) =\n\nwiXf\n\nN\n\nXi=1\n\nwiZ \u2020(s)xi(t \u00a1 s)ds\n\n(2)\n\nu(t) =\n\nXi=1\nwhere xi(t) =Pf \u2013(t \u00a1 tf\n\ni ) denotes the spike train of the presynaptic neuron i. The above\nmodel is a special case of the spike response model with escape noise [2]. For vanishing\nrefractoriness \u00bfrefr ! 0 and \u00bfabs ! 0, the above model reduces to an inhomogeneous\nPoisson process.\n\nFor a given set of presynaptic spikes in an interval [0; T ], hence for a given time course of\n\n\fmembrane potential fu(t)jt 2 [0; T ]g, the model generates an output spike train\n\n\u2013(t \u00a1 tf )\n\ny(t) =Xf\n\nwith \ufb01ring times ftf jf = 1; : : : ; ng with a probability density\n\nP (yju) = exp\"Z T\n\n0\n\n(y(t) log \u2030(t) \u00a1 \u2030(t)) dt# :\n\n(3)\n\n(4)\n\nwhere \u2030(t) is given by Eq. (1), i.e., \u2030(t) = g(\ufb02u(t)) R(t). Since the refractory variable R\ndepends on the \ufb01ring time ^t of the previous output spike, we sometimes write \u2030(tj^t) instead\nof \u2030(t) in order to make this dependence explicit. Equation (4) can then be re-expressed in\nterms of the survivor function S(tj^t) = e\u00a1R t\n^t \u2030(sj^t)ds and the interval distribution Q(tj^t) =\n\u2030(tj^t)S(tj^t) in a more transparent form:\n\nn\n\nP (yju) = 0\nYf =1\n@\n\nQ(tf jtf \u00a11)1\n\nA S(T jtn);\n\nwhere t0 = 0 and n is the number of postsynaptic spikes in [0; T ]. In words, the probability\nthat a speci\ufb01c output spike train y occurs can be calculated from the interspike intervals\nQ(tf jtf \u00a11) and the probability that the neuron \u2018survives\u2019 from the last spike at time tn to\ntime T without further \ufb01ring.\n\n2.2 Fisher information and mutual information\n\nLet us consider input spike trains with stationary statistics. These input spike trains generate\nan input potential u(t) with an average value u0 and standard deviation (cid:190). Assuming a\nweak dependence of g on the membrane potential u, i.e., for small \ufb02, we expand g around\ng0 = g(0) to obtain g(\ufb02u(t)) = g0 + g0\n0 [\ufb02u(t)]2=2 + O(\ufb023) where g0 is the\nvalue of g in the absence of input and the next terms describe the in\ufb02uence of the input.\nHere and in the following, all calculations will be done to order \ufb02 2.\nIn the limit of small \ufb02, the mutual information is given by [15]\n\n0\ufb02u(t) + g00\n\n(5)\n\n(6)\n\n(7)\n\n(8)\n\nI(Y ; X) =\n\ndt0\u00a7(t \u00a1 t0)J0(t \u00a1 t0) + O(\ufb023);\n\n\ufb022\n\n2 Z T\n\n0\n\ndtZ T\n\n0\n\nwith the autocovariance function of the membrane potential\n\nwith \u00a2u(t) = u(t) \u00a1 u0 and Fisher information\n\n\u00a7(t \u00a1 t0) = h\u00a2u(t)\u00a2u(t0)iX ;\n\nJ0(t \u00a1 t0) = \u00a1* @2 log P (yju)\n\n@\ufb02u(t)@\ufb02u(t0)\ufb02\ufb02\ufb02\ufb02\ufb02=0+Y j\ufb02=0\n\n;\n\nwith h\u00a2iY j\ufb02=0 = R \u00a2 P (yj\ufb02 = 0)dy and h\u00a2iX = R \u00a2 P (x)dx. Note that the Fisher\n\ninformation (8) is to be evaluated at the constant g0, i.e., at the value \ufb02u = 0, whereas\nthe autocovariance in Eq. (7) is calculated with respect to the mean membrane potentital\nu0 = hu(t)iX which is in general different from zero. The derivation of (6) is based\non the assumption that the variability of the output signal is small and g(\ufb02u) does not\ndeviate much from g0, i.e., it corresponds to the regime of small signal-to-noise ratio.\nIt is well known that the information capacity of the Gaussian channel is given by the\nlog of the signal-to-noise ratio [16], and the mutual information is proportional to the\n\n\fsignal-to-noise ratio when it is small. The relation between the Fisher information, the\nmutual information, and optimal tuning curves has previously been established in the\nregime of large signal-to-noise ratio [17].\n\nWe introduce the following notation: Let \u201e0 = hy(t)iY j\ufb02=0 = h\u2030(t)iY j\ufb02=0 be the spon-\ntaneous \ufb01ring rate in the absence of input and \u201e\u00a11\n0 hy(t)y(t0)iY j\ufb02=0 = \u2013(t \u00a1 t0) + \u201e0[1 +\n`(t \u00a1 t0)] be the postsynaptic \ufb01ring probability at time t given a postsynaptic spike at t0,\ni.e., the autocorrelation function of Y . From the theory of stationary renewal processes [2]\n\n\u201e0 = \u2022Z s Q0(s)ds\u201a\u00a11\n\n;\n\n\u201e0[1 + `(s)] = Q0(jsj) +Z Q0(s0)\u201e0[1 + `(jsj \u00a1 s0)] \u00a3(jsj \u00a1 s0)ds0;\n\nwhere Q0(s) = g0R(s)e\u00a1g0[(s\u00a1\u00bfabs)\u00a1\u00bfrefr arctan(s\u00a1\u00bfabs)=\u00bfrefr] is the interval distribution for\nconstant g = g0. The interval distribution vanishes during the absolute refractory time \u00bfabs;\ncf. Fig. 1.\n\n(9)\n\n(A)\n\n(B)\n\n0.05\n\n0.04\n\n0.03\n\n0.02\n\n0.01\n\n0\n\n)\ns\n(\n0\nQ\nPSfrag replacements\n\n0.2\n\n0\n\n\u22120.2\n\n\u22120.4\n\n\u22120.6\n\n\u22120.8\n\n)\ns\n(\n`\n\nPSfrag replacements\n\n\u22120.01\n0\n\n`(s)\n\n20\n\n40\n\n60\n\n80\n\ns [ms]\n\nQ0(s)\n\n100\n\n\u22121\n\n0\n\n10\n\n20\n\n30\n\ns [ms]\n\n40\n\n50\n\nFigure 1: Interspike interval distribution Q0 and normalized autocorrelation function `.\nThe circles show numerical results, the solid line the theory.\n\nThe Fisher information of (8) is calculated from (4) to be\n\nwith the instantaneous \ufb01ring rate \u20300(t) = g0R(t). Hence the mutual information is\n\nFor an interpretation of Eq. (11) we note that (cid:190)2 = \u00a7(0) is the variance of the mem-\nbrane potential and depends on the statistics of the presynaptic input whereas \u201e0 is the\nspontaneous \ufb01ring rate which characterizes the output of the postsynaptic neuron. Hence,\nEquation (11) contains both pre- and postsynaptic factors.\n\n3 Results: Optimal spike-timing dependent learning rule\n\nIn the previous section we have calculated the mutual information between presynaptic\ninput spike trains and the output of the postsynaptic neuron under the assumption of small\n\n0\n\nh\u20300(t)iY j\ufb02=0\n\ng0\u00b62\nJ0(t \u00a1 t0) = \u2013(t \u00a1 t0) (cid:181) g0\ng0\u00b62Z T\n2 (cid:181) g0\ng0\u00b62\n2 (cid:181) g0\n\nI(Y ; X) =\n\n\ufb022\n\n0\n\n0\n\n\ufb022\n\n=\n\ndt \u201e0(cid:190)2\n\n0\n\nT \u201e0(cid:190)2:\n\n(10)\n\n(11)\n\n(12)\n\n\f\ufb02uctuations of g. The mutual information depends on parameters of the model neuron, in\nparticular the synaptic weights that characterize the ef\ufb01cacy of the connections between\npre- and postsynaptic neurons. In this section, we will optimize the mutual information\nby changing the synaptic weights in an appropriate fashion. To do so we will proceed in\nseveral steps.\n\nFirst, based on gradient ascent we derive a batch learning rule of synaptic weights that\nmaximizes the mutual information. In a second step, we transform the batch rule into an\nonline rule that reduces to the batch version when averaged. Finally, in subsection 3.2, we\nwill see that the online learning rule shares properties with STDP, in particular a biphasic\ndependence upon the relative timing of pre- and postsynaptic spikes.\n\n3.1 Learning rule for spiking model neuron\n\nIn order to keep the analysis as simple as possible, we suppose that the input spike trains\nare independent Poisson trains, i.e., h\u00a2xi(t)\u00a2xj(t0)iX = \u201di\u2013(t \u00a1 t0)\u2013ij, where \u00a2xi(t) =\nxi(t) \u00a1 \u201di with rate \u201di = hxi(t)iX. Then we obtain the variance of the membrane potential\n(13)\n\nw2\n\nj \u201dj\n\n(cid:190)2 = h[\u00a2u(t)]2iX = \u20202Xj\n\nwith \u20202 =R \u20202(s)ds.\n\nApplying gradient ascent to (11) with an appropriate learning rate \ufb01, we obtain the batch\nlearning rule of synaptic weights as\n\n\u00a2wi = \ufb01\n\n@I(Y ; X)\n\n@wi\n\n\u2026 \ufb01\n\n\ufb022\n\n0\n\n2 (cid:181) g0\n\ng0\u00b62Z T\n\n0\n\ndt \u201e0\n\n@(cid:190)2\n@wi\n\n:\n\n(14)\n\nThe derivative of \u201e0 with respect to wi vanishes, since \u201e0 is the spontaneous \ufb01ring rate in\nthe absence of input. We note that both \u201e0 and (cid:190)2 are de\ufb01ned by an ensemble averages, as\nis typical for a \u2018batch\u2019 rule.\n\nWhile there are many candidates of online learning rule that give (14) on average, we\nare interested in rules that depend directly on neuronal spikes rather than mean rates. To\n\nproceed it is useful to write (cid:190)2 = h[\u00a2u(t)]2iX with \u00a2u = Pi wi\u00a2\u2020i(t) where \u00a2\u2020i(t) =\nR \u2020(s)\u00a2xi(t \u00a1 s)ds.\n\ndepends on both the postsynaptic \ufb01ring statistics and presynaptic autocorrelation is\n\nIn this notation, one simple form of an online learning rule that\n\ndwi\ndt\n\ng0\u00b62\n= \ufb01\ufb022(cid:181) g0\n\n0\n\ny(t)\u00a2\u2020i(t)\u00a2u(t);\n\n(15)\n\nHence weights are updated with each postsynaptic spike with an amplitude proportional\nto an online estimate of the membrane potential variance calculated as the product of\n\u00a2u and \u00a2\u2020i.\nIndeed, to order \ufb020, the input and the output spikes are independent;\nhy(t)\u00a2\u2020i(t)\u00a2u(t)iY;X = hy(t)iY j\ufb02=0h\u00a2\u2020i(t)\u00a2u(t)iX and the average of (15) leads back\nto (14).\n\n3.2 STDP function as a spike-pair effect\n\nApplication of the online learning rule (15) during a trial of duration T , yields a total\nchange of the synaptic ef\ufb01cacy which depends on all the presynaptic spikes via the factor\n\u00a2\u2020i; on the postsynaptic potential via the factor \u00a2u; and on the postsynaptic spike train\ny(t). In order to extract the spike pair effect evoked by a given presynaptic spike at tpre\nand a postsynaptic spike at tpost, we average over x and y given the pair of spikes. The\nspike pair effect up to the second order of \ufb02 is therefore described as\n\ni\n\n\u00a2wi(tpost \u00a1 tpre\n\ni\n\n) = \ufb01\ufb022(cid:181) g0\n\ng0\u00b62Z T\n\n0\n\n0\n\ndthy(t)iY jtpost;\ufb02=0h\u00a2\u2020i(t)\u00a2u(t)iXjtpre\n\ni\n\n;\n\n(16)\n\n\fwhere h\u00a2iY jtpost;\ufb02=0 = R dy \u00a2 P (yjtpost; \ufb02 = 0) and h\u00a2iXjtpre\n\n).\nNote that the leading factor of Eq.\n(16) is already of order \ufb02 2, so that all other fac-\ntors have to be evaluated to order \ufb020. Suppressing all terms containing \ufb02, we obtain\nP (yjtpost; u) \u2026 P (yjtpost; \ufb02 = 0) and from the Bayes formula P (xjtpre\n; tpost) =\n\n= R dx \u00a2 P (xjtpre\n\ni\n\ni\n\ni\n\nP (tpostjx;tpre\n\nhP (tpostjx;tpre\n\ni\n\ni\n\n)\n)iXjt\n\nP (xjtpre\n\ni\n\n) \u2026 P (xjtpre\n\ni\n\n).\n\npre\ni\n\ni\n\nIn order to see the contribution of tpre\nspikes at tpre\nwe insert hy(t)iY jtpost;\ufb02=0 = \u2013(t\u00a1tpost)+\u201e0[1+`(t\u00a1tpost)] and h\u00a2\u2020i(t)\u00a2u(t)iXjtpre\nwi[\u20202(t \u00a1 tpre) + \u20202\u201di] into Eq. (16) and decompose \u00a2wi(tpost \u00a1 tpre\nfour terms: the drift term \u00a2w0\n\nand tpost, we think of separating the effects caused by\n; tpost from the mean weight evolution caused by all other spikes. Therefore\n=\n) into the following\nT \u201e0\u20202wi\u201di of the batch learning (14) that\n\ni\n\ni\n\ni\n\ndoes not depend on tpre\nthat is triggered by the presynaptic spike at tpre\n\ni\n\nor tpost; the presynaptic component \u00a2wpre\n\n; the postsynaptic component \u00a2wpost\n\n=\n\ni\n\ni\n\ng0\u00b72\ni = \ufb01\ufb022\u2021 g0\n\n0\n\n\u201e0\u20202wi\n\n\ufb01\ufb022\u2021 g0\n\nat tpost; and the correlation component\n\n0\n\ng0\u00b72h1 + \u201e0R T\n0 `(t \u00a1 tpost)dti \u20202wi\u201di that is triggered by the postsynaptic spike\n)dt# (17)\ng0\u00b62\n= \ufb01\ufb022(cid:181) g0\n\nwi\"\u20202(tpost \u00a1 tpre\n\n) + \u201e0Z T\n\n`(t \u00a1 tpost)\u20202(t \u00a1 tpre\n\n0\n\n0\n\ni\n\ni\n\n\u00a2wcorr\n\ni\n\nthat depends on the difference of the pre- and postsynaptic spike timing.\n\ng0\u00b72\ni = \ufb01\ufb022\u2021 g0\n\n0\n\n(A)\n\n(B)\n\nFigure 2: (A) The effect from EPSP: the \ufb01rst term in the square bracket of (17). (B) The\neffect from refractoriness: the second term in the square bracket of (17). (C) Temporal\nlearning window \u00a2wcorr\n\nof (17).\n\ni\n\ni\n\ni\n\nIn the following, we choose a simple exponential EPSP \u2020(t) = \u00a3(s)e\u00a1s=\u00bfu with a time\nconstant \u00bfu = 10 ms. The parameters are N = 100, \u201di = 40 Hz for all i, wi = (N \u00bfu\u201di)\u00a11,\n\ufb01 = 1 and \ufb02 = 0:1.\nof (17). The \ufb01rst term of (17) indicates the contribution of a\nFigure 2 shows \u00a2wcorr\npresynaptic spike at tpre\nto increase the online estimation of membrane potential variance\nat time tpost, whereas the second term represents the effect of the refractory period on\npostsynaptic \ufb01ring intensity, i.e., the normalized autocorrelation function convolved with\nthe presynaptic contribution term. Due to the averaging of h\u00a2iY jtpost;\ufb02=0 and h\u00a2iXjtpre\nin\n(16), this optimal temporal learning window is local in time; we do not need to impose a\nmemory span [14] to restrict the negative part of the learning window.\nFigure 3 compares \u00a2wi of (16) with numerical simulations of (15). We note a good agree-\nment between theory and simulation. We recall, that all calculations, and hence the STDP\nfunction of (17) are valid for small \ufb02, i.e., for small \ufb02uctuation of g.\n\ni\n\n1\n\n0.8\n\n0.6\n\n0.4\n\n0.2\n\nPSfrag replacements\n\nPSfrag replacements\n\ntpost \u00a1 tpre\n\ni\n\n)\ns\n(\n2\n\n[ms]\u2020\n\ntpost \u00a1 tpre\n\n\u201e0(` \u2044 \u20202)(s)\n\u00a2wcorr\n\ni\n\n0\n\n\u221250\n\n\u221225\n\n0\n\n25\n\ns [ms]\n\n0.05\n\n0\n\n\u22120.05\n\n\u22120.1\n\n\u22120.15\n\n\u22120.2\n\n\u221250\n\n)\ns\n(\n)\n2\n\u2020\n[ms]\n\u2044\n`\n\u20202(s)\n(\n0\n\u201e\n\u00a2wcorr\n\ni\n\n50\ni\n\n(C)\n\nx 10\u22125\n\n15\n\nPSfrag replacements\nr\ns [ms]\nr\no\nc\nw\n\u00a2\n\u20202(s)\n\u201e0(` \u2044 \u20202)(s)\n\ni\n\n\u221225\n\n0\n\n25\n\n50\n\n10\n\n5\n\n0\n\n\u22125\n\u221250\n\n\u221225\n\n0\n\n25\n\n50\n\ns [ms]\n\ntpost \u00a1 tpre\n\n[ms]\n\ni\n\n\fx 10\u22124\n\n3.6\n\n3.4\n\n3.2\n\n3\n\n2.8\n\n2.6\n\n2.4\n\n2.2\n\ni\n\nw\n\u00a2\n\n\u221250\n\n\u221225\n\n0\ntpost \u00a1 tpre\n\ni\n\n[ms]\n\n25\n\n50\n\nPSfrag replacements\n[ms]\n\ntpost \u00a1 tpre\n\ni\n\nFigure 3: The comparison of the analytical result of (16) ( solid line ) and the numerical\nsimulation of the online learning rule (15) ( circles ). For the simulation, the conditional\naverage h\u00a2wiiX;Y jtpre\ndt over 200 ms around spike pairs\nwith the given interval tpost \u00a1 tpre\n\n;tpost is evaluated by integrating dwi\n\n;\n\ni\n\ni\n\n4 Conclusion\n\nIt is important for neurons especially in primary sensory systems to send information from\nprevious processing circuits to neurons in other areas while capturing the essential features\nof its input. Mutual information is a natural quantity to be maximized from this perspec-\ntive. We introduced an online learning rule for synaptic weights that increases information\ntransmission for small input \ufb02uctuation. Introduction of the temporal properties of the tar-\nget neuron enables us to analyze the temporal properties of the learning rule required to\nincrease the mutual information. Consequently, the temporal learning window is given in\nterms of the time course of EPSPs and the autocorrelation function of the postsynaptic neu-\nron. In particular, neuronal refractoriness plays a major role and yields the negative part\nof the learning window. Though we restrict our analysis here to excitatory synapses with\nindependent spike trains, it is straightforward to generalize the approach to a mixture of ex-\ncitatory and inhibitory neurons with weakly correlated spike trains as long as the synaptic\nweights are small enough. The analytically derived temporal learning window is similar to\nthe experimentally observed bimodal STDP window [1]. Since the effective time course\nof EPSPs and the autocorrelation function of output spike trains vary from one part of\nthe brain to another, it is important to compare those functions with the temporal learning\nwindow in biological settings.\n\nAcknowledgments\n\nT.T. is supported by the Japan Society for the Promotion of Science and a Grant-in-Aid for\nJSPS Fellows; J.-P.P. is supported by the Swiss National Science Foundation. We thank Y.\nAviel for discussions.\n\n\fReferences\n\n[1] G. Bi and M. Poo. Synaptic modi\ufb01cation of correlated activity: Hebb\u2019s postulate revisited.\n\nAnnu. Rev. Neurosci., 24:139\u2013166, 2001.\n\n[2] W. Gerstner and W. M. Kistler. Spiking Neuron Models. Cambridge University Press, 2002.\n[3] R. Kempter, W. Gerstner, and J. L. van Hemmen. Hebbian learning and spiking neurons. Phys.\n\nRev. E, 59:4498\u20134514, 1999.\n\n[4] W. Gerstner and W. M. Kistler. Mathematical formulations of hebbian learning. Biol. Cybern.,\n\n87:404\u2013415, 2002.\n\n[5] R. G\u00a8utig, R. Aharonov, S. Rotter, and H. Sompolinsky. Learning input correlations through\n\nnonlinear temporally asymmetric hebbian plasticity. J. Neurosci., 23(9):3697\u20133714, 2003.\n\n[6] R. B. Stein. The information capacity of nerve cells using a frequency code. Biophys. J.,\n\n7:797\u2013826, 1967.\n\n[7] W. Bialek, F. Rieke, R. de Ruyter van Stevenick, and D. Warland. Reading a neural code.\n\nScience, 252:1854\u20131857, 1991.\n\n[8] F. Rieke, D. Warland, R. R. van Steveninck, and W. Bialek. Spikes. MIT Press, 1997.\n[9] R. Linsker. Self-organization in a perceptual network. Computer, 21:105\u2013117, 1988.\n[10] J-P. Nadal and N. Parga. Nonlinear neurons in the low-noise limit: a factorial code maximizes\n\ninformation transfer. Network: Comput.Neural Syst., 5:565\u2013581, 1994.\n\n[11] J-P Nadal, N. Brunel, and N Parga. Nonlinear feedforward networks with stochastic outputs:\n\ninfomax implies redundancy reduction. Network: Comput. Neural Syst., 9:207\u2013217, 1998.\n\n[12] A. J. Bell and T. Sejnowski. An information-maximization approach to blind separation and\n\nblind deconvolution. Neural Comput., 7(6):1004\u20131034, 1995.\n\n[13] J. J. Hop\ufb01eld. Encoding for computation: recognizing brief dynamical patterns by exploiting\neffects of weak rhythms on action-potential timing. Proc. Natl. Acad. Sci. USA, 101(16):6255\u2013\n6260, 2004.\n\n[14] G. Checkik. Spike-timing-dependent plasticity and relevant mutual information maximization.\n\nNeural Comput., 15:1481\u20131510, 2003.\n\n[15] V. V. Prelov and E. C. van der Meulen. An asymptotic expression for the information and\ncapacity of a multidimensional channel with weak input signals. IEEE. Trans. Inform. Theory,\n39(5):1728\u20131735, 1993.\n\n[16] T. M. Cover and J. A. Thomas. Elements of Information Theory. New York: Wiley, 1991.\n[17] N. Brunel and J-P. Nadal. Mutual information, \ufb01sher information, and population coding. Neu-\n\nral Comput., 10:1731\u20131757, 1998.\n\n\f", "award": [], "sourceid": 2588, "authors": [{"given_name": "Taro", "family_name": "Toyoizumi", "institution": null}, {"given_name": "Jean-pascal", "family_name": "Pfister", "institution": null}, {"given_name": "Kazuyuki", "family_name": "Aihara", "institution": null}, {"given_name": "Wulfram", "family_name": "Gerstner", "institution": null}]}