{"title": "Efficient Spike-Coding with Multiplicative Adaptation in a Spike Response Model", "book": "Advances in Neural Information Processing Systems", "page_first": 1835, "page_last": 1843, "abstract": "Neural adaptation underlies the ability of neurons to maximize encoded information over a wide dynamic range of input stimuli. While adaptation is an intrinsic feature of neuronal models like the Hodgkin-Huxley model, the challenge is to integrate adaptation in models of neural computation. Recent computational models like the Adaptive Spike Response Model implement adaptation as spike-based addition of fixed-size fast spike-triggered threshold dynamics and slow spike-triggered currents. Such adaptation has been shown to accurately model neural spiking behavior over a limited dynamic range. Taking a cue from kinetic models of adaptation, we propose a multiplicative Adaptive Spike Response Model where the spike-triggered adaptation dynamics are scaled multiplicatively by the adaptation state at the time of spiking. We show that unlike the additive adaptation model, the firing rate in the multiplicative adaptation model saturates to a maximum spike-rate. When simulating variance switching experiments, the model also quantitatively fits the experimental data over a wide dynamic range. Furthermore, dynamic threshold models of adaptation suggest a straightforward interpretation of neural activity in terms of dynamic signal encoding with shifted and weighted exponential kernels. We show that when thus encoding rectified filtered stimulus signals, the multiplicative Adaptive Spike Response Model achieves a high coding efficiency and maintains this efficiency over changes in the dynamic signal range of several orders of magnitude, without changing model parameters.", "full_text": "Ef\ufb01cient Spike-Coding with Multiplicative\n\nAdaptation in a Spike Response Model\n\nSander M. Bohte\nCWI, Life Sciences\n\nAmsterdam, The Netherlands\n\nS.M.Bohte@cwi.nl\n\nAbstract\n\nNeural adaptation underlies the ability of neurons to maximize encoded informa-\ntion over a wide dynamic range of input stimuli. Recent spiking neuron mod-\nels like the adaptive Spike Response Model implement adaptation as additive\n\ufb01xed-size fast spike-triggered threshold dynamics and slow spike-triggered cur-\nrents. Such adaptation accurately models neural spiking behavior over a limited\ndynamic input range. To extend ef\ufb01cient coding over large changes in dynamic in-\nput range, we propose a multiplicative adaptive Spike Response Model where the\nspike-triggered adaptation dynamics are scaled multiplicatively by the adaptation\nstate at the time of spiking. We show that, unlike the additive adaptation model,\nthe \ufb01ring rate in our multiplicative adaptation model saturates to a realistic max-\nimum spike-rate regardless of input magnitude. Additionally, when simulating\nvariance switching experiments, the model quantitatively \ufb01ts experimental data\nover a wide dynamic range. Dynamic threshold models of adaptation furthermore\nsuggest a straightforward interpretation of neural activity in terms of dynamic dif-\nferential signal encoding with shifted and weighted exponential kernels. We show\nthat when thus encoding recti\ufb01ed \ufb01ltered stimulus signals, the multiplicative adap-\ntive Spike Response Model achieves a high coding ef\ufb01ciency and maintains this\nef\ufb01ciency over changes in the dynamic signal range of several orders of magni-\ntude, without changing model parameters.\n\n1\n\nIntroduction\n\nThe ability of neurons to adapt their responses to greatly varying sensory signal statistics is central\nto ef\ufb01cient neural coding [1, 2, 3, 4, 5, 6, 7]. Consequently, accurate models for the underlying\nmechanisms can provide insight into the nature of neural coding itself. For this, models of neural\ncomputation have to account for adaptation in a manner consistent with both experimental \ufb01ndings\nand notions of ef\ufb01cient neural coding.\nNeural computation is often reduced to a linear-nonlinear-poisson (LNP) model: input signals are\n\ufb01ltered, followed by a thresholding function that determines the \ufb01ring probability of the neuron. In\nthe Generalized Linear Model (GLM) [8] a refractory response in the form of a post-spike \ufb01lter is\nadded (\ufb01gure 1). With experimental responses \ufb01tted to such LNP models, adaptation is found to\nadjust both the effective gain in the thresholding function and the linear \ufb01ltering function [9, 10].\nNeural adaptation responds primarily to changes in local stimulus contrast or, equivalently, to the\nlocal detection threshold [11, 12], and a number of theoretical studies account for adaptation from\nthe perspective of optimal contrast estimation [12, 13]. Recent work by Ozuysal & Baccus [14]\nsuggests that in a Linear-Nonlinear \ufb01rst-order Kinetics model (LNK), the gain depends on the local\ncontrast of the \ufb01ltered and recti\ufb01ed input signal.\n\n1\n\n\fFigure 1: Generalized Linear Model (GLM) of neural computation.\n\nWith substantial spike-rate adaptation occurring on a time scale of just tens of milliseconds [4, 5],\nadapting neurons necessarily generate at most tens of spikes in that period. From an adaptive coding\nperspective, this implies that for a neuron\u2019s adaptation to be computable by downstream neurons,\nthe adaptation effects have to be derivable from just the emitted spike-train. Spike-based models are\nthus central when accounting for adaptation as adaptive neural coding.\nIn variations of adaptive integrate-and-\ufb01re neurons [15, 16, 17], adaptation can be incorporated as\na combination of two mechanisms: spike-triggered adaptation currents and a dynamical action-\npotential threshold. In such models, the adaptation mechanisms together increase the distance be-\ntween the reversal potential and the threshold, effectively changing the gain of the neuron. The\nadaptive Spike Response Model [16, 17] in particular has been shown to be effective for modeling\nneural behavior in response to input currents with limited dynamic range [17]. On longer timescales,\nspike-triggered adaptation currents \ufb01t a power-law decay rather than an exponential decay, linking\nto observations of long-range power-law rate-adaptation [18, 19, 20, 21, 17].\nStill, in spite of its success, the additive model of adaptation in adaptive Spike Response Model\neffectively changes neural gain with at most a \ufb01xed step-size, and thus cannot respond quickly to\nchanges in signal variance that are large compared to this step-size. In particular, Brette [22] argues\nthat adaptation modulation has to be multiplicative for neurons to respond with the same level of\nneural activity to drastic changes in dynamic range, as is observed experimentally (e.g. [4]).\nWe augment the adaptive Spike Response Model with multiplicative adaptation dynamics. We show\nthat such a multiplicative adaptive Spike Response Model quantitatively matches neural responses in\nvariance switching experiments and maximizes information transfer. Furthermore, we demonstrate\nthat the model\u2019s effective gain responds to changes in either mean or variance of the \ufb01ltered signal,\nsimilar to the LNK kinetic model in [14].\nIn the adaptive Spike Response Model, gain modulation derives from the difference between the\nadapted reversal potential and the dynamic threshold. This suggests a straightforward interpreta-\ntion of spike-trains in terms of threshold-based detection of discernible signal levels in the recti\ufb01ed\n\ufb01ltered input signal: adaptive spike-coding. We show how non-linear signal encoding with a multi-\nplicative adaptive Spike Response Model maintains a high coding ef\ufb01ciency for stimuli that vary in\nmagnitude over several orders of magnitude, unlike the additive version of the adaptive Spike Re-\nsponse Model. The coding ef\ufb01ciency is further comparable to the additive adaptive Spike Response\nModel when the adaptation step-size in the latter is optimized for the local dynamic range.\n\n2 Spike-rate Adaptation in the Spike Response Model\n\nWe follow Naud et al [17] in modeling adaptation in an augmented Spike-Response Model [23]. In\nthe adaptive Spike Response Model (aSRM), the dynamics of the (normalized) membrane-potential\nV (t) are described as a sum of integrated input current I(t) and spike-triggered currents \u03b7(t):\n\nV (t) =\n\n\u03c6(t \u2212 s)I(s)ds \u2212\n\n\u03c6(t \u2212 s)\n\n\u03b7(s \u2212 ti)ds,\n\n(1)\n\n(cid:90)\n\nwhere {ti} denotes the set of past emitted spikes, and the kernel \u03c6(t) is a fast exponential low-pass\n\ufb01lter on membrane currents:\n\n(cid:90)\n\n(cid:88)\n\n{ti}\n\n(cid:18)\u2212t\n\n(cid:19)\n\n,\n\n\u03c4m\n\n\u03c6(t) = \u03c60 exp\n\n2\n\nLinear (cid:31)lterSpikingNonlinearitypost-spike-(cid:31)lteroutputdelayinputg(t)s(t)u(t){t }i\fwith \u03c4m determined by the membrane capacitance and conductance, and is typically on the order of\nseveral milliseconds [23, 17] .\nThe dynamical threshold is computed as the sum of a resting threshold V0 and spike-triggered thresh-\nold dynamics \u03b3(t):\n\nVT (t) = V0 +\n\n\u03b3(t \u2212 ti).\n\n(2)\n\n(cid:88)\n\n{ti}\n\nSpikes are generated either deterministically when V (t)\u2212 VT (t) becomes positive, or stochastically\nfollowing an inhomogeneous point process with conditional \ufb01ring rate:\n\n\u03bb(t|V (t), VT (t)) = \u03bb0 exp\n\n,\n\n(3)\n\n(cid:18) V (t) \u2212 VT (t)\n\n(cid:19)\n\n\u2206V\n\nwhere \u2206V determines the slope of the exponential function; small values of \u2206V approximate a\nneuron with a deterministic threshold. Naud et al [17] report that the threshold kernel \u03b3(t) is best\n\ufb01tted with an exponentially decaying function, whereas the shape of the spike-triggered current \u03b7(t)\ndepends on the type of neuron, and furthermore for longer timescales best \ufb01ts a decaying power-law:\n\u03b7(t \u2212 ti) \u221d (t \u2212 ti)\u2212\u03b2 for t >> ti, with \u03b2 \u2248 1.\nWe can denote the effective neural threshold \u03d1 as the amount of input that will trigger a spike. In\nthe adaptive Spike Response Model this amounts to the sum of the dynamic threshold, VT (t), and\n\nthe (\ufb01ltered) spike-triggered current: \u03d1 \u221d VT (t) +(cid:82) \u03c6(t \u2212 s)(cid:80){ti} \u03b7(s \u2212 ti)ds. We can move the\n\nreset response from (1) to the dynamic threshold (2) to obtain adaptation as the effective threshold\ndynamics \u03d1(t):\n\n\u03d1(t) = \u03d10 +\n\n\u03b3(t \u2212 ti) +\n\n\u03c6(t \u2212 s)\u03b7(s \u2212 ti)ds\n\n,\n\n(4)\n\n(cid:90)\n\n(cid:21)\n\n(cid:20)\n\n(cid:88)\n\n{ti}\n\nwhere \u03d10 = V0 denotes the effective threshold for an inactive neuron. As the adaptation dynamics\nin this model are strictly additive, we will refer to it further as the additive aSRM.\nThe maximum effective threshold in the additive aSRM is limited by the maximum number of spikes\nthat can be generated within the short time-window reported for variance adaptation. Effectively,\nthe refractory period determines the upper bound for the adaptation step-size, and adaptation speed\nis upper-bounded by this value times the number of generated spikes.\n\n2.1 Multiplicative Dynamic Adaptation\n\nWe propose a modi\ufb01cation of the additive aSRM where the effective spike-triggered adaptation is\nnot a \ufb01xed quantity but depends on the effective adaptation at the time of spiking. We include the\nmultiplicative interaction in the aSRM by scaling the effective adaptation in (4) with the current\nadaptation value at the time of spiking:\n\n\u03d1(t) = \u03d10 +\n\n\u03d1(ti)\n\n\u03b3(t \u2212 ti) +\n\n\u03c6(t \u2212 s)\u03b7(s \u2212 ti)ds\n\n.\n\n(5)\n\n(cid:20)\n\n(cid:88)\n\n{ti}\n\n(cid:90)\n\n(cid:21)\n\nFor sparse spiking and adaptation response kernels that decay fairly rapidly to zero, such multi-\nplicative adaptive threshold dynamics are approximately similar to the effective threshold dynamics\nin (4). For rapid signal variance transitions however, the multiplicative dynamics ensure that the\neffective threshold adaptation can rapidly range over multiple orders of magnitude.\nThe key difference in adaptation dynamics for the two aSRM models is illustrated in Figure 2. For\na given spike-train, the respective adaptation magnitudes are plotted in Figure 2a , and the neural\nresponses to different levels of step-size current injections are shown in Figure 2b. The additive\naSRM responds to an increasing input current with a \ufb01ring rate that is essentially only bounded\nby the refractory response; the \ufb01ring rate in the aSRM with multiplicative adaptation saturates at a\nmuch lower value as the effective threshold catches up with the magnitude of the injected current.\n2.2 Adaptive Spike-Coding\n\nThe interpretation of spike-triggered adaptation as dynamic neural gain in the Spike Response Model\nsuggests a straightforward application to a spike-based neural coding model. Spike-rate adaptation\n\n3\n\n\fFigure 2: Illustration of multiplicative and additive threshold adaptation dynamics. (a) Effective\nadaptation as a sum of threshold dynamics (solid lines) and spike-triggered currents (dashed lines)\ngiven an input spike-train (black dots). Red lines correspond to additive adaptation dynamics, blue\nlines to multiplicative. (b) Firing rate as a function of signal strength. Red solid line is response for\n(stochastic) additive aSRM, blue solid line for the stochastic multiplicative aSRM; dotted blue line\ncorresponds to a deterministic version of the multiplicative aSRM.\n\nhas been extensively studied from the point of view of optimal contrast estimation or signal threshold\ndetection [13, 12]. In particular the notion of signal threshold detection suggests a simple model\nwhere individual spikes signal that the neuron has detected that its internally computed value has\nreached a level distinguishable from the local noise level [11].\nTaking the standard Linear-Non-Linear model of neural computation, we follow Ozuysal & Baccus\n[14] in assuming that it is the recti\ufb01ed \ufb01ltered version of the stimulus signal, u(t), that is encoded\nby the spikes emitted by a neuron. We then de\ufb01ne the Linear-Non-Linear-Adaptive-Thresholding\n(LNL-AT) model as greedy differential signaling: if the signal u(t) exceeds a threshold value \u03d1(ti)\nat time ti, a spike is generated communicating a scaled response kernel \u03d1(ti)\u03ba(t\u2212ti) to downstream\nneurons. This response kernel is then also subtracted from the signal u(t), and the dynamic threshold\nis updated to account for threshold adaptation (\ufb01gure 3). In such greedy differential spike-coding,\nthe signal u(t) is effectively approximated as a sum of shifted and weighted response kernels:\n\n(cid:88)\n\n\u02c6u(t) =\n\n\u03d1(ti)\u03ba(t \u2212 ti).\n\n\ufb01ltered reset function(cid:82) \u03c6(t \u2212 s)\u03b7(t)ds is interpreted as a response kernel \u03ba(t \u2212 ti):\n\nThis adaptive spike-coding model corresponds to the multiplicative adaptive SRM in (5), where the\n\nti