{"title": "Short-Term Depression in VLSI Stochastic Synapse", "book": "Advances in Neural Information Processing Systems", "page_first": 1817, "page_last": 1824, "abstract": "We report a compact realization of short-term depression (STD) in a VLSI stochastic synapse. The behavior of the circuit is based on a subtractive single release model of STD. Experimental results agree well with simulation and exhibit expected STD behavior: the transmitted spike train has negative autocorrelation and lower power spectral density at low frequencies which can remove redundancy in the input spike train, and the mean transmission probability is inversely proportional to the input spike rate which has been suggested as an automatic gain control mechanism in neural systems. The dynamic stochastic synapse could potentially be a powerful addition to existing deterministic VLSI spiking neural systems.", "full_text": "Short-Term Depression in VLSI Stochastic Synapse\n\nPeng Xu, Timothy K. Horiuchi, and Pamela Abshire\n\nDepartment of Electrical and Computer Engineering, Institute for Systems Research\n\nUniversity of Maryland, College Park, MD 20742\n\npxu,timmer,pabshire@umd.edu\n\nAbstract\n\nWe report a compact realization of short-term depression (STD) in a VLSI sto-\nchastic synapse. The behavior of the circuit is based on a subtractive single re-\nlease model of STD. Experimental results agree well with simulation and exhibit\nexpected STD behavior: the transmitted spike train has negative autocorrelation\nand lower power spectral density at low frequencies which can remove redun-\ndancy in the input spike train, and the mean transmission probability is inversely\nproportional to the input spike rate which has been suggested as an automatic\ngain control mechanism in neural systems. The dynamic stochastic synapse could\npotentially be a powerful addition to existing deterministic VLSI spiking neural\nsystems.\n\n1 Introduction\n\nSynapses are the primary locations in neural systems where information is processed and transmit-\nted. Synaptic transmission is a stochastic process by nature, i.e. it has been observed that at central\nsynapses transmission proceeds in an all-or-none fashion with a certain probability. The synaptic\nweight has been modeled as R = npq [1], where n is the number of quantal release sites, p is the\nprobability of release per site, and q is some measure of the postsynaptic effect. The synapse un-\ndergoes constant changes in order to learn from and adapt to the ever-changing outside world. The\nvariety of synaptic plasticities differ in the triggering condition, time span, and involvement of pre-\nand postsynaptic activity. Regulation of the vesicle release probability has been considered as the\nunderlying mechanism for various synaptic plasticities [1\u20133]. The stochastic nature of the neural\ncomputation has been investigated and the bene\ufb01ts of stochastic computation such as energy ef\ufb01-\nciency, communication ef\ufb01ciency, and computational ef\ufb01ciency have been shown [4\u20136]. Recently\nthere is increasing interest in probabilistic modeling of brain functions [7]. VLSI stochastic synapse\ncould provide a useful hardware tool to investigate stochastic nature of the synapse and also function\nas the basic computing unit for VLSI implementation of stochastic neural computation.\n\nAlthough adaptive deterministic VLSI synapses have been extensively studied and developed for\nneurally inspired VLSI learning systems [8\u201313], stochastic synapses have been dif\ufb01cult to imple-\nment in VLSI because it is hard to properly harness the probabilistic behavior, normally provided\nby noise. Although stochastic behavior in integrated circuits has been investigated in the context of\nrandom number generators (RNGs) [14], these circuits either are too complicated to use for a sto-\nchastic synapse or suffer from poor randomness. Therefore other approaches were explored to bring\nrandomness into the systems. Stochastic transmission was implemented in software using a lookup\ntable and a pseudo random number generator [15]. Stochastic transition between potentiation and\ndepression has been demonstrated in bistable synapses driven by stochastic spiking behavior at the\nnetwork level for stochastic learning [16].\n\nPreviously we reported the \ufb01rst VLSI stochastic synapse. Experimental results demonstrated true\nrandomness as well as the adjustable transmission probability. The implementation with \u223c 15 tran-\nsistors is compact for these added features, although there are much more compact deterministic\n\n\fsynapses with as few as \ufb01ve transistors. We also proposed the method to implement plasticity and\ndemonstrated the implementation of STD by modulating the probability of spike transmission. Like\nits deterministic counterpart, this stochastic synapse operates on individual spike train inputs; its\nstochastic character, however, creates the possibility of a broader range of computational primitives\nsuch as rate normalization of Poisson spike trains, probabilistic multiplication, or coincidence de-\ntection. In this paper we extend the subtractive single release model of STD to the VLSI stochastic\nsynapse. We present the simulation of the new model. We describe a novel compact VLSI imple-\nmentation of a stochastic synapse with STD and demonstrate extensive experimental results showing\nthe agreement with both simulation and theory over a range of conditions and biases.\n\n2 VLSI Stochastic Synapse and Plasticity\n\nFigure 1: Schematic of the stochastic synapse with STD.\n\nPreviously we demonstrated a compact stochastic synapse circuit exhibiting true randomness and\nconsuming very little power (10-44 \u00b5W). The core of the structure is a clocked, cross-coupled dif-\nferential pair comparator with input voltages Vi+ and Vi\u2212, as shown in the dashed box in Fig. 1.\nIt uses competition between two intrinsic circuit noise sources to generate random events. The dif-\nferential design helps to reduce the in\ufb02uence from other noise sources. When a presynaptic spike\narrives, Vpre\u223c goes low, and transistor M5 shuts off. Vo+ and Vo\u2212 are nearly equal and the circuit is\nin its metastable state. When the two sides are closely matched, the imbalance between Vo+ and Vo\u2212\ncaused by current noise in M1-M4 eventually triggers positive feedback, which drives one output\nto Vc and the other close to ground. We use a dynamic buffer, shown in the dotted box in Fig. 1,\nto generate rail-to-rail transmitted spikes Vtran. Vtran either goes high (with probability p) or stays\nlow (with probability 1 \u2212 p) during an input spike, emulating stochastic transmission.\nFabrication mismatch in an uncompensated stochastic synapse circuit would likely permanently bias\nthe circuit to one solution. In this circuit, \ufb02oating gate inputs to a pFET differential pair allow the\nmismatch to be compensated. By controlling the common-mode voltage of the \ufb02oating gates, we\noperate the circuit such that hot-electron injection occurs only on the side where the output voltage\nis close to ground. Over multiple clock cycles hot-electron injection works in negative feedback\nto equalize the \ufb02oating gate voltages, bringing the circuit into stochastic operation. The procedure\ncan be halted to achieve a speci\ufb01c probability or allowed to reach equilibrium (50% transmission\nprobability).\n\nThe transmission probability can be adjusted by changing the input offset or the \ufb02oating gate\ncharges. The higher Vg+ is, the lower p is. The probability tuning function is closely \ufb01tted by\nan error function f(v) = 0.5\n, where \u00b5 is the input offset voltage for p = 50%,\n\u03b4 is the standard deviation characterizing the spread of the probability tuning, and v = Vi\u2212 \u2212 Vi+\nis the input offset voltage. Synaptic plasticity can be implemented by dynamically modulating the\nprobability. Input offset modulation is suitable for short-term plasticity. Short-term depression is\ntriggered by the transmitted input spikes Vtran to emulate the probability decrease because of vesi-\ncle depletion. Short-term facilitation is triggered by the input spikes Vpre to emulate the probability\n\n1 + erf\n\nv\u2212\u00b5\u221a\n\n2\u03b4\n\n(cid:179)\n\n(cid:179)\n\n(cid:180)(cid:180)\n\nVdd2Vi+VcVo-Vg+Vg-Vo+M1M2M3M4IbiasM5Vpre~VwVpVi-M7M6VddVddVhVwVicmVrVicmVrVpreVbiasVo+Vo-CVtran\fincrease because of presynaptic Ca2+ accumulation. Nonvolatile storage at the \ufb02oating gate is suit-\nable for long-term plasticity. STDP can be implemented by modulating the probability depending\non the precise timing relation between the pre- and postsynaptic spikes.\n\n3 Short-Term Depression: Model and Simulation\n\nAlthough long-term plasticity has attracted much attention because of its apparent association with\nlearning and memory, the functional role of short-term plasticity has only recently begun to be un-\nderstood. Recent evidence suggests that short-term synaptic plasticity is involved in many functions\nsuch as gain control [17], phase shift [18], coincidence detection, and network recon\ufb01guration [19].\nIt has also been shown that depressing stochastic synapses can increase information transmission\nef\ufb01ciency by \ufb01ltering out redundancy in presynaptic spike trains [5].\n\nActivity dependent short-term changes in synaptic ef\ufb01cacy at the macroscopic level are determined\nby activity dependent changes in vesicle release probability at the microscopic level. We will focus\non STD here. STD during repetitive stimulation results from a decrease in released vesicles. Since\nthere is a \ufb01nite pool of vesicles, and released vesicles cannot be replenished immediately, a success-\nful release triggered by one spike potentially reduces the probability of release triggered by the next\nspike. We propose an STD model based on our VLSI stochastic synapse that closely emulates the\nsimple subtractive single release model [5, 20]. A presynaptic spike that is transmitted reduces the\ninput offset voltage v at the VLSI stochastic synapse by \u2206v, so that the transmission probability p(t)\nis reduced. Between successful releases, v relaxes back to its maximum value vmax exponentially\nwith a time constant \u03c4d so that p(t) relaxes back to its maximum value pmax as well. The model can\nbe written as\n\nv(t+) = v(t\u2212) \u2212 \u2206v, successful transmission at t\ndv(t)\n\n= vmax \u2212 v(t)\n\n\u03c4d\n\ndt\np(t) = f(v(t))\n\n(3)\nFor an input spike train with Poisson arrivals, the model can be expressed as a stochastic differential\nequation\n\n(4)\nwhere dNp\u00b7r(t) is a Poisson counting process with rate p \u00b7 r(t), and r(t) is the input spike rate. By\ntaking the expectation E(\u00b7) on both sides, we obtain a differential equation\n\n\u03c4d\n\ndv = vmax \u2212 v\n\ndt \u2212 \u2206v \u00b7 dNp\u00b7r(t)\n\n(1)\n\n(2)\n\n(5)\n\n(cid:180)(cid:180)\n\n(6)\n\ndE(v)\n\ndt\n\n= vmax \u2212 E(v)\n\n\u03c4d\n\n\u2212 \u2206v \u00b7 E(p)r(t)\n\n(cid:179)\n\n(cid:179)\n\nWhen v is reduced, the probability that it will be reduced again becomes smaller. v is effectively\nconstrained to a small range where we can approximate the function f(v) = 0.5\nby a linear function f(v) = av + 0.5, where \u00b5 = 0 for simplicity. We can then solve for E(p) at\nsteady state:\n\n1 + erf\n\nv\u2212\u00b5\u221a\n\n2\u03b4\n\npss \u2248 avmax + 0.5\n1 + a\u2206v\u03c4dr\n\n\u2248 pmax\na\u2206v\u03c4dr\n\n\u221d 1\nr\n(cid:179)\n\n(cid:179)\n\n(cid:180)(cid:180)\n\nTherefore the steady state mean probability is inversely proportional to the input spike rate when\na\u2206v\u03c4dr (cid:192) 1. This is consistent with prior work that modeled STD at the macroscopic level [17].\nWe simulated the model (1)-(3). We use the function f(v) = 0.5\n, obtained\nfrom the best \ufb01t of the experimental data. Initially v is set to 5 mV which sets pmax close to 1.\nAlthough the transformation from v to p is nonlinear, both simulation and experimental data show\nthat this implementation exhibits behavior similar to the model with the linear approximation and\nthe biological data. Fig. 2(a) and 2(b) show that the mean probability is a linear function of the\ninverse of the input spike rate at various \u2206v and \u03c4d for high input spike rates. Both \u2206v and \u03c4d affect\nthe slope of the linear relation, following the trend suggested by (6): the bigger the \u2206v or the bigger\nthe \u03c4d, the smaller the slope is. Fig. 3 shows a simulation of the transient probability for a period\nof 200 ms. Fig. 4 shows that the output spike train exhibits negative autocorrelation at small time\nintervals and lower power spectral density (PSD) at low frequencies. This is a direct consequence\nof STD.\n\n1 + erf\n\n2\u00b72.16\n\nv\u221a\n\n\f(a) \u2206v = 2, 4, 6 mV, \u03c4d = 100 ms.\n\n(b) \u03c4d = 100, 200, 300 ms, \u2206v = 2 mV.\n\nFigure 2: Mean probability as a function of input spike rate from simulation. Data were collected at\ninput rates from 100 Hz to 1000 Hz at 100 Hz intervals. The solid lines show the least mean square\n\ufb01t for input rates from 400 Hz to 1000 Hz.\n\nFigure 3: Simulated probability trajectory over 200 ms period. r = 100 Hz, \u03c4 = 100 ms, \u2206v = 2\nmV.\n\n(a) Autocorrelation.\n\n(b) Power spectral density.\n\nFigure 4: Characterization of the output spike train from the simulation of the stochastic synapse\nwith STD. r = 100 Hz, \u03c4d = 200 ms, \u2206v = 6 mV, Vmax = 5 mV.\n\n00.0020.0040.0060.0080.0100.050.10.150.20.250.30.350.41/rp \u2206v = 2 mV\u2206v = 4 mV\u2206v = 6 mV00.0020.0040.0060.0080.0100.050.10.150.20.250.30.350.41/rp \u03c4d = 100 ms\u03c4d = 200 ms\u03c4d = 300 ms02040608010012014016018020000.10.20.30.40.50.6Time (ms)p(t)01020304050\u22120.0200.020.040.060.080.1IntervalsAutocorrelation01020304050\u221280\u221260\u221240\u221220020Frequency (Hz)PSD (dB)\f4 VLSI Implementation of Short-Term Depression\n\nWe implemented this model using the stochastic synapse circuit described above (see Fig. 1). Both\ninputs are restored up to an equilibrium value Vicm by tunable resistors implemented by subthreshold\npFETs operating in the ohmic region. To change the transmission probability we only need to\nmodulate one side of the input, in this case Vi\u2212. The resistor and capacitor provide for exponential\nrecovery of the voltage to its equilibrium value. The input Vi\u2212 is modulated by transistors M6\nand M7 based on the result of the previous spike transmission. Every time a spike is transmitted\nsuccessfully, a pulse with height Vh and width Tp is generated at Vp. Tp is same as the input\nspike pulse width. This pulse discharges the capacitor with a small current determined by Vw and\nreduces Vi\u2212 by a small amount, thus decreasing the transmission probability. The value of the\ntunable resistors is controlled by the gate voltage of the pFETs, Vr. When Vi\u2212 is reduced, the\nprobability that it will be reduced again becomes smaller. Since the probability tuning only occurs\nin a small voltage range (\u223c 10 mV), the change in Vi\u2212 is limited to this small range as well. Under\nthis special condition, the resistance implemented by the subthreshold pFET is linear and large (\u223c\nG\u2126). With capacitance as small as 100 fF, the exponential time constant is tens of milliseconds and\nis adjustable. Similar control circuits can be applied to Vi+ to implement short-term facilitation.\nThe update mechanism would then be driven by the presynaptic spike rather than the successfully\ntransmitted spike. The extra components on the left provide for future implementation of short-term\nfacilitation and also symmetrize the stochastic synapse, improving its randomness.\n\n5 Experimental Results\n\nThe circuit has been fabricated in a commercially-available 0.5 \u00b5m CMOS process with 2 polysil-\nicon layers and 3 metal layers. The layout size of the stochastic synapse is 151.9 \u00b5m \u00d7 91.7 \u00b5m\nand the layout size of the STD block is 35 \u00b5m \u00d7 32.2 \u00b5m. A 2-to-1 multiplexer with size 35 \u00b5m\n\u00d7 30 \u00b5m is used to enable or disable STD. As a proof of concept, the layout of the circuit is quite\nconservative. Assuming no loss of performance, the existing circuit area could be reduced by 50%.\n\nThe circuit uses a nominal power supply of 5 V for normal operation. The differential pair compara-\ntor uses a separate power supply for hot-electron injection. Each \ufb02oating-gate pFET has a tunnelling\nstructure, which is a source-drain connected pFET with its gate connected to the \ufb02oating node. A\nseparate power supply provides the tunnelling voltage to the shorted source and drain (tunnelling\nnode). When the tunnelling voltage is high enough (\u223c14-15 V), electron tunnels through the sil-\nicon dioxide, from the \ufb02oating gate to the tunnelling node. We use this phenomenon to remove\nelectrons from the \ufb02oating gate only during initialization. Alternatively Ultra-Violet (UV) activated\nconductances may be used to remove electrons from the gate to avoid the need for special power\nsupplies.\n\nTo begin the test, we \ufb01rst remove residual charges on the \ufb02oating gates in the stochastic synapse.\nWe set Vicm = 2 V. We raise the power supply of the differential pair comparator to 5.3 V to\nfacilitate the hot-electron injection. We use the negative feedback operation of hot-electron injection\ndescribed above to automatically program the circuit into its stochastic regime. We halt the injection\nby lowering the power supply to 5 V. During this procedure, STD is disabled, so that the probability\nat this operating point is the synaptic transmission probability without any dynamics.\n\nWe then enable STD. We use a signal generator to generate pulse signals which serve as input\nspikes. Although spike trains are better modeled by Poisson arrivals, the averaging behavior should\nbe similar for deterministic spike trains which make testing easier. We use Ibias = 100 nA. The\npower consumption of the STD block is much smaller than the stochastic synapse. The total power\nconsumption is about 10 \u00b5W.\n\nWe collect output spikes from the depressing stochastic synapse at an input spike rate of 100 Hz. We\ndivide time into bins according to the input spike rate so that in each bin there is either 1 or 0 output\nspike. In this way, we convert the output spike train into a bit sequence s(k). We then compute the\nnormalized autocorrelation, de\ufb01ned as A(n) = E(s(k)s(k + n)) \u2212 E2(s(k)), where n is the num-\nber of time intervals between two bits. A(0) gives the variance of the sequence. For two bits with\ndistance n > 0, A(n) = 0 if they are independent, indicating good randomness, and A(n) < 0 if\nthey are anticorrelated, indicating the depressing effect of preceding spikes on the later spikes. Fig.\n5 shows the autocorrelation of the output spike trains at two different Vr. There is signi\ufb01cant nega-\n\n\ftive correlation at small time intervals and little correlation at large time intervals, as expected from\nSTD. Fig. 6 shows the PSD of the output spike trains from the same data shown in Fig. 5. Clearly,\nthe PSD is reduced at low frequencies. The time constant of STD increases with Vr so that the larger\nVr is, the longer the period of the negative autocorrelation is and the lower the frequencies where\npower is reduced. This agrees with simulation results. Notice that the autocorrelation and PSD for\nVr = 1.59 V show very close similarity to the simulation results in Fig. 4. Normally redundant\ninformation is represented by positive autocorrelation in the time domain, which is characterized by\npower at low frequencies. By reducing the low frequency component of the spike train, redundant\ninformation is suppressed and overall information transmission ef\ufb01ciency is improved. If the nega-\ntive autocorrelation of the synaptic dynamics matches the positive autocorrelation in the input spike\ntrain, the redundancy is cancelled and the output is uncorrelated [5].\n\nFigure 5: Autocorrelation of output spike trains from the VLSI stochastic synapse with STD for\nan input spike rate of 100 Hz. Autocorrelation at zero time represents the sequence variance, and\nnegative autocorrelation at short time intervals indicates STD.\n\nFigure 6: Power spectral density of output spike trains from the VLSI stochastic synapse with STD\nfor an input spike rate of 100 Hz. Lower PSD at low frequencies indicates STD.\n\nWe collect output spikes in response to 104 input spikes at input spike rates from 100 Hz to 1000\nHz with 100 Hz intervals. Fig. 7(a) shows that the mean transmission probability is inversely pro-\nportional to the input spike rate for various pulse widths when the rate is high enough. This matches\nthe theoretical prediction in (6) very well. By scaling the probability with the input spike rate, the\nsynapse tends to normalize the DC component of input frequency and preserve the neuron dynamic\nrange, thus avoiding saturation due to fast \ufb01ring presynaptic neurons and retaining sensitivity to less\nfrequently \ufb01ring neurons [17]. The slope of mean probability decreases as the pulse width increases.\nSince the pulse width determines the discharging time of the capacitor at Vi\u2212, the larger the pulse\nwidth, the larger the \u2206v is and the smaller the slope is. Fig. 7(b) shows that a\u2206v\u03c4d scales linearly\nwith the pulse width. The discharging current is approximately constant, thus \u2206v is proportional to\nthe pulse width.\n\n01020304050\u22120.1\u22120.0500.050.10.150.20.25IntervalsAutocorrelationVr = 1.56 V01020304050\u22120.0200.020.040.060.080.1IntervalsAutocorrelationVr = 1.59 V01020304050\u221280\u221260\u221240\u221220020Frequency (Hz)PSD (dB)Vr = 1.56 V01020304050\u221280\u221260\u221240\u221220020Frequency (Hz)PSD (dB)Vr = 1.59 V\f(a) Mean probability as a function of input spike\nrate for pulse width Tp =10, 20, 30, 40, 50 \u00b5s.\nData were collected at input rates from 100 Hz to\n1000 Hz at 100 Hz intervals. The dotted lines show\nthe least mean square \ufb01t from 200 Hz to 1000 Hz.\n\n(b) a\u2206v\u03c4d as a function of the pulse width. The\ndotted line shows the least mean square \ufb01t, f (x) =\n0.0008x + 0.0017.\n\nFigure 7: Steady state behavior of VLSI stochastic synapse with STD for different pulse widths.\n\nWe perform the same experiments for different Vr and Vw. As Vr increases, the slope of mean\nr decreases. This is due to the increasing \u03c4d = RC,\ntransmission probability as a linear function of 1\nwhere the equivalent resistance R from the pFET increases with Vr. Fig. 8(a) shows that a\u2206v\u03c4d\nis approximately an exponential function of Vr, indicating that the equivalent R of the pFET is ap-\nproximately exponential to its gate voltage Vr. For Vw, the slope of mean transmission probability\ndecreases as Vw increases. This is due to the increasing \u2206v with Vw. Fig. 8(b) shows that a\u2206v\u03c4d is\napproximately an exponential function of Vw, indicating that the discharging current from the tran-\nsistor M6 is approximately exponential to its gate voltage Vw. This matches the I-V characteristics\nof the MOSFET in subthreshold.\n\n(a) a\u2206v\u03c4d as a function of Vr. The dotted line shows\nthe least mean square \ufb01t, f (x) = e(44.54x\u221272.87).\n\n(b) a\u2206v\u03c4d as a function of Vw.\nThe dotted\nline shows the least mean square \ufb01t, f (x) =\ne(15.47x\u22129.854).\n\nFigure 8: The effect of biases Vr and Vw on the depressing behavior.\n\n6 Conclusion\n\nWe designed and tested a VLSI stochastic synapse with short-term depression. The behavior of\nthe depressing synapse agrees with theoretical predictions and simulation. The strength and time\nduration of the depression can be tuned by the biases. The circuit is compact and consumes low\npower. It is a good candidate to bring randomness and rich dynamics into VLSI spiking neural\n\n00.0020.0040.0060.0080.0100.10.20.30.40.50.60.70.80.911/rp 10 us20 us30 us40 us50 us10203040500.010.020.030.04Pulse width (\u00b5s)a\u2206v\u22c5\u03c4d1.551.561.571.581.590.020.040.060.080.10.120.14Vr (V)a\u2206v\u22c5\u03c4d0.30.350.40.450.500.020.040.060.080.10.12Vw (V)a\u2206v\u22c5\u03c4d\fsystems, such as for rate-independent coincidence detection of Poisson spike trains. However, the\napplication of such dynamic stochastic synapses in large networks still remains a challenge.\n\nReferences\n\n[1] C. Koch, Biophysics of Computation: Information Processing in Single Neurons. New York,\n\nNY: Oxford University Press, 1999.\n\n[2] M. V. Tsodyks and H. Markram, \u201cThe neural code between neocortical pyramidal neurons\ndepends on neurotransmitter release probability,\u201d Proc. Natl. Acad. Sci. USA, vol. 94, pp. 719\u2013\n723, 1997.\n\n[3] W. Senn, H. Markram, and M. Tsodyks, \u201cAn algorithm for modifying neurotransmitter release\nprobability based on pre- and postsynaptic spike timing,\u201d Neural Computation, vol. 13, pp.\n35\u201367, 2000.\n\n[4] W. Maass and A. M. Zador, \u201cDynamic stochastic synapses as computational units,\u201d Neural\n\nComput., vol. 11, no. 4, pp. 903\u2013917, 1999.\n\n[5] M. S. Goldman, P. Maldonado, and L. F. Abbott, \u201cRedundancy reduction and sustained \ufb01ring\n\nwith stochastic depressing synapses,\u201d J. Neurosci., vol. 22, no. 2, pp. 584\u2013591, 2002.\n\n[6] W. B. Levy and R. A. Baxter, \u201cEnergy-ef\ufb01cient neuronal computation via quantal synaptic\n\nfailures,\u201d J. Neurosci., vol. 22, no. 11, pp. 4746\u20134755.\n\n[7] R. Rao, B. Olshausen, and M. Lewicki, Eds., Statistical Theories of the Brain. MIT Press,\n\n2001.\n\n[8] C. Diorio, P. Hasler, B. A. Minch, and C. Mead, \u201cA single-transistor silicon synapse,\u201d IEEE\n\nTrans. Electron Devices, vol. 43, pp. 1972\u20131980, Nov. 1996.\n\n[9] P. H\u00a8a\ufb02iger and M. Mahowald, \u201cSpike based normalizing Hebbian learning in an analog VLSI\narti\ufb01cial neuron,\u201d Int. J. Analog Integr. Circuits Signal Process., vol. 18, no. 2-3, pp. 133\u2013139,\n1999.\n\n[10] S.-C. Liu, \u201cAnalog VLSI circuits for short-term dynamic synapses,\u201d EURASIP Journal on\n\nApplied Signal Processing, vol. 2003, pp. 620\u2013628, 2003.\n\n[11] E. Chicca, G. Indiveri, and R. Douglas, \u201cAn adaptive silicon synapse,\u201d in Proc. IEEE Int. Symp.\n\nCircuits Systems, vol. 1, Bangkok, Thailand, May 2003, pp. 81\u201384.\n\n[12] A. Bo\ufb01ll, A. F. Murray, and D. P. Thompson, \u201cCircuits for VLSI implementation of temporally\nasymmetric Hebbian learning,\u201d in Advances in Neural Information Processing Systems, S. B.\nT. G. Dietterich and Z. Ghahramani, Eds. Cambridge, MA, USA: MIT Press, 2002.\n\n[13] G. Indiveri, E. Chicca, and R. Douglas, \u201cA VLSI array of low-power spiking neurons and\nbistable synapses with spike-timing dependent plasticity,\u201d IEEE Trans. Neural Networks,\nvol. 17, pp. 211\u2013221, 2006.\n\n[14] C. S. Petrie and J. A. Connelly, \u201cA noise-based IC random number generator for applications\n\nin cryptography,\u201d IEEE Trans. Circuits Syst. I, vol. 47, no. 5, pp. 615\u2013621, May 2000.\n\n[15] D. H. Goldberg, G. Cauwenberghs, and A. G. Andreou, \u201cProbabilistic synaptic weighting in\na recon\ufb01gurable network of VLSI integrate-and-\ufb01re neurons,\u201d Neural Networks, vol. 14, pp.\n781\u2013793, 2001.\n\n[16] S. Fusi, M. Annunziato, D. Badoni, A. Salamon, and D. J. Amit, \u201cSpike driven synaptic plastic-\nity: theory, simulation, VLSI implementation,\u201d Neural Computation, vol. 12, pp. 2227\u20132258,\n2000.\n\n[17] L. F. Abbott, J. A. Varela, K. Sen, and S. B. Nelson, \u201cSynaptic depression and cortical gain\n\ncontrol,\u201d Science, vol. 275, pp. 220\u2013224, 1997.\n\n[18] F. S. Chance, S. B. Nelson, and L. F. Abbott, \u201cSynaptic depression and the temporal response\n\ncharacteristics of V1 cells,\u201d J. Neurosci., vol. 18, no. 12, pp. 4785\u20134799, 1998.\n\n[19] F. Nadim and Y. Manor, \u201cThe role of short-term synaptic dynamics in motor control,\u201d Curr.\n\nOpin. Neurobiol., vol. 10, pp. 683\u2013690, Dec. 2000.\n\n[20] R. S. Zucker, \u201cShort-term synaptic plasticity,\u201d Ann. Rev. Neurosci., vol. 12, pp. 13\u201331, 1989.\n\n\f", "award": [], "sourceid": 365, "authors": [{"given_name": "Peng", "family_name": "Xu", "institution": null}, {"given_name": "Timothy", "family_name": "Horiuchi", "institution": null}, {"given_name": "Pamela", "family_name": "Abshire", "institution": null}]}