{"title": "Learning to classify complex patterns using a VLSI network of spiking neurons", "book": "Advances in Neural Information Processing Systems", "page_first": 1009, "page_last": 1016, "abstract": null, "full_text": "Learning to classify complex patterns using a VLSI\n\nnetwork of spiking neurons\n\nSrinjoy Mitra\u2020, Giacomo Indiveri\u2020 and Stefano Fusi \u2020\u2207\n\n\u2207Center for Theoretical Neuroscience, Columbia University, New York\n\n\u2020Institute of Neuroinformatics, UZH|ETH, Zurich\nsrinjoy|giacomo|fusi@ini.phys.ethz.ch\n\nAbstract\n\nWe propose a compact, low power VLSI network of spiking neurons which can\nlearn to classify complex patterns of mean \ufb01ring rates on\u2013line and in real\u2013time.\nThe network of integrate-and-\ufb01re neurons is connected by bistable synapses that\ncan change their weight using a local spike\u2013based plasticity mechanism. Learning\nis supervised by a teacher which provides an extra input to the output neurons\nduring training. The synaptic weights are updated only if the current generated\nby the plastic synapses does not match the output desired by the teacher (as in\nthe perceptron learning rule). We present experimental results that demonstrate\nhow this VLSI network is able to robustly classify uncorrelated linearly separable\nspatial patterns of mean \ufb01ring rates.\n\n1 Introduction\n\nSpike driven synaptic plasticity mechanisms have been thoroughly investigated in recent years to\nsolve two important problems of learning: 1) how to modify the synapses in order to generate\nnew memories 2) how to protect old memories against the passage of time, and the overwriting of\nnew memories by ongoing activity. Temporal patterns of spikes can be encoded with spike-timing\ndependent plasticity (STDP) mechanisms (e.g. see [1, 2]). However, STDP in its simplest form is\nnot suitable for learning patterns of mean \ufb01ring rates [3], and most of the proposed STDP learning\nalgorithms solved the problems of memory encoding and memory preservation only for relatively\nsimple patterns of mean \ufb01ring rates.\n\nRecently a new model of stochastic spike-driven synaptic plasticity has been proposed [4] that is\nvery effective in protecting old learned memories, and captures the rich phenomenology observed\nin neurophysiological experiments on synaptic plasticity, including STDP protocols. It has been\nshown that networks of spiking neurons that use this synaptic plasticity model can learn to classify\ncomplex patterns of spike trains ranging from stimuli generated by auditory/vision sensors to im-\nages of handwritten digits from the MNIST database [4]. Here we describe a neuromorphic VLSI\nimplementation of this spike-driven synaptic plasticity model and present classi\ufb01cation experiments\nusing the VLSI device that validate the model\u2019s implementation. The silicon neurons and synapses\ninside the chip are implemented using full custom hybrid analog/digital circuits, and the network\u2019s\nspikes are received in input and transmitted in output using asynchronous digital circuits. Each\nspike is represented as an Address-Event, where the address encodes either the source neuron or the\ndestination synapse. This device is part of an increasing collection of spike-based computing chips\nthat have been recently developed within the framework of Address-Event Representation (AER)\nsystems [5, 6]. There are even multiple implementations of the same spike-driven plasticity model\nbeing investigated in parallel [7, 8]. The focus of this paper is to show that the VLSI device pro-\nposed here can successfully classify complex patterns of spike trains, producing results that are in\naccordance with the theoretical predictions.\n\n1\n\n\fFigure 1: Layout of a test chip comprising a network of I&F neurons and plastic synapses. The\nplacement of a single neuron along with its synapses is highlighted in the top part of the \ufb01gure.\nOther highlighted circuits are described in the test.\n\nIn Section 2 we describe the main features of the spike-based plasticity model and show how they are\nwell suited for future scaled CMOS VLSI technologies; in Section 3 we characterize the function-\nality of the spike-based learning circuits; in Section 4 we show control experiments on the learning\nproperties of the VLSI network; and in Section 5 we present experimental results on complex pat-\nterns of mean \ufb01ring rates. In Section 6 we present the concluding remarks and point out future\noutlooks and potential applications of this system.\n\n2 Implementation of the spike-based plasticity mechanism\n\nPhysical implementations of long lasting memories, either biological or electronic, are confronted\nwith two hard limits: the synaptic weights are bounded (they cannot grow inde\ufb01nitely or become\nnegative), and the resolution of the synapse is limited (i.e. the synaptic weight cannot have an in\ufb01nite\nnumber of states). These constraints, usually ignored by the vast majority of software models, have\nstrong impact on the classi\ufb01cation performance of the network, and on its memory storage capacity.\nIt has been demonstrated that the number of random uncorrelated patterns p which can be classi\ufb01ed\nor stored in a network of neurons connected by bounded synapses grows only logarithmically with\nthe number of synapses [9]. In addition, if each synapse has a n stable states (i.e. its weight has to\ntraverse n states to go from the lower bound to the upped bound), then the number of patterns p can\ngrow quadratically n. However, this can happen only in unrealistic scenarios, where \ufb01ne tuning of\nthe network\u2019s parameters is allowed. In more realistic scenarios where there are inhomogeneities\nand variability (as is the case for biology and silicon) p is largely independent of n [9].\n\nTherefore, an ef\ufb01cient strategy for implementing long lasting memories in VLSI networks of spiking\nneurons is to use a large number of synapses with only two stable states (i.e. n = 2), and to modify\ntheir weights in a stochastic manner, with a small probability. This slows down the learning process,\nbut has the positive effect of protecting previously stored memories from being overwritten. Using\nthis strategy we can build large networks of spiking neurons with very compact learning circuits\n(e.g. that do not require local Analog-to-Digital Converters or \ufb02oating gate cells for storing weight\nvalues). By construction, these types of devices operate in a massively parallel fashion and are fault-\ntolerant: even if a considerable fraction of the synaptic circuits is faulty due to fabrication problems,\nthe overall functionality of the chip is not compromised. This can be a very favorable property in\nview of the potential problems of future scaled VLSI processes.\n\nThe VLSI test chip used to carry out classi\ufb01cation experiments implementing such strategy is shown\nin Fig. 1. The chip comprises 16 low-power integrate-and-\ufb01re (I&F) neurons [5] and 2048 dynamic\nsynapses. It was fabricated using a standard 0.35\u00b5m CMOS technology, and occupies an area of\n6.1mm2 . We use an AER communication infrastructure that allows the chip to receive and transmit\nasynchronous events (spikes) off-chip to a workstation (for data logging and prototyping) and/or to\nother neuromorphic event-based devices [10]. An on-chip multiplexer can be used to recon\ufb01gure\nthe neuron\u2019s internal dendritic tree connectivity. A single neuron can be connected to 128, 256, 512\nor 1024 synapses. Depending on the multiplexer state the number of active neurons decrease from\n16 to 2. In this work we con\ufb01gured the chip to use all 16 neurons with 128 synapses per neuron.\nThe synapses are divided into different functional blocks: 4 are excitatory with \ufb01xed (externally\nadjustable) weights, 4 inhibitory and 120 excitatory with local learning circuits.\n\nEvery silicon neuron in the chip can be used as a classi\ufb01er that separates the input patterns into two\ncategories. During training, the patterns to be classi\ufb01ed are presented to the pre-synaptic synapses,\n\n2\n\n\fsynapses\n\nbistable\n\npre\n\nAER\ninput\n\nS1\n\nw\n\nS2\n\nDPI\nIEPSC\n\n(a)\n\nVUP\n\nVDN\n\nVmem\n\nSoma\n\nVmth\n\n+\n\nVmem\n\nVUP\n\nIUP\n\nCC2\n\nIk1\nIB\n\nCC1\n\nIk2\n\nIk3\n\nCC3\n\nIDN\n\nVDN\n\nDPI\n\nVspk\n\nI [Ca]\n\nStop Learning\n\nI&F block\n\n(b)\n\nFigure 2: (a) Plastic synapse circuits belonging to the neuron\u2019s dendritic tree. The synaptic weight\nnode w is modi\ufb01ed when there is a pre-synaptic input (i.e. when S1 and S2 are on) depending on\nthe values of VU P and VDN . In parallel, the bistable circuit slowly drives the node w toward either of\nits two stable states depending on its amplitude. The DPI is a pulse integrator circuit that produces\nan Excitatory Post-Synaptic Current (IEPSC), with an amplitude that depends on the synaptic weight\nw. (b) Neuron\u2019s \u201csoma\u201d block diagram with stop-learning module. It comprises a low-power I&F\nneuron block, a DPI integrator, a voltage comparator and a three current comparators(CC). Winner-\ntake-all (WTA) circuits are used as current comparators that set the output to be either the bias\ncurrent IB, or zero. The voltage comparator enables either the IU P or the IDN block, depending on\nthe value of Vmem with respect to Vmth. The voltages VU P and VDN are used to broadcast the values\nof IU P and IDN to the neuron\u2019s dendritic tree.\n\nin parallel with a teacher signal that represents the desired response. The post-synaptic neuron\nresponds with an activity that is proportional to its net input current, generated by the input pattern\nweighted by the learned synaptic ef\ufb01cacies, and by the teacher signal. If the neuron\u2019s mean activity is\nin accordance with the teacher signal (typically either very high or very low), then the output neuron\nproduces the correct response. In this case the the synapses should not be updated. Otherwise, the\nsynapses are updated at the time of arrival of the (Poisson distributed) input spikes, and eventually\nmake a transition to one of the two stable states. Such stochasticity, in addition to the \u2019stop-learning\u2019\nmechanism which prevents the synapses from being modi\ufb01ed when the output is correct, allows\neach neuron to classify a wide class of highly correlated, linearly separable patterns. Furthermore,\nby using more than one neuron per class, it is possible to classify also complex non-linearly separable\npatterns [4].\n\n3 The VLSI learning circuits\n\nThe learning circuits are responsible for locally updating the synaptic weights with the spike-based\nlearning rule proposed in [4].\n\nUpon the arrival of a pre-synaptic spike (an address-event), the plastic synapse circuit updates its\nweight w according to the spike-driven learning rule. The synapse then produces an Excitatory\nPost-Synaptic Current (EPSC) with an amplitude proportional to its weight, and with an exponential\ntime course that can be set to last from microseconds to several hundreds of milliseconds [11]. The\nEPSC currents of all synapses afferent to the target neuron are summed into the neuron\u2019s membrane\ncapacitance, and eventually the I&F neuron\u2019s membrane potential exceeds a threshold and the circuit\ngenerates an output spike. As prescribed by the model of [4], the post-synaptic neuron\u2019s membrane\npotential, together with its mean \ufb01ring rate are used to determine the weight change values \u2206w.\nThese weight change values are expressed in the chip as subthreshold currents. Speci\ufb01cally, the\nsignal that triggers positive weight updates is represented by an IU P current, and the signal that\ntriggers weight decreases if represented by the IDN current.\n\nThe weight updates are performed locally at each synapse, in a pre-synaptic weight update module,\nwhile the \u2206w values are computed globally (for each neuron), in a post-synaptic weight control\nmodule.\n\n3\n\n\fa\nC\n\nV\n\nN\nD\n\nV\n\n3.2\n3\n2.8\n2.6\n2.4\n0\n3.2\n3\n2.8\n2.6\n0\n0.4\n\nP\nU\n\nV\n\n0.2\n\n0\n0\n\n1\n\n1\n\n1\n\n0.5\n\n0.5\n\n0.5\n\nTime (s)\n\n(a)\n\n1.5\n\n1.5\n\n1.5\n\n1\n\nm\ne\nm\n\nV\n\n0\n0\n\n3\n\nN\nD\n\nV\n\n2.8\n\n2.6\n0\n0.4\n\nP\nU\n\nV\n\n0.2\n\n0.01\n\n0.02\n\n0.03\n\n0.04\n\n0.05\n\n0.01\n\n0.02\n\n0.03\n\n0.04\n\n0.05\n\n0\n0\n\n0.01\n\n0.02\n\nTime (s)\n\n0.03\n\n0.04\n\n0.05\n\n(b)\n\nFigure 3: Post-synaptic circuit data. (a) State of the VU P and VDN voltages as a function of the cal-\ncium concentration voltage VCa. (b) State of the VU P and VDN voltages as function of the membrane\npotential Vmem. This data corresponds to a zoomed-version of the data shown in (a) for VCa \u2248 2.8V .\n\n3.1 Pre-synaptic weight-update module\n\nThis module, shown in Fig. 2(a), comprises four main blocks: an input AER interfacing circuit [12],\na bistable weight refresh circuit, a weight update circuit and a log-domain current-mode integrator,\ndubbed the \u201cdiff-pair integrator\u201d (DPI) circuit, and fully characterized in [11]. Upon the arrival of an\ninput event (pre-synaptic spike), the asynchronous AER interfacing circuits produce output pulses\nthat activate switches S1 and S2. Depending on the values of IU P and IDN , mirrored from the post-\nsynaptic weight control module, the node w charges up, discharge toward ground, or does not get\nupdated. The same input event activates the DPI circuit that produces an EPSC current (IEPSC) with\nan amplitude that depends on the synaptic weight value w. In parallel, the bistable weight refresh\ncircuit slowly drives w toward one of two stable states depending on whether it is higher or lower\nthan a set threshold value. The two stable states are global analog parameters, set by external bias\nvoltages.\n\n3.2 Post-synaptic weight control module\n\nThis module is responsible for generating the two global signals VU P and VDN , mirrored to all\nsynapses belonging to the same dendritic tree. Post-synaptic spikes (Vspk), generated in the soma\nare integrated by an other instance of the DPI circuit to produce a current ICa proportional to the\nneuron\u2019s average spiking activity. This current is compared to three threshold values, Ik1, Ik2, and\nIk3 of Fig. 2(b), using three current-mode winner-take-all circuits [13]. In parallel, the instantaneous\nvalue of the neuron\u2019s membrane potential Vmem is compared to the threshold Vmth (see Fig. 2(b)).\nThe values of IU P and IDN depend on the state of the neuron\u2019s membrane potential and its average\nfrequency. Speci\ufb01cally, if Ik1 < ICa < Ik3 and Vmem > Vmth, then IU P = IB. If Ik1 < ICa < Ik2 and\nVmem < Vmth, then IDN = IB. Otherwise both IU P, and IDN are null.\n\nTo characterize these circuits we injected a step current in the neuron, produced a regular output\nmean \ufb01ring rate, and measured the voltages VCa, VU P, and VDN (see Fig. 3(a)). VCa is the gate volt-\nage of the P-FET transistor producing ICa, while VDN , VU P are the gate voltages of the P- and N-FET\ntransistors mirroring IDN and IU P respectively (Fig. 2(a)). The neuron\u2019s spikes are integrated and\nthe output current ICa increases with an exponential pro\ufb01le over time (VCa decreases accordingly\nover time, as shown in Fig. 3(a)). The steady-state asymptotic value depends on the average input\nfrequency, as well as the circuit\u2019s bias parameters [11]. As ICa becomes larger than the \ufb01rst thresh-\nold Ik1 (VCa decreases below the corresponding threshold voltage) both VU P and VDN are activated.\nWhen ICa becomes larger than the second threshold Ik2 the VDN signal is deactivated, and \ufb01nally\nas ICa becomes larger than the third threshold Ik3, also the VU P signal is switched off. The small \u223c\n300mV changes in VU P and VDN produce subthreshold currents (IU P and IDN ) that are mirrored to the\nsynapses (Fig. 2(a)). In Fig. 3(b) the VDN and VU P signals are zoomed in along with the membrane\npotential of the post-synaptic neuron (Vmem), for values of VCa \u223c 2.8V . Depending on the state of\n\n4\n\n\fm\ne\nm\n\nV\n\nw\n\nV\n\n1.5\n1\n0.5\n0\n0\n3\n2.5\n2\n1.5\n0\n\n2\n\ne\nr\np\n\n0\n0\n\n0.05\n\n0.1\n\n0.15\n\n0.2\n\n0.25\n\n0.05\n\n0.1\n\n0.15\n\n0.2\n\n0.25\n\n0.05\n\n0.1\n\n0.15\n\n0.2\n\n0.25\n\nTime(s)\n\n(a)\n\nm\ne\nm\n\nV\n\nw\n\nV\n\n1.5\n1\n0.5\n0\n0\n3\n2.5\n2\n1.5\n0\n\n2\n\ne\nr\np\n\n0\n0\n\n0.05\n\n0.1\n\n0.15\n\n0.2\n\n0.25\n\n0.05\n\n0.1\n\n0.15\n\n0.2\n\n0.25\n\n0.05\n\n0.1\n\n0.15\n\n0.2\n\n0.25\n\nTime(s)\n\n(b)\n\nFigure 4: Stochastic synaptic LTP transition: in both sub-\ufb01gures the non-plastic synapse is stim-\nulated with Poisson distributed spikes at a rate of 250Hz, making the post-synaptic neuron \ufb01re at\napproximately 80Hz; and the plastic synapse is stimulated with Poisson distributed spike trains of\n100Hz. (a) The updates in the synaptic weight did not produce any LTP transition during the 250ms\nstimulus presentation. (b) The updates in the synaptic weight produced an LTP transition that re-\nmains consolidated.\n\nVmem, the signals VU P and VDN are activated or inactivated. When not null, currents IU P and IDN are\ncomplementary in nature: only one of the two is equal to IB.\n\n4 Stochastic plasticity\n\nTo characterize the stochastic nature of the weight update process we stimulated the neuron\u2019s plastic\nsynapses with Poisson distributed spike trains. When any irregular spike train is used as a pre-\nsynaptic input, the synaptic weight voltage crosses the synapse bistability threshold in a stochastic\nmanner, and the probability of crossing the threshold depends on the input\u2019s mean frequency. There-\nfore Long Term Potentiation (LTP) or Long Term Depression (LTD) occur stochastically even when\nthe mean \ufb01ring rates of the input and the output are always the same. In Fig. 4 we show two in-\nstances of a learning experiment in which the mean input \ufb01ring rate (bottom row) was 100Hz, and\nthe mean output \ufb01ring rate (top row) was 80Hz. Although these frequencies were the same for both\nexperiments, LTP occurred only in one of the two cases (compare synaptic weight changes in middle\nrow of both panels). In this experiment we set the ef\ufb01cacy of the \u201chigh\u201d state of all plastic synapses\nto a relatively low value. In this way the neuron\u2019s mean output \ufb01ring rate depends primarily on the\nteacher signal, irrespective of the states of plastic synapses.\n\nOne essential feature of this learning rule is the non-monotonicity of both the LTP/LTD probabilities\nas a function of the post-synaptic \ufb01ring frequency \u03bdpost [4]. Such a non-monotonicity is essential\nto slow down and eventually stop-learning when \u03bdpost is very high or very low (indicating that\nthe learned synaptic weights are already correctly classifying the input pattern). In Fig. 5 we show\nexperimental results where we measured the LTP and LTD transitions of 60 synapses over 20 training\nsessions: for the LTD case (top row) we initialized the synapses to a high state (white pixel) and\nplotted a black pixel if its \ufb01nal state was low, at the end of the training session. The transitions\n(white to black) are random in nature and occur with a probability that \ufb01rst increases and then\ndecreases with \u03bdpost . An analogous experiment was done for the LTP transitions (bottom row), but\nwith complementary settings (the initial state was set to a low value). In Fig. 5(b) we plot the LTD\n(top row) and LTP (bottom row) probabilities measured for a single synapse. The shape of these\ncurves can be modi\ufb01ed by acting on the post-synaptic weight control module bias parameters such\nas Ik1\u2212k3, or IB.\n\n5\n\n\f5\n\n20\n\n100\n\n180\n\n320\n\n500\n\n700\n\n900\n\n1\n\n20\n\n40\n\n1\n\n20\n\n40\n\nr\ne\nb\nm\nu\nn\n\n \n\ne\ns\np\na\nn\ny\nS\n\nr\ne\nb\nm\nu\nn\n\n \n\ne\ns\np\na\nn\ny\nS\n\n1\n\n20\n\n(a)\n\n1\n\n)\n\nD\nT\nL\n(\np\n\n0.5\n\n0\n1\n\n)\n\nP\nT\nL\n(\np\n\n0.5\n\n0\n0\n\n200\n\n600\n\n400\n(Hz)\n\npost\n(b)\n\nFigure 5: (a) LTD and LTP transitions of 60 synapses measured across 20 trials, for different values\nof post-synaptic frequency \u03bdpost (top label on each panel). Each black pixel represents a low synaptic\nstate, and white pixel a high one. On x-axis of each panel we plot the trial number (1 to 20) and y-axis\nshows the state of the synapses at the end of each trial. In the top row we show the LTD transitions\nthat occur after initializing all the synapses to high state.\nIn the bottom row we show the LTP\ntransition that occur after initializing the synapses to low state. The transitions are stochastic and the\nLTP/LTD probabilities peak at different frequencies before falling down at higher \u03bdpost validating\nthe stop-learning algorithm. No data was taken for the gray panels. (b) Transition probabilities\nmeasured for a single synapse as a function \u03bdpost . The transition probabilities can be reduced by\ndecreasing the value of IB. The probability peaks can also be modi\ufb01ed by changing the biases that\nset Ik1\u2212k3. (Fig. 2(b))\n\nT+\n\nT\u2212\n\nExcitatory synapse, non\u2212plastic\n\nInhibitory synapse, non\u2212plastic\n\nExcitatory synapse, plastic\n\nHigh input state (30Hz)\nLow input state (2Hz)\nIntegrate and Fire neuron\n\nC+\n\nC\n\nFigure 6: A typical training scenario with 2 random binary spatial patterns. High and low inputs are\nencoded with generate Poisson spike trains with mean frequencies of 30Hz and 2Hz respectively.\nBinary patterns are assigned to the C+ or C\u2212 class arbitrarily. During training patterns belonging\nto the C+ class are combined with a T + (teacher) input spike train of with 250Hz mean \ufb01ring rate.\nSimilarly, patterns belonging to the C\u2212 class are combined with a T \u2212 spike train of 20Hz mean\n\ufb01ring rate. New Poisson distributed spike trains are generated for each training iterations.\n\n5 Classi\ufb01cation of random spatial patterns\n\nIn order to evaluate the chip\u2019s classi\ufb01cation ability, we used spatial binary patterns of activity, ran-\ndomly generated (see Fig. 6). The neuron\u2019s plastic synapses were stimulated with Poisson spike\ntrains of either high (30Hz) or low (2Hz) mean \ufb01ring rates. The high/low binary state of the input\nwas chosen randomly, and the number of synapses used was 60. Each 60-input binary pattern was\nthen randomly assigned to either a C+ or a C\u2212 class.\nDuring training, spatial patterns belonging to the C+ class are presented to the neuron in conjunction\nwith a T + teacher signal (i.e. a 250Hz Poisson spike train). Conversely patterns belonging to the\nC\u2212 class are combined with a T \u2212 teacher signal of 20Hz. The T + and T \u2212 spike trains are presented\nto the neuron\u2019s non-plastic synapses. Training sessions with C+ and C\u2212 patterns are interleaved\nin a random order, for 50 iterations. Each stimulus presentation lasted 500ms, with new Poisson\ndistributions generated at each training session.\n\n6\n\nn\n\fAfter training, the neuron is tested to see if it can correctly distinguish between patterns belonging\nto the two classes C+ and C\u2212. The binary patterns used during training are presented to the neuron\nwithout the teacher signal, and the neuron\u2019s mean \ufb01ring rate is measured.\nIn Fig. 7(a) we plot\nthe responses of two neurons labeled neuron-A and neuron-B. Neuron-A was trained to produce a\nhigh output \ufb01ring rate in response to patterns belonging to class C+, while neuron-B was trained to\nrespond to patterns belonging to class C\u2212. As shown, a single threshold (e.g. at 20Hz) is enough to\nclassify the output in C+ (high frequency) and C\u2212 (low frequency) class.\n\n80\n\n60\n\n40\n\n20\n\n)\nz\nH\n\n(\n\nt\ns\no\np\n\n0\n\n1\n\n2\n\n3\n\nneuron\u2212A\n\n4\n\n1\n\n(a)\n\n0.5\n\n0\n\n0.5\n\n)\n\nt\ns\no\np\n\n(n\np\n\n)\n\nt\ns\no\np\n\n(n\np\n\n4\n\n0\n\n0\n\n50\n\n150\n\n100\npost(Hz)\n(b)\n\n2\n\n3\n\nneuron\u2212B\n\nFigure 7: Classi\ufb01cation results, after training on 4 patterns. (a) Mean output frequencies of neurons\ntrained to recognize class C+ patterns (Neuron-A), and class C\u2212 patterns (Neuron-B). Patterns 1, 2\nbelong to class C+, while patterns 3, 4 belong to class C\u2212. (b) Output frequency probability distribu-\ntion, for all C+ patterns (top) and C\u2212 patterns (bottom) computed over 20 independent experiments.\n\nFig. 7(b) shows the probability distribution of post-synaptic frequencies (of neuron-A) over different\nclassi\ufb01cation experiments, each done with new sets of random spatial patterns.\n\nTo quantify the chip\u2019s classi\ufb01cation behavior statistically, we employed a Receiver Operating Char-\nacteristics (ROC) analysis [14]. Figure 8(a) shows the area under the ROC curve (AUC) plotted on\ny-axis for increasing number of patterns. An AUC magnitude of 1 represents 100% correct classi-\n\ufb01cation while 0.5 represents chance level. In Fig. 8(b) the storage capacity (p) \u2013expressed as the\nnumber of patterns with AUC larger than 0.75\u2013 is plotted against the number of synapses N. The top\nand bottom traces show the theoretical predictions from [3], with (p\u221d 2\u221aN) and without (p\u221d \u221aN)\nthe stop learning condition, respectively. The performance of the VLSI system with 20, 40 and 60\nsynapses and the stop-learning condition lie within the two theoretical curves.\n\n6 Conclusions\n\nWe implemented in a neuromorphic VLSI device a recently proposed spike-driven synaptic plastic-\nity model that can classify complex patterns of spike trains [4]. We presented results from the VLSI\nchip that demonstrate the correct functionality of the spike-based learning circuits, and performed\nclassi\ufb01cation experiments of random uncorrelated binary patterns, that con\ufb01rm the theoretical pre-\ndictions. Additional experiments have demonstrated that the chip can be applied to the classi\ufb01cation\nof correlated spatial patterns of mean \ufb01ring rates and as well [15]. To our knowledge, the classi\ufb01-\ncation performance achieved with this chip has not yet been reported for any other silicon system.\nThese results show that the device tested can perform real-time classi\ufb01cation of sequences of spikes,\nand is therefore an ideal computational block for adaptive neuromorphic sensory-motor systems and\nbrain-machine interfaces.\n\nAcknowledgment\n\nThis work was supported by the Swiss National Science Foundation grant no. PP00A106556, the\nETH grant no. TH02017404, and by the EU grants ALAVLSI (IST-2001-38099) and DAISY (FP6-\n2005-015803).\n\n7\n\nn\nn\n\f1\n\n0.9\n\n0.8\n\n0.7\n\n0.6\n\nC\nU\nA\n\n0.5\n2\n\n4\n\n10\n\n12\n\n6\n8\n# patterns\n\n(a)\n\ny\nt\ni\nc\na\np\na\nc\n \n\ne\ng\na\nr\no\nS\n\nt\n\n15\n\n10\n\n5\n\n0\n\n20\n\n40\n\n# input synapses\n\n60\n\n(b)\n\nFigure 8: (a). Area under ROC curve (AUC) measured by performing 50 classi\ufb01cation experiments.\n(b) Storage capacity (number of patterns with AUC value \u2265 0.75) as a function of the number of\nplastic synapses used. The solid line represents the data obtained from chip, while top and bottom\ntraces represent the theoretical predictions with and without the stop learning condition.\n\nReferences\n\n[1] R. G\u00a8utig and H. Sompolinsky. The tempotron: a neuron that learns spike timing\u2013based decisions. Nature\n\nNeuroscience, 9:420\u2013428, 2006.\n\n[2] R.A. Legenstein, C. N\u00a8ager, and W. Maass. What can a neuron learn with spike-timing-dependent plastic-\n\nity? Neural Computation, 17(11):2337\u20132382, 2005.\n\n[3] S. Fusi and W. Senn. Eluding oblivion with smart stochastic selection of synaptic updates. Chaos, An\n\nInterdisciplinary Journal of Nonlinear Science, 16(026112):1\u201311, 2006.\n\n[4] J. Brader, W. Senn, and S. Fusi. Learning real world stimuli in a neural network with spike-driven synaptic\n\ndynamics. Neural Computation, 2007. (In press).\n\n[5] G. Indiveri, E. Chicca, and R. Douglas. A VLSI array of low-power spiking neurons and bistable synapses\nIEEE Transactions on Neural Networks, 17(1):211\u2013221, Jan\n\nwith spike\u2013timing dependent plasticity.\n2006.\n\n[6] J. Arthur and K. Boahen. Learning in silicon: Timing is everything.\n\nIn Y. Weiss, B. Sch\u00a8olkopf, and\nJ. Platt, editors, Advances in Neural Information Processing Systems 18. MIT Press, Cambridge, MA,\n2006.\n\n[7] D. Badoni, M. Giulioni, V. Dante, and P. Del Giudice. An aVLSI recurrent network of spiking neurons\nwith recon\ufb01gurable and plastic synapses. In Proceedings of the IEEE International Symposium on Circuits\nand Systems, pages 1227\u20131230. IEEE, IEEE, May 2006.\n\n[8] G. Indiveri and S. Fusi. Spike-based learning in VLSI networks of integrate-and-\ufb01re neurons. In Proc.\n\nIEEE International Symposium on Circuits and Systems, ISCAS 2007, pages 3371\u20133374, 2007.\n\n[9] S. Fusi and L. F. Abbott. Limits on the memory storage capacity of bounded synapses. Nature Neuro-\n\nscience, 10:485\u2013493, 2007.\n\n[10] E. Chicca, P. Lichtsteiner, T. Delbr\u00a8uck, G. Indiveri, and R.J. Douglas. Modeling orientation selectivity\nusing a neuromorphic multi-chip system. In Proceedings of IEEE International Symposium on Circuits\nand Systems, pages 1235\u20131238. IEEE, 2006.\n\n[11] C. Bartolozzi and G. Indiveri. Synaptic dynamics in analog VLSI. Neural Computation, 19:2581\u20132603,\n\nOct 2007.\n\n[12] K. A. Boahen. Point-to-point connectivity between neuromorphic chips using address-events.\n\nIEEE\n\nTransactions on Circuits and Systems II, 47(5):416\u201334, 2000.\n\n[13] J. Lazzaro, S. Ryckebusch, M.A. Mahowald, and C.A. Mead. Winner-take-all networks of O(n) com-\nplexity. In D.S. Touretzky, editor, Advances in neural information processing systems, volume 2, pages\n703\u2013711, San Mateo - CA, 1989. Morgan Kaufmann.\n\n[14] T. Fawcett. An introduction to ROC analysis. Pattern Recognition Letters, (26):861\u2013874, 2006.\n\n[15] S. Mitra, G. Indiveri, and S. Fusi. Robust classi\ufb01cation of correlated patterns with a neuromorphic VLSI\nnetwork of spiking neurons. In IEEE Proceedings on Biomedical Circuits and Systems (BioCAS08), 2008.\n(In press).\n\n8\n\n\f", "award": [], "sourceid": 602, "authors": [{"given_name": "Srinjoy", "family_name": "Mitra", "institution": null}, {"given_name": "Giacomo", "family_name": "Indiveri", "institution": null}, {"given_name": "Stefano", "family_name": "Fusi", "institution": null}]}