{"title": "Model of a Biological Neuron as a Temporal Neural Network", "book": "Advances in Neural Information Processing Systems", "page_first": 85, "page_last": 91, "abstract": "", "full_text": "Model of a Biological Neuron as a Temporal \n\nNeural Network \n\nSean D. Murphy and Edward W. Kairiss \n\nInterdepartmental Neuroscience Program, Department of Psychology, \n\nand The Center for Theoretical and Applied Neuroscience, \n\nYale University, \n\nBox 208205, New Haven, CT 06520 \n\nAbstract \n\nA biological neuron can be viewed as a device that maps a multidimen(cid:173)\nsional temporal event signal (dendritic postsynaptic activations) into a \nunidimensional temporal event signal (action potentials). We have \ndesigned a network, the Spatio-Temporal Event Mapping (STEM) \narchitecture, which can learn to perform this mapping for arbitrary bio(cid:173)\nphysical models of neurons. Such a network appropriately trained, \ncalled a STEM cell, can be used in place of a conventional compartmen(cid:173)\ntal model in simulations where only the transfer function is important, \nsuch as network simulations. The STEM cell offers advantages over \ncompartmental models in terms of computational efficiency, analytical \ntractabili1ty, and as a framework for VLSI implementations of biologi(cid:173)\ncal neurons. \n\n1 INTRODUCTION \nDiscovery of the mechanisms by which the mammalian cerebral cortex processes and \nstores information is the greatest remaining challenge in the brain sciences. Numerous \nmodeling studies have attempted to describe cortical information processing in frame(cid:173)\nworks as varied as holography, statistical physics, mass action, and nonlinear dynamics. \nYet, despite these theoretical studies and extensive experimental efforts, the functional \narchitecture of the cortex and its implementation by cortical neurons are largely a mystery. \n\nOur view is that the most promising approach involves the study of computational models \nwith the following key properties: (1) Networks consist of large (> 103 ) numbers of neu(cid:173)\nrons; (2) neurons are connected by modifiable synapses; and (3) the neurons themselves \npossess biologically-realistic dynamics. \nProperty (1) arises from extensive experimental observations that information processing \nand storage is distributed over many neurons. Cortical networks are also characterized by \nsparse connectivity: the probability that any two local cortical neurons are synaptically \nconnected is typically less than 0.1. These and other observations suggest that key features \nof cortical dynamics may not be apparent unless large, sparsely-connected networks are \nstudied. \n\nProperty (2) is suggested by the accumulated evidence that (a) memory formation is sub(cid:173)\nserved by use-dependent synaptic modification, and (b) Hebbian synaptic plasticity is \npresent in many areas of the brain thought to be important for memory. It is also well \nknown that artificial networks composed of elements that are connected by Hebb-like syn(cid:173)\napses have powerful computational properties. \n\n\f86 \n\nSean D. Murphy, Edward W. Kairiss \n\nProperty (3) is based on the assumption that biological neurons are computationally more \ncomplex than. for example. the processing elements that compose artificial (connectionist) \nneural networks. Although it has been difficult to infer the computational function of cor(cid:173)\ntical neurons directly from experimental data, models of neurons that explicitly incorpo(cid:173)\nrate biophysical components (e.g. neuronal geometry, channel kinetics) suggest a \ncomplex, highly non-linear dynamical transfer function. Since the \"testability\" of a model \ndepends on the ability to make predictions in terms of empirically measurable single-neu(cid:173)\nron firing behavior, a biologically-realistic nodal element is necessary in the network \nmodel. \nBiological network models with the above properties (e.g. Wilson & Bower, 1992; Traub \nand Wong, 1992) have been handicapped by the computationally expensive single-neuron \nrepresentation. These \"compartmental\" models incorporate the neuron's morphology and \nmembrane biophysics as a large (102 _104) set of coupled, non-linear differential equa(cid:173)\ntions. The resulting system is often stiff and requires higher-order numerical methods and \nsmall time-steps for accurate solution. Although the result is a realistic approximation of \nneuronal dynamics, the computational burden precludes exhaustive study of large net(cid:173)\nworks for functionality such as learning and memory. \n\nThe present study is an effort to develop a computationally efficient representation of a \nsingle neuron that does not compromise the biological dynamical behavior. We take the \nposition that the \"dynamical transfer function\" of a neuron is essential to its computational \nabstraction, but that the underlying molecular implementation need not be explicitly repre(cid:173)\nsented unless it is a target of analysis. We propose that a biological neuron can be viewed \nas a device that performs a mapping from multidimensional spatio-temporal (synaptic) \nevents to unidimensional temporal events (action potentials). This computational abstrac(cid:173)\ntion will be called a Spatio-Temporal Event Mapping (STEM) cell. We describe the archi(cid:173)\ntecture of the neural net that implements the neural transfer function, and the training \nprocedure required to develop realistic dynamics. Finally, we discuss our preliminary \nanalyses of the performance of the model when compared with the full biophysical repre(cid:173)\nsentation. \n\n2 STEM ARCHITECTURE \n\nThe architecture of the STEM cell is similar to that found in neural nets for temporal \nsequence processing (e.g. review by Mozer, in press). In general, these networks have 2 \ncomponents: (1) a short-term memory mechanism that acts as a preprocessor for (2) a non(cid:173)\nlinear feedforward network. For example, de Vries & Principe (1992) describe the utility \nof the gamma net, a real-time neural net for temporal processing, in time series prediction. \nThe preprocessor in the gamma net is the gamma memory structure, implemented as a net(cid:173)\nwork of adaptive dispersive elements (de Vries & Principe, 1991). The preprocessor in our \nmodel (the \"tau layer\", described below) is somewhat simpler, and is inspired by the tem(cid:173)\nporal dynamics of membrane conductances found in biological neurons. \n\nThe STEM architecture (diagrammed in Figure 1) works by building up a vectorial repre(cid:173)\nsentation of the state of the neuron as it continuously receives incoming synaptic activa(cid:173)\ntions, and then labeling that vector space as either \"FIRE\" or \"DON'T FIRE\". This is \naccomplished with the use of four major components: (1) TAU LAYER: a layer of nodes \nthat continuously maps incoming synaptic activations into a finite-dimensional vector \nspace (2) FEEDBACK TAU NODE: a node that maintains a vectorial representation of \nthe past activity of the cell itself (3) MLP: a multilayer perceptron that functions as a non(cid:173)\nlinear spatial mapping network that performs the \"FIRE\" / \"NO-FIRE\" labeling on the tau \nlayer output (4) OUTPUT FILTER: this adds a refractory period and threshold to the MLP \noutput that contrains the format of the output to be discrete-time events. \n\n\fModel of Biological Neuron as a Temporal Neural Network \n\n87 \n\ninput (spike train at each synapse) \n\nI spatio-temporalto spatial projection layer ! _ \n\nl nonlinear spatial mapping network I \n\n- spike train output \n\n\" V \nu \n\nQ \n\nI output processor : \n\nFigure 1: Information Flow in The STEM Cell \n\nThe tau layer (Fig. 2) consists of N + 1 tau nodes, where N is the number of synapses on \nthe cell, and the extra node is used for feedback. Each tau node consists of M tau units. \nEach tau unit within a single tau node receives an identical input signal. Each tau unit \nwithin a tau node calculates a second-order rise-and-decay function with unique time con(cid:173)\nstants. The tau units within a tau node translate arbitrary temporal events into a vector \nform, with each tau-unit corresponding to a different vector component. Taken as a whole, \nall of the tau unit outputs of the tau node layer comprise a high-dimensional vector that \nrepresents the overall state of the neuron. Functionally, the tau layer approximates a one(cid:173)\nto-one mapping between the spatio-temporal input and the tau-unit vector space. \n\nThe output of each tau unit in the tau layer is fed into the input layer of a multilayer per(cid:173)\nceptron (MLP) which, as will be explained in the next section, has been trained to label the \ntau-layer vector as either FIRE or NO-FIRE. The output of the MLP is then fed into an \noutput filter with a refractory period and threshold. The STEM architecture is illustrated in \nFig. 3. \n\n(A) \n\n(afferents) \n\nsynapse # 1,* ~ 3~\u00b7\" N~ I ~+TAU_NODE,a~r \n\n(feedback from action potentials) \n\n(all outputs go to MLP) \n\np~esynaptic \n\nmput \n\n(C) \n\nTAU-UNIT DYNAMICS \n\n(B) \n\nd ' \n-xl \ndt I \n\n, \n\n= a)-(cid:173)\n\nI \n\nxi \n'to \nI \n\n_d T,; \ndt I \n\n..\\ = xl-(cid:173)\n\n, \nI \n\nTi \nI \n't. \nI \n\nI \n\nI I \n\nai(t) \nx~(~ \nT~(t~ \n\ntM....r\"-(cid:173)\n.~~ \nTlll __ \n\nTAU NODE \n\nTAU UNITS \n\nFigure 2: Tau Layer Schematic. (A) the tau layer has an afferent section, with N tau \nnodes, and a single-node feedback section. (B) Each tau node contains M tau units, and \ntherefore has 1 input and M outputs (C) Each of the M tau units in a tau node has a rise \nand decay function with different constants. The equations are given for the ith tau unit of \nthe jth tau node. a is input activity, x an internal state variable, and T the output. \n\n\f88 \n\nSean D. Murphy, Edward W. Kairiss \n\nafferent inputs (axons) \n\nsingle tau node from output with B \n\ninternal tau units \n\n& IaU node layer \n\nsN \n\n/ \n\nhidden layer of MLP \n\nsingle output unit of MLP \n\nSchematic of output filter processin~ \n\nMLP \n\n',))f\\\",AU M -n-;f-\n\noutput filter I I I \n\nB units \nin 1st layer \nfrom feedback \n\nNxM units in axon subset of \n1st layer of multilayer perceptron \n\nx = full forward connections \n\nbetween layers \n\noutput axon \n\nFigure 3: STEM Architecture. Afferent activity enters the tau layer, where it is con(cid:173)\nverted into a vectorial representation of past spaiotemporal activity. The MLP maps this \nvector into a FlRE/NO-FIRE output unit, the continuous value of which is converted to \na discrete event signal by the refractory period and threshold of the output filter. \n\n3 STEM TRAINING \n\nThere are six phases to training the STEM cell: \n\n(1) Biology: anatomical and physiological data are collected on the cell to be modeled. \n\n(2) Compartmental Model: a compartmental model of the cell is designed, typically with a \nsimulation environment such as GENESIS. As much biological detail as possible is incor(cid:173)\nporated into the model. \n(3) Transfer Function Trials: many random input sequences are generated for the compart(cid:173)\nmental model. The firing response of the model is recorded for each input sequence. \n\n(4) Samplin~ assi~nments; In the next step, sampling will need to be done on the affect of \nthe input sequences on the STEM tay layer. The timing of the sampling is calculated by \nseparating the response of the compartmental model on each trial into regions where no \nspikes occur, and regions surrounding spikes. High-rate sampling times are determined for \nspike regions, and lower rate times are determined for quiet regions. \n\n(5) Tau layer trials: the identical input sequences applied to the compartmental model in \nstep #3 are applied to an isolated tau layer of the STEM cell. The spike events from the \ncompartmental model are used as input for the feedback node. For each input sequence, \nthe tay layer is sampled at the times calculated in step #4, and the vector is labeled as \nFIRE or NO-FIRE (0 or 1). \n\n(6) MLP training: conjugate-gradient and line-search methods are used to train the multi(cid:173)\nlayer perceptron using the results of step #5 as training vectors. \n\n\fModel of Biological Neuron as a Temporal Neural Network \n\n89 \n\nTraining is continued until a minimal performance level is reached, as determined by com(cid:173)\nparing the response of the STEM cell to the original compartmental model on novel input \nsequences. \n\n4 RESULTS \n\nThe STEM cell has been initially evaluated using Roger Traub's (1991) compartmental \nmodel of a hippocampal CAl cell, implemented in GENESIS by Dave Beeman. This is a \nrelatively simple model structurally, with 19 compartments connected in a linear segment, \nwith the soma in the middle. Dynamically, however, it is one of the most accurate and \nsophisticated models published, with on the order of 100 voltage- and Ca++ sensitive \nmembrane channel mechanisms. 94 synapses were placed on the model. Each synapse \nrecevied a random spike train with average frequency 10Hz during training. A diagram of \nthe model and the locations of synaptic input is given in Fig. 4. \n\nInputs going to a single compartment were treated as members of a common synapse, so \nthere were a total of 13 tau nodes, with 5 tau units per node, for a total of 65 tau units, plus \n5 additional units from the feedback tau node. These fed into 70 units in the input layer of \nthe MLP. Two STEM cells were trained, one on a passive shell of the CAl cell, and the \nother with all of the membrane channels included. Both used 70 units in the hidden layer \n\n1200m \n\n880m \n\n6mlfllll_m \n\n14m \n\nmwmiil \nFig. 4 Structure of Traub's CAl cell \n\n8m \n\nreceived 8 synaptic inputs \n\ncompartmental model \n\n-42. \n\n-+4. \n-44'., \n\n-48. \n-150. \n-!5Z. \n-64. \n\n-116. \n-ee. \n\nSTEM cell \n10.0