{"title": "New Hardware for Massive Neural Networks", "book": "Neural Information Processing Systems", "page_first": 201, "page_last": 210, "abstract": null, "full_text": "201 \n\nNEW HARDWARE FOR MASSIVE NEURAL NETWORKS \n\nD.  D.  Coon and A.  G. U.  Perera \nApplied Technology Laboratory \n\nUniversity of Pittsburgh \nPittsburgh, PA  15260. \n\nABSTRACT \n\nTransient phenomena associated with forward biased silicon p + - n - n + struc(cid:173)\ntures at 4.2K show remarkable similarities with biological neurons.  The devices  play \na  role  similar to the  two-terminal switching elements in  Hodgkin-Huxley  equivalent \ncircuit  diagrams.  The  devices  provide simpler  and  more  realistic  neuron  emulation \nthan  transistors  or op-amps.  They  have  such  low  power  and  current  requirements \nthat  they  could  be  used  in  massive  neural  networks.  Some  observed  properties  of \nsimple  circuits  containing  the  devices  include  action  potentials,  refractory  periods, \nthreshold behavior, excitation, inhibition, summation over synaptic inputs, synaptic \nweights,  temporal  integration, memory,  network  connectivity modification  based  on \nexperience, pacemaker activity, firing  thresholds, coupling to sensors with graded sig(cid:173)\nnal  outputs  and  the  dependence  of firing  rate  on input  current.  Transfer functions \nfor simple artificial neurons with spiketrain inputs and spiketrain outputs have been \nmeasured  and correlated with input coupling. \n\nINTRODUCTION \n\nHere we  discuss the simulation of neuron phenomena by electronic processes in \nsilicon from the point of view of hardware for new approaches to electronic processing \nof information which  parallel the means by which information is processed in intelli(cid:173)\ngent organisms.  Development of this hardware basis is  pursued through exploratory \nwork on circuits which exhibit some basic features of biological neural networks.  Fig. 1 \nshows  the  basic  circuit  used  to obtain spiketrain outputs.  A  distinguishing feature \nof this hardware basis is  the spontaneous generation of action potentials as a  device \nphysics feature. \n\n) ! -__  ,----_O_u-f)t put  JLJLL \n\nR \n\nFigure 1:  Spontaneous, \nneuronlike spiketrain \ngenerating circuit.  The \nspikes  are nearly equal in \namplitude so that \ninformation is  contained in \nthe frequency  and \ntemporal pattern of the \nspiketrain generation. \n\n\u00a9 American Institute of Physics 1988 \n\n\f202 \n\nTWO-TERMINAL  SWITCHING  ELEMENTS \n\nThe  use  of transistor  based  circuitry 1 is  avoided  because  transistor  electrical \ncharacteristics  are  not  similar  to  neuron  characteristics.  The  use  of devices  with \nfundamentally  non-neuronlike character increases  the  complexity  of artificial  neural \nnetworks.  Complexity would be  an important drawback for  massive  neural networks \nand  most  neural  networks  in  nature  achieve  their  remarkable  performance  through \ntheir massive size.  In addition) transistors have three terminals whereas the switching \nelements  of Hodgkin-Huxley  equivalent  circuits  have  two  terminals.  Motivated  in \npart by  Hodgkin-Huxley  equivalent  circuit  diagrams) we  employ  two-terminal  p+  -\nn - n+  devices which execute transient switching between low  conductance  and high \nconductance states.  (See Fig. 2)  We  call these devices injection mode devices  (IMDs). \nIn the  \"OFF-STATE\",  a  typical current through the devices  is  '\" 100fA/mm2) and \nin  the  \"ON-STATE\"  a  typical  current  is  '\"  10mA/mm2.  Hence  this  device  is  an \nextremely good switch with a ON / 0 F F ratio of 1011.  As in real neurons2, the current \nin the device is a function of voltage and time, not only voltage.  The devices  require \ncryogenic  cooling but this results in an advantageously low  quiescent power drain of \n<  1 nanowatt/cm2 of chip  area and  the very low  leakage  currents mentioned  above. \nIn addition, the highly unique ability of the neural networks described here to operate \nin  a  cryogenic environment is an important advantage for  infrared image  processing \nat  the  focal  plane  (see  Fig.  3  and  further  discussion  below).  Vision systems  begin \nprocessing  at  the  focal  plane  and  there  are  many  benefits  to  be  gained  from  the \nvision system approach to IR image processing. \n\n-----/  ...-. - - - - -\n\n/\n\n\\ \n\nI \n\nI( V, t) \n\nI \n\nR \n\nVD C ~--~--VV~--~------~ \n\nIR \n\nC \n\n+Q \n\n- Q \n\n;;SS:Ulse \n\nOutput \n\n1----0 \n\nSwitching  element \n\nFigure  2: \nin  Hodgkin-Huxley  equivalent  cir(cid:173)\ncuits. \n\nFigure 3:  Single stage conversion of \ninfrared intensity to spiketrain fre(cid:173)\nquency with a neuron-like semicon(cid:173)\nductor  device.  No  pre-amplifiers \nare necessary. \n\nCoding  of graded  input  signals  (see  Fig.  4)  such  as  photocurrents  into  ac(cid:173)\n\ntion  potential  spike  trains  with  millimeter  scale  devices  has  been  experimentally \ndemonstrated3  with  currents  from  1  IlA  down  to  about  1  picoampere  with  coding \nnoise referred  to input of < 10 femtoamperes.  Coding of much smaller current levels \nshould be possible with smaller devices.  Figure 5 clearly shows the threshold behavior \nof the IMD.  For devices studied to date, a  transition from action potential output to \ngraded signal output is  observed for  input currents of the order of 0.5 picoamperes 1~ \n\n\f203 \n\n4 \n\n10 \n\n--.. \no \nZ \no \nU \nw \n(f) \n\nFigure 4:  Coding of NIR-VISmLE-UV intensity into firing  frequency  of a  spiketrain \nand  the  experimentally determined  firing  rate vs.  the  input current for  one  device. \nNote that the dynamic range is  about 107 . \n\nCURRENT  (AMPERES) \n\n> \n'0 \n\n'(cid:173)> \no \n\nE2 \n- - -PL  \n\nUBI) \n\nFigure 5:  mustration of the threshold firing  of the \ndevice  in response  to input step functions. \n\n500 fLS/ div \n\nThis transition is remarkably well  described in von  Neumann's discussion5 ,6  of \n\nthe  mixed  character of neural  elements which  he  relates  to  the concept  of sublimi(cid:173)\nnal  stimulation  levels  which  are  too  low  to produce  the stereotypical  all-or-nothing \nresponse.  Neural  network  modelers  frequently  adopt  viewpoints  which  ignore  this \ninteresting mixed character.  The von  Neumann viewpoint links the mixed character \nto  concepts  of nonlinear  dynamics  in  a  way  which is  not  apparent  in  recent  neural \nnetwork  modeling  literature.  The  scaling  down  of IMD  size  should  result  in  even \nlower current requirements for  all-or-nothing response. \n\nDEVICE PHYSICS \n\nRecently, neuronlike action potential transients in  IMDs have been the subject \nof considerable  research3 ,4,7,8,9,1O,1l,12,13.  In  the  simple  circuits  of  Fig.  1,  the  IMD \ngives  rise to a spontaneous neuronlike spiketrain output.  Between pulses, the IMD is \npolarized in the sense that it is in a low  conductance state with a substantial voltage \noccurring across it, even though it is  forward  biased.  The low  conductance has been \nattributed  to  small  interfacial  work  functions  due  to band  offsets  at  the  n+ -n  and \np+ -n interfaces8 \u2022 \n\nLow  temperatures  inhibit  thermionic injection of electrons  and  holes  into  the \nn-region  from  the  n+ -layer  and  p+ -layer  impurity  bands14 .  Pulses  are  caused  by \n\n\f204 \n\nswitching  to  depolarized  states  with  low  diode  potential  drops  and  large  injection \ncurrents which are believed to be triggered by the slow buildup of a small thermionic \ninjection current from the n+ -layer into the n-region.  The injection current can cause \nimpact  ionization  of n-region  donor  impurities  resulting  in  an  increasingly  positive \nspace charge which further enhances the injection current to the point where the IMD \nabruptly switches to the low conductance state with large injection current.  Switching \ntimes  are  typically  under  lOOns.  Charging  of the  load  capacitance  CL  cuts  off the \nlarge injection  current  and  resets  the  diode  to its low  conductance  state.  The  load \ncapacitor CL  then  discharges  through RL.  During the CL  discharging  time constant \nRLCL  the voltage across  the  IMD  itself is  low  and  therefore  the bias voltage  would \nhave  to be  raised  substantially to cause  further  firing.  Thus,  RLCL  is  analogous  to \nthe refractory period of a neuron.  The output pulses of an IMD generally have about \nthe same amplitude while  the  rate of pulsing  varies over  a wide  range depending  on \nthe bias voltage  and the presence  of electromagnetic radiation. 7,8,10 \n\n~ DETECTOR  ARRAY \n\u00a2=::I TRANSIENT  SENSING \n\n\u00a2=::I MOTION  SENSING  - TRACKING \n\n2-D  PARALLEL  OUTPUT \n\nFigure 6:  lllustrative \nlaminar architecture \nshowing stacked wafers in \n3-dimensions. \n\nLAMINAR \n\nNEURAL  NETWORK \n\nREAL TIME PARALLEL ASYNCHRONOUS  PROCESSING \n\nThe devices  described  here  could  form  the hardware basis for  a  parallel asyn(cid:173)\n\nchronous processor in  much the same way  that transistors form  the basis for  digital \ncomputers.  The devices could be used to construct networks which could perform real \ntime signal processing.  Pulse propagation through silicon chips  (parallel firethrough, \nsee  Fig.  7)  as  opposed  to the  lateral  planar  propagation  in  conventional  integrated \ncircuits  has  been  proposed. 1S  This  would  permit  the  use  of laminar,  stacked  wafer \narchitectures.  See  Fig. 6. \n\nSuch  architectures  would  eliminate  the  serial  processing  limitations  of stan(cid:173)\n\ndard processors which utilize multiplexing and charge transfer.  There  are additional \nadvantages in terms of elimination of pre-amplifiers and reduction in power consump(cid:173)\ntion.  The approach would  utilize the  low  power,  low  noise deviceslO  described  here \nto perform input signal-to-frequency conversion in every processing  channel. \n\nPOWER CONSUMPTION  FOR A BRAIN SCALE  SYSTEM \n\nThe low  power and low  current requirements together with the electronic sim(cid:173)\n\nplicity  (lower parts-count as compared  with transistor  and op-amp  approaches)  and \n\n\fINPUTS \n\n111111111111  \n\n;1\"*\"*\"* *'\"*\"* '*\"* '* '*\"* \"*1  Siwafer \n;1* * * * '* *\"*\"*\"* *' * *1  Siwafer \n;1* * * * * * * * ** * *1 Siwaf.r \n;1*\"*\"*\"* * ** *\"* '*\"* \"*1  Siwaf.r \n;1* * *\"* * * *\"*\"* '*\"* \"*1  Siwaf.r \n\nI  1 Si  wafer \n; I I I I I I I I I I I I  \n! \n\n! \n\n! \n\n! \n\n! \n\n! \n\n! \n\n! \n\n! \n\n! \n\n! \n\n! \n\n205 \n\nFigure  7:  Schematic  illus(cid:173)\ntration  of  the  signal  flow \npattern through a real time \nparallel  asynchronous  pro(cid:173)\ncessor consisting of stacked \nsilicon wafers. \n\nOUTPUTS \n\nthe  natural  emulation  of neuron  features  means  that  the  approach  described  here \nwould  be especially  advantageous for  very  large  neural  networks, e.g.  systems com(cid:173)\nparable to supercomputers in which power dissipation and system complexity are im(cid:173)\nportant considerations.  The  power  consumption of large scale  analog16  and  digital17 \nsystems  is  always  a  major  concern.  For  example,  the  power  consumption  of  the \nCRAY XMP-48 is  of the order of 300 kilowatts.  For the devices  described  here,  the \npower consumption is very low.  For these devices, we  have observed quiescent power \ndrains of about  1 n W /cm2 and pulse power consumption of about 500 nJ/pulse/cm2 \u2022 \nWe  estimate  that  a  system  with  1011  active  10~m x  10~m elements  (comparable \nto the  number of neurons  in  the  brain18)  all  firing  with  an  average  pulse  rate of 1 \nKHz  (corresponding to a  high neuronal firing  rateS)  would  consume about  50 watts. \nThe  quiescent  power  drain  for  this  system  would  be  0.1  milliwatts.  Thus,  power \n(P)  requirements for  such an artificial neural network with the size scale  (1011  pulse \ngenerating  elements)  of the  human  brain  and  a  range  of activity  between  zero  and \nthe  maximum  conceivable  sustained  activity  for  neurons  in  the  brain would  be  0.1 \nmilliwatts  <  P  <  50  watts for  10 micron  technology.  For comparison, we  note  that \nvon  Neumann's estimate for  the  power  dissipation  of the  brain is  of order  10  to  25 \nwatts. S,6  Fabrication of a  1011  element 10 ~m artificial neural network would  require \nprocessing of about  1500 four  inch wafers. \n\nNETWORK CONNECTIVITY \n\nFor a  network with coupling between many IMD's3  we  have shown\"  that \n\n(1) \n\nwhere  Vj  is  the  voltage  across  the  diode  and  the  input  capacitance  Cj  of the  i-th \nnetwork node, Rj represents a leakage resistance in parallel with Cil and Ij  represents \nan external current input to the i-th diode.  iJ=1,2,3, .....  label different network nodes \nand  Tij  incoporates  coupling  between  network  elements.  Equation  1  has  the  same \nform  as  equations  which  occur  in  the  Hopfield  modeI2o,21,22,23  for  neural  networks. \nSejnowski has  also  discussed  similar  equations in  connection with skeleton  filters  in \n\n\f206 \n\nINPUTS \n\nR \n\nc); \no---j \no---j~~~~~'fi-~ \no---j \n~: \n\nR \n\nOUTPUTS \n\n;~ \nt----o \n~f---r----t---t t----o \nt----o \n:P---o \n\nFigure 8:  a)  Main features  of a  typical  neuron from  Kandel  and  Schwartz.19  b)  Our \nartificial neuron) which shows the summation over synaptic inputs and fan-out. \n\nTRANSMISSION  LINE \n\nthe brain. 24\u202225  Nonlinear threshold behavior of IMD)s enters through F(V)  as it does \nin the neural network  models. \n\nIn  Fig. 8-b a range of input capacitances is possible.  This range of capacitances \nis related to the range of possible synaptic weights.  The circuit in Fig. 8 accomplishes \npulse  height  discrimination  and  each  pulse  can  contribute  to  the  charge  stored  on \nthe  central  node  capacitance  C.  The  charge  added  to C  during  each  input  pulse  is \nlinearly related to the input capacitance except at extreme limits.  The range of input \ncapacitances for a particular experiment was .002 J-lF  to .2 J-lF  which differ by a factor \nof about  100.  The  effect  of various  input  capacitance  values  (synaptic  weights)  on \ninput-output firing  rates is shown in Fig. 9.  Also the Fig. 8-b shows many capacitive \ninputs/outputs to/from a single IMD. i.e.  fan-in  and fan-out.  For pulses which arrive \nat different  inputs  at  about  the same  time)  the effect  of the pulses  is  additive.  The \ntime within which inputs are summed is just the stored  charge lifetime.  Summation \nover many inputs is  an important feature of neural information processing. \n\nEXCITATION)  INHIBITION) MEMORY \n\nBoth excitatory and inhibitory input circuits are shown in Fig. 10.  Input pulses \ncause  the  accumulation  of charge  on  C  in  excitatory  circuits  and  the  depletion  of \ncharge on C  in  inhibitory circuits.  Charge associated  with input spiketrains  is  inte(cid:173)\ngrated/stored on C. The temporally integrated charge is  depleted by the firing  of the \nIMD.  Thus)  the storage time  is  related to the  firing  rate.  After  an  input spiketrain \nraises the potential across C  to a  value above the firing  threshold) the resulting IMD \n\n\fO.2}J F \n\n5 \n\nO.03}JF \n\n207 \n\nFigure 9:  Output pulse \nrate vs.  the input \npulse rate for  different \ninput capacitance \nvalues Ci  values \n\n,--.,. \nN \nI \n.;,< \n\n4 \n\nW \nt-\n~ 3 \nw \nVl \n---1 \n::J \n(L  2 \nt-\n::J \nCL \n\nt-6  1 \n\n(0) \n\n(b) \n\n20 \n\n40 \n\n60 \n\n80 \n\n100 \n\nINPUT  PULSE  RATE  (Hz ) \n\nR \n\nI NP~ I----'-~-r_{)l__--,----<>OUTPUT \n\nFigure  10:  Circuits which incorporate rec(cid:173)\ntifying  synaptic  inputs.  a)  an  excitatory \ninput.  b)  an  inhibitory input. \n\nR \n\nR' \n\nINP~ \n\nc\u00b7 L \n\nR' L \n\noutput spiketrain  codes  the input information.  The output firing  rate is  linearly  re(cid:173)\nlated to the input firing rate times the synaptic coupling strength (linearly related to \nCi).  See Fig. 9.  If the input ceases, then the potential across C relaxes back to a value \njust below the firing  threshold.  When  not firing,  the  IMD  has  a  high  impedance.  If \nthere  is  negligible  leakage  of charge from  C, then  V can  remain near  V T  (threshold \nvoltage)  for  a  long  time  and  a  new  input signal  will quickly  take  the  IMD  over  the \nfiring  threshold.  See Fig. 11.  We  have observed stored charge lifetimes of 56 days and \nlonger  times may  be  acheivable.  The lifetime  of charge stored on C  can  be reduced \nby adding a  resistance in parallel with C. \n\nFrom the discussion of integration, we see that long term storage of charge on C \nis equivalent to long term memory.  The memory can be  read by seeing if a new input \npulse  or spiketrain  produces  a  prompt output  pulse  or spiketrain.  The  read  signal \ninput  channel  in  Fig.  8-b  can  be  the  same  as  or different  from  the  channel  which \nresulted in the charge storage.  In either case memory would produce  a change in the \npattern of connectivity if the circuit was  imbedded  in  a  neural network.  Changes  in \npatterns of connectivity are similar  to Hebb's ruie considerations26  in which memory \nis associated with increases in the strength (weight) of synaptic couplings.  Frequently, \n\n\f208 \n\n-\n\nQJ -o \n\na:: \n\n13 \n\n11 \nInput  Potential \n\nFigure 11:  Firing rate vs.  the bias voltage. \nThe region where  the firing  is  negligible is \nassociated  with memory.  The state of the \nmemory  is  associated  with  the  proximity \nto the firing  threshold. \n\nthe increase in synaptic weights is  modeled  by increased conductance whereas in the \ncircuits in Figs.  lO(a) and 8-b memory is achieved by integration and charge storage. \nNote that for  these particular circuits, the memory is not eraseable although volatile \n(short term)  memory  can easily  be  constructed by adding a  resistor in parallel with \nC. Thus, a  continuous range of memory lifetimes can be  achieved. \n\n2-D  PARALLEL ASYNCHRONOUS  CHIP-TO-CHIP TRANSMISSION \n\nFor  many  IMD's the output pulse  heights  for  a  circuit  like  that in  Fig.  1  are \n>3 volts.  Thus, output from  the first  stage or any later stage of the  network  could \neasily  be  transmitted  to other  parts  of an overall  system.  Two-dimensional  arrays \nof  devices  on  different  chips  could  be  coupled  by  indium  bump  bonding  to  form \nthe laminar architecture  described  above.  Planar technology could  be  used for  local \nlateral interconnections in  the  processor.  (See  Fig.  7)  In  addition to transmission  of \nelectrical pulses, optical transmission is possible because the pulses can directly drive \nLED's. \n\nEmerging  GaAs-on-Si  technology  is  interesting  as  a  means  of fabricating  two \ndimensional  emitter  arrays.  Optical  transmission  is  not  necessary  but  it  might  be \nuseful  (A)  for  processed  image data transfer,  (B)  for  coupling  to  an optical proces(cid:173)\nsor, or  (C)  to provide  2-0 optical interconnects  between chips bearing  2-D  arrays of \np+ - n - n+  diodes.  Note that with optical interconnects between chips, the circuits \nemployed here would be internal receivers.  The p-i-n diodes employed in the present \nwork  would  be well  suited  to the receiver role.  An  interesting possibility  would  en(cid:173)\ntail the  use  optical interconnects between  chips  to achieve  local,  lateral interaction. \nThis would  be accomplished by  having each optical emitter in a  2-D  array  broadcast \nlocally  to multiple receivers rather than to a single  receiver.  Similarly, each receiver \nwould  have  a  reeeptive field  extending over multiple transmitters.  It is  also possible \nthat an optical element could be placed in the gap between  parallel transmitter and \nreceiver  planes  to structure,  control  or  alter  2-D  patterns of interconnection.  This \nwould  be  an  alternative  to a  planar technology  approach  to lateral interconnection. \nIT  the  optical elements were  active  then  the system  would  constitute  a  hybrid opti(cid:173)\ncal/electronic processor, whereas if passive optical elements were employed, we would \nregard  the system as an optoelectronic processor.  In either case, we  picture the pro(cid:173)\ncessing functions of temporal integration, spatial summation over inputs, coding  and \npulse generation as residing on-chip. \n\n\f209 \n\nACKNOWLEDGEMENTS \n\nThe  work  was  supported  in  part  by  U.S.  DOE  under  contract  #DE-AC02-\n\n80ER10667 and NSF  under grant #  ECS-8603075. \n\nReferences \n\n[1]  L.  D.  Harmon, Kybernetik 1,89 (1961). \n\n[2]  A.  L.  Hodgkin and A.  F.  Huxley, J.  Physioll17, 500 (1952). \n\n[3]  D.  D.  Coon  and A.  G.  U.  Perera, Int. J.  Electronics 63, 61  (1987). \n\n[4]  K.  M.  S.  V.  Bandara,  D.  D.  Coon  and  R.  P.  G.  Karunasiri,  Infrared  'lransient \n\nSensing, to be  published. \n\n[5]  J.  von  Neumann,  The  Computer  and  the  Brain,  Yale  University  Press,  New \n\nHaven and  London, 1958. \n\n[6]  J. von  Neumann,  Collected  Works,  Pergamon Press,  New  York,  1961. \n\n[7]  D.  D.  Coon  and A.  G. U.  Perera, Int. J.  Infrared  and  Millimeter  Waves  7, 1571 \n\n(1986). \n\n[8]  D.  D.  Coon  and  S.  D.  Gunapala, J.  Appl.  Phys 57, 5525  (1985). \n\n[9]  D.  D.  Coon,  S.  N.  Ma and A.  G. U.  Perera, Phys.  Rev.  Let.  58, 1139  (1987). \n\n[10]  D.  D. Coon  and A.  G.  U.  Perera, Applied  Physics  Letters 51, 1711  (1987). \n\n[11]  D.  D.  Coon  and  A.  G.  U.  Perera, Solid-State Electronics 29, 929  (1986). \n\n[12]  D.  D. Coon  and A.  G.  U.  Perera, Applied  Physics  Letters 51, 1086  (1987). \n\n[13]  K.  M. S.  V.  Bandara, D.D.  Coon and R.  P. G.  Karunasiri, Appl.  Phys. Lett 51, \n\n961  (1987). \n\n[14]  Y.  N.  Yang,  D.  D.  Coon  and  P.  F.  Shepard,  Applied  Physics  Letters  45,  752 \n\n(1984). \n\n[15]  D.  D.  Coon and A.  G. U.  Perera, Int. J. IR and Millimeter Waves 8, 1037 (1987). \n\n[16]  M.  A.  Sivilotti,  M.  R.  Emerling  and  C.  A.  Mead,  VLSI Arcbitectures  for  Im(cid:173)\n\nplementation of Neural  Networks, Neural Networks for  Computing, A.J.P., \n1986, pp. 408-413. \n\n[17]  R.  W.  Keyes,  Proc.  IEEE 63, 740  (1975). \n\n[18]  E.  R.  Kandel  and  J.  H.  Schwartz,  Principles  of Neural  Science,  Elsevier,  New \n\nYork,  1985. \n\n\f210 \n\n[19]  E.  R.  Kandel  and  J.  H.  Schwartz,  Principles  of Neural  Science,  Elsevier,  New \nYork,  1985,  page  15,  Reproduced  by  permission  of Elsevier  Science  Publishing \nCo., N.Y .. \n\n[20]  J. J.  Hopfield,  Proc.  Nat!. Acad.  Sci.  U.S.A 81, 3088  (1984). \n\n[21]  J.  J.  Hopfield  and D.  W.  Tank, BioI.  Cybern 52, 141  (1985). \n\n[22]  J.  J.  Hopfield  and D.  W.  Tank, Science  233,625 (1986). \n\n[23]  D.  W.  Tank  and J. J. Hopfield,  IEEE. Circuits Syst.  CAS-33, 533  (1986). \n\n[24]  T. J.  Sejnowski, J. Math. Biology  4, 303  (1977). \n\n[25]  T.  J.  Sejnowski,  Skeleton  Filters  in  tbe Brain,  Lawrence  Erlbaum,  New  Jersey, \n\n1981, pp.  189-212, edited by  G. E.  Hinton and J. A.  Anderson. \n\n[26]  J.  L.  McClelland,  D.  E.  Rumelhart  and  the  PDP research  group,  Parallel  Dis(cid:173)\ntributed Processing,  The MIT Press, Cambridge,  Massachusetts,  1986,  two vol(cid:173)\numes. \n\n\f", "award": [], "sourceid": 22, "authors": [{"given_name": "Darryl", "family_name": "Coon", "institution": null}, {"given_name": "A.", "family_name": "Perera", "institution": null}]}