{"title": "Pulsestream Synapses with Non-Volatile Analogue Amorphous-Silicon Memories", "book": "Advances in Neural Information Processing Systems", "page_first": 763, "page_last": 769, "abstract": null, "full_text": "Real-Time Control of a Tokamak Plasma \n\nUsing Neural Networks \n\nChris M Bishop \n\nNeural Computing Research Group \nDepartment of Computer Science \n\nAston University \n\nBirmingham, B4 7ET, U.K. \n\nc.m .bishop@aston .ac.uk \n\nPaul S Haynes, Mike E U Smith, Tom N Todd, \n\nDavid L Trotman and Colin G Windsor \n\nAEA Technology, Culham Laboratory, \n\nOxfordshire OX14 3DB \n\n(Euratom/UKAEA Fusion Association) \n\nAbstract \n\nThis paper presents results from the first use of neural networks \nfor the real-time feedback control of high temperature plasmas in \na tokamak fusion experiment. The tokamak is currently the prin(cid:173)\ncipal experimental device for research into the magnetic confine(cid:173)\nment approach to controlled fusion. \nIn the tokamak, hydrogen \nplasmas, at temperatures of up to 100 Million K, are confined \nby strong magnetic fields. Accurate control of the position and \nshape of the plasma boundary requires real-time feedback control \nof the magnetic field structure on a time-scale of a few tens of mi(cid:173)\ncroseconds. Software simulations have demonstrated that a neural \nnetwork approach can give significantly better performance than \nthe linear technique currently used on most tokamak experiments. \nThe practical application of the neural network approach requires \nhigh-speed hardware, for which a fully parallel implementation of \nthe multilayer perceptron, using a hybrid of digital and analogue \ntechnology, has been developed. \n\n\f1008 \n\nC. Bishop, P. Haynes, M. Smith, T. Todd, D. Trotman, C. Windsor \n\n1 \n\nINTRODUCTION \n\nFusion of the nuclei of hydrogen provides the energy source which powers the sun. \nIt also offers the possibility of a practically limitless terrestrial source of energy. \nHowever, the harnessing of this power has proved to be a highly challenging prob(cid:173)\nlem. One of the most promising approaches is based on magnetic confinement of a \nhigh temperature (107 - 108 Kelvin) plasma in a device called a tokamak (from the \nRussian for 'toroidal magnetic chamber') as illustrated schematically in Figure 1. \nAt these temperatures the highly ionized plasma is an excellent electrical conduc(cid:173)\ntor, and can be confined and shaped by strong magnetic fields. Early tokamaks \nhad plasmas with circular cross-sections, for which feedback control of the plasma \nposition and shape is relatively straightforward. However, recent tokamaks, such as \nthe COMPASS experiment at Culham Laboratory, as well as most next-generation \ntokamaks, are designed to produce plasmas whose cross-sections are strongly non(cid:173)\ncircular. Figure 2 illustrates some of the plasma shapes which COMPASS is de(cid:173)\nsigned to explore. These novel cross-sections provide substantially improved energy \nconfinement properties and thereby significantly enhance the performance of the \ntokamak. \n\nz \n\nR \n\nFigure 1: Schematic cross-section of a tokamak experiment show(cid:173)\ning the toroidal vacuum vessel (outer D-shaped curve) and plasma \n(shown shaded). Also shown are the radial (R) and vertical (Z) co(cid:173)\nordinates. To a good approximation, the tokamak can be regarded \nas axisymmetric about the Z-axis, and so the plasma boundary can \nbe described by its cross-sectional shape at one particular toroidal \nlocation. \n\nUnlike circular cross-section plasmas, highly non-circular shapes are more difficult to \nproduce and to control accurately, since currents through several control coils must \nbe adjusted simultaneously. Furthermore, during a typical plasma pulse, the shape \nmust evolve, usually from some initial near-circular shape. Due to uncertainties \nin the current and pressure distributions within the plasma, the desired accuracy \nfor plasma control can only be achieved by making real-time measurements of the \nposition and shape of the boundary, and using error feedback to adjust the currents \nin the control coils. \n\nThe physics of the plasma equilibrium is determined by force balance between the \n\n\fReal-Time Control of Tokamak Plasma Using Neural Networks \n\n1009 \n\ncircle \n\nellipse \n\nO-shape \n\nbean \n\nFigure 2: Cross-sections of the COMPASS vacuum vessel showing \nsome examples of potential plasma shapes. The solid curve is the \nboundary of the vacuum vessel, and the plasma is shown by the \nshaded regions. \n\nthermal pressure of the plasma and the pressure of the magnetic field, and is rel(cid:173)\natively well understood. Particular plasma configurations are described in terms \nof solutions of a non-linear partial differential equation called the Grad-Shafranov \n(GS) equation. Due to the non-linear nature of this equation, a general analytic \nsolution is not possible. However, the GS equation can be solved by iterative nu(cid:173)\nmerical methods, with boundary conditions determined by currents flowing in the \nexternal control coils which surround the vacuum vessel. On the tokamak itself it \nis changes in these currents which are used to alter the position and cross-sectional \nshape of the plasma. Numerical solution of the GS equation represents the stan(cid:173)\ndard technique for post-shot analysis of the plasma, and is also the method used \nto generate the training dataset for the neural network, as described in the next \nsection. However , this approach is computationally very intensive and is therefore \nunsuitable for feedback control purposes. \n\nFor real-time control it is necessary to have a fast (typically:::; 50J.lsec.) determi(cid:173)\nnation of the plasma boundary shape. This information can be extracted from a \nvariety of diagnostic systems, the most important being local magnetic measure(cid:173)\nments taken at a number of points around the perimeter of the vacuum vessel. \nMost tokamaks have several tens or hundreds of small pick up coils located at care(cid:173)\nfully optimized points around the torus for this purpose. We shall represent these \nmagnetic signals collectively as a vector m . \n\nFor a large class of equilibria, the plasma boundary can be reasonably well repre(cid:173)\nsented in terms of a simple parameterization, governed by an angle-like variable B, \ngiven by \n\nR(B) \nZ(B) \n\nRo + a cos(B + 8 sinB) \nZo + a/\\,sinB \n\n(1) \n\nwhere we have defined the following parameters \n\n\f1010 \n\nC. Bishop, P. Haynes, M. Smith, T. Todd, D. Trotman, C. Windsor \n\nRo \nZo \na \nK \n6 \n\nradial distance of the plasma center from the major axis of the torus, \nvertical distance of the plasma center from the torus midplane, \nminor radius measured in the plane Z = Zo, \nelongation, \ntriangularity. \n\nWe denote these parameters collectively by Yk. The basic problem which has to be \naddressed, therefore, is to find a representation for the (non-linear) mapping from \nthe magnetic signals m to the values of the geometrical parameters Yk, which can \nbe implemented in suitable hardware for real-time control. \n\nThe conventional approach presently in use on many tokamaks involves approxi(cid:173)\nmating the mapping between the measured magnetic signals and the geometrical \nparameters by a single linear transformation. However, the intrinsic non-linearity \nof the mappings suggests that a representation in terms of feedforward neural net(cid:173)\nworks should give significantly improved results (Lister and Schnurrenberger, 1991; \nBishop et a/., 1992; Lagin et at., 1993). Figure 3 shows a block diagram of the \ncontrol loop for the neural network approach to tokamak equilibrium control. \n\nNeural \nNetwork \n\nFigure 3: Block diagram of the control loop used for real-time \nfeedback control of plasma position and shape. \n\n2 SOFTWARE SIMULATION RESULTS \n\nThe dataset for training and testing the network was generated by numerical so(cid:173)\nlution of the GS equation using a free-boundary equilibrium code. The data base \ncurrently consists of over 2,000 equilibria spanning the wide range of plasma po(cid:173)\nsitions and shapes available in COMPASS. Each equilibrium configuration takes \nseveral minutes to generate on a fast workstation. The boundary of each configu(cid:173)\nration is then fitted using the form in equation 1, so that the equilibria are labelled \nwith the appropriate values of the shape parameters. Of the 120 magnetic signals \navailable on COMPASS which could be used to provide inputs to the network, a \n\n\fReal-Time Control o/Tokamak PLasma Using Neural Networks \n\n1011 \n\nsubset of 16 has been chosen using sequential forward selection based on a linear \nrepresentation for the mapping (discussed below) . \nIt is important to note that the transformation from magnetic signals to flux surface \nparameters involves an exact linear invariance. This follows from the fact that, if all \nof the currents are scaled by a constant factor, then the magnetic fields will be scaled \nby this factor, and the geometry of the plasma boundary will be unchanged. It is \nimportant to take advantage of this prior knowledge and to build it into the network \nstructure, rather than force the network to learn it by example. We therefore \nnormalize the vector m of input signals to the network by dividing by a quantity \nproportional to the total plasma current. Note that this normalization has to be \nincorporated into the hardware implementation of the network, as will be discussed \nin Section 3. \n\n4 \n\n01 2 \nc .5. \nCIS \n:E \n1iI \n~ -2 \n::J \n\n0- \u00b0 \n\n-4 \n\n2 \n\n01 c .5. \n:E \u00b0 \n~ \n1iI \nCD c \n::J \n\n-2 \n\n1.2 \n\ngo.8 \n.5. \n0-\nCIS \n:E \n1iI0.4 \nCD c \n::J \n\n\u00b0 \n\nDatabase \n\nDatabase \n\nDatabase \n\n4 \n\n2 \n\n-2 \n\n-4 \n\n~ \n\n~ \u00b0 \n\nCD \nZ \n~ \n:::I \nCD z \n\n1.2 \n\n~O.8 \n~ \nCD z \n~O.4 \n:::I \nCD \nZ \n\n\u00b0 \n\n.2 \n\n.2 \n\nDatabase \n\nDatabase \n\nDatabase \n\nFigure 4: Plots of the values from the test set versus the values \npredicted by the linear mapping for the 3 equilibrium parameters, \ntogether with the corresponding plots for a neural network with 4 \nhidden units. \n\nThe results presented in this paper are based on a multilayer perceptron architecture \nhaving a single layer of hidden units with 'tanh' activation functions , and linear \noutput units. Networks are trained by minimization of a sum-of-squares error using \na standard conjugate gradients optimization algorithm, and the number of hidden \n\n\fJ012 \n\nC. Bishop, P. Haynes, M. Smith, T. Todd, D. Trotman, C. Windsor \n\nunits is optimized by measuring performance with respect to an independent test \nset. Results from the neural network mapping are compared with those from the \noptimal linear mapping, that is the single linear transformation which minimizes \nthe same sum-of-squares error as is used in the neural network training algorithm, \nas this represents the method currently used on a number of present day tokamaks. \n\nInitial results were obtained on networks having 3 output units, corresponding to \nthe values of vertical position ZQ, major radius RQ, and elongation K; these being \nparameters which are of interest for real-time feedback control. The smallest nor(cid:173)\nmalized test set error of 11.7 is obtained from the network having 16 hidden units. \nBy comparison, the optimal linear mapping gave a normalized test set error of 18.3. \nThis represents a reduction in error of about 30% in going from the linear mapping \nto the neural network. Such an improvement, in the context of this application, is \nvery significant. \n\nFor the experiments on real-time feedback control described in Section 4 the cur(cid:173)\nrently available hardware only permitted networks having 4 hidden units, and so we \nconsider the results from this network in more detail. Figure 4 shows plots of the \nnetwork predictions for various parameters versus the corresponding values from \nthe test set portion of the database. Analogous plots for the optimal linear map \npredictions versus the database values are also shown. Comparison of the corre(cid:173)\nsponding figures shows the improved predictive capability of the neural network, \neven for this sub-optimal network topology. \n\n3 HARDWARE IMPLEMENTATION \n\nThe hardware implementation of the neural network must have a bandwidth of 2: \n20 kHz in order to cope with the fast timescales of the plasma evolution. It must \nalso have an output precision of at least (the the analogue equivalent of) 8 bits in \norder to ensure that the final accuracy which is attainable will not be limited by the \nhardware system. We have chosen to develop a fully parallel custom implementation \nof the multilayer perceptron, based on analogue signal paths with digitally stored \nsynaptic weights (Bishop et al., 1993). A VME-based modular construction has \nbeen chosen as this allows flexibility in changing the network architecture, ease of \nloading network weights, and simplicity of data acquisition. Three separate types \nof card have been developed as follows: \n\n\u2022 Combined 16-input buffer and signal normalizer. \n\nThis provides an analogue hardware implementation of the input normal(cid:173)\nization described earlier. \n\u2022 16 x 4 matrix multiplier \n\nThe synaptic weights are produced using 12 bit frequency-compensated \nmultiplying DACs (digital to analogue converters) which can be configured \nto allow 4-quadrant multiplication of analogue signals by a digitally stored \nnumber. \n\n\u2022 4-channel sigmoid module \n\nThere are many ways to produce a sigmoidal non-linearity, and we have \nopted for a solution using two transistors configured as along-tailed-pair, \n\n\fReal-Time Control of Tokamak Plasma Using Neural Networks \n\n1013 \n\nto generate a 'tanh ' sigmoidal transfer characteristic. The principal draw(cid:173)\nback of such an approach is the strong temperature sensitivity due to the \nappearance of temperature in the denominator of the exponential transistor \ntransfer characteristic. An elegant solution to this problem has been found \nby exploiting a chip containing 5 transistors in close thermal contact. Two \nof the transistors form the long-tailed pair, one of the transistors is used \nas a heat source, and the remaining two transistors are used to measure \ntemperature. External circuitry provides active thermal feedback control, \nand stability to changes in ambient temperature over the range O\u00b0C to 50\u00b0C \nis found to be well within the acceptable range. \n\nThe complete network is constructed by mounting the appropriate combination \nof cards in a VME rack and configuring the network topology using front panel \ninterconnections. The system includes extensive diagnostics, allowing voltages at \nall key points within the network to be monitored as a function of time via a series \nof multiplexed output channels. \n\n4 RESULTS FROM REAL-TIME FEEDBACK CONTROL \n\nFigure 5 shows the first results obtained from real-time control of the plasma in \nthe COMPASS tokamak using neural networks. The evolution of the plasma elon(cid:173)\ngation, under the control of the neural network, is plotted as a function of time \nduring a plasma pulse. Here the desired elongation has been preprogrammed to \nfollow a series of steps as a function of time. The remaining 2 network outputs \n(radial position Ro and vertical position Zo) were digitized for post-shot diagnosis , \nbut were not used for real-time control. The solid curve shows the value of elonga(cid:173)\ntion given by the corresponding network output, and the dashed curve shows the \npost-shot reconstruction of the elongation obtained from a simple 'filament' code, \nwhich gives relatively rapid post-shot plasma shape reconstruction but with limited \naccuracy. The circles denote the elongation values given by the much more accurate \nreconstructions obtained from the full equilibrium code. The graph clearly shows \nthe network generating the required elongation signal in close agreement with the \nreconstructed values. The typical residual error is of order 0.07 on elongation values \nup to around 1.5. Part of this error is attributable to residual offset in the integra(cid:173)\ntors used to extract magnetic field information from the pick-up coils, and this is \ncurrently being corrected through modifications to the integrator design. An addi(cid:173)\ntional contribution to the error arises from the restricted number of hidden units \navailable with the initial hardware configuration. While these results represent the \nfirst obtained using closed loop control, it is clear from earlier software modelling of \nlarger network architectures (such as 32- 16-4) that residual errors of order a few % \nshould be attainable. The implementation of such larger networks is being persued, \nfollowing the successes with the smaller system. \n\nAcknowledgements \n\nWe would like to thank Peter Cox, Jo Lister and Colin Roach for many useful \ndiscussions and technical contributions. This work was partially supported by the \nUK Department of Trade and Industry. \n\n\f1014 \n\nC. Bishop, P. Haynes, M. Smith, T. Todd, D. Trotman, C. Windsor \n\n1.8 \n\nshot 9576 \n\nC) \n\nc: o \n~ 14 \nc: o as \n\n\u2022 \n\n1.0 \n\n0.0 \n\n0.1 \n\n0.2 \n\ntime (sec.) \n\nFigure 5: Plot of the plasma elongation K. as a function of time \nduring shot no. 9576 on the COMPASS tokamak, during which the \nelongation was being controlled in real-time by the neural network. \n\nReferences \n\nBishop C M, Cox P, Haynes P S, Roach C M, Smith M E U, Todd T N and Trotman \nD L, 1992. A neural network approach to tokamak equilibrium control. In Neural \nNetwork Applications, Ed. J G Taylor, Springer Verlag, 114-128. \n\nBishop C M, Haynes P S, Roach C M, Smith ME U, Todd T N, and Trotman D L. \n1993. Hardware implementation of a neural network for plasma position control in \nCOMPASS-D. In Proceedings of the 17th. Symposium on Fusion Technology, Rome, \nItaly. 2 997-1001. \n\nLagin L, Bell R, Davis S, Eck T, Jardin S, Kessel C, Mcenerney J, Okabayashi \nM, Popyack J and Sauthoff N. 1993. Application of neural networks for real-time \ncalculations of plasma equilibrium parameters for PBX-M, In Proceedings of the \n17th. Symposium on Fusion Technology, Rome, Italy. 21057-106l. \n\nLister J Band Schnurrenberger H. 1991. Fast non-linear extraction of plasma \nparameters using a neural network mapping. Nuclear Fusion. 31, 1291-1300. \n\n\fPulsestream Synapses with Non-Volatile \nAnalogue Amorphous-Silicon Memories. \n\nA.J. Holmes, A.F. Murray, S. Churcher and J. Hajto \n\nDepartment of Electrical Engineering \n\nUniversity of Edinburgh \n\nEdinburgh, EH9 3JL \n\nDept. of Applied Physics and Electronics, \n\nM. J. Rose \n\nDundee University \nDundee DD14HN \n\nAbstract \n\nA novel two-terminal device, consisting of a thin lOooA layer of p+ \na-Si:H sandwiched between Vanadium and Chromium electrodes, \nexhibits a non-volatile, analogue memory action. This device stores \nsynaptic weights in an ANN chip, replacing the capacitor previously \nused for dynamic weight storage. Two different synapse designs are \ndiscussed and results are presented. \n\n1 \n\nINTRODUCTION \n\nAnalogue hardware implementations of neural networks have hitherto been ham(cid:173)\npered by the lack of a straightforward (local) analogue memory capability. The \nideal storage mechanism would be compact, non-volatile, easily reprogrammable, \nand would not interfere with the normal silicon chip fabrication process. \nTechniques which have been used to date include resistors (these are not generally \nreprogrammable, and suffer from being large and difficult to fabricate with any accu(cid:173)\nracy), dynamic capacitive storage [4] (this is compact, reprogrammable and simple, \nbut implies an increase in system complexity, arising from off-chip refresh circuitry), \n\n\f764 \n\nA. J. Holmes, A. F. Murray, S. Churcher, J. Hajto, M. J. Rose \n\nEEPROM (\"floating gate\") memory [5] (which is compact, reprogrammable, and \nnon-volatile, but is slow, and cannot be reprogrammed in situ), and local digital \nstorage (which is non-volatile, easily programmable and simple, but consumes area \nhorribly). \nAmorphous silicon has been used for synaptic weight storage [1, 2], but only as \neither a high-resistance fixed weight medium or a binary memory. \nIn this paper, we demonstrate that novel amorphous silicon memory devices can be \nincorporated into standard CMOS synapse circuits, to provide an analogue weight \nstorage mechanism which is compact, non-volatile, easily reprogrammable, and sim(cid:173)\nple to implement. \n\n2 a-Si:H MEMORY DEVICES \n\nThe a-Si:H analogue memory device [3] comprises a lOooA thick layer of amorphous \nsilicon (p+ a-Si:H) sandwiched between Vanadium and Chromium electrodes. \nThe a-Si device takes the form of a two-terminal, programmable resistor. It is an \n\"add-on\" to a conventional CMOS process, and does not demand that the normal \nCMOS fabrication cycle be disrupted. The a-Si device sits on top of the completed \nchip circuitry, making contact with the CMOS arithmetic elements via holes cut in \nthe protective passivation layer, as shown in Figure 1. \n\nCMOS Passivation \n\nFigure 1: The construction of a-Si:H Devices on a CMOS chip \n\nAfter fabrication a number of electronic procedures must be performed in order to \nprogram the device to a given resistance state. \n\nProgramming, and Pre-Programming Procedures \n\nBefore the a-Si device is usable, the following steps must be carried out: \n\n\u2022 Forming: This is a once-only process, applied to the a-Si device in its \n\"virgin\" state, where it has a resistance of several MO. A series of 300ns \npulses, increasing in amplitude from 5v to 14v, is applied to the device \nelectrodes. This creates a vertical conducting channel or filament whose \napproximate resistance is 1KO. This filament can then be programmed to \na value in the range lKO to 1 MO . The details of the physical mechanisms \nare not yet fully established, but it is clear that conduction occurs through \na narrow (sub-micron) conducting channel. \n\n\fPulsestream Synapses with Non-Volatile Analogue Amorphous-Silicon Memories \n\n765 \n\n\u2022 Write: To decrease the device's resistance, negative \"Write\", pulses are \n\napplied. \n\n\u2022 Erase: To increase the device's resistance, positive\" Erase\" , pulses are ap(cid:173)\n\nplied. \n\n\u2022 Usage: Pulses below O.5v do not change the device resistance. The resis(cid:173)\n\ntance can therefore be utilised as a weight storage medium using a voltage \nof less than O.5v without causing reprogramming. \n\nProgramming pulses, which range between 2v and 5v, are typically 120ns in du(cid:173)\nration. Programming is therefore much faster than for other EEPROM (floating \ngate) devices used in the same context, which use a series of 100jls pulses to set the \nthreshold voltage [5]. \nThe following sections describe synapse circuits using the a-Si:H devices. These \nsynapses use the reprogrammable a-Si:H resistor in the place of a storage capacitor \nor EEPROM cell. These new synapses were implemented on a chip referred to as \nASiTEST2, consisting of five main test blocks, each comprising of four synapses \nconnected to a single neuron. \n\n3 The EPSILON based synapse \n\nThe first synapse to be designed used the a-Si:H resistor as a direct replacement for \nthe storage capacitor used in the EPSILON [4] synapse. \n\n+Sv \n\n.. t. Storage 1 I \n\nOriginal \n\nV \n\nNeuron \nCircuitry \n\n> Vw \n\n~:l: Capacitor \n... <0-----------.,... __ \n\nO.5v \n\nEPSILON Synapse \n\nFigure 2: The EPSILON Synapse with a-Si:H weight storage \n\nIn the original EPSILON chip the weight voltage was stored as a voltage on a \ncapacitor. In this new synapse design, shown in Figure 2, the a-Si:H resistance is \nset such that the voltage drop produced by Iset is equivalent to the original weight \nvoltage, Vw, that was stored dynamically on the capacitor. \n\nA new, simpler, synapse, which can be operated from a single +5v supply, was also \nbe included on the ASiTEST2 chip. \n\n\f766 \n\nA. J. Holmes, A. F. Murray, S. Churcher, J. Hajto, M. J. Rose \n\n4 The MkII synapse \n\nThe circuit is shown in Figure 3. The a-Si:H memory is used to store a current, \nIasi. This current is subtracted from a zero current, Isy...:z\" to give a weight current \n, +/-Iw, which adds or subtracts charge from the activity capacitor, Cact, thus \nimplementing excitation or inhibition respectively. \n\nFor the circuit to function correctly we must limit the voltage on the activity ca(cid:173)\npacitor to the range [1.5v,3.5v], to ensure that the transistors mirroring Isy_z and \nIasi remain in saturation. As Figure 3 shows, there are few reference signals and \nthe circuit operates from a single +5v power supply rail, in sharp contrast to many \nearlier analogue neural circuits, including our own. \n\n+5v PWm 1 v\"\" \n\nII . .r--\\. \n~ 881 ....L \n\nVsel \n-.L Comparator \n\n* Cact \n\nPWout \n..rL \n\nVramp ~ \n\nOv \n\nE \nMirror Set \n\n;. \"'E~-----------:~~ E \n\nSynapse \n\n;. \n\nNeuron \n\nPower Supplies \n\nV5_0=5.Ov \n\nReferences \nVrstv\u00b72.5v \nIsy_z=5uA \n\nTail Currents \n\nIneu=4uA \n\nFigure 3: The MkII synapse \n\nOn first inspection the main drawback of this design would appear to be a reliance \non the accuracy with which the zero current Isy...:z, is mirrored across an entire chip. \nThe variation in this current means that two cells with the same synapse resistance \ncould produce widely differing values of Iw. However, during programming we \ndo not use the resistance of the a-Si:H device as a target value. We monitor the \nvoltage on Cact for a given PWin signal, increasing or decreasing the resistance \nof the a-Si:H device until the desired voltage level is achieved. \nExample: To set a weight to be the maximum positive value, we adjust the a-Si \nresistance until a PWin signal of 5us, the maximum input signal, gives a voltage of \n3.5v on the integration capacitor. \nWe are able to set the synapse weight using the whole integration range of [1.5v,3.5v] \nby only closing V sel for the desired synapse during programming. In normal op(cid:173)\nerating mode all four Vsel switches will be closed so that the integration charge is \nsummed over all four local capacitors. \n\n\fPulsestream Synapses with Non-Volatile Analogue Amorphous-Silicon Memories \n\n767 \n\n4.1 Example - Stability Test \n\nAs an example of the use of integration voltage as means of monitoring the resistance \nof a particular synapse we have included a stability test. This was carried out on \none of the test chips which contained the MkII synapse. \nThe four synapses on the test chip were programmed to give different levels of \nactivation. The chip was then powered up for 30mins each day during a 7-day \nperiod, and the activation levels for each synapse were measured three times. \n\n3.5 ,----~-_._--;--__..___r...:....,.-_._,.__-,.--.,..,.-__..__:___., \n\nStability Test - PWin = 3us \n\ntestl \n\ntest2 \n\ntest3 \n\ntest4 \n\ntestS \n\ntest6 \n\ntest7 \n\n3 \n\nt:'\" \n~ \n-~ \n1:1 \n\u2022\n\n't \n\nI \n\n\u2022\n\n\u00b7 \n,\n. \n\u2022 \n\n\u2022 \nI \nI \n\u2022 \n\u2022 \n\u2022 \nI \n\n, \n\n\u2022 ~. \n\n25 \n\u2022 \n\n- ~ -:- - ~ -:- -\n\n-:- - - . of - ~-:- - {,o-:- - -\n\n. , . . . . \n. \n\u2022 \n\n. -. . \n- - ~ - - ~ - - --:- - - ~ - - -~- - - w. ----r s2 \n2 ~ - - ~ - - -~- - - ~ - - L ---L- --~ --~ -s3 \n\n-- - .;. '\" - -: -011>- - :.. ~ - ~ -oGii - -:- -\n\ni-- ~ -\n\n..; -sl \n\n\u00b7 \n\n. \n\n. \n\n. \n\n-\n\n-\n\n\u2022 \n\n. \n\u2022 \n\n, \n, \n, \n, \nI \n\u2022 \n: \n\n\u2022 \n\n. \n\u2022 \n\n\u2022 \n\u2022 \n\u2022 \nI \n\u2022 \nI \n\u2022 \n\n' \n. \n\n\u2022 \n\u2022 \n\u2022 \n\u2022 \n\u2022 \nI \n: \n\nI \n, \nI \n, \nI \n\u2022 \n\u2022 \n\nI \n\n. \n\u2022 \n~-s4 \n. \n\u2022 \n\n\u2022 \n\u2022 \n\u2022 \n\u2022 \n\u2022 \n\u2022 \n\u2022 \n\nI \n\n\u2022 \n\n. \n\u2022 \n\n, \n\u2022 \n\u2022 \n, \n\u2022 \n\u2022 \n\u2022 \n\n10 \n\n20 \n\n30 \n\n40 \n\n50 \n\n60 \n\n70 \n\n80 \n\n90 \n\nMeasurement Index \n\nFigure 4: ASiTEST2- Stability Test \n\nAs figure 4 shows, the memories remain in the same resistance state (i.e retain their \nprogrammed weight value) over the whole 7-day period. Separate experiments on \nisolated devices indicate much longer hold times - of the order of months at least. \n\n5 ASiTEST3 \n\nRecently we have received our latest, overtly neural, a-Si:H based test chip. This \ncontains an 8x8 array of the MkII synapses. \n\nThe circuit board for this device has been constructed and partially tested while \nthe ASiTEST3 chips are awaiting the deposition of the a-Si:H layers. We have been \nable to use an ASiTEST2 chip containing two of the MkII synapse test blocks i.e. \n8 synapses and 2 neurons to exercise much of the board's functionality. \n\nThe test board contains a simple state machine which has four different states: \n\n\u2022 State 0: Load Input Pulsewidths into SRAM from PC. \n\u2022 State 1: Apply Input Pulsewidth signals to chipl. \n\u2022 State 2: Use Vramp to generate threshold function for chipl. The resulting \nPulsewidth outputs are used as the inputs to chip2, as well as being stored \n\n\f768 \n\nA. J. Holmes, A. F. Murray, S. Churcher, J. Hajto, M. J. Rose \n\nin SRAM . \n\n\u2022 State 3: Use Vramp to generate threshold function for chip2. Read resulting \n\nPulsewidth Outputs into SRAM . \n\n\u2022 State 0: Read Output Pulsewidths from SRAM into PC. \n\nThe results obtained during a typical test cycle are shown in Figure 5. \n\nIE-- Statel --;,,;,,*1 EE-- State2 -----;>~i~ State3 ---;!>~I \n~r-IF~~~--r-------~---------l \n4v \n3v \n2v \nIv \nOv~ . . . . . . . . . \n\nPWin_O \n\n3.5v r;;;;;\"\"-~,\"\",,,or;:::::;;:;::;;;;:;;;::;;t---~----t--------, \n\n~:~ ....... ~;a~;\"\"\"'\" ~'Sig~;,id\"\"\"\"\" ... ~\"Li~\"\"\"\" ...... \n\n....................... ,. \n\n....... . \n\n.... ..... .......... \n\n.. .. \n\n2.Ov \nl.~ \n\ne \u00b7_ e _____ . . . . . . ___ \u2022 __ __ _ \n\n\u2022 ____________ . . . . . . . . _____ \u2022 \u2022 \u2022 \u2022 ____ \n\n. . . . . . . . . . . \n\n5v \n4v \n3v \n2v \n:;; ~,...,. ............... -..--t; \u2022. ~ . ..:... ~ __ --! .......... ~~.n-.-~~~~~~......J \n\n10. \n\nIS. \n\nFigure 5: ASiTEST3 Board Scope Waveforms \n\nAs this figure shows different ramp signals, corresponding to different threshold \nfunctions, can be applied to chipl and chip2 neurons. \n\n10.0 .----.,..----r------.----r----.------, \n\nSingle Buffer PulscWidth Sweeps \n\n9.0 \n\n8.0 \n\n! 7.0 - ~ \ni6.o~ \n\n~ 5'O~~~~~~~-~~-+++~~~~N~~~ \n\nJ:: \n\n2D \n\n~----\n\n1.0 \n\no \n\no.oL---~--~- --~--\u00ad\n\n0.5 \n\n1.0 \n\n1.5 \n\nNeal-Syal \nN~~JIII \nN~~ya2 \n\n2.0 \n\n2.5 \n\n3.0 \n\nPulscwldlh Input [WI) \n\nFigure 6: ASiTEST3 Board - MkII Synapse Characteristic \n\nWhile the signals shown in Figure 5 appear noisy the multiplier characteristic that \nthe chip produces is still admirably linear, as shown in Figure 6. In this experiment \nall eight synapses on a test chip were programmed into different resistance states \nand PWin was swept from 0 to 3us. \n\n\fPulsestream Synapses with Non- Volatile Analogue Amorphous-Silicon Memories \n\n769 \n\n6 Conclusions \n\nWe have demonstrated the use of novel a-Si:H analogue memory devices as a means \nof storing synaptic weights in a Pulsewidth ANN. We have also demonstrated the \noperation of an interface board which allows two 8x8 ANN chips, operating as a \ntwo layer network, to be controlled by a simple PC interface card. \n\nThis technology is most suitable for small networks in, for example, remote con(cid:173)\ntrol and other embedded-system applications where cost and power considerations \nfavour a single all-inclusive ANN chip with non-volatile, but programmable weights. \nAnother possible application of this technology is in large networks constructed \nusing Thin Film Technology(TFT). If TFT's were used in place of the CMOS tran(cid:173)\nsistors then the area constraint imposed by crystalline silicon would be removed, \nallowing truly massively parallel networks to be integrated. \n\nIn summary - the a-Si:H analogue memory devices described in this paper provide a \nroute to an analogue, non-volatile and fast synaptic weight storage medium. At the \npresent time neither the programming nor storage mechanisms are fully understood \nmaking it difficult to compare this new device with more established technologies \nsuch as the ubiquitous Floating-Gate EEPROM technique. Current research is \nfocused on firstly, improving the yield on the a-Si:H device which is unacceptably \nlow at present, a demerit that we attribute to imperfections in the a-Si fabrication \nprocess and secondly, improving understanding of the device physics and hence the \nprogramming and storage mechanisms. \n\nAcknowledgements \n\nThis research has been jointly funded by BT, and EPSRC (formerly SERC), the \nEngineering and Physical Sciences Research Council. \n\nReferences \n\n[1] W. Hubbard et al.(1986) Electronic Neural Networks AlP Conference Proceed(cid:173)\n\nings - Snowbird 1986 :227-234 \n\n[2] H.P. Graf (1986) VLSI Implementation of a NN memory with several hundreds \n\nof neurons AlP Conference Proceedings - Snowbird 1986 :182-187. \n\n[3] M.J. Rose et al (1989) Amorphous Silicon Analogue Memory Devices Journal \n\nof Non-Crystalline Solids 1(115):168-170 \n\n[4] A.Hamilton et al. (1992) Integrated Pulse-Stream Neural Networks - Results, \n\nIssues and Pointers IEEE Transactions on N.N.s 3(3):385-393 \n\n[5] M.Holler, S.Tam, H.Castro and R.Benson (1989) An Electrically Trainable ANN \n\nwith 10240 Floating Gate Synapses. Int Conf on N.N.s Proc :191-196 \n\n[6] A.F.Murray and A.V.W.Smith.(1987) Asynchronous Arithmetic for VLSI Neu(cid:173)\n\nral Systems. Electronics Letters 23(12):642-643 \n\n[7] A.J. Holmes et al. (1993) Use of a-Si:H Memory Devices for Non-volatile Weight \n\nStorage in ANNs. Proc lCAS 15 :817-820 \n\n\f\f", "award": [], "sourceid": 1006, "authors": [{"given_name": "A.", "family_name": "Holmes", "institution": null}, {"given_name": "Alan", "family_name": "Murray", "institution": null}, {"given_name": "Stephen", "family_name": "Churcher", "institution": null}, {"given_name": "J.", "family_name": "Hajto", "institution": null}, {"given_name": "M.", "family_name": "Rose", "institution": null}]}