{"title": "Direction Selective Silicon Retina that uses Null Inhibition", "book": "Advances in Neural Information Processing Systems", "page_first": 756, "page_last": 763, "abstract": null, "full_text": "Direction Selective Silicon Retina \n\nthat uses N uIl Inhibition \n\nRonald G. Benson and Tobi Delbriick \n\nComputation and Neural Systems Program, 139-74 \n\nCalifornia Institute of Technology \n\nemail: benson@cns.caltech.edu and tdelbruck@caltech.edu \n\nPasadena CA 91125 \n\nAbstract \n\nBiological retinas extract spatial and temporal features in an attempt to \nreduce the complexity of performing visual tasks. We have built and tested \na silicon retina which encodes several useful temporal features found in ver(cid:173)\ntebrate retinas. The cells in our silicon retina are selective to direction, \nhighly sensitive to positive contrast changes around an ambient light level, \nand tuned to a particular velocity. Inhibitory connections in the null di(cid:173)\nrection perform the direction selectivity we desire. This silicon retina is \non a 4.6 x 6.8mm die and consists of a 47 x 41 array of photoreceptors. \n\n1 \n\nINTRODUCTION \n\nThe ability to sense motion in the visual world is essential to survival in animals. \nVisual motion processing is indispensable; it tells us about predators and prey, our \nown motion and image stablization on the retina. Many algorithms for performing \nearly visual motion processing have been proposed [HK87] [Nak85]. A key salient \nfeature of motion is direction selectivity, ie the ability to detect the direction of \nmoving features. We have implemented Barlow and Levick's model, [BHL64], which \nhypothesizes inhibition in the null direction to accomplish direction selectivity. \n\nIn contrast to our work, Boahen, [BA91], in these proceedings, describes a silicon \nretina that is specialized to do spatial filtering of the image. Mahowald, [Mah91], \ndescribes a silicon retina that has surround interactions and adapts over mulitiple \ntime scales. Her silicon retina is designed to act as an analog preprocessor and \n\n756 \n\n\fDirection Selective Silicon Retina that uses Null Inhibition \n\n757 \n\nPixels inhibit to the left \n\nPreferred \n.Null ~ \n\nPreferred dPrection \n\n~ Photoreceptor \nL~ DS cell \n.. Inhibition \n\n(b) \n\n(a) \n\nFigure 1: Barlow and Levick model of direction selectivity (DS). (a) Shows \nhow two cells are connected in an inhibitory fashion and (b) a mosaic of such \ncells. \n\nso the gain of the output stage is rather low. In addition there is no rectification \ninto on- and off-pathways. This and earlier work on silicon early vision systems \nhave stressed spatial processing performed by biological retinas at the expense of \ntemporal processing. \n\nThe work we describe here and the work described by Delbriick, [DM9l], emphasizes \ntemporal processing. Temporal differentiation and separation of intensity changes \ninto on- and off-pathways are important computations performed by vertebrate \nretinas. Additionally, specialized vertebrate retinas, [BHL64], have cells which are \nsensitive to moving stimuli and respond maximally to a preferred direction; they \nhave almost zero response in the opposite or null direction. We have designed and \ntested a silicon retina that models these direction selective velocity tuned cells. \nThese receptors excite cells which respond to positive contrast changes only and \nare selective for a particular direction of stimuli. Our silicon retina may be useful \nas a preprocessor for later visual processing and certainly as an enhancement for \nthe already existing spatial retinas. It is a striking demonstration of the perceptual \nsaliency of contrast changes and directed motion in the visual world. \n\n2 \n\nINHIBITION IN THE NULL DIRECTION \n\nBarlow and Levick, [BHL64]' described a mechanism for direction selectivity found \nin the rabbit retina which postulates inhibitory connections to achieve the desired \ndirection selectivity. Their model is shown in Figure l(a). As a moving edge \npasses over the photoreceptors from left to right, the left photoreceptor is excited \nfirst, causing its direction selective (DS) cell to fire. The right photoreceptor fires \nwhen the edge reaches it and since it has an inhibitory connection to the left DS \ncell, the right photoreceptor retards further output from the left DS cell. If an edge \nis moving in the opposite or null direction (right to left), the activity evoked in the \nright photoreceptor completely inhibits the left DS cell from firing, thus creating a \ndirection selective cell. \n\n\f758 \n\nBenson and Delbriick \n\nInhibition to left \n\nInhibition from right \n\nIr \nQ \n\nr Preferred Direction \n\n~ \n\nPhotoreceptor \n\nDS cell \n\nFigure 2: Photoreceptor and direction selective (DS) cell. The output of the \nhigh-gain, adaptive photoreceptor is fed capacitively to the input of the DS \ncell. The output of the photoreceptor sends inhibition to the left. Inhibition \nfrom the right photoreceptors connect to the input of the DS cell. \n\nIn the above explanation with the edge moving in the preferred direction (left to \nright), as the edge moves faster, the inhibition from leading photoreceptors truncates \nthe output of the DS cell ever sooner. In fact, it is this inhibitory connection which \nleads to velocity tuning in the preferred direction. \nBy tiling these cells as shown in Figure l(b), it is possible to obtain an array of \ndirectionally tuned cells. This is the architecture we used in our chip. Direction \nselectivity is inherent in the connections of the mosaic, ie the hardwiring of the \ninhibitory connections leads to directionally tuned cells. \n\n3 PIXEL OPERATION \n\nA pixel consists of a photoreceptor, a direction selective (DS) cell and inhibition to \nand from other pixels as shown in Figure 2. The photoreceptor has high-gain and \nis adaptive [Mah91, DM91]. The output from this receptor, Vp , is coupled into the \nDS cell which acts as a rectifying gain element, [MS91], that is only sensitive to \npositive-going transitions due to increases in light intensity at the receptor input. \nAdditionally, the output from the photoreceptor is capacitively coupled to the in(cid:173)\nhibitory synapses which send their inhibition to the left and are coupled into the \nDS cell of the neighboring cells. \n\nA more detailed analysis of the DS cell yields several insights into this cell's func(cid:173)\ntionality. A step increase of 6. V at Vp , caused by a step increase in light intensity \nincident upon the phototransistor, results in a charge injection of Cc6. V at Vi. This \ncharge is leaked away by QT at a rate IT, set by voltage VT. Hence, to first order, \nthe output pulse width T is simply \n\nT = Cc6.V. \n\nIT \n\nThere is also a threshold minimum step input size that will result in enough change \n\n\fDirection Selective Silicon Retina that uses Null Inhibition \n\n759 \n\n1.6 \n\n-.. 1.2 \n> \n'-' \n~ 0.8 \n<Il = 0 c. 0.4 \n\n<Il \n~ \n~ \n\nOutput \n\n0.0 \n\nInput intensity \n\n0 40 80 120 160 200 \n\nTime (msec) \n\nFigure 3: Pixel response to intensity step. Bottom trace is intensity; top trace \nis pixel output. \n\nin Vi to pull Vout all the way to ground. This threshold is set by Cc and the gain of \nthe photoreceptor. \n\nWhen the input to the rectifying gain element is not a step, but instead a steady \nincrease in voltage, the current lin flowing into node Vi is \n\nlin = CcVp. \n\nWhen this current exceeds IT there is a net increase in the voltage Vi, and the \noutput Vout will quickly go low. The condition lin = IT defines the threshold \ninput stimuli resu~ting in an lin < IT are not \nlimit for stimuli detection, i.e. \nperceptible to the pixel. For a changing intensity I, the adaptive photoreceptor \nstage outputs a voltage Vp proportional to j / I, where I is the input light intensity. \nThis photoreceptor behavior means that the pixel threshold will occur at whatever \nj / I causes Cc Vp to exceed the constant current I r. \nThe inhibitory synapses (shown as Inhibition from right in Figure 2) provide addi(cid:173)\ntionalleakage from Vi resulting in a shortened response width from the DS cell. \nThis analysis suggests that a characterization of the pixel should investigate both \nthe response amplitude, measured as pulse width versus input intensity step size, \nand the response threshold, measured with temporal intensity contrast. In the next \nsection we show such measurements. \n\n4 CHARACTERIZATION OF THE PIXEL \n\nWe have tested both an isolated pixel and a complete 2-dimensional retina of 47 x 41 \npixels. Both circuits were fabricated in a 2J.tm p-well CMOS double poly process \navailable through the MOSIS facility. The retina is scanned out onto a monitor using \na completely integrated on-chip scanner[MD91]. The only external components are \na video amplifier and a crystal. \n\nWe show a typical response of the isolated pixel to an input step of intensity in \nFigure 3. In response to the input step increase of intensity, the pixel output goes \nlow and saturates for a time set by the bias Vr in Figure 2. Eventually the pixel \nrecovers and the output returns to its quiescent level. \nIn response to the step \ndecrease of intensity there is almost no response as seen in Figure 3. \n\n\f760 \n\nBenson and Delbriick \n\nU-16O \nIII \nrIJ \n!,120 \n..c= \n.... \nbe 80 = III -III \nrIJ 40 -::l \n\nA.. \n\n~~ \n/ \n\n1.8 \n\n2.2 \n\nStep Contrast \n\n(a) \n\nTemporal Frequency (Hz) \n\n(b) \n\nFigure 4: (a) Pulse width of response as function of input contrast step size. \nThe abscissa is measured in units of ratio-intensity, i.e., a value of 1 means \nno intensity step, a value of 1.1 means a step from a normalized intensity of 1 \nto a normalized intensity of 1.1, and so forth. The different curves show the \nresponse at different absolute light levels; the number in the figure legend is \nthe log of the absolute intensity. (b) Receptor threshold measurements. At \neach temporal frequency, we determined the minimum necessary amplitude of \ntriangular intensity variations to make the pixel respond. The different curves \nwere taken at different background intensity levels, shown to the left of each \ncurve. For example, the bottom curve was taken at a background level of 1 \nunit of intensity; at 8 Hz, the threshold occurred at a variation of 0.2 units of \nintensity. \n\nThe output from the pixel is essentially quantized in amplitude, but the resulting \npulse has a finite duration related to the input intensity step. The analysis in \nSection 3 showed that the output pulse width, T, should be linear in the input \nintensity contrast step. In Figure 4{ a), we show the measured pulse-width as a \nfunction of input contrast step. To show the adaptive nature of the receptor, we \ndid this same measurement at several different absolute intensity levels. \n\nOur silicon retina sees some features of a moving image and not others. Detection \nof a moving feature depends on its contrast and velocity. To characterize this \nbehavior, we measured a receptor's thresholds for intensity variations, as a function \nof temporal frequency. \n\nThese measurements are shown in Figure 4(b); the curves define \"zones of visibil(cid:173)\nity\"; if stimuli lie below a curve, they are visible, if they fall above a curve they \nare not. (The different curves are for different absolute intensity levels.) For low \ntemporal frequencies stimuli are visible only if they are high contrast; at higher \ntemporal frequencies, but still below the photoreceptor cutoff frequency, lower con(cid:173)\ntrast stimuli are visible. Simply put, if the input image has low contrast and is \nslowly moving, it is not seen. Only high contrast or quickly moving features are \nsalient stimuli. More precisely, for temporal frequencies below the photoreceptor \ncutoff frequency, the threshold occurs at a constant value of the temporal intensity \ncontrast j / I. \n\n\fDirection Selective Silicon Retina that uses Null Inhibition \n\n761 \n\nPreferred \nfN uu-\n\nPhotoreceptors \n\nExcitatio \n\n(a) \n\nPreferred \n\nL \nR \nInhib \n\n\"'----- DS \n\n---- Inhib \n\n' - - - - - DS \n\nL \nR \n\n-0.1 sec \n\n(b) \n\nFigure 5: (a) shows the basic connectivity of the tested cell. (b) top trace is \nthe response due to an edge moving in the preferred direction (left to right). \n(b) bottom trace is the response due to an edged moving in the null direction \n(right to left). \n\n5 NULL DIRECTION INHIBITION PROPERTIES \n\nWe performed a series of tests to characterize the inhibition for various orientations \nand velocities. The data in Figure 5(b) shows the outputs of two photo receptors, \nthe inhibitory signal and the output of a DS cell. The top panel in Figure 5(b) shows \nthe outputs in the preferred direction and the bottom panel shows them in the null \ndirection. Notice that the out pu t of the left photoreceptor (L in Figure 5 (b) top \npanel) precedes the right (R). The output of the DS cell is quite pronounced, but is \ntruncated by the inhibition from the right photoreceptor. On the other hand, the \nbottom panel shows that the output of the DS cell is almost completely truncated \nby the inhibitory input. \n\nA DS cell receives most inhibition when the stimulus is travelling exactly in the null \ndirection. As seen in Figure 6(a) as the angle of stimulus is rotated, the maximum \nresponse from the DS cell is obtained when the stimulus is moving in the preferred \ndirection (directly opposite to the null direction). As the bar is rotated toward the \nnull direction, the response of the cell is reduced due to the increasing amount of \ninhibition received from the neighboring photo receptors. \nIf a bar is moving in the preferred direction with varying velocity, there is a velocity, \nVmaz , for which the DS cell responds maximally as shown in Figure 6(b). As the \nbar is moved faster than Vmaz , inhibition arrives at the cell sooner, thus truncating \nthe response. As the cell is moved slower than V maz, less input is provided to the \nDS cell as described in Section 3. In the null direction (negative in Figure 6(b\u00bb) the \ncell does not respond, as expected, until the bar is travelling fast enough to beat \nthe inhibition's onset (recall delay from Figure 5). \n\nIn Figure 7 we show the response of the entire silicon retina to a rotating fan. When \nthe fan blades are moving to the left the retina does not respond, but when moving \nto the right, note the large response. Note the largest response when the blades are \nmoving exactly in the preferred direction. \n\n\f762 \n\nBenson and Delbruck \n\n160 \n\n-;;-120 \n8 \n\n~ \n\nQ) \nrn \n\u00a7 80 \n0.. \n~ \n~ 40 \n\n-0.8 \n\n-0.4 \n\n0.8 \nVelocity (arbitrary units) \n\n0.0 \n\n0.4 \n\n(a) \n\n(b) \n\n(a) polar plot which shows the pixels are directionally tuned. \n\nFigure 6: \n(b) shows velocity tuning of the DS cell (positive velocities are in the pre(cid:173)\nferred direction). \n\n(a) \n\n(b) \n\nFigure 7: (a) Rotating fan used as stimulus to the retina. (b) Output of the \nretina. \n\n\fDirection Selective Silicon Retina that uses Null Inhibition \n\n763 \n\n6 CONCLUSION \n\nWe have designed and tested a silicon retina that detects temporal changes in an \nimage. The salient image features are sufficiently high contrast stimuli, relatively \nfast increase in intensity (measured with respect to the recent past history of the \nintensity), direction and velocity of moving stimuli. These saliency measures result \nin a large compression of information, which will be useful in later processing stages. \n\nAcknowledgments \n\nOur thanks to Carver Mead and John Hopfield for their guidance and encourage(cid:173)\nment, to the Office of Naval Research for their support under grant NAV N00014-\n89-J-1675, and, of course, to the MOSIS fabrication service. \n\nReferences \n\n[BA91] K. Boahen and A. Andreou. A contrast sensitive silicon retina with re(cid:173)\n\nciprocal synapses. In S. Hanson J. Moody and R. Lippmann, editors, \nAdvances in Neural Information Processing Systems, Volume 4. Morgan \nKaufmann, Palo Alto, CA, 1991. \n\n[BHL64] H.B. Barlow, M.R. Hill, and W.R. Levick. Retinal ganglion cells respond(cid:173)\ning selectively to direction and speed of image motion in the rabbit. J. \nPhysiol., 173:377-407, 1964. \n\n[DM91] T. Delbriick and Carver Mead. Silicon adaptive photoreceptor array that \nIn Proc. SPIE 1541, volume \n\ncomputes temporal intensity derivatives. \n1541-12, pages 92-99, San Diego, CA, July 1991. Infrared Sensors: De(cid:173)\ntectors, Electronics, and Signal Processing. \n\n[HK87] E. Hildreth and C. Koch. The analysis of visual motion: From computa(cid:173)\ntional theory to neuronal mechanisms. Annual Review in Neuroscience, \n10:477-533, 1987. \n\n[Mah91] M.A. Mahowald. Silicon retina with adaptive photoreceptor. In SPIE \nTechnical Symposia on Optical Engineering and Photonics in Aerospace \nSensing, Orlando, FL, April 1991. Visual Information Processing: From \nNeurons to Chips. \n\n[MD91] C.A. Mead and T. Delbriick. Scanners for use in visualizing analog VLSI \ncircuitry. Analog Integrated Circuits and Signal Processing, 1:93-106, 1991. \n[MS91] C.A. Mead and R. Sarpeshkar. An axon circuit. Internal Memo, Physics \n\nof Computation Laboratory, Caltech, 1991. \n\n[Nak85] K. Nakayama. Biological image motion processing: A review. Vision \n\nResearch, 25(5):625-660, 1985. \n\n\f", "award": [], "sourceid": 528, "authors": [{"given_name": "Ronald", "family_name": "Benson", "institution": null}, {"given_name": "Tobi", "family_name": "Delbr\u00fcck", "institution": null}]}