{"title": "An Analog VLSI Chip for Thin-Plate Surface Interpolation", "book": "Advances in Neural Information Processing Systems", "page_first": 687, "page_last": 694, "abstract": null, "full_text": "AN ANALOG VLSI CHIP FOR \n\nTHIN-PLATE SURFACE INTERPOLATION \n\n687 \n\nJohn G. Harris \n\nCalifornia Institute of Technology \n\nComputation and Neural Systeins Option, 216-76 \n\nPasadena, CA 91125 \n\nABSTRACT \n\nReconstructing a surface from sparse sensory data is a well-known \nproblem iIi computer vision. This paper describes an experimental \nanalog VLSI chip for smooth surface interpolation from sparse depth \ndata. An eight-node ID network was designed in 3J.lm CMOS and \nsuccessfully tested. The network minimizes a second-order or \"thin(cid:173)\nplate\" energy of the surface. The circuit directly implements the cou(cid:173)\npled depth/slope model of surface reconstruction (Harris, 1987). In \naddition, this chip can provide Gaussian-like smoothing of images. \n\nINTRODUCTION \n\nReconstructing a surface from sparse sensory data is a well-known problem in \ncomputer vision. Early vision modules typically supply sparse depth, orientation, \nand discontinuity information. The surface reconstruction module incorporates \nthese sparse and possibly conflicting measurements of a surface into a consistent, \ndense depth map. \n\nThe coupled depth/slope model provides a novel solution to the surface reconstruc(cid:173)\ntion problem (Harris, 1987). A ID version of this model has been implemented; \nfortunately, its extension to 2D is straightforward. Figure 1 depicts a high-level \nschematic of the circuit. The di voltages represent noisy and possibly sparse input \ndata, the ZiS are the smooth output values, and the PiS are the explicitly computed \nslopes. The vertical data resistors (with conductance g) control the confidence in \nthe input data. In the absence of data these resistors are open circuits. The \nhorizontal chain of smoothness resistors of conductance ..\\ forces the derivative of \nthe data to be smooth. This model is called the coupled depth/slope model be(cid:173)\ncause of the coupling between the depth and slope representations provided by the \nsubtractor elements. The subtractors explicitly calculate a slope representation \nof the surface. Any depth or slope node can be made into a constraint by fixing \na voltage source to the proper location in the network. Intuitively, any sudden \nchange in slope is smoothed out with the resistor mesh. \n\n\f688 \n\nHarris \n\nFigure 1. The coupled depth/slope model. \n\nThe tri-directional subtractor device (shown in Figure 2) is responsible for the \ncoupling between the depth and slope representations. If nodes A and B are set \nwith ideal voltage sources, then node C will be forced to A - B by the device. This \ncircuit element is unusual in that all of its terminals can act as inputs or outputs. \nIf nodes Band C are held constant with voltage sources, then the A terminal is \nfixed to B + C. If A and C are input, then B becomes A - C. When further \nconstraints are added, this device dissipates a power proportional to (A - B - C)2. \nIn the limiting case of a continuous network, the total dissipated power is \n\n(1) \n\nThe three terms arise from the power dissipated in the sub tractors and in the two \ndifferent types of resistors. Energy minimization techniques and standard calculus \nof variations have been used to formally show that the reconstructed surfaces, z, \nsatisfy the 1D biharmonic equation between input data points (Harris, 1987). In \nthe tw~dimensional formulation, z is a solution of \n\n(2) \n\nThis interpolant, therefore, provides the same results as minimizing the energy of \na thin plate, which has been commonly used in surface reconstruction algorithms \non digital computers (Grimson, 1981; Terzopoulos, 1983). \n\nIMPLEMENTATION \n\nThe eight-node 1D network shown in Figure 1 was designed in 3J.lm CMOS (Mead, \n1988) and fabricated through MOSIS. Three important components of the model \nmust be mapped to analog VLSI: the two different types of resistors and the sub(cid:173)\ntractors. The vertical confidence resistors are built with simple transconductance \n\n\fAn Analog VLSI Chip for Thin-Plate Surface Interpolation \n\n689 \n\nB \n\nA \n\nc \n\nFigure 2. Tri-directional subtract constraint device. \n\namplifiers (transamps) connected as followers. The bias voltage of the transamp \nfollower determines its conductance (g) and therefore signifies the certainty of \nthe data. If there are no data for a given location, the corresponding transamp \nfollower is turned off. The horizontal smoothness resistors are implemented with \nMead's saturating resistor (Mead, 1988). Since conventional CMOS processes lack \nadequate resistive elements, we are forced to build resistors out of transistor el(cid:173)\nements. The bias voltage for Mead's resistor allows the effective conductance of \nthese circuit elements to vary over many orders of magnitude. \n\nThe most difficult component to implement in analog VLSI is the subtract con(cid:173)\nstraint device. Its construction led to a general theory of constraint boxes which \ncan be used to implement all sorts of constraints which are useful in early vi(cid:173)\nsion (Harris, 1988). The implementation of the subtract constraint device is a \nstraightforward application of constraint box theory. Figure 3 shows a generic n \nterminal constraint box enforcing a constraint F on its voltage terminals. The \nconstraints are enforced by generating a feedback current lie for each constrained \nvoltage terminal. Suppose F can be written as \n\nOne possible feedback equation which implements this constraint is given by \n\n8F \n1.: = -F -\n8Vl: \n\n(4) \n\nWhen this particular choice of feedback current is used, the constraint box min(cid:173)\nimizes the least-squares error in the constraint equation (Harris, 1989). Notice \nthat F can be scaled by any arbitrary scaling factor. This scaling factor and the \ncapacitance at each node determine the speed of convergence of a single constraint \nbox. \n\n\f690 \n\nHarris \n\nV \nt-------i~-.. k \n\nFigure 3. Generic n terminal constraint box. \n\nThe subtract constraint box given in Figure 2 requires a constraint of A - B = C, \nwhich leads to the following error equation: \n\nF(A,B,C) = A- B - C \n\nStraightforward application of constraint box theory yields \n-F 8A = -(A - B - C) \n\n8F \n\n1B \n\n-F~ = (A- B - C) \n\nIe \n\n-\n\n-F :~ = (A - B - C) \n\n(5) \n\n(6) \n\nwhere lA, I B , and Ie represent feedback currents that must be generated by the \ndevice. \n\nThese current feedback equations can be implemented with two modified wide(cid:173)\nrange transamps (see Figure 4). In its linear range, a single transamp produces \na current proportional to the difference of its two inputs. The negative input \nto each transamp is indicated by an inverting circle. The transamps have been \nmodified to produce four outputs, two positive and two negative. The negative \noutputs are also represented by inverting circles. Because the difference terminal \nC can be positive or negative, it is measured with respect to a voltage reference \nVREF. VREF is a global signal which defines zero slope. As seen in Figure 4, the \n\n\fAn Analog VLSI Chip for Thin-Plate Surface Interpolation \n\n691 \n\nA \n\nB \n\nI----V \n\nREF \nc \n\nIB \n\nIe \n\n~ ________________ -J \n\nFigure 4. Tri-directional subtract constraint box. \n\nproper combination of positive and negative outputs from the two transamps are \nfed back to the voltage terminals to implement the feedback equations given in \neq. (6). \n\nAnalog networks which solve most regularizable early vision problems can be de(cid:173)\nsigned with networks consisting solely of linear resistances and batteries (Poggio \nand Koch, 1985). Unfortunately, many times these networks contain negative re(cid:173)\nsistances that are troublesome to implement in analog VLSI. For example, the \ncircuit shown in Figure 5 computes the same solutions as the coupled depth/slope \nnetwork described in this paper. \nInterestingly, a 2-D implementation of this \nidea was implemented in the 1960s using inductors and capacitors (Volynskii and \nBukhman, 1965). Proper choice of the frequency of alternating current allowed the \ncircuit elements to act as pure positive and negative impedances. Unfortunately, \nnegative resistances are troublesome to implement, especially in analog VLSI. \nOne of the big advantages of using constraint boxes to implement early vision \nalgorithms is that the resulting networks do not require negative resistances. \n\nANALYSIS \n\nFigure 6 shows a sample output of the circuit. Data (indicated by vertical dashed \nlines) were supplied at nodes 2, 5, and 8. As expected, the chip finds a smooth \nsolution (solid line) which extrapolates beyond the known data points. It is well(cid:173)\nknown that a single resistive grid minimizes the first-order or membrane energy of \na surface. Luo, Koch, and Mead (1988) have implemented a 48x48 resistive grid \nto perform surface interpolation. Figure 6 also shows the simulated performance \nof a first-order energy or membrane energy minimization. Data points are again \nsupplied at nodes 2, 5, and 8. In contrast to the second-order chip results, the \nsolution (dashed line) is much more jagged and does not extrapolate outside of \n\n\f692 \n\nHarris \n\n\u2022 \n\ng \n\n-R \n\n-R \n\n-R \n\n-R \n\n-R \n\n-R \n\nFigure 5. A negative-resistor resistor solution to the ID biharmonic equation. \n\nI .\u2022 \n\n1.7 \n\n-. \n~ 1.6 \n\nI \nI \n\n1.5 \n\n1.4 \n\n.... \n\n.... .... \n\n~ \n\n~ \n\n~ \n\n~ \n\n~ \n\n~ \n\n~ \n\n/ \n~ \n\n~ \n\n/ \n\n/ \n~ \n\n~ \n\n13+------+------+------+------~----~------~----~ \n\u2022 \n\n4 \n\n7 \n\n2 \n\n3 \n\n5 \n\n6 \n\nI \n\nFigure 6. Measured data from the second-order chip (solid line) and simulated \nfirst-order result ( dashed line). \n\n\fAn Analog VLSI Chip for Thin-Plate Surface Interpolation \n\n693 \n\n1.0 \n\n0.' \n\n0.6 \n\n0.4 \n\nG.2 \n\nI \n\nI \n\nI \n\nI \n\nI \n\n/ \n\n/ \n\n/ \n\n, , , , , , , \n\n\" \n\no.o+r-~. ~ .. -.-----.: \n\n.. . ... \n\n. ' . \n\n.... .... ---\n\n.... \n\n... ,. ..... \n\n42+---~~--~----~--~~-~---~--~~--+--~--~ \n\n-5 \n\n\"\" \n\n-3 \n\n-2 \n\n-I \n\n0 \n\n2 \n\n3 \n\n4 \n\n5 \n\nFigure 7. Graphical comparison of ID analytic Green's functions for first-order \n(dashed line), second-order (dotted line) and Gaussian (solid line). \n\nthe known data points (for example, see node 1). Interestingly, psychophysics \nexperiments support the smoother interpolant used by the second-order coupled \ndepth/slope chip (Grimson, 1981). Unlike the second-order network, the first(cid:173)\norder network is not rigid enough to incorporate either orientation constraints or \norientation discontinuities (Terzopoulos, 1983). \n\nImage smoothing is a special case of surface interpolation where the data are \ngiven on a dense grid. The first-order network is a poor smoothing operator. \nA comparison of analytic Green's function of first and second-order networks is \nshown in Figure 7 (the first-order shown with a dashed line and the second(cid:173)\norder with a solid line). Note that the analytic Green's function of the second(cid:173)\norder network (solid line) and that of standard Gaussian convolution (dotted line) \nare nearly identical. This fact was pointed out by Poggio, Voorhees, and Yuille \n(1986), when they suggested the use of the second-order energy to regularize \nthe edge detection problem. Gaussian convolution has been claimed by many \nauthors to be the \"optimal\" smoothing operator and is commonly used as the \nfirst stage of edge detection. Though the second-order network can be used to \nsmooth images, Gaussian convolution cannot be used to solve the more difficult \nproblem of interpolating from sparse data points. \n\n\f694 \n\nHarris \n\nCONCLUSION \n\nBiharmonic surface interpolation has been successfully demonstrated in analog \nVLSI. To test true performance, we plan to combine a larger version of this chip \nwith an analog stereo network. Work has already started on building the necessary \ncircuitry for discontinuity detection during surface reconstruction. The Gaussian(cid:173)\nlike smoothing effect of this network will be further explored through building a \nnetwork with photoreceptors supplying dense data input. \n\nAcknowledgements \n\nSupport for this research was provided by the Office of Naval Research and the \nSystem Development Foundation. The author is a Hughes Aircraft Fellow and \nthanks Christof Koch and Carver Mead for their ongoing support. Additional \nthanks to Berthold Horn for several helpful suggestions. \nReferences \n\nGrimson, W.E.L. From Images to Surfaces, MIT Press, Cambridge, (1981). \n\nHarris, J.G. A new approach to surface reconstruction: the coupled depth/slope \nmodel, Proc. IEEE First Inti. Con! Computer Vision, pp. 277-283, London, \n(1987). \n\nHarris, J.G. Solving early vision problems with VLSI constraint networks, Neural \n\nArchitectures for Computer Vision Workshop, AAAI-88, Minneapolis, Min(cid:173)\nnesota, Aug. 20 (1988). \n\nHarris, J .G. Designing analog constraint boxes to solve energy minimization prob(cid:173)\n\nlems in vision, submitted to INNS Neural Networks Conference, Washington \nD.C., June (1989) \n\nLuo, J., Koch, C., and Mead, C. An experimental subthreshold, analog CMOS tw