{"title": "Range Image Restoration Using Mean Field Annealing", "book": "Advances in Neural Information Processing Systems", "page_first": 594, "page_last": 601, "abstract": null, "full_text": "594 \n\nRange Image Restoration \n\nusing Mean Field Annealing \n\nGriff L. Bilbro \n\nWesley E. Snyder \n\nCenter for Communications and Signal Processing \n\nNorth Carolina State University \n\nRaleigh, NC \n\nAbstract \n\nA new optimization strategy, Mean Field Annealing, is presented. \nIts application to MAP restoration of noisy range images is derived \nand experimentally verified. \n\n1 \n\nIntroduction \n\nThe application which motivates this paper is image analysis; specifically the anal(cid:173)\nysis of range images. We [BS86] [GS87] and others [YA85][BJ88] have found that \nsurface curvature has the potential for providing an excellent, view-invariant fea(cid:173)\nture with which to segment range images. Unfortunately, computation of curvature \nrequires, in turn, computation of second derivatives of noisy data. \nWe cast this task as a restoration problem: Given a measurement g(z, y), we assume \nthat g(z, y) resulted from the addition of noise to some \"ideal\" image fez, y) which \nwe must estimate from three things: \n\n1. The measurement g(z, y). \n2. The statistics of the noise, here assumed to be zero mean with variance (1'2. \n3. Some a priori knowledge of the smoothness of the underlying surface(s). \n\nWe will turn this restoration problem into a minimization, and solve that mini(cid:173)\nmization using a strategy called Mean Field A nnealing. A neural net appears to be \nthe ideal architecture for the reSUlting algorithm, and some work in this area has \nalready been reported [CZVJ88]. \n\n2 \n\nSimulated Annealing and Mean Field Anneal-\n\u2022 Ing \n\nThe strategy of SSA may be summarized as follows: \nLet H(f) be the objective function whose minimum we se~k, wher~ /is somt' pa(cid:173)\nrameter vector. \nA parameter T controls the algorithm. The SSA algorithm begins at a relatively \nhigh value of T which is gradually reduced. Under certain conditions, SSA will \nconverge to a global optimum, [GGB4] [RS87] \n\nH (f) = min{ H (fie)} V fie \n\n(1) \n\n\fRange Image Restoration Using Mean Field Annealing \n\n595 \n\neven though local minima may occur. However, SSA suffers from two drawbacks: \n\n\u2022 It is slow, and \n\u2022 there is no way to directly estimate [MMP87] a continuously-valued I or its \n\nderivatives. \n\nThe algorithm presented in section 2.1 perturbs (typically) a single element of fat \neach iteration. In Mean Field Annealing, we perturb the entire vector f at each \niteration by making a deterministic calculation which lowers a certain average of \nH, < H(f) >, at the current temperature. We thus perform a rather conventional \nnon-linear minimization (e.g. gradient descent), until a minimum is found at that \ntemperature. We will refer to the minimization condition at a given T as the \nequilibrium for that T. Then, T is reduced, and the previous equilibrium is used as \nthe initial condition for another minimization. \nMFA thus converts a hard optimization problem into a sequence of easier problems. \nIn the next section, we justify this approach by relating it to SSA. \n\n2.1 Stochastic Simulated Annealing \nThe problem to be solved is to find j where j minimizes H(f). SSA solves this \nminimization with the following strategy: \n\n1. Define PT ex e- H / T . \n2. Find the equilibrium conditions on PT, at the current temperature, T. By equi(cid:173)\n\nlibrium, we mean that any statistic ofpT(f) is constant. These statistics could \nbe derived from the Markov chain which SSA constructs: jO, p, ... , IN, ... , al(cid:173)\nthough in fact such statistical analysis is never done in normal running of an \nSSA algorithm. \n\n3. Reduce T gradually. \n4. As T --+ 0, PT(f) becomes sharply peaked at j, the minimum. \n\n2.2 Mean Field Annealing \n\nIn Mean Field Annealing, we provide an analytic mechanism for approximating the \nequilibrium at arbitrary T. In MFA, we define an error function, \n\nEMF(Z, T) = Tln--=-H- - + \n\n-H \n\nfe--ordl \nf eTdl \n\n-Hfl \n\nfe T (H-Ho)dl \n- --. \n\n- / j ---\n\nf e- TdJ \n\nwhich follows from Peierl's inequality [BGZ76]: \n\nF ~ Fo+ < H - Ho > \n\n(2) \n\n(3) \n\nwhere F = -Tlnf e---r-dl and Fo = -Tlnf e T dl . The significance of EMF is as \nfollows: the minimum of EMF determines the best approximation given the form \n\n-H \n\n-Hg \n\n\f596 \n\nBilbro and Snyder \n\nof Ho to the equilibrium statistics of the SSA-generated MRF at T. We will then \nanneal on T. In the next section, we choose a special form for Ho to simplify this \nprocess even further. \n\n1. Define some Ho(f, z) which will be used to estimate H(f). \n2. At temperature T, minimize EMF(Z) where EMF is a functional of Ho and \nH which characterizes the difference between Ho and H. The process of \nminimizing EMF will result in a value of the parameter z, which we will \ndenote as ZT. \n\n3. Define HT(f) = Ho(f, ZT) and for(f) ex e- iiT /T. \n\n3 \n\nImage Restoration Using MFA \n\nWe choose a Hamiltonian which represents both the noise in the image, and our a \npriori knowledge of the local shape of the image data. \n\nHN = L.J -2 2 (Ii - gil \n2 \n\n\"\" 1 \n, \n(1' \n\n\u2022 \n\n(4) \n\n(5) \n\nwhere 18( represents [Bes86] the set of values of pixels neighboring pixel i (e.g. the \nvalue of I at i along with the I values at the four nearest neighbors of i); A is some \nscalar valued function of that set of pixels (e.g. the 5 pixel approximation to the \nLaplacian or the 9 pixel approximation to the quadratic variation); and \n\n(6) \n\nThe noise term simply says that the image should be similar to the data, given noise \nof variance (1'2. The prior term drives toward solutions which are locally planar. Re(cid:173)\ncently, a simpler V(z) = z2 and a similar A were successfully used to design a neural \nnet [CZVJ88] which restores images consisting of discrete, but 256-valued pixels. \nOur formulation of the prior term emphasizes the importance of \"point processes,\" \nas defined [WP85] by Wolberg and Pavlidis. While we accept the eventual necessity \nof incorporating line processes [MMP87] [Mar85] [GG84] [Gem87] into restoration, \nour emphasis in this paper is to provide a rigorous relationship between a point \nprocess, the prior model, and the more usual mathematical properties of surfaces. \nUsing range imagery in this problem makes these relationships direct. By adopting \nthis philosophy, we can exploit the results of Grimson [Gri83] as well as those of \nBrady and Horn [BH83] to improve on the Laplacian. \nThe Gaussian functional form of V is chosen because it is mathematically conve(cid:173)\nnient for Boltzmann statistics and beca.use it reflects the following shape properties \nrecommended for grey level images in the literature and is especially important if \n\n\fRange Image Restoration Using Mean Field Annealing \n\n597 \n\nline processes are to be omitted: Besag [Bes86] notes that lito encourage smooth \nvariation\", V(A) \"should be strictly increasing\" in the absolute value of its argu(cid:173)\nment and if \"occasional abrupt changes\" are expected, it should \"quickly reach a \nmaximum\" . \nRational functions with shapes similar to our V have been used in recent stochastic \napproaches to image processing [GM85]. In Eq. 6, T is a \"soft threshold\" which \nrepresents our prior knowledge of the probability of various values of \\7 2 f (the \nLaplacian of the undegraded image). For T large, we imply that high values of \nthe Laplacian are common - f is highly textured; for small values of T, we imply \nthat f is generally smooth. We note that for high values of T, the prior term is \ninsignificant, and the best estimate of the image is simply the data. \nWe choose the Mean Field Hamiltonian to be \n\nand find that the optimal ZT approximately minimizes \n\n(7) \n\n(8) \n\nboth at very high and very low T . We have found experimentally that this approx(cid:173)\nimation to ZT does anneal to a satisfactory restoration. At each temperature, we \nuse gradient descent to find ZT with the following approximation to the gradient of \n: \n\n(9) \n\nand \n\n-b \n\nV(r\u00b7) -\n\n, - y'2;(T+T) \n\n.. ? \n\ne- 2( .. h) \n. \n\nDifferentiating Eq. 8 with this new notation, we find \n\nSince 6'+11,; is non-zero only when i + v = i, we have \n\n8 < H > _ :J!j - gj L L \n\n2 + \n\n-\n\n(1' \n\n8 \n\n:J! . \n) \n\nII \n\nIT'(. ) \n\n')+11 \n\n-II \n\nand this derivative can be used to find the equilibrium condition. \n\nAlgorithm \n\n(lO) \n\n(11) \n\n(12) \n\n\f598 \n\nBilbro and Snyder \n\n1. Initially, we use the high temperature assumption, which eliminates the prior \n\nterm entirely, and results in \n\nZ; =g;; for \n\nT = 00. \n\n(13) \n\nThis will provide the initial estimate of z. Any other estimate quickly con(cid:173)\nverges to g. \n2. Given an image z;, form the image ri = (L \u00ae z);, where the \u00ae indicates \n\nconvolution. \n\nP \n\n, \n\n..? \n\n~T+T)T+T \n\n3. Create the image V. = V' (r\u00b7) = - -----l= _-.!'L e - :II(T~ .. ) \u2022 \n4. Using 12, perform ordinary non-linear minimization of < H > starting from \nthe current z. The particular strategy followed is not critical. We have \nsuccessfully used steepest descent and more sophisticated conjugate gradi(cid:173)\nent [PFTV88] methods. The simpler methods seem adequate fot Gaussian \nnoise. \n\n5. Update z to the minimizing z found in step 4. \n6. Reduce T and go to 2. When T is sufficiently close to 0, the algorithm is \n\ncomplete. \n\nIn step 6 above, T essentially defines the appropriate low-temperature stopping \npoint. In section 5, we will elaborate on the determination of T and other such \nconstants. \n\n4 Performance \n\nIn this section, we describe the performance of the algorithm as it is applied to \nseveral range images. We will use range images, in which the data is of the form \n\nz = z(z, y). \n\n(14) \n\n4.1 \n\nImages With High Levels of Noise \n\nFigure 1 illustrates a range image consisting of three objects, a wedge (upper left), \na cylinder with rounded end and hole (right), and a trapezoidal block viewed from \nthe top. The noise in this region is measured at (1' = 3units out of a total range of \nabout 100 units. Unsophisticated smoothing will not estimate second derivatives of \nsuch data without blurring. Following the surface interpolation literature, [Gri83] \n[BB83] we use the quadratic variation as the argument of the penalty function for \nthe prior term to \n\n(15) \n\nand performing the derivative in a manner analogous to Eq. 11 and 12. The \nLaplacian of the restoration is shown in Figure 2. Figure 3 shows a cross-section \ntaken as indicated by the red line on Figure 2. \n\n\fFig. 1 Original rallge image \n\nFig. 2 Laplacian of the restored image \n\nI \n\nn \n\nJ~~ l\\~lll \n\nFig. 3 Cross section \nThrough Laplacian along \nRed Line \n\n4.2 Comparison With Existing Techniques \n\nAccurate computation of surface derivatives requires extremely good smoothing of \nsurface noise, while segmentation requires preservat.ion of edges. One suc.h adapt.ive \nsmoothing technique,[Ter87] iterative Gaussian smoothing (IGS) has been success(cid:173)\nfully applied to range imagery. [PB87] Following this strategy, step edges are first \ndetected, and smoothing is then applied using a small center-weighted kernel. At \nedges, an even smaller kernel, called a \"molecule\", is used to smooth right up to \nthe edge without blurring the edge. The smoothing is then iterated. \n\n\f600 \n\nBilbro and Snyder \n\nThe results, restoration and Laplacian, of IGS are not nearly as sharp as those \nshown in Figure 2. \n\n5 Determining the Parameters \n\nAlthough the optimization strategy described in section 3 has no hard thresholds, \nseveral parameters exist either explicitly in Eq. 8 or implicitly in the iteration. \nGood estimates of these parameters will result in improved performance, faster \nconvergence, or both. The parameters are: \n\n(1' the standard deviation of the noise \nb the relative magnitude of the prior term \n11 = T + T the initial temperature and \nT the final temperature. \n\nThe decrement in T which defines the annealing schedule could also be considered \na parameter. However, we have observed that 10% or less per step is always good \nenough. \nWe find that for depth images of polyhedral scenes, T = 0 so that only one parameter \nis problem dependent: (1'. For the more realistic case of images which also contain \ncurved surfaces, however, see our technical report [BS88], which also describes the \nMFA derivation in much more detail. \nThe standard deviation of the noise must be determined independently for each \nIt is straightforward to estimate (1' to within 50%, and we have \nproblem class. \nobserved experimentally that performance of the algorithm is not sensitive to this \norder of error. \nWe can analytically show that annealing occurs in the region T:::::: IL12(1'2 and choose \nTJ = 2ILI2(1'2. Here, ILI2 is the squared norm of the operator Land ILI2 = 20 for \nthe usual Laplacian and ILI2 = 12.5 for the quadratic variation. \nFurther analysis shows that b = .J2;ILI(1' is a good choice for the coefficient of the \nprior term. \n\nReferences \n\n[Bes86] \n\nJ. Besag. On the statistical analysis of dirty pictures. Journal of the \nRoyal Stati6ticCJl Society, B 48(3), 1986. \n\n[BGZ76] E. Brezin, J. C. Le Guillon, and J. Zinn-Justin. Field theoretical ap(cid:173)\n\nproach to critical phenomena. In C. Domb and M.S. Green, editors, \nPhCJ6e Tran6ition6 and Critical Phenomena, chapter 3, Academic Press, \nNew York, 1976. \nM. Brady and B.K.P Horn. Symm~tric operators for surface interpola(cid:173)\ntion. CVGIP, 22, 1983. \nP.J. Besl and R.C. Jain. Segmentation through variable-order surface \nfitting. IEEE PAMI, 10(2), 1988. \n\n[BH83] \n\n[BJ88] \n\n\fRange Image Restoration Using Mean Field Annealing \n\n601 \n\n[BS86] \n\n[BS88] \n\nG. Bilbro and W. Snyder. A Linear Time Theory for Recognizing Sur(cid:173)\nface6 in 3-D. Technical Report CCSP-NCSU TR-86/8, Center for Com(cid:173)\nmunications and Signal Processing, North Carolina State University, \n1986. \nG. L. Bilbro and W. E. Snyder. Range Image Re6toration U6ing Mean \nField Annealing. Technical Report NETR 88-19, Available from the \nCenter for Communications and Signal Processing, North Carolina State \nUniversity, 1988. \n\n[CZVJ88] R. Chellappa, Y.-T. Zhou, A. Vaid, and B. K. Jenkins. Image restoration \nusing a neural network. IEEE Tran6action6 on ASSP, 36(7):1141-1151, \nJuly 1988. \n\n[Gem87] D. Geman. Stochastic model for boundary detection. Vi6ion and Image \n\n[GG84] \n\n[GM85] \n\nComputing, 5(2):61-65, 1987. \nD. Geman and S. Geman. Stochastic relaxation, Gibbs Distributions, \nand the Bayesian restoration of images. IEEE Tran6action6 on PAMI, \nPAMI-6(6):721-741, November 1984. \nS. Geman and D.E. McClure. Bayesian image analysis: an application \nto single photon emission tomography. Proceeding6 of the American \nStati6tical A uociation, Stati6tical Computing Section, 12-18, 1985. \n\nrGri83] W.E.L. Grimson. An implementation of computational theory of visual \n\n[GS87] \n\n[Mar85] \n\nsurface interpolation. CVGIP, 22, 1983. \nB. R. Groshong and W. E. Snyder. Range image segmentation for object \nrecognition. In 18th Pitt6burgh Conference on Modeling and Simulation, \nPittsburgh, PA, April 1987. \nJ .L. Marroquin. Probabili6tic Solution to Inver6e Problem6. PhD thesis, \nM.LT, Cambridge, MA, September 1985. \n\n[MMP87] J. Marroquin, S. Mitter, and T. Poggio. Probabilistic solution of ill(cid:173)\n\n[PB87] \n\nposed problems in computational vision. Journal of American Stati6tical \nAuociation, 82(397):76-89, March 1987. \nT. Ponce and M. Brady. Toward a surface primal sketch. In T. Kanade, \neditor, Three Dimen6ional Machine Vi6ion, Kluwer Press, 1987. \n\n[PFTV88] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling. \n\n[RS87] \n\n[Ter87] \n\n[WP85] \n\n[YA85] \n\nNumerical Recipe6 in C. Cambridge University Press, 1988. \nF. Romeo and A. Sangiovanni-Vencentelli. Probabalistic hill climbing \nalgorithms: properties and applications. In Chapel Hill Conference on \nVLSI, Computer Science Press, Chapel Hill, NC, 1987. \nD. Terzopoulos. The role of constraints and discontiuities in visible(cid:173)\nsurface reconstruction. \npages 1073-1077, 1987. \nG. Wolberg and T. Pavlidis. Restoration of binary images using stochas(cid:173)\ntic relaxation with annealing. Pattern Recognition Letter6, 3(6):375-388, \nDecember 1985. \nM. Brady A. Yiulle and H. Asada. Describing surfaces. CVGIP, August \n1985. \n\nIn Proc. of 7th International Conf. on AI, \n\n\f", "award": [], "sourceid": 153, "authors": [{"given_name": "Griff", "family_name": "Bilbro", "institution": null}, {"given_name": "Wesley", "family_name": "Snyder", "institution": null}]}