{"title": "Multiscale Fields of Patterns", "book": "Advances in Neural Information Processing Systems", "page_first": 82, "page_last": 90, "abstract": "We describe a framework for defining high-order image models that can be used in a variety of applications. The approach involves modeling local patterns in a multiscale representation of an image. Local properties of a coarsened image reflect non-local properties of the original image. In the case of binary images local properties are defined by the binary patterns observed over small neighborhoods around each pixel. With the multiscale representation we capture the frequency of patterns observed at different scales of resolution. This framework leads to expressive priors that depend on a relatively small number of parameters. For inference and learning we use an MCMC method for block sampling with very large blocks. We evaluate the approach with two example applications. One involves contour detection. The other involves binary segmentation.", "full_text": "Multiscale Fields of Patterns\n\nPedro F. Felzenszwalb\n\nBrown University\n\nProvidence, RI 02906\npff@brown.edu\n\nJohn G. Oberlin\nBrown University\n\nProvidence, RI 02906\n\njohn oberlin@brown.edu\n\nAbstract\n\nWe describe a framework for de\ufb01ning high-order image models that can be used\nin a variety of applications. The approach involves modeling local patterns in a\nmultiscale representation of an image. Local properties of a coarsened image re-\n\ufb02ect non-local properties of the original image. In the case of binary images local\nproperties are de\ufb01ned by the binary patterns observed over small neighborhoods\naround each pixel. With the multiscale representation we capture the frequency\nof patterns observed at different scales of resolution. This framework leads to\nexpressive priors that depend on a relatively small number of parameters. For in-\nference and learning we use an MCMC method for block sampling with very large\nblocks. We evaluate the approach with two example applications. One involves\ncontour detection. The other involves binary segmentation.\n\n1\n\nIntroduction\n\nMarkov random \ufb01elds are widely used as priors for solving a variety of vision problems such as\nimage restoration and stereo [5, 8]. Most of the work in the area has concentrated on low-order\nmodels involving pairs of neighboring pixels. However, it is clear that realistic image priors need to\ncapture higher-order properties of images.\nIn this paper we describe a general framework for de\ufb01ning high-order image models that can be used\nin a variety of applications. The approach involves modeling local properties in a multiscale repre-\nsentation of an image. This leads to a natural low-dimensional representation of a high-order model.\nWe concentrate on the problem of estimating binary images. In this case local image properties can\nbe captured by the binary patterns in small neighborhoods around each pixel.\nWe de\ufb01ne a Field of Patterns (FoP) model using an energy function that assigns a cost to each 3x3\npattern observed in an image pyramid. The cost of a pattern depends on the scale where it appears.\nFigure 1 shows a binary image corresponding to a contour map from the Berkeley segmentation\ndataset (BSD) [12, 2] and a pyramid representation obtained by repeated coarsening. The 3x3 pat-\nterns we observe after repeated coarsening depend on large neighborhoods of the original image.\nThese coarse 3x3 patterns capture non-local image properties. We train models using a maximum-\nlikelihood criteria. This involves selecting pattern costs making the expected frequency of patterns\nin a random sample from the model match the average frequency of patterns in the training images.\nUsing the pyramid representation the model matches frequencies of patterns at each resolution.\nIn practice we use MCMC methods for inference and learning. In Section 3 we describe an MCMC\nsampling algorithm that can update a very large area of an image (a horizontal or vertical band of\npixels) in a single step, by combining the forward-backward algorithm for one-dimensional Markov\nmodels with a Metropolis-Hastings procedure.\nWe evaluated our models and algorithms on two different applications. One involves contour detec-\ntion. The other involves binary segmentation. These two applications require very different image\npriors. For contour detection the prior should encourage a network of thin contours, while for bi-\n\n1\n\n\f(a)\n\n(b)\n\n(c)\n\nFigure 1: (a) Multiscale/pyramid representation of a contour map. (b) Coarsest image scaled up\nfor better visualization, with a 3x3 pattern highlighted. The leftmost object in the original image\nappears as a 3x3 \u201ccircle\u201d pattern in the coarse image. (c) Patches of contour maps (top) that coarsen\nto a particular 3x3 pattern (bottom) after reducing their resolution by a factor of 8.\n\nnary segmentation the prior should encourage spatially coherent masks. In both cases we can design\neffective models using maximum-likelihood estimation.\n\n1.1 Related Work\n\nFRAME models [24] and more recently Fields of Experts (FoE) [15] de\ufb01ned high-order energy\nmodels using the response of linear \ufb01lters. FoP models are closely related. The detection of 3x3\npatterns at different resolutions corresponds to using non-linear \ufb01lters of increasing size. In FoP we\nhave a \ufb01xed set of pre-de\ufb01ned non-linear \ufb01lters that detect common patterns at different resolutions.\nThis avoids \ufb01lter learning, which leads to a non-convex optimization problem in FoE.\nA restricted set of 3x3 binary patterns was considered in [6] to de\ufb01ne priors for image restoration.\nBinary patterns were also used in [17] to model curvature of a binary shape. There has been recent\nwork on inference algorithms for CRFs de\ufb01ned by binary patterns [19] and it may be possible to\ndevelop ef\ufb01cient inference algorithms for FoP models using those techniques.\nThe work in [23] de\ufb01ned a variety of multiresolution models for images based on a quad-tree rep-\nresentation. The quad-tree leads to models that support ef\ufb01cient learning and inference via dynamic\nprogramming, but such models also suffer from artifacts due to the underlying tree-structure. The\nwork in [7] de\ufb01ned binary image priors using deep Boltzmann machines. Those models are based\non a hierarchy of hidden variables that is related to our multiscale representation. However in our\ncase the multiscale representation is a deterministic function of the image and does not involve extra\nhidden variables as [7]. The approach we take to de\ufb01ne a multiscale model is similar to [9] where\nlocal properties of subsampled signals where used to model curves.\nOne of our motivating applications involves detecting contours in noisy images. This problem has a\nlong history in computer vision, going back at least to [16], who used a type of Markov model for\ndetecting salient contours. Related approaches include the stochastic completion \ufb01eld in [22, 21],\nspectral methods [11], the curve indicator random \ufb01eld [3], and the recent work in [1].\n\n2 Fields of Patterns (FoP)\nLet G = [n] \u00d7 [m] be the grid of pixels in an n by m image. Let x = {x(i, j) | (i, j) \u2208 G} be a\nhidden binary image and y = {y(i, j) | (i, j) \u2208 G} be a set of observations (such as a grayscale or\ncolor image). Our goal is to estimate x from y.\nWe de\ufb01ne a CRF p(x|y) using an energy function that is a sum of two terms,\n\np(x|y) =\n\n1\n\nZ(y)\n\nexp(\u2212E(x, y)) E(x, y) = EFoP(x) + Edata(x, y)\n\n(1)\n\n2\n\n\f2.1 Single-scale FoP Model\n\nThe single-scale FoP model is one of the simplest energy models that can capture the basic properties\nof contour maps or other images that contain thin objects. We use x[i, j] to denote the binary pattern\nde\ufb01ned by x in the 3x3 window centered at pixel (i, j), treating values outside of the image as 0. A\nsingle-scale FoP model is de\ufb01ned by the local patterns in x,\n\nEFoP(x) =\n\nV (x[i, j]).\n\n(2)\n\n(cid:88)\n\n(i,j)\u2208G\n\nHere V is a potential function assigning costs (or energies) to 3x3 binary patterns. The sum is\nover all 3x3 windows in x, including overlapping windows. Note that there are 512 possible binary\npatterns in a 3x3 window. We can make the model invariant to rotations and mirror symmetries\nby tying parameters together. The resulting model has 102 parameters (some patterns have more\nsymmetries than others) and can be learned from smaller datasets. We used invariant models for all\nof the experiments reported in this paper.\n\n2.2 Multiscale FoP Model\n\nTo capture non-local statistics we look at local patterns in a multiscale representation of x. For a\nmodel with K scales let \u03c3(x) = x0, . . . , xK\u22121 be an image pyramid where x0 = x and xk+1 is\na coarsening of xk. Here xk is a binary image de\ufb01ned over a grid Gk = [n/2k] \u00d7 [m/2k]. The\ncoarsening we use in practice is de\ufb01ned by a logical OR operation,\n\nxk+1(i, j) = xk(2i, 2j) \u2228 xk(2i + 1, 2j) \u2228 xk(2i, 2j + 1)k \u2228 xk(2i + 1, 2j + 1)\n\n(3)\n\nThis particular coarsening maps connected objects at one scale of resolution to connected objects at\nthe next scale, but other coarsenings may be appropriate in different applications.\nA multiscale FoP model is de\ufb01ned by the local patterns in \u03c3(x),\n\nEFoP(x) =\n\nV k(xk[i, j]).\n\n(4)\n\nK\u22121(cid:88)\n\n(cid:88)\n\nk=0\n\n(i,j)\u2208Gk\n\nThis model is parameterized by K potential functions V k, one for each scale in the pyramid \u03c3(x).\nIn many applications we expect the frequencies of a 3x3 pattern to be different at each scale. The\npotential functions can encourage or discourage speci\ufb01c patterns to occur at speci\ufb01c scales.\nNote that \u03c3(x) is a deterministic function and the pyramid representation does not introduce new\nrandom variables. The pyramid simply de\ufb01nes a convenient way to specify potential functions over\nlarge regions of x. A single potential function in a multiscale model can depend on a large area of\nx due to the coarsenings. For large enough K (proportional to log of the image size) the Markov\nblanket of a pixel can be the whole image.\nWhile the experiments in Section 5 use the conditional modeling approach speci\ufb01ed by Equation\n(1), we can also use EFoP to de\ufb01ne priors over binary images. Samples from these priors illus-\ntrate the information that is captured by a FoP model, specially the added bene\ufb01t of the multiscale\nrepresentation. Figure 2 shows samples from FoP priors trained on contour maps of natural images.\nThe empirical studies in [14] suggest that low-order Markov models can not capture the empirical\nlength distribution of contours in natural images. A multiscale FoP model can control the size\ndistribution of objects much better than a low-order MRF. After coarsening the diameter of an object\ngoes down by a factor of approximately two, and eventually the object is mapped to a single pixel.\nThe scale at which this happens can be captured by a 3x3 pattern with an \u201con\u201d pixel surrounded by\n\u201coff\u201d pixels (this assumes there are no other objects nearby). Since the cost of a pattern depends on\nthe scale at which it appears we can assign a cost to an object that is based loosely upon its size.\n\n2.3 Data Model\n\nLet y be an input image and \u03c3(y) be an image pyramid computed from y. Our data models are\nde\ufb01ned by sums over pixels in the two pyramids \u03c3(x) and \u03c3(y). In our experiments y is a graylevel\n\n3\n\n\f(a)\n\n(b)\n\n(c)\n\nFigure 2: (a) Examples of training images T extracted from the BSD. (b) Samples from a single-\nscale FoP prior trained on T . (c) Samples from a multiscale FoP prior trained on T . The multiscale\nmodel is better at capturing the lengths of contours and relationships between them.\n\nimage with values in {0, . . . , M \u2212 1}. The pyramid \u03c3(y) is de\ufb01ned in analogy to \u03c3(x) except that\nwe use a local average for coarsening instead of the logical OR,\n\nyk+1(i, j) = (cid:98)(yk(2i, 2j) + yk(2i + 1, 2j) + yk(2i, 2j + 1) + yk(2i + 1, 2j + 1))/4(cid:99)\n\nThe data model is parameterized by K vectors D0, . . . , DK\u22121 \u2208 RM\n\nEdata(x, y) =\n\nxk(i, j)Dk(yk(i, j))\n\nK\u22121(cid:88)\n\n(cid:88)\n\nk=0\n\n(i,j)\u2208Gk\n\n(5)\n\n(6)\n\nHere Dk(yk(i, j)) is an observation cost incurred when xk(i, j) = 1. There is no need to include an\nobservation cost when xk(i, j) = 0 because only energy differences affect the posterior p(x|y).\nWe note that it would be interesting to consider data models that capture complex relationships\nbetween local patterns in \u03c3(x) and \u03c3(y). For example a local maximum in yk(i, j) might give\nevidence for xk(i, j) = 1, or a particular 3x3 pattern in xk[i, j].\n\n2.4 Log-Linear Representation\n\nThe energy function E(x, y) of a FoP model can be expressed by a dot product between a vector of\nmodel parameters w and a feature vector \u03c6(x, y). The vector \u03c6(x, y) has one block for each scale.\nIn the k-th block we have: (1) 512 (or 102 for invariant models) entries counting the number of\ntimes each 3x3 pattern occurs in xk; and (2) M entries counting the number of times each possible\nvalue for y(i, j) occurs where xk(i, j) = 1. The vector w speci\ufb01es the cost for each pattern in each\nscale (V k) and the parameters of the data model (Dk). We then have that E(x, y) = w \u00b7 \u03c6(x, y).\nThis log-linear form is useful for learning the model parameters as described in Section 4.\n\n3\n\nInference with a Band Sampler\n\nIn inference we have a set of observations y and want to estimate x. We use MCMC methods [13]\nto draw samples from p(x|y) and estimate the posterior marginal probabilities p(x(i, j) = 1|y).\nSampling is also used for learning model parameters as described in Section 4.\nIn a block Gibbs sampler we repeatedly update x by picking a block of pixels B and sampling new\nvalues for xB from p(xB|y, xB). If the blocks are selected appropriately this de\ufb01nes a Markov chain\nwith stationary distribution p(x|y).\nWe can implement a block Gibbs sampler for a multiscale FoP model by keeping track of the image\npyramid \u03c3(x) as we update x. To sample from p(xB|y, xB) we consider each possible con\ufb01guration\n\n4\n\n\ffor xB. We can ef\ufb01ciently update \u03c3(x) to re\ufb02ect a possible con\ufb01guration for xB and evaluate the\nterms in E(x, y) that depend on xB. This takes O(K|B|) time for each con\ufb01guration for xB. This\nin turn leads to an O(K|B|2|B|) time algorithm for sampling from p(xB|y, x \u00afB). The running time\ncan be reduced to O(K2|B|) using Gray codes to iterate over con\ufb01gurations for xB.\nHere we de\ufb01ne a band sampler that updates all pixels in a horizontal or vertical band of x in a single\nstep. Consider an n by m image x and let B be a horizontal band of pixels with h rows. Since\n|B| = mh a straightforward implementation of block sampling for B is completely impractical.\nHowever, for an Ising model we can generate samples from p(xB|y, xB) in O(m22h) time using\nthe forward-backward algorithm for Markov models. We simply treat each column of B as a single\nvariable with 2h possible states. A similar idea can be used for FoP models.\nLet S be a state space where a state speci\ufb01es a joint con\ufb01guration of binary values for the pixels in\na column of B. Note that |S| = 2h. Let z1, . . . , zm be a representation of xB in terms of the state\nof each column. For a single-scale FoP model the distribution p(z1, . . . , zn|y, x \u00afB) is a 2nd-order\nMarkov model. This allows for ef\ufb01cient sampling using forward weights computed via dynamic\nprogramming. Such an algorithm takes O(m23h) time to generate a sample from p(xB|y, xB),\nwhich is ef\ufb01cient for moderate values of h.\nIn a multiscale FoP model the 3x3 patterns in the upper levels of \u03c3(x) depend on many columns of\nB. This means p(z1, . . . , zn|x \u00afB) is no longer 2nd-order. Therefore instead of sampling xB directly\nwe use a Metropolis-Hastings approach. Let p be a multiscale FoP model we would like to sample\nfrom. Let q be a single-scale FoP model that approximates p. Let x be the current state of the\nMarkov chain and x(cid:48) be a proposal generated by the single-scale band sampler for q. We accept x(cid:48)\nwith probability min(1, ((p(x(cid:48)|y)q(x|y))/(p(x|y)q(x(cid:48)|y)))). Ef\ufb01cient computation of acceptance\nprobabilities can be done using the pyramid representations of x and y. For each proposal we update\n\u03c3(x) to \u03c3(x(cid:48)) and compute the difference in energy due to the change under both p and q.\nOne problem with the Metropolis-Hastings approach is that if proposals are rejected very often the\nresulting Markov chain mixes slowly. We can avoid this problem by noting that most of the work\nrequired to generate a sample from the proposal distribution involves computing forward weights\nthat can be re-used to generate other samples. Each step of our band sampler for a multiscale FoP\nmodel picks a band B (horizontal or vertical) and generates many proposals for xB, accepting each\none with the appropriate acceptance probability. As long as one of the proposals is accepted the\nwork done in computing forward weights is not wasted.\n\n4 Learning\n\nWe can learn models using maximum-likelihood and stochastic gradient descent. This is similar to\nwhat was done in [24, 15, 20]. But in our case we have a conditional model so we maximize the\nconditional likelihood of the training examples.\nLet T = {(x1, yi), . . . , (xN , yN )} be a training set with N examples. We de\ufb01ne the training ob-\njective using the negative log-likelihood of the data plus a regularization term. The regularization\nensures no pattern is too costly. This helps the Markov chains used during learning and inference to\nmix reasonably fast. Let L(xi, yi) = \u2212 log p(xi|yi). The training objective is given by\n\n\u2207O(w) = \u03bbw +\n\n\u03c6(xi, yi) \u2212 Ep(x|yi)[\u03c6(x, yi)].\n\n(8)\nHere Ep(x|yi)[\u03c6(x, yi)] is the expectation of \u03c6(x, yi) under the posterior p(x|yi) de\ufb01ned by the\ncurrent model parameters w. A stochastic approximation to the gradient \u2207O(w) can be obtained\ni from p(x|yi). Let \u03b7 be a learning rate. In each stochastic gradient descent step we\nby sampling x(cid:48)\nsample x(cid:48)\n\nN(cid:88)\ni from p(x|yi) and update w as follows\n\nw := w \u2212 \u03b7(\u03bbw +\n\n\u03c6(xi, yi) \u2212 \u03c6(x(cid:48)\n\ni, yi)).\n\n(9)\n\nO(w) =\n\nThis objective is convex and\n\nL(xi, yi).\n\n(7)\n\nN(cid:88)\n\ni=1\n\n||w||2 +\n\n\u03bb\n2\n\nN(cid:88)\n\ni=1\n\ni=1\n\n5\n\n\fTo sample the x(cid:48)\ni we run N Markov chains, one for each training example, using the band sampler\nfrom Section 3. After each model update we advance each Markov chain for a small number of steps\nusing the latest model parameters to obtain new samples x(cid:48)\ni.\n\n5 Applications\n\nTo evaluate the ability of FoP to adapt to different problems we consider two different applications.\nIn both cases we estimate hidden binary images x from grayscale input images y. We used ground-\ntruth binary images x from standard datasets and synthetic observations y. For the experiments\ndescribed here we generated y by sampling a value y(i, j) for each pixel independently from a\nnormal distribution with standard deviation \u03c3y and mean \u00b50 or \u00b51, depending on x(i, j),\n\ny(i, j) \u223c N (\u00b5x(i,j), \u03c32\ny).\n\n(10)\n\nWe have also done experiments with more complex observation models but the results we obtained\nwere similar to the results described here.\n\n5.1 Contour Detection\n\nThe BSD [12, 2] contains images of natural scenes and manual segmentations of the most salient\nobjects in those images. We used one manual segmentation for each image in the BSD500. From\neach image we generated a contour map x indicating the location of boundaries between segments.\nTo generate the observations y we used \u00b50 = 150, \u00b51 = 100 and \u03c3y = 40 in Equation (10). Our\ntraining and test sets each have 200 examples. We \ufb01rst trained a 1-scale FoP model. We then trained\na 4-level FoP model using the 1-level model as a proposal distribution for the band sampler (see\nSection 3). Training each model took 2 days on a 20-core machine. During training and testing\nwe used the band sampler with h = 3 rows. Inference involves estimating posterior probabilities\nfor each pixel by sampling from p(x|y). Inference on each image took 20 minutes on an 8-core\nmachine.\nFor comparison we implemented a baseline technique using linear \ufb01lters. Following [10] we used the\nsecond derivative of an elongated Gaussian \ufb01lter together with its Hilbert transform. The \ufb01lters had\nan elongation factor of 4 and we experimented with different values for the base standard deviation\n\u03c3b of the Gaussian. The sum of squared responses of both \ufb01lters de\ufb01nes an oriented energy map. We\nevaluated the \ufb01lters at 16 orientations and took the maximum response at each pixel. We performed\nnon-maximum suppression along the dominant orientations to obtain a thin contour map.\nFigure 3 illustrates our results on 3 examples from the test set. Results on more examples are\navailable in the supplemental material. For the FoP models we show the posterior probabilities for\neach pixel p(x(i, j) = 1|y). The darker pixels have higher posterior probability. The FoP models\ndo a good job suppressing noise and localizing the contours. The multiscale FoP model in particular\ngives fairly clean results despite the highly noisy inputs. The baseline results at lower \u03c3b values\nsuffer from signi\ufb01cant noise, detecting many spurious edges. The baseline at higher \u03c3b values\nsuppresses noise at the expense of having poor localization and missing high-curvature boundaries.\nFor a quantitative evaluation we compute precision-recall curves for the different models by thresh-\nolding the estimated contour maps at different values. Figure 4 shows the precision-recall curves.\nThe average precision (AP) was found by calculating the area under the precision-recall curves. The\n1-level FoP model AP was 0.73. The 4-level FoP model AP was 0.78. The best baseline AP was\n0.18 obtained with \u03c3b = 1. We have also done experiments using lower observation noise levels \u03c3y.\nWith low observation noise the 1-level and 4-level FoP results become similar and baseline results\nimprove signi\ufb01cantly approaching the FoP results.\n\n5.2 Binary Segmentation\n\nFor this experiment we obtained binary images from the Swedish Leaf Dataset [18]. We focused on\nthe class of Rowan leaves because they have complex shapes. Each image de\ufb01nes a segmentation\nmask x. To generate the observations y we used \u00b50 = 150, \u00b51 = 100 and \u03c3y = 100 in Equation\n(10). We used a higher \u03c3y compared to the previous experiment because the 2D nature of masks\nmakes it possible to recover them under higher noise. We used 50 examples for training and 25\n\n6\n\n\fx\np\na\nm\n\nr\nu\no\nt\nn\no\nC\n\ny\nn\no\ni\nt\na\nv\nr\ne\ns\nb\nO\n\n1\n=\n\nb\n\u03c3\ne\nn\ni\nl\ne\ns\na\nB\n\n4\n=\n\nb\n\u03c3\ne\nn\ni\nl\ne\ns\na\nB\n\n1\nP\no\nF\n\n4\nP\no\nF\n\nFigure 3: Contour detection results. Top-to-bottom: Hidden contour map x, input image y, output\nof oriented \ufb01lter baseline with \u03c3b = 1 and \u03c3b = 4, output of 1-level and 4-level FoP model.\n\nexamples for testing. We trained FoP models with the same procedure and parameters used for the\ncontour detection experiment. For a baseline, we used graph-cuts [5, 4] to perform MAP inference\nwith an Ising model. We set the data term using our knowledge of the observation model and picked\nthe pairwise discontinuity cost minimizing the per-pixel error rate in the test set.\nFigure 5 illustrates the results of the different methods. Results on other images are available in the\nsupplemental material. The precision-recall curves are in Figure 4. Graph-cuts yields a precision-\nrecall point, with precision 0.893 and recall 0.916. The 1-level FoP model has a higher precision\nof 0.915 at the same recall. The 4-level FoP model raises the precision to 0.929 at the same recall.\n\n7\n\n\f(a) Contour detection\n\n(b) Binary segmentation\n\nFigure 4: (a) Precision-recall curves for the contour detection experiment. (b) Precision-recall curves\nfor the segmentation experiment (the graph-cuts baseline yields a single precision-recall point).\n\nMask x\n\nObservation y\n\nGraph-cuts\n\nFoP 1\n\nFoP 4\n\nFigure 5: Binary segmentation examples. The 4-level FoP model does a better job recovering pixels\nnear the object boundary and the stem of the leaves.\n\nThe differences in precision are small because they are due to pixels near the object boundary but\nthose are the hardest pixels to get right. In practice the 4-level FoP model recovers more detail when\ncompared to graph-cuts. This can be seen by visual inspection of the leaf stem and boundaries.\n\n6 Conclusion\n\nWe described a general framework for de\ufb01ning high-order image models. The idea involves mod-\neling local properties in a multiscale representation of an image. This leads to a natural low-\ndimensional parameterization for high-order models that exploits standard pyramid representations\nof images. Our experiments demonstrate the approach yields good results on two applications that\nrequire very different image priors, illustrating the broad applicability of our models. An interesting\ndirection for future work is to consider FoP models for non-binary images.\n\nAcknowledgements\n\nWe would like to thank Alexandra Shapiro for helpful discussions and initial experiments related to\nthis project. This material is based upon work supported by the National Science Foundation under\nGrant No. 1161282.\n\n8\n\n\fReferences\n[1] S. Alpert, M. Galun, B. Nadler, and R. Basri. Detecting faint curved edges in noisy images. In ECCV,\n\n2010.\n\n[2] Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. Contour detection and hierarchical\n\nimage segmentation. PAMI, 33(5):898\u2013916, May 2011.\n\n[3] J. August and S.W. Zucker. Sketches with curvature: The curve indicator random \ufb01eld and markov\n\nprocesses. PAMI, 25(4):387\u2013400, April 2003.\n\n[4] Y. Boykov and V. Kolmogorov. An experimental comparison of min-cut/max-\ufb02ow algorithms for energy\n\nminimization in vision. PAMI, 26(9):1124\u20131137, Sep 2004.\n\n[5] Y. Boykov, O. Veksler, and R.Zabih. Ef\ufb01cient approximate energy minimization via graph cuts. PAMI,\n\n20(12):1222\u20131239, Nov 2001.\n\n[6] X. Descombes, J.F. Mangin, E. Pechersky, and M. Sigelle. Fine structures preserving markov model for\n\nimage processing. In SCIA, 1995.\n\n[7] S.M. Eslami, N. Heess, and J. Winn. The shape boltzmann machine: a strong model of object shape.\n\nCVPR, 2012.\n\n[8] P. F. Felzenszwalb and D. P. Huttenlocher. Ef\ufb01cient belief propagation for early vision.\n\n2006.\n\nIJCV, 70(1),\n\n[9] P. F. Felzenszwalb and J. Schwartz. Hierarchical matching of deformable shapes. CVPR, 2007.\n[10] T. Leung and J. Malik. Contour continuity in region-based image segmentation. ECCV, pages 544\u2013559,\n\n1998.\n\n[11] Shyjan Mahamud, Lance R. Williams, Karvel K. Thornber, and Kanglin Xu. Segmentation of multiple\n\nsalient closed contours from real images. PAMI, 25(4):433\u2013444, 2003.\n\n[12] David R. Martin, Charless C. Fowlkes, and Jitendra Malik. Learning to detect natural image boundaries\n\nusing local brightness, color, and texture cues. PAMI, 26(5):530\u2013549, 2004.\n\n[13] R. Neal. Probabilistic inference using markov chain monte carlo methods. Technical Report CRG-TR-\n\n93-1, Computer Science, University of Toronto, 1993.\n\n[14] Xiaofeng Ren, Charless Fowlkes, and Jitendra Malik. Learning probabilistic models for contour comple-\n\ntion in natural images. IJCV, 77(1-3):47\u201363, 2008.\n\n[15] Stefan Roth and Michael J. Black. Fields of experts. IJCV, 82(2):205\u2013229, 2009.\n[16] A. Shashua and S. Ullman. Structural saliency: The detection of globally salient structures using a locally\n\nconnected network. In ICCV, pages 321\u2013327, 1988.\n\n[17] A. Shekhovtsov, P. Kohli, and C. Rother. Curvature prior for mrf-based segmentation and shape inpaint-\n\ning. In DAGM, 2012.\n\n[18] O. J. O. S\u00a8oderkvist. Computer vision classi\ufb01cation of leaves from swedish trees. Master\u2019s thesis,\n\nLink\u00a8oping University, September 2001.\n\n[19] Rustem Takhanov and Vladimir Kolmogorov. Inference algorithms for pattern-based crfs on sequence\n\ndata. In ICML, 2013.\n\n[20] T. Tieleman. Training restricted boltzmann machines using approximations to the likelihood gradient. In\n\nICML, 2008.\n\n[21] Lance R. Williams and David W. Jacobs. Local parallel computation of stochastic completion \ufb01elds.\n\nNeural Computation, 9(4):859\u2013881, 1997.\n\n[22] Lance R. Williams and David W. Jacobs. Stochastic completion \ufb01elds: A neural model of illusory contour\n\nshape and salience. Neural Computation, 9(4):837\u2013858, 1997.\n\n[23] A.S. Willsky. Multiresolution markov models for signal and image processing. Proceedings of the IEEE,\n\n90(8):1396\u20131458, 2002.\n\n[24] S. C. Zhu, Y. N. Wu, and D.B. Mumford. Filters, random \ufb01elds and maximum entropy (FRAME):\n\nTowards a uni\ufb01ed theory for texture modeling. IJCV, 27(2):1\u201320, 1998.\n\n9\n\n\f", "award": [], "sourceid": 88, "authors": [{"given_name": "Pedro", "family_name": "Felzenszwalb", "institution": "Brown University"}, {"given_name": "John", "family_name": "Oberlin", "institution": "Brown University"}]}