{"title": "Message Errors in Belief Propagation", "book": "Advances in Neural Information Processing Systems", "page_first": 609, "page_last": 616, "abstract": null, "full_text": " Message Errors in Belief Propagation\n\n\n\n Alexander T. Ihler, John W. Fisher III, and Alan S. Willsky\n Department of Electrical Engineering and Computer Science\n Massachusetts Institute of Technology\n ihler@mit.edu, fisher@csail.mit.edu, willsky@mit.edu\n\n\n\n\n Abstract\n\n Belief propagation (BP) is an increasingly popular method of perform-\n ing approximate inference on arbitrary graphical models. At times,\n even further approximations are required, whether from quantization or\n other simplified message representations or from stochastic approxima-\n tion methods. Introducing such errors into the BP message computations\n has the potential to adversely affect the solution obtained. We analyze\n this effect with respect to a particular measure of message error, and show\n bounds on the accumulation of errors in the system. This leads both to\n convergence conditions and error bounds in traditional and approximate\n BP message passing.\n\n\n1 Introduction\n\nGraphical models and message-passing algorithms defined on graphs are a growing field of\nresearch. In particular, the belief propagation (BP, or sum-product) algorithm has become\na popular means of solving inference problems exactly or approximately. One part of\nits appeal is its optimality for tree-structured graphical models (models which contain no\nloops). However, its is also widely applied to graphical models with cycles. In these cases\nit may not converge, and if it does its solution is approximate; however in practice these\napproximations are often good. Recently, further justifications for loopy belief propagation\nhave been developed, including a few convergence results for graphs with cycles [13].\n\nThe approximate nature of loopy BP is often a more than acceptable price for efficient in-\nference; in fact, it is sometimes desirable to make additional approximations. There may be\na number of reasons for this--for example, when exact message representation is compu-\ntationally intractable, the messages may be approximated stochastically [4] or determinis-\ntically by discarding low-likelihood states [5]. For BP involving continuous, non-Gaussian\npotentials, some form of approximation is required to obtain a finite parametrization for\nthe messages [68]. Additionally, graph simplification by edge removal may be regarded\nas a coarse form of message approximation. Finally, one may wish to approximate the\nmessages and reduce their representation size for another reason--to decrease the commu-\nnications required for distributed inference applications. In a distributed environment, one\nmay approximate the transmitted message to reduce its representational cost [9], or discard\nit entirely if it is deemed \"sufficiently similar\" to the previously sent version [10]. This may\nsignificantly reduce the amount of communication required.\n\nGiven that message approximation may be desirable, we would like to know what effect\nthe introduced errors have on our overall solution. To characterize the effect in graphs\n\n\f\nwith cycles, we analyze the deviation from a solution given by \"exact\" loopy BP (not, as is\ntypically considered, the deviation of loopy BP from the true marginal distributions). In the\nprocess, we also develop some results on the convergence of loopy BP. Section 3 describes\nthe major themes of the paper; but first we provide a brief summary of belief propagation.\n\n\n2 Graphical Models and Belief Propagation\n\nGraphical models provide a convenient means of representing conditional independence\nrelations among large numbers of random variables. Specifically, each node s in a graph\nis associated with a random variable xs, while the set of edges E is used to describe the\nconditional dependency structure of the variables. A distribution satisfies the conditional\nindependence relations specified by an undirected graph if it factors into a product of poten-\ntial functions defined on the cliques (fully-connected subsets) of the graph; the converse\nis also true if p(x) is strictly positive [11]. Here we consider graphs with at most pairwise\ninteractions (a typical assumption in BP), where the distribution factors according to\n\n p(x) = st(xs, xt) s(xs) (1)\n (s,t)E s\n\nThe goal of belief propagation [12], or BP, is to compute the marginal distribution p(xt) at\neach node t. BP takes the form of a message-passing algorithm between nodes, expressed\nin terms of an update to the outgoing message from each node t to each neighbor s in terms\nof the (previous iteration's) incoming messages from t's neighbors t,\n\n mts(xs) ts(xt, xs)t(xt) mut(xt)dxt (2)\n\n ut\\s\n\nTypically each message is normalized so as to integrate to unity (and we assume that such\nnormalization is possible). At any iteration, one may calculate the belief at node t by\n\n M i(x mi (x\n t t) t(xt) ut t) (3)\n ut\n\nFor tree-structured graphical models, belief propagation can be used to efficiently perform\nexact marginalization. Specifically, the iteration (2) converges in a finite number of itera-\ntions (at most the length of the longest path in the graph), after which the belief (3) equals\nthe correct marginal p(xt). However, as observed by [12], one may also apply belief prop-\nagation to arbitrary graphical models by following the same local message passing rules\nat each node and ignoring the presence of cycles in the graph; this procedure is typically\nreferred to as \"loopy\" BP.\n\nFor loopy BP, the sequence of messages defined\n\n 1\nby (2) is not guaranteed to converge to a fixed 1\n\npoint after any number of iterations. Under rela-\n 3\n 2 2 4 3\ntively mild conditions, one may guarantee the ex-\n\n 4\nistence of fixed points [13]. However, they may 4 2 3 4\nnot be unique, nor are the results exact (the be-\n 1 3 1 1 1 2\nlief M it does not converge to the true marginal).\nIn practice however the procedure often arrives at\na reasonable set of approximations to the correct Figure 1: For a graph with cycles, one\n may show an equivalence between\nmarginal distributions. n it-\n erations of loopy BP and the n-level\nIt is sometimes convenient to think of loopy BP computation tree (shown here for n = 3\nin terms of its computation tree [2]. The n-level and rooted at node 1; example from [2]).\ncomputation tree rooted at some node t is a tree-\nstructured \"unrolling\" of the graph, so that n iterations of loopy BP on the original graph\nis equivalent at the node t to exact inference on the computation tree. An example of this\nstructure is shown in Figure 1.\n\n\f\n m(x) log m/ ^\n m\n\n\n ^\n m(x) } logd(e)\n (a) (b)\n\nFigure 2: (a) A message m(x), solid, and its approximation ^\n m(x), dashed. (b) Their log-ratio\nlog m(x)/ ^\n m(x); log d (e) characterizes their similarity by measuring the error's dynamic range.\n\n\n3 Overview of Results\n\nTo orient the reader, we lay out the order and general results which are obtained in this pa-\nper. We begin by considering multiplicative error functions which describe the difference\nbetween a \"true\" message m(x) (typically meaning consistent with some BP fixed-point)\nand some approximation ^\n m(x) = m(x) e(x). We apply a particular functional measure\nd (e) (defined below) and show how this measure behaves with respect to the BP equa-\ntions (2) and (3). When applied to traditional BP, this results in a novel sufficient condition\nfor its convergence to a unique solution, specifically\n\n d (\n max ut)2 - 1 < 1, (4)\n (s,t)E d (\n u ut)2 + 1\n t \\s\n\nand may be further improved in most cases. The condition (4) is shown to be slightly\nstronger than the sufficient condition given in [2]. More importantly, however, the method\nin which it is derived allows us to generalize to many other situations:\n\n The condition (4) is easily improved for graphs with irregular geometry or potential\n strengths\n The method also provides a bound on the distance between any two BP fixed points.\n The same methodology may be applied to the case of quantized or otherwise approx-\n imated messages, yielding bounds on the ensuing error (our original motivation).\n By regarding message errors as a stochastic process and applying a few additional\n assumptions, a similar analysis obtains alternate, tighter estimates (though not\n necessarily bounds) of performance.\n\n4 Message Approximations\n\nIn order to discuss the effects and propagation of errors introduced to the BP messages,\nwe first require a measure of the difference between two messages. Although there are\ncertainly other possibilities, it is very natural to consider the message deviations (which we\ndenote ets) to be multiplicative, or additive in the log-domain, and examine a measure of\nthe error's dynamic range:\n\n ^\n mts(xs) = mts(xs)ets(xs) d (ets) = max ets(a)/ets(b) (5)\n a,b\n\nThen, we have that mts(x) = ^\n mts(x)x if and only if log d (ets) = 0. This measure may\nalso be related to more traditional error measures, including an absolute error on log m(x),\na floating-point precision on m(x), and the Kullback-Leibler divergence D(m(x) ^\n m(x));\nfor details, see [14]. In this light our analysis of message approximation (Section 5.3)\nmay be equivalently regarded as a statement about the required precision for an accurate\nimplementation of loopy BP. Figure 2 shows an example message m(x) and approximation\n^\nm(x) along with their associated error e(x).\n\nTo facilitate our analysis, we split the message update operation (2) into two parts. In the\nfirst, we focus on the message products\n\n Mts(xt) t(xt) mut(xt) Mt(xt) t(xt) mut(xt) (6)\n u u\n t \\s t\n\n\f\nwhere as usual, the proportionality constant is chosen to normalize M . We show the mes-\nsage error metric is (sub-)additive, i.e. that the errors in each incoming message (at most)\nadd in their impact on M . The second operation is the message convolution\n\n mts(xs) ts(xt, xs)Mts(xt)dxt (7)\n\nwhere M is a normalized message or product of messages. We demonstrate a level of\ncontraction, that is, the approximation of mts is measurably better than the approximation\nof Mts used to construct it.\n\nWe use the convention that lowercase quantities (mts, ets, . . .) refer to messages and mes-\nsage errors, while uppercase ones (Mts, Ets, Mt, . . .) refer to products of messages or\nerrors--all incoming messages to node t (Mt and Et), or all except the one from s (Mts\nand Ets). Due to space constraints, many omitted details and proofs can be found in [14].\n\n4.1 Additivity and Error Contraction\n\nThe log of (5) is sub-additive, since for several incoming messages { ^\n mut(x)} we have\n log d (Ets) = log d ^\n Mts/Mts = log d eut log d (eut) (8)\n\nWe may also derive a minimum rate of contraction on the errors. We consider the message\nfrom t to s; since all quantities in this section relate to mts and Mts we suppress the\nsubscripts. The error measure d (e) is given by\n (x (x\n d (e)2 = d ( ^\n m/m)2 = max t, a)M (xt)E(xt)dxt t, b)M (xt)dxt (9)\n a,b (xt, a)M (xt)dxt (xt, b)M (xt)E(xt)dxt\nsubject to certain constraints, such as positivity of the messages and potentials. Since\n f, g > 0, f (x) dx / g(x) dx max f (x)/g(x) (10)\n x\nwe can directly obtain the two bounds:\n d (e)2 d (E)2 and d (e)2 d ()4 (11)\nwhere we have extended the measure d () to\nfunctions of two variables (describing a mini-\nmum rate of mixing across the potential) by log d (E)\n log d ()2\n (a, b) (e)d\n d ()2 = max . (12)\n a,b,c,d (c, d) log log d()2 d(E)+1\n d()2+d(E)\nHowever, with some work one may show [14] the\nstronger measure of contraction, log d (E) \n d ()2 d (E) + 1\n d (e) . (13) Figure 3: Bounds on the error output\n d ()2 + d (E) d (e) as a function of the error in the\nSketch of proof: While the full proof is rather involved, product of incoming messages d (E).\nwe outline the procedure here. First, use (10) to show\nthat the maximum of (9) given d () is attained by potentials of the form (x, a) 1 + KA(x)\nand (x, b) 1 + KB(x), where K = d ()2 - 1 and A and B take on only values {0, 1},\nalong with a similar form for E(x). Then define the variables MA = R M (x)A(x), MAE =\nR M(x)A(x)E(x), etc., and optimize given the constraints 0 MA, MB, ME 1, MAE \nmin[MA, ME], and MBE max[0, ME - (1 - MB)] (where the last constraint arises from the\nfact that ME + MB - MBE 1). Simplifying and taking the square root yields (13).\n\nThe bound (13) is shown in Figure 3; note that it improves both error bounds (11), shown\nas straight lines. In the next section, we use (8)-(13) to analyze the behavior of loopy BP.\n\n5 Implications in Graphs with Cycles\n\nWe begin by examining loopy BP with exact message passing, using the previous results\nto derive a new sufficient condition for convergence to a unique fixed point. When this\n\n\f\ncondition is not satisfied, we instead obtain a bound on the relative distances between any\ntwo fixed points of the loopy BP equations. We then consider the effect of introducing\nadditional errors into the messages passed at each iteration, showing sufficient conditions\nfor this operation to converge, and a bound on the resulting error from exact loopy BP.\n\n5.1 Convergence of Loopy BP & Fixed Point Distance\n\nTatikonda and Jordan [2] showed that the convergence and fixed points of loopy BP may\nbe considered in terms of a Gibbs measure on the graph's computation tree, implying that\nloopy BP is guaranteed to converge if the graph satisfies Dobrushin's condition [15]. Do-\nbrushin's condition is a global measure and difficult to verify; given in [2] is a sufficient\ncondition (often called Simon's condition):\n\n max log d (ut) < 1 (14)\n t\n ut\n\nwhere d () is defined as in (12). Using the previous section's analysis, we may argue\nsomething slightly stronger. Let us take the \"true\" messages mts to be any fixed point of\nBP, and \"approximate\" them at each iteration by performing loopy BP from some arbitrary\ninitial conditions. Now suppose that the largest message-product error log d (Eut) in any\nnode u with parent t at level i of the computation tree (corresponding to iteration n - i out\nof n total iterations of loopy BP) is bounded above by some constant log i. Note that this\nis trivially true (at any i) for the constant log i = max(u,t)E |t| log d (ut)2. Now, we\nmay bound d (Ets) at any replicate of node t with parent s on level i - 1 of the tree by\n\n d (\n log d (E ut)2 i + 1\n ts) gts(log i) = log . (15)\n d (\n u ut)2 + i\n t \\s\n\n\nand we may define log i-1 = maxt,s gts(log i) to bound the error at level i-1. Loopy BP\nwill converge if the sequence i, i-1, . . . is strictly decreasing for all > 1, i.e. gts(z) < z\nfor all z > 0. This is guaranteed by the conditions gts(0) = 0, g (0) < 1 (z) < 0\n ts and gts .\nThe first is easy to show, the third can be verified by algebra, and the condition g (0) < 1\n ts\ncan be rewritten to give the convergence criterion\n\n d (\n max ut)2 - 1 < 1 (16)\n (s,t)E d (\n u ut)2 + 1\n t \\s\n\nWe may relate (16) to Simon's condition (14) by expanding the set t \\ s to the larger t\nand noting that log x x2-1 for all x 1 with equality as x 1. Doing so, we see\n x2+1\nthat Simon's condition is sufficient to guarantee (16), but that (16) may be true (implying\nconvergence) when Simon's condition is not satisfied. The improvement over Simon's con-\ndition becomes negligible as connectivity increases (assuming the graph has approximately\nequal-strength potentials), but can be significant for low connectivity. For example, if the\ngraph consists of a single loop then each node t has at most two neighbors. In this case,\nthe contraction (16) tells us that the outgoing message in either direction is always closer\nto the BP fixed point than the incoming message. Thus we obtain the result of [1], that (for\nfinite-strength potentials) BP always converges to a unique fixed point on graphs containing\na single loop. Simon's condition, on the other hand, is too loose to demonstrate this fact.\n\nIf the condition (16) is not satisfied, then the sequence { i} is not always decreasing and\nthere may be multiple fixed points. In this case, the sequence { i} as defined will decrease\nuntil it reaches the largest value such that maxts gts(log ) = log . Since the choice of\ninitialization was arbitrary, we may opt to initialize to any other fixed point, and observe\nthat the difference Et between these two fixed point beliefs is bounded by\n d (\n log d (E ut)2 + 1\n t) log (17)\n d (\n u ut)2 +\n t\n\n\f\n 10 Simple bound, grids (a) and (b)\n ) t 8 Nonuniform bound, grid (a)\n Nonuniform bound, grid (b)\n 6 Simons condition\n (Ed 4\n 2\n log \n 00 0.5 1 1.5 2 2.5\n\n (a) (b) (c)\n\n\nFigure 4: Two small (5 5) grids, with (a) all equal-strength potentials log d ()2 = and (b)\nseveral weaker ones (log d ()2 = .5, thin lines). The methods described provide bounds (c) on the\ndistance between any two fixed points as a function of potential strength , all of which improve on\nSimon's condition. See text for details.\n\nThus, the fixed points of BP lie in some potentially small set. If log is small (the con-\ndition (16) is nearly satisfied) then although we cannot guarantee convergence to a unique\nfixed point, we can guarantee that every fixed point and our estimate are all mutually close\n(in a log-ratio sense).\n\n5.2 Improving the Bounds by Path-counting\n\nIf we are willing to put a bit more effort into our bound-computation, we may be able to\nimprove it.In particular the proofs of (16)-(17) assume that, as a message error propagates\nthrough the graph, repeated convolution with only the strongest set of potentials is possible.\nBut often even if the worst potentials are quite strong, every cycle which contains them\nalso contains several weaker potentials. Using an iterative algorithm much like BP itself,\nwe may obtain a more globally aware estimate of error propagation.\n\nLet us consider a message-passing procedure (potentially performed offline) in which node\nt passes a (scalar) bound its on the message error d eits at iteration i to its neighbor s.\nThe bound may be initialized to 1 = d (\n ts ts)2, and the next iteration's (updated) outgoing\nbound is given by the pair of equations\n d ( + 1\n log i+1 = log ts)2 its log i = log i\n ts ts ut (18)\n d (ts)2 + its ut\\s\n\nHere, as in Section 5.1, its bounds the error d (Ets) in the product of incoming messages.\n\nIf (18) converges to log i 0\n ts for all t, s we may guarantee a unique fixed point for\nloopy BP; if not, we may compute log i = log i\n t ut to obtain a bound on the belief\n t\nerror at each node t. If every node is identical (same number of neighbors, same potential\nstrengths) this yields the same bound as (17); however, if the graph or potential strengths are\ninhomogeneous it provides a strictly stronger bound on loopy BP convergence and errors.\n\nThis situation is illustrated in Figure 4--we specify two 55 grids in terms of their potential\nstrengths and compute bounds on the log-range of their fixed point beliefs. (While potential\nstrength does not completely specify the graphical model, it is sufficient for all the bounds\nconsidered here.) One grid (a) has equal-strength potentials log d ()2 = , while the other\nhas many weaker potentials (/2). The worst-case bounds are the same (since both have\na node with four strong neighbors), shown as the solid curve in (c). However, the dashed\ncurves show the estimate of (18), which improves only slightly for the strongly coupled\ngraph (a) but considerably for the weaker graph (b). All three bounds give considerably\nmore information than Simon's condition (dotted vertical line).\n\n5.3 Introducing additional errors\n\nAs discussed in the introduction, we may wish to introduce or allow additional errors in our\nmessages at each stage, in order to improve the computational or communication efficiency\nof the algorithm. This may be the result of an actual distortion imposed on the message\n\n\f\n(perhaps to decrease its complexity, for example quantization), from censoring the message\nupdate (reusing the message from the previous iteration) when the two are sufficiently\nsimilar, or from approximating or quantizing the model parameters (potential functions).\nAny of these additional errors can be easily incorporated into our framework.\n\nIf at each iteration, we introduce an additional (perhaps stochastic) error to each message\nwhich has a dynamic range bounded by some constant , the relationships of (18) become\n\n d ( + 1\n log i+1 = log ts)2 its + log log i = log i\n ts ts ut (19)\n d (ts)2 + its ut\\s\n\nand gives a bound on the steady-state error (distance from a fixed point) in the system.\n\n\n5.4 Stochastic Analysis\n\nUnfortunately, the above bounds are often pessimistic compared to actual performance. By\ntreating the perturbations as stochastic we may obtain a more realistic estimate (though no\nlonger a strict bound) on the resulting error. Specifically, let us describe the error func-\ntions log ets(xs) for each xs as a random variable with mean zero and variance 2ts. By\nassuming that the errors in each incoming message are uncorrelated, we obtain additivity\nof their variances: 2 = 2\n ts u ut. The assumption of uncorrelated errors is clearly\n t \\s\nquestionable since propagation around loops may couple the incoming message errors, but\nis common in quantization analysis, and we shall see that it appears reasonable in practice.\n\nWe would also like to estimate the contraction of variance incurred in the convolution step.\nWe may do so by applying a simple sigma-point quadrature (\"unscented\") approxima-\ntion [16], in which the standard deviation of the convolved function mts(xs) is estimated\nby applying the same nonlinearity (13) to the standard deviation of the error on the incom-\ning product Mts. Thus, similarly to (18) and (19), we have\n 2\n d (\n 2 = log ts)2 ts + 1 + (log )2 (log 2\n ts ts)2 = ut (20)\n ts + d (ts)2 ut\\s\n\nThe steady-state solution of (20) yields an estimate of the variances of the log-belief log pt\nby 2 = 2\n t u ut; this estimate is typically much smaller than the bound (18) due to the\n t\nstrict sub-additive relationship between the standard deviations. Although it is not a bound,\nusing a Chebyshev-like argument we may conclude that, for example, the 2t distance will\nbe greater than the typical errors observed in practice.\n\n6 Experiments\n\nWe demonstrate the error bounds for perturbed messages with a set of Monte Carlo trials.\nIn particular, for each trial we construct a binary-valued 5 5 grid with uniform poten-\ntial strengths, which are either (1) all positively correlated, or (2) randomly chosen to be\npositively or negatively correlated (equally likely); we also assign random single-node po-\ntentials to each xs. We then run a quantized version of BP, rounding each log-message\nto discrete values separated by 2 log (ensuring that the newly introduced error satisifies\nd (e) ). Figure 5 shows the maximum belief error in each of 100 trials of this procedure\nfor various values of .\n\nAlso shown are the bound on belief error developed in Section 5.3 and the 2 estimate\ncomputed assuming uncorrelated message errors. As can be seen, the stochastic estimate\nis often a much tighter, more accurate assessment of error, but it does not possess the\nsame strong theoretical guarantees. Since, as observed in analysis of quantization and\nstability in digital filtering [17], the errors introduced by quantization are typically close to\nindependent, the assumptions of the stochastic estimate are reasonable and empirically we\nobserve that the estimate and actual errors behave similarly.\n\n\f\n Strict bound\n 1\n 10 Stochastic estimate\n ) 101\n )\n Positive corr. potentials\n t t\n Mixed corr. potentials\n\n 0\n (E (E\n 10\n d 100\n d\n 1\n 10\n log 10 -1\n log\n ax ax\n 2\n 10\n m 10 -2\n log m log \n 3\n 10 3 2 1 0 10 -3\n 10 10 10 10 10-3 10 -2 10 -1 100\n\n\n (a) log d ()2 = .25 (b) log d ()2 = 1\n\nFigure 5: Maximum belief errors incurred as a function of the quantization error. The scatterplot\nindicates the maximum error measured in the graph for each of 200 Monte Carlo runs; this is strictly\nbounded above by the solution of (18), solid, and bounded with high probability (assuming uncorre-\nlated errors) by (20), dashed.\n\n7 Conclusions\n\nWe have described a particular measure of distortion on BP messages and shown that it is\nsub-additive and measurably contractive, leading to sufficient conditions for loopy BP to\nconverge to a unique fixed point. Furthermore, this enables analysis of quantized, stochas-\ntic, or other approximate forms of BP, yielding sufficient conditions for convergence and\nbounds on the deviation from exact message passing. Assuming the perturbations are un-\ncorrelated can often give tighter estimates of the resulting error. For additional details as\nwell as some further consequences and extensions, see [14].\n\nThe authors would like to thank Erik Sudderth, Martin Wainwright, Tom Heskes, and Lei Chen for\nmany helpful discussions. This research was supported in part by MIT Lincoln Laboratory under\nLincoln Program 2209-3023 and by ODDR&E MURI through ARO grant DAAD19-00-0466.\n\nReferences\n\n [1] Y. Weiss. Correctness of local probability propagation in graphical models with loops. Neural\n Computation, 12(1), 2000.\n [2] S. Tatikonda and M. Jordan. Loopy belief propagation and gibbs measures. In UAI, 2002.\n [3] T. Heskes. On the uniqueness of loopy belief propagation fixed points. To appear in Neural\n Computation, 2004.\n [4] D. Koller, U. Lerner, and D. Angelov. A general algorithm for approximate inference and its\n application to hybrid Bayes nets. In UAI 15, pages 324333, 1999.\n [5] J. M. Coughlan and S. J. Ferreira. Finding deformable shapes using loopy belief propagation.\n In ECCV 7, May 2002.\n [6] E. B. Sudderth, A. T. Ihler, W. T. Freeman, and A. S. Willsky. Nonparametric belief propagation.\n In CVPR, 2003.\n [7] M. Isard. PAMPAS: Realvalued graphical models for computer vision. In CVPR, 2003.\n [8] T. Minka. Expecatation propagation for approximate bayesian inference. In UAI, 2001.\n [9] A. T. Ihler, J. W. Fisher III, and A. S. Willsky. Communication-constrained inference. Technical\n Report TR-2601, Laboratory for Information and Decision Systems, 2004.\n[10] L. Chen, M. Wainwright, M. Cetin, and A. Willsky. Data association based on optimization in\n graphical models with application to sensor networks. Submitted to Mathematical and Com-\n puter Modeling, 2004.\n[11] P. Clifford. Markov random fields in statistics. In G. R. Grimmett and D. J. A. Welsh, editors,\n Disorder in Physical Systems, pages 1932. Oxford University Press, Oxford, 1990.\n[12] J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufman, San Mateo, 1988.\n[13] J. S. Yedidia, W. T. Freeman, and Y. Weiss. Constructing free energy approximations and\n generalized belief propagation algorithms. Technical Report 2004-040, MERL, May 2004.\n[14] A. T. Ihler, J. W. Fisher III, and A. S. Willsky. Message errors in belief propagation. Technical\n Report TR-2602, Laboratory for Information and Decision Systems, 2004.\n[15] Hans-Otto Georgii. Gibbs measures and phase transitions. Studies in Mathematics. de Gruyter,\n Berlin / New York, 1988.\n[16] S. Julier and J. Uhlmann. A general method for approximating nonlinear transformations of\n probability distributions. Technical report, RRG, Dept. of Eng. Science, Univ. of Oxford, 1996.\n[17] A. Willsky. Relationships between digital signal processing and control and estimation theory.\n Proc. IEEE, 66(9):9961017, September 1978.\n\n\f\n", "award": [], "sourceid": 2679, "authors": [{"given_name": "Alexander", "family_name": "Ihler", "institution": null}, {"given_name": "John", "family_name": "Fisher", "institution": null}, {"given_name": "Alan", "family_name": "Willsky", "institution": null}]}