{"title": "Linear programming analysis of loopy belief propagation for weighted matching", "book": "Advances in Neural Information Processing Systems", "page_first": 1273, "page_last": 1280, "abstract": "Loopy belief propagation has been employed in a wide variety of applications with great empirical success, but it comes with few theoretical guarantees. In this paper we investigate the use of the max-product form of belief propagation for weighted matching problems on general graphs. We show that max-product converges to the correct answer if the linear programming (LP) relaxation of the weighted matching problem is tight and does not converge if the LP relaxation is loose. This provides an exact characterization of max-product performance and reveals connections to the widely used optimization technique of LP relaxation. In addition, we demonstrate that max-product is effective in solving practical weighted matching problems in a distributed fashion by applying it to the problem of self-organization in sensor networks.", "full_text": "Linear Programming Analysis of Loopy Belief\n\nPropagation for Weighted Matching\n\nSujay Sanghavi, Dmitry M. Malioutov and Alan S. Willsky\n\nLaboratory for Information and Decision Systems\n\nMassachusetts Institute of Technology\n\nCambridge, MA 02139\n\n{sanghavi,dmm,willsky}@mit.edu\n\nAbstract\n\nLoopy belief propagation has been employed in a wide variety of applications with\ngreat empirical success, but it comes with few theoretical guarantees. In this paper\nwe investigate the use of the max-product form of belief propagation for weighted\nmatching problems on general graphs. We show that max-product converges to the\ncorrect answer if the linear programming (LP) relaxation of the weighted matching\nproblem is tight and does not converge if the LP relaxation is loose. This provides\nan exact characterization of max-product performance and reveals connections to\nthe widely used optimization technique of LP relaxation. In addition, we demon-\nstrate that max-product is effective in solving practical weighted matching prob-\nlems in a distributed fashion by applying it to the problem of self-organization in\nsensor networks.\n\n1 Introduction\n\nLoopy Belief Propagation (LBP) and its variants [6, 9, 13] have been shown empirically to be effec-\ntive in solving many instances of hard problems in a wide range of \ufb01elds. These algorithms were\noriginally designed for exact inference (i.e. calculation of marginals/MAP estimates) in probability\ndistributions whose associated graphical models are tree-structured. While some progress has been\nmade in understanding their convergence and accuracy on general \u201cloopy\u201d graphs (see [8, 12, 13]\nand their references), it still remains an active research area.\n\nIn this paper we study the application of the widely used max-product form of LBP (or simply\nmax-product (MP) algorithm), to the weighted matching problem. Given a graph G = (V, E) with\nnon-negative weights we on its edges e \u2208 E, the weighted matching problem is to \ufb01nd the heaviest\nset of mutually disjoint edges (i.e. a set of edges such that no two edges share a node). Weighted\nmatching is a classic problem that has played a central role in computer science and combinatorial\noptimization, with applications in resource allocation, scheduling in communications networks [10],\nand machine learning [5]. It has often been perceived to be the \u201ceasiest non-trivial problem\u201d, and\none whose analysis and solution has inspired methods (or provided insights) for a variety of other\nproblems. Weighted matching thus naturally suggests itself as a good candidate for the study of\nconvergence and correctness of algorithms like max-product.\n\nWeighted matching can be naturally formulated as an integer program. The technique of linear\nprogramming (LP) relaxation involves replacing the integer constraints with linear inequality con-\nstraints. This relaxation is tight if the resulting linear program has an integral optimum. LP relax-\nation is not always tight for the weighted matching problem. The primary contribution of this paper\nis an exact characterization of max-product performance for the matching problem, which also es-\ntablishes a link to LP relaxation. We show that (i) if the LP relaxation is tight then max-product\n\n1\n\n\fconverges to the correct answer, and (ii) if the LP relaxation is not tight then max-product does not\nconverge.\n\nWeighted matching is a special case of the weighted b-matching problem, where there can be up to\nbi edges touching node i (setting all bi = 1 reduces to simple matching). All the results of this paper\nhold for the general case of b-matchings on arbitrary graphs. However, in the interests of clarity, we\nprovide proofs only for the conceptually easier case of simple matchings where bi = 1. The minor\nmodi\ufb01cations needed for general b-matchings will appear in a longer publication. In prior work,\nBayati et. al [2] established that max-product converges for weighted matching in bipartite graphs,\nand [5] extended this result to b-matching. These results are implied by our result1, as for bipartite\ngraphs, the LP relaxation is always tight.\n\nIn Section 2 we set up the weighted matching problem and its LP relaxation. We describe the max-\nproduct algorithm for weighted matching in Section 3. The main result of the paper is established in\nSection 4. Finally, in Section 5 we apply b-matching to a sensor-network self-organization problem\nand show that max-product provides an effective way to solve the problem in a distributed fashion.\n\n2 Weighted Matching and its LP Relaxation\n\nSuppose that we are given a graph G with weights we, we also positive integers bi for each node\ni \u2208 V . A b-matching is any set of edges such that the total number of edges in the set incident\nto any node i is at most bi. The weighted b-matching problem is to \ufb01nd the b-matching of largest\nweight. Weighted b-matching can be naturally formulated as the following integer program (setting\nall bi = 1 gives an integer program for simple matching):\n\nwexe,\n\nmax Xe\u2208E\n\nxe \u2264 bi\n\nfor all i \u2208 V,\n\nIP :\n\ns.t. Xe\u2208Ei\n\nxe \u2208 {0, 1} for all e \u2208 E\n\nHere Ei is the set of edges incident to node i. The linear programming (LP) relaxation of the above\nproblem is to replace the constraint xe \u2208 {0, 1} with the constraint xe \u2208 [0, 1], for each e \u2208 E. We\ndenote the corresponding linear program by LP. Throughout this paper, we will assume that LP has\na unique optimum. The LP relaxation is said to be tight if the unique optimum is integral (i.e. one in\nwhich all xe \u2208 {0, 1}). Note that the LP relaxation is not tight in general. Apart from the bipartite\ncase, the tightness of LP relaxation is a function of both the weights and the graph structure2.\n\n3 Max-Product for Weighted Matching\n\nWe now formulate weighted b-matching on G as a MAP estimation problem by constructing a\nsuitable probability distribution. This construction is naturally suggested by the form of the integer\nprogram IP. Associate a binary variable xe \u2208 {0, 1} with each edge e \u2208 E, and consider the\nfollowing probability distribution:\n\np(x) \u221d Yi\u2208V\n\n\u03c8(xEi)Ye\u2208E\n\nexp(wexe),\n\n(1)\n\nwhich contains a factor \u03c8(xEi) for each node i \u2208 V , the value of which is \u03c8(xEi) = 1 if\n\nof p, and e to refer both to the edges of G and variables of p. The factor \u03c8(xEi) enforces the con-\nstraint that at most one edge incident to node i can be assigned the value \u201c1\u201d. It is easy to see that,\n\nPe\u2208Ei xe \u2264 bi, and 0 otherwise. Note that we use i to refer both to the nodes of G and factors\nfor any x, p(x) = exp(Pe wexe) if the set of edges {e|xe = 1} constitute a b-matching in G, and\n\np(x) = 0 otherwise. Thus the max-weight b-matching of G corresponds to the MAP estimate of p.\n\n1However, [2] uses a graphical model which is different from ours to represent weighted matching.\n2A simple example: G is a cycle of length 3, all the bi = 1. If all we = 1, LP relaxation is loose: setting\n\neach xe = 1\n\n2 is the optimum. However, if instead the weights are {1, 1, 3}, then LP relaxation is tight.\n\n2\n\n\fThe factor-graph version of the max-product algorithm [6] passes messages between variables and\nthe factors that contain them (for the formulation in (1), each variable is a member of exactly two\nfactors). The output is an estimate \u02c6x of the MAP of p. We now present the max-product update\nequations simpli\ufb01ed for p in (1). In the following e and (i, j) denote the same edge. Also, for two\nsets A and B the set difference is denoted by the notation A \u2212 B.\nMax-Product for Weighted Matching\n\n(INIT) Set t = 0 and initialize each message to be uniform.\n(ITER) Iteratively compute new messages until convergence as follows:\ne\u2192i[xe] = exp(xewe) \u00d7 mt\n\nVariable to Factor:\n\nmt+1\n\nj\u2192e[xe]\n\nFactor to Variable:\n\nmt+1\n\ni\u2192e[xe] = max\n\nxEi\u2212e(\u03c8(xEi) Ye0\u2208Ei\u2212e\n\nmt\n\ne0\u2192i[xe0 ])\n\nAlso, at each t compute beliefs nt\n\ne[xe] = exp(wexe) \u00d7 mt\n\ni\u2192e[xe] \u00d7 mt\n\nj\u2192e[xe]\n\n(ESTIM) Upon convergence, output estimate \u02c6x: for each edge set \u02c6xe = 1 if ne[1] > ne[0], and\n\n\u02c6xe = 0 otherwise.\n\nRemark: If the degree |Ei| of a node is large, the corresponding factor \u03c8(xEi) will depend on many\nvariables. In general, for very large factors it is intractable to compute the \u201cfactor to variable\u201d update\n(and even to store the factors in memory). However, for our problem the special form of \u03c8 makes\n\nif all bi = 1, we have that\n\nthis step easy even for large degrees: for each edge e \u2208 Ei compute ae = max\u00b31, mt\nae0 \u00d7 Ye0\u2208Ei\u2212e\n\ni\u2192e[1] = Ye0\u2208Ei\u2212e\n\ni\u2192e[0] = max\n\ne0\u2192i[0]\n\nmt+1\n\nmt+1\n\ne0\u2208Ei\u2212e\n\nmt\n\nmt\n\n,\n\ne\u2192i[1]\n\ne\u2192i[0]\u00b4. Then,\n\nmt\n\ne0\u2192i[0]\n\nThe simpli\ufb01cation for general b is as follows: let Fe \u2282 Ei \u2212 e be the set of bi variables in Ei \u2212 e\nwith the largest values of ae, and let Ge \u2282 Ei \u2212 e be the set of bi \u2212 1 variables with largest ae. Then,\n\nmt+1\n\ni\u2192e[1] = Ye0\u2208Ge\n\nae0 Ye0\u2208Ei\u2212e\n\nThese updates are linear in the degree |Ei|.\n\nmt\n\ne0\u2192i[0]\n\n,\n\nmt+1\n\ni\u2192e[0] = Ye0\u2208Fe\n\nae0 Ye0\u2208Ei\u2212e\n\nmt\n\ne0\u2192i[0]\n\nThe Computation Tree for Weighted Matching\n\nOur proofs rely on the computation tree interpretation [12, 11] of loopy max-product beliefs, which\nwe now describe for the special case of simple matching (bi = 1). Recall the variables of p corre-\nspond to edges in G, and nodes in G correspond to factors. For any edge e, the computation tree\nTe(1) at time 1 is just the edge e, the root of the tree. Both endpoints of the root are leaves. The tree\nTe(t) at time t is generated from Te(t \u2212 1) by adding to each leaf of Te(t \u2212 1) a copy of each of\nits neighbors in G, except for the neighbor that is already present in Te(t \u2212 1). The weights of the\nedges in Te are copied from the corresponding edges in G.\nSuppose M is a matching on the original graph G, and Te is a computation tree. Then, the image\nof M in Te is the set of edges in Te whose corresponding copy in G is a member of M. We now\nillustrate the ideas of this section with a simple example.\nExample 1: Consider the \ufb01gure above. G appears on the left, the numbers are the edge weights and\nthe letters are node labels. The max-weight matching M \u2217 = {(a, b), (c, d)} is depicted in bold. In\nthe center plot we show T(a,b)(4), the computation tree at time t = 4 rooted at edge (a, b). Each\nnode is labeled in accordance to its copy in G. The bold edges in the middle tree depict the matching\nwhich is the image of M \u2217 onto T(a,b)(4). The weight of this matching is 6.6, and it is easy to see\nthat any matching on T(a,b)(4) that includes the root edge will have weight at most 6.6. On the right\nwe depict M, the max-weight matching on the tree T(a,b)(4). M has weight 7.3. In this example we\nsee that even though (a, b) is in the unique optimal matching in G, the beliefs at the root are such\nthat n4\n(a,b)[1]. Note also that the dotted edges are not an image of any matching in the\noriginal graph G. This example thus illustrates how \u201cspurious\u201d matchings in the computation tree\ncan lead to incorrect beliefs, and estimates.\n\n(a,b)[0] > n4\n\n3\n\n\fb\n\n1\n\n1.1\n\na\n\n1\n\nc\n\n1\n\n1.1\n\nd\n\nd\n\na\n\na\n\nc\n\nd\n\nb\n\nc\n\na\n\nd\n\nc\n\nb\n\na\n\na\n\nb\n\nd\n\nb\n\na\n\na\n\nb\n\nc\n\nd\n\nc\n\nb\n\nc\n\na\n\nd\n\na\n\na\n\nb\n\nd\n\nb\n\na\n\nd\n\na\n\n4 Main Result: Equivalence of LP Relaxation and Loopy Max-product\n\nIn this section we formally state the main result of this paper, and give an outline of the proofs.\n\nTheorem 1 Let G = (V, E) be a graph with nonnegative real weights we on the edges e \u2208 E.\nAssume the linear programming relaxation LP has a unique optimal solution. Then, the following\nholds:\n\n1. If the LP relaxation is tight, i.e.\n\nif the unique solution is integral, then the max-product\n\nconverges and the resulting estimate is the optimal matching.\n\n2. If the LP relaxation is not tight, i.e. if the unique solution contains fractional values, then\n\nthe max-product does not converge.\n\nThe above theorem implies that LP relaxation and Max-product will both succeed, or both fail, on\nthe same problem instances, and thus are equally powerful for the weighted matching problem. We\nnow prove the two parts of the theorem. In the interest of brevity and clarity, the theorem and the\nproofs are presented for the conceptually easier case of simple matchings, in which all bi = 1. Also,\nfor the purposes of the proofs we will assume that \u201cconvergence\u201d means that there exists a \u03c4 < \u221e\nsuch that the maximizing assignment arg maxxe nt\n\ne(xe) remains constant for all t > \u03c4 .\n\nProof of Part 1: Max-Product is as Powerful as LP Relaxation\n\nSuppose LP has an integral optimum. Consider now the linear-programming dual of LP, denoted\nbelow as DUAL.\n\nDUAL :\n\ns.t.\n\nzi\n\nmin Xi\u2208V\n\nwij \u2264 zi + zj\nzi \u2265 0 for all i \u2208 V\n\nfor all (i, j) \u2208 E,\n\nThe following lemma states that the standard linear programming properties of complimentary\nslackness hold in the strict sense for the weighted matching problem (this is a special case of [3,\nex. 4.20]).\n\nLemma 1 (strict complimentary slackness) If the solution to LPis unique and integral, and M \u2217\nis the optimal matching, then there exists an optimal dual solution z to DUAL such that\n\n1. For all (i, j) \u2208 M \u2217, we have wij = zi + zj\n\n2. There exists \u0001 > 0 such that for all (i, j) /\u2208 M \u2217 we have wij \u2264 zi + zj \u2212 \u0001\n\n3. if no edge in M \u2217 is incident on node i, then zi = 0\n\n4. zi \u2264 maxe we for all i\n\nLet t \u2265 2wmax\n, where wmax = maxe we is the weight of the heaviest edge, and \u0001 is as in part 2 of\nLemma 1 above. Suppose now that there exists an edge e /\u2208 M \u2217 for which the belief at time t is\nincorrect, i.e nt\n\ne[0]. We now show that this leads to a contradiction.\n\ne[1] > nt\n\n\u0001\n\n4\n\n\fe[1] > nt\n\ne[0] means that there is a matching M in Te(t) such that (a) the root e \u2208 M, and\nRecall that nt\nT be the image of M \u2217 onto Te(t). By de\ufb01nition,\n(b) M is a max-weight matching on Te(t). Let M \u2217\nT . From e, build an alternating path P by successively adding edges as follows: \ufb01rst add e,\ne /\u2208 M \u2217\nthen add all edges adjacent to e that are in M \u2217\nT , then all their adjacent edges that are in M, and so\nforth until no more edges can be added \u2013 this will occur either because no edges are available that\nmaintain the alternating structure, or a leaf of Te(t) has been reached. Note that this will be a path,\nbecause M and M \u2217\nT are matchings and so any node in Te(t) can have at most one edge adjacent to\nit in each of the two matchings.\n\nFor illustration, consider Example 1 of section 3. M \u2217\nT is in the center plot and M is on the right.\nThe above procedure for building P would yield the path adcabcda that goes from the left-most leaf\nto the right-most leaf. It is easy to see that this path alternates between edges in M and M \u2217\nT .\nWe now show that w(P \u2229 M \u2217\nT ) > w(P \u2229 M ). Let z be the dual optimum corresponding to the\n\u0001 above. Suppose \ufb01rst that neither endpoint of P is a leaf of Te(t). Then, from parts 1 and 3 of\nLemma 1 it follows that\n\nw(P \u2229 M \u2217\n\nT ) = X(i,j)\u2208P \u2229M \u2217\n\nT\n\nwij = X(i,j)\u2208P \u2229M \u2217\n\nT\n\nzi + zj = Xi\u2208P\n\nzi.\n\nOn the other hand, from part 2 of Lemma 1 it follows that\n\nw(P \u2229 M ) = X(i,j)\u2208P \u2229M\n\nwij \u2264 X(i,j)\u2208P \u2229M\n\nzi + zj \u2212 \u0001 = \u00c3Xi\u2208P\n\nzi! \u2212 \u0001|P \u2229 M |.\n\nT ) > w(P \u2229 M ). A similar argument,\nNow by construction the root e \u2208 P \u2229 S, and hence w(P \u2229 M \u2217\nwith minor modi\ufb01cations, holds for the case when one or both endpoints of P are leaves of Te. For\nthese cases we would need to make explicit use of the fact that t \u2265 2wmax\nWe now show that M cannot be an optimal matching in Te(t). We do so by \u201c\ufb02ipping\u201d the edges in\nP to obtain a matching with higher weight. Speci\ufb01cally, let M 0 = M \u2212 (P \u2229 M ) + (P \u2229 M \u2217\nT ) be\nthe matching containing all edges in M except the ones in P , which are replaced by the edges in\nT . It is easy to see that M 0 is a matching in Te(t), and that w(M 0) > w(M ). This contradicts\nP \u2229 M \u2217\nthe choice of M, and shows that for e /\u2208 M \u2217 the beliefs satisfy nt\ne[0] for all t large enough.\nThis means that the estimate has converged and is correct for e. A similar argument can be used\nto show that the max-product estimate converges to the correct answer for e \u2208 M \u2217 as well. Hence\nmax-product converges globally to the correct M \u2217.\n\ne[1] \u2264 nt\n\n.\n\n\u0001\n\nProof of Part 2: LP Relaxation is as Powerful as Max-Product\n\nSuppose the optimum solution of LP contains fractional values. We now show that in this case\nmax-product does not converge. As a \ufb01rst step we have the following lemma.\n\nLemma 2 If Max-Product converges, the resulting estimate is M \u2217.\n\nThe proof of this lemma uses the result in [12], that states that if max-product converges then the re-\nsulting estimates are \u201clocally optimal\u201d: the posterior probability of the max-product assignment can\nnot be increased by changing values in any induced subgraph in which each connected component\ncontains at most one loop. For the weighted matching problem this local optimality implies global\noptimality, because the symmetric difference of any two matchings is a union of disjoint paths and\ncycles. The above lemma implies that, for the proof of part 2 of the theorem, it is suf\ufb01cient to show\nthat max-product does not converge to the correct answer M \u2217. We do this by showing that for any\ngiven \u03c4 , there exists a t \u2265 \u03c4 such that the max-product estimate at time t will not be M \u2217.\nWe \ufb01rst provide a combinatorial characterization of when the LP relaxation is loose. Let M \u2217 be the\nmax-weight matching on G. An alternating path in G is a path in which every alternate edge is in\nM \u2217, and each node appears at most once. A blossom is an alternating path that wraps onto itself,\nsuch that the result is a single odd cycle C and a path R leading out of that cycle3. The importance\nof blossoms for matching problems is well-known [4]. A bad blossom is a blossom in which the\nedge weights satisfy\n\nw(C \u2229 M \u2217) + 2w(R \u2229 M \u2217) < w(C \u2212 M \u2217) + 2w(R \u2212 M \u2217).\n\n3The path may be of zero length, in which case the blossom is just the odd cycle.\n\n5\n\n\fExample: On the right is a bad blossom:\nbold edges are in M \u2217, numbers are edge\nweights and alphabets are node labels. Cycle\nC in this case is abcdu, and path R is cf ghi.\n\na\n\n3\n\nu\n\n3\n\n3\n\nb\n\n3\n\nc\n\n3\n\nd\n\n1\n\n1\n\nf\n\nh\n\n1\n\ng\n\n0.5\n\ni\n\nA dumbbell is an alternating path that wraps onto itself twice, such that the result is two disjoint odd\ncycles C1 and C2 and an alternating path R connecting the two cycles. In a bad dumbbell the edge\nweights satisfy\n\nw(C1 \u2229 M \u2217) + w(C2 \u2229 M \u2217) + 2w(R \u2229 M \u2217) < w(C1 \u2212 M \u2217) + w(C2 \u2212 M \u2217) + 2w(R \u2212 M \u2217).\n\nExample: On the right is a bad dumbbell.\nCycles C1 and C2 are abcdu and f ghij, and\n(c, f ) is the path R.\n\na\n\n3\n\nu\n\n3\n\n3\n\nb\n\n3\n\n3\n\nd\n\nc\n\n1\n\nf\n\ng\n\n3\n\n3\n\nj\n\n3\n\n3\n\nh\n\n3\n\ni\n\nProposition 1 If LP relaxation is loose, then in G there exists either a bad blossom, or a bad\ndumbbell.\n\nProof. The proof of this proposition will appear in a longer version of this paper. (It is also in the\nappendix submitted along with the paper).\n\nSuppose now that max-product converges to M \u2217 by iteration \u03c4 , and suppose also there exists a bad\nblossom B1 in G. For an edge e \u2208 B1 \u2229 M \u2217 consider the computation tree Te(\u03c4 + |V |) for e at time\n\u03c4 + |V |. Let M be the optimal matching on the tree. From the de\ufb01nition of convergence, it follows\nthat near the root e, M will be the image of M \u2217 onto Te: for any edge e0 in the tree at distance less\nthan |V | from the root, e0 \u2208 M if and only if its copy in G is in M \u2217.\nThis means that the copies in Te of the edges in B1 will contain an alternating path P in Te: every\nalternate edge of P will be in M. For the bad blossom example above, the alternating path is\nihgf cbaudcf ghi (it will go once around the cycle and twice around the path of the blossom). Make\na new matching M 0 on Te(\u03c4 + |V |) by \u201cswitching\u201d the edges in this path: M 0 = M \u2212 (M \u2229 P ) +\n(P \u2212 M ). Then, it is easy to see that\n\nw(M ) \u2212 w(M 0) = w(C \u2229 M \u2217) + 2w(R \u2229 M \u2217) \u2212 w(C \u2212 M \u2217) \u2212 2w(R \u2212 M \u2217).\n\nBy assumption B1 is a bad blossom, and hence we have that w(M ) < w(M 0), which violates the\noptimality of M. Thus, max-product does not converge to M \u2217 if there exists a bad blossom. A\nsimilar proof precludes convergence to M \u2217 for the case when there is a bad dumbbell. It follows\nfrom Proposition 1 that if LP relaxation is loose, then max-product cannot converge to M \u2217.\n\n5 Sensor network self-organization via b-matching\n\nWe now consider the problem of sensor network self-organization. Suppose a large number of low-\ncost sensors are deployed randomly over an area, and as a \ufb01rst step of any communication or remote\nsensing application the sensors have to organize themselves into a network [1]. The network should\nbe connected, and robust to occasional failing links, but at the same time it should be sparse (i.e.\nhave nodes with small degrees) due to severe limitations on power available for communication.\n\nSimply connecting every pair of sensors that lie within some distance d of each other (close enough\nto communicate reliably) may lead to large clusters of very densely connected components, and\nnodes with high degrees. Hence, sparser networks with fewer edges are needed [7]. The throughput\nof a link drops fast with distance, so the sparse network should mostly contain short edges. The\nsparsest connected network is achieved by a spanning tree solution. However, a spanning tree may\nhave nodes with large degrees, and a single failed link disconnects it. An interesting set of sparse\nsubgraph constructions with various tradeoffs addressing power ef\ufb01ciency in wireless networks is\nproposed in [7].\n\n6\n\n\f1\n\n0.8\n\n0.6\n\n0.4\n\n0.2\n\n0\n\n\u22120.2\n\n\u22120.4\n\n\u22120.6\n\n\u22120.8\n\n\u22121\n\n1\n\n0.8\n\n0.6\n\n0.4\n\n0.2\n\n0\n\n\u22120.2\n\n\u22120.4\n\n\u22120.6\n\n\u22120.8\n\n\u22121\n\n\u22121\n\n\u22120.8\n\n\u22120.6\n\n\u22120.4\n\n\u22120.2\n\n0\n\n0.2\n\n0.4\n\n0.6\n\n0.8\n\n1\n\n\u22121\n\n\u22120.8\n\n\u22120.6\n\n\u22120.4\n\n\u22120.2\n\n0\n\n0.2\n\n0.4\n\n0.6\n\n0.8\n\n1\n\n(a)\n\n(b)\n\nFigure 1: Network with N = 100 nodes. (a) Nodes within d = 0.5 are connected by an edge. (b)\nSparse network obtained by b-matching with b = 5.\n\n0.12\n\n0.1\n\n0.08\n\n0.06\n\n0.04\n\n0.02\n\n0\n\n \n0\n\n \n\nN = 50\nN = 100\nN = 200\n\n10\n\n20\n\n30\n\n40\n\n50\n\n60\n\n1.002\n\n1\n\n0.998\n\n0.996\n\n0.994\n\n0.992\n\n0.99\n\n0.988\n\n0.986\n\n0.984\n\n \n\n \n\nBP, b=3\nBP, b=5\nBP, b=10\nLP, b=3\nLP, b=5\nLP, b=10\n\n40\n\n60\n\n80\n\n100\n\n120\n\n140\n\n160\n\n180\n\n200\n\n(a)\n\n(b)\n\nFigure 2: (a) Histogram of node degrees versus node density. (b) Average fraction of the LP upper\nbound on optimal cost obtained using LP relaxation and max-product.\n\nWe consider using b-matching to \ufb01nd a sparse power-ef\ufb01cient subgraph. We assign edge weights to\nbe proportional to the throughput of the link. For typical sensor network applications the received\npower (which can be used as a measure of throughput) decays as d\u2212p with distance, where p \u2208 [2, 4].\nWe set p = 3 for concreteness, and let the edge weights be we = d\u2212p\ne . The b-matching objective\nis now to maximize the total throughput (received power) among sparse subgraphs with degree at\nmost b. We use the max-product algorithm to solve weighted b-matching in a distributed fashion.\nFor our experiments we randomly disperse N nodes in a square region [\u22121, 1] \u00d7 [\u22121, 1]. First we\ncreate the adjacency graph for nodes that are close enough to communicate, we set the threshold to\nbe d = 0.5. In Figure 2(a) we plot the histogram over a 100 trials of resulting node degrees. Clearly,\nas N increases, nodes have increasingly higher degrees.\nNext we apply max-product (MP) and LP relaxation4 to solve the b-matching objective. As we have\nestablished earlier, the performance of LP relaxation, and hence, of MP for b-matching depends on\nthe existence of \u2019bad blossoms\u2019, i.e. odd-cycles where the weights on the edges are quite similar.\nWe show in simulations that bad blossoms appear rarely for the random graphs and weights in our\nconstruction, and LP-relaxation and MP produce nearly optimal b-matchings. For the cases where\nLP relaxation has fractional edges, and MP has oscillating (or non-converged) edges, we erase them\nfrom the \ufb01nal matching and ensure that LP and MP solutions are valid matchings. Also, instead of\ncomparing LP and MP costs to the optimal b-matching cost, we compare them to the LP upper bound\non the cost (the cost of the fractional LP solution). This avoids the need to \ufb01nd optimal b-matchings.\nIn \ufb01gure 1 we plot the dense adjacency graph for N = 100 nodes, and the much sparser b-matching\nsubgraph with b = 5 obtained by MP. Now, consider \ufb01gure 2(b). We plot the percentage of the LP\n\n4LP is not practical for sensor networks, as it is not easily distributed.\n\n7\n\n\f1\n\n0.95\n\n0.9\n0\n\nFraction disconnected\nMean power stretch\n\nb = 5\n5/100\n3.64\n\nb = 7\n0/100\n1.45\n\nb = 10\n0/100\n1.06\n\n5\n\n10\n\n15\n\n(a)\n\n20\n\n25\n\n30\n\n(b)\n\nFigure 3: (a) Average fraction of the LP upper bound on optimal cost obtained using T iterations\nof max-product. (b) A table showing probability of disconnect, and the power stretch factor for\nN = 100 averaged over 100 trials.\n\nupper bound obtained by MP and by rounded LP relaxation. It can be seen that both LP and MP\nproduce nearly optimal b-matchings, with more than 98 percent of the optimal cost. The percentage\ndecreases slowly with sensor density (with higher N), but improves for larger b. An important per-\nformance metric for sensor network self-organization is the power-stretch factor5, which compares\nthe weights of shortest paths in G to weights of shortest paths in the sparse subgraph. In \ufb01gure 3(b)\nwe display the maximum power stretch factor over all pairs of nodes, averaged over 100 trials. For\nb = 10 there is almost no loss in power by using the sparse subgraph. A limitation of the b-matching\nsolution is that connectedness of the subgraph is not guaranteed. In fact, for b = 1 it is always\ndisconnected. However, as b increases, the graph gets rarely disconnected. In \ufb01gure 3(b) we display\nprobability of disconnect over 100 trials. For b = 10 and N = 100 in a longer simulation, the sparse\nsubgraph got disconnected twice over 500 trials.\nIn \ufb01gure 3(a) we study the performance of MP versus the number of iterations. We run MP for a \ufb01xed\nnumber of iterations, remove oscillating edges to get a valid matching, and plot the average fraction\nof the LP upper bound that the solution gets. We set b = 5, and N = 100. Quite surprisingly, MP\nachieves a large percentage of the optimal cost even with as few as 3 iterations. After 20 this \ufb01gure\nexceeds 99 percent.\n\nReferences\n\n[1] I.F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, \u201cA survey on sensor networks,\u201d IEEE\n\nCommunications Magazine, vol. 40, no. 8, pp. 102\u2013114, Aug. 2002.\n\n[2] M. Bayati, D. Shah, and M. Sharma, \u201cMaximum weight matching via max-product belief propagation,\u201d in\n\nISIT, Sept. 2005, pp. 1763 \u2013 1767.\n\n[3] D. Bertsimas and J. Tsitsiklis. Linear Opitimization. Athena Scienti\ufb01c, 1997.\n[4] J. Edmonds, \u201cPaths, trees and \ufb02owers,\u201d Canadian Journal of Mathematics, vol. 17, pp. 449\u2013467, 1965.\n[5] B. Huang and T. Jebara, \u201cLoopy belief propagation for bipartite maximum weight b-matching,\u201d in Arti\ufb01cial\n\nIntelligence and Statistics (AISTATS), March 2007.\n\n[6] F. Kschischang, B. Frey, and H. Loeliger, \u201cFactor graphs and the sum-product algorithm,\u201d IEEE Transac-\n\ntions on Information Theory, vol. 47, no. 2, pp. 498\u2013519, Feb. 2001.\n\n[7] X. Y. Li, P. J. Wan, Y. Wang, and O. Frieder, \u201cSparse power ef\ufb01cient topology for wireless networks,\u201d in\n\nProc. IEEE Hawaii Int. Conf. on System Sciences, Jan. 2002.\n\n[8] D. Malioutov, J. Johnson, and A. Willsky, \u201cWalk-sums and belief propagation in Gaussian graphical mod-\n\nels,\u201d Journal of Machine Learning Research, vol. 7, pp. 2031\u20132064, Oct. 2006.\n\n[9] J. Pearl. Probabilistic inference in intelligent systems. Morgan Kaufmann, 1988.\n[10] L. Tassiulas and A. Ephremides Stability properties of constrained queueing systems and scheduling\npolicies for maximum throughput in multihop radio networks IEEE Trans. on Automatic Control, vol. 37,\nno. 12, Dec. 1992.\n\n[11] S. Tatikonda and M. Jordan, \u201cLoopy belief propagation and Gibbs measures,\u201d in Uncertainty in Arti\ufb01cial\n\nIntelligence, vol. 18, 2002, pp. 493\u2013500.\n\n[12] Y. Weiss and W. Freeman, \u201cOn the optimality of solutions of the max-product belief-propagation algo-\n\nrithm in arbitrary graphs,\u201d IEEE Trans. on Information Theory, vol. 47, no. 2, pp. 736\u2013744, Feb. 2001.\n\n[13] J. Yedidia, W. Freeman, and Y. Weiss. Understanding belief propagation and its generalizations. Exploring\n\nAI in the new millennium, pages 239\u2013269, 2003.\n\n5To compute the power-stretch the edges are weighted by d3, i.e. the power needed to get a \ufb01xed throughput.\n\n8\n\n\f", "award": [], "sourceid": 905, "authors": [{"given_name": "Sujay", "family_name": "Sanghavi", "institution": null}, {"given_name": "Dmitry", "family_name": "Malioutov", "institution": null}, {"given_name": "Alan", "family_name": "Willsky", "institution": null}]}