{"title": "Replicator Equations, Maximal Cliques, and Graph Isomorphism", "book": "Advances in Neural Information Processing Systems", "page_first": 550, "page_last": 556, "abstract": null, "full_text": "Replicator Equations, Maximal Cliques, \n\nand Graph Isomorphism \n\nMarcello Pelillo \n\nDipartimento di Informatica \n\nUniversita Ca' Foscari di Venezia \n\nVia Torino 155, 30172 Venezia Mestre, Italy \n\nE-mail: pelillo@dsi.unive.it \n\nAbstract \n\nWe present a new energy-minimization framework for the graph \nisomorphism problem which is based on an equivalent maximum \nclique formulation. The approach is centered around a fundamental \nresult proved by Motzkin and Straus in the mid-1960s, and recently \nexpanded in various ways, which allows us to formulate the maxi(cid:173)\nmum clique problem in terms of a standard quadratic program. To \nsolve the program we use \"replicator\" equations, a class of simple \ncontinuous- and discrete-time dynamical systems developed in var(cid:173)\nious branches of theoretical biology. We show how, despite their \ninability to escape from local solutions, they nevertheless provide \nexperimental results which are competitive with those obtained us(cid:173)\ning more elaborate mean-field annealing heuristics. \n\nINTRODUCTION \n\n1 \nThe graph isomorphism problem is one of those few combinatorial optimization \nproblems which still resist any computational complexity characterization [6]. De(cid:173)\nspite decades of active research, no polynomial-time algorithm for it has yet been \nfound. At the same time, while clearly belonging to N P, no proof has beel1 pro(cid:173)\nvided that it is NP-complete. Indeed, there is strong evidence that this cannot be \nthe case for, otherwise, the polynomial hierarchy would collapse [5]. The current \nbelief is that the problem lies strictly between the P and NP-complete classes. \nBecause of its theoretical as well as practical importance, the problem has attracted \nmuch attention in the neural network community, and various powerful heuris(cid:173)\ntics have been developed [11, 18, 19, 20]. Following Hopfield and Tank's seminal \nwork [10], the typical approach has been to write down a (continuous) energy func(cid:173)\ntion whose minimizers correspond to the (discrete) solutions being sought, and then \nconstruct a dynamical system which converges toward them. Almost invariably, all \nthe algorithms developed so far are based on techniques borrowed from statistical \nmechanics, in particular mean field theory, which allow one to escape from poor \n\n\fReplicator Equations, Maximal Cliques, and Graph Isomorphism \n\n551 \n\nlocal solutions. \n\nIn this paper, we develop a new energy-minimization framework for the graph iso(cid:173)\nmorphism problem which is based on the idea of reducing it to the maximum clique \nproblem, another well-known combinatorial optimization problem. Central to our \napproach is a powerful result originally proved by Motzkin and Straus [13], and \nrecently extended in various ways [3, 7, 16], which allows us to formulate the maxi(cid:173)\nmum clique problem in terms of an indefinite quadratic program. We then present \na class of straightforward continuous- and discrete-time dynamical systems known \nin mathematical biology as replicator equations, and show how, thanks to their \ndynamical properties, they naturally suggest themselves as a useful heuristic for \nsolving the proposed graph isomorphism program. The extensive experimental re(cid:173)\nsults presented show that, despite their simplicity and their inherent inability to \nescape from local optima, replicator dynamics are nevertheless competitive with \nmore sophisticated deterministic annealing algorithms. The proposed formulation \nseems therefore a promising framework within which powerful continuous-based \ngraph matching heuristics can be developed, and is in fact being employed for solv(cid:173)\ning practical computer vision problems [17J. More details on the work presented \nhere can be found in [15J. \n\n2 A QUADRATIC PROGRAM FOR GRAPH \n\nISOMORPHISM \n\n2.1 GRAPH ISOMORPHISM AS CLIQUE SEARCH \nLet G = (V, E) be an undirected graph, where V is the set of vertices and E ~ V x V \nis the set of edges. The order of G is the number of its vertices, and its size is the \nnumber of edges. Two vertices i,j E V are said to be adjacent if (i,j) E E. The \nadjacency matrix of G is the n x n symmetric matrix A = (aij) defined as follows: \naij = 1 if (i,j) E E, aij = a otherwise. \nGiven two graphs G' = (V', E') and Gil = (V\", E\") having the same order and \nsize, an isomorphism between them is any bijection \u00a2 : V' -t V\" such that \n(i,j) E E' {:} (\u00a2(i),\u00a2(j)) E E\", for all i,j E V'. Two graphs are said to be \nisomorphic if there exists an isomorphism between them. The graph isomorphism \nproblem is therefore to decide whether two graphs are isomorphic and, in the af(cid:173)\nfirmative, to find an isomorphism. Barrow and Burstall [IJ introduced the notion \nof an association graph as a useful auxiliary graph structure for solving general \ngraphjsubgraph isomorphism problems. The association graph derived from G' \nand Gil is the undirected graph G = (V, E), where V = V' X V\" and \n\nE = {((i, h), (j, k)) E V x V \n\n: i:f= j, h:f= k, and (i,j) E E' {:} (h, k) E E\"} . \n\nGiven an arbitrary undirected graph G = (V, E), a subset of vertices C is called a \nclique if all its vertices are mutually adjacent , i.e. , for all i,j E C we have (i,j) E E. \nA clique is said to be maximal if it is not contained in any larger clique, and \nmaximum if it is the largest clique in the graph. The clique number, denoted by \nw(G), is defined as the cardinality of the maximum clique. \n\nThe following result establishes an equivalence between the graph isomorphism \nproblem and the maximum clique problem (see [15J for proof). \n\nTheorem 2.1 Let G' and Gil be two graphs of order n , and let G be the correspond(cid:173)\ning association graph. Then, G' and Gil are isomorphic if and only if w(G) = n. In \nthis case, any maximum clique of G induces an isomorphism between G' and Gil , \nand vice versa. \n\n\f552 \n\nM. Pelillo \n\n2.2 CONTINUOUS FORMULATION OF MAX-CLIQUE \n\nLet G = (V, E) be an arbitrary undirected graph of order n, and let Sn denote the \nstandard simplex of lRn: \n\nSn={xElRn : Xi~O foralli=l. .. n, and tXi=I}. \n\nz== 1 \n\nGiven a subset of vertices C of G, we will denote by XC its characteristic vector \nwhich is the point in Sn defined as xI = 1/ICI if i E C, xi = 0 otherwise, where ICI \ndenotes the cardinality of C. \n\nNow, consider the following quadratic function: \n\nx T Ax \n\nf(x) = \n\n(1) \nwhere \"T\" denotes transposition. The Motzkin-Straus theorem [13] establishes a \nremarkable connection between global (local) maximizers of fin Sn and maximum \n(maximal) cliques of G. Specifically, it states that a subset of vertices C of a \ngraph G is a maximum clique if and only if its characteristic vector XC is a global \nmaximizer of the function f in Sn. A similiar relationship holds between (strict) \nlocal maximizers and maximal cliques [7, 16]. \n\nOne drawback associated with the original Motzkin-Straus formulation relates to \nthe existence of spurious solutions, i.e., maximizers of f which are not in the form \nof characteristic vectors [16]. In principle, spurious solutions represent a problem \nsince, while providing information about the order of the maximum clique, do not \nallow us to extract the vertices comprising the clique. Fortunately, there is straight(cid:173)\nforward solution to this problem which has recently been introduced and studied \nby Bomze [3]. Consider the following regularized version of function f: \n\nj (x) = x T Ax + ~ X T X \n\n. \n\n(2) \n\nThe following is the spurious-free counterpart of the original Motzkin-Straus theo(cid:173)\nrem (see [3] for proof). \n\nTheorem 2.2 Let C be a subset of vertices of a graph G, and let X C be its charac(cid:173)\nteristic vector. Then the following statements hold: \n(a) C is a maximum clique of G if and only if XC is a global maximizer of j over \n\nthe simplex Sn. Its order is then given by ICI = 1/2(1 - f(x C )). \n\n(b) C is a maximal clique of G if and only if XC is a local maximizer of j in Sn. \n(c) All local (and hence global) maximizers of j over Sn are strict. \n\nUnlike the Motzkin-Straus formulation, the previous result guarantees that all max(cid:173)\nimizers of j on Sn are strict, and are characteristic vectors of maximal/maximum \ncliques in the graph. In an exact sense, therefore, a one-to-one correspondence ex-\nists between maximal cliques and local maximizers of j in Sn on the one hand, and \nmaximum cliques and global maximizers on the other hand. \n\n2.3 A QUADRATIC PROGRAM FOR GRAPH ISOMORPHISM \n\nLet G' and Gil be two arbitrary graphs of order n, and let A denote the adjacency \nmatrix of the corresponding association graph, whose order is assumed to be N. \nThe graph isomorphism problem is equivalent to the following program: \n\nmaXImIze \nsubject to x E SN \n\nj(x) = xT (A + ~ IN)X \n\n(3) \n\n\fReplicator Equations. Maximal Cliques. and Graph Isomorphism \n\n553 \n\nMore precisely, the following result holds, which is a straightforward consequence \nof Theorems 2.1 and 2.2. \n\nTheorem 2.3 Let G' and Gil be two graphs of order n, and let x* be a global \nsolution of program (3), where A is the adjacency matrix of the association graph \nof G' and Gil . Then, G' and Gil are isomorphic if and only if j(x*) = 1 - 1/2n. \nIn this case, any global solution to (3) induces an isomorphism between G' and Gil, \nand vice versa. \n\nIn [15] we discuss the analogies between our objective function and those proposed \nin the literature (e.g., [18, 19]). \n\n3 REPLICATOR EQUATIONS AND GRAPH \n\nISOMORPHISM \n\nLet W be a non-negative n x n matrix, and consider the following dynamical system: \n\n~Xi(t) = Xi(t) (\"i(t) - t.X;(t)\";(t)) , \n\ni = 1. . . n \n\nwhere 7ri(t) = 2:.7=1 WijXj(t), i = 1 . . . n , and its discrete-time counterpart: \n\nxi(t+1)=2:. n\n\nXi(t)7ri(t) \nj = l x] t 7r] t \n\n() . ( ) ' \n\ni=l .. . n. \n\n(4) \n\n(5) \n\nIt is readily seen that the simplex Sn is invariant under these dynamics, which \nmeans that every trajectory starting in Sn will remain in Sn for all future times. \nBoth (4) and (5) are called replicator equations in theoretical biology, since they \nare used to model evolution over time of relative frequencies of interacting, self(cid:173)\nreplicating entities [9]. The discrete-time dynamical equations turn also out to be \na special case of a general class of dynamical systems introduced by Baum and \nEagon [2] in the context of Markov chain theory. \nTheorem 3.1 If W is symmetric, then the quadratic polynomial F(x) = xTWx is \nstrictly increasing along any non-constant trajectory of both continuous-time (4) and \ndiscrete-time (5) replicator equations. Furthermore, any such trajectory converges \nto a (unique) stationary point. Finally, a vector x E Sn is asymptotically stable \nunder (4) and (5) if and only if x is a strict local maximizer of F on Sn. \n\nThe previous result is known in mathematical biology as the Fundamental Theorem \nof Natural Selection [9, 21]. As far as the discrete-time model is concerned, it \ncan be regarded as a straightforward implication of the more general Baum-Eagon \ntheorem [2]. The fact that all trajectories of the replicator dynamics converge to a \nstationary point is proven in [12]. \n\nRecently, there has been much interest in evolutionary game theory around the \nfollowing exponential version of replicator equations , which arises as a model of \nevolution guided by imitation [8, 21]: \n\n:t Xi (t) = Xi(t) (L:7~1 ~:;;;~ .. '(t) - 1), i = l... n \n\n(6) \n\nwhere K, is a positive constant. As K, tends to 0, the orbits of this dynamics approach \nthose of the standard, first-order replicator model (4), slowed down by the factor \n\n\f554 \n\nM Pelillo \n\nK. Hofbauer [8] has recently proven that when the matrix W is symmetric, the \nquadratic polynomial F defined in Theorem 3.1 is also strictly increasing, as in \nthe first-order case. After discussing various properties of this, and more general \ndynamics, he concluded that the model behaves essentially in the same way as the \nstandard replicator equations, the only difference being the size of the basins of \nattraction around stable equilibria. A customary way of discretizating equation (6) \nis given by the following difference equations: \nxi(t)e\"1l';(t) \n. X \u00b7 t e\"1l'J \n)=1 \n\nXi(t + 1) = L:n \n\ni = l. .. n \n\n(7) \n\n( ) \n\n) \n\n(t)' \n\nwhich enjoys many of the properties of the first-order system (5), e.g., they have \nthe same set of equilibria. \n\nThe properties discussed above naturally suggest using replicator equations as a \nuseful heuristic for the graph isomorphism problem. Let G' and G\" be two graphs \nof order n, and let A denote the adjacency matrix of the corresponding N-vertex \nassociation graph G. By letting \n\nW = A + \"2IN \n\n1 \n\nwe know that the replicator dynamical systems, starting from an arbitrary initial \nstate, will iteratively maximize the function j(x) = xT(A + !IN)x in SN, and will \neventually converge to a strict local maximizer which, by virtue of Theorem 2.2 will \nthen correspond to the characteristic vector of a maximal clique in the association \ngraph. This will in turn induce an isomorphism between two subgraphs of G' and \nG\" which is \"maximal,\" in the sense that there is no other isomorphism between \nsubgraphs of G' and G\" which includes the one found. Clearly, in theory there is no \nguarantee that the converged solution will be a global maximizer of j, and therefore \nthat it will induce an isomorphism between the two original graphs . Previous work \ndone on the maximum clique problem [4, 14], and also the results presented in this \npaper, however, suggest that the basins of attraction of global maximizers are quite \nlarge, and very frequently the algorithm converges to one of them. \n\n4 EXPERIMENTAL RESULTS \nIn the experiments reported here, the discrete-time replicator equation (5) and its \nexponential counterpart (7) with K = 10 were used. The algorithms were started \nfrom the barycenter of the simplex and they were stopped when either a maximal \nclique was found or the distance between two successive points was smaller than a \nfixed threshold, which was set to 10-17 . In the latter case the converged vector was \nrandomly perturbed, and the algorithm restarted from the perturbed point . Because \nof the one-to-one correspondence between local maximizers and maximal cliques, \nthis situation corresponds to convergence to a saddle point. All the experiments \nwere run on a Sparc20. \n\nUndirected 100-vertex random graphs were generated with expected connectivities \nranging from 1 % to 99%. For each connectivity value, 10'0 graphs were produced and \neach of them had its vertices randomly permuted so as to obtain a pair of isomorphic \ngraphs. Overall, therefore, 1500 pairs of isomorphic graphs were used. Each pair \nwas given as input to the replicator models and, after convergence, a success was \nrecorded when the cardinality of the returned clique was equal to the order of the \ngraphs given as input (Le., 100) .1 Because of the stopping criterion employed, this \n\n1 Due to the high computational time required, in the 1 % and 99% cases the first-order \n\nreplicator algorithm (5) was tested only on 10 pairs, instead of 100. \n\n\fReplicator Equations, Maximal Cliques, and Graph Isomorphism \n\n/\n\n- -\n\n--- . \n\n-- . ~--.-- . -\n\n- - -\n\nf '00 ~, \n.i 75 1 \nU ! 50 j / \nt c \ni \n.. \ni \n\n25 I \n\nI -(cid:173)\nI \n\n-.--.. -. . -. \n\n555 \n\n-. . . . \\ \n\\ \u2022 \\ \n\n\\ \n\nI \n\n' \n\nI \niii \n\n001 003 0 05 0' 0.2 0.3 \n\n0 \" as 06 0.7 a a 09 0 95 0 97 099 \n\n001 003 0 .05 01 \n\n0 .2 03 0 \" as 06 07 08 0.9 095 a 97 0 99 \n\nExpecled connectivity \n\nExpected connecllvlty \n\nFigure 1: Percentage of correct isomorphisms obtained using the first-order (left) and the \nexponential (right) replicator equations, as a function of the expected connectivity. \n\n100000 -- - (cid:173)\n\n(\u00b1I ~1U7 (1) \n\n\u00a5 10000 \n!!!. ! 1000 \n\n'00 \n\n'0 \n\n-\n\n-\n\n-\n\n.~~om