{"title": "Message Passing Inference with Chemical Reaction Networks", "book": "Advances in Neural Information Processing Systems", "page_first": 2247, "page_last": 2255, "abstract": "Recent work on molecular programming has explored new possibilities for computational abstractions with biomolecules, including logic gates, neural networks, and linear systems.  In the future such abstractions might enable nanoscale devices that can sense and control the world at a molecular scale.  Just as in macroscale robotics, it is critical that such devices can learn about their environment and reason under uncertainty. At this small scale, systems are typically modeled as chemical reaction networks. In this work, we develop a procedure that can take arbitrary probabilistic graphical models, represented as factor graphs over discrete random variables, and compile them into chemical reaction networks that implement inference.  In particular, we show that marginalization based on sum-product message passing can be implemented in terms of reactions between chemical species whose concentrations represent probabilities.  We show algebraically that the steady state concentration of these species correspond to the marginal distributions of the random variables in the graph and validate the results in simulations.  As with standard sum-product inference, this procedure yields exact results for tree-structured graphs, and approximate solutions for loopy graphs.", "full_text": "Message Passing Inference with Chemical Reaction\n\nNetworks\n\nWyss Institute for Biologically Inspired Engineering\n\nSchool of Engineering and Applied Sciences\n\nNils Napp\n\nRyan Prescott Adams\n\nHarvard University\n\nCambridge, MA 02138\n\nHarvard University\n\nCambridge, MA 02138\n\nnnapp@wyss.harvard.edu\n\nrpa@seas.harvard.edu\n\nAbstract\n\nRecent work on molecular programming has explored new possibilities for com-\nputational abstractions with biomolecules, including logic gates, neural networks,\nand linear systems. In the future such abstractions might enable nanoscale devices\nthat can sense and control the world at a molecular scale. Just as in macroscale\nrobotics, it is critical that such devices can learn about their environment and rea-\nson under uncertainty. At this small scale, systems are typically modeled as chem-\nical reaction networks. In this work, we develop a procedure that can take arbitrary\nprobabilistic graphical models, represented as factor graphs over discrete random\nvariables, and compile them into chemical reaction networks that implement infer-\nence. In particular, we show that marginalization based on sum-product message\npassing can be implemented in terms of reactions between chemical species whose\nconcentrations represent probabilities. We show algebraically that the steady state\nconcentration of these species correspond to the marginal distributions of the ran-\ndom variables in the graph and validate the results in simulations. As with stan-\ndard sum-product inference, this procedure yields exact results for tree-structured\ngraphs, and approximate solutions for loopy graphs.\n\n1 Introduction\n\nRecent advances in nanoscale devices and biomolecular synthesis have opened up new and exciting\npossibilities for constructing microscopic systems that can sense and autonomously manipulate the\nworld. Necessary to such advances is the development of computational mechanisms and associated\nabstractions for algorithmic control of these nanorobots. Work on molecular programming has ex-\nplored the power of chemical computation [3, 6, 11] and resulted in in vitro biomolecular implemen-\ntations of various such abstractions, including logic gates [16], arti\ufb01cial neural networks [9, 10, 14],\ntiled self-assembly models [12, 15], and linear functions and systems [4, 13, 20]. Similarly, in\nvivo gene regulatory networks can be designed that when transformed into cells implement devices\nsuch as oscillators [8], intracellularly coupled oscillators [7], or distributed algorithms like pattern\nformation [1]. Many critical information processing tasks can be framed in terms of probabilistic\ninference, in which noisy or incomplete information is accumulated to produce statistical estimates\nof hidden structure. In fact, we believe that this particular computational abstraction is ideally suited\nto the noisy and often poorly characterized microscopic world. In this work, we develop a chemical\nreaction network for performing inference in probabilistic graphical models. We show that message\npassing schemes, such as belief propagation, map relatively straightforwardly onto sets of chemical\nreactions, which can be thought of as the \u201cassembly language\u201d of both in vitro and in vivo compu-\ntation at the molecular scale. The long-term possibilities of such technology are myriad: adaptive\ntissue-sensitive drug delivery, in situ chemical sensing, and identi\ufb01cation of disease states.\n\n1\n\n\fAbstract Problem & Algorithm\n\nLow-Level \u201dAssembly\u201d Language\n\nPhysical\n\nImplementation\n\n\u03c81\n\n\u03c82\n\nP(1\u21921)\n\nS(1\u21921)\n\nP(2\u21922)\n\nS(2\u21922)\n\nP(1\u21923)\n\nP(2\u21923)\n\nx1\n\n\u03c83\n\nx2\n\nS(3\u21921)\n\nS(3\u21922)\n\n(a)\n\nS(3\u21922)\n\n0\n\nS(3\u21922)\n\n0\n\n1\n\nS(1\u21921)\nS(1\u21921)\n\n2\n\n\u03bar\nGGGGB S(1\u21921)\n\u03bar\nGGGGB S(1\u21921)\n\n0\n\n0\n\n.\n.\n.\n\n1\n\n+ P(1\u21923)\n+ P(1\u21923)\n\n2\n\n\u03c83(1,1)\nGGGGGGGB S(3\u21922)\n\u03c83(2,1)\nGGGGGGGB S(3\u21922)\n\n1\n\n1\n\n.\n.\n.\n(b)\n\n1\n\n+ P(1\u21923)\n+ P(1\u21923)\n\n2\n\n(c)\n\nFigure 1:\nInference at different levels of abstraction. (a) Factor graph over two random variables.\nInference can be performed ef\ufb01ciently by passing messages (shown as gray arrows) between ver-\ntices, see Section 2. (b) Message passing implemented at a lower level of abstraction. Chemical\nspecies represent the different components of message vectors. The chemical reaction networks\nconstructed in Section 3 perform the same computation as the sum-product message passing algo-\nrithm. (c) Schematic representation of DNA strand displacement. A given reaction network can be\nimplemented in different physical systems, e.g. DNA strand displacement cascades [5, 17].\n\nAt the small scales of interest systems are typically modeled as deterministic chemical reaction net-\nworks or their stochastic counterparts that explicitly model \ufb02uctuations and noise. However, chem-\nical reactions are not only models, but can be thought of as speci\ufb01cations or abstract computational\nframeworks themselves. For example, arbitrary reaction networks can be simulated by DNA strand\ndisplacement systems [5, 17], where some strands correspond to the chemical species in the specify-\ning reaction network. Reactions rates in these systems can be tuned over many orders of magnitude\nby adjusting the toehold length of displacement steps, and high order reactions can be approximated\nby introducing auxiliary species. We take advantage of this abstraction by \u201dcompiling\u201d the sum-\nproduct algorithm for discrete variable factor graphs into a chemical reaction network, where the\nconcentrations of some species represent conditional and marginal distributions of variables in the\ngraph. In some ways, this representation is very natural: while normalization is a constant concern\nin digital systems, our chemical design conserves species within some subsets and thus implicitly\nand continuously normalizes its estimates. The computation is complete when the reaction network\nreaches equilibrium. Variables in the graph can be conditioned upon by adjusting the reaction rates\ncorresponding to unary potentials in the graph.\n\nSection 2 provides a brief review of factor graphs and the sum-product algorithm. Section 3 in-\ntroduces notation and concepts for chemical reaction networks. Section 4 shows how inference on\nfactor graphs can be compiled into reaction networks, and in Section 5, we show several example\nnetworks and compare the results of molecular simulations to standard digital inference procedures.\n\nTo aid parsing the potentially tangled notation resulting from mixing probabilistic inference tools\nwith chemical reaction models, this paper follows these general notational guidelines: capital letters\ndenote constants, such as set sizes, and other quantities, such as tuples and message types; lower\ncase letters denote parameters, such as reaction rates and indices; bold face letters denote vectors\nand subscripts elements of that vector; scripted upper letters indicate sets; random variables are\nalways denoted by x or their vector version; and species names have a sans-serif font.\n\n2 Graphical Models and Probabilistic Inference\n\nGraphical models are popular tools for reasoning about complicated probability distributions. In\nmost types of graphical models, vertices represent random variables and edges re\ufb02ect dependence\nstructure. Here, we focus on the factor graph formalism, in which there are two types of vertices\nthat have a bipartite structure: variable nodes (typically drawn as circles), which represent random\nvariables, and factor nodes (typically drawn as squares), which represent potentials (also called\ncompatibility functions) coupling the random variables. Factor graphs, encode the factorization of a\nprobability distribution and therefore its conditional independence structure. Other graphical mod-\nels, such as bayesian networks, can be converted to factor graphs, and thus factor graph algorithms\nare directly applicable to other types of graphical models, see [2, Ch. 8].\n\n2\n\n\fLet G be a factor graph over N random variables {xn}N\nn=1 where xn takes one of Kn discrete values.\nThe global N -dimensional random variable x takes on values in the (potentially huge) product space\nK = QN\nn=1{1, ..., Kn}. The other nodes of G are called factors and every edge in G connects\nexactly one factor node and one variable node. In general, G can have J factors {\u03c8j(xj)}J\nj=1 where\nwe use xj to indicate the subset of random variables that are neighbors of factor j, i.e. {xn | n \u2208\nne(j)}. Each xj takes on values in the (potentially much smaller) space Kj = Qn\u2208ne(j){1, ..., Kn},\nand each \u03c8j is a non-negative scalar function on Kj. Together the structure of G and the particular\nfactors \u03c8j de\ufb01ne a joint distribution on x\n\nPr(x) = Pr(x1, x2, \u00b7 \u00b7 \u00b7 , xN ) =\n\n1\nZ\n\nJ\n\nY\nj=1\n\n\u03c8j(xj),\n\n(1)\n\nwhere Z is the appropriate normalizing constant. Figure 1a shows a simple factor graph with two\nvariable nodes and three factors. It implies that the the joint distribution x1 and x2 has the form\nPr(x1, x2) = 1\nThe sum-product algorithm (belief propagation) is an dynamic programming technique for perform-\ning marginalization in a factor graph. That is, it computes sums of the form\n\nZ \u03c81(x1)\u03c82(x2)\u03c83(x1, x2).\n\nPr(xn) =\n\n1\nZ X\n\nx\\xn\n\nJ\n\nY\nj=1\n\n\u03c8j(xj).\n\n(2)\n\nFor tree-structured factor graphs, the sum-product algorithm ef\ufb01ciently recovers the exact marginals.\nFor more general graphs the sum-product algorithm often converges to useful approximations, in\nwhich case it is called loopy belief propagation.\n\nThe sum-product algorithm proceeds by passing \u201cmessages\u201d along the graph edges. There are two\nkinds of messages messages from a factor node to a variable node and messages from a variable\nnode to a factor node. In order to make clear what quantities are represented by chemical species\nconcentrations in Section 4, we use somewhat unconventional notation. The kth entry of the sum\nmessage from factor node j to variable node n is denoted S(j\u2192n)\nand the entire Kn-dimensional\nvector is denoted by S(j\u2192n). The kth entry of the product message from variable n to factor node j\nis denoted by P(n\u2192j)\nand the entire Kj-dimensional vector is denoted P(n\u2192j). Figure 1a shows a\nsimple factor graph with message names and their directions shown as gray arrows. Sum messages\nfrom j are computed as the weighted sum of product messages over the domain Kj of \u03c8j\n\nk\n\nk\n\nS(j\u2192n)\n\nk\n\n= X\nn=k\n\nkj\n\nn\u2032\u2208ne(j)\\n\n\n\u03c8j(xj = kj) Y\n\nP(n\u2032\u2192j)\n\nkj\n\nn\u2032\n\n,\n\n(3)\n\nn = k to the set of all\nwhere ne(j)\\n refers to the variable node neighbors of j except n and kj\nkj \u2208 Kj where the entry in the dimension of n is \ufb01xed to k. Product messages are computed by\ntaking the component-wise product of incoming sum messages\nS(j \u2032\u2192n)\n\nP(n\u2192j)\n\n(4)\n\n.\n\nk\n\n= Y\nj \u2032\u2208ne(n)\\j\n\nk\n\nUp to normalization, the marginals can be computed from the product of incoming sum messages\n\nPr(xn = k) = Y\n\nS(j\u2192n)\n\nk\n\n.\n\nj\u2208ne(n)\n\n(5)\n\nThe sum-product algorithm corresponds to \ufb01xed-point iterations that are minimizing the Bethe free\nenergy. This observation leads to both partial-update or damped variants of sum-product, as well\nas asynchronous versions [18, Ch.6][19]. The validity of damped asynchronous sum-product is\nwhat enables us to frame the computation as a chemical reaction network. The continuous ODE\ndescription of species concentrations that represent messages can be thought of as an in\ufb01nitesimally\nsmall version of damped asynchronous update rules.\n\n3 Chemical Reaction Networks\n\nThe following model describes how a set of M chemical species Z = {Z1, Z2, \u00b7 \u00b7 \u00b7 , ZM } interact\nand their concentrations evolve over time. Each reaction has the general form\n\n3\n\n\fr1Z1 + r2Z2 + \u00b7 \u00b7 \u00b7 + rM ZM\n\n\u03ba\n\nGGGGGGGB p1Z1 + p2Z2 + \u00b7 \u00b7 \u00b7 + pM ZM .\n\n(6)\n\nIn this generic representation most of the coef\ufb01cients rm \u2208 N and pm \u2208 N are typically zero (where\nN indicates non-negative integers). The species on the left with non-zero coef\ufb01cients are called\nreactants and are consumed during the reaction. The species on the right with non-zero entries\nare called products and are produced during the reaction. Species that participate in a reaction,\ni.e., rm > 0, but where no net consumption or production occurs, i.e. rm = pm, are called catalysts.\nThey change the dynamics of a particular reaction without being changed themselves.\n\nA reaction network over a given set of\nR = {R1, R2, \u00b7 \u00b7 \u00b7 , RQ}, where each reaction is a triple of reaction parameters (6),\n\nspecies consists of a set of Q reactions\n\n(7)\nFor example, in a reaction Rq \u2208 R where species Z1 and Z3 form a new chemical species Z2 at a\nrate of \u03baq, the reactant vector rq is zero everywhere except at rq\n3 = 1. The associated product\nvector pq is zero everywhere except at pq\n2 = 1. In the reaction notation where non-participating\nspecies are dropped reaction Rq is can be compactly written as\n\nRq = (rq , \u03baq, pq).\n\n1 = rq\n\nZ1 + Z3\n\n\u03baq\n\nGGGGGGGB Z2.\n\n(8)\n\n3.1 Mass Action Kinetics\n\nThe concentration of each chemical species Zm is denoted by [Zm]. A reaction network describes\nthe evolution of species concentrations as a set of coupled non-linear differential equations. For each\nspecies concentration [Zm] the rate of change is given by mass action kinetics,\n\nd[Zm]\n\ndt\n\n=\n\nQ\n\nX\nq=1\n\n\u03baq\n\nM\n\nY\nm\u2032=1\n\n[Zm\u2032 ]rq\n\nm\u2032 (pq\n\nm \u2212 rq\n\nm).\n\n(9)\n\nBased on the fact that reactant coef\ufb01cients appear as powers, the sum PM\nm=1 rm is called the order\nof a reaction. For example, if the only reaction in a network were the second order reaction (8), the\nconcentration dynamics of [Z1] would be\n\nd[Z1]\ndt\n\n= \u2212\u03baq[Z1][Z3].\n\n(10)\n\nSimilar to [4] we design reaction networks where the equilibrium concentration of some species\ncorresponds to the results we are interested in computing. The reaction networks in the following\nsection conserve mass and do not require \ufb02ux in or out of the system, and therefore the solutions are\nguaranteed to be bounded. While we cannot rule out oscillations in general, the message passing\nmethods these reactions are simulating correspond to an energy minimization problem. As such, we\nsuspect that the particular reaction networks presented here always converge to their equilibrium.\n\n4 Representing Graphical Models with Reaction Networks\n\nIn the following compilation procedure, each message and marginal probability is represented by\na set of distinct chemical species. We design networks that cause them to interact in such a way\nthat, at steady state, the concentration of some species represent the marginal distributions of the\nvariable nodes in a factor graph. When information arrives the network equilibrates to the new,\ncorrect, value. Since messages in the sum-product inference algorithm are computed from other\nmessages, the reaction networks that implement sending messages describe how species from one\nmessage catalyze the species of another message.\n\n, has an associated chemical species S(j\u2192n)\n\nBeliefs and messages are represented as concentrations of chemical species: each component of a\nsum message, S(j\u2192n)\n; each component of a product\nk\nk\nmessage, P(n\u2192j)\n, has an associated chemical species P(n\u2192j)\n; and each component of a marginal\nprobability distribution, Pr(xn = k), has an associated chemical species Pn\nk . In addition, each\nmessage and marginal probability distribution has a chemical species with a zero subscript that rep-\nresents unassigned probability mass. Together, the set of species associated with a messages or\n\nk\n\nk\n\n4\n\n\fmarginal probability are called a belief species, and the reaction networks presented in the subse-\nquent sections are designed to conserve species \u2013 and by extension their concentrations \u2013 with each\nsuch set. For example, the concentration of belief species P n = {Pn\nk=0 of Pr(xn) have a constant\nsum, PKn\nk ] , determined by the initial concentrations. These sets belief species are a chemi-\ncal representation of the left hand sides of Equations 3\u20135. The next few sections present reaction\nnetworks whose dynamics implement their right hand sides.\n\nk=0[Pn\n\nk }Kn\n\n4.1 Belief Recycling Reactions\n\nEach set of belief species has an associated set of reactions that recycle assigned probabilities to the\nunassigned species. By continuously and dynamically reallocating probability mass, the resulting\nreaction network can adapt to changing potential functions \u03c8j, i.e. new information.\nFor example, the factor graph shown in Figure 1a has 8 distinct sets of belief species \u2013 2 representing\nmarginal probabilities of x1 and x2, and 6 (ignoring messages to leaf factor nodes) representing\nmessages. The associate recycling reactions are\n\nP1\nk\nP2\nk\n\n\u03bar\n\nGGGGB P1\n0\n\u03bar\nGGGGB P2\n0\n\nk\n\nS(1\u21921)\nS(2\u21922)\n\nk\n\n\u03bar\nGGGGB S(1\u21921)\n\u03bar\nGGGGB S(2\u21922)\n\n0\n\n0\n\nk\n\nS(3\u21921)\nS(3\u21922)\n\nk\n\n\u03bar\nGGGGB S(3\u21921)\n\u03bar\nGGGGB S(3\u21922)\n\n0\n\n0\n\nk\n\nP(1\u21923)\nP(2\u21923)\n\nk\n\n\u03bar\nGGGGB P(1\u21923)\n\u03bar\nGGGGB P(2\u21923)\n\n0\n\n0\n\n.\n(11)\n\nBy choosing a smaller rate \u03bar less of the probability mass will be unassigned at steady state, i.e.\nquantities will be closer to normalized, however the speed at which the reaction network reaches\nsteady state decreases, see Section 5.\n\n4.2 Sum Messages\n\nIn the reactions that implement messages from factor to variable nodes, message species of incoming\nmessages catalyze the assignment of message species belonging to outgoing messages. The entries\nin factor tables determine the associated rate constants. The kth message component from a factor\nnode \u03c8j to the variable node xn is implemented by a reactions of the form\n\nS(j\u2192n)\n\n0\n\n+ X\n\nP(n\u2032\u2192j)\n\nkj\n\nn\u2032\n\nn\u2032\u2208ne(j)\\n\n\n\u03c8j (xj =kj )\nGGGGGGGB S(j\u2192n)\n\nk\n\n+ X\n\nP(n\u2032\u2192j)\n\nkj\n\nn\u2032\n\nn\u2032\u2208ne(j)\\n\n\n,\n\n(12)\n\nwhere the nth component of kj is clamped to k, kj\nfor each sum message species are given by\n\nn = k. Using the law of mass action, the kinetics\n\nd[S(j\u2192n)\n\nk\ndt\n\n]\n\n\u03c8j(xj = kj)[S(j\u2192n)\n\n0\n\n= X\nn=k\n\nkj\n\n[P(n\u2032\u2192j)\n\nkj\n\n] Y\nn\u2032\u2208ne(j)\\n\n\nn\u2032\n\n] \u2212 \u03bar[S(j\u2192n)\n\nk\n\n].\n\n(13)\n\nAt steady state the concentration of S(j\u2192n)\n\nk\n\nis given by\n\n\u03bar\n\n[S(j\u2192n)\n\n0\n\n]\n\n[S(j\u2192n)\n\nk\n\n] = X\nn=k\n\nkj\n\n\u03c8j(xj = kj ) Y\n\n[P(n\u2032\u2192j)\n\nkj\n\nn\u2032\n\nn\u2032\u2208ne(j)\\n\n\n],\n\n(14)\n\nk\n\n] species concentrations have the same factor\n\nwhere all [S(j\u2192n)\n. Their relative concentra-\ntions are exactly the message according to the to Equation (3). As \u03bar decreases, the concentration of\nunassigned probability mass decreases and the concentration normalized by the constant sum of all\nthe related belief species can be interpreted as a probability. For example, the four factor-to-variable\nmessages in Figure 1(a) can be implemented with the following reactions\n\n[S(j\u2192n)\n\n\u03bar\n\n0\n\n]\n\nS(1\u21921)\n\n0\n\nS(2\u21922)\n\n0\n\n\u03c81(k)\n\nGGGGGGGB S(1\u21921)\n\n\u03c82(k)\n\nGGGGGGGB S(2\u21922)\n\nk\n\nk\n\n\u03c83(k,k\u2032)\nGGGGGGGB S(3\u21921)\n\u03c83(k\u2032,k)\nGGGGGGGB S(3\u21922)\n\nk\n\nk\n\nS(3\u21921)\n\n0\n\nS(3\u21922)\n\n0\n\nk\u2032\n\n+ P(2\u21923)\n+ P(1\u21923)\n\nk\u2032\n\n5\n\nk\u2032\n\n+ P(2\u21923)\n+ P(1\u21923)\n\nk\u2032\n\n.\n\n(15)\n\n\f\u03c81\n\nx1\n\n\u03c84\n\n\u03c83\n\nx3\n\n\u03c82\n\nx2\n\n\u03c86\n\nx4\n\n(a)\n\n\u03c85\n\n\u03c87\n\n\u03c81\n\nx1\n\n\u03c82\n\n\u03c83\n\nx2\n\n\u03c84\n\nx3\n\n\u03c86\n\n\u03c85\n\n(b)\n\n\u03c83(1)\n\n2\n\n\u03c83(2)\n\n1\n\n\u03c83(3)\n\n1\n\n\u03c87(1)\n\n1\n\n\u03c87(2)\n\n1\n\n\u03c81(1)\n\n1\n\n\u03c81(2)\n\n0.1\n\n\u03c8\u2032\n\n1(1)\n0.1\n\n\u03c8\u2032\n\n1(2)\n1\n\n\u03c84(1, \u00b7)\n\u03c84(2, \u00b7)\n\n\u03c84(\u00b7, 1)\n\n1\n0.1\n\n\u03c84(\u00b7, 2)\n\n0.1\n3\n\n\u03c85(1, \u00b7)\n\u03c85(2, \u00b7)\n\n\u03c82(1)\n\n1\n\u03c85(\u00b7, 1)\n\n\u03c82(2)\n\n0.1\n\n0.1\n3\n\n\u03c85(\u00b7, 2)\n\n2\n0.1\n\n\u03c85(\u00b7, 3)\n\n0.1\n1\n\n\u03c86(1, \u00b7)\n\u03c86(2, \u00b7)\n\n\u03c86(\u00b7, 1)\n\n0.1\n1\n\n\u03c86(\u00b7, 2)\n\n0.1\n0.1\n\n(c)\n\nFigure 2: Examples of non-trivial factor graphs. (a) Four variable factor graph with binary factors.\nThe factor leafs can be used to specify information about a particular variable. (b) Example of a\nsmall three variable cyclic graph. (c) Factors for (a) used in simulation experiments in Section. 5.1.\n\n4.3 Product Messages\n\nReaction networks that implement variable to factor node messages have a similar, but slightly\nsimpler, structure. Again, each components species of the message is catalyzed by all incoming\nmessages species but only of the same component species. The rate constant for all product message\nreactions is the same \u03baprod resulting in reactions of the following form\n\nP(n\u2192j)\n\n0\n\nS(j \u2032\u2192n)\n\n+ X\n\nk\nj \u2032\u2208ne(n)\\j\n\n\u03baprod\n\nGGGGGGGB P(n\u2192j)\n\nk\n\nS(j \u2032\u2192n)\n\n+ X\n\nk\nj \u2032\u2208ne(n)\\j\n\n.\n\n(16)\n\nThe dynamics of the message component species is given by\n\nd[P(n\u2192j)\n\nk\ndt\n\n]\n\n= \u03baprod[P(n\u2192j)\n\n0\n\n] Y\nj \u2032\u2208ne(n)\\j\n\n[S(j \u2032\u2192n)\n\nk\n\n] \u2212 \u03bar[P(n\u2192j)\n\nk\n\n].\n\nAt steady state the concentration of P(n\u2192j)\n\nis given by\n\nk\n\n\u03bar\n\n\u03baprod[P(n\u2192j)\n\n0\n\n]\n\n[P(n\u2192j)\n\nk\n\n] = Y\n\n[S(j \u2032\u2192n)\n\nk\n\n].\n\nj \u2032\u2208ne(n)\\j\n\n(17)\n\n(18)\n\n[P(n\u2192j)\nSince all component species of product messages have the same multiplier\n],\nthe steady state species concentrations compute the correct message according to Equation 4. For\nexample, the two different sets of variable to factor messages in Figure 1a are\n\n\u03baprod[P(n\u2192j)\n\n\u03bar\n\nk\n\n0\n\n]\n\nP(1\u21923)\n\n+ S(2\u21922)\nSimilarly, the reactions to compute the marginal probabilities of x1 and x2 in Figure 1a are\n\n+ S(1\u21921)\n\n+ S(1\u21921)\n\n+ S(2\u21922)\n\nP(2\u21923)\n\n0\n\n0\n\nk\n\nk\n\nk\n\nk\n\nk\n\nk\n\n\u03baprod\nGGGGB P(1\u21923)\n\n\u03baprod\nGGGGB P(2\u21923)\n\n0 + S(3\u21921)\nP1\n0 + S(3\u21922)\nP2\n\nk\n\nk\n\nk\n\n+ S(1\u21921)\n+ S(2\u21922)\n\nk\n\n\u03baprod\nGGGGB P1\n\u03baprod\nGGGGB P2\n\nk + S(3\u21921)\nk + S(3\u21922)\n\nk\n\nk\n\nk\n\n+ S(1\u21921)\n+ S(2\u21922)\n\n.\n\nk\n\n(19)\n\n.\n\n(20)\n\nThe two rate constants \u03baprod and \u03bar can be adjusted to tradeoff speed vs. accuracy, see Section 5.\nTogether, reactions for recycling probability mass, implementing sum-product messages, and imple-\nmenting product messages de\ufb01ne a reaction network whose equilibrium computes the messages and\nmarginal probabilities via the sum-product algorithm. As probability mass is continuously recycled,\nmessages computed on partial information will readjust and settle to the correct value. There is a\nclear dependence of messages. Sum messages from leaf nodes do not depend on any other mes-\nsages. Once they are computed, i.e. the reactions have equilibrated, the message species continue to\ncatalyze the next set of messages until they have reached the the correct value, etc.\n\n6\n\n\fPr(x1)\n\nPr(x2)\n\nPr(x3)\n\n1\n0\n.\n0\n=\n\nr\n\u03ba\n\n1\n.\n0\n=\n\nr\n\u03ba\n\nexact\nslow\nfast\n\nPr(x1)\n\n0.692\n0.690\n0.661\n\n0.308\n0.306\n0.294\n\nPr(x2)\n\n0.598\n0.583\n0.449\n\n0.402\n0.393\n0.302\n\n0.392\n0.394\n0.379\n\nPr(x3)\n0.526\n0.520\n0.508\n\n0.083\n0.083\n0.080\n\nPr(x4)\n\n0.664\n0.665\n0.646\n\n0.336\n0.333\n0.326\n\nFigure 3: Inference results for factor graph in Figure 2(a). Colored boxes show the trajectories of\na belief species set in a simulated reaction network. The simulation time (3000 sec) is along the\nx\u2013dimension. Half way though the simulation the factor attached to x1 changes from \u03c81 to \u03c8\u2032\n1, and\nthe exact marginal distribution for each period is shown as a back-white dashed line. The white area\nat the top indicates unassigned probability mass. These plots show the clear tradeoff between speed\n(higher value of \u03bar) and accuracy (less unassigned probability mass). The exact numerical answers\nat 3000 sec are given in the table.\n\n5 Simulation Experiments\n\nThis section presents simulation results of factor graphs that have been compiled into reaction net-\nworks via the procedure in Section 4. All simulations were performed using the SimBiology Toolbox\nin Matlab with the \u201csundials\u201d solver. The conserved concentrations for all sets of belief species were\nset to 1, so plots of concentrations can be directly interpreted as probabilities. Figure 2 shows two\ngraphical models for which we present detailed simulation results in the next two sections.\n\n5.1 Tree-Structured Factor Graphs\n\nTo demonstrate the functionality and features of the compilation procedure described in Section 4,\nwe compiled the 4 variable factor graph shown in Figure 2a into a reaction network. When x1, x2, x3\nand x4 have discrete states K1 = K2 = K4 = 2 and K3 = 3, the resulting network has 64 chemical\nspecies and 105 reactions. The largest reaction is of 5th order to compute the marginal distribution\nof x2. We instantiated the factors as shown in Figure 2c and initialized all message and marginal\nspecies to be uniform. To show that the network continuously performs inference and can adapt\n1 half way through the simulation. In terms of\nto new information, we changed the factor \u03c81 to \u03c8\u2032\ninformation, the new factor implies that Pr(x1 = 2) is suddenly more likely. In terms of reactions\nthe change means that S(1\u21921)\nIn a biological reaction\nnetwork, such a change could be induced by up-regulating, or activating a catalyst due to a new\nchemical signal. This new information changes the probability distribution of all variables in the\ngraph and the network equilibrates to these new values, see Figure 3.\nThe only two free parameters are \u03baprod and \u03bar. Since only \u03bar has an direct effect on all sets of belief\nspecies, we \ufb01xed \u03baprod = 50 and varied \u03bar. Small values of \u03bar results in better approximation as less\nof the probability mass in each belief species set is in an unassigned state. However, small values of\n\u03bar slow the dynamics of the network. Larger values of \u03bar result in faster dynamics, but more of the\nprobability mass remains unassigned, top white area in Figure 3. We should note, that at equilibrium,\nthe relative assignments of probabilities are still correct, see Equation 14 and Equation 18.\n\nis now more likely to turn into S(1\u21921)\n\n0\n\n.\n\n2\n\nThe compilation procedure also works for factor graphs with larger factors. When replacing the two\nof the binary factors \u03c85 and \u03c86 in Figure 2a with a new tertiary factor \u03c8\u2032\n5 that is connected to x2,x3,\nand x4 the compiled reaction network has 58 species and 115 reactions. The largest reaction is of\norder 4. Larger factors can reduce the number of species since there are fewer edges and associated\nmessages to represent, however, the domain sizes Kj of the individual factors grows exponentially\nand in the number of neighbors and thus require more reactions to implement.\n\n7\n\n\f(a)\n\n(b)\n\nFigure 4:\n(a) The belief of Pr(x1 = 1) as function of iteration in loopy belief propagation. All\nmessages are updated simultaneously at every time step. After 100 iterations the oscillations abate\nand the belief converges to the correct estimate indicated by the dashed line. (b) Trajectory of PAi\nspecies concentrations. The simulation time is 3000 sec and the different colors indicate the belief\nof about either of the two states. The dotted line indicates the exact marginal distribution of x1.\n\n5.2 Loopy Belief Propagation\nThese networks can also be used on factor graphs that are not trees. Figure 2b shows a cyclic graph\nwhich we compiled to reactions and simulated. When Kn = 2 for all variables the resulting reaction\nnetwork has 54 species and 84 reactions. We chose factor tables that anti-correlate neighbors and\nleaf factors that prefer the same state.\n\nFigure 4 shows the results of performing both loopy belief propagation and simulation results for\nthe compiled reaction network. Both exhibit decaying oscillations, but settle to the correct marginal\ndistribution. Since the reaction network is essentially performing damped loopy belief propagation\nwith an in\ufb01nitesimal time step, the reaction network implementation should always converge.\n\n6 Conclusion\n\nWe present a compilation procedure for taking arbitrary factor graphs over discrete random vari-\nables and construct a reaction network that performs the sum-product message passing algorithm\nfor computing marginal distributions.\n\nThese reaction networks exploit the fact that the message structure of the sum-product algorithm\nmaps neatly onto the model of mass action kinetics. By construction, conserved sets of belief species\nin the network perform implicit and continuous normalization of all messages and marginal distri-\nbutions. The correct behavior of the network implementation relies on higher order reactions to\nimplement multiplicative operations. However, physically high order reaction are exceedingly un-\nlikely to proceed in a single step. While we can simulate and validate our implementation with\nrespect to the mass action model, a physical implementation will require an additional translation\nstep, e.g. along the lines of [17] with intermediate species of binary reactions.\n\nOne aspect that this paper did not address, but we believe is important, is how parameter uncer-\ntainty and noise affect the reaction network implementations of inference algorithms. Ideally, they\nwould be robust to parameter uncertainty and random \ufb02uctuations. To address the former one could\ndirectly compute the parameter sensitivity in this deterministic model. To address the latter, we\nplan to look at other semantic interpretations of chemical reaction networks, such as the linear noise\napproximation or the stochastic chemical kinetics model.\n\nIn addition to further analyzing this particular algorithm we would like to implement others, e.g.\nmax-product, parameter learning, and dynamic state estimation, as reaction networks. We believe\nthat statistical inference provides the right tools for tackling noise and uncertainty at a microscopic\nlevel, and that reaction networks are the right language for specifying systems at that scale.\n\nAcknowledgements\n\nWe are grateful to Wyss Institute for Biologically Inspired Engineering at Harvard, especially Prof.\nRadhika Nagpal, for supporting this research. We would also like to thank our colleagues and\nreviewers for their helpful feedback.\n\n8\n\n\fReferences\n\n[1] Subhayu Basu, Yoram Gerchman, Cynthia H. Collins, Frances H. Arnold, and Ron Weiss. A synthetic\n\nmulticellular system for programmed pattern formation. Nature, 434:1130\u20131134, 2005.\n\n[2] Christopher M. Bishop. Pattern Recognition and Machine Learning. Information Science and Statistics.\n\nSpringer, 2006.\n\n[3] Luca Cardelli and Gianluigi Zavattaro. On the computational power of biochemistry. In Katsuhisa Hori-\nmoto, Georg Regensburger, Markus Rosenkranz, and Hiroshi Yoshida, editors, Algebraic Biology, volume\n5147 of Lecture Notes in Computer Science, pages 65\u201380. Springer Berlin Heidelberg, 2008.\n\n[4] Ho-Lin Chen, David Doty, and David Soloveichik. Deterministic function computation with chemical\nreaction networks. In Darko Stefanovic and Andrew Turber\ufb01eld, editors, DNA Computing and Molec-\nular Programming, volume 7433 of Lecture Notes in Computer Science, pages 25\u201342. Springer Berlin\nHeidelberg, 2012.\n\n[5] Yuan-Jyue Chen, Neil Dalchau, Niranjan Srinivas, Andrew Phillips, Luca Cardelli, David Soloveichik,\nand Georg Seelig. Programmable chemical controllers made from DNA. Nature Nanotechnology,\n8(10):755\u2013762, October 2013.\n\n[6] Matthew Cook, David Soloveichik, Erik Winfree, and Jehoshua Bruck. Programmability of chemical\nIn Algorithmic Bioprocesses, Natural Computing Series, pages 543\u2013584. Springer\n\nreaction networks.\nBerlin Heidelberg, 2009.\n\n[7] Tal Danino, Octavio Mondragon-Palomino, Lev Tsimring, and Jeff Hasty. A synchronized quorum of\n\ngenetic clocks. Nature, 463:326\u2013330, 2010.\n\n[8] Michael B. Elowitz and Stanislas Leibler. A synthetic oscillatory network of transcriptional regulators.\n\nNature, 403:335\u2013338, 2000.\n\n[9] A Hjelmfelt, E D Weinberger, and J Ross. Chemical implementation of neural networks and Turing\n\nmachines. Proceedings of the National Academy of Sciences, 88(24):10983\u201310987, 1991.\n\n[10] Erik Winfree Jongmin Kim, John J. Hop\ufb01eld. Neural network computation by in vitro transcriptional\n\ncircuits. In Advances in Neural Information Processing Systems 17 (NIPS 2004). MIT Press, 2004.\n\n[11] Marcelo O. Magnasco. Chemical kinetics is Turing universal. Phys. Rev. Lett., 78:1190\u20131193, Feb 1997.\n[12] Chengde Mao, Thomas H. LaBean, John H. Reif, and Nadrian C. Seeman. Logical computation using\n\nalgorithmic self-assembly of DNA triple-crossover molecules. Nature, 407:493\u2013496, 2000.\n\n[13] K. Oishi and E. Klavins. Biomolecular implementation of linear I/O systems. Systems Biology, IET,\n\n5(4):252\u2013260, 2011.\n\n[14] Lulu Qian, Erik Winfree, and Jehoshua Bruck. Neural network computation with DNA strand displace-\n\nment cascades. Nature, 475:368\u2013372, 2011.\n\n[15] Paul W. K Rothemund, Nick Papadakis, and Erik Winfree. Algorithmic self-assembly of DNA Sierpinski\n\ntriangles. PLoS Biol, 2(12):e424, 12 2004.\n\n[16] Georg Seelig, David Soloveichik, David Yu Zhang, and Erik Winfree. Enzyme-free nucleic acid logic\n\ncircuits. Science, 314(5805):1585\u20131588, 2006.\n\n[17] David Soloveichik, Georg Seelig, and Erik Winfree. DNA as a universal substrate for chemical kinetics.\n\nProceedings of the National Academy of Sciences, 107(12):5393\u20135398, 2010.\n\n[18] Benjamin Vigoda. Analog Logic: Continuous-Time Analog Circuits for Statistical Signal Processing.\n\nPhD thesis, Massachusetts Institute of Technology, 2003.\n\n[19] Jonathan S. Yedidia, W.T. Freeman, and Y. Weiss. Constructing free-energy approximations and gen-\nInformation Theory, IEEE Transactions on, 51(7):2282\u20132312,\n\neralized belief propagation algorithms.\n2005.\n\n[20] David Yu Zhang and Georg Seelig. DNA-based \ufb01xed gain ampli\ufb01ers and linear classi\ufb01er circuits. In\nYasubumi Sakakibara and Yongli Mi, editors, DNA Computing and Molecular Programming, volume\n6518 of Lecture Notes in Computer Science, pages 176\u2013186. Springer Berlin Heidelberg, 2011.\n\n9\n\n\f", "award": [], "sourceid": 1083, "authors": [{"given_name": "Nils", "family_name": "Napp", "institution": "Harvard University"}, {"given_name": "Ryan", "family_name": "Adams", "institution": "Harvard University"}]}