{"title": "Action-Model Based Multi-agent Plan Recognition", "book": "Advances in Neural Information Processing Systems", "page_first": 368, "page_last": 376, "abstract": "Multi-Agent Plan Recognition (MAPR) aims to recognize dynamic team structures and team behaviors from the observed team traces (activity sequences) of a set of intelligent agents. Previous MAPR approaches required a library of team activity sequences (team plans) be given as input. However, collecting a library of team plans to ensure adequate coverage is often difficult and costly. In this paper, we relax this constraint, so that team plans are not required to be provided beforehand. We assume instead that a set of action models are available. Such models are often already created to describe domain physics; i.e., the  preconditions and effects of effects actions. We propose a novel approach for recognizing multi-agent team plans based on such action models rather than libraries of team plans. We encode the resulting MAPR problem as a \\emph{satisfiability problem} and solve the problem using a state-of-the-art weighted MAX-SAT solver. Our approach also allows for incompleteness in the observed plan traces. Our empirical studies demonstrate that our algorithm is both effective and efficient in comparison to state-of-the-art MAPR methods based on plan libraries.", "full_text": "Action-Model Based Multi-agent Plan Recognition\n\nHankz Hankui Zhuo\n\nDepartment of Computer Science\n\nSun Yat-sen University, Guangzhou, China 510006\n\nzhuohank@mail.sysu.edu.cn\n\nQiang Yang\n\nHuawei Noah\u2019s Ark Research Lab\n\nCore Building 2, Hong Kong Science Park, Shatin, Hong Kong\n\nqyang@cse.ust.hk\n\nSubbarao Kambhampati\n\nDepartment of Computer Science and Engineering\n\nArizona State University, Tempe, Arizona, US 85287-5406\n\nrao@asu.edu\n\nAbstract\n\nMulti-Agent Plan Recognition (MAPR) aims to recognize dynamic\nteam structures and team behaviors from the observed team traces (ac-\ntivity sequences) of a set of intelligent agents. Previous MAPR ap-\nproaches required a library of team activity sequences (team plans) be\ngiven as input. However, collecting a library of team plans to ensure\nadequate coverage is often dif\ufb01cult and costly. In this paper, we relax\nthis constraint, so that team plans are not required to be provided be-\nforehand. We assume instead that a set of action models are available.\nSuch models are often already created to describe domain physics; i.e.,\nthe preconditions and effects of effects actions. We propose a novel ap-\nproach for recognizing multi-agent team plans based on such action\nmodels rather than libraries of team plans. We encode the resulting\nMAPR problem as a satis\ufb01ability problem and solve the problem using\na state-of-the-art weighted MAX-SAT solver. Our approach also allows\nfor incompleteness in the observed plan traces. Our empirical studies\ndemonstrate that our algorithm is both effective and ef\ufb01cient in compar-\nison to state-of-the-art MAPR methods based on plan libraries.\n\n1\n\nIntroduction\n\nMulti-Agent Plan Recognition (MAPR) seeks an explanation of observed team-action traces. From\nthe activity sequences of a set of agents, MAPR aims to identify the dynamic team structures and\nteam behaviors of agents. The MAPR problem has important applications in analyzing data from\nautomated monitoring, situation awareness, intelligence surveillance and analysis [4]. Many ap-\nproaches have been proposed in the past to automatically recognize team plans given an observed\nteam trace as input. For instance, Banerjee et al. [4, 3] proposed to formalize MAPR with a new\nmodel. They solved MAPR problems using a \ufb01rst-cut approach, provided that a fully observed team\ntrace and a library of full team plans were given as input. To relax the full observability constraint,\nZhuo and Li [19] proposed a MARS system to recognize team plans based on partially observed team\ntraces and libraries of partial team plans.\n\n1\n\n\fDespite the success of these previous approaches, they all assume that a library of team plans has\nbeen collected beforehand and provided as input. However, there are many applications where col-\nlecting and maintaining a library of team plans is dif\ufb01cult and costly. For example, in military opera-\ntions, it is dif\ufb01cult and expensive to collect team plans, since activities of team-mates may consume\nlots of resources such as ammunition and human labor. Collecting a smaller library is not an option\nsince it is infeasible to recognize team plans if they are not covered by the library. It is thus useful to\ndesign approaches for solving the MAPR problem where we do not require libraries of team plans\nto be known.\nIn this paper, we advocate replacing the plan library with a compact action model of the domain. In\ncontrast to plan libraries, action models are easier to specify (in terms of preconditions and effects\nof each type of activity). Moreover, in principle action models provide full coverage to recognize\nany team plans. The speci\ufb01c algorithmic framework we develop is called DARE, which stands for\nDomain- model based multi-Agent REcognition, to recognize multi-agent plans. DARE does not\nrequire plan libraries to be given as input. Instead, DARE takes as input a team trace and a set of\naction models. DARE also allows the observed traces to be incomplete, i.e., there can be To \ufb01ll these\ngaps, DARE leverages all possible constraints both from the plan traces and from its knowledge\nof how a plan works in terms of its causal structure. To do this, DARE \ufb01rst builds a set of hard\nconstraints that encode the correctness property of the team plans, and a set of soft constraints that\nencode the optimal utility property of team plans based on the input team trace and action models.\nAfter that, it solves all these constraints using a state-of-the-art weighted MAX-SAT solver, such as\nMaxSatz [10], and converts the solution to a set of team plans as output.\nWe organize the rest of the paper as follows. In the next section, we \ufb01rst introduce the related\nwork including single agent plan recognition and multi-agent plan recognition, and then give our\nformulation of the MAPR problem. After that, we present DARE and discuss its properties. Finally,\nwe evaluate DARE in the experimental section and present our conclusions.\n\n2 Related work\n\nThe plan recognition problem has been addressed by many researchers. Kautz and Allen proposed\nan approach to recognize plans based on parsing observed actions as sequences of subactions and\nessentially model this knowledge as a context-free rule in an \u201caction grammar\u201d [9]. Bui et al. pre-\nsented approaches to probabilistic plan recognition problems [5, 7]. Instead of using a library of\nplans, Ramrez and Geffner [12] proposed an approach to solving the plan recognition problem us-\ning slightly modi\ufb01ed planning algorithms, assuming the action models were given as input. Note\nthat action models can be created by experts or learnt by previous systems, such as ARMS [18] and\nLAMP [20]. Singla and Mooney proposed an approach to abductive reasoning using a \ufb01rst-order\nprobabilistic logic to recognize plans [15]. Amir and Gal addressed a plan recognition approach to\nrecognizing student behaviors using virtual science laboratories [1]. Ramirez and Geffner exploited\noff-the-shelf classical planners to recognize probabilistic plans [13]. Despite the success of these\nsystems, a limitation is that they all focus only on single agent plans.\nFor multi-agent plan recognition, Sukthankar and Sycara presented an approach that leveraged sev-\neral types of agent resource dependencies and temporal ordering constraints in the plan library to\nprune the size of the plan library considered for each observation trace [16]. Avrahami-Zilberbrand\nand Kaminka preferred a library of single agent plans to team plans, but identi\ufb01ed dynamic teams\nbased on the assumption that all agents in a team executing the same plan under the temporal con-\nstraints of that plan [2]. The constraint on activities of the agents that can form a team can be severely\nlimiting when team-mates can execute coordinated but different behaviors.\nInstead of using the assumption that agents in the same team should execute a common activity,\nbesides the approaches introduced in the introduction section [4, 3, 19], Sadilek and Kautz provided\na uni\ufb01ed framework to model and recognize activities that involved multiple related individuals\nplaying a variety of roles [14]; Masato et al. proposed a probabilistic model based on conditional\nrandom \ufb01elds to automatically recognize the composition of teams and team activities in relation to\na plan [11]. In these systems, although coordinated activities can be recognized, they either assume\nthere is a set of real-world GPS data available, or assume that team traces and team plans can be\nfully observed. In this paper, we allow that: (1) agents can execute coordinated different activities in\na team, (2) team traces can be partial, and (3) neither GPS data nor team plans are needed.\n\n2\n\n\f3 Problem De\ufb01nition\n\nWe \ufb01rst de\ufb01ne a team trace. Let = {1, 2, . . . , n} be a set of agents, and O = [otj] be an\nobserved team trace. Let otj be the observed activity executed by agent j at time step t, where\n0 < t \uf8ff T and 0 < j \uf8ff n. A team trace O is partial, if some elements in O are empty (denoted by\nnull), i.e., there are missing values in O.\nWe then de\ufb01ne an action model. In the STRIPS language [6], an action model is a tuple\nha, Pre(a), Add(a), Del(a)i, where a is an action name with zero or more parameters, Pre(a) is\na list of preconditions of a, Add(a) is a list of add effects, and Del(a) is a list of deleting effects.\nA set of action models is denoted by A. An action name with zero of more parameters is called an\nactivity. An observed activity otj in a partial team trace O is either an instantiated action of A or\nnoop or null, where noop is an empty activity that does nothing.\nAn initial state s0 is a set of propositions that describes a closed world state from which the team\ntrace O starts to be observed. In other words, activities at time step t = 0 can be applied in the initial\nstate s0. When we say an activity can be applied in a state, we mean the activity\u2019s preconditions are\nsatis\ufb01ed by the state. A set of goals G, each of which is a set of propositions, describes the probable\ntargets of the team trace. We assume s0 and G can both be observed by sensing devices.\nA team is composed of a subset of agents 0 = {j1, j2, . . . , jm}. A team plan is de\ufb01ned as\np = [atk]0<k\uf8ffm\n0<t\uf8ffT , where m \uf8ff n, and atk is an activity or noop. A set of correct team plans P is\nrequired to have properties P1-P5.\nP1: P is a partition of the team trace O, i.e., each element of O should be in exactly one p of P and\neach activity of p should be an element of O;\nP2: P should cover all the observed activities, i.e., for each p 2 P and 0 < t \uf8ff T and 0 < k \uf8ff m,\nif otjk 6= null, then atk = otjk, where atk 2 p and otjk 2O ;\nP3: P is executable starting from s0 and achieves some goal g 2 G, i.e., at\u21e4 is executable in state\nst1 for all 0 < t \uf8ff T , and achieves g after step T , where at\u21e4 = hat1, at2, . . . , atmi;\nP4: Each team plan p 2 P is associated with a likelihood \u00b5(p): P 7! R+. \u00b5(p) speci\ufb01es the\nlikelihood of recognizing team plan p, and can be affected by many factors, including the\nnumber of agents in the team, the cost of executing p, etc. The value of \u00b5(p) is composed of\ntwo parts \u00b51(Nactivity(p)) and \u00b52(Nagent(p)), i.e., \u00b5(p) =\n\u00b51(Nactivity(p))+\u00b52(Nagent(p)),\nwhere \u00b51(Nactivity(p)) depends on Nactivity(p), the number of activities of p, and\n\u00b52(Nagent(p)) depends on Nagent(p), the number of agents (i.e., team-mates) of p.\nGenerally, \u00b51(Nactivity(p)) (or \u00b52(Nagent(p))) becomes larger when Nactivity(p) (or\nNagent(p)) increases. Note that more agents would have a smaller likelihood (or larger\ncost) to coordinate these agents to successfully execute p. Thus, we require that \u00b52 should\nsatisfy the condition: \u00b52(n1 + n2) > \u00b52(n1) + \u00b52(n2). For each goal g 2 G, the output\nplan P should have the largest likelihood, i.e.,\n\n1\n\nP = arg max\n\nP 0 Xp2P 0\n\n\u00b5(p),\n\nwhere P 0 is a team-plan set that achieves g. Note that we presume that teams are (usually)\norganized with the largest likelihood.\n\nP5: Any pair of interacting agents must belong to the same team plan. In other words, if an agent\ni interacts with another agent j, i.e., i provides or deletes some conditions of j, then\ni and j should be in the same team, and activities of agents in the same team compose a\nteam plan. Agents exist in exactly one team plan, i.e., team plans do not share any common\nagents.\n\nOur multi-agent plan recognition problem can be stated as: Given a partially observed team trace\nO, a set of action models A, an initial state s0, and a set of goals G, the recognition algorithm must\noutput a set of team plans P with the maximal likelihood to achieve some goal g 2 G, where P\nsatis\ufb01es the properties P1-P5.\n\n3\n\n\fFigure 1: An example of the input and output of our problem from the blocks domain. (I) is an input\nexample, where \u201c(b) the initial state s0\u201d is a set of propositions: {(ontable A)(ontable B)(ontable\nF)(ontable G)(on C B)(on D F)(on E D)(clear A)(clear C)(clear E)(clear G)(handempty)}; \u201c(c) goals\n{g}\u201d is a goal set composed of one goal g, and g is composed of propositions: {(ontable A)(ontable\nD)(ontable E)(on B A)(on C B)(on F D)(on G F)(clear C)(clear G)(clear E)}. (II) is an output exam-\nple, which is the set of team plans {p1, p2}.\n\nFigure 1 shows an example multi-agent plan recognition problem from blocks world1. In part (a) of\nFigure 1(I), the \ufb01rst column indicates the time steps from 1 to 5. h1, . . . , h5 are \ufb01ve hoist agents.\nThe value null suggests the missing observation, and noop suggests the empty activity. We assume\n\u00b51 and \u00b52 are de\ufb01ned by: \u00b51(k) = k, and \u00b52(k) = k2. Based on \u00b51 and \u00b52, the corresponding\noutput is shown in Figure 1(II), which is the set of two team plans {p1, p2}.\n4 DARE Algorithm Framework\n\nAlgorithm 1 below describes the plan recognition process in DARE. In the subsequent subsections,\nwe describe each step of this algorithm in detail.\n\nAlgorithm 1 An overview of our algorithm framework\ninput: a partial team trace O, an initial state s0, a set of goals G, and a set of action models A;\noutput: a set of team plans P ;\n1: max = 0;\n2: for each g 2 G do\n3:\n4:\n5:\n6:\n7:\n8:\n9:\nend if\n10:\n11: end for\n12: return P ;\n\nbuild a set of candidate activities \u21e5;\nbuild a set of hard constraints based on \u21e5;\nbuild a set of soft constraints based on the likelihood \u00b5;\nsolve all the constraints using a weighted MAX-SAT solver, with hmax0, soli as output;\nif max0 > max then\n\nmax = max0;\nconvert the solution sol to a set of team plans P 0, and let P = P 0;\n\n4.1 Candidate activities\n\nIn Step 3 of Algorithm 1, we build a set of candidate activities \u21e5 by instantiating each parameter\nof action models in A with all objects in the initial state s0, team trace O and goal g. We perform\nthe following phases. We \ufb01rst scan each parameter of propositions (or activities) in s0, O, and g,\nand collect sets of different objects (note that each set of objects corresponds to a type, e.g., there\n\n1http://www.cs.toronto.edu/aips2000/\n\n4\n\n(a). team tracepickup(?x \u2013 block)precondition: (handempty)(clear ?x)(ontable ?x)effect: (holding ?x)(not (handempty)) (not (clear ?x))(not (ontable))putdown(?x \u2013 block)precondition: (holding ?x)effect: (clear ?x) (ontable ?x)(handempty) (not (holding ?x))unstack(?x \u2013 block ?y \u2013 block)precondition: (on ?x ?y) (clear ?x) (handempty)effect: (holding ?x)(clear ?y)(not (clear ?x)) (not (handempty))(not (on ?x ?y))stack(?x \u2013 block ?y \u2013 block)precondition: (holding ?x) (clear ?y)Effect: (not (holding ?x))(not (clear ?y)) (clear ?x) (handempty) (on ?x ?y)(d). action modelsa1: unstack(C B); a2: stack(B A); a3: unstack(E D); a4: stack (C B); a5: stack(F D); a6: putdown(D); a7: pickup(G); T123h1a1h3noopnoopnullnulla2h4nullnulla645a4noopnullnoopnoopnooph5a7noopnullnoopnullnullnullh2a3a5nullCBAEDFG(b). initial stateCBAEDFG(c). goals {g}(I). inputsT123h1a1h3noopnoopnoopa8a245a4noopnoopnoopa1: unstack(C B); a2: stack(B A); a3: unstack(E D); a4: stack (C B); a5: stack(F D); a6: putdown(D); a7: pickup(G); a8: pickup(B); a9: unstack(D F); a10: putdown(E); a11: pickup(F); a12: stack(G F);T123h4noopa9a645noopnooph5a7noopnoopnoopa12a10a11h2a3a5noop(a). team plan p1(b). team plan p2(II). outputs\fis a type \u201cblock\u201d in the blocks domain). Second, we substitute each parameter of each action model\nin A with its corresponding objects (the correspondence relationship is re\ufb02ected by type, i.e., the\nparameters of action models and objects should belong to the same type), which results in a set of\ndifferent activities, called candidate activities \u21e5. Note that we also add an noop activity in \u21e5.\nFor example, there are seven objects {A, B, C, D, E, F, G} corresponding to type \u201cblock\u201d in Fig-\nure 1(I). The set of candidate activities \u21e5 is: {noop, pickup(A), pickup(B), pickup(C), pickup(D),\npickup(E), pickup(F), pickup(G), . . .}, where the \u201cdots\u201d suggests other activities that are generated\nby instantiating parameters of actions \u201cputdown, stack, unstack\u201d.\n\n4.2 Hard constraints\n\nWith the set of candidate activities \u21e5, we build a set of hard constraints to ensure the properties P1 to\nP3 in Step 4 of Algorithm 1. We associate each element otj 2O with a variable vtj, i.e., we have a\nset of variables V = [vtj]0<j\uf8ffn\n0<t\uf8ffT , which is also called a variable matrix. Each variable in the variable\nmatrix will be assigned with a speci\ufb01c activity in candidate activities \u21e5, and we will partition these\nvariables to attain a set of team plans that have the properties P1-P5 based on the assignments.\nAccording to properties P2 and P3, we build two kinds of hard constraints: Observation constraints\nand Causal-link constraints. Note that P1 is guaranteed since the set of team plans that is output is\na partition of the team trace.\n\n(otj 6= null) ! (vtj = otj).\n\nObservation constraints For P2, i.e., given a team plan p = [atk]0<k\uf8ffm\n\n0<t\uf8ffT composed of agents\n0 = {j1, j2, . . . , jm}, if otjk 6= null, then atk = otjk, this suggests vtjk should have\nthe same activity of otjk if otjk 6= null, since the team plan p is a partition of V and atk\nis an element of of p. Thus, we build hard constraints as follows. For each 0 < t \uf8ff T and\n0 < j \uf8ff n, we have\nWe call this kind of hard constraints the observation constraints, since they are built based\non the partially observed activities of O.\nCausal-link constraints For P3, i.e., each team plan p should be executable starting from the initial\nstate s0, this suggests each row of variables hvt1, vt2, . . . , vtni should be executable, where\n0 < t \uf8ff T . Note that \u201cexecutable\u201d suggests that the preconditions of vtj should be satis\ufb01ed.\nThis means, for each 0 < t \uf8ff T and 0 < j \uf8ff n, the following constraints should be\nsatis\ufb01ed:\n\u2022 each precondition of vtj either exists in the initial state s0 or is added by vt0j0, and is\nnot deleted by any activity between t0 and t, where t0 < t and 0 < j0 \uf8ff n.\n\u2022 likewise, each proposition in goal g either exists in the initial state s0 or is added\nby vt0j0, and is not deleted by any activity between t0 and T , where t0 < T and\n0 < j0 \uf8ff n.\n\nWe call this kind of hard constraints causal-link constraints, since they are created accord-\ning to the causal link requirement of executable plans.\n\n4.3 Soft constraints\n\nIn Step 5 of Algorithm 1, we build a set of soft constraints based on the likelihood function \u00b5. Each\nvariable in V can be assigned with any element of the candidate activities \u21e5. We require that all\nvariables in V should be assigned with exactly one activity from \u21e5. For each ha1, a2, . . . , a|V |i 2\n\u21e5 \u21e5 . . . \u21e5 \u21e5, we have\n\n^0<i\uf8ff|V |\n\n(vi = ai).\n\nWe calculate the weights of these constraints by the following phases. First, we partition the variable\nmatrix V based on property P5 into a set of team plans P , i.e., agent i provides or deletes some\nconditions of j, then i and j should be in the same team, and activities of agents in the same\nteam compose a team plan. Second, for all team plans, we calculate the total likelihood \u00b5(P ), i.e.,\n\n\u00b5(P ) =Xp2P\n\n\u00b5(p) =Xp2P\n\n1\n\n\u00b51(Nactivity(p)) + \u00b52(Nagent(p))\n\n,\n\n5\n\n\fand let \u00b5(P ) be the weights of the soft constraints. Note that we aim to maximize the total likelihood\nwhen solving these constraints (together with hard constraints) with a weighted MAX-SAT solver.\n\n4.4 Solving the constraints\n\nIn Step 6 of Algorithm 1, we put both hard and soft constraints together, and solve these constraints\nusing Maxsatz [10], a MAX-SAT solver . The solution sol is an assignment for all variables in V ,\nand max0 is the total weight of the satis\ufb01ed constraints corresponding to the solution sol. In Step 8\nof Algorithm 1, we partition V into a set of team plans P based on P5.\nAs an example, in (a) of Figure 1(I), the team trace\u2019s corresponding variable in V is assigned with\nactivities, which means the null values in (a) of Figure 1(I) are replaced with the corresponding\nassigned activities in V . According to property P5, we can simply partition the team trace into two\nteam plans, as is shown in Figure 1(II), by checking preconditions and effects of activities in the\nteam trace.\n\n4.5 Properties of DARE\n\nDARE can be shown to have the following properties:\n\nTheorem 1: (Conditional Soundness) If the weighted MAX-SAT solver is powerful enough to\n\noptimally solve all solvable SAT problems, DARE is sound.\n\nTheorem 2: (Conditional Completeness) If the weighted MAX-SAT solver we exploit in DARE is\n\ncomplete, DARE is also complete.\n\nFor Theorem 1, we only need to check that the solutions output by DARE satisfy P1-P5. P2 and P3\nare guarranteed by observation constraints and causal-link constraints; P4 is guaranteed by the soft\nconstraints built in Section 4.3 and the MAX-SAT solver; P1 and P5 are both guaranteed by the\npartition step in Section 4.4, i.e., partitioning the variable matrix into a set of team plans; that is to\nsay, the conditional soundness property holds.\nFor Theorem 2, since all steps in Algorithm 1, except Step 6 that calls a weighted MAX-SAT solver,\ncan be executed in \ufb01nite time, the completeness property only depends on the weighted MAX-SAT\nsolver, which means the conditional completeness property holds.\n\n5 Experiments\n\n5.1 Dataset and Evaluation Criterion\nWe evaluate DARE in three planning domains: blocks, driverlog2 and rovers2. We modify the three\ndomains for multi-agent setting. In blocks, there are multiple hoists, which are viewed as agents\nthat perform actions of pickup, putdown, stack and unstack. In driverlog, there are multiple trucks,\ndrivers and hoists, which are agents that can group together to form different teams (trucks and\ndrivers can be in the same team, likewise for hoists.). In rovers, there are multiple rovers that can\ngroup together to form different teams. For each domain, we set T = 50 and generate 50 team traces\nwith the size of T \u21e5 n for each n 2{ 20, 40, 60, 80, 100}. For each team trace, we have a set of\noptimal team plans (which is viewed as the ground truth), denoted by Ptrue, and its corresponding\ngoal gtrue, which best explains the team trace according to the likelihood function \u00b5. We de\ufb01ne the\nlikelihood function by: \u00b5 = \u00b51 + \u00b52, where \u00b51(k) = k and \u00b52(k) = k2, as is presented in the end\nof the problem de\ufb01nition section.\nWe randomly delete a subset of activities from each team trace with respect to a speci\ufb01c percentage\n\u21e0. We will test different \u21e0 values with 0%, 10%, 20%, 30%, 40%, 50%. As an example, \u21e0 = 10%\nsuggests there are 10 activities deleted from a team trace with 100 activities. We also randomly\nadd 10 additional goals, together with gtrue, to form the goal set G, as is presented in the problem\nde\ufb01nition section. We de\ufb01ne the accuracy  by:\n\n =\n\nthe number of correctly recognized team plan sets\n\nthe total number of team traces\n\n,\n\n2http://planning.cis.strath.ac.uk/competition/\n\n6\n\n\fwhere \u201ccorrectly recognized team plan sets\u201d suggests the recognized team plan sets and goals are\nthe same as the expected team plan sets {Ptrue} and goals G.\nWe generate 100 team plans as the library as is described by MARS [19], and compare the recognition\nresults with MARS as a baseline.\n\n5.2 Experimental Results\n\nWe evaluate DARE in the following aspects: (1) accuracy with respect to different number of agents;\n(2) accuracy with respect to different percentages of null values; and (3) the running time.\n\n5.2.1 Varying the number of agents\n\n(a). blocks\n\n(b). driverlog\n\n(c). rovers\n\n0.95\n\n0.9\n\n0.85\n\n0.8\n\n\u03bb\n\n0.75\n\n0.7\n\n0.65\n\n0.6\n\n20\n\n\u2190 MARS\n\n\u2190 DARE\n\n40\n\n60\n\n80\n\n100\n\nnumber of agents\n\n0.95\n\n0.9\n\n0.85\n\n0.8\n\n\u03bb\n\n0.75\n\n0.7\n\n0.65\n\n0.6\n\n20\n\n\u2190 MARS\n\nDARE\u2192\n\n40\n\n60\n\n80\n\n100\n\nnumber of agents\n\n0.95\n\n0.9\n\n0.85\n\n0.8\n\n\u03bb\n\n0.75\n\n0.7\n\n0.65\n\n0.6\n\n20\n\n\u2190 MARS\n\n\u2190 DARE\n\n40\n\n60\n\n80\n\n100\n\nnumber of agents\n\nFigure 2: Accuracies with respect to different number of agents\n\nWe would like to evaluate the change of accuracies when the number of agents increases. We set the\npercentage of null values to be 30%, and also ran DARE \ufb01ve times to calculate an average of accu-\nracies. The result is shown in Figure 2. From the \ufb01gure, we found that the accuracies of both DARE\nand MARS generally decreased when the number of agents increased. This is because the problem\nspace is enlarged when the number of agents increases, which makes the available information be\ndecreased comparing to the large problem space, and not enough to attain high accuracies.\nWe also found that the accuracy of DARE was lower than MARS at the beginning, and then became\nbetter than MARS as the number of agents became larger. This indicates that DARE has better per-\nformance in handling large number of agents based on action models. This is because DARE builds\nthe MAX-SAT problem space (described as proposition variables and constraints) based on model\ninferences (i.e., action models), while MARS is based on instances (i.e., plan library). When the num-\nber of agents is small, the problem space built by MARS is smaller than that built by DARE; when the\nnumber of agents becomes larger, the problem space built by MARS becomes larger than that built\nby DARE; the larger the problem space is, the more dif\ufb01cult it is for MAX-SAT to solve the problem;\nthus, DARE performs worse than MARS with less agents, while better with more agents.\n\n(a). blocks\n\n(b). driverlog\n\n(c). rovers\n\n\u2190 DARE\n\n\u2190 MARS\n\n1.05\n\n1\n\n0.95\n\n0.9\n\n\u03bb\n\n0.85\n\n0.8\n\n0.75\n\n0.7\n\n1.05\n\n1\n\n0.95\n\n0.9\n\n0.85\n\n0.8\n\n0.75\n\n\u03bb\n\n\u2190 MARS\n\nDARE\u2192\n\n1.05\n\n1\n\n0.95\n\n0.9\n\n\u03bb\n\n0.85\n\n0.8\n\n0.75\n\n0.7\n\n\u2190 MARS\n\n\u2190 DARE\n\n0.65\n\n0\n\n10\n\n20\n30\npercentage\n\n40\n\n50\n\n0.7\n\n0\n\n10\n\n20\n30\npercentage\n\n40\n\n50\n\n0.65\n\n0\n\n10\n\n20\n30\npercentage\n\n40\n\n50\n\nFigure 3: Accuracies with respect to different percentages of null values.\n\n5.2.2 Varying the percentage of null values\nWe set the number of agents to be 60, and run DARE \ufb01ve times to calculate average of accuracies\nwith a percentage \u21e0 of null values. We found both accuracies of DARE and MARS decreased when\n\n7\n\n\fthe percentage \u21e0 increased, due to less information provided when the percentage increasing. When\nthe percentage is 0%, both DARE and MARS can recognize all the team traces successfully.\nBy observing all three domains in Figure 3, we \ufb01nd that DARE does not function as well as MARS\nwhen the percentage of incompleteness is large. This relative advantage for the library-based ap-\nproach is due in large part to the fact that all team plans to be recognized are covered by the small\nlibrary in the experiment, and the library of team plans will help reduce the recognition problem\nspace compared to DARE. We conjecture that if the team plans to be recognized are not covered by\nthe library (because of the size restrictions on the library), DARE will perform better than MARS. In\nthis case, MARS cannot successfully recognize some team plans.\n\n5.2.3 The running time\n\n(a) blocks\n\n(b) driverlog\n\n(c) rovers\n\n)\ns\nd\nn\no\nc\ne\ns\n(\n \n\ne\nm\n\ni\nt\n \n\nu\np\nc\n\n1200\n\n1000\n\n800\n\n600\n\n400\n\n200\n\n0\n\n20\n\n60\n\n40\n80\nnumber of agents\n\n100\n\n)\ns\nd\nn\no\nc\ne\ns\n(\n \n\ne\nm\n\ni\nt\n \n\nu\np\nc\n\n1200\n\n1000\n\n800\n\n600\n\n400\n\n200\n\n0\n\n20\n\n60\n\n40\n80\nnumber of agents\n\n100\n\n)\ns\nd\nn\no\nc\ne\ns\n(\n \n\ne\nm\n\ni\nt\n \n\nu\np\nc\n\n1200\n\n1000\n\n800\n\n600\n\n400\n\n200\n\n0\n\n20\n\n60\n\n40\n80\nnumber of agents\n\n100\n\nFigure 4: The CPU time of DARE.\n\nWe show the average CPU time of DARE over 50 team traces with respect to different number of\nagents in Figure 4. As can be seen from the \ufb01gure, the running time increases polynomially with the\nnumber of input agents. This can be veri\ufb01ed by \ufb01tting the relationship between the number of agents\nand the running time to a performance curve with a polynomial of order 2 or 3. For example, the \ufb01t\npolynomial for blocks is 0.0821x2 + 20.1171x  359.8.\n6 Final Remark\n\nIn this paper, we presented a system called DARE for recognizing multi-agent team plans from\nincomplete observed plan traces based on action models . This approach has signi\ufb01cant advantage\nover previous approaches that make use of a library of prede\ufb01ned team plans. Such plan libraries\nare dif\ufb01cult to obtain in many applications. With the action model based approach, we \ufb01rst build a\nset of candidate activities, and then build sets of hard and soft constraints to \ufb01nally recognize team\nplans. Our experiments show that DARE is effective in three benchmark domains compared to the\nstate-of-the-art multi-agent plan recognition system MARS that relies on a library of team plans. Our\napproach is thus well suited for scenarios where collecting a library of team plans is infeasible before\nperforming team plan recognition tasks.\nIn the current work, we assume that the action models are complete. A more realistic assumption\nis to allow the models to be incomplete [8, 17]. In future, we plan to extend DARE to work with\nincomplete action models. Another assumption in the current model is that it expects as input the\nalternative sets of goals, one of which the observed plan is expected to be targeting. We plan to relax\nthis so DARE can take as input a set of potential goals, with the understanding that the observed plan\nis achieving a bounded subset of these goals. We believe that both these extensions can be easily\naccommodated into the MAX-SAT framework of DARE.\n\nAcknowledgments\nHankz Hankui Zhuo thanks Natural Science Foundation of Guangdong Province of China (No.\nS2011040001869) and Research Fund for the Doctoral Program of Higher Education of China\n(No. 20110171120054) for the support of this research. Qiang Yang thanks Hong Kong RGC GRF\nProjects 621010 and 621211 for the support of this research. Kambhampati\u2019s research is supported\nin part by the NSF grant IIS201330813 and ONR grants N00014-09-1-0017, N00014-07-1-1049,\nand N000140610058.\n\n8\n\n\fReferences\n\n[1] Ofra Amir and Yaakov (Kobi) Gal. Plan recognition in virtual laboratories. In Proceedings of\n\nIJCAI, 2011.\n\n[2] Dorit Avrahami-Zilberbrand and Gal A. Kaminka. Towards dynamic tracking of multi-agents\nteams: An initial report. In Proceedings of the AAAI Workshop on Plan, Activity, and Intent\nRecognition (PAIR 2007), 2007.\n\n[3] Bikramjit Banerjee and Landon Kraemer. Branch and price for multi-agent plan recognition.\n\nIn Proceedings of AAAI, 2011.\n\n[4] Bikramjit Banerjee, Landon Kraemer, and Jeremy Lyle. Multi-agent plan recognition: formal-\n\nization and algorithms. In Proceedings of AAAI, 2010.\n\n[5] Hung H. Bui. A general model for online probabilistic plan recognition. In Proceedings of\n\nIJCAI, 2003.\n\n[6] R. Fikes and N. J. Nilsson. STRIPS: A new approach to the application of theorem proving to\n\nproblem solving. Arti\ufb01cial Intelligence Journal, pages 189\u2013208, 1971.\n\n[7] Christopher W. Geib and Robert P. Goldman. A probabilistic plan recognition algorithm based\n\non plan tree grammars. Arti\ufb01cial Intelligence, 173(11):1101\u20131132, 2009.\n\n[8] Subbarao Kambhampati. Model-lite planning for the web age masses: The challenges of plan-\n\nning with incomplete and evolving domain models. In AAAI, 2007.\n\n[9] Henry A. Kautz and James F. Allen. Generalized plan recognition. In Proceedings of AAAI,\n\n1986.\n\n[10] Chu Min LI, Felip Manya, Nouredine Mohamedou, and Jordi Planes. Exploiting cycle struc-\nIn In proceedings of 12th international conference on the Theory and\n\ntures in Max-SAT.\nApplications of Satis\ufb01ability Testing (SAT-09), pages 467\u2013480, 2009.\n\n[11] Daniele Masato, Timothy J. Norman, Wamberto W. Vasconcelos, and Katia Sycara. Agent-\n\noriented incremental team and activity recognition. In Proceedings of IJCAI, 2011.\n\n[12] Miquel Ramrez and Hector Geffner. Plan recognition as planning. In Proceedings of IJCAI,\n\n2009.\n\n[13] Miquel Ramrez and Hector Geffner. Probabilistic plan recognition using off-the-shelf classical\n\nplanners. In Proceedings of AAAI, 2010.\n\n[14] Adam Sadilek and Henry Kautz. Recognizing multi-agent activities from gps data. In Pro-\n\nceedings of AAAI, 2010.\n\n[15] Parag Singla and Raymond Mooney. Abductive markov logic for plan recognition. In Pro-\n\nceedings of AAAI, 2011.\n\n[16] Gita Sukthankar and Katia Sycara. Hypothesis pruning and ranking for large plan recognition\n\nproblems. In Proceedings of AAAI, 2008.\n\n[17] Minh Do. Tuan Nguyen, Subbarao Kambhampati. Synthesizing robust plans under incomplete\n\ndomain models. In Proc. AAAI Workshop on Generalized Planning, 2011.\n\n[18] Qiang Yang, Kangheng Wu, and Yunfei Jiang. Learning action models from plan examples\n\nusing weighted MAX-SAT. Arti\ufb01cial Intelligence, 171:107\u2013143, February 2007.\n\n[19] Hankz Hankui Zhuo and Lei Li. Multi-agent plan recognition with partial team traces and plan\n\nlibraries. In Proceedings of IJCAI, 2011.\n\n[20] Hankz Hankui Zhuo, Qiang Yang, Derek Hao Hu, and Lei Li. Learning complex action models\n\nwith quanti\ufb01ers and implications. Arti\ufb01cial Intelligence, 174(18):1540 \u2013 1569, 2010.\n\n9\n\n\f", "award": [], "sourceid": 194, "authors": [{"given_name": "Hankz", "family_name": "Zhuo", "institution": null}, {"given_name": "Qiang", "family_name": "Yang", "institution": null}, {"given_name": "Subbarao", "family_name": "Kambhampati", "institution": null}]}