{"title": "Incremental A*", "book": "Advances in Neural Information Processing Systems", "page_first": 1539, "page_last": 1546, "abstract": "", "full_text": "Incremental A\n\nS. Koenig and M. Likhachev\nGeorgia Institute of Technology\n\nCollege of Computing\nAtlanta, GA 30312-0280\n\n\u0001 skoenig, mlikhach\n\n@cc.gatech.edu\n\nAbstract\n\nIncremental search techniques \ufb01nd optimal solutions to series of similar\nsearch tasks much faster than is possible by solving each search task\nfrom scratch. While researchers have developed incremental versions of\nuninformed search methods, we develop an incremental version of A*.\nThe \ufb01rst search of Lifelong Planning A* is the same as that of A* but\nall subsequent searches are much faster because it reuses those parts of\nthe previous search tree that are identical to the new search tree. We then\npresent experimental results that demonstrate the advantages of Lifelong\nPlanning A* for simple route planning tasks.\n\n1 Overview\nArti\ufb01cial intelligence has investigated knowledge-based search techniques that allow one\nto solve search tasks in large domains. Most of the research on these methods has studied\nhow to solve one-shot search problems. However, search is often a repetitive process,\nwhere one needs to solve a series of similar search tasks, for example, because the actual\nsituation turns out to be slightly different from the one initially assumed or because the\nsituation changes over time. An example for route planning tasks are changing traf\ufb01c\nconditions. Thus, one needs to replan for the new situation, for example if one always\nwants to display the least time-consuming route from the airport to the conference center\non a web page. In these situations, most search methods replan from scratch, that is, solve\nthe search problems independently. Incremental search techniques share with case-based\nplanning, plan adaptation, repair-based planning, and learning search-control knowledge\nthe property that they \ufb01nd solutions to series of similar search tasks much faster than is\npossible by solving each search task from scratch. Incremental search techniques, however,\ndiffer from the other techniques in that the quality of their solutions is guaranteed to be as\ngood as the quality of the solutions obtained by replanning from scratch.\n\nAlthough incremental search methods are not widely known in arti\ufb01cial intelligence and\ncontrol, different researchers have developed incremental search versions of uninformed\nsearch methods in the algorithms literature. An overview can be found in [FMSN00].\nWe, on the other hand, develop an incremental version of A*, thus combining ideas from\nthe algorithms literature and the arti\ufb01cial intelligence literature. We call the algorithm\nLifelong Planning A* (LPA*), in analogy to \u201clifelong learning\u201d [Thr98], because it reuses\n\n\u0003 We thank Anthony Stentz for his support. The Intelligent Decision-Making Group is partly supported by NSF awards under contracts IIS-\n\n9984827, IIS-0098807, and ITR/AP-0113881. The views and conclusions contained in this document are those of the authors and should not be\ninterpreted as representing the of\ufb01cial policies, either expressed or implied, of the sponsoring organizations and agencies or the U.S. government.\n\n\u0002\n\finformation from previous searches. LPA* uses heuristics to focus the search and always\n\ufb01nds a shortest path for the current edge costs. The \ufb01rst search of LPA* is the same as that\nof A* but all subsequent searches are much faster. LPA* produces at least the search tree\nthat A* builds. However, it achieves a substantial speedup over A* because it reuses those\nparts of the previous search tree that are identical to the new search tree.\n\n. Similarly,\n\n.\n\n\b?\u000f@\n\n2 The Route Planning Task\nLifelong Planning A* (LPA*) solves the following search task: It applies to \ufb01nite graph\nsearch problems on known graphs whose edge costs can increase or decrease over time.\ndenotes the \ufb01nite set of vertices of the graph.\n\u0002\u0001\u0004\u0003\u0005\u0003\u0007\u0006\t\b\u000b\n\r\f\u000e\nof vertex\ndenotes the cost of moving from vertex\n\u001c\u001e\u001d\nalways determines a shortest path from a given start vertex\nvertex\nuse\nfrom\n\ndenotes the set of successors\n.\n\b\u0010\u000f\u0011\n. LPA*\nto a given goal\n, knowing both the topology of the graph and the current edge costs. We\n, that is, the length of a shortest path\n\ndenotes the set of predecessors of vertex\n\b\u0015\"(\u000f)*\u0001+\u0003\u0005\u0003,\u0006\t\b\u000b\n\n\u0012\r\u0013\u0015\u0014\u000b\u0016\u0017\u0006\u0018\b\u0019\n\u001a\f\u001b\n\n\b\u0010\u000f\u0011\n\u0003\u0007\u0006\u0018\b \u001f!\b\u0019\"#\n%$'&\n\u000f\u0011\n\b87\u00059\n0;:\nto denote the start distance of vertex\n<>=,\u0006\t\b\u000b\n\nto\n\b\u000b-/.10\u00052A.\n\nTo motivate and test LPA*, we use a special case of these search tasks that is easy to\nvisualize. We apply LPA* to navigation problems in known eight-connected gridworlds\nwith cells whose traversability can change over time. They are either traversable (with cost\none) or untraversable. LPA* always determines a shortest path between two given cells of\nthe gridworld, knowing both the topology of the gridworld and which cells are currently\nblocked. This is a special case of the graph search problems on eight-connected grids whose\nedge costs are either one or in\ufb01nity. As an approximation of the distance between two cells,\nwe use the maximum of the absolute differences of their x and y coordinates. This results\nin consistent heuristics that are for eight-connected grids what Manhattan distances are for\nfour-connected grids.\n\nto vertex\n\b,-/.10324.5\u000f6\n\n3 Reusing Information from Previous Searches\nThe graph search problems can be solved with traditional graph-search methods, such as\nbreadth-\ufb01rst search, if they update the shortest path every time some edge costs change.\nThey typically do not take advantage of information from previous searches. The following\nexample, however, shows that this can be advantageous.\n\n\u001c5CED\u000bF\n\nshown in Figure 1. The original gridworld is\nConsider the gridworlds of size\nshown on top and the changed gridworld is shown at the bottom. The traversability of only\na few cells has changed. In particular, three blocked cells became traversable (namely, B3,\nC5, and D2) and three traversable cells became blocked (namely, A1, A4, D3). Thus, two\npercent of the cells changed their status but the obstacle density remained the same. The\n\ufb01gure shows the shortest paths in both cases, breaking ties towards the north. Note that we\nassume that one can squeeze through diagonal obstacles. (This is just an artifact of how we\ngenerated the underlying graphs from the mazes.) The shortest path changed since one cell\non the original shortest path became blocked.\n\nOnce the start distances of all cells are known, one can easily trace back a shortest path\nfrom the start cell to the goal cell by always greedily decreasing the start distance, starting\nto\nat the goal cell. This is similar to how A* traces the shortest path back from\nusing the search tree it has constructed. Thus, we only need to determine the start\n-/.10324.\ndistances. The start distances are shown in each traversable cell of the original and changed\ngridworlds. Those cells whose start distances in the changed gridworld have changed from\nthe corresponding ones in the original gridworld are shaded gray.\n\n\b\u00157\u00059\n\n03:\n\nThere are two different ways of decreasing the search effort for determining the start dis-\ntances for the changed gridworld. First, some start distances have not changed and thus\nneed not get recomputed. This is what DynamicSWSF-FP [RR96] does. (DynamicSWSF-\n\n\n\b\n\b\nB\n\b\n\fOriginal Eight-Connected Gridworld\n2\n7\n6\n\n5\n11\n10\n\n12\n\n11\n\n11\n\n7\n\n3\n\n8\n\n10\n\n14\nChanged Eight-Connected Gridworld\n7\n6\n\n11\n10\n\n12\n\n11\n\n11\n\n7\n\nA\n\nB\n\nC\nD\n\n1\n\n7\n\n5\n4\n3\n2\n2\n2\n3\n4\n5\n6\n\n7\n\n5\n5\n4\n\n2\n2\n3\n4\n5\n6\n\n7\n6\n\n4\n\n3\n\n3\n4\n\n6\n7\n8\n\n7\n6\n\n5\n\n3\n\n3\n4\n\n6\n7\n8\n\n7\n\n4\n\n1\n\n6\n7\n8\n\n7\n\n4\n\n1\n\n6\n7\n8\n\n5\n\n1\n1\n\n5\n5\n\n3\n2\n1\nsstart\n\n6\n\n6\n7\n7\n\n5\n\n1\n1\n\n5\n5\n6\n\n3\n2\n1\nsstart\n\n6\n\n6\n7\n7\n8\n\n7\n6\n5\n4\n3\n2\n\n2\n\n4\n5\n6\n7\n8\n\n7\n6\n5\n4\n3\n2\n\n2\n\n4\n5\n6\n7\n8\n\n3\n3\n\n3\n\n4\n4\n\n6\n7\n8\n\n9\n\n7\n8\n9\n\n9\n\n3\n3\n\n3\n\n6\n7\n8\n\n4\n4\n\n7\n8\n9\n\n10\n9\n8\n7\n6\n5\n5\n\n5\n\n8\n8\n8\n9\n\n10\n9\n8\n7\n6\n5\n5\n\n5\n\n8\n8\n8\n9\n\n10\n9\n8\n\n6\n\n6\n\n8\n\n9\n9\n\n6\n\n6\n\n8\n\n9\n9\n\n10\n9\n8\n\n9\n\n9\n9\n9\n\n4\n11\n10\n9\n8\n\n8\n\n8\n\n8\n8\n9\n10\n\n11\n10\n9\n8\n\n9\n8\n7\n\n7\n\n7\n7\n7\n8\n9\n\n9\n8\n7\n\n12\n\n8\n\n9\n9\n9\n\n8\n8\n9\n10 10\n\n7\n\n7\n7\n7\n8\n9\n\n10\n\n16 16\n15\n14\n13\n\n14\n14\n14\n\nsgoal\n13\n\n13\n13\n\n19\n18\n17\n16\n\nsgoal\n13\n\n13\n13\n\n14\n\n14\n\n14\n14\n14\n14\n\n19\n\n17\n17\n17\n\n14\n\n14\n\n14\n14\n14\n14\n\n12\n\n12\n12\n\n12\n12\n12\n\n15\n\n12\n12\n\n12\n12\n12\n\n14\n\n16\n\n15\n15\n16\n15\n15 16\n16\n16\n16\n16\n\n15\n15\n15\n15\n15\n15\n15\n15\n\n18\n18\n18\n18\n\n15\n15\n15\n15\n15\n15\n15\n15\n\n16\n\n16\n\n19\n\n19\n18\n17\n16\n16\n16\n\n16\n\n16\n\n13\n12\n11 11\n10\n\n11\n10 11\n10\n10\n10\n10\n10\n11\n12\n\n11\n11\n11\n\n16\n15\n14\n\n12\n11\n\n11\n11\n11\n\n14\n13\n\n11\n10\n10\n10\n10\n10\n11\n11\n\nFigure 1: Simple Gridworld\n\nFP, as originally stated, searches from the goal vertex to the start vertex and thus maintains\nestimates of the goal distances rather than the start distances.\nIt is a simple matter of\nrestating it to search from the start vertex to the goal vertex. Furthermore, DynamicSWSF-\nFP, as originally stated, recomputes all goal distances that have changed. To avoid bias-\ning our experimental results in favor of LPA*, we changed the termination condition of\nDynamicSWSF-FP so that it stops immediately after it is sure that it has found a shortest\npath.) Second, heuristic knowledge, in form of approximations of the goal distances, can\nbe used to focus the search and determine that some start distances need not get computed\nat all. This is what A* [Pea85] does. We demonstrate that the two ways of decreasing the\nsearch effort are orthogonal by developing LPA* that combines both of them and thus is\nable to replan faster than either DynamicSWSF-FP or A*.\n\nFigure 2 shows in gray those cells whose start distances each of the four algorithms recom-\nputes. (To be precise: it shows in gray the cells that each of the four algorithms expands.)\nDuring the search in the original gridworld, DynamicSWSF-FP computes the same start\ndistances as breadth-\ufb01rst search during the \ufb01rst search and LPA* computes the same start\ndistances as A*. During the search in the changed gridworld, however, both incremental\nsearch (DynamicSWSF-FP) and heuristic search (A*) individually decrease the number of\nstart distances that need to get recomputed compared to breadth-\ufb01rst search, and together\n(LPA*) decrease the number even more.\n\n4 Lifelong Planning A*\nto\nLifelong Planning A* (LPA*) is an incremental version of A* that uses heuristics\ncontrol its search. As for A*, the heuristics approximate the goal distances of the vertices\n. They need to be consistent, that is, satisfy\nfor\nall vertices\n\b\u0007\u0006\n\u0001\u001b\b\n\n\u0004\u0003\u0005\n03:\t\n\u0002\u0001\n.\n\u000f5\u0002\u0001+\u0003\n\u0003\u0007\u0006\t\b\u000b\n\n. These values\nof the start distance\nLPA* maintains an estimate\n\u0006\t\b\u000b\n\ndirectly correspond to the g-values of an A* search. They are carried forward from search to\nsearch. LPA* also maintains a second kind of estimate of the start distances. The rhs-values\nare one-step lookahead values based on the g-values and thus potentially better informed\n\n\u001c and\n\n\u0006\u0018\b\u000b\n\n$E\u0003\u0007\u0006\u0018\b,\u001f;\b\nof each vertex\n\n<+=\u0015\u0006\t\b\u000b\n\nwith\n\n\u000f5\n\nand\n\n\u0006\t\b\u000b\n\n7\u00059\n0;:\n\n\u0006\t\b\n7\u00059\n\n\b\u000b\"\n\n\u0006\t\b\n\n\n\b\n\n\n\"\n\"\n\n\b\n<\n\b\n\fOriginal Eight-Connected Gridworld\n\nuninformed search\n\nbreadth-\ufb01rst search\n\nheuristic search\n\nA*\n\nsstart\n\nsgoal\n\nsstart\n\nDynamicSWSF-FP (with early termination)\n\nLifelong Planning A*\n\nsstart\n\nsgoal\n\nsstart\n\nChanged Eight-Connected Gridworld\n\nuninformed search\n\nbreadth-\ufb01rst search\n\nheuristic search\n\nA*\n\nsstart\n\nsgoal\n\nsstart\n\nDynamicSWSF-FP (with early termination)\n\nLifelong Planning A*\n\nsstart\n\nsgoal\n\nsstart\n\nsgoal\n\nsgoal\n\nsgoal\n\nsgoal\n\nh\nc\nr\na\ne\ns\n \ne\nt\ne\nl\np\nm\no\nc\n\nh\nc\nr\na\ne\ns\n \nl\na\nt\nn\ne\nm\ne\nr\nc\nn\ni\n\nh\nc\nr\na\ne\ns\n \ne\nt\ne\nl\np\nm\no\nc\n\nh\nc\nr\na\ne\ns\n \nl\na\nt\nn\ne\nm\ne\nr\nc\nn\ni\n\nFigure 2: Performance of Search Methods in the Simple Gridworld\n\nthan the g-values. They always satisfy the following relationship:\n\n\u0002\u0001\u0004\u0003\u0006\u0005\u0007\u0003\t\b\u000b\n\r\f\u000f\u000e\n\u001b\u001d\u001c\u001f\u001e\n\n\u0012\u0015 \"!$#%\u0018'&)(\t*\u001f\u0012,+\n\n\u0005.-%\u0005\u0007\u0003\u0013/0\b21435\u0005\u0007\u0003\u0019/\u00076\u001a\u0003\t\b,\b\n\notherwise.\n\nif\u0003\u0010\n\u0011\u0003\u0013\u0012\u0015\u0014\u0017\u0016\u0019\u0018\u001a\u0014\n\n(1)\n\nA vertex is called locally consistent iff its g-value equals its rhs-value. This is similar to\nsatisfying the Bellman equation for undiscounted deterministic sequential decision prob-\nlems. Thus, this concept is important because the g-values of all vertices equal their start\ndistances iff all vertices are locally consistent. However, LPA* does not make every vertex\nlocally consistent. Instead, it uses the heuristics\nto focus the search and update only\nthe g-values that are relevant for computing a shortest path from\n\nto\n\n.\n\n\u0006\t\b\u000b\n\nLPA* maintains a priority queue\nthat always contains exactly the locally inconsistent\nvertices. These are the vertices whose g-values LPA* potentially needs to update to make\nthem locally consistent. The keys of the vertices in the priority queue correspond to the\nf-values used by A*, and LPA* always expands the vertex in the priority queue with the\nsmallest key, similar to A* that always expands the vertex in the priority queue with the\nsmallest f-value. By expanding a vertex, we mean executing \u0001 10-16\n(numbers in brackets\nrefer to line numbers in Figure 3). The key\nis a vector with two components:\n\nof vertex\n\n\b,-/.10324.\n\n739\n\n0;:\n\n\u0006\u0018\b\u000b\n\n\n\b\n7\n\u0002\n8\n\b\n\freturns the smallest priority of all vertices in priority queue\nand returns the\n\n.\n\n;\n\n;\n\n. Finally, U.Remove*\u001f\u0012,+ removes vertex\u0012\n\nThe pseudocode uses the following functions to manage the priority queue: U.TopKey*\n+ deletes the vertex with the smallest priority in priority queue \u0001\n. (If \u0001\n\u0003\u0005\u0004\u0006\u0003\b\u0007 .) U.Pop*\nis empty, then U.TopKey*\nreturns \u0002\ninto priority queue \u0001 with priority\u000b\nvertex. U.Insert*\u001f\u0012\n\t\f\u000b\u0019+\nfrom priority queue \u0001\ninserts vertex\u0012\nprocedure CalculateKey*\n\u0012,+\n*\u0014\u0013\t*\u001f\u0012,+\u0015\t\u0007\u0018\u0017\u0016\u0013\u0012\n*\u0014\u0013\n\u0012,+\u0015\t\"\u0018\u0017\u0016\u0013\u0012\n*\u001f\u0012\u0015+.+\u0019\u0018\u001a\u0016\u0002*\u001f\u0012,+\n\r 01\u000e\n*\u001f\u0012,+\u0017+\nreturn\u0002\n\u0007 ;\n\u000f\u0011\u0010\n\u0004\f\u000f\u0011\u0010\nprocedure Initialize*\n\r 02\u000e\n\u0001\u001c\u001b\n\u001d\u0005\u001e\n! \u001f\u000b\u0018\u0017\u0016\t\u0012\u001a*\u001f\u0012,+\n\r 03\u000e\n\u0012,+\nfor all\u0012\n\u0018\u0017\u0016\u0013\u0012\n*\u001f\u0012\"!$#&%\u0017'(#\u0015+\n\r 04\u000e\n\u001d\u0005) ;\n\u0016\u0002*\u001f\u0012\n!$#&%\u0017'*#\n!\u0015#\u0006%+'*#\n\r 05\u000e U.Insert*\u001f\u0012\n+ ;\nprocedure UpdateVertex*-,5+\n+\u0019\u0018;:\nif*\u0014,/.\n+\u0013\u0018\n\u0016\u0013\u0012\n*\u0014\u0013\t*\u001f\u0012\n*\u0014,\u0002+\n!$#&%\u0017'(#\n\r 06\u000e\n'\"4$5\u00176-798\n 1032\n\u000f\u0011\u0010\n+ U.Remove*-,5+ ;\nif*\u0014,\u0010!\n\r 07\u000e\n*-,5+.+ U.Insert*\u0014,=\t CalculateKey*-,5+.+ ;\n*-,5+<.\n\u0018\u0017\u0016\u0013\u0012\n\r 08\u000e\nif*\nprocedure ComputeShortestPath*\u001f+\n@ CalculateKey*\u001f\u0012\"A\"B\n+?>\n+ OR\u0018\u0017\u0016\t\u0012\u001a*\u001f\u0012\"A\"B\n+<.\n%\u0017C\n%\nC\n\r 09\u000e while* U.TopKey*\n\r 10\u000e\n\u001d U.Pop*\n+ ;\n\u0018\u0017\u0016\u0013\u0012\n*-,5+.+\nif*\u0014\u0013\t*-,5+ED\n\r 11\u000e\n*-,5+\n*-,5+ ;\n\u0018\u0017\u0016\u0013\u0012\n\r 12\u000e\n! \u001fF,3:$:\n*-,5+ UpdateVertex*\u001f\u0012,+ ;\n\r 13\u000e\nfor all\u0012\n\r 14\u000e\n*-,5+\n\r 15\u000e\n! \u001fF,3:$:\n*-,5+3G\n\r 16\u000e\n\u000e UpdateVertex*\u001f\u0012,+ ;\nfor all\u0012\nprocedure Main*\n\r 17\u000e\nInitialize*\n+ ;\n\r 18\u000e\n\r 19\u000e\nComputeShortestPath*\u001f+ ;\n\r 20\u000e\nfor all directed edges*-,F\t\u0015H\u0019+ with changed edge costs\n\r 21\u000e\n*-,F\t\u0015H\u0019+ ;\nUpdate the edge cost:\n\r 22\u000e\nUpdateVertex*\u0014H\u0019+ ;\n\r 23\u000e\n\nFigure 3: Lifelong Planning A*.\n\n\t\f,5+.+ ;\n*\u001f\u0012*A\"B\n\nWait for changes in edge costs;\n\nforever\n\n+\u0017+\n\n*\u001f\u0012\n\nelse\n\n;\n\n%\nC\n\nI=N\nand\n\n\u0001XRUT&V\n\n\u0005\u0007\u0003\t\b\nM\n\"\u0018\u0006\t\b\u000b\n\n\u0005\u0007\u0003\t\b$O\"6\n8?W\n\u0006\t\b\u000b\n\nIEL\n\u0005\u0007\u0003\t\b\u000b\nKJ\n\u0001SRUT&V\n\u0006\t\b\u000b\n\u0005\u001fA\u0013\n\u0006\u0018\b\u0019\n4\n\n\u0006\t\b\u000b\n\n, denoted by\n8(\u0006\t\b\u000b\nZY\n\"/\u0006\t\b\u000b\n\n).\n8QP,\u0006\t\b\u000b\n\n(2)\nwhere\n\u0001 1\n. Keys are\n\u0017\b\n\u0017\b\n\u0006\u0018\b\u0019\n4\n\n\u0006\t\b\u000b\n\u0005\u001fA\u0013\ncompared according to a lexicographic ordering. For example, a key\nis smaller than\n8(\u0006\t\b\u000b\n\nor equal to a key\nor (\n\u0006\t\b\u000b\n\n\u0006\u0018\b\u000b\n\n\u0006\u0018\b\u0019\n\nand\nused\ncorresponds directly to the f-values \\\u0002\u0006\t\b\u000b\n\n\u0001\u001b<+=\u0007\u0006\u0018\b\u0019\n\n\u0006\t\b\u000b\n\nby A* because both the g-values and rhs-values of LPA* correspond to the g-values of\ncorresponds to the\nA* and the h-values of LPA* correspond to the h-values of A*.\n\u0006\u0018\b\u0019\n\n8QP\u0007\u0006\t\b\u000b\n\n8[W \u0006\t\b\u000b\n\u001a$\ng-values of A*. LPA* expands vertices in the order of increasing kP -values and vertices\nwith equal kP -values in order of increasing kW -values. This is similar to A* that expands\n\nvertices in the order of increasing f-values (since the heuristics are consistent) and vertices\nwith equal f-values that are on the same branch of the search tree in order of increasing\ng-values (since it grows the search tree).\n\n, iff either\n\n\u0006\u0018\b\u000b\n\n\u0006\u0018\b\u000b\n\n, then\n\n\u0001`_\n\nimplies that\n\n\\\u0002\u0006\t\b\u000b\nbaA<\n\n. When LPA*\n\n=\u0007\u0006\t\b\u000b\n*c and thus LPA* expands overconsistent vertices in the same order\n\n\u0006\t\b\u000b\n\nbecause vertex\n<+=,\u0006\t\b\u000b\n\n, LPA* sets the g-value of vertex\n\nis called overconsistent iff\nA locally inconsistent vertex\n\u0006\t\b\u000b\n^]\n\u0017\b\nexpands a locally overconsistent vertex \u0001 12-13\n\u0006\t\b\u000b\n\n+\b\n<+=,\u0006\t\b\u000b\n\nhas the smallest key among all locally inconsistent vertices.\n\u0006\u0018\b\u0019\n\n\u0017\b\n\u0006\u0018\b\u0019\n\nto its rhs-\nas A*. During the expansion of vertex\nvalue and thus its start distance \u0001 12\n, which is the desired value and also makes the vertex\nlocally consistent. Its g-value then no longer changes until LPA* terminates. A locally\ninconsistent vertex\n. When LPA* expands\na locally underconsistent vertex \u0001 15-16\n, then it simply sets the g-value of the vertex to\n. This makes the vertex either locally consistent or locally overconsistent. If\nin\ufb01nity \u0001 15\nthe expanded vertex was locally overconsistent, then the change of its g-value can affect\nthe local consistency of its successors \u0001 13\n. Similarly, if the expanded vertex was locally\n. LPA* therefore updates\nunderconsistent, then it and its successors can be affected \u0001 16\nrhs-values of these vertices, checks their local consistency, and adds them to or removes\nthem from the priority queue accordingly.\n\nis called underconsistent iff\n\n<\u0004\u0006\u0018\b\u0019\n\n\u0006\u0018\b\u0019\n\n\u0017\b\n\n+\n\u0001\n+\n\u0012\n\u0012\n*\n+\n\u001d\n\u0013\n*\n\u001d\n\u0003\n\t\n\u0002\n+\n\u0004\n)\n\u0007\n\u001d\n\u0012\n\u001d\n\u0012\n!\n \n \n\u0001\n\u0013\n\u001d\n\u001d\n\u0013\n,\n\u0013\n\u001d\n\u0013\n\u001d\n\u0003\n\n,\n+\nI\n\u0006\n<\n\u0003\n\n\u0006\n<\n\u0002\n8\n$\n8\n8\nP\n\u001d\n8\n\"\nP\n8\nP\n\u0001\n8\n\"\nP\n8\n\"\nW\n\u0003\n\n8\nW\n\b\n<\n\u0013\n\u0002\n\u0013\n\u0001\n\b\n\u0013\n\u0001\n8\n\b\n\b\n\u0002\n\b\n\u001d\n\u0013\n\u0002\n\u0002\n\u0002\n\u0002\n\f0;:\n\n\b\u000b739\n\n0;:\n\n\b87\u00059\n\n03:\n\nat which point in time the g-value of\n\nis locally consistent and the key of the vertex to expand\nLPA* expands vertices until\n739\n0;:\n. This is similar to A* that expands vertices until it\nnext is no smaller than the key of\n7\u00059\nequals its start distance and the\nexpands\nf-value of the vertex to expand next is no smaller than the f-value of\n. It turns out\n\b\u00157\u00059\n0;:\nthat LPA* expands a vertex at most twice, namely at most once when it is underconsistent\nand at most once when it is overconsistent. Thus, ComputeShortestPath\nreturns after a\n\u0006\u0018\n\nnumber of vertex expansions that is at most twice the number of vertices.\nafter the search, then there is no \ufb01nite-cost path from\n\nIf\nOtherwise, one can trace back a shortest path from\nthe current vertex\nuntil\nnot use backpointers.\n\n.\n\b87\u00059\n0;:\nby always moving from\n\u0003\u0007\u0006\u0018\b\u0019\"\t\u001f;\b\u000b\n\nis reached (ties can be broken arbitrarily), similar to what A* can do if it does\n\nto\nthat minimizes\n\b\u000b\"\n\n\b\u0007-/.10324.\n, to any predecessor\n\n.1032A.\n\u0006\t\b\u000b\"#\n\n, starting at\n\n7\u00059\n\n03:\n\n\u0006\t\b87\u00059\n\n0;:\n\n7\u00059\n\n0;:\n\nto\n\n-/.10324.\n\n\b\u000b-/.10324.\n\nThe resulting version of LPA* is shown in Figure 3. The main function Main() \ufb01rst calls\nInitialize() to initialize the search problem \u0001 17\n. Initialize() sets the initial g-values of\nall vertices to in\ufb01nity and sets their rhs-values according to Equation 1 \u0001 03-04\n. Thus,\nis the only locally inconsistent vertex and is inserted into the otherwise\ninitially\nempty priority queue with a key calculated according to Equation 2 \u0001 05\n. This initialization\nguarantees that the \ufb01rst call to ComputeShortestPath() performs exactly an A* search, that\nis, expands exactly the same vertices as A* in exactly the same order, provided that A*\nbreaks ties between vertices with the same f-values suitably. Notice that, in an actual\nimplementation, Initialize() only needs to initialize a vertex when it encounters it during\nthe search and thus does not need to initialize all vertices up front. This is important because\nthe number of vertices can be large and only a few of them might be reached during the\nsearch. LPA* then waits for changes in edge costs \u0001 20\n. If some edge costs have changed,\nit calls UpdateVertex() \u0001 23\nto update the rhs-values and keys of the vertices potentially\naffected by the changed edge costs as well as their membership in the priority queue if they\nbecome locally consistent or inconsistent, and \ufb01nally recalculates a shortest path \u0001 19\n5 Optimizations of Lifelong Planning A*\nThere are several simple ways of optimizing LPA* without changing its overall operation.\nThe resulting version of LPA* is shown in Figure 4. First, a vertex sometimes gets removed\nfrom the priority queue and then immediately reinserted with a different key. For example,\n. In this case,\na vertex can get removed on line \u0001 07\nit is often more ef\ufb01cient to leave the vertex in the priority queue, update its key, and only\nchange its position in the priority queue. Second, when UpdateVertex\ncom-\nputes the rhs-value for a successor of an overconsistent vertex it is unnecessary to take the\nminimum over all of its respective predecessors. It is suf\ufb01cient to compute the rhs-value\nas the minimum of its old rhs-value and the sum of the new g-value of the overconsistent\nvertex and the cost of moving from the overconsistent vertex to the successor. The reason\nis that only the g-value of the overconsistent vertex has changed. Since it decreased, it can\non line \u0001 16\nonly decrease the rhs-values of the successor. Third, when UpdateVertex\ncomputes the rhs-value for a successor of an underconsistent vertex, the only g-value that\nhas changed is the g-value of the underconsistent vertex. Since it increased, the rhs-value\nof the successor can only get affected if its old rhs-value was based on the old g-value of\nthe underconsistent vertex. This can be used to decide whether the successor needs to get\nupdated and its rhs-value needs to get recomputed \u0001 21\u2019\n. Fourth, the second and third op-\ntimization concerned the computations of the rhs-values of the successors after the g-value\nof a vertex has changed. Similar optimizations can be made for the computation of the\nrhs-value of a vertex after the cost of one of its incoming edges has changed.\n\nand then be reentered on line \u0001 08\n\non line \u0001 13\n\n\u0006\u0018\n\n\u0006\t\n\n.\n\n6 Analytical and Experimental Results\nWe can prove the correctness of ComputeShortestPath().\n\n\b\n\b\n<\n\n\u0001\n&\n\b\n-\n\b\n\b\n\b\n<\n\u0003\n\b\n\u0002\n\u0002\n\u0002\n\u0002\n\u0002\n\u0002\n\u0002\n\u0002\n\u0002\n\u0002\n\u0002\n\freturns a vertex with the smallest priority of all vertices in\n\n. (If \u0001\nis empty, then U.TopKey*\n\u0012\n\t$\u000b\u0019+ changes the priority of vertex\u0012\nfrom priority queue \u0001\nremoves vertex\u0012\n\nreturns \u0002\nin priority queue \u0001\n\n.\n\n. (It\n\n\u00041\u0003\b\u0007 .)\nto\u000b\n\n;\n\n%\nC\n\n%\nC\n\n\u0004\u0006\u000f\u0011\u0010\n\n;\n\nThe pseudocode uses the following functions to manage the priority queue: U.Top*\u001f+\nreturns the smallest priority of all vertices in priority queue \u0001\npriority queue \u0001\n. U.TopKey*\nU.Insert*\u001f\u0012\n\t\u0015\u000b\ninto priority queue \u0001 with priority\u000b\ninserts vertex\u0012\n. U.Update*\ndoes nothing if the current priority of vertex\u0012 already equals\u000b\n.) Finally, U.Remove*\u001f\u0012,+\nprocedure CalculateKey*\n\u0012,+\n*\u0014\u0013\t*\u001f\u0012,+\u0015\t\u0007\u0018\u0017\u0016\u0013\u0012\n\u0012,+\u0015\t\u0007\u0018\u0017\u0016\u0013\u0012\n*\u0014\u0013\n\u0018\u001a\u0016\u0002*\u001f\u0012,+\n\r 01\u2019\u000e\n*\u001f\u0012\u0015+.+\n*\u001f\u0012,+.+\n\u0007 ;\nreturn\u0002\n\u000f\u0011\u0010\nprocedure Initialize*\n\r 02\u2019\u000e\n\u0013\t*\u001f\u0012,+\n! \u001f\n\u0018\u0017\u0016\u0013\u0012\n\r 03\u2019\u000e\nfor all\u0012\n*\u001f\u0012\u0015+\n\u0018\n\u0016\u0013\u0012\n!$#\u0006%+'*#\n\r 04\u2019\u000e\n) ;\n\u0016\u0002*\u001f\u0012\n!\u0015#\u0006%+'*#\n!$#\u0006%+'*#\n\r 05\u2019\u000e U.Insert*\u001f\u0012\n+ ;\nprocedure UpdateVertex*-,5+\n+ U.Update*-,F\t CalculateKey*\u0014,\u0002+.+ ;\n*-,5+ AND,\n*\u0014,\u0002+<.\nif (\u0013\n\u0018\u0017\u0016\u0013\u0012\n\r 06\u2019\u000e\n\r 07\u2019\u000e else if*\u0014\u0013\t*-,5+\n\u0018\n\u0016\u0013\u0012\n*\u0014,\u0002+ AND,\n+ U.Insert*-,=\t CalculateKey*-,5+.+ ;\n+ U.Remove*-,\u0002+ ;\n\r 08\u2019\u000e else if*\u0014\u0013\t*-,5+\n\u0018\n\u0016\u0013\u0012\n*\u0014,\u0002+ AND,\nprocedure ComputeShortestPath*\u001f+\n+ OR\u0018\u0017\u0016\u0013\u0012\n\u0013\t*\u001f\u0012\nA\"B\nA\"B\nA\"B\n\r 09\u2019\u000e while* U.TopKey*\n%\nC\n@ CalculateKey*\n*\u001f\u0012\n\r 10\u2019\u000e\n\u001d U.Top*\n+ ;\nif*\u0014\u0013\t*-,5+\n*-,5+.+\n\u0018\u0017\u0016\u0013\u0012\n\r 11\u2019\u000e\n\u0013\t*-,5+\n\u0018\n\u0016\u0013\u0012\n*\u0014,\u0002+ ;\n\r 12\u2019\u000e\nU.Remove*-,5+ ;\n\r 13\u2019\u000e\nfor all\u0012\u000b!Z\u001f=,9:(:\u001a*-,5+\n\r 14\u2019\u000e\n*\u001f\u0012,+\f\t1\u0013\n*\u001f\u0018\u0017\u0016\u0013\u0012\n*\u0014,\u0002+\u0019\u0018/:\u001a*-,F\t\"\u0012,+.+ ;\n\u0018\u0017\u0016\u0013\u0012\n!$#\u0006%+'*#\n\r 15\u2019\u000e\nif*\u001f\u0012\n*\u001f\u0012,+\n\u000f\u0011\u0010\n\r 16\u2019\u000e\nUpdateVertex*\u001f\u0012,+ ;\n\r 17\u2019\u000e\n\u0013\t*-,\u0002+ ;\n\u0013\u0017B\nC\u00145\n\r 18\u2019\u000e\n\u0013\t*-,5+\n\r 19\u2019\u000e\nfor all\u0012\u000b!Z\u001f=,9:(:\u001a*-,5+\u0019G\n\r 20\u2019\u000e\n,\u0002+\n*-,F\t\"\u0012,+ OR\u0012\n\u0018\b:\n\u0013\u0017B\nif*\u001f\u0018\u0017\u0016\t\u0012\u001a*\u001f\u0012,+\nC\u00145\n\r 21\u2019\u000e\n\u0018\b:\u001a*\u001f\u0012\n*\u0014\u0013\t*\u001f\u0012\n+\t\u0018\u0017\u0016\u0013\u0012\n!\u0015#\u0006%+'*#\n\r 22\u2019\u000e\n!$8\n'*4(5\u00176\nif*\n*\u001f\u0012,+\n \f032\n\u000f\u0011\u0010\n\r 23\u2019\u000e\nUpdateVertex*\u001f\u0012,+ ;\nprocedure Main*\n\r 24\u2019\u000e\nInitialize*\u001f+ ;\n\r 25\u2019\u000e\n\r 26\u2019\u000e\nComputeShortestPath*\u001f+ ;\n\r 27\u2019\u000e\nfor all directed edges*-,F\t\fH\u0019+ with changed edge costs\n\r 28\u2019\u000e\n:\u001a*-,F\t\fH\u0019+ ;\n\r 29\u2019\u000e\nC\u00145\nUpdate the edge cost:\u001a*-,=\t1H\u0019+ ;\n\r 30\u2019\u000e\n:\u001a*-,=\t1H\u0019+ )\nif (:*B\nC\u00145\n\r 31\u2019\u000e\n\u0018\b:\u001a*-,=\t1H\u0019+.+ ;\n*-,5+\n*\u001f\u0018\u0017\u0016\t\u0012\u001a*-H\u0019+\u0015\t\u0006\u0013\n*-H\n\u0018\u0017\u0016\u0013\u0012\nif*-H .\n!$#&%\u0017'*#\n\r 32\u2019\u000e\n\u0018;:\"B\nelse if*\u001f\u0018\u0017\u0016\u0013\u0012\n*-H\u0019+\n\u0013\t*-,5+\n\r 33\u2019\u000e\n*\u0014\u0013\t*\u001f\u0012\nif*-H .\n\u0018\u0017\u0016\u0013\u0012\n*-H\n\u0018;:\u001a*\u001f\u0012\n!$#&%\u0017'*#\n\r 34\u2019\u000e\n'\"4$5\n6\u0003\u0002\u00178\n 1032\nUpdateVertex*-H\n\r 35\u2019\u000e\n+ ;\n\n\t1H\u0019+.+ ;\n\nWait for changes in edge costs;\n\n\u000f\u0011\u0010\n\u000f\u0011\u0010\n\nforever\n\n+.+\n\nelse\n\nC\u00145\n\n;\n\n\t\u0007\u0012,+.+ ;\n\nFigure 4: Lifelong Planning A* (optimized version)\n\nTheorem 1 ComputeShortestPath() terminates and one can then trace back a shortest path from\n\n\u0012\u0015\u0014\u0017\u0016\u0019\u0018\u001a\u0014\n\n\b21435\u0005\u0007\u0003\n\n6\u001a\u0003\t\b until\u0003\n\nto\u0003\n\n\u0003\u0013\u0012\u0015\u0014\u0017\u0016\u0013\u0018'\u0014\nthat minimizes-%\u0005\u0007\u0003\n\n\u0013\u0005\u0004'\u0016\u0007\u0006 by always moving from the current vertex\u0003 , starting at\u0003\n\n\u0013\u0005\u0004'\u0016\u0007\u0006 , to any predecessor\u0003\n\nis reached (ties can be broken arbitrarily).\n\n(The proofs can be found in [LK01].) We now compare breadth-\ufb01rst search, A*,\nDynamicSWSF-FP, and the optimized version of LPA* experimentally.\n(We use\nDynamicSWSF-FP with the same optimizations that we developed for LPA*, to avoid bi-\nasing our experimental results in favor of LPA*.) The priority queues of all four algorithms\nwere implemented as binary heaps. Since all algorithms determine the same paths (if they\nbreak ties suitably), we need to compare their total search time until a shortest path has\nbeen found. Since the actual runtimes are implementation-dependent, we instead use three\nmeasures that all correspond to common operations performed by the algorithms and thus\n(that is, updates\nof the g-values, similar to backup operations of dynamic programming for sequential de-\n(for example, to read or change\ntheir values), and the total number of heap percolates\n(exchanges of a parent and child\nin the heap). Note that we count two vertex expansions, not just one vertex expansion, if\nLPA* expands the same vertex twice, to avoid biasing our experimental results in favor of\nLPA*.\n\nheavily in\ufb02uence their runtimes: the total number of vertex expansions \b\ncision problems), the total number of vertex accesses \b\n\t\n\f\u000b\n\nAll of our experiments use \ufb01fty eight-connected gridworlds that have size \n\n\u001c\u001eC\n\n\u001c and an\n\n+\n+\n\u0003\n+\n\u0012\n\u0012\n*\n+\n\u0001\n\u001b\n\u001d\n\u001e\n\u001d\n\u001d\n\u0003\n*\n\u0012\n+\n\u001d\n\t\n\u0002\n+\n\u0004\n)\n\u0007\n\u001d\n!\n\u0001\n.\n\u001d\n\u0001\n!\n\u0001\n\u001d\n!\n\u0001\n+\n>\n\u0012\n+\n.\n\u001d\n,\nD\n\u001d\n.\n\u001d\n\u0012\n+\n\u001d\n\u0012\n\u001d\n\u001d\n\u0003\n\n,\n\u000e\n\u001d\n\u001d\n\u0012\n.\n\u001d\n\u0012\n\u001d\n\u0012\n!\n \n+\n \n+\n:\nB\n\u001d\nD\n\u001d\n\u0012\n+\n+\n\u001d\n\u0012\n\u001d\n+\n\u001d\n\u0012\n+\n+\n\u001d\n\u0012\n!\n \n+\n \n/\n/\n/\n\u0014\n\n\fobstacle density of 40 percent. The start cell is at coordinates (34, 20) and the goal cell\nis at coordinates (5, 20), where the upper leftmost cell is at coordinates (0, 0). For each\ngridworld, the initial obstacle con\ufb01guration was generated randomly. Then, it was changed\n500 times in a row, each time by making eight randomly chosen blocked cells traversable\nand eight randomly chosen traversable cells blocked. Thus, each time one percent of the\ncells changed their status but the obstacle density remained the same. After each of the\n500 changes, the algorithms recomputed a shortest path from the start cell to the goal cell.\nFor each of the four algorithms and each of the three performance measures, the following\ntable reports the mean of the performance measure for the 500 changes: both its average\nover the \ufb01fty mazes and its 95-percent con\ufb01dence interval over the \ufb01fty mazes (assuming a\nnormal distribution with unknown variance). The table con\ufb01rms the observations made in\nSection 3: LPA* outperforms the other three search methods according to all three perfor-\nmance measures.\n\ncomplete search\n\nincremental search\n\nve =\nva =\nhp =\n\nve =\nva =\nhp =\n\nuninformed search\nbreadth-\ufb01rst search\n\n1331.7\n26207.2\n5985.3\n\nheuristic search\n\n4.4\n84.0\n19.7\n\nve =\nva =\nhp =\n\nA*\n\n284.0\n6177.3\n1697.3\n\n5.9\n129.3\n39.9\n\nDynamicSWSF-FP\n\nLifelong Planning A*\n\n2.0\n75.0\n16.9\n\n173.0\n5697.4\n956.2\n\n4.9\n167.0\n26.6\n\nve =\nva =\nhp =\n\n25.6\n1235.9\n240.1\n\nWe have also applied LPA* successfully to more complex planning tasks, including the\nkind of route planning tasks that Focussed Dynamic A* [Ste95] applies to. The results will\nbe reported separately.\n\nReferences\n\n[LK01]\n\n[Pea85]\n\n[FMSN00] D. Frigioni, A. Marchetti-Spaccamela, and U. Nanni. Fully dynamic algo-\nrithms for maintaining shortest paths trees. Journal of Algorithms, 34(2):251\u2013\n281, 2000.\nM. Likhachev and S. Koenig. Lifelong Planning A* and Dynamic A* Lite: The\nproofs. Technical report, College of Computing, Georgia Institute of Technol-\nogy, Atlanta (Georgia), 2001.\nJ. Pearl. Heuristics: Intelligent Search Strategies for Computer Problem Solv-\ning. Addison-Wesley, 1985.\nG. Ramalingam and T. Reps. An incremental algorithm for a generalization of\nthe shortest-path problem. Journal of Algorithms, 21:267\u2013305, 1996.\nA. Stentz. The focussed D* algorithm for real-time replanning. In Proceedings\nof the International Joint Conference on Arti\ufb01cial Intelligence, pages 1652\u2013\n1659, 1995.\nSebastian Thrun. Lifelong learning algorithms. In S. Thrun and L. Pratt, edi-\ntors, Learning To Learn. Kluwer Academic Publishers, 1998.\n\n[Thr98]\n\n[RR96]\n\n[Ste95]\n\n\n\n\n\n\n\n\n\n\n\n\n\n\f", "award": [], "sourceid": 2003, "authors": [{"given_name": "S.", "family_name": "Koenig", "institution": null}, {"given_name": "M.", "family_name": "Likhachev", "institution": null}]}