{"title": "Structural and Behavioral Evolution of Recurrent Networks", "book": "Advances in Neural Information Processing Systems", "page_first": 88, "page_last": 95, "abstract": null, "full_text": "Structural and Behavioral Evolution \n\nof Recurrent Networks \n\nGregory M. Saunders, Peter J. Angeline, and Jordan B. Pollack \n\nLaboratory for Artificial Intelligence Research \n\nDepartment of Computer and Information Science \n\nThe Ohio State University \nColumbus, Ohio  43210 \n\nsaunders@cis.ohio-state.edu \n\nAbstract \n\nThis paper introduces GNARL, an evolutionary program which induces \nrecurrent neural networks that are structurally unconstrained. In contrast \nto constructive and destructive algorithms, GNARL employs a popula(cid:173)\ntion of networks and uses a fitness  function's  unsupervised feedback to \nguide  search  through  network  space.  Annealing  is  used  in  generating \nboth  gaussian  weight  changes  and  structural  modifications.  Applying \nGNARL  to  a complex  search and collection task demonstrates  that the \nsystem is capable of inducing networks with complex internal dynamics. \n\n1  INTRODUCTION \nA variety of methods  to induce network architecture exist.  Some start with a very  simple \nnetwork and incrementally add nodes and links (Hanson 1990; Fahlman & Lebiere,  1990; \nFahlman  1991; Chen,  et aI.,  1993);  others  start with  a  large  network  and  then prune off \nsuperfluous pieces (Mozer & Smolensky,  1989; Cun, Denker, and SolI a,  1990; Hassibi & \nStork,  1993; amlin & Giles,  1993). But these constructive and destructive algorithms are \nmonotonic  extremes that  ignore  a  more  moderate  solution:  \"dynamically add  or remove \npieces of architecture as  needed.\" Moreover, by exclusively exploring either feedforward \nnetworks (e.g., Ash,  1989), fully-connected recurrent networks (e.g., Chen, et al.  1993), or \nsome restricted middle ground (e.g., Fahlman,.1991), these algorithms allow only limited \nstructural change. Finally, constructive and destructive algorithms are supervised methods \n\n88 \n\n\fStructural and Behavioral Evolution of Recurrent Networks \n\n89 \n\n\u2022 \n\nnum-in input units \n\nRandom(max-hidden) hidden units \n\nRandom(max-links) links \n\nnum-out output units \n\nFigure 1:  Sample initial network. The number of input nodes and number of output nodes \nis  fixed  for  the  particular  task,  but  the  number  of hidden  units  and  the  connectivity \n(although bounded), is random. \n\nwhich rely  on complex predicates to determine  when to add or delete  pieces  of network \narchitecture (e.g., \"when rate of improvement falls below threshold\"). \nGenetic algorithms (Holland 1975), on the other hand, are unsupervised methods which can \ninduce networks by making stochastic modifications to a population of bitstrings, each of \nwhich is interpreted as a network. Most studies, however, still assume a fixed structure for \nthe network (e.g., Belew et aI.,  1990; Jefferson, et al., 1991; see also Schaffer, et al.  1992), \nand those  that do not allow  only limited  structural change  (e.g.,  Potter,  1992,  and  Karu(cid:173)\nnanithi et al.,  1992). \nEvolutionary  programming  (Fogel,  1992)  is  an  alternate  optimization  technique  which, \nwhen applied to network induction, obviates the need for a bitstring-to-network mapping \nby  mutating  networks  directly.  Furthermore,  because EP does  not employ  crossover (an \noperator of questionable efficacy on distributed representations), it is a better candidate for \ninducing network structures (Angeline, Saunders, and Pollack,  1993; Fogel et al.,  1990). \n\n2  THE GNARL ALGORITHM \nGNARL  (GeNeralized  Acquisition  of Recurrent  Links)  is  an  evolutionary  program  that \nnon-monotonically constructs recurrent networks  to  solve a given task.  It begins with an \ninitial population of n random individuals; a sample network N is  shown in Figure  1.  The \nnumber of input nodes (num-in) and number of output nodes (num-out) are fixed for a given \ntask; the number of hidden nodes  as  well as the connections among them are free to vary. \n\nSelf-links as  well  as  general loops are allowed.  Thus GNARL's  search space is  N = {N: \n\nnetwork N has num-in input nodes and num-out output nodes}. \nIn each epoch of search, the  networks are ranked by a user-supplied fitness  function f:  N \n~ R, where R represents the reals. Reproduction of the best n/2 individuals entails modi(cid:173)\nfying both the weights and structure of each parent network N. First, the temperature T(N) \nis calculated: \n\nwhere fmax  (provided by the user)  is  the maximum possible fitness  for a given task.  This \n\nT(N)  = I_ f (N) \nf max \n\n(1) \n\n\f90 \n\nSaunders, Angeline, and Pollack \n\nmeasure of N's performance is  used to anneal the structural and parametric (Barto,  1990) \nsimilarity  between  parent  and  offspring,  so  that  networks  with  a  high  temperature  are \nmutated severely, and those with a low temperature are mutated only slightly. This allows \na coarse-grained search initially, and a finer-grained search as a network approaches a solu(cid:173)\ntion (cf. Kirkpatrick et aI.,  1983). \nMore concretely, parametric mutations are accomplished by perturbing each weight with \ngaussian noise, whose variance is T(Ny2: \n\nW  f- W + Normal  (0; T (N)), \n\n'v'w  E  N \n\n(2) \n\nStructural mutations are accomplished by: \n\n\u2022 adding  k] hidden nodes with probability Palld-node \n\u2022 deleting  k2 hidden nodes with probability Pdelete-node \n\u2022 adding  k3links with probability Padd-link \n\u2022 deleting  k4 links with probability Pdelete-link \n\nwhere each kj  is  selected  uniformly  from  a  user-defined  range,  again  annealed  by  T(N). \nWhen a node is added, it is  initialized without connections; when a node is deleted, all its \nincident links are removed. All new links are initialized to O.  (See also Angeline, Saunders, \nand Pollack,  1993.) \n\n3  RESULTS \nGNARL was  tested on a simple control task - the Tracker task of Jefferson, et al.  (1991) \nand Koza (1992). In this problem, a simulated ant is placed on a two-dimensional toroidal \ngrid and must maximize the number of pieces of food it collects in a given time period (Fig(cid:173)\nure  2a).  Each ant is  controlled by a network with two input nodes and four output nodes \n(Figure 2b). At each step, the action whose corresponding output node has maximum acti(cid:173)\nvation is performed. Fitness is the number of grid positions cleared within 200 time steps. \nThe  experiments  used a population of 100 networks. In  the  first  run  (2090 generations), \nGNARL found  a network (Figure 3b) that cleared  81  grid positions  within  the 200 time \nsteps. Figure 4 shows the state of the output units of the network over three different sets \nof inputs. Each point is a triple of the form (move, right, left). (No-op is not shown because \nit was never used in the final  network.) Figure 4a shows the result of supplying to the net(cid:173)\nwork 200 \"food\" inputs - a fixed point that executes \"Move.\" Figure 4b shows the sequence \nof states reached when 200 \"no food\" signals are supplied to the network - a collection of \npoints  describing  a  limit cycle of length 5 that repeatedly executes  the  sequence \"Right, \nRight, Right, Right, Move.\" These two attractors determine the response of the network to \nthe task (Figure 4c,d); the additional points in Figure 4c are transients encountered as the \nnetwork alternates between these attractors. \nHowever, not all evolved network behaviors are so simple as to approximate an FSA (Pol(cid:173)\nlack, 1991). In a second run (1595 generations) GNARL induced a network that cleared 82 \ngrid points within the 200 time steps. Figure 5 demonstrates the behavior of this network. \nOnce  again, the  \"food\" attractor,  shown  in  Figure  5a, is  a  single  point in  the  space  that \nalways executes \"Move.\" The \"no food\" behavior, however, is not an FSA; instead, it is  a \n\n\fStructural and Behavioral Evolution of Recurrent Networks \n\n91 \n\n\" \n, \n\nrp1 \n\nMove  Turn left  Tum right  N o-op \n6) \n\n@ \n\nFood \n\nNo food \n\n(a) \n\n(b) \n\nFigure 2:  The ant problem. (a) The trail is connected initially, but becomes progressively \nmore difficult to follow. The underlying 2-d grid is toroidal, so that position \"P\" is the first \nbreak in the trail. The ellipse indicates the 7 pieces of food that the network of the second \nrun failed to reach. (b) The semantics of the I/O units for the ant network. The first input \nnode  denotes  the  presence  of food  in  the  square  directly  in  front  of the  ant;  the  second \ndenotes  the  absence  of  food  in  this  same  square.  No-op,  from  Jefferson,  allows  the \nnetwork  to  stay  in  one  position  while  activation  flows  through  recurrent  links.  This \nparticular network \"eats\" 42 pieces of food  before spinning endlessly in place at position \nP,  illustrating a very deep local minimum in the  search space. \n\nquasiperiodic trajectory of points shaped like a \"D\" in output space (Figure Sb). The place(cid:173)\nment of the \"D\" is in the \"Move / Right\" corner of the space and encodes a complex alter(cid:173)\nnation between these two operations (Figure Sd). \n\n4  CONCLUSIONS \nArtificial  architectural  constraints  (such  as  \"feedforwardness\")  close  the  door on  entire \nclasses of behavior;  forced liberties (such as  assumed full  recurrence)  may  unnecessarily \nincrease structural complexity or learning time.  By relying on a simple stochastic process, \nGNARL strikes  a  middle ground  between these two,  allowing  the  network's complexity \nand behavior to emerge in response to the demands of the task. \n\nAcknowledgments \n\nThe  research  reported  in  this  paper  has  been  partially  supported  by  Office  of  Naval \nResearch grants NOOO14-93-1-00S9  and  NOOO14-92-J-119S.  We  are indebted to  all  those \nwho read and reviewed this work, especially John Kolen, Ed Large, and Barbara Becker. \n\n\f92 \n\nSaunders, Angeline, and Pollack \n\n,.\\ \n\n\u2022 \n\" . \\ \n./ \n\n1 \n1 \nI \n1 \n1 \n1 \n\n\\ \n\\ \n\\ \n\\ \n\\ \n\\ \n\n1 \nI \n\n\" \n\ni. \n\n, . , , , \n\u2022 \n\\~ \n~\"\". \n\n, \n, \n, \nI \n, \n, \n\n: \n\nI \nI \n\nI \n\n\\ \n\n\\ \n\\ \n\nMove  Left  Right No-op \n\n~: \n\nFood \n\nNo food \n\n(a) \n\n(c) \n\n-----------------------\n\n(b) \n\nFigure 3:  The Tracker Task, first run. (a) The best network in the initial population. Nodes \no &  1  are  input,  nodes  5-8  are  output,  and  nodes  2-4  are  hidden  nodes.  (b)  Network \ninduced by  GNARL after 2090 generations. Forward links are dashed;  bidirectional links \n&  loops are solid. The light gray connection between nodes 8 and  13  is the sole backlink. \nThis network clears the trail in 319 epochs. (c) Jefferson et al.'s fixed network structure for \nthe Tracker task. \n\nReferences \n\nAngeline, P.,  Saunders, G., Pollack, J.  (1993). An evolutionary algorithm that constructs \nrecurrent neural networks. LAIR Technical Report 93-PA-GNARL, The Ohio State Uni(cid:173)\nversity, Columbus Ohio. To be published in IEEE Transactions on Neural Networks. \n\n\fStructural and Behavioral Evolution of Recurrent Networks \n\n93 \n\n1 \n\n1 \n\n1 \n\n(a) \n\n1 \n\n1 \n\n(b) \n\n1 \n\n.~  400 \n..... . ;;) \no \n0.. \n>< \n\n1 \n\n(c) \n\nFigure 4:  Limit behavior of the network that clears the trail in 319 steps. Graphs show the \nstate  of  the  output  units  Move,  Right,  Left.  (a)  Fixed  point  attractor  that  results  for \nsequence of 200 \"food\" signals; (b) Limit cycle attractor that results when a sequence of \n200 \"no food\"  signals is  given to network;  (c)  All  states visited while traversing the trail; \n(d) The x position of the ant over time when run on an empty grid. \n\nAsh, T.  (1989). \"Dynamic node creation in backpropagation networks,\" Connection Sci(cid:173)\nence,  1 :365-375. \nBarto, A. G. (1990). Connectionist learning for control. In Miller, W. T. III, Sutton, R. S., \nand Werbos, P. J., editors, Neural Networksfor Control. Chapter 1, pages 5-58. MIT Press, \nCambridge. \nBelew, R.  K., McInerney, J., and Schraudolf, N.  N.  (1990). Evolving networks: Using the \ngenetic algorithm with connectionist learning. Technical Report CS90-174, University of \nCalifornia, San Diego. \n\n\f94 \n\nSaunders, Angeline, and Pollack \n\n1 \n\n1 \n\n(b) \n\n3 \n\n.... \n\u00a7  3000 \n. -g  2000 \no \n0.. \n>< \n\n1 \n\n(a) \n\n1 \n\n1 \n\n(c) \n\nFigure 5:  Limit behavior of the  network of the  second run.  Graphs  show the  state of the \noutput units  Move,  Right,  Left.  (a) Fixed point attractor that results  for  sequence of 500 \n\"food\"  signals;  (b)  Limit cycle  attractor that results  when  a  sequence  of 500 \"no  food\" \nsignals  is  given  to  network;  (c)  All  states  visited  while  traversing  the  trail;  (d)  The  x \nposition of the ant over time when run on an empty grid. \n\nChen, D., Giles, C., Sun, G., Chen, H., Less, Y., and Goudreau, M.  (1993). Constructive \nlearning of recurrent neural networks. IEEE International Conference on Neural Networks, \n3:1196-1201. \nCun, Y.L., Denker, J., and SoIIa, S. (1990). Optimal brain damage. In Touretzky, D., editor, \nAdvances in Neural Information Processing Systems 2. Morgan Kaufmann. \n\nFahlman, S. and Lebiere, C. (1990). The cascade-correlation architecture. In Touretzky, D. \nS., editor, Advances in Neural Information Processing Structures 2, pages 524-532. Mor(cid:173)\ngan Kaufmann. \nFahlman, S. (1991). The recurrent cascade-correlation architecture. In Lippmann, R., \n\n\fStructural and Behavioral Evolution of Recurrent Networks \n\n95 \n\nMoody, J., and Touretzky, D., editors, Advances in Neural Information Processing Systems \n3, pages 190-196. Morgan Kaufmann, San Mateo. \nFogel, D.  (1992). Evolving Artificial Intelligence.  Ph.D. thesis, University of California, \nSan Diego. \nFogel, D., Fogel, L., and Porto, V. W. (1990). Evolving neural networks. Biological Cyber(cid:173)\nnetics.  63:487~93. \nHanson, S. J. (1990). Meiosis networks. In Touretzky, D., editor,Advances in NeuralInfor(cid:173)\nmation Processing Systems 2, pages 533-541. Morgan Kaufmann, San Mateo. \nHassibi, B. and Stork, D. G. (1993). Second order derivatives for network pruning: Optimal \nbrain surgeon. In Hanson, S. J., Cowan, J. D., and Giles, C. L., editors, Advances in Neural \nInformation Processing Systems 5, pages 164-171. Morgan Kaufmann. \nHolland, J.  (1975). Adaptation in Natural and Artificial Systems.  The University of Mich(cid:173)\nigan Press, Ann Arbor, MI. \nJefferson, D., Collins, R, Cooper, C., Dyer, M., Flowers, M., Korf, R, Taylor, C., and \nWang, A. (1991). Evolution as a theme in artificial life: The genesys/tracker system. In \nLangton, C. G., Taylor, C., Farmer, J. D., and Rasmussen, S., editors, Artificial Life II: Pro(cid:173)\nceedings of the  Workshop on Artificial Life. pages 549-577 . Addison-Wesley. \nKarunanithi, N., Das, R, and Whitley, D. (1992). Genetic cascade learning for neural net(cid:173)\nworks. In Proceedings of COGANN-92 International Workshop on Combinations of \nGenetic Algorithms and Neural Networks. \nKirkpatrick, S., Gelatt, C. D., and Vecchi, M. P.  (1983). Optimization by simulated anneal(cid:173)\ning. Science, 220:671-680. \nKoza, J. (1992). Genetic evolution and co-evolution of computer programs. In Christopher \nG. Langton, Charles Taylor, J. D. F.  and Rasmussen, S., editors, Artificial Life II. Addison \nWesley Publishing Company, Reading Mass. \nMozer, M. and Smolensky, P.  (1989). Skeletonization: A technique for trimming the fat \nfrom a network via relevance assessment. In Touretzky, D., editor, Advances in Neural \nInformation Processing Systems 1, pages 107-115. Morgan Kaufmann, San Mateo. \nOmlin, C. W. and Giles, C. L. (April 1993). Pruning recurrent neural networks for improved \ngeneralization performance. Technical Report Tech Report No 93-6, Computer Science \nDepartment, Rensselaer Polytechnic Institute. \nPollack, J. B. (1991). The induction of dynamical recognizer. Machine Learning. 7:227-\n252. \nPotter, M.  A.  (1992). A genetic cascade-correlation learning algorithm. In Proceedings of \nCOGANN-92 International Workshop on Combinations of Genetic Algorithms and Neural \nNetworks. \nSchaffer, J. D., Whitley, D., and Eshelman, L. J.  (1992). Combinations of genetic algo(cid:173)\nrithms and neural networks: A survey of the state of the art. In Proceedings of COGANN-\n92 International Workshop on Combinations of Genetic Algorithms and Neural Networks. \n\n\f", "award": [], "sourceid": 722, "authors": [{"given_name": "Gregory", "family_name": "Saunders", "institution": null}, {"given_name": "Peter", "family_name": "Angeline", "institution": null}, {"given_name": "Jordan", "family_name": "Pollack", "institution": null}]}