{"title": "Evolution and Learning in Neural Networks: The Number and Distribution of Learning Trials Affect the Rate of Evolution", "book": "Advances in Neural Information Processing Systems", "page_first": 804, "page_last": 810, "abstract": null, "full_text": "EVOLUTION AND LEARNING IN \n\nNEURAL NETWORKS: THE NUMBER \nAND DISTRIBUTION OF LEARNING \n\nTRIALS AFFECT THE RATE OF \n\nEVOLUTION \n\nRon Keesing and David G. Stork* \n\nRicoh California Research Center \n2882 Sand Hill Road Suite 115 \n\nMenlo Park, CA 94025 \nstork@crc.ricoh.com \n\nAbstract \n\nand \n\n*Dept. of Electrical Engineering \n\nStanford University \nStanford, CA 94305 \n\nstork@psych.stanford.edu \n\nLearning can increase the rate of evolution of a population of \nbiological organisms (the Baldwin effect). Our simulations \nshow that in a population of artificial neural networks \nsolving a pattern recognition problem, no learning or too \nmuch learning leads to slow evolution of the genes whereas \nan intermediate amount is optimal. Moreover, for a given \ntotal number of training presentations, fastest evoution \noccurs if different individuals within each generation receive \ndifferent numbers of presentations, rather than equal \nnumbers. Because genetic algorithms (GAs) help avoid \nlocal minima in energy functions, our hybrid learning-GA \nsystems can be applied successfully to complex, high(cid:173)\ndimensional pattern recognition problems. \n\nINTRODUCTION \nThe structure and function of a biological network derives from both its \nevolutionary precursors and real-time learning. Genes specify (through \ndevelopment) coarse attributes of a neural system, which are then refined \nand \nbased on experience in an environment containing more information -\n\n804 \n\n\fEvolution and Learning in Neural Networks \n\n805 \n\nmore unexpected infonnation -\nthan the genes alone can represent. Innate \nneural structure is essential for many high level problems such as scene \nanalysis and language [Chomsky, 1957]. \nAlthough the Central Dogma of molecular genetics [Crick, 1970] implies \nthat information learned cannot be directly transcribed to the genes, such \ninformation can appear in the genes through an indirect Darwinian process \n(see below). As such, learning can change the rate of evolution -\nthe \nBaldwin effect [Baldwin, 1896]. Hinton and Nowlan [1987] considered a \nclosely related process in artificial neural networks, though they used \nstochastic search and not learning per se. We present here analyses and \nsimulations of a hybrid evolutionary-learning system which uses gradient(cid:173)\ndescent learning as well as a genetic algorithm, to determine network \nconnections. \nConsider a population of networks for pattern recognition, where initial \nsynaptic weights (weights \"at birth\") are detennined by genes. Figure 1 \nshows the Darwinian fitness of networks (i.e., how many patterns each can \ncorrectly classify) as a function the weights. Iso-fitness contours are not \nconcentric, in general. The tails of the arrows represent the synaptic \nweights of networks at birth. In the case of evolution without learning, \nnetwork B has a higher fitness than does A, and thus would be \npreferentially selected. In the case of gradient-descent learning before \nselection, however, network A has a higher after-learning fitness, and \nwould be preferentially selected (tips of arrows). Thus learning can change \nwhich individuals will be selected and reproduce, in particular favoring a \nnetwork (here, A) whose genome is \"good\" (i.e., initial weights \"close\" to \nthe optimal), despite its poor performance at birth. Over many generations, \nthe choice of \"better\" genes for reproduction leads to new networks which \nrequire less learning to solve the problem -\nthey are closer to the optimal. \nThe rate of gene evolution is increased by learning (the Baldwin effect). \n\nIso-fitness contours \n\nA \n\nWeight 1 \n\nFigure 1: \nIso-fitness contours in \nsynaptic weight space. The black region \ncorresponds to perfect classifications \n(fitness = 5). The weights of two \nnetworks are shown at birth (tails of \narrows), and after learning (tips of \narrows). At birth, 8 has a higher fitness \nscore (2) than does A (1); a pure genetic \nalgorithm (without learning) would \nreproduce 8. Wit h \npreferentially \nlearning, though, A has a higher fitness \nscore (4) than 8 (2), and would thus be \npreferentially reproduced. Since A's \ngenes are \"better\" than 8's, learning can \nlead to selection of better genes. \n\n\f806 \n\nKeesing and Stork \n\nSurprisingly, too much learning leads to slow evolution of the genome, \nsince after sufficient training in each generation, all networks can perform \nperfectly on the pattern recognition task, and thus are equally likely to pass \non their genes, regardless of whether they are \"good\" or \"bad.\" In Figure \n1, if both A and B continue learning, eventually both will identify all five \npatterns correctly. B will be just as likely to reproduce as A, even though \nA's genes are \"better.\" Thus the rate of evolution will be decreased -\ntoo \nmuch learning is worse than an intermediate amount -\nlearning. \n\nor even no -\n\nSIMULA TION APPROACH \nOur system consists of a population of 200 networks, each for classifying \npixel images of the first five letters of the alphabet. The 9 x 9 input grid is \nconnected to four 7 x 7 sets of overlapping 3 x 3 orientation detectors; \neach detector is fully connected by modifiable weights to an output layer \ncontaining five category units (Fig. 2). \n\ntrainable \nweights \n\nA \n\nB \n\nC \n\n0 \n\nE \n\nV'J \nQ) \n.~ \n~ \n0 \nOJ) \nQ) \n....... \n~ \nu \n\n....... \n=' \n~ \n\n.~ \n\nfully interconnected \n\n((,~ \n~~~~ \n\nFigure 2: Individual network architecture. The 9x9 pixel input is detected by each of \nfour orientation selective input layers (7x7 unit arrays), which are fully connected by \ntrainable weights to the five category units. The network is thus a simple perceptron \nwith 196 (=4x7x7) inputs and 5 outputs. Genes specify the initial connection strengths. \n\nEach network has a 490-bit gene specifying the initial weights (Figure 3). \nFor each of the 49 filter positions and 5 categories, the gene has two bits \n\n\fEvolution and Learning in Neural Networks \n\n807 \n\nwhich specify which orientation is initially most strongly connected to the \ncategory unit (by an arbitrarily chosen factor of 3:1). During training, the \nweights from the filters to the output layer are changed by (supervised) \nperceptron learning. Darwinian fitness is given by the number of patterns \ncorrectly classified after training. We use fitness-proportional reproduction \nand the standard genetic algorithm processes of replication, mutation, and \ncross-over [Holland, 1975]. Note that while fitness may be measured after \ntraining, reproduction is of the genes present at birth, in accord with the \nCentral Dogma. This is llil1 a Lamarkian process. \n\nA detector B detector C detector D detector ..... . \n\npit 011010101001 0 101 00111010 100101 01001 01 010010101 00101 010010 101001010 10 10 1 001 010100 ... \n\n001010110001001110100110100101010101101010101011 ... \n\n\u2022 \n\n~ possible \n\n~@IJ ~ QI]~ gene \n\nvalues at \n\nRelative \ninitial \nweights \nbetween \nfilters (at a \n\nspatial \nposition) \n\nand \n\ncategory \n\n3 \n\n1 \n\n1 \n\n1 \n\n1 \n\n3 \n\n1 \n\n1 \n\n1 \n\n1 \n\n3 \n\n1 \n\nposition \n\n1 ~ a spatial \n1 ~ \n1 ~ \n3 \n~ \n\nFigure 3: The genetic representation of a network. For each of the five category units, \n49 two-bit numbers describe which of the four orientation units is most strongly \nconnected at each position within the 7x7 grid. This unit is given a relative connection \nstrength of 3, while the other three orientation units at that position are given a relative \nstrength of 1. \n\nFor a given total number of teaching presentations, reproductive fitness \nmight be defined in many ways, including categorization score at the end of \nlearning or during learning; such functions will lead to different rates of \nevolution. We show simulations for two schemes: in uniform learning each \nnetwork received the same number (e.g., 20) of training presentations; in \n\n\f808 \n\nKeesing and Stork \n\ndistributed learning networks received a randomly chosen number (10, 34, \n36, 16, etc.) of presentations. \n\nRESULTS AND DISCUSSION \nFigure 4 shows the population average fitness at birth. The lower curve \nshows the performance of the genetic algorithm alone; the two upper curves \nrepresent genetypic evolution -\nthe amount of information within the genes \n- when the genetic algorithm is combined with gradient-descent learning. \nLearning increases the rate of evolution -\nboth uniform and distributed \nlearning are significantly better than no learning. The fitness after learning \nin a generation (not shown) is typically only 5% higher than the fitness at \nbirth. Such a small improvement at a single generation cannot account for \nthe overall high performance at later generations. A network's performance \n-\nis more dependent upon its ancestors having \nlearned than upon its having learned the task. \n\neven after learning -\n\nPop. Avg. Fitness at Birth for \n\nDifferent Learning Schemes \n\n= S~--------------------~ \n... \nm \n-\nas \n\n4 \n\n(t) \n(t) 3 \nCD \nC \n!:: u.. 2 \n. C) \n> cr:: 1 \n. \nD.. \n0 0 \nD.. \n\n0 \n\n20 \n\n40 \n\n60 \n\n80 100 \n\nGeneration \n\nFigure 4: Learning guides the rate of \nevolution. \nIn uniform learning, every \nnetwork in every generation receives 20 \nlearning presentations; in the distributed \nlearning scheme, any network receives a \nnumber of patterns randomly chosen \nbetween 0 and 40 presentations (mean = \n20). Clearly, evolution with learning \nleads to superior genes (fitness at birth) \nthan evolution without learning. \n\nAve. Fitness at Generation 100 \nDepends on Amount of Training \n.cs-r--------------. \nt: \n_ 4 \nas \n(t) \n\nDistributed Learning \n\n(t) 3 \nCD \n\nC -u.. \n2 \nai \n> cr:: 1 \n\nD.. \n0 \nQ. 0 \n\n1 \nAvg. \n\n10 \n\n100 \n\n1000 \n\nLearning Trials per Indlv. \n\nSelectivity of learning(cid:173)\n\nFigure 5: \nevolution interactions. Too little or too \nmuch learning leads to slow evolution \n(population fitness at birth at generation \n100) while an intermediate amount of \nlearning leads to significantly higher such \nfitness. This effect is significant in both \nlearning schemes. (Each point represents \nthe mean of five simulation runs.) \n\n\fEvolution and Learning in Neural Networks \n\n809 \n\nFigure 5 illustrates the tuning of these learning-evolution interactions, as \ndiscussed above: too little or too much learning leads to poorer evolution \nthan does an intermediate amount of learning. Given excessive learning \n(e.g., 500 presentations) all networks perform perfectly. This leads to the \nslowest evolution, since selection is independent of the quality of the genes. \nNote too in Fig. 4 that distributed learning leads to significantly faster \nevolution (higher fitness at any particular generation) than uniform learning. \nIn the uniform learning scheme, once networks have evolved to a point in \nweight space where they (and their offspring) can identify a pattern after \nlearning, there is no more \"pressure\" on the genes to evolve. In Figure 6, \nboth A and B are able to identify three patterns correctly after uniform \nlearning, and hence both will reproduce equally. However, in the \ndistributed learning scheme, one of the networks may (randomly) receive a \nsmall amount of learning. In such cases, A's reproductive fitness will be \nunaffected, because it is able to solve the patterns without learning, while \nB's fitness will decrease significantly. Thus in the distributed learning \nscheme (and in schemes in which fitness is determined in part during \nlearning), there is \"pressure\" on the genes to improve at every generation. \nDiversity is, a driving force for evolution. Our distributed learning scheme \nleads to a greater diversity of fitness throughout a population. \n\nIso-fitness contours \n\nFigure 6: Distributed learning leads \nto faster evolution \nthan uniform \nlearning. In uniform learning, (shown \nabove) A and B have equal reproductive \nfitness, even though A has \"better\" \ngenes. In distributed learning, A will \nbe more likely to reproduce when it \n(randomly) receives a small amount of \nlearning (shorter arrow) than B will \nunder similar circumstances. \nThus \n\"better\" genes will be more likely to \nreproduce, leading to faster evolution. \n\nWeight 1 \n\nCONCLUSIONS \nEvolutionary search via genetic algorithms is a powerful technique for \navoiding local minima in complicated energy landscapes [Goldberg, 1989; \nPeterson, 1990], but is often slow to converge in large problems. \nConventional genetic approaches consider only the reproductive fitness of \n\n\f810 \n\nKeesing and Stork \n\nthe genes; the slope of the fitness landscape in the immediate vicinity of the \ngenes is ignored. Our hybrid evolutionary-learning approach utilizes the \ngradient of the local fitness landscape, along with the fitness of the genes, in \ndetennining survival and reproduction. \nWe have shown that this technique offers advantages over evolutionary \nsearch alone in the single-minimum landscape given by perceptron learning. \nIn a simple pattern recognition problem, the hybrid system performs twice \nas well as a genetic algorithm alone. A hybrid system with distributed \nlearning, which increases the \"pressure\" on the genes to evolve at every \ngeneration, performs four times as well as a genetic algorithm. In addition, \nwe have demonstrated that there exists an optimal average amount of \nlearning in order to increase the rate of evolution -\ntoo little or too much \nlearning leads to slower evolution. In the extreme case of too much \nlearning, where all networks are trained to perfect performance, there is no \nimprovement of the genes. The advantages of the hybrid approach in \nlandscapes with multiple minima can be even more pronounced [Stork and \nKeesing, 1991]. \n\nAcknowledgments \nThanks to David Rumelhart, Marcus Feldman, and Aviv Bergman for \nuseful discussions. \n\nReferences \nBaldwin, J. M. \"A new factor in evolution,\" American Naturalist 30,441-\n\nChomsky, N. Syntactic Structures The Hague: Mouton (1957) \nCrick, F. W. \"Central Dogma of Molecular Biology,\" Nature 227, 561-563 \n\n451 (1896) \n\n(1970) \n\nGoldberg, D. E. Genetic Algorithms in Search, Optimization & \n\nMachine Learning Reading, MA: Addison-Wesley (1989). \n\nHinton, G. E. and Nowlan, S. 1. \"How learning can guide evolution,\" \n\nComplex Systems 1,495-502 (1987) \n\nHolland, J. H. Adaptation in Natural and Artificial Systems \n\nUniversity of Michigan Press (1975) \n\nPeterson, C. \"Parallel Distributed Approaches to Combinatorial \n\nOptimization: Benchmanrk Studies on Traveling Salesman Problem,\" \nNeural Computation 2, 261-269 (1990). \n\nStork, D. G. and Keesing, R. \"The distribution of learning trials affects \n\nevolution in neural networks\" (1991, submitted). \n\n\f", "award": [], "sourceid": 420, "authors": [{"given_name": "Ron", "family_name": "Keesing", "institution": null}, {"given_name": "David", "family_name": "Stork", "institution": null}]}