{"title": "Patterns of damage in neural networks: The effects of lesion area, shape and number", "book": "Advances in Neural Information Processing Systems", "page_first": 35, "page_last": 42, "abstract": "", "full_text": "Patterns of damage in neural networks: \n\nThe effects of lesion area, shape and \n\nnumber \n\nEytan Ruppin and James A. Reggia \u2022 \n\nDepartment of Computer Science \n\nA.V. Williams Bldg. \n\nUniversity of Maryland \nCollege Park, MD 20742 \n\nruppin@cs.umd.edu \n\nreggia@cs.umd.edu \n\nAbstract \n\nCurrent understanding of the effects of damage on neural networks \nis rudimentary, even though such understanding could lead to im(cid:173)\nportant insights concerning neurological and psychiatric disorders. \nMotivated by this consideration, we present a simple analytical \nframework for estimating the functional damage resulting from fo(cid:173)\ncal structural lesions to a neural network. The effects of focal le(cid:173)\nsions of varying area, shape and number on the retrieval capacities \nof a spatially-organized associative memory. Although our analyti(cid:173)\ncal results are based on some approximations, they correspond well \nwith simulation results. This study sheds light on some important \nfeatures characterizing the clinical manifestations of multi-infarct \ndementia, including the strong association between the number of \ninfarcts and the prevalence of dementia after stroke, and the 'mul(cid:173)\ntiplicative' interaction that has been postulated to occur between \nAlzheimer's disease and multi-infarct dementia. \n\n*Dr. Reggia is also with the Department of Neurology and the Institute of Advanced \n\nComputer Studies at the University of Maryland. \n\n\f36 \n\nEytan Ruppin, James A. Reggia \n\n1 \n\nIntroduction \n\nUnderstanding the response of neural nets to structural/functional damage is im(cid:173)\nportant for a variety of reasons, e.g., in assessing the performance of neural network \nhardware, and in gaining understanding of the mechanisms underlying neurologi(cid:173)\ncal and psychiatric disorders. Recently, there has been a growing interest in con(cid:173)\nstructing neural models to study how specific pathological neuroanatomical and \nneurophysiological changes can result in various clinical manifestations, and to in(cid:173)\nvestigate the functional organization of the symptoms that result from specific brain \npathologies (reviewed in [1, 2]). In the area of associative memory models specifi(cid:173)\ncally, early studies found an increase in memory impairment with increasing lesion \nseverity (in accordance with Lashley's classical 'mass action' principle), and showed \nthat slowly developing lesions have less pronounced effects than equivalent acute \nlesions [3]. Recently, it was shown that the gradual pattern of clinical deterioration \nmanifested in the majority of Alzheimer's patients can be accounted for, and that \ndifferent synaptic compensation rates can account for the observed variation in the \nseverity and progression rate of this disease [4]. However, this past work is limited \nin that model elements have no spatial relationships to one another (all elements \nare conceptually equidistant). Thus, as there is no way to represent focal (localized) \ndamage in such networks, it has not been possible to study the functional effect of \nfocal lesions and to compare them with that caused by diffuse lesions. \n\nThe limitations of past work led us to use spatially-organized neural network for \nstudying the effects of different types of lesions (we use the term lesion to mean \nany type of structural and functional damage inflicted on an initially intact neural \nnetwork). The elements in our model, which can be thought of as representing \nneurons, or micro-columnar units, form a 2-dimensional array (whose edges are \nconnected, forming a torus to eliminate edge effects), and each unit is connected \nprimarily to its nearby neighbors, as is the case in the cortex [5]. It has recently been \nshown that such spatially-organized attractor networks can function reasonably well \nas associative memory devices [6]. This paper presents the first detailed analysis of \nthe effects of lesions of various size, form and number on the memory performance \nof spatially-organized attractor neural networks. Assuming that these networks are \na plausible model of some frontal and associative cortical areas (see, e.g., [7]), our \nresults shed light on the clinical progress of disorders such as stroke and dementia. \n\nIn the next section, we derive a theoretical framework that characterizes the effects \nof focal lesions on an associative network's performance. This framework, which \nis formulated in very general terms, is then examined via simulations in Section 3, \nwhich show a remarkable quantitative fit with the theoretical predictions, and are \ncompared with simulations examining performance with diffuse damage. Finally, \nthe clinical significance of our results is discussed in Section 4. \n\n2 Analyzing the effects of focal lesions \n\nOur analysis pertains to the case where in the pre-damaged network, all units \nhave an approximately similar average level of activity 1. A focal structural lesion \n\nIThis is true in general for associative memory networks, when the activity of each unit \n\nis averaged over a sufficiently long time span. \n\n\fPatterns of Damage ill NeuraL Networks \n\n37 \n\n(anatomical lesion), denoting the area of damage and neuronal death, is modeled \nby clamping the activity of the lesioned units to zero. As a result of this primary \nlesion, the activity of surrounding units may be decreased, resulting in a secondary \narea of functional lesion, as illustrated in Figure 1. We are primarily interested in \nlarge focal lesions, where the area s of the lesion is significantly greater than the \nlocal neighborhood region from which each unit receives its inputs. Throughout our \nanalysis we shall hold the working assumption that, traversing from the border of \nthe lesion outwards, the activity of units gradually rises from zero until it reaches \nits normal, predamaged levels, at some distance d from the lesion's border (see \nFigure 1). As s is large and d is determined by local interactions on the borders of \nthe structural lesion, we may reasonably assume that the value of d is independent \nof the lesion size, and depends primarily on the specific network characteristics , \nsuch as it architecture, dynamics, and memory load. \n\nFigure 1: A sketch of a structural (dark shading) and surrounding functional (light \nshading) rectangular lesion. \n\nLet the intact baseline performance level of the network be denoted as P(O), and \nlet the network size be A. The network 's performance denotes how accurately it \nretrieves the correct memorized patterns given a set of input cues, and is defined \nformally below . A structural lesion of area s (dark shading in Figure 1), causing \na functional lesion of area ~s (light shading in Figure 1), will then result In a \nperformance level of approximately \n\nP(s) = P(O) [A - (s + ~3)l + Pt:..~3 = P(O) _ (~P~3)/(A _ s) , \n\nA-s \n\n(1) \n\nwhere Pt:.. denotes the average level of performance over ~3 and ~P = P(O) - Pt:.. . \nP( s) hence reflects the performance level over the remaining viable parts of the \nnetwork , discarding the structurally damaged region. Bearing these definitions in \nmind, a simple analysis shows that the effect of focal lesions is governed by the \nfollowing rules. \nConsider a symmetric, circular structural lesion of size s = 71-r2 . ~3' the area of \nfunctional damage following such a lesion is then (assuming large lesions and hence \nVS> d) \n\nRule 1: \n\n(2) \n\n\f38 \n\nEytan Ruppin, James A. Reggia \n\n1.0 .--~----r-------r------.--, \n\n0.8 \n\n0.2 \n\n-k .1 \n--- k=5 \nk=25 \n\n0.0 '---~----'\"---~-~---'--' \n1500.0 \n\n1000.0 \n\n0.0 \n\n500.0 \n\nLesion size \n\nFigure 2: Theoretically predicted network performance as a function of a single \nfocal structural lesion's size (area): analytic curves obtained for different k values; \nA = 1600. \n\nand \n\npes) ~ P(O) - Ak..JS \n-s \n\n, \n\n(3) \n\nfor some constant k = yi4;db..P. Thus, the area of functional damage surrounding \na single focal structural lesion is proportional to the square root of the structural \nlesion's area. Some analytic performancejlesioning curves (for various k values) are \nillustrated in Figure 2. Note the different qualitative shape of these curves as a \nfunction of k. Letting x = sjA be the fraction of structural damage, we have \n\nP(x) ~ P(O) _ k.JX _1_ \n1-x.,jA , \n\n(4) \n\nthat is, the same fraction x of damage results in less performance decrease in larger \nnetworks. This surprising result testifies to the possible protective value of having \nfunctional 'modular' cortical networks of large size. \n\nExpressions 3 and 4 are valid also when the structural lesion has a square shape. \nTo study the effect of the structural lesion's shape, we consider the area b.. 8 [n] of \na functional lesion resulting from a rectangular focal lesion of size s = a . b (see \nFigure 1), where, without loss of generality, n = ajb ~ 1. Then, for large n, we find \nthat the functional damage of a rectangular structural lesion of fixed size increases \nas its shape is more elongated, following \nRule 2: \n\nand \n\npes) ~ P(O) _ k.,foS \n2(A - s) \n\n(5) \n\n(6) \n\n\fPatterns of Damage in Neural Networks \n\n39 \n\nTo study the effect of the number of lesions, consider the area .6. 3 m of a functional \nlesion composed of m focal rectangular structural lesions (with sides a = n\u00b7 b), each \nof area s/m. We find that the functional damage increases with the number offocal \nsub-lesions (while total structural lesion area is held constant), according to \nRule 3: \n\nand \n\npes) ~ P(O) _ k.;mns \n2(A - s) \n\n(7) \n\n(8) \n\nWhile Rule 3 presents a lower bound on the functional damage which may actually \nbe significantly larger and involves no approximations, Rule 2 presents an upper \nbound on the actual functional damage. As we shall show in the next section, the \nnumber of lesions actually affects the network performance significantly more than \nits precise shape. \n\n3 Numerical Simulation Results \n\nWe now turn to examine the effect of lesions on the performance of an associa(cid:173)\ntive memory network via simulations. The goal of these simulations is twofold. \nFirst, to examine how accurately the general but approximate theoretical results \npresented above describe the actual performance degradation in a specific associa(cid:173)\ntive network. Second, to compare the effects of focal lesions to those of diffuse \nones, as the effect of diffuse damage cannot be described as a limiting case within \nthe framework of our analysis. Our simulations were performed using a standard \nTsodyks-Feigelman attractor neural network [8]. This is a Hopfield-like network \nwhich has several features which make it more biologically plausible [4], such as low \nactivity and non-zero positive thresholds. In all the experiments, 20 sparse random \n{O, I} memory patterns (with a fraction of p ~ 1 of l's) were stored in a network \nof N = 1600 units, placed on a 2-dimensionallattice. The network has spatially \norganized connectivity, where each unit has 60 incoming connections determined \nrandomly with a Gaussian probability (z) = J1/27rexp( _z2 /2(1'2), where z is the \ndistance between two units in the array. When (1' is small, each unit is connected \nprimarily to its nearby neighbors. As in [4], the cue input patterns are presented \nvia an external field of magnitude e = 0.035, and the noise level is T = 0.005. The \nperformance of the network is measured (over the viable, non-lesioned units) by the \nstandard overlap measure which denotes the similarity between the final state S \nthe network converges to and the memory pattern ~!l that was cued in that trial, \ndefined by \n\nm!l(t) = \n\nP \n\nN \n\n(1 ~ )N I:(~r - p)Si(t) . \n\nP \n\ni=l \n\n(9) \n\nIn all simulations we report the average overlap achieved over 100 trials. \nWe first studied the network's performance at various (1' values. Figure 3a displays \nhow the performance of the network degrades when diffuse structural lesions of \nincreasing size are inflicted upon it (i.e., randomly selected units are clamped to \nzero), while Figure 3b plots the performance as a function of the size of a single \nsquare-shaped focal lesion. As is evident, spatially-organized connectivity enables \n\n\f40 \n\nEytan Ruppin, James A. Reggia \n\nthe network to maintain its memory retrieval capacities in face of focal lesions of \nconsiderable size. Diffuse lesions are always more detrimental than single focal \nlesions of identical size. Also plotted in Figure 3b is the analytical curve calculated \nvia expression (3) (with k = 5), which shows a nice fit with the actual performance \nof the spatially-connected network parametrized by (J' = 1. Concentrating on the \nstudy of focal lesions in a spatially-connected network, we adhere to the values \n(J' = 1 and k = 5 hereafter, and compare the analytical and numerical results. With \nthese values, the analytical curves describing the performance of the network as a \nfunction of the fraction of the network lesioned (obtained using expression 4) are \nsimilar to the corresponding numerical results. \n\n(a) \n\n1.0 r-----,----~-------, \n\n(b) \n\n1.0 , -- - - - , - - - - - r - -- - - - , \n\n- -- \",gna_1 \nsigna =3 \n\"'gna= 10 \n--- \"'gnam30 \n\n0.8 \n\n0.6 \n\n0.4 \n\n0.2 \n\n0.8 \n\n0.6 \n\n0.4 \n\n0.2 \n\n.... ~ .... \n\n~...: .... ~.: .. -~, \n\n',~''', ~-... ~. ; - .. -\\ \n\".-\n\n, \n\n\\ \n\n.\\ \n\n\\ \n\\ ' , \n\\ \n\\ \n\\ \n\\ \n\\ \n\\ \n\\ \n\\ \n\\ \n\n\\ , \n\\ \n\\ \n\\ \n\\ \n\n\\ \n\\ \n\" \n\n'. . \\ \n\n'. \n\n\\ \n\n'. \n\n\\ \n, \\ \n'.\\ \n\n--- sigma =1 \n... , sigma _3 \nsigma -10 \n--- sigma =30 \n\nAnalytical .osuns, k = 5 \" , \n\n....................... \n\n'-\n\n0.0 L-_~ _ _'___~_~ ___ ---' \n\n0.0 \n\n500.0 \n\n1000.0 \n\n1500.0 \n\nLesion sIZe \n\n0.0 ':--~__;:::'::_:_-~-::-:':c_:__---= \n\n1000.0 \n\n1500.0 \n\n0.0 \n\n500.0 \n\nLesion size \n\nFigure 3: Network performance as a function of lesion size: simulation results \nobtained in four different networks, each characterized by a distinct distribution of \nspatially-organized connectivity. (a) Diffuse lesions. (b) Focal lesions. \n\nTo examine Rule 2, a rectangular structural lesion of area s = 300 was induced in \nthe network. As shown in Figure 4a, as the ratio n between the sides is increased, \nthe network's performance further decreases, but this effect is relatively mild. The \nmarkedly stronger effect of varying the lesion number (described by Rule 3) is \ndemonstrated in figure 4b, which shows the effect of multiple lesions composed of \n2,4,8 and 16 separate focal lesions. For comparison, the performance achieved with \na diffuse lesion of similar size is plotted on the 20'th x-ordinate. It is interesting \nto note that a sufficiently large multiple focal lesion (s = 512) can cause a larger \nperformance decrease than a diffuse lesion of similar size. That is, at some point, \nwhen the size of each individual focal lesion becomes small in relation to the spread \nof each unit's connectivity, our analysis looses its validity, and Rule 3 ceases to hold. \n\n\fPatterns of Damage in Neural Networks \n\n41 \n\n(a) \n\nli-\ni \n~ \n\n0.930 \n\n0.910 \n\n0.890 \n\n0.870 \n\n<>--<> SlrnuIeIion \"'1U~s \n- . - Analytical reou~ \n\n\" \n\n\" \n\n, \n\n\" \n\n\" \n\n(b) \n\n0.90 ,---'-~-\"-----'----r-----, \n\n<>--tI Sinulation (s D 256) \n...... \" Sinulation (s \u2022 512) \n- - - - Analytic (s =256) \n- . - Analytic (s \u2022 512) \n\n0.80 \n\n0.80 \n\n0.850 '---~-'---'----:\"-~~-~-----' \n6.0 \n\n1.0 \n\n3.0 \n\n0.0 \n\n2.0 \n\n4.0 \n\n5.0 \n\n0.50 '----'----\"------'-~--'-----' \n25.0 \n\n20.0 \n\n15.0 \n\n10.0 \n\n5.0 \n\n0.0 \n\nRectangular ratio n \n\nNo. 01 sub-lesions m \n\nFigure 4: Network performance as a function of focal lesion shape (a) and num(cid:173)\nber (b). Both numerical and analytical results are displayed. In Figure 4b, the \nx-ordinate denotes the number of separate sub-lesions (1,2,4,8,16) , and, for com(cid:173)\nparison, the performance achieved with a diffuse lesion of similar size is plotted on \nthe 20'th x-ordinate. \n\n4 Discussion \n\nWe have presented a simple analytical framework for studying the effects of focal \nlesions on the functioning of spatially organized neural networks. The analysis \npresented is quite general and a similar approach could be adopted to investigate \nthe effect offocallesions in other neural models. Using this analysis, specific scaling \nrules have been formulated describing the functional effects of structural focal lesions \non memory retrieval performance in associative attractor networks. The functional \nlesion scales as the square root of the size of a single structural lesion, and the form of \nthe resulting performance curve depends on the impairment span d. Surprisingly, \nthe same fraction of damage results in significantly less performance decrease in \nlarger networks, pointing to their relative robustness. As to the effects of shape and \nnumber, elongated structural lesions cause more damage than more symmetrical \nones. However, the number of sub-lesions is the most critical factor determining \nthe functional damage and performance decrease in the model. Numerical studies \nshow that in some conditions multiple lesions can damage performance more than \ndiffuse damage, even though the amount of lost innervation is always less in a \nmultiple focal lesion than with diffuse damage. \n\nBeyond its computational interest, the study of the effects of focal damage on the \nperformance of neural network models can lead to a better understanding of func(cid:173)\ntional impairments accompanying focal brain lesions. In particular, we are inter(cid:173)\nested in multi-infarct dementia, a frequent cause of dementia (chronic deterioration \nof cognitive and memory capacities) characterized by a series of multiple, aggregat-\n\n\f42 \n\nEytan Ruppin, James A. Reggia \n\ning focal lesions. Our results indicate a significant role for the number of infarcts in \ndetermining the extent of functional damage and dementia in multi-infarct disease. \nIn our model, multiple focal lesions cause a much larger deficit than their simple \n'sum', i.e., a single lesion of equivalent total size. This is consistent with clinical \nstudies that have suggested the main factors related to the prevalence of dementia \nafter stroke to be the infarct number and site, and not the overall infarct size, which \nis related to the prevalence of dementia in a significantly weaker manner [9, 10]. Our \nmodel also offers a possible explanation to the 'multiplicative' interaction that has \nbeen postulated to occur between co-existing Alzheimer and multi-infarct dementia \n[10], and to the role of cortical atrophy in increasing the prevalence of dementia after \nstroke; in accordance with our model, it is hypothesized that atrophic degenerative \nchanges will lead to an increase in the value of d (and hence of k) and increase \nthe functional damage caused by a lesion of given structural size. This hypothesis, \ntogether with a detailed study of the effects of the various network parameters on \nthe value of d, are currently under further investigation. \nAcknowledgements \nThis research has been supported by a Rothschild Fellowship to Dr. Ruppin and \nby Awards NS29414 and NS16332 from NINDS. \n\nReferences \n\n[1] J. Reggia, R. Berndt, and L. D'Autrechy. Connectionist models in neuropsy(cid:173)\n\nchology. In Handbook of Neuropsychology, volume 9. 1994, in press. \n\n[2] E. Ruppin. Neural modeling of psychiatric disorders. Network: Computation \n\nin Neural Systems, 1995. Invited review paper, to appear. \n\n[3] J .A. Anderson. Cognitive and psychological computation with neural models. \n\nIEEE Trans. on Systems,Man, and Cybernetics, SMC-13(5):799-815, 1983. \n\n[4] D. Horn, E. Ruppin, M. Usher, and M. Herrmann. Neural network modeling of \nmemory deterioration in alzheimer's disease. Neural Computation, 5:736-749, \n1993. \n\n[5] A. M. Thomson and J. Deuchars. Temporal and spatial properties of local \n\ncircuits in the neocortex. Trends in neuroscience, 17(3):119-126, 1994. \n\n[6] J .M. Karlholm. Associative memories with short-range higher order couplings. \n\nNeural Networks, 6:409-421, 1993. \n\n[7] F.A.W. Wilson, S.P.O Scalaidhe, and P.S. Goldman-Rakic. Dissociation of \nobject and spatial processing domains in primate prefrontal cortex. Science, \n260:1955-1958,1993. \n\n[8] M.V. Tsodyks and M.V. Feigel'man. The enhanced storage capacity in neural \n\nnetworks with low activity level. Europhys. Lett., 6:101 - 105, 1988. \n\n[9] T.K. Tatemichi, M.A. Foulkes, J.P. Mohr, J.R. Hewitt, D. B. Hier, T .R. Price, \nand P.A. Wolf. Dementia in stroke survivors in the stroke data bank cohort. \nStroke, 21:858-866, 1990. \n\n[10] T. K. Tatemichi. How acute brain failure becomes chronic: a view of the \n\nmechanisms of dementia related to stroke. Neurology, 40:1652-1659, 1990. \n\n\f", "award": [], "sourceid": 942, "authors": [{"given_name": "Eytan", "family_name": "Ruppin", "institution": null}, {"given_name": "James", "family_name": "Reggia", "institution": null}]}